Best local LLMs for German: open models compared
Local LLMs run on your own hardware, so your documents stay in-house. Which is the best local LLM for German depends on licence, size and language support. Here you will find ten open model families compared on facts, taken directly from what their publishers state. To find the best local LLM for German documents, use our free test set: 30 questions in German that you score yourself.
As of 5 November 2026
Key points
- The table shows
details from the publishers’ model cards and licences , sorted alphabetically. - 5 of 10 publishers explicitly list
German as a supported language: EuroLLM, Granite 4.2, Llama, Mistral and Teuken-7B. If German is missing from the list, that says nothing yet about quality. - “Open” does not mean “unconditional”: user thresholds, EU clauses, usage policies and non-commercial versions are in the
licences , not in the model name. - Test for yourself: our
German RAG test set with 30 questions, reference answers and a score sheet is free to download under CC BY 4.0.
What this page does, and what it does not.
Anyone who wants to run local AI in the company has to choose a model at some point. Which criteria matter is explained in our post Open-source LLMs: six criteria for choosing. This page is its concrete companion: the current model families with their details side by side, and a tool to check candidates in German.
What you will not find here: scores, winners or a “top 10”. Until we have measured the models ourselves, any order would be an opinion. We will publish our own results with the test set on this page later.
From the publishers
Sizes, licence, context, German and release date from the publishers’ own details, with a link to the source.
Memory recalculated
Each model’s memory need comes from the same formulas as the LLM VRAM calculator.
Test for yourself
30 questions in German about a fictional company handbook, with reference answers and a score sheet.
run local AI in your company →Six criteria for choosing a model →
Ten open model families for local use.
We included families with freely downloadable weights that can run on your own hardware and matter for German-speaking companies. Phi-4 is in the table as a counter-example: its publisher explicitly does not intend it for multilingual use. The order is alphabetical and says nothing about quality.
| Model family · publisher | Sizes (parameters) | Licence | Context (tokens) | German officially listed | Released | VRAM (4-bit, 1 user, 8k) |
|---|---|---|---|---|---|---|
Swiss AI Initiative (ETH Zurich, EPFL) | 8B, 70B (version 2509; newer v1.1 with 0.5B–4B and v1.5 have been released) | Apache 2.0 plus acceptable use policy: indemnify the universities against third-party claims; output filter for deletion requests recommended | 65,536 | not named individually (1,811 languages) | Sep 2025 | 8B: 7.4 GB 70B: 50.1 GB Calculate → |
EuroLLM project (European research consortium) | 9B, 22B (version 2512) | Apache 2.0 | 32,768 | yes (24 official EU languages and 11 more) | Dec 2025 | 9B: 8.5 GB 22B: 17.8 GB Calculate → |
Google DeepMind | E2B (5.1B incl. embeddings), E4B (8B), 12B, 31B; MoE 26B A4B (25.2B total, 3.8B active) | Apache 2.0 | 128K (E2B, E4B), 256K (others) | not named individually (35+ languages, trained on 140+) | from Apr 2026 | E4B: 7.0† GB 26B A4B: 20.1† GB 31B: 30.1† GB Calculate → |
OpenAI | MoE 20b (21B total, 3.6B active), MoE 120b (117B total, 5.1B active) | Apache 2.0 plus a short usage policy (comply with applicable law) | 128K | not named | Aug 2025 | 20b: 15.1† GB 120b: 78.2† GB Calculate → |
IBM | 3B, 8B, 30B (dense) | Apache 2.0 | 131,072 (extensible to 512K per model card) | yes (12 languages) | Aug 2026 | 3B: 4.1 GB 8B: 8.2 GB 30B: 22.5 GB Calculate → |
Meta | Llama 3.3: 70B; Llama 4: Scout MoE 109B (17B active), Maverick MoE 400B (17B active) | Own licences (Llama 3.3 / Llama 4 Community License): extra licence above 700M monthly active users, “Built with Llama”, acceptable use policy. Llama 4 multimodal: rights under Section 1(a) not granted to individuals domiciled in, or companies with their principal place of business in, the EU | 3.3: 128K; Scout: 10M; Maverick: 1M | yes (3.3: 8 languages, 4: 12 languages) | Dec 2024 (3.3), Apr 2025 (4) | 3.3 70B: 50.1 GB Scout: 73.9† GB Calculate → |
Mistral AI | Ministral 3: 3B, 8B, 14B; Mistral Small 4: MoE 119B (6B active) | Apache 2.0 | 256K | yes | Dec 2025 (Ministral 3), Mar 2026 (Small 4) | 3B: 4.5 GB 8B: 8.1 GB 14B: 11.6 GB Small 4: not calculated‡ Calculate → |
Microsoft | 14B | MIT | 16K | no: according to the model card “not intended to support multilingual use” | Dec 2024 | 14B: 12.4 GB Calculate → |
Qwen team (Alibaba) | 27B (dense); Flash-Next: MoE 125B (6B active, plus 51B n-gram embedding and 4B MTP); 2.4T-A95B: MoE 2.4T (95B active) | 27B: Apache 2.0. Flash-Next: Qwen Community License 1.0; 2.4T: Qwen3.8-Max License. Both: model name in the UI above 100M MAU or US$20M monthly revenue; companies running a “Model as a Service” or AI work assistant business need an extra licence (for the 2.4T model only above US$50M group revenue in 12 months); purely internal use without third-party access exempt | 262,144 (extensible to 1M) | not named individually (201 languages and dialects) | Aug 2026 | 27B: 21.5† GB Calculate → |
OpenGPT-X (research project funded by the German economy ministry) | 7B | v0.4 “commercial”: Apache 2.0; v0.6: CC BY-NC 4.0, non-commercial only | 4,096 | yes (24 official EU languages) | Nov 2024 (v0.4) | 7B: 6.0 GB (4k) Calculate → |
- Apertus: Apertus-8B-Instruct-2509 model card (Swiss AI Initiative)
- EuroLLM: EuroLLM-22B-Instruct-2512 model card
- EuroLLM: EuroLLM-22B release post, 14 Dec 2025
- Gemma 4: Gemma 4 31B model card (Google DeepMind)
- Gemma 4: Gemma 4 licence page (Apache 2.0)
- gpt-oss: gpt-oss-20b model card and USAGE_POLICY
- Granite 4.2: Granite 4.2 release post (Granite Team, IBM), 25 Aug 2026
- Llama: Llama 3.3 model card
- Llama: Llama 4 model card
- Llama: Llama 4 Community License Agreement
- Llama: Llama 4 Acceptable Use Policy
- Mistral: Ministral 3 14B model card
- Mistral: Mistral 3 release post, 2 Dec 2025
- Mistral: Mistral Small 4 release post, 16 Mar 2026
- Phi-4: phi-4 model card (Microsoft)
- Qwen3.8: Qwen3.8-27B model card
- Qwen3.8: Qwen3.8-Flash-Next model card and licence
- Qwen3.8: Qwen3.8 repository (release notes)
- Teuken: Teuken-7B-instruct-commercial-v0.4 model card
- Teuken: Teuken-7B-instruct-v0.6 model card
- VRAM: own calculation with the formulas of the LLM VRAM calculator; bits per weight for 4-bit (Q4_K_M) from the llama.cpp quantize README.
Why models differ in how well they handle German.
Three factors play a role, as far as is known today. First,
Second,
What this means for your choice: the details in the table narrow down the candidates but do not decide. Whether a model correctly answers your German questions from your documents only shows in a test.
Tokenisation
If a model splits German words into many small pieces, less text fits in the context and answers take longer.
Training data
The share of German text in training is rarely published. European projects such as EuroLLM and Teuken focus on EU languages.
Commitment
“Supported” means the publisher lists the language explicitly, in some cases based on its own tests. It is no proof of quality for your documents.
Licence pitfalls for companies.
The weights of every model in the table can be downloaded freely. What you may do with them is governed by the licence, and that sometimes differs within one model family. The points below come from the licence texts themselves. They are no substitute for legal advice; as a rule, have the licence of the model you plan to use checked by your legal team.
Same name, different licence
Teuken-7B v0.4 “commercial” is under Apache 2.0, the newer v0.6 under CC BY-NC 4.0, so for non-commercial purposes only. With Qwen3.8, the 27B model is under Apache 2.0 and the larger models under Qwen’s own licences.
Thresholds and attribution
Llama requires a separate licence above 700 million monthly active users and the notice “Built with Llama” when redistributing. The Qwen licences require the model name to be shown in the interface above 100 million monthly active users or US$20 million monthly revenue.
EU clause for Llama 4
According to the acceptable use policy, the rights under Section 1(a) for multimodal Llama 4 models are not granted to individuals domiciled in the EU or companies with their principal place of business there; end users of a product that incorporates such models are exempt. Meta describes Scout and Maverick as natively multimodal.
Offering to third parties
The Qwen licences for Flash-Next and the 2.4T model require an extra licence for any commercial use if your company runs a “Model as a Service” or AI work assistant (coding or office productivity) business; for the 2.4T model only above US$50M revenue in twelve months. Purely internal use without third-party access is exempt.
Use policies
Some models add their own policies to the licence: the acceptable use policy for Llama, a short usage policy for gpt-oss. With Apertus you agree, among other things, to indemnify the universities against third-party claims, and you are advised to update an output filter for deletion requests every six months.
Document it
Record which model runs in which version under which licence, as with any other software. When you switch models, check the licence again.
Model size and hardware: what fits where.
The key figure for hardware is GPU memory. The chart shows, for one to three sizes per family, how much the language model alone needs at 4 bit for one request. Up to about 8B, a model fits with headroom on a GPU with 16 GB; models between 27B and 31B need 24 to 48 GB, depending on architecture; a 70B model needs 80 GB or two GPUs.
With MoE models, the whole model always sits in memory, even though only a small part computes per token. An MoE model therefore often answers faster than a dense model of the same size but does not need less memory. And in operation, every concurrent request adds KV cache: with many users it is often larger than the model itself.
Calculate the memory need of any model →Local LLM hardware: plan your server →
How to test local models yourself.
A good test compares models under the same conditions with questions whose correct answer you know. That is exactly what the
The test set, handbook and questions are in German by design, because the point is to test German answer quality. Everything is under Creative Commons Attribution 4.0 (CC BY 4.0): you may use, change and share it, commercially too, credited as “RAG-Testset Deutsch, RAGSuite (ragsuite.de), CC BY 4.0”. An English README explains the files and the scoring.
Pick candidates
Two or three models from the table whose licence suits your use and that fit your hardware.
Fix the conditions
Same quantisation, same retrieval (chunk size, number of hits, embedding model) and the same system prompt for all. Note the versions.
Ask the questions
Ingest the handbook as one document and ask all 30 questions in German, each in a new conversation.
Score
Compare each answer with the reference answer and score it with the table below. Ideally two people score independently.
Repeat with your own documents
Repeat the process with 20 to 30 real questions from your daily work. That is what decides.
Five categories, 30 questions.
Each category tests a typical weakness when answering from documents. The examples are taken from the test set; the questions are in German.
| Category | Questions | What it tests | Example |
|---|---|---|---|
| Faktenfrage (fact) | 7 | One fact from one section | “Wo befindet sich der Sanitätsraum?” (Where is the first-aid room?) |
| Zahl/Datum (number/date) | 8 | Exact numbers, amounts, times and deadlines | “Wie hoch darf ein Hotel in Hamburg pro Nacht kosten?” (hotel cap in Hamburg) |
| mehrere Abschnitte (several sections) | 5 | Combine two passages or draw a simple inference | “Ich arbeite in der Frühschicht und bin krank …” (early shift, sick: by when to call in?) |
| nicht beantwortbar (unanswerable) | 5 | The handbook does not contain the answer. Correct: saying so instead of guessing | “Wie viele Tage Sonderurlaub …” (special leave for a wedding: not in the handbook) |
| Umlaute/Fachbegriffe (umlauts/terms) | 5 | Names with umlauts and special characters, German technical terms | “Was ist ein EMPB …” (Erstmusterprüfbericht, first article inspection report) |
Scoring rubric: correct, partial, wrong, correctly declined.
Four levels are enough for two people to reach similar results. The key level is “correctly declined”: a model that honestly passes when information is missing is more valuable for company documents than one that invents plausible-sounding answers.
| Score | Points | When |
|---|---|---|
| richtig (correct) | 2 | Correct and complete; the right passage is cited or clearly used. |
| teilweise (partial) | 1 | Core is right, but incomplete or with a minor error, such as a wrong time with the right procedure. |
| falsch (wrong) | 0 | Wrong, invented, or refused although the answer is in the handbook. |
| korrekt verweigert (correctly declined) | 2 | Only for “nicht beantwortbar”: the model says the handbook does not contain the answer. An invented answer counts as wrong. |
Maximum: 60 points. Also report the result per category: a model that scores well overall but invents answers to “nicht beantwortbar” questions is risky for company documents.
Printable score sheet.
Enter 0, 1 or 2 points per model. The reference answers and source sections are in the CSV and JSON files; that keeps the sheet short and the answers out of view of whoever asks the questions.
| No. | Category | Question (German) | Model A | Model B | Model C |
|---|---|---|---|---|---|
| Q01 | Faktenfrage | Was stellt die Musterwerk GmbH her? | |||
| Q02 | Faktenfrage | Wer ist Geschäftsführerin der Musterwerk GmbH? | |||
| Q03 | Faktenfrage | Wie heißt das Reiseportal, in dem Dienstreisen genehmigt und abgerechnet werden? | |||
| Q04 | Faktenfrage | Wo befindet sich der Sanitätsraum? | |||
| Q05 | Faktenfrage | Mit welchem Formular beantrage ich mobiles Arbeiten? | |||
| Q06 | Faktenfrage | Wohin kommen beanstandete Teile aus einer Kundenreklamation? | |||
| Q07 | Faktenfrage | Darf ich Kundendaten beim mobilen Arbeiten auf meinem privaten Laptop speichern? | |||
| Q08 | Zahl/Datum | Wie viele Urlaubstage habe ich bei einer Fünftagewoche? | |||
| Q09 | Zahl/Datum | Bis wann muss Resturlaub aus dem Vorjahr genommen werden? | |||
| Q10 | Zahl/Datum | Wann ist am Freitag Kernzeit? | |||
| Q11 | Zahl/Datum | Wie viel Kilometergeld gibt es für Fahrten mit dem privaten Pkw? | |||
| Q12 | Zahl/Datum | Wie hoch darf ein Hotel in Hamburg pro Nacht kosten? | |||
| Q13 | Zahl/Datum | Wie lautet die interne Notrufnummer? | |||
| Q14 | Zahl/Datum | Wie viele Beschäftigte hat Musterwerk? | |||
| Q15 | Zahl/Datum | In welchem Takt fährt der Shuttlebus zwischen Werk Nord und Haus Süd? | |||
| Q16 | mehrere Abschnitte | Ich arbeite in der Frühschicht und bin krank. Bis wann muss ich mich melden, und um wie viel Uhr ist das? | |||
| Q17 | mehrere Abschnitte | Ich arbeite in der Fertigung im Werk Nord. Darf ich an einzelnen Tagen mobil arbeiten, und wann beginnt meine Spätschicht? | |||
| Q18 | mehrere Abschnitte | Ich arbeite im Haus Süd. Wo kann ich mittags warm essen, und wie komme ich dorthin? | |||
| Q19 | mehrere Abschnitte | Ich habe einen Laptop mit Personaldaten verloren. Wen informiere ich, wie und bis wann? | |||
| Q20 | mehrere Abschnitte | Ich bekomme Besuch für eine Führung durch Halle 2. Was muss ich vorher und vor Ort beachten? | |||
| Q21 | nicht beantwortbar | Wie viele Tage Sonderurlaub gibt es bei der eigenen Hochzeit? | |||
| Q22 | nicht beantwortbar | Wie hoch ist das Tagegeld für Verpflegung bei Dienstreisen? | |||
| Q23 | nicht beantwortbar | Wie lautet die Durchwahl der Geschäftsführerin? | |||
| Q24 | nicht beantwortbar | Wie viele Tage Bildungsurlaub nach Landesgesetz stehen mir zusätzlich zu? | |||
| Q25 | nicht beantwortbar | Welche Note braucht man in der jährlichen Sicherheitsunterweisung zum Bestehen? | |||
| Q26 | Umlaute/Fachbegriffe | Was ist ein EMPB, und wann wird er erstellt? | |||
| Q27 | Umlaute/Fachbegriffe | Bis wann müssen bei einem 8D-Report die Sofortmaßnahmen feststehen, und bis wann der vollständige Bericht? | |||
| Q28 | Umlaute/Fachbegriffe | Ein Messschieber trägt einen roten Aufkleber. Was bedeutet das, und wie oft werden Messmittel kalibriert? | |||
| Q29 | Umlaute/Fachbegriffe | Wer ist Vorsitzender des Betriebsrats, und wie erreiche ich ihn? | |||
| Q30 | Umlaute/Fachbegriffe | Ab wann muss ich bei Arbeitsunfähigkeit eine ärztliche Bescheinigung vorlegen, und muss ich einen Papierschein einreichen? | |||
| Total | maximum 60 points |
Our own results: to follow.
We will measure selected models from the table with exactly this test set on our own hardware and publish the results here, with all settings, so you can check them.
Have you already used the test set? We welcome your results and suggestions via the contact form.
Questions about local LLMs.
What is the best local LLM for German?
There is no general answer. The publishers of EuroLLM, Granite 4.2, Llama 3.3 and Llama 4, Ministral 3 and Mistral Small 4, and Teuken-7B explicitly list German as a supported language. Whether a model answers your documents well only shows in a test with your own questions, for example with our free test set.
Which local AI models can companies use free of charge?
The weights of every model in the table can be downloaded free of charge, but the conditions differ. Apertus (with an additional acceptable use policy), EuroLLM, Gemma 4, gpt-oss, Granite 4.2, Ministral 3, Mistral Small 4, Qwen3.8-27B and Teuken-7B version v0.4 are under Apache 2.0; Phi-4 is under MIT. Llama uses its own licences with conditions, and so do the larger Qwen3.8 models. Teuken-7B v0.6 is released for non-commercial purposes only.
How much GPU memory does a local AI model need?
At 4 bit, for one request with 8,192 tokens, an 8B model needs about 8 GB by our calculation, a 30B model about 22 GB and a 70B model about 50 GB. An embedding model and a reranker add 4.4 GB, and every additional concurrent request needs more memory. Check your own set-up with the LLM VRAM calculator.
Can a company in the EU use Llama 4?
That is a question for your legal team. The Llama 4 Acceptable Use Policy says that for multimodal Llama 4 models, the rights under Section 1(a) of the licence are not granted to individuals domiciled in the EU or companies with their principal place of business there; the restriction does not apply to end users of a product that incorporates such models. Meta describes Scout and Maverick as natively multimodal. The Llama 3.3 Acceptable Use Policy contains the same clause, but it only covers multimodal models, and according to its model card Llama 3.3 70B is text-only.
What does “German officially supported” mean?
The publisher explicitly names German as a supported language in the model card or release post. For Llama 3.3, Meta explains that the supported languages meet its thresholds for safety and helpfulness, and that the model may also answer in other languages. If German is missing from the list, that does not mean the model is poor at German, only that the publisher does not commit to it.
How current is the table?
The table is as of 5 November 2026. All details come from the publishers’ model cards, licence files and release posts, each checked on that day. We review the table regularly and when important new models appear; the as-of date applies.
Can I use and change the test set?
Yes. The handbook, questions and answers are under Creative Commons Attribution 4.0 (CC BY 4.0). You may use, change and share them, commercially too, as long as you credit the source: “RAG-Testset Deutsch, RAGSuite (ragsuite.de), CC BY 4.0”. Musterwerk GmbH and everything in the handbook are fictional.
Find the right model for your documents.
Mihaela reviews licence, hardware and test results with you and helps compare two or three candidates on your own documents.