Tokenizer: how many tokens does your text have?
Language models count in tokens, not words. Paste a text and see at once roughly how many tokens it has, what a request costs and whether it fits typical context windows. The estimate runs entirely in your browser.
Estimate · usually within 10–20%
Your text in tokens, and whether it fits the context window.
Tokens
Context windows
What is a token?
A token is a piece of text a language model computes with, often part of a word, sometimes a whole word, a punctuation mark or a space. In English there are roughly 1.3 tokens per word. German texts usually need more, because words are longer and compounded.
Tokens decide two things: what a request costs with an API provider and whether question, context and answer fit the model’s context window. Knowing tokens helps you plan costs and answer lengths more precisely.
A transparent estimate instead of a black box.
Splitting
Words are split into pieces of about four characters, numbers into three, punctuation into two.
In your browser
Real tokenizers are model-specific and often run on someone else’s server. Our estimate sends nothing.
Accuracy
Usually within 10–20%. For billing, use your model’s own tokenizer.
How many tokens do typical texts have?
| Text | Roughly | Note |
|---|---|---|
| One question | 20–50 tokens | short, hardly any cost |
| One page of running text | 500–800 tokens | German towards the upper end |
| Ten passages as context | 2,000–4,000 tokens | the largest item in RAG |
| A 100-page handbook | 50,000–80,000 tokens | does not fit every window |
Estimates for German and English texts. Tables, code and numbers increase the token count.
Why tokens matter especially for RAG.
With retrieval-augmented generation, not only the question goes to the model but also the passages found. The context is often ten times longer than the question, and it drives the cost. Choosing and trimming passages well saves tokens and gives better answers.
Context window calculator →Chunking playground →AI cost calculator →
Questions about the tokenizer.
How many tokens does a word have?
In English there are on average about 1.3 tokens per word, roughly four characters per token. German texts usually have more tokens per word because words are longer and often compounded. Code, numbers and lots of punctuation push the ratio up further. The tool shows the ratio for your text.
Is the token count exact?
No, it is an estimate. An exact count needs the specific model’s tokenizer, and every model splits text slightly differently. Our estimate is usually within 10 to 20 percent. That is enough for planning and comparisons; for billing, check with your provider’s tokenizer.
Why not use real tokenizers?
Real tokenizers are model-specific and come with large data files. Many online tools send your text to a server for this. Our tool runs entirely in the browser without external libraries, so your text never leaves your computer. That matters especially for confidential documents.
What does a request cost?
That depends on tokens and price: tokens divided by one million times the price per million tokens. Enter your model’s price to see the cost for your text. Keep in mind that answers usually cost more than inputs. For the whole month, use the AI cost calculator.
Is my text stored?
No. The text stays in your browser and is not sent to us or third parties. So you can also check confidential documents. Close the page and the text is gone. This applies to all developer tools on ragsuite.de, without exception.
Save tokens without losing quality.
RAGSuite picks matching passages, shows the source and lets you choose the model per project. Install the Community Edition or try the Cloud.