AI utility - On-device estimate
LLM Token Counter
and Context Estimator.
Estimate token ranges for prose, multilingual text, code, prompts, or page copy, model how much fits, and plan approximate overlapping chunks for retrieval workflows.
Transparent assumptions
Profiles show their characters-per-token ratio
Local processing
No text upload, model API, or storage
Workspace
LLM Token Counter and Context Estimator
Runs in this browser. Text is not uploaded or stored. Results are estimates, not an exact model tokenizer.
RAG Chunk Planner
Plan approximate retrieval chunks from the same text. Content remains on this device.
Paste text to plan retrieval chunks.
Tokenization depends on the exact model, tokenizer version, language, whitespace, punctuation, code, and special tokens. Verify hard limits with the tokenizer used by the target model.
Interpretation rules
Estimate a budget, not a promise.
- The selected profile exposes its characters-per-token assumption
- The range is deliberately wider than the typical estimate
- Instructions and expected output reduce the available input budget
- The boundary preview is approximate and stops at a word boundary
- The RAG planner prefers paragraph, sentence, and word boundaries
- Chunk overlap is explicit and never duplicates a complete chunk
- Exact limits require the tokenizer used by the target model
Context windows describe model input and output capacity. They are not a search-engine crawl budget and do not predict whether an AI system will index, retrieve, cite, or trust a page.
Use it well
Use a range before you depend on a hard limit.
A local heuristic is useful for planning and comparing drafts. Before sending a long or expensive request, verify the final prompt with the exact tokenizer and context rules of the chosen model.
01
Choose the text profile
02
Set context or RAG chunk parameters
03
Review ranges, boundaries, and chunk excerpts
Common questions
How do I estimate the number of AI tokens in text?
Paste the text, choose the closest profile, and read the typical estimate plus its wider range. Use the exact model tokenizer when a hard API or context limit matters.
Why is this a token estimate instead of an exact count?
Different model families and versions use different token vocabularies and special-token rules. A browser-only generic tool cannot claim one exact result for every model.
What is the difference between characters, words, and tokens?
Characters and words describe the source text. Tokens are model-specific pieces that can represent a word, part of a word, punctuation, whitespace, code, or another symbol. For general text length metrics, use the Text Character Counter.
What is an LLM context window?
It is the model limit shared by input, instructions, retrieved material, tool messages, and generated output. The usable capacity can be smaller than the advertised maximum in a particular product or workflow.
How can I see how far an LLM may read into a page?
Set the available context and reserve. The tool converts the remaining typical token budget into an approximate character boundary and shows the ending excerpt. A real product may truncate, chunk, summarize, or retrieve text differently.
Is an LLM context budget the same as crawl budget?
No. Crawl budget concerns how search-engine crawlers allocate requests to a site. Context budget concerns how much tokenized material a model can process in one operation.
Why do language, code, and structured data change token counts?
Token vocabularies compress common patterns differently. Less familiar words, scripts, punctuation-heavy code, JSON, tables, and unusual whitespace can require more tokens than ordinary Latin prose.
How do I split text into chunks for RAG?
Paste the source, choose the closest estimation profile, then set a target chunk size and overlap. The planner prefers paragraph, sentence, and word boundaries and shows the approximate token range and source-character span of every chunk.
What chunk size and overlap should I use?
There is no universal value. Start with a size that keeps one useful idea and its evidence together, then test retrieval with real questions. Use enough overlap to preserve context at boundaries without repeatedly retrieving the same material.
Does this prove that an AI system can retrieve my content?
No. Chunk planning only helps inspect potential retrieval units. Retrieval also depends on parsing, indexing, embeddings, query matching, filters, ranking, and the product implementation. Use the AI Extractability Checker to inspect whether the visible page copy exposes clear, self-contained passages.
Is my prompt or page text uploaded?
No. Estimation happens in this browser and no model API is called. If you are preparing an AI content workflow, you can build the next instruction locally with the AI Content Prompt Builder.