IBM Granite AI in Your Browser — Chat, Documents, Speech & Search

No. The only network requests are for the model weights themselves, fetched once from Hugging Face's CDN and then cached by your browser. After that your prompts, documents and audio are processed entirely on your device. Disconnect from the internet once a model has loaded and everything still works — which is the simplest proof available.

Because you are downloading an actual neural network, not calling an API. A 350M-parameter model quantised to 4-bit is roughly 350 MB of weights. That is the price of the model living on your machine instead of someone else's. It happens once — the browser caches it, and returning visits load from disk in seconds. The loader tells you when that has happened.

Yes, and it is unavoidable. Loading happens in two phases. First the weights download, which is network-bound and usually quick. Then ONNX Runtime has to parse the graph, allocate GPU buffers and compile a WebGPU shader for every operator in the network. That second phase is compute-bound, happens entirely on your machine, and reports no progress of any kind — there is no event to hook into, which is why you get an elapsed timer rather than a percentage.

Expect a few seconds for the 350M and embedding models, and up to a couple of minutes for Micro 3.4B on a modest GPU. It only happens on the first load of each model; afterwards both the weights and much of the compiled result are cached, and startup drops to seconds.

It holds the actual model weights. The ONNX format stores a network as a protobuf file, which has a 2 GB size limit, so anything larger keeps the graph structure in model_q4f16.onnx and moves the weight tensors into one or more external .onnx_data files beside it. Seeing several of them — _data, _data_1 and so on — is completely normal for the bigger models and simply means the weights were split across files.

No. Weights are stored in the browser's Cache Storage, so the second visit skips the download entirely and the progress panel says loaded from cache. Clearing site data or browsing history removes them and the next load downloads again. Each model is cached separately, so trying the 1B after the 350M is a fresh download.

WebGPU is the browser API that lets a page use your graphics card for computation. With it, generation runs at a usable speed. Without it the page falls back to WASM on your CPU, which still works but is several times slower — tolerable for the 30M embedding model, painful for the 3.4B chat model. WebGPU ships in recent Chrome, Edge and Brave on desktop; Safari and Firefox support is newer and sometimes behind a flag.

350M for speed — it loads fastest and handles rewriting, summarising and simple questions. 1B is the balanced default once you know you want this, noticeably better at following instructions. Micro 3.4B is the strongest and the slowest, worth the 2.2 GB only on a machine with a decent GPU. Start small; you can always size up, and the smaller model stays cached.

Not close, and it would be dishonest to suggest otherwise. Frontier models have hundreds of billions of parameters; a 1B model is roughly 0.3% of that. What these are genuinely good at is bounded, mechanical work — rewriting a paragraph, summarising something you paste in, extracting structured data, drafting boilerplate. Do not rely on them for facts, current events or multi-step reasoning. The trade you are making is capability for privacy and cost.

That is a numerical failure, not a prompting problem. Granite 4.0 uses a hybrid Mamba/SSM architecture where state accumulates across every token generated. If the weights are quantised too aggressively for that maths to hold up on your hardware, the values collapse, every token scores identically, and the model emits the same character forever.

The fix is precision. Each model needs a different one — the 350M runs at fp16, the 1B at q4, and Micro 3.4B at q4f16, matching what IBM ships in their own reference demo. Those are the defaults here. If you still hit it, change Precision in the bar above to fp32, which is the most numerically robust option, and load the model again. The page detects a run of repeated characters and stops rather than filling the screen.

They describe how precisely the model's numbers are stored. fp32 is full 32-bit precision — largest download, most reliable. fp16 halves that. q4 compresses the weights to 4 bits while computing in higher precision, and q4f16 does both, giving the smallest file.

Smaller is not simply worse-but-workable: below a certain point the arithmetic stops being stable, especially for SSM layers. Bigger models tolerate heavy quantisation better because they have more redundancy, which is why Micro is fine at q4f16 while the 350M is not.

Yes, more readily than a large model. Small models hallucinate confidently, particularly on dates, names, numbers and anything requiring knowledge rather than manipulation. Treat every factual claim as unverified. Where they are reliable is when you give them the material — paste the text and ask for a summary or a rewrite, rather than asking what they know.

It is IBM's small document-understanding model — it looks at an image and produces text. It handles photographed pages, scans, receipts, forms, screenshots and simple tables. It is the most downloaded Granite browser model by a wide margin, which is why it is here. Handwriting is unreliable, very low-resolution images produce poor results, and complex multi-column layouts can come out in an unexpected order.

Because Docling does not produce plain prose — it produces DocTags, a structured markup that records not just the words but where they sit on the page. The <loc_…> values are coordinates, and tags like <otsl>, <section_header> and <caption> record document structure. That is the whole point of the model: it understands layout rather than just running OCR.

The panel strips all of that for readability by default. Tick show raw DocTags to see the original, which is what you want if you intend to rebuild the document's structure rather than just lift the text out of it.

Docling is prompt-directed rather than one-size-fits-all, and these are the phrasings IBM uses in their own demo. Convert this page to docling handles a full page. Convert this table to OTSL targets a table specifically — OTSL is a compact table markup that preserves cells and spans. Convert chart to OTSL pulls the underlying values out of a chart image. Convert code to text is tuned for code screenshots. Using the matching prompt gives noticeably better results than a generic instruction.

Not directly — it takes images. For a PDF, convert the page to an image first with the PDF to JPG converter and drop that in. If your PDF already contains a real text layer you do not need a model at all: copy the text out, which is faster and perfectly accurate.

Granite Speech 4.0 covers English, French, German, Spanish, Portuguese and Japanese. Accuracy is best on clear single-speaker audio. Background noise, overlapping speakers and heavy accents all degrade it, as they do every ASR system. It transcribes rather than diarises, so it will not tell you who said what.

Because the browser library cannot run that model yet. IBM publish a transformers.js code sample for Granite Speech, but it calls a class named GraniteSpeechForConditionalGeneration that does not exist in any released transformers.js 3.x — 3.8.1 is current, and it ships GraniteForCausalLM and GraniteMoeHybridForCausalLM but no speech class. The sample targets unreleased code.

Rather than leave a button that downloads several hundred megabytes and then fails, the tab detects the missing class at startup and disables itself. Everything else is unaffected. When the class lands in a transformers.js release the detection flips automatically and the tab starts working. In the meantime, local Whisper does browser-based transcription today.

Both. Granite Speech does bidirectional speech translation as well as transcription, so it can take French, German, Spanish or Portuguese audio and return English text directly — no separate translation step. Pick the target from the dropdown above the Run button. Transcribing as spoken is the default, and if the model gets a prompt it does not understand it falls back to plain transcription rather than failing.

Yes — press Record from microphone, allow access, and press it again to stop. The recording stays in the page and is transcribed locally, exactly like a dropped file. Nothing is uploaded, which makes it usable for the kind of meeting note you would not send to a transcription service.

There is no imposed limit, but the whole clip is decoded into memory and processed in one pass, so a few minutes is comfortable and an hour-long recording will be slow or may exhaust memory. For long recordings, split the audio and transcribe in sections.

It converts every line and your query into a vector — a list of numbers representing meaning — then ranks lines by how close they sit to the query. That is why “I can't pay for my order” finds “My card was declined at checkout” despite sharing no words. It is the retrieval half of what people call RAG, running in your browser on a 35 MB model.

Sorting support tickets by topic, finding the closest existing FAQ entry to a new question, deduplicating a list of feature requests, or matching a description against a product catalogue. Anything where people describe the same thing in different words and keyword search consequently fails.

Granite is IBM's family of open models aimed at enterprise use — trained with documented data provenance, which matters to organisations that need to answer questions about what a model was trained on. They are released under Apache 2.0, a genuinely permissive licence: commercial use, modification and redistribution are all allowed, with no user-count restrictions of the kind some “open” model licences include.

Yes. Apache 2.0 places no restriction on what you do with what the model produces, and this page adds none. Check the licence text yourself if you are making a decision that matters — but for ordinary commercial use, Granite is about as unencumbered as open models get.

Once a model is cached, yes, completely. It is genuinely useful on a plane, behind a restrictive corporate firewall, or anywhere the network is unreliable. The one caveat is that the page itself and the transformers.js library must load first, so visit once while connected before relying on it.

Partly. Mobile WebGPU support is still patchy and phones have far less memory, so the CPU fallback usually kicks in. The 35 MB embedding model is fine. The chat models will be slow, and Micro will likely fail outright on a phone. This is a desktop tool in practice.

A token is roughly three-quarters of a word, and the counter shows how many the model produces each second along with how long the first one took. Speed depends almost entirely on your GPU and the model size — a discrete GPU running the 350M model feels instant, while an integrated one running Micro can drop to a few tokens per second. If it is slower than you want, pick a smaller model.

Yes. Open System prompt & sampling under the chat and rewrite the system prompt — it is applied to every message. Temperature controls randomness: near 0 makes it repeatable and literal, which suits extraction and code, while 0.7–1.0 gives more varied prose. Max new tokens caps the response length; raise it if answers are cut off mid-sentence.

No. The conversation lives in the page and is gone when you close the tab — nothing is written to storage and nothing is uploaded. Use Export .md if you want to keep it. Deliberately nothing is persisted, since the point of a local tool is that sensitive material does not linger.

Technically it is well suited to it — Apache 2.0, no data egress, no vendor account. Two practical notes: the one-time weight download comes from Hugging Face's CDN, which some corporate proxies block, and you should confirm your own policy on running models locally. Nothing here transmits company data anywhere, which is usually the hard part of that conversation.

It hit the max new tokens limit. Raise it in the sampling panel — 1024 or 2048 is sensible for longer answers. The trade-off is time: every extra token is more work for your GPU, and the default of 512 keeps responses quick.

Free, no account, no ads, no message cap and no subscription. There is nothing to meter — the compute is yours, not a server's, so it costs nothing to run regardless of how much you use it. Analytics on this site are privacy-first and cookie-free and record only that a page was viewed.

For the larger models, including Granite 4.1, use Ollama or LM Studio on your desktop — both take the GGUF builds directly and will run 8B and above comfortably on decent hardware. On Apple Silicon, the MLX builds are faster still. This page exists for the case where installing anything is not an option.