Free browser-based AI, explained
Getting started
What is this, exactly?
A real language model running inside your browser tab — not a front-end for someone else's API. You pick an open-weights model, it downloads once, and from then on every prompt is processed on your own GPU or CPU. There is no account, no server doing the thinking, and no per-message cost.
How do I run an LLM in my browser?
Open the page, pick a model and press Load model. The weights download once, are cached in your browser, and then run locally — on your GPU through WebGPU, or on your CPU in the Mobile / WASM tab. There is no install, no command line and no server involved.
Do I need to download or install anything?
No app, no runtime, no admin rights. The only thing that downloads is the model file itself, straight into the browser cache. Nothing is written to your Downloads folder and nothing is registered with the operating system — clearing site data removes every trace.
Do I need an account, an API key or an OpenAI subscription?
None of the three. Because the model runs on your own hardware there is nothing to authenticate against and no per-token bill. You never enter an email address, a key or a card number.
Why does the first load take so long?
You are downloading the model itself — anything from ~70 MB for the smallest mobile model to several gigabytes for a 7B one. It is stored with the browser's Cache API, so the second visit starts in seconds and needs no network at all. A cached model is marked with a badge in the picker.
Does it work offline?
Yes, once the weights are cached. You can switch off Wi-Fi entirely and keep chatting — a useful test in itself, because a tool that still answers with the network disconnected clearly is not calling a server. Only the first download of each model needs a connection.
What does “Best for my device” do?
It asks your GPU for the largest buffer it can allocate and picks a model that fits: Llama 3.2 1B on modest hardware, Qwen 3 4B on a mid-range GPU, and Qwen 3 8B on a card with several gigabytes to spare. It is a starting point rather than a verdict — you can always pick something bigger or smaller by hand, and the note under the picker shows each model's memory needs.
Is it free? Are there any limits?
Completely free, with no account, no subscription, no message cap and no watermark. There is no rate limit either, because there is no server to rate-limit — the only ceiling is how fast your own hardware can generate tokens.
Documents, images, thinking & modes
What does 🧠 Think do?
Qwen 3 models can reason before they answer. With Think on, they write out their chain of thought first — shown in a collapsible 💭 block with how long it took — and then answer. It makes arithmetic, logic and multi-step instructions noticeably more accurate, at the cost of time: Qwen 3 0.6B wrote about 550 thinking tokens before answering whether 91 is prime. Turn it off for quick replies. DeepSeek R1 and Ministral Reasoning always think; the toggle only appears for models that support switching.
Can it read images or screenshots?
Yes, with Phi 3.5 Vision. Pick it, load it, and attach an image with 🖼 — or paste a screenshot straight into the message box, or drop a file onto the chat. The image is resized to at most 1024 px inside your browser before the model sees it, and it never leaves your device. It is a 2.77 GB download and needs about 4 GB of GPU memory.
Can I chat with a PDF or a document?
Yes. Press 📎 or drop a file on the chat: PDF, TXT, Markdown, CSV, JSON, HTML or source code, up to 25 MB. Short files go into your message in full. Longer ones are split into overlapping parts and indexed locally with IBM's Granite Embedding Multilingual R2 (a 98 MB one-time download); for every question the four most relevant parts are handed to the chat model, labelled with the file and part number so you can check the source. Because the models here have a 4K-token window, this is what makes a 40-page PDF usable at all.
What is JSON mode?
Tick { } JSON and the answer is constrained to valid JSON: WebLLM applies a grammar while the model generates, so it cannot produce anything else. It is ideal with the Extract JSON template — “Sophie lives in Gent” comes back as {"name": "Sophie", "city": "Gent"} every time, ready to paste into code.
Is there a distraction-free or full-screen mode?
Yes. Simple hides everything on the page except the chat and lets it fill the window; the page remembers it, and ?mode=simple opens straight into it. Full screen takes the chat over your whole screen — press Esc to come back. The model stays loaded either way.
What happens when a conversation gets too long?
These models see about 4,000 tokens at a time. The small meter under the input shows how full that window is. When a conversation outgrows it, the oldest turns are quietly left out of what the model sees (they stay on your screen), so the chat keeps working instead of failing with an error.
Choosing a model
Which model should I pick?
Llama 3.2 1B is still the sensible default: 695 MB, quick to load and reliable at everyday instructions. For visible step-by-step reasoning pick Qwen 3 (0.6B, 1.7B, 4B or 8B) and switch 🧠 Think on. Qwen 3 4B is the best quality-per-gigabyte here if your GPU has about 3.5 GB free, and Qwen 3 8B the strongest overall. To ask about pictures, Phi 3.5 Vision is the only model that reads images. The new Qwen 3.5 models are very fast but answer each message on its own in this runtime. Press 🎯 Best for my device to let the page choose from your GPU's limits.
What do 0.5B, 1B, 3B and 7B actually mean?
The number of parameters, in billions — roughly, how much the model knows and how well it can reason. Each step up costs download size, memory and speed: a 1B model is around 880 MB and feels instant, a 3B is about 2 GB and noticeably more coherent, a 7B is over 4 GB and needs a proper graphics card. Quality rises with size, but so does the wait.
Which model is best for coding?
Qwen 2.5 Coder is trained specifically on code and comes in three sizes here: 1.5B for quick completions and shell one-liners, 3B for most everyday scripting, and 7B — the best coding model on the page — if your GPU has about 5 GB free. Qwen 3 4B with Think on is the better choice when you need the model to reason about why code fails rather than just write it. None of them will replace a frontier model on a large codebase.
What are the DeepSeek R1 distills?
Small models trained to imitate DeepSeek R1's long chain of thought. They write their reasoning before the answer; the page now folds that reasoning into a collapsible 💭 Thought for N s block so the answer stays readable, and keeps it out of the conversation history so later turns stay within the context window.
Which model is best for languages other than English?
The Qwen families are the strongest multilingual options here: Qwen 3 officially covers more than 100 languages and dialects, and Qwen 2.5 handles Chinese, Dutch, French, German, Spanish and many more well for its size. Gemma 3 and Phi 4 Mini are good alternatives. Small models drift more outside English, so keep prompts explicit — setting the system prompt to “always answer in Dutch” works better than asking mid-conversation. The Translate templates help too.
Can I run a 7B model?
Only with a GPU that has roughly 5–6 GB of free video memory — the weights alone are 4–5 GB before the conversation cache. The Large group holds them: Qwen 3 8B (the strongest model on the page), Llama 3.1 8B, Qwen 2.5 Coder 7B, the two DeepSeek R1 distills, Qwen 3.5 9B and Mistral 7B. The note under the picker shows how much GPU memory each one needs. On integrated graphics they either fail to allocate or crawl, and a 3–4B model is the better trade.
Which models does the Mobile / WASM tab use?
Small ones that run on the CPU: SmolLM2 135M (137 MB), SmolLM2 360M (365 MB, the default), Qwen 2.5 0.5B and Qwen 3 0.6B for other languages, and Llama 3.2 1B and SmolLM2 1.7B when quality matters more than speed. All run in 8-bit, which measured about 7× faster than 4-bit on the CPU (13.9 vs 1.9 tokens/s for SmolLM2 360M), and in a background thread so the page never freezes while it writes.
If I switch models, does it download again?
Each model is cached separately, so the first switch to a new model downloads it and every later switch is instant. Models you have already cached are flagged in the picker, which makes it easy to hop between a fast one and a smarter one without paying the download twice.
What is a context window, and which model has the largest?
It is how much text the model can hold in mind at once — your system prompt, the conversation so far, any document parts and its own reply all count against it. Many of these models support 32K tokens or more on paper, but WebLLM runs its builds with a window of about 4,000 tokens to keep GPU memory in check. The meter under the input shows how full it is; when a chat outgrows it, the oldest turns are left out of what the model sees. For long documents, attach them with 📎 instead of pasting — only the relevant parts are sent.
What does “q4f16” in the model name mean?
It is the quantisation: the weights are compressed to 4 bits with 16-bit floating point used during computation. That is what makes a multi-billion-parameter model small enough to download and fit in browser memory. The cost is a small amount of accuracy, which is a bargain compared with not being able to run the model at all.
Privacy and data
Does my data get sent anywhere?
No. Prompts and replies never leave the tab. The single network request the tool makes is the one-time download of the model weights from the Hugging Face CDN; after that you can disconnect entirely and it keeps working. There is no inference server, because your device is the inference server.
Are my chats stored or logged?
Nothing is ever uploaded. By default the page also keeps your recent conversations in this browser's local storage so you can reopen them from History — they never leave your device, and you can switch 💾 Save chats off or delete all history at any time. Attached documents and images are never stored; only the text of the conversation is.
Is browser-based AI genuinely private, or just marketed that way?
Genuinely, and you can check it yourself rather than take anyone's word for it. There is no API key in the page to send anything with, no endpoint to send it to, and the model answers with the network switched off — which is impossible for a cloud chatbot.
How can I verify nothing is uploaded?
Two ways. Open your browser's developer tools, go to the Network tab, and watch it while you send a message: after the model has loaded there are no requests at all. Or simply disable Wi-Fi and keep chatting — if the answers still come, they are being generated locally.
Can I paste confidential or GDPR-sensitive text into it?
Technically nothing leaves your device, so there is no transfer to a third party and no processor to sign an agreement with — which is exactly why local models are attractive for sensitive text. That said, your own organisation's policy still applies, so check it before pasting client data anywhere, including here.
Does the model download tell anyone what I am doing?
Downloading the weights is an ordinary file request to a CDN, so that CDN sees a file being fetched, as it would for any image or script. It carries no prompt, no conversation and no identity beyond a normal web request — and it happens once per model, not per message.
Does it work in private or incognito mode?
It works, but private windows discard storage when you close them, so the model downloads again the next time. If you plan to use it regularly, a normal window keeps the cache and saves you the wait.
Speed and hardware
What hardware do I need?
For the desktop tab, a GPU that supports WebGPU and has enough free memory for the model: roughly 1 GB for a 1B model, 2–3 GB for a 3B, and 5 GB or more for a 7B. Integrated graphics handle the 0.5B–1B models comfortably. If none of that applies to your machine, the Mobile / WASM tab runs on the CPU on practically anything.
How fast should it be?
On a modern discrete GPU a 1B model typically streams faster than you can read, and a 3B model still feels conversational. On integrated graphics expect it to be readable but unhurried, and in CPU / WASM mode expect a few words per second. The live tok/s counter under the chat box shows exactly what your device is doing.
Why is generation slow or stuttering?
Usually the model is too large for the hardware, so memory is being shuffled instead of used. Drop a size class, close other GPU-heavy tabs — video calls and 3D pages compete for the same memory — and check that your laptop is not in a battery-saver profile that caps the GPU. A long conversation also slows things down, because the whole history is reprocessed; reset the chat to get the speed back.
Why does the answer stop in the middle?
It hit the max tokens limit, which defaults to 1024. For scripts, long code or detailed documents raise it to 2048, 4096 or 8192 next to the Send button. It is a deliberate cap rather than a failure — without it a small model can ramble for a very long time.
What does the temperature slider do?
It controls how adventurous the sampling is. Low values (around 0.2) make the model repetitive but predictable, which is what you want for extraction, JSON and code. Higher values (0.8 and above) produce more varied writing at the cost of accuracy. The default of 0.7 is a reasonable middle for chat.
How much faster is GPU mode than WASM mode?
Measured on the same Apple-silicon laptop: Qwen 3 0.6B wrote about 60–80 tokens a second on WebGPU, while the CPU models manage 3–35 depending on size (SmolLM2 135M 35/s, 360M 14/s, Llama 3.2 1B 3.5/s). WASM mode exists so the tool still works on hardware and browsers without WebGPU.
It says out of memory, or the GPU device was lost.
The model did not fit. Choose a smaller one, close other tabs using the GPU, and try again — the error message names the limit it hit. On laptops that switch between integrated and discrete graphics, forcing the browser onto the discrete GPU in the system settings often solves it outright.
Will it drain my battery or heat the laptop?
While it is generating, yes — you are running a neural network on your own silicon, so the fans may spin up much as they would during a game. It only draws power while a reply is being produced; the model sitting in memory idle costs nothing.
Browsers, storage and troubleshooting
Which browsers support WebGPU today?
Chrome and Edge have shipped it by default since version 113 on Windows, macOS and ChromeOS, and on Android 12+ since Chrome 121. Safari enables it by default in macOS Tahoe 26, iOS 26 and iPadOS 26. Firefox ships it on Windows from version 141 and on Apple Silicon Macs from 147, with Linux and Android still in Nightly. Anything not on that list can still use the Mobile / WASM tab.
Why does it say “WebGPU not available”?
Your browser, or that machine's graphics driver, is not exposing WebGPU — common on older browser versions, on Linux with certain drivers, in some virtual machines, and where enterprise policy disables it. The page detects this and moves you to the Mobile / WASM tab automatically, so you can still use the tool; updating the browser is the fix if you want GPU speed.
Does it work on iPhone or iPad?
Yes, in the Mobile / WASM tab, with one caveat: iOS Safari enforces a hard memory ceiling of roughly 1–1.5 GB per tab, so only the smallest model (SmolLM2 135M) loads reliably. Larger ones crash the tab with “a problem repeatedly occurred”. For anything more capable, open the page on a desktop.
Does it work on Android?
Yes. Chrome on Android 12 and later supports WebGPU on most recent chipsets, and where it does not, the WASM tab runs on the CPU. Stick to the small models — phone memory limits bite long before the model quality does.
It will not download the model on my work network.
Corporate proxies and content filters frequently block the Hugging Face CDN, which is where the weights come from, and some inspect and break large downloads. That is a network policy question rather than a browser one — the same page usually works immediately on a home connection or a phone hotspot.
Where are the weights stored, and how do I delete them?
In the browser's Cache Storage for this site, not in your file system. Clearing site data (or “Cookies and other site data” for this domain) removes every cached model and frees the space; the tool then behaves like a first visit. Nothing outside the browser profile is touched.
Can the browser throw the cached model away?
Yes. Browsers evict cached data when disk space runs low or a profile is cleaned, and private windows discard it on close. If a model that loaded instantly yesterday starts downloading again, that is what happened — nothing is broken.
How much disk space will this use?
Only what the models you actually load take up: from 137 MB for the smallest CPU model and 204 MB for the smallest GPU model to about 5 GB for Qwen 3.5 9B. Attaching a large document also downloads the 100 MB embedding model once. Every model you try is kept until you clear site data, so trying several does add up — worth remembering on a machine that is short on space.
What it is good at — and what it is not
Why is it less capable than ChatGPT or Claude?
Because it is smaller by three orders of magnitude. Frontier models run on server racks with hundreds of billions of parameters; a 1B model in a browser tab is a fraction of a percent of that, and it is remarkable that it works at all. Judge it as a fast local assistant for small jobs, not as a replacement for a frontier model.
So what is it actually good at?
Short, well-defined text work: rewriting a paragraph formally or casually, summarising something you paste in, drafting a reply, extracting fields into JSON, explaining a concept simply, and small code and query snippets. The preset buttons above the chat box are shortcuts to exactly those tasks. Anything that needs current facts or long reasoning chains is the wrong job for it.
Can it search the web or read my files?
It has no internet access. It can read files you attach: text, Markdown, CSV, JSON, code and PDF. Small files are added to your message whole; larger ones are split into parts and searched locally with a multilingual embedding model, so only the most relevant passages are shown to the chat model. Everything happens in your browser — the file content is never uploaded.
Does it make things up?
More often than a large model, yes. Small models are confident and frequently wrong about names, dates, numbers and anything specialised, and they have no way to look anything up. Treat factual claims as drafts to verify, and lean on the tasks where the source material is in your prompt — rewriting, summarising, extracting.
Can I change its personality or behaviour?
Yes. Open System prompt (persona) under the model picker and describe how it should respond — “you are a terse senior engineer”, “always answer in Dutch”, “reply only with JSON”. It applies to every message in the conversation, and it is by far the most effective way to improve the output of a small model.
Can it write code?
For small, self-contained things, yes — a regex, a shell one-liner, a function, a KQL query from the preset. Use Qwen 2.5 Coder or Phi 3.5 Mini, keep the request narrow, raise max tokens so it is not truncated, and read what it produces before running it. It has no idea what is in your codebase.
Can I use the output commercially?
Generally yes, but the licence belongs to the model rather than to this page, and they differ: Llama has Meta's community licence, Gemma has Google's terms of use, most Qwen and SmolLM2 releases are Apache 2.0 and Phi is MIT. If the output is going into a product, read the licence for the specific model you used.
Under the hood
What technology powers this?
The desktop tab uses WebLLM 0.2.85 from MLC AI, which compiles open-weights models to run on WebGPU. The mobile tab uses Hugging Face transformers.js 4.3 on ONNX Runtime Web, executing 8-bit models on the CPU through WebAssembly in a Web Worker. Document search uses IBM's Granite Embedding Multilingual R2, and PDFs are read with Mozilla's pdf.js. Everything around them is plain JavaScript and CSS in a single HTML file.
Where do the model weights come from?
From the Hugging Face CDN, in the pre-compiled MLC format for the WebGPU tab and ONNX for the WASM tab. They are open-weights models published by Meta, Alibaba (Qwen), Microsoft (Phi), Google DeepMind (Gemma), Mistral AI, Hugging Face (SmolLM2), DeepSeek and IBM (the Granite embedding model used for document search) — the same files anyone can download and run locally.
Is this the same as Ollama or LM Studio?
Same idea, different delivery. Those are desktop applications you install, and they will always be faster and support far larger models because they use your hardware directly. This runs the model in a browser tab with nothing installed at all, which makes it the easier way to try a local model — on a locked-down machine, on someone else's computer, or simply to see what the fuss is about.
When should I use a cloud model instead?
When the task needs breadth, accuracy on facts, long documents, up-to-date information or serious reasoning — a frontier model will do in one attempt what a 1B model cannot do at all. Use this when the text is sensitive, when you are offline, when you want an instant answer without opening an account, or when the job is small enough that a small model is simply the quicker tool.
免费浏览器端 AI 详解
快速上手
这到底是什么?
一个真正的语言模型运行在你的浏览器标签页里——而不是别人 API 的前端界面。你选择一个开放权重模型,下载一次,此后每一条提示都在你自己的 GPU 或 CPU 上处理。没有账户、没有服务器替你"思考",也没有按消息计费。
如何在浏览器中运行 LLM?
打开页面,选择一个模型,点击"加载模型"。权重只需下载一次,会缓存在你的浏览器中,之后就在本地运行——在 Desktop 标签页通过 WebGPU 使用你的 GPU,或在 Mobile / WASM 标签页使用你的 CPU。全程无需安装、无需命令行、也不涉及任何服务器。
我需要下载或安装什么吗?
不需要 App、不需要运行时、也不需要管理员权限。唯一会下载的是模型文件本身,直接存入浏览器缓存。不会写入你的"下载"文件夹,也不会在操作系统中注册任何内容——清除网站数据即可抹去所有痕迹。
我需要账户、API key 或 OpenAI 订阅吗?
三者都不需要。因为模型运行在你自己的硬件上,没有任何需要验证身份的对象,也没有按 token 计费的账单。你无需输入邮箱、密钥或银行卡号。
为什么第一次加载这么慢?
你正在下载模型本身——最小的移动端模型约 70 MB,而 7B 模型则有好几 GB。它通过浏览器的 Cache API 存储,所以第二次访问几秒钟就能启动,完全不需要联网。已缓存的模型会在选择器中标出徽章。
它能离线使用吗?
可以,只要权重已缓存。你可以完全关闭 Wi-Fi 继续聊天——这本身就是一个很好的测试,因为断网后仍能回答问题,显然说明它没有在调用服务器。只有每个模型第一次下载时需要联网。
“最适合我的设备”是做什么的?
它会询问你的 GPU 能分配的最大缓冲区,然后挑选合适的模型:普通硬件用 Llama 3.2 1B,中端 GPU 用 Qwen 3 4B,有充足显存的显卡用 Qwen 3 8B。这只是一个起点而非定论——你随时可以手动选择更大或更小的模型,选择器下方的提示会显示每个模型的显存需求。
它是免费的吗?有任何限制吗?
完全免费,无需账户、无需订阅、没有消息上限,也没有水印。也没有速率限制,因为根本没有服务器可供限速——唯一的上限是你自己硬件生成 token 的速度。
文档、图片、思考与模式
🧠 思考有什么用?
Qwen 3 模型可以在回答前先推理。打开思考后,它会先写出思维链——显示在可折叠的 💭 区块中,并标出用时——然后再回答。这会让算术、逻辑和多步骤指令明显更准确,代价是时间:Qwen 3 0.6B 在回答“91 是否为质数”之前写了约 550 个思考 token。需要快速回复时请关闭。DeepSeek R1 和 Ministral Reasoning 总是会思考;该开关只对支持切换的模型显示。
它能读取图片或截图吗?
可以,使用 Phi 3.5 Vision。选择并加载它,然后用 🖼 附加图片——或者直接把截图粘贴到输入框,或拖放文件到聊天区。图片在交给模型之前会在浏览器内缩放到最多 1024 像素,绝不会离开你的设备。该模型下载 2.77 GB,需要约 4 GB 显存。
我能和 PDF 或文档对话吗?
可以。点击 📎 或把文件拖到聊天区:PDF、TXT、Markdown、CSV、JSON、HTML 或源代码,最大 25 MB。短文件会整篇加入消息;较长的文件会被切分成有重叠的片段,并用 IBM 的 Granite Embedding Multilingual R2(一次性下载 98 MB)在本地建立索引;每次提问时,最相关的四个片段会连同文件名和片段编号交给聊天模型,方便你核对出处。由于这些模型的窗口只有 4K token,正是这一点让 40 页的 PDF 变得可用。
什么是 JSON 模式?
勾选 { } JSON 后,回答会被约束为合法的 JSON:WebLLM 在生成时应用语法约束,模型无法输出其他内容。它与“提取 JSON”模板搭配最佳——“Sophie lives in Gent”每次都会返回 {"name": "Sophie", "city": "Gent"},可直接粘贴到代码中。
有没有无干扰或全屏模式?
有。简洁模式会隐藏页面上除聊天以外的所有内容,让聊天填满窗口;页面会记住这个选择,?mode=simple 可直接进入。全屏让聊天占满整个屏幕——按 Esc 返回。两种方式下模型都保持加载。
对话太长时会怎样?
这些模型一次大约能看到 4,000 个 token。输入框下方的小进度条显示窗口已用多少。当对话超出窗口时,最早的轮次会被悄悄地从模型所见内容中省略(仍保留在你的屏幕上),因此聊天会继续工作,而不是报错。
选择模型
我应该选哪个模型?
Llama 3.2 1B 依然是稳妥的默认选择:695 MB,加载快,日常指令表现可靠。想看逐步推理,选 Qwen 3(0.6B、1.7B、4B 或 8B)并打开 🧠 思考。如果 GPU 有约 3.5 GB 空闲显存,Qwen 3 4B 是本页性价比最高的模型;Qwen 3 8B 则整体最强。想问图片相关的问题,Phi 3.5 Vision 是唯一能读图的模型。新的 Qwen 3.5 模型非常快,但在此运行时中每条消息都单独回答。点击 🎯 最适合我的设备,让页面根据你的 GPU 限制自动选择。
0.5B、1B、3B 和 7B 到底是什么意思?
这是参数量,以十亿(B)为单位——大致代表模型知道多少、推理能力有多强。每上一个档次都会增加下载体积、内存占用和运行速度的代价:1B 模型约 880 MB,感觉是即时响应;3B 模型约 2 GB,明显更连贯;7B 模型超过 4 GB,需要一块像样的显卡。质量随体积提升,但等待时间也随之增加。
哪个模型最适合编程?
Qwen 2.5 Coder 专门针对代码训练,这里提供三种规格:1.5B 适合快速补全和 shell 单行命令,3B 适合大多数日常脚本,7B 是本页最好的编程模型(需约 5 GB 空闲显存)。如果你需要模型分析代码为什么出错,而不只是写代码,开启“思考”的 Qwen 3 4B 更合适。它们都无法在大型代码库上取代前沿模型。
什么是 DeepSeek R1 蒸馏模型?
经过训练来模仿 DeepSeek R1 长思维链的小模型。它们会先写出推理过程再给出答案;页面现在会把推理过程折叠进可展开的 💭 思考了 N 秒 区块,让答案保持易读,并且不把推理写入对话历史,让后续轮次不超出上下文窗口。
哪个模型最适合非英语场景?
Qwen 系列是这里最强的多语言选择:Qwen 3 官方支持 100 多种语言和方言,Qwen 2.5 就其体积而言也能很好地处理中文、荷兰语、法语、德语、西班牙语等。Gemma 3 和 Phi 4 Mini 也是不错的替代。小模型在英语之外更容易跑偏,所以提示要明确——在系统提示词里写“始终用中文回答”比对话中途提出效果更好。“翻译”模板也有帮助。
我能运行 7B 模型吗?
只有在 GPU 有约 5–6 GB 空闲显存时才行——仅权重就有 4–5 GB,还不算对话缓存。“大型”分组中包括:Qwen 3 8B(本页最强)、Llama 3.1 8B、Qwen 2.5 Coder 7B、两个 DeepSeek R1 蒸馏模型、Qwen 3.5 9B 和 Mistral 7B。选择器下方的提示会显示每个模型所需的显存。在集成显卡上,它们要么分配失败,要么极慢,3–4B 模型是更好的选择。
Mobile / WASM 标签页使用哪些模型?
在 CPU 上运行的小模型:SmolLM2 135M(137 MB)、SmolLM2 360M(365 MB,默认)、用于其他语言的 Qwen 2.5 0.5B 和 Qwen 3 0.6B,以及更看重质量时的 Llama 3.2 1B 和 SmolLM2 1.7B。全部以 8 位运行——在 CPU 上实测比 4 位快约 7 倍(SmolLM2 360M:13.9 对 1.9 tokens/秒)——并在后台线程中运行,因此生成时页面绝不会卡住。
切换模型会重新下载吗?
每个模型分别缓存,所以第一次切换到新模型会下载,之后每次切换都是即时的。已缓存的模型会在选择器中标记出来,方便你在快速模型和更强模型之间自如切换,而不必重复下载。
什么是上下文窗口?哪个模型最大?
它是模型一次能“记住”多少文本——系统提示词、目前的对话、文档片段以及它自己的回复都会占用。许多模型名义上支持 32K 甚至更多 token,但 WebLLM 为了控制显存,以约 4,000 token 的窗口运行这些模型。输入框下方的进度条显示已用多少;对话超出窗口时,最早的轮次会从模型所见内容中省略。长文档请用 📎 附加,而不是直接粘贴——只会发送相关的部分。
模型名称中的“q4f16”是什么意思?
这是量化方式:权重被压缩到 4 位,计算时使用 16 位浮点数。这正是让一个数十亿参数的模型小到能下载并装入浏览器内存的原因。代价是少量精度损失,但相比完全无法运行模型而言,这是很划算的权衡。
隐私与数据
我的数据会被发送到任何地方吗?
不会。 提示词和回复永远不会离开这个标签页。这个工具发出的唯一网络请求,是从 Hugging Face CDN 一次性下载模型权重;之后你可以完全断网,它依然能正常工作。这里没有推理服务器,因为你的设备本身就是推理服务器。
我的对话会被存储或记录吗?
任何内容都不会上传。默认情况下,页面会把最近的对话保存在本浏览器的本地存储中,方便你从历史对话重新打开——它们绝不会离开你的设备,你可以随时关闭 💾 保存对话 或删除全部历史。附加的文档和图片从不保存,只保存对话文字。
浏览器端 AI 是真的私密,还是只是营销说法?
是真的私密,而且你可以自己验证,不必只听别人的说法。页面中没有任何可用来发送数据的 API key,也没有可以发送数据的目标地址,而且断网后模型依然能回答——这对云端聊天机器人来说是不可能做到的。
我如何验证没有任何内容被上传?
两种方法。打开浏览器的开发者工具,切到 Network 标签页,发送消息时观察它:模型加载完成后,不会出现任何网络请求。或者干脆关闭 Wi-Fi 继续聊天——如果依然能得到回答,说明它是在本地生成的。
我可以粘贴机密或涉及 GDPR 的敏感文本吗?
从技术上讲,任何内容都不会离开你的设备,因此不存在向第三方传输的问题,也不需要与任何处理者签署协议——这正是本地模型对敏感文本很有吸引力的原因。话虽如此,你所在组织的政策仍然适用,所以在这里粘贴客户数据之前,请先确认一下相关规定。
下载模型会向任何人透露我在做什么吗?
下载权重只是一次普通的文件请求,发往一个 CDN,该 CDN 看到的就是"有一个文件被获取",和获取任何图片或脚本没有区别。它不携带任何提示词、对话内容,也没有超出普通网络请求的身份信息——而且每个模型只发生一次,不是每条消息都发生。
它能在隐私 / 无痕模式下使用吗?
可以,但隐私窗口在关闭时会清除存储,所以下次使用时模型会重新下载。如果你打算经常使用,普通窗口能保留缓存,省去等待时间。
速度与硬件
我需要什么样的硬件?
对于桌面标签页,需要一块支持 WebGPU、并有足够空闲内存运行模型的 GPU:1B 模型大约需要 1 GB,3B 模型需要 2–3 GB,7B 模型则需要 5 GB 以上。集成显卡可以轻松应对 0.5B–1B 模型。如果你的设备都不满足,Mobile / WASM 标签页几乎可以在任何设备的 CPU 上运行。
它应该有多快?
在现代独立显卡上,1B 模型的生成速度通常比你阅读的速度还快,3B 模型也依然感觉像正常对话。在集成显卡上,预期是可读但不算快,而在 CPU / WASM 模式下,预期是每秒几个词。聊天框下方的实时 tok/s 计数器会准确显示你设备的表现。
为什么生成速度慢或卡顿?
通常是模型对硬件来说太大了,内存在被反复调度而不是被有效利用。降低一个体积档次、关闭其他占用 GPU 的标签页——视频通话和 3D 页面会争抢同样的内存——并确认笔记本没有处于限制 GPU 性能的省电模式。长对话也会拖慢速度,因为整个历史记录都要被重新处理;重置对话即可恢复速度。
为什么回答会中途停止?
它触及了 max tokens 上限,默认值是 1024。对于脚本、长代码或详细文档,请在 Send 按钮旁将其调高到 2048、4096 或 8192。这是一个刻意设置的上限,而不是故障——没有它,小模型可能会长篇大论、没完没了。
温度滑块是做什么的?
它控制采样的"大胆程度"。较低的值(约 0.2)会让模型重复但可预测,这适合用于提取信息、JSON 和代码。较高的值(0.8 及以上)会产生更多样化的文字,但准确性会下降。默认值 0.7 是聊天场景比较合理的折中。
GPU 模式比 WASM 模式快多少?
在同一台 Apple 芯片笔记本上实测:Qwen 3 0.6B 在 WebGPU 上每秒约输出 60–80 个 token,而 CPU 模型视体积在每秒 3–35 个之间(SmolLM2 135M 35/秒、360M 14/秒、Llama 3.2 1B 3.5/秒)。WASM 模式的存在,是为了让没有 WebGPU 的硬件和浏览器也能使用这个工具。
提示内存不足,或者“GPU device was lost”。
模型放不下了。选择一个更小的模型,关闭其他占用 GPU 的标签页,再试一次——错误信息会指出具体触及了哪个限制。在会在集成显卡和独立显卡之间切换的笔记本上,在系统设置中强制浏览器使用独立显卡,往往能彻底解决问题。
它会耗电或让笔记本发热吗?
在生成过程中,会的——你是在自己的芯片上运行一个神经网络,风扇可能会像玩游戏时一样转起来。它只在生成回复时耗电;模型闲置在内存中时不会产生任何额外功耗。
浏览器、存储与故障排查
目前哪些浏览器支持 WebGPU?
Chrome 和 Edge 自 113 版本起,在 Windows、macOS 和 ChromeOS 上默认已支持,Android 12+ 上则从 Chrome 121 起支持。Safari 在 macOS Tahoe 26、iOS 26 和 iPadOS 26 中默认启用。Firefox 在 Windows 上从 141 版本起支持,在 Apple Silicon Mac 上从 147 版本起支持,Linux 和 Android 上仍处于 Nightly 阶段。不在此列表中的浏览器仍可使用 Mobile / WASM 标签页。
为什么提示“WebGPU 不可用”?
你的浏览器,或者该设备的显卡驱动,没有开放 WebGPU 支持——这在较旧的浏览器版本、部分驱动的 Linux 系统、某些虚拟机,以及禁用了该功能的企业策略环境中很常见。页面检测到这一点后会自动把你切换到 Mobile / WASM 标签页,让你依然能使用工具;如果想要 GPU 速度,升级浏览器就是解决办法。
它能在 iPhone 或 iPad 上使用吗?
可以,通过 Mobile / WASM 标签页,但有一个注意事项:iOS Safari 对每个标签页强制施加约 1–1.5 GB 的硬性内存上限,所以只有最小的模型(SmolLM2 135M)能可靠加载。更大的模型会让标签页崩溃,提示“a problem repeatedly occurred”。如果需要更强的能力,请在桌面端打开此页面。
它能在 Android 上使用吗?
可以。Android 12 及更高版本上的 Chrome 在大多数近期芯片上支持 WebGPU,不支持时则由 WASM 标签页在 CPU 上运行。请优先选择小模型——手机的内存限制往往比模型质量更早成为瓶颈。
在我的工作网络上无法下载模型。
企业代理和内容过滤器经常会屏蔽 Hugging Face CDN(权重正是从这里下载的),有些还会检查并中断大文件下载。这是网络策略问题,而不是浏览器问题——同一个页面在家庭网络或手机热点上通常能立即正常工作。
权重存储在哪里?我该如何删除它们?
存储在浏览器针对该网站的 Cache Storage 中,而不是你的文件系统里。清除网站数据(或针对该域名清除“Cookie 及其他网站数据”)会移除所有已缓存的模型并释放空间;之后工具的表现就如同首次访问一样。浏览器配置文件之外的任何内容都不会受到影响。
浏览器会自动丢弃已缓存的模型吗?
会的。当磁盘空间不足或配置文件被清理时,浏览器会清除缓存数据,隐私窗口在关闭时也会丢弃缓存。如果一个昨天还能瞬间加载的模型突然又开始重新下载,原因就在这里——并不是出了什么故障。
这会占用多少磁盘空间?
只占用你实际加载过的模型:最小的 CPU 模型 137 MB、最小的 GPU 模型 204 MB,最大的 Qwen 3.5 9B 约 5 GB。附加大型文档还会一次性下载约 100 MB 的嵌入模型。你试过的每个模型都会保留,直到你清除站点数据,所以多试几个会累积起来——在空间紧张的电脑上值得注意。
它擅长什么——以及不擅长什么
为什么它不如 ChatGPT 或 Claude 强大?
因为它的体积小了三个数量级。前沿模型运行在拥有数千亿参数的服务器机群上;浏览器标签页里的 1B 模型只是其中的一小部分,而它还能正常工作,已经很了不起了。把它当作一个处理小任务的快速本地助手,而不是前沿模型的替代品。
那它究竟擅长什么?
短小、边界清晰的文字工作:把一段话改写得更正式或更随意、总结你粘贴进来的内容、起草回复、把信息提取为 JSON、简单地解释一个概念,以及小段代码和查询语句。聊天框上方的预设按钮正是这些任务的快捷方式。任何需要最新事实或长链条推理的任务,都不适合交给它。
它能搜索网络或读取我的文件吗?
它无法联网。它可以读取你附加的文件:文本、Markdown、CSV、JSON、代码和 PDF。小文件会整篇加入你的消息;较大的文件会被切分,并用多语言嵌入模型在本地检索,只把最相关的部分交给模型。文件本身绝不会离开你的设备。
它会编造内容吗?
比大模型更容易,是的。小模型对名称、日期、数字和任何专业内容都表现得"很自信却常常出错",而且它无法查证任何信息。把事实性的结论当作需要核实的草稿,并优先使用那些原始材料就在你提示词里的任务——改写、总结、提取。
我可以改变它的个性或行为方式吗?
可以。在模型选择器下方打开"系统提示词(角色设定)",描述它应该如何回应——例如“你是一位言简意赅的资深工程师”、“始终用中文回答”、“只用 JSON 回复”。它会应用到对话中的每一条消息,是提升小模型输出质量最有效的方法。
它能写代码吗?
对于小型、独立的任务,可以——比如正则表达式、shell 单行命令、一个函数,或者预设里的 KQL 查询。使用 Qwen 2.5 Coder 或 Phi 3.5 Mini,把需求写得具体一些,调高 max tokens 以免被截断,并在运行之前先检查它写出的内容。它并不了解你代码库里的任何内容。
我可以将输出用于商业用途吗?
一般来说可以,但许可证属于模型本身,而不是这个页面,而且各不相同:Llama 遵循 Meta 的社区许可证,Gemma 遵循 Google 的使用条款,大多数 Qwen 和 SmolLM2 版本采用 Apache 2.0,Phi 采用 MIT 许可证。如果输出要用于产品,请查阅你所使用的具体模型的许可证。
技术原理
是什么技术驱动了这个工具?
桌面标签页使用来自 MLC AI 的 WebLLM 0.2.85,它把开放权重模型编译为可在 WebGPU 上运行的形式。移动标签页使用 Hugging Face transformers.js 4.3(基于 ONNX Runtime Web),在 Web Worker 中通过 WebAssembly 在 CPU 上运行 8 位模型。文档检索使用 IBM 的 Granite Embedding Multilingual R2,PDF 由 Mozilla 的 pdf.js 读取。其余部分都是单个 HTML 文件中的纯 JavaScript 和 CSS。
模型权重来自哪里?
来自 Hugging Face CDN:WebGPU 标签页使用预编译的 MLC 格式,WASM 标签页使用 ONNX 格式。它们是由 Meta、阿里巴巴(Qwen)、微软(Phi)、Google DeepMind(Gemma)、Mistral AI、Hugging Face(SmolLM2)、DeepSeek 和 IBM(用于文档检索的 Granite 嵌入模型)发布的开放权重模型——任何人都可以下载并在本地运行的同一批文件。
这和 Ollama 或 LM Studio 是一回事吗?
理念相同,交付方式不同。那些是需要安装的桌面应用程序,由于直接使用你的硬件,它们始终会更快,也能支持更大的模型。而这个工具在浏览器标签页里运行模型,完全不需要安装任何东西,这让它成为尝试本地模型更简单的方式——无论是在受限设备上、别人的电脑上,还是单纯想看看这是怎么回事。
什么时候应该改用云端模型?
当任务需要广度、事实准确性、长文档、最新信息或严肃的推理时——前沿模型一次就能完成 1B 模型完全做不到的事。而在文本敏感、你处于离线状态、你想不开账户就获得即时回答,或者任务足够小、小模型本身就是更快捷的工具时,请使用这个方案。