My Chinese AI Is Learning From an American One

The AI assistant I am building for my family runs on a Chinese model.

It is called Dobby, and it runs on Alibaba’s Qwen3.8-27B on the Mac Studio I wrote about last month, which finally arrived after six months. At 4-bit the model takes about 15GB.

After months of waiting, the machine was mine, and so was the model on it. Dobby no longer depended on someone else’s API staying cheap or staying open.

I also wanted to learn fine-tuning by doing it, so before choosing, I looked at what I could download: Meta’s Muse Glimmer, OpenAI’s gpt-oss and Nvidia’s Nemotron. Qwen3.8-27B is small enough to train at home, very good for its size, and its Apache 2.0 license lets me change it however I want.

Not everything in the house is Chinese. Dobby’s voice will be Nvidia’s Nemotron VoiceChat, an American open model.

I pay for two of the US frontier labs and use them every day. So I am a customer of the American labs and, it turns out, a beneficiary of the Chinese ones.

Who wrote Dobby’s answers

I want to fine-tune Qwen so that Dobby answers the way I want. For that, I need example conversations: a question, and the answer Dobby should give. My starting point was simple. I asked Codex, OpenAI’s coding agent, to generate them, and it gave me a dataset of 24 conversations.

They cover three things: how Dobby should talk, how it should read a note and quote a fact, and when it should call a tool. For now they use made-up projects with names like Atlas and Beacon. Next, I want to replace them with questions my family actually asks.

So the first training data for my Chinese model came from an American one. Anthropic calls this distillation: training one model on another model’s answers.

Anthropic wrote about this on 12 October. Its post explains how Anthropic now defends Claude against distillation. It says Zhipu distilled cyber capabilities from leading US models while developing GLM-5.3, and that requests to Moonshot’s Kimi models were secretly forwarded to Claude.

Was I a distiller? I went back to Anthropic’s February report to see what a real one looks like. It named DeepSeek, Moonshot and MiniMax and counted more than 16 million exchanges through about 24,000 fraudulent accounts. I had one account, in my own name, and 24 conversations.

What I did was the same kind of thing, but very small. Anthropic’s cases also include things I did not do: hundreds of thousands of fake accounts, tricks to pull out Claude’s hidden reasoning, and distilled models that ship with weak safeguards or end up in military and surveillance systems.

The word free-ride

OpenAI, the company whose agent wrote Dobby’s answers, told a US House select committee that DeepSeek’s next model should be understood in the context of “its ongoing efforts to free-ride on the capabilities developed by OpenAI and other US frontier labs.”

That felt unfair to me. OpenAI and Anthropic trained their models on writing from the public internet, and most of the people who wrote it were never asked. They call that fair use.

When a Chinese lab learns from their models, it is called free-riding. Anthropic’s February report says distillation attacks “reinforce the rationale for export controls,” and those controls reach every AI company in China, including Alibaba, whose model runs Dobby. That feels one-sided to me.

Let’s talk about the KV cache

Dobby has one machine with 128GB of memory, and nothing to add when it runs out. A model has to fit, and keep fitting as a conversation grows.

The part that grows is the key-value cache, the attention data a model keeps for every token it has already read. The longer we talk to Dobby, the more memory that takes.

Dobby’s model, Qwen3.8-27B, keeps about 64KB of cache for every token. At its longest context, 262,144 tokens, that is 16GB on top of the 15GB model. Alibaba built it so that only 16 of its 64 layers keep a full cache. If every layer did, the same conversation would need 64GB.

Claude can now read up to 1 million tokens in one request. If Qwen could read that much, its cache alone would be 64GB, half of Dobby’s machine. I do not know how Anthropic stores Claude’s cache, but I think holding it is one of the biggest costs of long context.

That is for one conversation. Every conversation needs its own cache, so if three people talked to Dobby at once, each with a long conversation, Qwen would need 48GB of cache. Today Dobby answers one request at a time and keeps the cache of only two conversations. When a third one starts, the oldest is dropped, and that person’s conversation has to be read again from the start.

As I understand it, this is also what prompt caching is. Coding agents send much of the same context again and again, and the provider can keep the cache it already built instead of building it again. On 22 September, Opus 5.5 cut the price of those cache reads from $0.50 to $0.20 per million tokens, a 60% drop, while input and output prices fell 20%.

Chinese labs have spent years making this cache smaller, under chip limits that began in 2022. A lab that cannot buy more hardware has to waste less of what it has. DeepSeek published its fixes. Its V2 paper reported 93.3% less KV-cache memory than its earlier 67B model, and it has kept shrinking the cache since.

DeepSeek said this smaller cache is what lets it keep caches on low-cost disks for hours or days. Its cache hits cost $0.003 per million tokens, against $0.20 for Opus 5.5. Moonshot published Mooncake, the system Kimi uses to spread caches across the memory and SSDs of many machines.

I wondered whether that research had anything to do with the Opus price. I do not know. Anthropic says Opus 5.5 “requires less compute to serve than Opus 5, and its pricing reflects that,” and says nothing about cache design.

American labs also work on efficiency on their own. But the limits meant to slow China down may have pushed Chinese labs toward research that anyone can read for free, including me with my one Mac Studio.

Anthropic’s posts list what Chinese labs took. I would like to see them also mention what they have learned from Chinese labs’ published papers.

What China gave me

The more I looked, the more of my setup traced back to things Chinese labs gave away. Developers have built 151,448 models on Qwen alone, 2.6 times as many as on Meta’s models, according to Hugging Face’s summer report on open models. When my fine-tune of Dobby’s model works, it will be one more. The biggest Chinese models are starting to add non-commercial terms, though almost none of the smaller ones do.

When I went looking for a model to own, the American list was shorter, and its biggest models smaller. Meta put its new flagship Muse Spark behind an API and released the smaller Muse Glimmer. Anthropic, OpenAI and Google keep their flagships closed. Nvidia’s Nemotron is the exception, and I am grateful for it.

Chinese labs also keep prices low. DeepSeek’s API charges $0.15 per million input tokens for V4.1-Flash off-peak, and Opus 5.5 charges $4. They are different models, but DeepSeek’s price is now the floor American labs get compared against.

The papers that started all this were American. Google’s Attention Is All You Need is why any of these models exist. But as the American frontier labs stopped publishing how their flagships work, Chinese labs kept publishing theirs. Today a few American companies hold the best closed models. Chinese labs supply most of the strongest open weights, many of the open papers and the lowest prices, and that keeps things in balance.

If not for China

I tried to imagine my setup without Chinese labs.

Dobby would choose from fewer models, and the one I picked would not exist. Without DeepSeek at $0.15, there would be one less price pulling $4 down. Fewer frontier-scale efficiency tricks would be published, since American frontier labs keep their flagship architectures private.

The companies leading the race would also write the rules, with fewer outsiders to hold them to account. Most builders would rent capability from a few APIs on whatever terms those APIs set. Anthropic’s October post shows how those terms can change: stricter limits for accounts that keep probing for Claude’s reasoning, and charges even for the rare harmless request its filters block.

I will keep paying the American labs, and some of the distillation Anthropic describes deserves to be stopped. I will also keep training Qwen. Every answer Dobby gives my family will come from a model Alibaba gave away, taught partly by an American one.

If not for China, this would be a very different world.

In the American story, China plays the villain. I am fine with that. Thanos was a villain too, and he put it best: “Perfectly balanced, as all things should be.” For the last two years, Chinese labs have helped keep it that way.

The receipts: releases, licenses, chips, cache and prices

Open-weight releases, as listed on Hugging Face in mid-October:

• DeepSeek published V4.1-Flash on 10 September under the MIT license, the same terms as R1 in January 2025.
• Z.ai (formerly Zhipu) released the weights of GLM-5.3 in late August. Its MIT-licensed sibling, GLM-5.3-Flash, was downloaded more than 6.5 million times in the past month. GLM-5.3 is the model Anthropic’s October post accuses of distilling, and anyone can download it.
• Alibaba opened Qwen3.8, a 2.4-trillion-parameter model, in August. Moonshot’s Kimi K3 and MiniMax-M3 arrived in June.
• Baidu, which kept its flagship ERNIE closed, released ERNIE 4.5 under Apache 2.0 after R1 changed expectations.
• Hugging Face’s summer report found that “in almost every month of 2026, the largest and most performant open model from a Chinese lab was larger than any model an American lab released.”

Licenses are tightening at the top. Kimi K3 and the 2.4-trillion-parameter Qwen3.8 added non-commercial and revenue-share conditions, and the full GLM-5.3 has its own license. Hugging Face still found that almost none of the 178 Chinese releases above 20 billion parameters this year carry non-commercial restrictions.

Western labs are answering. Reflection AI announced Beam on 5 October as a match for “leading Chinese open models”, with weights promised this month. Mistral plans to release the weights of Mistral Large 4 on 27 October and says it will be the most capable open-weight model developed outside China.

The chip limits began in 2022 and were tightened in 2023 and clarified in 2024. Since January, the older Nvidia H200 can be licensed for China case by case, with a share of the revenue going to the US government, while the newest chips stay off-limits. Nvidia’s latest quarterly filing says it is “effectively foreclosed” from China’s data-centre market.

Dario Amodei’s essay on pacing the frontier writes that “chips will be the main determinant of China’s AI strength” and asks for a block on sales of advanced AI chips and chipmaking equipment to China. It also proposes that frontier companies in democracies agree on shared standards and limits on progress.

On the KV cache, the PagedAttention paper shows how much cache memory limits serving throughput. DeepSeek’s V2 paper introduced Multi-head Latent Attention and reported 93.3% less KV-cache memory than its earlier 67B model. The V4.1-Flash model card adds FP4 cache storage and attention layers that reuse each other’s cache, and reports 890 bytes of global KV cache per token, about 437 times smaller than DeepSeek V1. Dobby’s numbers come from Qwen3.8-27B’s config: 64 layers, 16 of them with full attention, each with 4 key-value heads of size 256. At 16 bits that is 2 × 16 × 4 × 256 × 2 bytes, or 64KB per token. If all 64 layers kept a full cache, a million tokens would need 256GB. Dobby’s model server is mlx-lm’s, set to answer one request at a time and to keep at most two conversations’ caches, 4GB in all.

Opus 5.5 standard API prices, in US dollars per million tokens:

Token category	Opus 5	Opus 5.5	Reduction
Input	$5.00	$4.00	20%
Output	$25.00	$20.00	20%
Cache writes	$6.25	$5.00	20%
Cache reads	$0.50	$0.20	60%

Anthropic estimates a 40% lower cost on typical workloads once fewer tokens per task are counted. For comparison, DeepSeek charges $0.003 per million tokens for a V4.1-Flash cache hit, against $0.20 for Opus 5.5.