NVIDIA released Nemotron 3.5 Bolt E4B 4GB on October 23, 2026, an open small language model built on Google's Gemma 4 E4B and post-trained with reinforcement learning only, no supervised fine-tuning. It has 8 billion total parameters with 4 billion effective, a 64,000-token context window, support for 35 languages, and a design goal of packing conversational agent intelligence into about 4GB of memory on GeForce RTX cards, Jetson modules, and RTX and DGX Spark systems.
The direct answer: on NVIDIA's October 2026 21-benchmark panel, Bolt E4B beats its small-model rivals on 15 tests and leads on instruction control, but it loses the multi-turn customer-task test to Qwen3.5-4B by 20 points, so the best small model depends on the job, not the headline.
The scoreboard
NVIDIA's technical page publishes the full panel: Bolt E4B against its predecessor Nemotron 3 Nano 4B, Qwen3.5-4B, and its own base model Gemma 4 E4B, all measured in BF16 with the same reasoning mode per benchmark. Here are the six most revealing rows, exactly as NVIDIA reports them.
| Benchmark (what it measures) | Bolt E4B | Gemma 4 E4B | Qwen3.5-4B | Nano 4B |
|---|---|---|---|---|
| IHEval (instruction hierarchy and priority) | 84.8 | 67.0 | 60.6 | 51.3 |
| Tau2-Bench (multi-turn customer tasks) | 55.1 | 38.5 | 75.1 | 29.2 |
| RoleBench (character speaking style) | 67.0 | 49.9 | 53.7 | 12.0 |
| MultiChallenge (multi-turn consistency) | 53.1 | 42.1 | 51.8 | 35.3 |
| BFCL-v3 (correct tools and arguments) | 68.5 | 66.4 | 70.9 | 61.1 |
| LongMemEval (recall across conversations) | 72.1 | 70.9 | 66.8 | 25.2 |
Two honesty notes belong with this table. First, these are vendor-measured numbers: NVIDIA graded its own model against rivals on its own chosen panel, and no independent verification exists yet. Second, two of NVIDIA's own new benchmarks in the panel, Brevity Bench and When2Call-MT, are listed as "to be released at a later date," so outside labs cannot reproduce those rows at all. The 15-of-21 win claim is NVIDIA's count across the full panel, and it is real as a statement about NVIDIA's measurements, not as an independent verdict.
The hardware reality
A small model's benchmark scores mean little without the hardware table, and NVIDIA publishes that too. All figures below use the Q4_K_M build at batch size 1, measured with llama.cpp.
| Hardware and workload | VRAM used | Time to first token | Generation speed |
|---|---|---|---|
| RTX 5070, short session (3K input, 50 output) | 3.5 GB | 548 ms | 133.4 tokens per second |
| RTX 5070, long context (32K cached, 2K input, 256 output) | 4.3 GB | 1,238 ms | 106.2 tokens per second |
| Jetson Orin Nano 8GB, short session | 4.2 GB | 5,785 ms | 15.5 tokens per second |
| Jetson Orin Nano 8GB, long context | 4.9 GB | 6,259 ms | 13.0 tokens per second |
Two numbers in this story need unspinning. The "4GB" in the name is a memory-footprint goal for target workloads, not the download size: the Q4_K_M GGUF file weighs 6.18 GB. And the Jetson numbers show the real trade: the same model that answers at 133 tokens per second on a desktop RTX 5070 crawls at 15.5 on the robot-grade Orin Nano, with a nearly six-second wait for the first token. NVIDIA's own demo pairs Bolt with Cosmos 3 Edge for vision on that 8GB module, which is a two-model squeeze into a small box, not a comfortable fit. NVIDIA also claims up to 80% fewer output tokens than peers in reasoning comparisons, and a DFlash parallel drafter worth up to 21% more throughput than Gemma 4 E4B's original drafter. Both claims are vendor-measured and unreproduced.
For calibration, NVIDIA's own chart puts cloud references above the local pack on its composite average: Gemini 3.8 Flash at 77.1 and Claude Sonnet 5.5 at 76.2. Local small models are closing the gap for conversational agent work. They have not closed it overall.
The bigger sibling, and the real race
Bolt E4B is the smallest of three Nemotron 3.5 models NVIDIA launched together. The middle tier is Bolt 74B-A9B, a 74-billion-parameter mixture-of-experts model with about 9 billion active parameters, squeezed by 2-bit quantization into a 24GB footprint for local research, coding, and debugging on RTX and DGX systems. The top tier is Super VL, the large model the Bolt family distills from. The strategy is a ladder: cloud-scale capability at the top, a desktop agent in the middle, a 4GB conversational agent at the edge.
That ladder is the context for our open-weight benchmark wars comparison, which showed the same pattern one tier up: open models winning on value and specific tests while the capability crown stays with the biggest closed systems. Small models are not a separate story from the benchmark wars. They are the same war, fought where the hardware is cheapest.
Why the World Bank is betting on small
The demand side of this story arrived five days before Bolt. The World Bank urged developing nations to build "small AI" models that run on mobile phones with limited connectivity, in local languages, for specific tasks. Its chief knowledge officer, Paschal Donohoe, said economic development that could have taken centuries might happen within decades with such tools, and cited Indian farmers in the World Bank's AgriConnect scheme who need soil analysis in their own language, where there will never be enough agricultural experts for every small farm.
The World Bank's August 2026 World Development Report, "The Promise of Artificial Intelligence," found one in five small firms surveyed across six developing economies already uses AI chatbots, mostly for search, writing, and translation. And on October 20, 2026, the Bank announced a shared, pay-as-you-go GPU initiative for governments in emerging markets, starting with Africa and Latin America, noting that sub-Saharan Africa holds 18% of the world's population but 0.6% of global data-center capacity, with about 900 million people in the region still offline. That is the market small models are actually for: not gamers with RTX cards, but farmers, teachers, and shopkeepers whose only computer is a phone.
Which small model for which job
Put the scoreboard, the hardware table, and the demand story together, and the honest buying guide writes itself. For a conversational agent that must follow instructions, hold a persona, and respect priority rules, Bolt E4B leads its class on NVIDIA's numbers, and its 4GB footprint leaves room for speech and vision models beside it. For multi-turn customer service tasks, Qwen3.5-4B wins Tau2-Bench by 20 points and is the better pick despite trailing elsewhere. For long-conversation recall, Bolt and Gemma 4 E4B are effectively tied. For a robot or kiosk on an 8GB Jetson module, budget for the six-second first-token wait, or keep the session short.
The same local thesis is playing out in voice. Block's Dessa model, trained on speech alone and running locally on a MacBook Pro, shows the other half of the edge: models that never send your data to a cloud at all. Small, local, and specific is becoming a coherent alternative to big, remote, and general. Bolt E4B is the strongest evidence yet that the alternative can hold a conversation. The Tau2 gap is the reminder that it cannot yet run the shop.
By ByteHerald Staff. NVIDIA's Nemotron 3.5 Bolt E4B 4GB technical page, published October 2026, reports Bolt E4B at 84.8 on IHEval, 55.1 on Tau2-Bench, 67.0 on RoleBench, and 53.1 on MultiChallenge, against Gemma 4 E4B, Qwen3.5-4B, and Nemotron 3 Nano 4B, with 15 wins across its 21-benchmark panel and RTX 5070 performance of 3.5 GB VRAM, 548 ms time to first token, and 133.4 tokens per second in short sessions. The Guardian reported on October 18, 2026 that the World Bank urged developing nations to build small AI models for phones with limited connectivity, citing chief knowledge officer Paschal Donohoe and the AgriConnect soil-analysis example. MarkTechPost reported on October 23, 2026 that the downloadable Q4_K_M GGUF file weighs 6.18 GB. The World Bank's World Development Report 2026, "The Promise of Artificial Intelligence," published in August 2026, found one in five small firms surveyed across six developing economies already uses AI chatbots, mostly for search, writing, and translation. The World Bank's AI Infrastructure and Readiness Initiative, announced on October 20, 2026, offers shared pay-as-you-go GPU access for governments in emerging markets, starting with Africa and Latin America, noting that sub-Saharan Africa holds 18% of the world's population but 0.6% of global data-center capacity, with about 900 million people in the region still offline.
Comments
Post a Comment