BitNet ESP32 Cluster: 1-Bit LLMs Reach $5 Microcontrollers
A seven-board BitNet ESP32 cluster and Cactus Compute's 2-bit Needle model show ultra-low-bit LLMs reaching cheap hardware. Here is how ternary weights work and where the limits are.
LLM on a Microcontroller: How Generative AI Fits in 512 KB SRAM
A 28.9M-parameter LLM now runs on an ESP32-S3 and a diffusion model draws faces on a $1 RP2350. Here is how memory tiering, quantization and DMA streaming make it work,…