Summary of the live topic "Getting GLM 5.2 running on my slow computer" based on the investigation: 1. What Changed: - The project "Colibrì" by JustVugg enables running the GLM-5.2 model, a 744 billion parameter Mixture-of-Experts (MoE) large language model, on a consumer-grade computer with about 25 GB RAM. - The key innovation is streaming the large number of experts (~21,504 experts, ~370 GB) from disk on demand, while keeping the dense part (~17B parameters) resident in RAM at int4 precision (~9.9 GB). - The engine is implemented in pure C with zero dependencies, no GPU required, and uses advanced techniques like MTP speculative decoding, integer dot-product kernels, async expert readahead, and quantization to int4. - The model is converted from the original FP8 format to int4 to reduce memory footprint and enable running on modest hardware. - The system uses an LRU cache and pinned hot-store for experts to optimize disk streaming. - The project achieves correctness validated against transformer oracles and supports true sampling with temperature and nucleus sampling tuned for int4 quantization noise. - Performance is slow but functional: cold start decoding speed is about 0.05-0.1 tokens per second on a 12-core laptop with NVMe SSD, improving with warm cache and pinned experts. 2. Why It Matters: - GLM-5.2 is a state-of-the-art open-source large language model with 744B parameters, 1 million token context window, and performance comparable to Claude 4.8, GPT-5.5, and Gemini 3.1 Pro on reasoning and coding benchmarks. - Running such a massive model locally on consumer hardware was previously infeasible due to memory and compute constraints. - Colibrì demonstrates that with clever streaming, quantization, and efficient C implementation, frontier-class models can be run on modest machines without GPUs. - This opens possibilities for privacy-preserving, offline, and cost-effective use of large language models. - The approach also highlights the trade-offs between latency and hardware cost, showing that even slow token generation can be useful for overnight or batch tasks. - The project is a one-person effort and invites community contributions. 3. Strongest Sources: - The Colibrì GitHub repository by JustVugg: https://github.com/JustVugg/colibri - The Hugging Face page hosting the pre-converted int4 GLM-5.2 model for Colibrì: https://huggingface.co/jlnsrk/GLM-5.2-colibri-int4 - Hacker News discussion thread showing community feedback and benchmarks: https://news.ycombinator.com/item?id=48842459 - The official GLM-5.2 FP8 model page by Z.ai on Hugging Face with benchmarks and architecture details: https://huggingface.co/zai-org/GLM-5.2-FP8 - Unsloth documentation on running GLM-5.2 locally with quantization and usage tips: https://unsloth.ai/docs/models/glm-5.2 In summary, Colibrì is a breakthrough in making the massive GLM-5.2 model accessible on slow, consumer-grade computers by streaming experts from disk and using quantization, enabling practical local use of frontier LLMs without expensive hardware. If you want, I can provide more technical details or community reactions from the HN thread.