cognisoc / zigllm
About

A book-shaped repository.

zigllm is an educational implementation of transformer architectures, written in Zig. It is structured the way a textbook is structured: six progressive layers, each one building on the layer below it. You read the project the way you read a book — from the bottom up.

Who this is for

  • ML / systems-curious learners who can already write a forward pass on paper but want to see one all the way down to memory layout.
  • Zig fans who want a substantial codebase that uses comptime, SIMD, and manual memory management for something more interesting than a hash map.
  • Anyone who learns by reading code. The 285+ tests serve as executable documentation: each test demonstrates a concept and validates the math.

What it actually is

The project is split into six layers. Layer 1 (foundation) is tensors and memory. Layer 2 is linear algebra — SIMD matrix ops and a wide set of quantization formats. Layer 3 is neural primitives — activations like SwiGLU and GELU, normalization like RMSNorm, positional encodings like RoPE. Layer 4 is transformer blocks: multi-head attention and feed-forward networks. Layer 5 is whole model architectures, of which 18 families are implemented (LLaMA, Mistral, GPT-2, Falcon, Qwen, Phi, GPT-J, GPT-NeoX, BLOOM, Mamba, BERT, Gemma, StarCoder, MoE, multi-modal, and more). Layer 6 is inference: the generation loop, KV caching, streaming, and sampling.

A note on honesty

zigllm is a pedagogical project. Educational clarity takes priority over micro-optimizations. The point is not to ship faster inference than llama.cpp — the point is to make the math of a modern transformer legible to a reader who is willing to follow it through the code.

How to read it

  1. Clone the repository and run zig build test to confirm the toolchain works.
  2. Open docs/01-foundations/ and read it alongside src/foundation/.
  3. Walk up the stack chapter by chapter. Don't skip — each layer assumes the previous one.
  4. When something feels handwavy, look for the matching test. The tests are the proof.

Requirements

  • Zig 0.14+.
  • A modern CPU. AVX/AVX2 is recommended for the SIMD paths, but not required.

License and contributions

zigllm is released under the MIT license. Contributions that improve educational value — clearer explanations, additional tests and edge cases, new architecture implementations, or visualization tools — are especially welcome. Please keep code readable; clarity takes priority.

Source: https://github.com/cognisoc/zigllm
Docs: https://docs.cognisoc.com/zigllm/