Quantization: Making Models Fit
A 70B model in FP16 needs 140GB. Two A100s just for weights. Quantize to FP8: one 80GB GPU. INT4: a MacBook.
Visual edition planned
This lesson isn’t interactive yet.
It is part of the curriculum and will get the same treatment as Phase 1 — a visual cover, hands-on labs, derivations with numeric checks and a quiz. Until then, the original lesson is the best place to read it:
Part of Phase 10 — LLMs from Scratch. Use the previous / next cards below to keep browsing the phase.