Model quantization: measure the cost of compression
Choose quantization by measuring memory, throughput, latency, fidelity, robustness and runtime compatibility.
Quantization is an intervention on the numerical substrate and must be evaluated like any other transformation.
Perplexity or a handful of prompts cannot reveal every loss on strict formats and edge cases.
Each quantized derivative has its own identity and does not automatically inherit source-model qualification.
What the architecture must make explicit
Cost is reduced at realization level: avoiding calls, shortening context, quantizing, routing or rematerializing a responsibility can matter more than isolated model optimization.
Move from a keyword to an inspectable engineering decision
Profile
Measure actual system costs, including retries, transfers, memory and verification.
Attribute
Determine which responsibilities still require a powerful model.
Transform
Test reduction, quantization, specialization, caching, routing or substitution.
Compare
Keep the same capability contract and publish exact hardware conditions.
A claim is only as strong as its binding
The gate must evaluate the exact output produced by the exact realization. Integrity, scientific validity and authority are related, but never interchangeable.
Questions that change the design
What does model quantization: measure the cost of compression require?
Quantization is an intervention on the numerical substrate and must be evaluated like any other transformation. Cost is reduced at realization level: avoiding calls, shortening context, quantizing, routing or rematerializing a responsibility can matter more than isolated model optimization.
Which evidence should be retained?
At minimum: exact realization identity, inputs and outputs, protocol version, measurements, failure cases and the decision rule. Each quantized derivative has its own identity and does not automatically inherit source-model qualification.
Does AMF prescribe a particular model?
No. AMF treats the model, runtime, data, tools and deterministic components as candidate materials. Their retention depends on measured contribution and declared constraints.