Optimizationmedium~35 min
Objective
Implement per-channel INT8 quantization for linear layers: compute scales, quantize/dequantize weights, run a forward pass, and measure compression.
Background
Your team needs to deploy a large transformer model to edge devices with limited memory. The model currently uses float32 weights (2.3 GB), but the target device only has 768 MB. Post-training quantization to INT8 can shrink the model by ~4x with minimal accuracy loss. You need to implement the quantization pipeline from scratch using only numpy (no PyTorch or specialized libraries).
Requirements
- 1.Compute per-channel (per-row) scale factors as max(|W_i|) / 127
- 2.Quantize weights to INT8 with proper clamping to [-128, 127]
- 3.Implement dequantization to reconstruct approximate float32 weights
- 4.Implement forward pass using quantized weights
- 5.Measure compression ratio (original float32 vs quantized int8 + scale overhead)
Evaluation (100 points)
Per-channel scale factor computed
Scale is computed as max(abs(W)) / 127 per row
Weights clamped to INT8 range
Quantized weights are clipped to [-128, 127] and cast to int8
Dequantization multiplies int8 by scale
dequantize() reconstructs float32 weights from int8 * scale
Forward pass uses matrix multiplication
forward() performs matmul with quantized weights
Compression ratio computed correctly
measure_compression uses 4 bytes for float32 and 1 byte for int8