chipping on: also curious. From the blogpost they make it sound like this is a QAT.
Quantization. After training, our Eureka Machine searched for quantization recipes that shrink Flint’s memory footprint while preserving its capabilities. Against common practice, it quantized the output layer (lm_head) to 4 bits. Because Flint has only 2.5 billion active parameters, its output layer is a large share of the weights read for each token. To recover accuracy, it added an offline distillation stage in which the quantized model trains on its own pre-sampled responses from a self-discovered coreset, with the 16-bit model as teacher. This nearly halved the number of training steps compared with our quantization-aware training baseline. The final NVFP4 model scores within 0.7 points of the 16-bit model across our benchmarks.
If so, this likely should be advertised and made more common as a suggested quant to take, as alternative labs found flint taking quants badly? re: AtomicChat says:
Flint is unusually sensitive to quantization. Even an 8-bit copy matches the original's next word only about 95% of the time on general text, where most models reach 98% or more. Read the table against that ceiling, and against the other builds below, which were measured the same way.
