gemma-3-12b-it-MPOAdd-v1

This model was derived from google/gemma-3-12b-it, and represents Magnitude-Preserving Orthogonal Addition (MPOAdd).

Projected abliteration has been applied to neasuring refusal direction, along with a second round of removal of projected contribution onto the harmless direction of layer targeted for intervention. Additionally, instead of subtracting/ablating away the refusal direction in toto, the directional component of the refusal direction was added, or enhanced, while preserving the norms of the layers subjected to intervention. The model will now push back against perceived harms in an exaggerated manner, as refusal over safety concerns is more strongly enforced. More details of norm preservation can be found in the article on Norm-Preserving Biprojected Abliteration.

These geometric tweaks defy the conventional wisdom that (blunt) ablation or addition damages the reasoning of models. Perplexity loss has been minimal when measured on Q8_0 GGUFs compared to baseline.