wire-2m-chinchilla / config.json
LisaMegaWatts's picture
Add Wire-2M (H16 L8, Chinchilla) checkpoint (step 0, bpb 1.9697)
df90c3a verified
Raw History Blame Contribute Delete
223 Bytes
{
"architecture": "WireNative",
"n_groups": 7,
"n_harmonics": 32,
"n_layers": 8,
"d_model": 448,
"n_kan_coeffs": 4,
"vocab_size": 256,
"seq_len": 256,
"training": {
"step": 0,
"bpb": 1.969687
}
}