Update README.md

README.md

@@ -1,15 +1,18 @@
 ---
 license: apache-2.0
-base_model: allenai/Olmo-3-7B-Think-SFT
+base_model:
+- allenai/Olmo-3-1125-32B
 language:
 - en
+datasets:
+- allenai/Dolci-Instruct-SFT
 ---
 
 ## Model Details
 <img alt="OLMo Logo" src="https://cdn-uploads.huggingface.co/production/uploads/65316953791d5a2611426c20/nC44-uxMD6J6H3OHxRtVU.png" width="242px" style="margin-left:'auto' margin-right:'auto' display:'block'">
 
 
-# Model Card for Olmo-3-32B-Instruct-SFT
+# Model Card for Olmo-3.1-32B-Instruct-SFT
 
 We introduce Olmo 3, a new family of 7B and 32B models both Instruct and Think variants. Long chain-of-thought thinking improves reasoning tasks like math and coding.
 
@@ -20,12 +23,12 @@ These models are pre-trained on the Dolma 3 dataset and post-trained on the Dolc
 
 The core models released in this batch include the following:
 
-| **Stage**               | **[Olmo 3 7B Think]** | **[Olmo 3 32B Think]** | **[Olmo 3 7B Instruct]** | **[Olmo 3 32B Instruct]** |
-|--------------------------|---------------|---------------|---------------|---------------|
-| **Base Model**           | [Olmo-3-7B](https://huggingface.co/allenai/Olmo-3-1025-7B) | [Olmo-3-32B](https://huggingface.co/allenai/Olmo-3-1125-32B) | | |
-| **SFT**                  | [Olmo-3-7B-Think-SFT](https://huggingface.co/allenai/Olmo-3-7B-Think-SFT) | [Olmo-3-32B-Think-SFT](https://huggingface.co/allenai/Olmo-3-32B-Think-SFT) | [Olmo-3-7B-Instruct-SFT](https://huggingface.co/allenai/Olmo-3-7B-Instruct-SFT) | [Olmo-3-32B-Instruct-SFT](https://huggingface.co/allenai/Olmo-3-32B-Instruct-SFT) |
-| **DPO**                  | [Olmo-3-7B-Think-DPO](https://huggingface.co/allenai/Olmo-3-7B-Think-DPO) | [Olmo-3-32B-Think-DPO](https://huggingface.co/allenai/Olmo-3-32B-Think-DPO) | [Olmo-3-7B-Instruct-DPO](https://huggingface.co/allenai/Olmo-3-7B-Instruct-DPO) | [Olmo-3-32B-Instruct-DPO](https://huggingface.co/allenai/Olmo-3-32B-Instruct-DPO) |
-| **Final Models (RLVR)**  | [Olmo-3-7B-Think](https://huggingface.co/allenai/Olmo-3-7B-Think) | [Olmo-3-32B-Think](https://huggingface.co/allenai/Olmo-3-32B-Think) | [Olmo-3-7B-Instruct](https://huggingface.co/allenai/Olmo-3-7B-Instruct) | [Olmo-3-32B-Instruct](https://huggingface.co/allenai/Olmo-3-32B-Instruct) |               
+| **Stage**               | **Olmo 3 7B Think** | **Olmo (3/3.1) 32B Think** | **Olmo 3 7B Instruct** | **Olmo 3.1 32B Instruct** |
+|--------------------------|-----------------------|------------------------|---------------------------|----------------------------|
+| **Base Model**           | [Olmo-3-7B](https://huggingface.co/allenai/Olmo-3-1025-7B) | [Olmo-3-32B](https://huggingface.co/allenai/Olmo-3-1125-32B) | [Olmo-3-7B](https://huggingface.co/allenai/Olmo-3-1025-7B) | [Olmo-3-32B](https://huggingface.co/allenai/Olmo-3-1125-32B) |
+| **SFT**                  | [Olmo-3-7B-Think-SFT](https://huggingface.co/allenai/Olmo-3-7B-Think-SFT) | [Olmo-3-32B-Think-SFT](https://huggingface.co/allenai/Olmo-3-32B-Think-SFT) | [Olmo-3-7B-Instruct-SFT](https://huggingface.co/allenai/Olmo-3-7B-Instruct-SFT) | [Olmo-3.1-32B-Instruct-SFT](https://huggingface.co/allenai/Olmo-3.1-32B-Instruct-SFT) |
+| **DPO**                  | [Olmo-3-7B-Think-DPO](https://huggingface.co/allenai/Olmo-3-7B-Think-DPO) | [Olmo-3-32B-Think-DPO](https://huggingface.co/allenai/Olmo-3-32B-Think-DPO) | [Olmo-3-7B-Instruct-DPO](https://huggingface.co/allenai/Olmo-3-7B-Instruct-DPO) | [Olmo-3.1-32B-Instruct-DPO](https://huggingface.co/allenai/Olmo-3.1-32B-Instruct-DPO) |
+| **Final Models (RLVR)**  | [Olmo-3-7B-Think](https://huggingface.co/allenai/Olmo-3-7B-Think) | [Olmo-3-32B-Think](https://huggingface.co/allenai/Olmo-3-32B-Think)<br>[Olmo-3.1-32B-Think](https://huggingface.co/allenai/Olmo-3.1-32B-Think) | [Olmo-3-7B-Instruct](https://huggingface.co/allenai/Olmo-3-7B-Instruct) | [Olmo-3.1-32B-Instruct](https://huggingface.co/allenai/Olmo-3.1-32B-Instruct) |
              
 
 ## Installation
@@ -40,8 +43,8 @@ pip install transformers>=4.57.0
 You can use OLMo with the standard HuggingFace transformers library:
 ```python
 from transformers import AutoModelForCausalLM, AutoTokenizer
-olmo = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3-32B-Instruct-SFT")
-tokenizer = AutoTokenizer.from_pretrained("allenai/Olmo-3-32B-Instruct-SFT")
+olmo = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3.1-32B-Instruct-SFT")
+tokenizer = AutoTokenizer.from_pretrained("allenai/Olmo-3.1-32B-Instruct-SFT")
 message = ["Language modeling is "]
 inputs = tokenizer(message, return_tensors='pt', return_token_type_ids=False)
 # optional verifying cuda
@@ -54,7 +57,7 @@ print(tokenizer.batch_decode(response, skip_special_tokens=True)[0])
 
 For faster performance, you can quantize the model using the following method:
 ```python
-AutoModelForCausalLM.from_pretrained("allenai/Olmo-3-32B-Instruct-SFT", 
+AutoModelForCausalLM.from_pretrained("allenai/Olmo-3.1-32B-Instruct-SFT", 
     torch_dtype=torch.float16, 
     load_in_8bit=True)  # Requires bitsandbytes
 ```
@@ -68,13 +71,13 @@ We have released checkpoints for these models. For post-training, the naming con
 
 To load a specific model revision with HuggingFace, simply add the argument `revision`:
 ```bash
-olmo = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3-32B-Instruct-SFT", revision="step_1375")
+olmo = AutoModelForCausalLM.from_pretrained("allenai/Olmo-3.1-32B-Instruct-SFT", revision="step_1375")
 ```
 
 Or, you can access all the revisions for the models via the following code snippet:
 ```python
 from huggingface_hub import list_repo_refs
-out = list_repo_refs("allenai/Olmo-3-32B-Instruct-SFT")
+out = list_repo_refs("allenai/Olmo-3.1-32B-Instruct-SFT")
 branches = [b.name for b in out.branches]
 ```
 
@@ -98,7 +101,7 @@ For more documentation, see the [GitHub readme](https://github.com/allenai/OLMo-
 - **Language(s) (NLP):** English
 - **License:** This model is licensed under Apache 2.0. It is intended for research and educational use in accordance with Ai2's [Responsible Use Guidelines](https://allenai.org/responsible-use).
 - **Contact:** Technical inquiries: `[email protected]`. Press: `[email protected]`
-- **Date cutoff:** Dec. 2023.
+- **Date cutoff:** Dec. 2024.
 
 
 ### Model Sources
@@ -108,37 +111,51 @@ For more documentation, see the [GitHub readme](https://github.com/allenai/OLMo-
     - Open-Instruct for DPO and RLVR: https://github.com/allenai/open-instruct
     - OLMo-Core for pre-training and SFT: https://github.com/allenai/OLMo-core
     - OLMo-Eval for evaluation: https://github.com/allenai/OLMo-Eval
-- **Paper:** [TBD]
-<!-- - **Technical blog post:** (URL)  -->
-<!-- - **W&B Logs:** [SFT](()), [DPO](()), [RLVR](()) -->
+- **Paper:**: https://allenai.org/papers/olmo3
 
 
 ## Evaluation
 
-| **Model** | **Math** |  |  | **Reasoning** |  |  | **Coding** |  |  | **IF** |  | **QA** |  |  |
-|----------|----------|----------|----------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|-------------|
-|          | AIME '24 | AIME '25 | OMEGA | BBH | Zebra Logic | AGI Eval | Human Eval+ | MBPP+ | LCB v3 | IFEval | IFBench | MMLU | PopQA | GPQA |
-| **Nemotron–Nano–9B–v2** | 72.1 | 58.9 | 42.4 | 86.2 | 60.8 | 83.1 | 89.7 | 66.1 | 83.4 | 86.0 | 34.6 | 84.3 | 17.9 | 56.2 |
-| **OpenThinker3–7B** | 67.7 | 57.2 | 38.4 | 77.1 | 34.9 | 78.6 | 87.4 | 61.4 | 68.2 | 51.7 | 23.0 | 77.4 | 18.0 | 48.0 |
-| **DeepSeek–R1–Qwen–7B** | 54.9 | 40.2 | 28.5 | 73.5 | 26.1 | 69.5 | 83.0 | 63.5 | 58.8 | 59.6 | 16.7 | 67.9 | 12.8 | 53.2 |
-| **Qwen 3 8B (w/ reasoning)** | **74.0** | **67.8** | 43.4 | 84.4 | 85.2 | 87.0 | 80.2 | **69.1** | **86.2** | **87.4** | 37.1 | 85.4 | 24.3 | 57.7 |
-| **Qwen 3 VL 8B Thinker** | 70.9 | 61.5 | 37.9 | **86.8** | **91.2** | **90.1** | 83.7 | 63.0 | 85.5 | 85.5 | 40.4 | **86.5** | **29.3** | **62.4** |
-| **OpenReasoning Nemo 7B** | 77.0 | 73.1 | 43.2 | 81.3 | 22.4 | 81.4 | 89.7 | 61.2 | 82.3 | 42.5 | — | 80.7 | 14.5 | 60.8 |
-| **Olmo-3-32B-Instruct-SFT** |  |  |  |  |  |  |  |  |  |  |  |  |  |  |
+| Metric | **Olmo 3.1 32B Instruct SFT** | **Olmo 3.1 32B Instruct DPO** | **Olmo 3.1 32B Instruct** | Apertus 70B | Qwen 3 32B (No Think) | Qwen 3 VL 32B Instruct | Qwen 2.5 32B | Gemma 3 27B | Gemma 2 27B | OLMo 2 32B |
+| :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- | :--- |
+| **Math** | | | | | | | | | | |
+| MATH | 74.4 | 86.6 | 93.4 | 36.2 | 84.3 | 95.1 | 80.2 | 87.4 | 51.5 | 49.2 |
+| AIME 2024 | 12.7 | 35.2 | 67.8 | 0.31 | 27.9 | 75.4 | 15.7 | 28.9 | 4.7 | 4.6 |
+| AIME 2025 | 8.2 | 23.3 | 57.9 | 0.1 | 21.3 | 64.2 | 13.4 | 22.9 | 0.9 | 0.9 |
+| OMEGA | 15.5 | 33.3 | 42.2 | 5.6 | 23.4 | 44.0 | 19.2 | 24.0 | 9.1 | 9.8 |
+| **Reasoning** | | | | | | | | | | |
+| BigBenchHard | 69.0 | 82.1 | 84.0 | 57.0 | 80.4 | 89.0 | 80.9 | 82.4 | 66.0 | 65.6 |
+| ZebraLogic | 30.6 | 51.1 | 61.7 | 9.0 | 28.4 | 86.7 | 24.1 | 24.8 | 17.2 | 13.3 |
+| AGI Eval English | 71.7 | 79.4 | 79.5 | 61.6 | 82.4 | 89.4 | 78.9 | 76.9 | 70.9 | 68.4 |
+| **Coding** | | | | | | | | | | |
+| HumanEvalPlus | 80.8 | 85.7 | 86.7 | 42.9 | 83.9 | 89.3 | 82.6 | 79.2 | 67.5 | 44.4 |
+| MBPP+ | 61.5 | 63.6 | 65.1 | 45.8 | 67.9 | 69.0 | 66.6 | 65.7 | 61.2 | 49.0 |
+| LiveCodeBench v3 | 35.4 | 49.6 | 54.7 | 9.7 | 57.5 | 70.2 | 49.9 | 39.0 | 28.7 | 10.6 |
+| **IF** | | | | | | | | | | |
+| IFEval | 87.7 | 87.3 | 88.8 | 70.4 | 87.5 | 88.1 | 81.9 | 85.4 | 62.1 | 85.8 |
+| IFBench | 29.7 | 36.3 | 39.7 | 26.0 | 31.3 | 37.2 | 36.7 | 31.3 | 27.8 | 36.4 |
+| **Knowledge & QA** | | | | | | | | | | |
+| MMLU | 79.0 | 81.9 | 80.9 | 70.2 | 85.8 | 88.7 | 84.6 | 74.6 | 76.1 | 77.1 |
+| PopQA | 23.7 | 28.5 | 25.0 | 33.5 | 25.9 | 25.7 | 28.0 | 30.2 | 30.4 | 37.2 |
+| GPQA | 41.3 | 47.9 | 48.6 | 27.9 | 54.4 | 61.4 | 44.6 | 45.0 | 39.9 | 36.4 |
+| **Chat** | | | | | | | | | | |
+| AlpacaEval 2 LC | 42.2 | 69.7 | 59.8 | 19.9 | 67.9 | 84.3 | 81.9 | 65.5 | 39.8 | 38.0 |
+| **Safety** | 92.1 | 88.9 | 89.5 | 77.1 | 81.6 | 85.8 | 82.2 | 68.8 | 74.4 | 84.2 |
+
 
 ## Model Details
 
 #### Stage 1: SFT
 - supervised fine-tuning on the Dolci-Think-SFT-7B dataset. This dataset consits of math, code, chat, and general knowledge queries.
-- Datasets: [Dolci-Think-SFT-7B](https://huggingface.co/datasets/allenai/dolci-thinking-sft), [Dolci-Instruct-SFT-7B](https://huggingface.co/datasets/allenai/dolci-instruct-sft)
+- Datasets: [Dolci-Think-SFT-7B](https://huggingface.co/datasets/allenai/dolci-thinking-sft), [Dolci-Instruct-SFT](https://huggingface.co/datasets/allenai/dolci-instruct-sft)
 
 #### Stage 2:DPO
 - direct preference optimization on the Dolci-Think-DPO-7B dataset. This dataset consits of math, code, chat, and general knowledge queries.
-- Datasets: [Dolci-Think-DPO-7B](https://huggingface.co/datasets/allenai/dolci-thinking-dpo), [Dolci-Instruct-DPO-7B](https://huggingface.co/datasets/allenai/dolci-3-instruct-dpo-with-metadata)
+- Datasets: [Dolci-Think-DPO-7B](https://huggingface.co/datasets/allenai/dolci-thinking-dpo), [Dolci-Instruct-DPO](https://huggingface.co/datasets/allenai/dolci-3-instruct-dpo-with-metadata)
 
 #### Stage 3: RLVR
 - reinforcement learning from verifiable rewards on the Dolci-Think-RL-7B dataset. This dataset consits of math, code, instruction-following, and general chat queries.
-- Datasets: [Dolci-Think-RL-7B](https://huggingface.co/datasets/allenai/Dolci-Think-RL-7B), [Dolci-Instruct-RL-7B](https://huggingface.co/datasets/allenai/Dolci-Instruct-RL-7B)
+- Datasets: [Dolci-Think-RL-7B](https://huggingface.co/datasets/allenai/Dolci-Think-RL-7B), [Dolci-Instruct-RL](https://huggingface.co/datasets/allenai/Dolci-Instruct-RL-7B)
 
 
 ## Bias, Risks, and Limitations