amara-o1

A fine-tuned coding model built on Qwen for elite problem-solving

Demo | Training | Benchmarks | Limitations

---

Model Details

amara-o1 is a specialized coding assistant fine-tuned from Qwen2.5-Coder, optimized for:

• 🧮 Complex algorithmic problem solving
• 🔐 Secure code generation and vulnerability detection
• 📊 Mathematical reasoning and computation
• 💡 Multi-step reasoning for challenging tasks

Attribute	Details
Base Model	Qwen/Qwen2.5-Coder-7B-Instruct
Parameters	7B
Training Method	QLoRA (4-bit quantization)
LoRA Rank	64
Context Length	32,768 tokens
License	MIT
Languages	Python, JavaScript, C++, Java, and 90+ more

---

What Makes amara-o1 Different?

amara-o1 has been fine-tuned on a carefully curated dataset combining:

1. 🏆 Competitive Programming - 5,000+ problems from Code Contests
2. 🧮 Advanced Mathematics - MATH-500 dataset for quantitative reasoning
3. 🔐 Security-First Coding - Vulnerability detection and secure programming patterns
4. 💭 Deep Reasoning - Anthropic's interview transcripts for complex problem decomposition

This multi-domain training enables amara-o1 to:

• Generate production-ready, secure code
• Solve competitive programming challenges
• Handle complex mathematical computations
• Break down ambiguous problems systematically

---

Usage

Quick Start

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

# Load model and tokenizer
model_name = "ramdev12345/amara-o1"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(
    model_name,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Generate code
prompt = """<|im_start|>user
Write a Python function to find the longest palindromic substring in a string using dynamic programming.<|im_end|>
<|im_start|>assistant"""

inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
outputs = model.generate(
    **inputs,
    max_new_tokens=512,
    temperature=0.7,
    top_p=0.9,
    do_sample=True
)

response = tokenizer.decode(outputs[0], skip_special_tokens=True)
print(response)

With vLLM (Recommended for Production)

from vllm import LLM, SamplingParams

llm = LLM(model="ramdev12345/amara-o1")
sampling_params = SamplingParams(temperature=0.7, top_p=0.9, max_tokens=512)

prompts = [
    "<|im_start|>user\nOptimize this bubble sort algorithm<|im_end|>\n<|im_start|>assistant"
]

outputs = llm.generate(prompts, sampling_params)
for output in outputs:
    print(output.outputs[0].text)

Chat Template

messages = [
    {"role": "user", "content": "Write a binary search tree implementation in Python"}
]

text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

inputs = tokenizer([text], return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=512)

---

Training

Training Configuration

Parameter	Value
Training Method	Supervised Fine-Tuning (SFT) with QLoRA
Quantization	4-bit NF4
LoRA Rank	64
LoRA Alpha	16
Batch Size	1 (per device)
Gradient Accumulation	8 steps
Learning Rate	2e-4
LR Schedule	Cosine with warmup
Epochs	2
Training Examples	~7,000
Hardware	1x NVIDIA A100 80GB
Training Time	~3 hours

Training Datasets

Dataset	Examples	Purpose
DeepMind Code Contests	5,000	Algorithmic problem solving
MATH-500	500	Mathematical reasoning
Code Vulnerability Security DPO	1,000	Secure coding practices
Anthropic Interviews	1,000	Complex reasoning patterns

LoRA Target Modules

q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj

---

Performance

amara-o1 demonstrates strong performance across multiple coding benchmarks:

Coding Capabilities

Task Type	Performance	Notes
Algorithm Implementation	⭐⭐⭐⭐⭐	Excellent on competitive programming
Code Security	⭐⭐⭐⭐⭐	Trained on vulnerability detection
Math Problems	⭐⭐⭐⭐	Strong symbolic reasoning
Code Explanation	⭐⭐⭐⭐⭐	Clear, detailed breakdowns
Multi-language Support	⭐⭐⭐⭐	90+ programming languages

Example Outputs

Input: "Write a function to detect if a linked list has a cycle"

Output:

def has_cycle(head):
    """
    Detect cycle in linked list using Floyd's algorithm.
    Time: O(n), Space: O(1)
    """
    if not head or not head.next:
        return False
    
    slow = head
    fast = head.next
    
    while slow != fast:
        if not fast or not fast.next:
            return False
        slow = slow.next
        fast = fast.next.next
    
    return True

---

Limitations

While amara-o1 is a powerful coding assistant, users should be aware of:

• Not a Replacement for Testing: Always test generated code thoroughly
• Security: Review security-critical code manually
• Domain Expertise: May require human oversight for specialized domains
• Hallucinations: Like all LLMs, may occasionally generate incorrect information
• License Compliance: Ensure generated code complies with your licensing requirements
• Bias: May reflect biases present in training data

---

Ethical Considerations

Intended Use

✅ Recommended Uses:

• Educational programming assistance
• Code prototyping and rapid development
• Algorithm implementation
• Security vulnerability analysis
• Code review and optimization

❌ Not Recommended:

• Generating malicious code
• Bypassing security measures
• Automating critical systems without human oversight
• Legal or financial decision-making

Bias and Safety

amara-o1 has been trained on diverse coding datasets, but may still reflect biases in:

• Programming paradigm preferences
• Language-specific idioms
• Solution approaches

Users should:

• Review outputs for appropriateness
• Apply domain expertise
• Follow security best practices
• Test thoroughly before deployment

---

System Requirements

Minimum Requirements

Component	Requirement
GPU Memory	16GB (with 4-bit quantization)
RAM	32GB recommended
Storage	15GB for model files

Recommended Setup

• GPU: NVIDIA A100, A6000, or RTX 4090
• Inference: Use vLLM or TGI for production
• Quantization: 4-bit or 8-bit for resource constraints

---

Citation

If you use amara-o1 in your research or applications, please cite:

@misc{amara-o1-2024,
  title={amara-o1: A Fine-tuned Coding Model for Advanced Problem Solving},
  author={ramdev12345},
  year={2024},
  howpublished={\url{https://huggingface.co/ramdev12345/amara-o1}},
}

Base Model Citation

@article{qwen2.5,
  title={Qwen2.5-Coder Technical Report},
  author={Qwen Team},
  journal={arXiv preprint},
  year={2024}
}

---

License

This model is released under the MIT License. See LICENSE for details.

The model inherits the license from its base model (Qwen2.5-Coder).

---

Acknowledgments

• Base Model: Qwen Team for Qwen2.5-Coder
• Training Datasets: DeepMind, Hugging Face, CyberNative, Anthropic
• Infrastructure: Modal Labs for training infrastructure
• Framework: Hugging Face Transformers, PEFT, TRL

---

Contact & Support

• Issues: GitHub Issues
• Discussions: Hugging Face Discussions
• Email: [[email protected]] - contact me here!

HIRE ME NOW!

---

Built with 💻 for the coding community

⭐ Star this repo | 🐛 Report bugs | 🤝 Contribute