Hugging Face's logo

QuantFactory
/
Llama-Deepsync-1B-GGUF

Text Generation
Transformers
GGUF
text-generation-inference
Llama
Code
CoT
Math
Deepsync
3b
conversational
Model card Files
xet
 Community
aashish1904 commited on
Commit
0f424b9
·
verified ·
1 Parent(s): ddbde15

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +120 -0
README.md ADDED
@@ -0,0 +1,120 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+
2
+ ---
3
+
4
+ license: creativeml-openrail-m
5
+ language:
6
+ - en
7
+ - de
8
+ - fr
9
+ - it
10
+ - pt
11
+ - hi
12
+ - es
13
+ - th
14
+ base_model:
15
+ - meta-llama/Llama-3.2-1B-Instruct
16
+ pipeline_tag: text-generation
17
+ tags:
18
+ - text-generation-inference
19
+ - Llama
20
+ - Code
21
+ - CoT
22
+ - Math
23
+ - Deepsync
24
+ - 3b
25
+ library_name: transformers
26
+
27
+ ---
28
+
29
+ [![QuantFactory Banner](https://lh7-rt.googleusercontent.com/docsz/AD_4nXeiuCm7c8lEwEJuRey9kiVZsRn2W-b4pWlu3-X534V3YmVuVc2ZL-NXg2RkzSOOS2JXGHutDuyyNAUtdJI65jGTo8jT9Y99tMi4H4MqL44Uc5QKG77B0d6-JfIkZHFaUA71-RtjyYZWVIhqsNZcx8-OMaA?key=xt3VSDoCbmTY7o-cwwOFwQ)](https://hf.co/QuantFactory)
30
+
31
+
32
+ # QuantFactory/Llama-Deepsync-1B-GGUF
33
+ This is quantized version of [prithivMLmods/Llama-Deepsync-1B](https://huggingface.co/prithivMLmods/Llama-Deepsync-1B) created using llama.cpp
34
+
35
+ # Original Model Card
36
+
37
+ <pre align="center">
38
+ .___ _______.
39
+ __| _/____ ____ ______ _________.__. ____ ____ /_ \_ |__
40
+ / __ |/ __ \_/ __ \\____ \/ ___< | |/ \_/ ___\ | || __ \
41
+ / /_/ \ ___/\ ___/| |_> >___ \ \___ | | \ \___ | || \_\ \
42
+ \____ |\___ >\___ > __/____ >/ ____|___| /\___ > |___||___ /
43
+ \/ \/ \/|__| \/ \/ \/ \/ \/
44
+ </pre>
45
+
46
+ The **Llama-Deepsync-1B** is a fine-tuned version of the **Llama-3.2-1B-Instruct** base model, designed for text generation tasks that require deep reasoning, logical structuring, and problem-solving. This model leverages its optimized architecture to provide accurate and contextually relevant outputs for complex queries, making it ideal for applications in education, programming, and creative writing.
47
+
48
+ With its robust natural language processing capabilities, **Llama-Deepsync-1B** excels in generating step-by-step solutions, creative content, and logical analyses. Its architecture integrates advanced understanding of both structured and unstructured data, ensuring precise text generation aligned with user inputs.
49
+
50
+ - Significantly **more knowledge** and has greatly improved capabilities in **coding** and **mathematics**, thanks to our specialized expert models in these domains.
51
+ - Significant improvements in **instruction following**, **generating long texts** (over 8K tokens), **understanding structured data** (e.g, tables), and **generating structured outputs** especially JSON. **More resilient to the diversity of system prompts**, enhancing role-play implementation and condition-setting for chatbots.
52
+ - **Long-context Support** up to 128K tokens and can generate up to 8K tokens.
53
+ - **Multilingual support** for over 29 languages, including Chinese, English, French, Spanish, Portuguese, German, Italian, Russian, Japanese, Korean, Vietnamese, Thai, Arabic, and more.
54
+
55
+ # **Model Architecture**
56
+
57
+ Llama 3.2 is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) to align with human preferences for helpfulness and safety.
58
+
59
+ # **Use with transformers**
60
+
61
+ Starting with `transformers >= 4.43.0` onward, you can run conversational inference using the Transformers `pipeline` abstraction or by leveraging the Auto classes with the `generate()` function.
62
+
63
+ Make sure to update your transformers installation via `pip install --upgrade transformers`.
64
+
65
+ ```python
66
+ import torch
67
+ from transformers import pipeline
68
+
69
+ model_id = "prithivMLmods/Llama-Deepsync-1B"
70
+ pipe = pipeline(
71
+ "text-generation",
72
+ model=model_id,
73
+ torch_dtype=torch.bfloat16,
74
+ device_map="auto",
75
+ )
76
+ messages = [
77
+ {"role": "system", "content": "You are a pirate chatbot who always responds in pirate speak!"},
78
+ {"role": "user", "content": "Who are you?"},
79
+ ]
80
+ outputs = pipe(
81
+ messages,
82
+ max_new_tokens=256,
83
+ )
84
+ print(outputs[0]["generated_text"][-1])
85
+ ```
86
+
87
+ Note: You can also find detailed recipes on how to use the model locally, with `torch.compile()`, assisted generations, quantised and more at [`huggingface-llama-recipes`](https://github.com/huggingface/huggingface-llama-recipes)
88
+
89
+ # **Run with Ollama [Ollama Run]**
90
+
91
+ Ollama makes running machine learning models simple and efficient. Follow these steps to set up and run your GGUF models quickly.
92
+
93
+ ## Quick Start: Step-by-Step Guide
94
+
95
+ | Step | Description | Command / Instructions |
96
+ |------|-------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------|
97
+ | 1 | **Install Ollama 🦙** | Download Ollama from [https://ollama.com/download](https://ollama.com/download) and install it on your system. |
98
+ | 2 | **Create Your Model File** | - Create a file named after your model, e.g., `metallama`. |
99
+ | | | - Add the following line to specify the base model: |
100
+ | | | ```bash |
101
+ | | | FROM Llama-3.2-1B.F16.gguf |
102
+ | | | ``` |
103
+ | | | - Ensure the base model file is in the same directory. |
104
+ | 3 | **Create and Patch the Model** | Run the following commands to create and verify your model: |
105
+ | | | ```bash |
106
+ | | | ollama create metallama -f ./metallama |
107
+ | | | ollama list |
108
+ | | | ``` |
109
+ | 4 | **Run the Model** | Use the following command to start your model: |
110
+ | | | ```bash |
111
+ | | | ollama run metallama |
112
+ | | | ``` |
113
+ | 5 | **Interact with the Model** | Once the model is running, interact with it: |
114
+ | | | ```plaintext |
115
+ | | | >>> Tell me about Space X. |
116
+ | | | Space X, the private aerospace company founded by Elon Musk, is revolutionizing space exploration... |
117
+ | | | ``` |
118
+
119
+ ## Conclusion
120
+ With Ollama, running and interacting with models is seamless. Start experimenting today!