Instructions to use flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M

Use Docker

docker model run hf.co/flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M

Ollama
How to use flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF with Ollama:
```
ollama run hf.co/flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M
```
Unsloth Desktop

How to use flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF with Docker Model Runner:
```
docker model run hf.co/flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M
```

Lemonade

How to use flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

⚡ FlashModel-Qwen-3.6-35B-A3B-Maxed — GGUF

GGUF quants of FlashModel-Qwen-3.6-35B-A3B-Maxed, a reasoning / coding / agentic / bilingual fine-tune of Qwen/Qwen3.6-35B-A3B.

Key capabilities:

Code: Rust, TypeScript, JavaScript, Python, QML, HTML, CSS, Bash
Controllable reasoning effort: reasoning_effort=low|medium|high|xhigh|max in the system prompt — real control, not truncation
Agentic tool use: file read/write, shell, git operations
Bilingual EN/RU dialogue (≈22% RU in training)
MTP speculative decoding (--spec-type draft-mtp): 77% draft acceptance; measured decode 29.2 vs 27.2 tok/s without MTP (about +7%, one 400-token sample, IQ3_XXS on a Radeon 680M iGPU).

Provided Quants

All quants use an importance matrix computed on this fine-tune with code-weighted calibration (local coding chats + Rust/TS/JS/Python/QML/CSS samples, plus some RU). Every tensor uses the quant's base type (stock llama-quantize, no expert overrides); the MTP head is kept at Q4_K/Q5_K in the low-bit quants. Sizes include the MTP head.

Why this range? Below IQ3_XXS quality degrades noticeably on code and reasoning; above Q5_K_M the gain is too small to justify the extra VRAM — for a 24 GB card the sweet spot is already covered. Q4_K_M is the default recommendation.

Link	Type	Size/GB	Notes
GGUF	IQ3_XXS	~14.1	floor — smaller starts to hurt quality
GGUF	IQ3_S	~15.4	better quality
GGUF	IQ4_XS	~18.2	good trade-off
GGUF	IQ4_NL	~18.5	balanced+
GGUF	Q4_K_M	~22.7	recommended for 24 GB VRAM
GGUF	Q5_K_S	~24.9	24 GB VRAM + offload some layers
GGUF	Q5_K_M	~26.5	ceiling — higher is pointless for this VRAM
imatrix	imatrix	—	importance matrix used for every quant above

Which one should I pick?

Your free VRAM	Pick
≥ 28 GB	`Q5_K_M` — best quality available
~26 GB	`Q5_K_S`
24 GB	`Q4_K_M` — default choice
~22 GB	`IQ4_XS`
≤ 20 GB	`IQ3_S`, then `IQ3_XXS`

Leave room for the KV cache: long reasoning traces need context. Size numbers are file sizes, not total memory use. Context length beyond 16k is not yet measured on this fine-tune.

mmproj

mmproj-Q8_0.gguf — vision encoder in Q8_0 (~0.45 GB). Use with vision-enabled builds.

Usage

llama.cpp (recommended)

# download one quant
huggingface-cli download flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF \
  FlashModel-Qwen-3.6-35B-A3B-Maxed-Q4_K_M.gguf mmproj-Q8_0.gguf --local-dir .

# server with MTP speculative decoding
llama-server -m FlashModel-Qwen-3.6-35B-A3B-Maxed-Q4_K_M.gguf \
  --mmproj mmproj-Q8_0.gguf --spec-type draft-mtp --spec-draft-n-max 2 \
  -c 32768 --flash-attn -ngl 99 --jinja

# effort control via system prompt
#   low    -> 1-2 sentences
#   medium -> paragraph
#   high   -> multi-paragraph reasoning
#   max    -> full chain-of-thought
system: "You are a coding and reasoning assistant.
reasoning_effort=high
Adjust the depth of your reasoning to the level given above."

Or pull straight from the Hub: llama-server -hf flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M --jinja

Ollama

ollama run hf.co/flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF:Q4_K_M

LM Studio

Search for flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF and pick a quant.

Recommended sampling (from Qwen3.6)

Mode	temperature	top_p	top_k	presence_penalty
Thinking, general	1.0	0.95	20	1.5
Thinking, precise coding	0.6	0.95	20	0.0
Non-thinking, general	0.7	0.8	20	1.5

Always pass --jinja so the embedded chat template (thinking blocks, tool calls) is used.

How these were made

Base: flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed (HF bf16 safetensors, LoRA merged, MTP head kept from the Qwen3.6-35B-A3B checkpoint).

Converted with llama.cpp/convert_hf_to_gguf.py to F16 (NextN/MTP tensors included), then quantized with stock llama-quantize using an imatrix computed on this fine-tune (code-weighted calibration). No per-tensor expert overrides; the MTP head is kept at Q4_K/Q5_K in the IQ3 quants.

No per-tensor expert overrides: each quant uses its base type for all tensors. The MTP head (block 40) is kept at Q4_K/Q5_K in the IQ3 quants because it never runs in a normal forward pass and so has no imatrix data.

Known eval results (baseline → finetuned)

Metric	Baseline	Finetuned
Tool Use (8 scenarios)	6/8	7/8
Agentic (5 tasks)	5/5	5/5
Effort monotonic (low→max)	❌	✅
RU Perplexity	—	5.41

Full benchmark suite (HumanEval, MBPP, GSM8K, MMLU, SWE-bench) planned — see the weights repo card for training details.

Downloads last month: 3,244

GGUF

Model size

36B params

Architecture

qwen35moe

Hardware compatibility

3-bit

4-bit

5-bit

View +1 variant

Model tree for flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF

Base model

Qwen/Qwen3.6-35B-A3B

Finetuned

flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed

Quantized

(3)

this model

flashback2k
/

FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF

⚡ FlashModel-Qwen-3.6-35B-A3B-Maxed — GGUF

Provided Quants

mmproj

Usage

llama.cpp (recommended)

Ollama

LM Studio

Recommended sampling (from Qwen3.6)

How these were made

Known eval results (baseline → finetuned)

Model tree for flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF

Datasets used to train flashback2k/FlashModel-Qwen-3.6-35B-A3B-Maxed-GGUF