Measured on device (edge-compat): Mac Studio M4 Max · LiteRT-LM 0.14.0 · GPU · decode 68.9 tok/s · prefill 971 tok/s · TTFT 278 ms (2026-07-23); Raspberry Pi 5 · LiteRT-LM 0.16.1 · CPU, 4 threads · decode 1.5 tok/s · prefill 11 tok/s · TTFT 25.66 s (2026-09-02). Record: https://github.com/john-rocky/edge-compat/blob/main/cards/polaris-4b/CARD.md

Polaris-4B-Preview — LiteRT-LM (blockwise int4)

POLARIS-Project/Polaris-4B-Preview converted to the LiteRT-LM (.litertlm) format for on-device inference with Google's LiteRT-LM runtime (the engine behind the official litert-community/* models).

Polaris-4B is an RL post-trained reasoning model built on Qwen3-4B (standard dense qwen3, Apache-2.0). It is tuned for hard competition math and works the problem inside a <think>…</think> chain before answering — a SOTA-for-size math reasoner that runs fully on a phone.


File	`model.litertlm` (~2.3 GB; embedding externalized so every section is <2 GiB → loads on iOS)
Quantization	int4 weights — blockwise (block 128) + OCTAV optimal-clipping, symmetric; embedding INT8
Compute	integer
Context (KV cache)	4096
Base model	POLARIS-Project/Polaris-4B-Preview (Apache-2.0)

What it's good at — hard math (AIME)

Polaris-4B's headline is competition math. Per the base model card, at ~4B params it reports AIME24 81.2 / AIME25 79.4, in the range of far larger frontier reasoners. It is optimized for long-chain hard-problem reasoning rather than grade-school arithmetic — give it a generous token budget (it thinks at length).

Usage

litert_lm_main \
  --model_path model.litertlm \
  --backend gpu \
  --input_prompt "Find the number of ordered pairs (a,b) of integers with 1<=a,b<=100 such that a*b is a perfect square."

The .litertlm bundle carries the tokenizer and a ChatML prompt template (<|im_start|>role\n … <|im_end|>). It emits a <think>…</think> chain then the final answer, and stops cleanly at <|im_end|>. Set a high max-tokens (≥2048) — a reasoning model truncated mid-thought produces no answer.

Run on Android

Install a recent Google AI Edge Gallery (1.0.16+ imports .litertlm directly from Hugging Face), import this repo (or push model.litertlm), pick the GPU backend, and chat. It's a ~2.3 GB / 4B model — GPU needs a ~12 GB+ device; free RAM first on smaller phones.

Performance

litert-lm benchmark (litert-lm 0.15.0) on an Apple M4 Max, -p 256 -d 256 --runs 3 (the tool averages three iterations), max-num-tokens 4096, warm-up run discarded, otherwise idle machine.

Device	Backend	Prefill (256)	Decode	TTFT	Load	Peak footprint
Apple M4 Max (macOS)	CPU	111 tok/s	17.9 tok/s	2.50 s	—	—
Apple M4 Max (macOS)	GPU (Metal)	1002 tok/s	69.1 tok/s	0.28 s	—	—

Reproducibility: the GPU rows repeat to within about 1% across invocations; the CPU rows are noisier — re-running the 1B control six times spread its CPU decode over 29.0–33.3 tok/s, so treat the CPU column as accurate to roughly ±7%.

The model loads and runs on iPhone, Android and desktop; no iPhone or desktop on-device timing was taken, and the Galaxy S26 GPU figures are withheld for the reason given in that section below.

Accuracy note (on-device int4 parity)

Measured on GSM8K (n=50, greedy, 0-shot chain-of-thought, max-tokens 2048):

Configuration	GSM8K
This model — LiteRT int4 (block128 + OCTAV)	82.0%

Non-degenerate, passes the local quality gate 8/8 with a clean stop at <|im_end|>. GSM8K undersells this model — it is tuned for AIME-level problems, and on easy arithmetic its long exploratory reasoning is not where its edge shows. block128 is used (rather than block32) because a 4B reasoning model's block32 weights can corrupt on the iPhone Metal GPU; block128 loads and runs stably across iPhone / Android / desktop.

Galaxy S26 — GPU backend

The published bundle runs on the Android GPU backend and generates.

file	GPU backend	delegation	peak
`model.litertlm`	runs	3274 / 3274 ops across 2 subgraphs on `LiteRT GPU`	1022 MB

Measured on a Samsung Galaxy S26 (SM-S942Q / SM8850, Android 16) with litert_lm_advanced_main from litert-lm 0.16.0, --backend=gpu --sampler_backend=cpu, prompt What is the capital of France?. Peak is the process high-water mark (VmHWM) sampled during that same run. Gated 2026-08-24.

The op counts above are the LiteRT GPU partitions. XNNPACK additionally takes 1 of the 4 nodes in decode_embedder and 1 of the 4 nodes in prefill_embedder_128; the runtime accepts that split.

No speed rows, on purpose. On this handset the GPU backend wins prefill and does not win decode, so a GPU throughput figure only means something beside a CPU row from the same handset, and no S26 CPU row exists for this model yet.

GPU wiring, including the Gallery import toggle: GPU guide.

Conversion

Converted with litert-torch: blockwise int4 (block 128) + OCTAV optimal-clipping, embedding INT8, KV cache 4096, ChatML template. Polaris-4B is a standard dense Qwen3ForCausalLM (with rope_scaling: yarn, exported with a cache within original_max_position_embeddings so base RoPE is exact), so it rides the existing Qwen3 converter with no custom graph code. externalize_embedder=True keeps every .litertlm section under the iOS ~2 GiB single-section mmap limit so it loads on iPhone.

2026-08-30 — tokenizer section replaced (weights unchanged)

The tokenizer in model.litertlm was a SentencePiece conversion of the model's BPE tokenizer, and the conversion lost the byte-level semantics: a standalone accented letter or symbol (é, ñ, ü, °, ·, …) was encoded to the id of a single-byte token instead of the token the upstream tokenizer uses, and any character without a whole-character vocabulary entry (emoji, most of Latin Extended-A) became the token the conversion had reused as UNK — the end-of-text token for this vocabulary. model.litertlm now embeds the upstream tokenizer.json (the same HF tokenizer path most bundles in this collection use).

Tokenizer-only change: every section of the bundle except the tokenizer is byte-identical to the previous file (verified by sha256 per section), so the weights, the graph and the chat template are unchanged and the speed and memory numbers on this card still describe this file per token — only the file's own sha256 differs. Verified on the LiteRT-LM runtime: the default turn, 7 probe strings, the 223 standalone characters U+00A1–U+017F and every special token now tokenize identically to the upstream tokenizer, and the four ASCII-only test questions answer byte-identically to the previous file (same ids in, same tokens out). Prompts containing accented letters, symbols or emoji reach the model differently from before, so individual answers to such prompts can change. Unless a row says otherwise, the accuracy figures on this card were measured on the previous file, and any on-device rows were measured on the previous file too — the on-device gate has not been re-run on this one (the runtime's tokenizer code is the same on macOS and on device; the weights and graph are byte-identical). If you downloaded before 2026-08-30, re-download.

Raspberry Pi 5 (CPU)

Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with litert-lm benchmark 0.16.1: CPU backend, 4 threads, 256 prefill + 256 decode tokens, --cache memory (the compile cache lives and dies with the process, so every invocation compiles the model from scratch; nothing is reused between runs), one warm-up plus one timed iteration per invocation, 3 invocations per file with cooldown in between. Values are the median across invocations (min–max in parentheses). No thermal throttling occurred during these runs (vcgencmd get_throttled stayed 0x0). Every file listed produced coherent text in a real generation on this backend before its numbers were recorded.