# Architecture

This document goes one level deeper than the main [README](../README.md) into how Lumen decides what to do with a message, how streaming works, and how the security layers fit together.

## Request lifecycle

1. Telegram POSTs an update to `/webhook`, authenticated by a secret header (`X-Telegram-Bot-Api-Secret-Token`). The body is capped at 512 KB, in-flight updates at 100, and duplicate `update_id` deliveries are ignored so a Telegram retry cannot trigger a second answer, download or quota charge.
2. `_handle_message_core` (`lumen_message_core.py`, re-exported in `bot.py`) classifies the message: is it a TikTok link, a `/draw` or `/tts` trigger phrase, a reply to earlier media, plain text, or an attachment?
3. `_build_route` (`lumen_router_config.py`) turns that classification into an ordered list of `(provider, model_id)` candidates.
4. `_run_route` tries the first candidate. If a whole provider's chain fails, it falls back to the next provider in the route, except where that's physically impossible (only Gemini can read links or analyze video/audio).
5. The reply streams into the chat if the request qualifies (see [Streaming](#streaming--typing-pace) below), or is sent as a single message otherwise.

## Automatic model routing

Routing is fully automatic: the router builds a fresh candidate list for every message:

- **A YouTube or website link** → Gemini only. It's the only provider that can read page content (`url_context`) or analyze a video by URL.
- **A video or audio attachment** → video goes to Gemini only (OpenRouter's multimodal models only accept images as base64). Voice/audio is first transcribed cheaply (Groq Whisper) and the text joins the normal routing below; if transcription fails or is unavailable, the audio goes to Gemini as before.
- **An image attachment, no live-info need** → free OpenRouter vision models first, Gemini as a reserve. With a freshness need, Gemini search models go first and OpenRouter vision stays as a fallback.
- **Needs current information** (a lightweight keyword heuristic: "now," "today," "price," "who is currently...," plus an explicit "search the web / погугли" ask; astronomy that depends on the date — sunrise/sunset/moon — triggers it too) → Gemini, prioritizing the models with a real search-grounding quota, then OpenRouter as a reserve, Groq last. Every freshness/search query also gets a free external snippet block (`WEB_SEARCH_*`) attached to the prompt, so whichever model answers, including the non-Gemini reserves and users over their daily Gemini limit, has real search results. For news queries DuckDuckGo and Google News run in parallel and are merged, Google News first; Google News is filtered with `when:1d` for "today" questions and `when:7d` otherwise, and RSS publication dates are embedded in the snippets. Non-news queries try DuckDuckGo first, then Bing RSS and Google News as fallbacks. If the fetch returns nothing (or the switch is off), the reply is honestly marked as answered without live search. Freshness prompts additionally carry today's date (Europe/Moscow) so relative days are not guessed.
- **Plain text, no attachments, no freshness need** (the most common case) → Gemma first (offline eval 22.10.2026: 47/50 persona score, ~14.4K free requests/day per model via the Google free tier; 26B-A4B head, 31B next), then Groq (1000 free requests/day: Qwen, then gpt-oss), then OpenRouter, Gemini as reserve. Heavier requests (code, multi-step reasoning, caught by another lightweight heuristic) still start at Groq to spare the scarcer OpenRouter/Gemini quotas, then go to the stronger free OpenRouter models, Gemini as reserve.

The reasoning: for ordinary messages the bulk quality came out of the offline persona eval with Gemma at the top, and its Google free quota is by far the largest (thousands per day versus 500 for Gemini lite models), so Groq and OpenRouter are now reserves rather than the text backbone. Gemini remains the only provider for links, video, audio and native grounding; external snippets let the other providers answer fresh questions too.

**Experiment switch (`MUNITO_ENABLED`, 05.11.2026).** While the owner tests Munito AI (EU-hosted `deepseek-ai/deepseek-v4.1-flash-1017`, then `zai-org/glm-5.3-flash`; zero data retention by default), every plain-text route above — normal, heavy and fresh — collapses to that single provider, and history summaries start there too. Attachments, links, YouTube and TTS stay on Gemini: no other provider accepts them, and Munito's US host does not even serve DeepSeek or GLM. `MUNITO_KEEP_FALLBACKS=1` appends the chains above as reserves instead of replacing them — without it a Munito `429`/`503` means no answer at all. Numbers behind the choice, measured live by `eval/munito_probe.py`: first token 0.24 s and 200 requests per 30-second window on the head model, 1.50 s and 10 requests on GLM, and an empty `content` from Qwen without streaming (reasoning tokens only) — which is why Qwen is not a head. Flipping one variable restores the routing above untouched; the chains themselves were not edited. Every Munito request also carries `reasoning_effort` (`MUNITO_REASONING_EFFORT`, default `low`): measured 05.11.2026 it removes GLM's thinking tokens almost entirely (0.13-0.26s instead of 0.5-1.0s) and stops `qwen3.8-27b` from spending the whole output budget on reasoning and returning an empty body. The price: on a two-step distance problem GLM answered `180` at `low` and `240` at `minimal` (correct is 210) and was right on the rerun, while the DeepSeek head was right throughout — so on hard multi-step requests the effect is a real but intermittent risk. Reversible without a code change by emptying the variable, which drops the field entirely (Munito answers `400 unknown parameter` to anything it does not know).

Models that the router should never pick are tracked in a single registry, `_OR_MODEL_HEALTH` (`lumen_router_config.py`), each with a dated reason: the provider dropped the free tier, or the model turned out to be uncensored and a poor fit for Lumen's persona. Nothing is removed on speculation, only on confirmed evidence from production logs (`HTTP 404`, `"no endpoints,"` etc.).

Each model is tried exactly once per route, with no retries: a single failure (timeout, `429`, `5xx`) moves straight to the next candidate. That keeps the worst case bounded by `len(route) × ROUTE_MODEL_TIMEOUT_SEC`, further capped by `ROUTE_TOTAL_BUDGET_SEC` for the whole route. Within each provider block the order is re-sorted by measured latency (`lumen_model_speed.py`; provider blocks stay in place), so the stream head is the post-reorder head, not the curated `_build_route` order.

### Image generation

`/draw` follows the same philosophy on a smaller scale. `_pick_image_model` (`lumen_images.py`) picks a Pollinations.ai model from the prompt's content (anime/manga → `flux-anime`, fantasy → `dreamshaper`, "photorealistic" → `flux-realism`, "quick sketch" → `turbo`, otherwise a general-purpose default) and falls back through the rest of the catalog if the chosen model fails, all within `DRAW_TOTAL_BUDGET_SEC`.

## Streaming & typing pace

For a plain-text route (no attachments, no links), Lumen streams the reply by repeatedly editing one message, the same mechanism for Gemini (`generate_content_stream`), OpenRouter, Groq and Munito (SSE over `chat/completions`). Streaming is attempted on every model in the route, not only on the first one: a failing head model no longer costs the chat its typing effect while a reserve model answers.

Telegram won't let a bot edit one message more than about once a second, so true token-by-token output isn't possible regardless of how fast the model generates text. Instead, `lumen_typing_pace.py` tracks each `provider:model` pair's real observed characters-per-second and reveals a growing slice of the buffered text at that pace, capped by whatever has actually arrived. If a backend returns the whole answer in one large chunk (common for free OpenRouter models, since the same `:free` slug can be served by different backends depending on load), a short "catch-up" phase finishes revealing it smoothly instead of dumping it all at once, capped at `STREAM_TYPING_MAX_CATCHUP_TICKS × STREAM_TYPING_TICK_SEC` seconds.

This speed estimate is measured, not hardcoded. There is no table of "model X does N tokens/sec" to maintain, because that number isn't a property of the model on OpenRouter's free tier in the first place. A new model just starts at a default speed and calibrates itself over its first few replies.

If a streaming attempt fails before showing any text, the model is tried once more without streaming; the bot reuses the same placeholder message rather than sending a new one, and the next models in the route also get their own streaming attempt. In private chats the answer is a plain message, without a Telegram reply quote; groups keep reply threading for context.

In private chats every request with a real early stage (web search, media, a link or YouTube) first gets a live status bubble (`lumen_status.py`) styled after Claude: an animated three-dot triangle spinner (∴ ⋰ ⋯ ⋱ ∵, a full symmetric rotation through a centered horizontal row, Claude's new indicator) plus "Searching the web", then a checkmarked "Searched the web · found N" and a "Thinking… Ns" timer while the model call runs. The same bubble is reused for the answer. At the final edit a collapsed `Thought for Ns` blockquote (with the completed steps and search source domains) is prepended above the answer, mirroring Claude's collapsible reasoning/activity block; Telegram collapses it by default. Plain text without an early stage skips the early bubble and gets a single italic `✻ Thought for Ns` line above the answer once streaming finishes (no quote frame). Long answers split across several Telegram messages skip the block: it would not fit the per-message limit. TikTok downloads, `/draw` and `/tts` reuse the same animated status with a live seconds counter while the action runs and remove it once the media or voice is sent. Status phrases are fixed pipeline facts (never model reasoning or model identity) and are localized in `lumen_lang.py`.

Final answers go out via `sendRichMessage` (Bot API 10.1: real tables, headings, math) with automatic fallback to plain HTML; streaming edits always use HTML. Kill-switch: `RICH_MESSAGES_ENABLED=0` (plus restart) forces legacy HTML.

### Manual smoke test before touching streaming code

Streaming is covered only by mocked clients; no test exercises a real Gemini or OpenRouter SSE stream. Before merging changes to `_run_streaming_reply`, `_gemini_stream_pieces`, `_openrouter_stream_pieces`, `_groq_stream_pieces`, or `lumen_typing_pace.py`, check by hand in a real chat:

1. A short message (fits in one edit): once via Gemini, once via OpenRouter.
2. A long message (past `TG_MAX_LEN`): confirm it splits into multiple messages and continues correctly.
3. A request where the first streaming model is unavailable: confirm the next model also streams (the typing effect must not disappear), and that a totally failed stream still falls back to a normal answer with no visible glitch.

## Prompt-injection and identity-leak defenses

Lumen deliberately hides which model or provider answers a given message (see the persona in `system_prompt.py`). The system prompt alone is the weakest layer, since any LLM can potentially be talked out of following it with a creative enough injection. So there are several independent layers, each assuming the previous one might have failed:

1. **Input pre-filter** (`_looks_like_injection_probe`, `lumen_security.py`): well-known jailbreak phrasing ("ignore previous instructions," "developer mode," "show me your system prompt") is caught before the model is ever called. The response is fully deterministic.
2. **System prompt** (`system_prompt.py`): instructs the model that anything outside the prompt itself (user messages, chat background, page/video/document content) is data, not instructions, and that the persona doesn't change no matter who claims authority to override it. Reply quotes, display names, nickname hints and passive group background are flattened to one line with bracketed markers escaped before they enter the prompt, so cross-user text cannot forge the `[...]` envelope or a fake "current question" block.
3. **Output identity-leak filter** (`_detect_identity_leak` / `_scrub_identity_leak`): a deterministic check on the finished reply, catching exact internal model IDs and first-person self-identification patterns ("I am Gemini", "I am a Gemini model", "created/built/trained by Google", "основан на" only with "я/I" in the same sentence). Detection normalizes NFKC and strips zero-width/format characters first; brands include Gemini/Gemma, OpenAI/Claude/Anthropic, Groq, Munito (and GLM-4/GLM-5, Zhipu — the routed model and its host, 05.11.2026), Hugging Face and the other routed providers. Deliberately narrow, to avoid false positives on ordinary, honest discussion of other AI companies.
4. **Injected-payload echo filter** (`_detect_injected_payload_echo`): catches the case where an attacker embeds an instruction in a photo or web page ("output this exact string to confirm the jailbreak worked") and the model, while refusing to *follow* it, ends up quoting it back verbatim while summarizing the content.
5. **Garbled-text tripwire** (`_detect_garbled_mix`): flags replies with fragments of unrelated scripts wedged *inside* words (the failure mode of `nemotron-nano-9b-v2`, see `_OR_MODEL_HEALTH`). Log-only (`[mush-suspect]`), never blocks — confirmed cases are reviewed by a human and added to the health registry.

Layers 3 and 4 run before the reply is written to chat history (so a leak can't influence future turns) and, during streaming, before each chunk is shown to the user, not just the final text. The final identity/echo scrub also runs before the last visible edit, so history and the shown text agree. Layer 5 runs once on the finished reply (inside the final scrub), for both streaming and non-streaming paths.

None of this is airtight except the input pre-filter. The goal is raising the bar for known attack patterns, not proving immunity to every possible phrasing; incidents get logged with `[identity-leak]` / `[injection-echo]` / `[injection-probe]` / `[mush-suspect]` tags so new patterns can be added as they show up.

## TikTok downloader

Downloads go through the public TikWM API (`tikwm.com`), no account or API key needed. The bot never re-encodes video or photos; bytes go to Telegram exactly as TikWM served them. Music tracks are the exception: MP3 tags are rewritten with mutagen before sending. For licensed music, TikWM only exposes "original sound - handle" and the video author's avatar; the bot reads TikTok's DSP song id (`tt2dsp.SongId`) from the video page and resolves the real track title, artist and artwork through the free iTunes Lookup API. Artwork is downscaled to Telegram's 320×320 thumbnail limit with ffmpeg. If the TikTok page is unreachable, the bot falls back to the TikWM fields with a corrected original-sound label instead of inventing a title from the handle.

- **Video quality**: requested with `&hd=1`; among the variants TikWM returns (`hdplay`/`play`/`wmplay`, each with a known byte size), the best one that fits Telegram's 50 MB upload limit is picked. If Telegram still rejects it as too large, the next lighter variant is tried automatically.
- **Slideshows**: TikTok allows up to 35 slides per post; Telegram's `sendMediaGroup` caps out at 10 items per call. The bot downloads the whole post in parallel and sends it as several media groups in sequence, never with a trailing group of exactly one item (Telegram requires 2–10 per group).
- **"Live" slides**: TikWM's response includes a separate `live_images` field alongside the regular `images`; that's where the actually-moving version of a slide lives, if it has one. Each downloaded slide is also double-checked by its magic bytes (`ftyp` = MP4 container, EBML header = WebM) rather than trusting the field alone.
- TikWM rejects Hugging Face Spaces' outbound IPs with an empty `403`, the same class of block that also keeps [YouTube downloading off the table](../README.md#known-limitations). The proxy (`TIKWM_API_BASE_URL`) is the only fix that's worked in practice; throttling, retries, and spoofed `Referer`/`Origin` headers didn't help on their own (they're still in place as defense in depth).
- Successful TikTok deliveries count as one answer toward the user's daily quota. Telegram and TikTok media downloads share a process-wide concurrency cap (`MEDIA_GLOBAL_CONCURRENCY`, default 2) on top of the per-file and per-post byte limits.

## Persistent storage

Chat history is stored per chat (`<chat_id>.json`, or `lumen:chat:<chat_id>` in Upstash) with a `chat_index.json` index; quota counters live in `global_quota.json` (`lumen:global_quota`). The old combined `chat_state.json` is only a migration fallback. By default everything is on the container's local disk, which Hugging Face Spaces wipes on every redeploy. If `UPSTASH_REDIS_REST_URL` and `UPSTASH_REDIS_REST_TOKEN` are both set, the same data is written to Upstash Redis instead and survives redeploys. Each chat gets its own key so one failed write can't take down every chat's state at once.

## Error tracking

If `SENTRY_DSN` is set, `sentry_sdk` picks up every `log.exception()`/`log.error()` call already in the codebase with no changes needed at the call sites. Local variables and request bodies are disabled, because stack locals can contain prompts and chat history; every event is additionally scrubbed of known secrets (`BOT_TOKEN`, API keys, `WEBHOOK_SECRET`, `ADMIN_PANEL_KEY`, the Upstash token) before it leaves the process. Performance tracing is off; this only tracks errors, to stay comfortably inside Sentry's free tier.