Do VLMs Need Vision Transformers? Evaluating State Space Models as Vision Encoders
Paper โข 2603.19209 โข Published โข 5
Magic Leap Community
community
AI & ML interests
This organization is maintained by the transformers team at Hugging Face and contains checkpoints for models developed by Magic Leap.
Recent Activity
View all activity
nielsrย
submitted a
paper to Daily Papers 9 days ago
nielsrย
submitted a
paper to Daily Papers 13 days ago
nielsrย
submitted a
paper to Daily Papers 14 days ago
nielsrย
authored a
paper 19 days ago
nielsrย
submitted 2
papers to Daily Papers about 1 month ago
nielsrย
submitted a
paper to Daily Papers 2 months ago
stevenbucailleย
posted an update 2 months ago
Post
247
LWDetr is available in ๐ค transformers !
Checkout the collection to find the original paper, model weights and a demo space : https://huggingface.co/collections/stevenbucaille/lwdetr
Checkout the collection to find the original paper, model weights and a demo space : https://huggingface.co/collections/stevenbucaille/lwdetr
nielsrย
submitted 2
papers to Daily Papers 3 months ago
Post
10328
deepseek-ai/DeepSeek-OCR is out! ๐ฅ my take โคต๏ธ
> pretty insane it can parse and re-render charts in HTML
> it uses CLIP and SAM features concatenated, so better grounding
> very efficient per vision tokens/performance ratio
> covers 100 languages
> pretty insane it can parse and re-render charts in HTML
> it uses CLIP and SAM features concatenated, so better grounding
> very efficient per vision tokens/performance ratio
> covers 100 languages
- 4 replies
Post
6959
large AI labs open-sourced a ton of models last week ๐ฅ
here's few picks, find even more here merve/sep-16-releases-68d13ea4c547f02f95842f05 ๐ค
> IBM released a new Docling model with 258M params based on Granite (A2.0) ๐ ibm-granite/granite-docling-258M
> Xiaomi released 7B audio LM with base and instruct variants (MIT) XiaomiMiMo/mimo-audio-68cc7202692c27dae881cce0
> DecartAI released Lucy Edit, open Nano Banana ๐ (NC) decart-ai/Lucy-Edit-Dev
> OpenGVLab released a family of agentic computer use models (3B/7B/32B) with the dataset ๐ป OpenGVLab/scalecua-68c912cf56f7ff4c8e034003
> Meituan Longcat released thinking version of LongCat-Flash ๐ญ meituan-longcat/LongCat-Flash-Thinking
here's few picks, find even more here merve/sep-16-releases-68d13ea4c547f02f95842f05 ๐ค
> IBM released a new Docling model with 258M params based on Granite (A2.0) ๐ ibm-granite/granite-docling-258M
> Xiaomi released 7B audio LM with base and instruct variants (MIT) XiaomiMiMo/mimo-audio-68cc7202692c27dae881cce0
> DecartAI released Lucy Edit, open Nano Banana ๐ (NC) decart-ai/Lucy-Edit-Dev
> OpenGVLab released a family of agentic computer use models (3B/7B/32B) with the dataset ๐ป OpenGVLab/scalecua-68c912cf56f7ff4c8e034003
> Meituan Longcat released thinking version of LongCat-Flash ๐ญ meituan-longcat/LongCat-Flash-Thinking
- 2 replies
Post
3499
IBM just released small swiss army knife for the document models: granite-docling-258M on Hugging Face ๐ฅ
> not only a document converter but also can do document question answering, understand multiple languages ๐คฏ
> best part: released with Apache 2.0 license ๐ use it with your commercial projects!
> it supports transformers, vLLM and MLX from the get-go! ๐ค
> built on SigLIP2 & granite-165M
model: ibm-granite/granite-docling-258M
demo: ibm-granite/granite-docling-258m-demo ๐
> not only a document converter but also can do document question answering, understand multiple languages ๐คฏ
> best part: released with Apache 2.0 license ๐ use it with your commercial projects!
> it supports transformers, vLLM and MLX from the get-go! ๐ค
> built on SigLIP2 & granite-165M
model: ibm-granite/granite-docling-258M
demo: ibm-granite/granite-docling-258m-demo ๐
Post
1260
a ton of image/video generation models and LLMs from big labs ๐ฅ
> Meta released facebook/mobilellm-r1-68c4597b104fac45f28f448e, smol LLMs for on-device use ๐ฌ
> Tencent released tencent/SRPO, high res image generation model and tencent/POINTS-Reader, cutting edge OCR ๐
> ByteDance released bytedance-research/HuMo, video generation from any input โฏ๏ธ
find more models, datasets, demos here merve/sep-11-releases-68c7dbfa26bea8cd921fa0ac
> Meta released facebook/mobilellm-r1-68c4597b104fac45f28f448e, smol LLMs for on-device use ๐ฌ
> Tencent released tencent/SRPO, high res image generation model and tencent/POINTS-Reader, cutting edge OCR ๐
> ByteDance released bytedance-research/HuMo, video generation from any input โฏ๏ธ
find more models, datasets, demos here merve/sep-11-releases-68c7dbfa26bea8cd921fa0ac
Post
1055
fan-favorite vision LM Florence-2 is now officially supported in transformers ๐ค
find all the models in
florence-community org ๐ซก
find all the models in
Post
1856
past week was great for open LLMs ๐ฅ merve/sep-1-releases-68bede0e729c12597eefd050
> Google released google/embeddinggemma-300m, new embedding model with 300M params
> new update to Kimi-K2 just landed moonshotai/Kimi-K2-Instruct-0905 ๐
> OpenBMB released a new version to MiniCPM with 8B params openbmb/MiniCPM4.1-8B
also soooo many Qwen-Image & Kontext LoRAs dropped!
> Google released google/embeddinggemma-300m, new embedding model with 300M params
> new update to Kimi-K2 just landed moonshotai/Kimi-K2-Instruct-0905 ๐
> OpenBMB released a new version to MiniCPM with 8B params openbmb/MiniCPM4.1-8B
also soooo many Qwen-Image & Kontext LoRAs dropped!
Post
3742
upgrade your transformers ๐ฅ
it comes with insanely capable models like merve/sam2-66ac9deac6fca3bc5482fe30, microsoft/kosmos-2.5, and more ๐ซก
I built a notebook you can run with free Colab T4 to walk through the API for new models ๐๐ปโโ๏ธ merve/smol-vision
fine-tuning will follow-up soon!
it comes with insanely capable models like merve/sam2-66ac9deac6fca3bc5482fe30, microsoft/kosmos-2.5, and more ๐ซก
I built a notebook you can run with free Colab T4 to walk through the API for new models ๐๐ปโโ๏ธ merve/smol-vision
fine-tuning will follow-up soon!
Post
6311
large AI labs have dropped so many open models last week ๐ฅ don't miss out on them
โ Apple released on-device vision LMs apple/fastvlm-68ac97b9cd5cacefdd04872e & apple/mobileclip2-68ac947dcb035c54bcd20c47
โ OpenGVLab released InternVL3.5, 32 new vision LMs with one based on gpt-oss! (OS) OpenGVLab/internvl35-68ac87bd52ebe953485927fb
โ MSFT released a killer small TTS model (OS) microsoft/VibeVoice-1.5B
find more herehttps://huggingface.co/collections/merve/august-29-releases-68b5a3754cfb8abf59e2b486
โ Apple released on-device vision LMs apple/fastvlm-68ac97b9cd5cacefdd04872e & apple/mobileclip2-68ac947dcb035c54bcd20c47
โ OpenGVLab released InternVL3.5, 32 new vision LMs with one based on gpt-oss! (OS) OpenGVLab/internvl35-68ac87bd52ebe953485927fb
โ MSFT released a killer small TTS model (OS) microsoft/VibeVoice-1.5B
find more herehttps://huggingface.co/collections/merve/august-29-releases-68b5a3754cfb8abf59e2b486
- 1 reply
Post
6097
first vision language model built off openai/gpt-oss-20b just dropped! ๐ฅ
InternVL3.5 comes with 32 models ๐คฏ pre-trained, fine-tuned, aligned in various sizes OpenGVLab/internvl35-68ac87bd52ebe953485927fb
comes with gpt-oss or Qwen3 for LLM part โคต๏ธ
InternVL3.5 comes with 32 models ๐คฏ pre-trained, fine-tuned, aligned in various sizes OpenGVLab/internvl35-68ac87bd52ebe953485927fb
comes with gpt-oss or Qwen3 for LLM part โคต๏ธ
- 1 reply
Post
3342
GPT-4.1-mini level model right in your iPhone ๐คฏ
openbmb/MiniCPM-V-4 is only 4B while surpassing GPT-4.1-mini in vision benchmarks ๐ฅ
allows commercial use as well!
openbmb/MiniCPM-V-4 is only 4B while surpassing GPT-4.1-mini in vision benchmarks ๐ฅ
allows commercial use as well!