Hugging Face's logo

qualcomm
/
MobileSam

Image Segmentation
PyTorch
foundation
android
Model card Files
xet
 Community
MobileSam / README.md
qaihm-bot's picture
qaihm-bot
v0.53.0
97608c3 verified
preview code
|
raw
history blame contribute delete
10.4 kB
metadata 
library_name: pytorch
license: other
tags:
  - foundation
  - android
pipeline_tag: image-segmentation

MobileSam: Optimized for Qualcomm Devices

Transformer based encoder-decoder where prompts specify what to segment in an image thereby allowing segmentation without the need for additional training. The image encoder generates embeddings and the lightweight decoder operates on the embeddings for point and mask based image segmentation.

This is based on the implementation of MobileSam found here. This repository contains pre-exported model files optimized for Qualcomm® devices. You can use the Qualcomm® AI Hub Models library to export with custom configurations. More details on model performance across various devices, can be found here.

Qualcomm AI Hub Models uses Qualcomm AI Hub Workbench to compile, profile, and evaluate this model. Sign up to run these models on a hosted Qualcomm® device.

Getting Started

There are two ways to deploy this model on your device:

Option 1: Download Pre-Exported Models

Below are pre-exported model assets ready for deployment.

Runtime Precision Chipset SDK Versions Download
ONNX float Universal QAIRT 2.42, ONNX Runtime 1.24.3 Download
QNN_DLC float Universal QAIRT 2.45 Download
TFLITE float Universal QAIRT 2.45 Download

For more device-specific assets and performance metrics, visit MobileSam on Qualcomm® AI Hub.

Option 2: Export with Custom Configurations

Use the Qualcomm® AI Hub Models Python library to compile and export the model with your own:

  • Custom weights (e.g., fine-tuned checkpoints)
  • Custom input shapes
  • Target device and runtime configurations

This option is ideal if you need to customize the model beyond the default configuration provided here.

See our repository for MobileSam on GitHub for usage instructions.

Model Details

Model Type: Model_use_case.semantic_segmentation

Model Stats:

  • Model checkpoint: vit_t
  • Input resolution: 720p (720x1280)
  • Number of parameters (sam_encoder): 6.95M
  • Model size (sam_encoder) (float): 26.6 MB
  • Number of parameters (sam_decoder): 6.16M
  • Model size (sam_decoder) (float): 23.7 MB

Performance Summary

Model Runtime Precision Chipset Inference Time (ms) Peak Memory Range (MB) Primary Compute Unit
decoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 2.622 ms 0 - 244 MB NPU
decoder ONNX float Snapdragon® 8 Elite Mobile 3.134 ms 0 - 234 MB NPU
decoder ONNX float Snapdragon® X2 Elite 2.686 ms 11 - 11 MB NPU
decoder ONNX float Snapdragon® X Elite 6.359 ms 11 - 11 MB NPU
decoder ONNX float Snapdragon® X Elite 6.359 ms 11 - 11 MB NPU
decoder ONNX float Snapdragon® 8 Gen 3 Mobile 4.146 ms 0 - 263 MB NPU
decoder ONNX float Qualcomm® QCS8550 (Proxy) 5.972 ms 0 - 15 MB NPU
decoder ONNX float Qualcomm® QCS9075 7.345 ms 4 - 7 MB NPU
decoder ONNX float Snapdragon® 8 Elite For Galaxy Mobile 3.134 ms 0 - 234 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 2.536 ms 4 - 194 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite Mobile 2.861 ms 2 - 192 MB NPU
decoder QNN_DLC float Snapdragon® X2 Elite 2.978 ms 4 - 4 MB NPU
decoder QNN_DLC float Snapdragon® X Elite 5.763 ms 4 - 4 MB NPU
decoder QNN_DLC float Snapdragon® X Elite 5.763 ms 4 - 4 MB NPU
decoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 3.709 ms 4 - 225 MB NPU
decoder QNN_DLC float Qualcomm® QCS8275 (Proxy) 12.61 ms 0 - 192 MB NPU
decoder QNN_DLC float Qualcomm® QCS8550 (Proxy) 5.336 ms 4 - 7 MB NPU
decoder QNN_DLC float Qualcomm® SA8775P 6.222 ms 0 - 192 MB NPU
decoder QNN_DLC float Qualcomm® SA8775P 6.222 ms 0 - 192 MB NPU
decoder QNN_DLC float Qualcomm® SA8775P 6.222 ms 0 - 192 MB NPU
decoder QNN_DLC float Qualcomm® QCS9075 6.383 ms 4 - 10 MB NPU
decoder QNN_DLC float Qualcomm® QCS8450 (Proxy) 11.638 ms 0 - 222 MB NPU
decoder QNN_DLC float Qualcomm® SA7255P 12.61 ms 0 - 192 MB NPU
decoder QNN_DLC float Qualcomm® SA8295P 7.713 ms 0 - 195 MB NPU
decoder QNN_DLC float Snapdragon® 8 Elite For Galaxy Mobile 2.861 ms 2 - 192 MB NPU
decoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 2.145 ms 0 - 192 MB NPU
decoder TFLITE float Snapdragon® 8 Elite Mobile 2.592 ms 0 - 189 MB NPU
decoder TFLITE float Snapdragon® 8 Gen 3 Mobile 3.41 ms 0 - 210 MB NPU
decoder TFLITE float Qualcomm® QCS8275 (Proxy) 11.877 ms 0 - 185 MB NPU
decoder TFLITE float Qualcomm® QCS8550 (Proxy) 4.967 ms 0 - 8 MB NPU
decoder TFLITE float Qualcomm® SA8775P 5.93 ms 0 - 187 MB NPU
decoder TFLITE float Qualcomm® SA8775P 5.93 ms 0 - 187 MB NPU
decoder TFLITE float Qualcomm® SA8775P 5.93 ms 0 - 187 MB NPU
decoder TFLITE float Qualcomm® QCS9075 5.954 ms 0 - 19 MB NPU
decoder TFLITE float Qualcomm® QCS8450 (Proxy) 8.045 ms 0 - 217 MB NPU
decoder TFLITE float Qualcomm® SA7255P 11.877 ms 0 - 185 MB NPU
decoder TFLITE float Qualcomm® SA8295P 7.262 ms 0 - 183 MB NPU
decoder TFLITE float Snapdragon® 8 Elite For Galaxy Mobile 2.592 ms 0 - 189 MB NPU
encoder ONNX float Snapdragon® 8 Elite Gen 5 Mobile 80.351 ms 20 - 1519 MB NPU
encoder ONNX float Snapdragon® 8 Elite Mobile 96.875 ms 8 - 1394 MB NPU
encoder ONNX float Snapdragon® X2 Elite 101.697 ms 14 - 14 MB NPU
encoder ONNX float Snapdragon® X Elite 167.909 ms 12 - 12 MB NPU
encoder ONNX float Snapdragon® X Elite 167.909 ms 12 - 12 MB NPU
encoder ONNX float Snapdragon® 8 Gen 3 Mobile 120.888 ms 20 - 3034 MB NPU
encoder ONNX float Qualcomm® QCS8550 (Proxy) 171.47 ms 12 - 32 MB NPU
encoder ONNX float Qualcomm® QCS9075 198.958 ms 12 - 15 MB NPU
encoder ONNX float Snapdragon® 8 Elite For Galaxy Mobile 96.875 ms 8 - 1394 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite Gen 5 Mobile 40.978 ms 8 - 1241 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite Mobile 49.148 ms 12 - 1129 MB NPU
encoder QNN_DLC float Snapdragon® X2 Elite 45.709 ms 12 - 12 MB NPU
encoder QNN_DLC float Snapdragon® X Elite 102.723 ms 12 - 12 MB NPU
encoder QNN_DLC float Snapdragon® X Elite 102.723 ms 12 - 12 MB NPU
encoder QNN_DLC float Snapdragon® 8 Gen 3 Mobile 65.661 ms 0 - 1320 MB NPU
encoder QNN_DLC float Qualcomm® QCS8275 (Proxy) 207.438 ms 1 - 1126 MB NPU
encoder QNN_DLC float Qualcomm® QCS8550 (Proxy) 99.672 ms 12 - 15 MB NPU
encoder QNN_DLC float Qualcomm® SA8775P 100.295 ms 0 - 1549 MB NPU
encoder QNN_DLC float Qualcomm® SA8775P 100.295 ms 0 - 1549 MB NPU
encoder QNN_DLC float Qualcomm® SA8775P 100.295 ms 0 - 1549 MB NPU
encoder QNN_DLC float Qualcomm® QCS9075 120.455 ms 12 - 30 MB NPU
encoder QNN_DLC float Qualcomm® QCS8450 (Proxy) 328.079 ms 12 - 1342 MB NPU
encoder QNN_DLC float Qualcomm® SA7255P 207.438 ms 1 - 1126 MB NPU
encoder QNN_DLC float Qualcomm® SA8295P 325.329 ms 1 - 1557 MB NPU
encoder QNN_DLC float Snapdragon® 8 Elite For Galaxy Mobile 49.148 ms 12 - 1129 MB NPU
encoder TFLITE float Snapdragon® 8 Elite Gen 5 Mobile 109.604 ms 32 - 1086 MB NPU
encoder TFLITE float Snapdragon® 8 Elite Mobile 129.101 ms 25 - 1105 MB NPU
encoder TFLITE float Snapdragon® 8 Gen 3 Mobile 166.461 ms 33 - 2313 MB NPU
encoder TFLITE float Qualcomm® QCS8275 (Proxy) 377.475 ms 33 - 1120 MB NPU
encoder TFLITE float Qualcomm® QCS8550 (Proxy) 218.803 ms 38 - 42 MB NPU
encoder TFLITE float Qualcomm® SA8775P 242.262 ms 34 - 1115 MB NPU
encoder TFLITE float Qualcomm® SA8775P 242.262 ms 34 - 1115 MB NPU
encoder TFLITE float Qualcomm® SA8775P 242.262 ms 34 - 1115 MB NPU
encoder TFLITE float Qualcomm® QCS9075 299.389 ms 32 - 109 MB NPU
encoder TFLITE float Qualcomm® QCS8450 (Proxy) 537.18 ms 0 - 1926 MB NPU
encoder TFLITE float Qualcomm® SA7255P 377.475 ms 33 - 1120 MB NPU
encoder TFLITE float Qualcomm® SA8295P 372.998 ms 22 - 1145 MB NPU
encoder TFLITE float Snapdragon® 8 Elite For Galaxy Mobile 129.101 ms 25 - 1105 MB NPU

License

  • The license for the original implementation of MobileSam can be found here.

References

Community