Instructions to use tencent/POINTS-Reader with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use tencent/POINTS-Reader with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="tencent/POINTS-Reader", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("tencent/POINTS-Reader", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use tencent/POINTS-Reader with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "tencent/POINTS-Reader" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/POINTS-Reader", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/tencent/POINTS-Reader
- SGLang
How to use tencent/POINTS-Reader with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "tencent/POINTS-Reader" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/POINTS-Reader", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "tencent/POINTS-Reader" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "tencent/POINTS-Reader", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use tencent/POINTS-Reader with Docker Model Runner:
docker model run hf.co/tencent/POINTS-Reader
| import copy | |
| from typing import Any, Dict | |
| from transformers import PretrainedConfig, Qwen2Config | |
| try: | |
| from transformers.models.qwen2_vl.configuration_qwen2_vl import Qwen2VLVisionConfig | |
| except ImportError: | |
| print('Please upgrade transformers to version 4.46.3 or higher') | |
| class POINTSV15ChatConfig(PretrainedConfig): | |
| model_type = "pointsv1.5_chat" | |
| is_composition = True | |
| """Configuration class for `POINTSV1.5`.""" | |
| def __init__(self, | |
| **kwargs) -> None: | |
| super().__init__(**kwargs) | |
| vision_config = kwargs.pop("vision_config", None) | |
| llm_config = kwargs.pop("llm_config", None) | |
| if isinstance(vision_config, dict): | |
| self.vision_config = Qwen2VLVisionConfig(**vision_config) | |
| else: | |
| self.vision_config = vision_config | |
| if isinstance(llm_config, dict): | |
| self.llm_config = Qwen2Config(**llm_config) | |
| else: | |
| self.llm_config = llm_config | |