# Does the REST API work with private repo?
**URL:**
**Author:**  [@sl02](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsl02)%5C
**Post date:** [January 5, 2023, 12:09pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-the-rest-api-work-with-private-repo%2F28987%2F1 "2023-01-05T12:09:54Z")
I was experimenting with the REST API with a private repo. Despite providing the user access token in the request header, I receive an error
```auto
import requests
from dotenv import load_dotenv
load_dotenv()
per_token = os.getenv('API_PER_TOKEN')
headers = {"Authorization": f"Bearer {per_token}"}
API_URL = "/static-proxy?url=https%3A%2F%2Fdatasets-server.huggingface.co%2Fis-valid%3Fdataset%3Dsl02%2Fnp-datasets"
def query():
response = requests.request("GET", API_URL, headers=headers)
return response.json()
data = query()
```
`{'error': 'The dataset does not exist, or is not accessible without authentication (private or gated). Please retry with authentication.'}`
However, when I make the repository public, it returns `{'valid': True}`. But, when I run the `first-rows` API, I get the following message
```auto
import requests
from dotenv import load_dotenv
load_dotenv()
per_token = os.getenv('API_PER_TOKEN')
headers = {"Authorization": f"Bearer {per_token}"}
API_URL = "/static-proxy?url=https%3A%2F%2Fdatasets-server.huggingface.co%2Ffirst-rows%3Fdataset%3Dsl02%2Fnp-datasets%26config%3Ddefault%26split%3Dtrain"
def query():
response = requests.request("GET", API_URL)
return response.json()
data = query()
```
`{'error': 'The response is not ready yet. Please retry later.'}`
The `load_dataset()` works in private mode when I set the `use_auth_token` argument. Any clue what I missing here?
---
**Author:**  [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [January 5, 2023, 4:22pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-the-rest-api-work-with-private-repo%2F28987%2F2 "2023-01-05T16:22:53Z")
Maybe @severo knows more, but IIRC the REST API is not available yet for private repos.
---
**Author:**  [@severo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsevero)%5C
**Post date:** [January 5, 2023, 4:28pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-the-rest-api-work-with-private-repo%2F28987%2F3 "2023-01-05T16:28:07Z")
Hi @sl02. The REST API uses the same rule as the dataset viewer (see [The Dataset Preview has been disabled on this dataset - #6 by severo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fthe-dataset-preview-has-been-disabled-on-this-dataset%2F21339%2F6))%3A it’s not available at all for the private datasets for now.
re “The response is not ready yet. Please retry later”: the responses to the API endpoints are pre-computed asynchronously and can take some time to be processed, depending on the dataset itself and on the load of the servers.
---
**Author:**  [@ymoslem](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fymoslem)%5C
**Post date:** [February 27, 2025, 5:18am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-the-rest-api-work-with-private-repo%2F28987%2F4 "2025-02-27T05:18:09Z")
Hello! The dataset preview is now available for the **Pro** accounts. Should not it be the case for the API? I cannot do something as simple as retrieving the URLs. Thanks!
```auto
headers = {"Authorization": f"Bearer {API_TOKEN}"}
reseponse = requests.get(f"/static-proxy?url=https%3A%2F%2Fdatasets-server.huggingface.co%2Fparquet%3Fdataset%3D%7Bdataset_name%7D")
json_data = reseponse.json()
urls = [f['url'] for f in json_data['parquet_files'] if f['split'] == 'test']
```
## Update
So now this works:
```auto
from datasets import load_dataset
import requests
headers = {"Authorization": f"Bearer {API_TOKEN}"}
API_URL = f"https://huggingface.co/api/datasets/{dataset_name}/parquet"
def query():
response = requests.get(API_URL, headers=headers)
json_data = response.json()["default"]
return json_data
urls = query()
print(urls)
```
However, if we try to download the retrieved URL, it does not work `FileNotFoundError`
```auto
test_dataset = load_dataset("parquet",
data_files={"test": urls["test"]},
split="test",
token=API_TOKEN
)
```
The only solution I found so far, is to manually download the retrieved URLs, something like:
```auto
# Manually download the files
import shutil
from tqdm.auto import tqdm
parquet_files = []
for n, url in tqdm(enumerate(urls["test"]), total=len(urls["test"])):
response = requests.get(url, headers=headers, stream=True)
with open(f"{n}.parquet", "wb") as f:
shutil.copyfileobj(response.raw, f)
parquet_files.append(f"{n}.parquet")
# Load dataset
test_dataset = load_dataset("parquet", data_files=parquet_files)
print(test_dataset)
```
---
**Author:**  [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [March 5, 2025, 2:39pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-the-rest-api-work-with-private-repo%2F28987%2F5 "2025-03-05T14:39:59Z")
Hi ! you can load the parquet files from the repo directly:
```python
load_dataset(dataset_name, revision="refs/convert/parquet")
```
and if you want to load specific files you can pass `data_files=[...]` (btw it accepts glob patterns)
---
**Author:**  [@ymoslem](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fymoslem)%5C
**Post date:** [March 10, 2025, 7:18am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-the-rest-api-work-with-private-repo%2F28987%2F6 "2025-03-10T07:18:58Z")
Thanks! I still receive `FileNotFoundError`. The issue, as in the original post, is that the repository is _private_. It is my repository, and I am logged in with an access token.
---
**Author:**  [@lhoestq](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flhoestq)%5C
**Post date:** [March 11, 2025, 3:20pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdoes-the-rest-api-work-with-private-repo%2F28987%2F7 "2025-03-11T15:20:02Z")
Can you check that your token has the right permissions ? I just tried on my side and I couldn’t reproduce the `FileNotFoundError` on a the parquet branch of a private repo with a token