# Will LFS related functionality come to hf\_api?
**URL:**
**Author:**  [@larryvrh](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flarryvrh)%5C
**Post date:** [March 21, 2025, 1:35am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwill-lfs-related-functionality-come-to-hf-api%2F146721%2F1 "2025-03-21T01:35:31Z")
Currently we can only access the LFS list/delete functionality through the web interface, which is very inconvenient to manage in cases where I need to upload and delete frequently.
Are there any plans to add these LFS management capabilities to the Hugging Face Python API (hf\_api)? This would be extremely helpful for users who need to programmatically manage large file storage.
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [March 21, 2025, 6:38am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwill-lfs-related-functionality-come-to-hf-api%2F146721%2F2 "2025-03-21T06:38:20Z")
I think it would be faster to ask the developer.š
@Wauplin
> **[huggingface/huggingface\_hub](https://github.com/huggingface/huggingface_hub/issues)**
>
> The official Python client for the Huggingface Hub. - huggingface/huggingface\_hub
---
**Author:**  [@Wauplin](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FWauplin)%5C
**Post date:** [March 21, 2025, 7:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwill-lfs-related-functionality-come-to-hf-api%2F146721%2F3 "2025-03-21T07:31:40Z")
Thanks for the ping š
@larryvrh what are you exactly trying to achieve? For context, the `upload_file`/`upload_folder`/`create_commit` methods already work correctly with LFS files (i.e. if file is too large or matches gitattributes rules, it will automatically be uploaded as an LFS pointer). Also you can use `list_repo_tree` to list files from the repo with their LFS status (i.e. is the file LFS or not, and if yes what is the pointer file). Finally you can also delete files from the repo using `delete_file`/`create_commit`, which works seamlessly for both regular and LFS files.
In general, the LFS protocol is kinda hidden to the end user when dealing with the `HfApi` client. HTTP requests are made to seamlessly work with any type or size of files. Here is a short explanation about it: [Git vs HTTP paradigm](https://huggingface.co/docs/huggingface_hub/concepts/git_vs_http).
Let me know if you have any precise question regarding LFS support in `HfApi` š¤
---
**Author:**  [@John6666](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJohn6666)%5C
**Post date:** [March 21, 2025, 7:35am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwill-lfs-related-functionality-come-to-hf-api%2F146721%2F4 "2025-03-21T07:35:39Z")
Thanks Wauplin!
---
**Author:**  [@larryvrh](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flarryvrh)%5C
**Post date:** [March 22, 2025, 1:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwill-lfs-related-functionality-come-to-hf-api%2F146721%2F5 "2025-03-22T01:26:29Z")
Hi, Wauplin, thanks for replying! My problem is that the LFS storage wonāt release properly even after we use the high level API to delete files. For example, I currently store my different checkpoints in different branches of a repo, each created from the initial revision:
```python
huggingface_hub.create_branch(repo_id=repo_id,
repo_type=repo_type,
branch=branch,
revision=huggingface_hub.list_repo_commits(repo_id=repo_id, repo_type=repo_type, token=token)[-1].commit_id,
token=token,
exist_ok=False)
```
However, when I want to delete some of the branches with the following code:
```python
api.delete_files(repo_id=repo_id, revision=branch, delete_patterns='*')
api.super_squash_history(repo_id=repo_id, branch=branch)
api.delete_branch(repo_id=repo_id, branch=branch)
```
The branch and files get successfully deleted, and Iām sure that those files arenāt referenced from any other branch, but the LFS storage wonāt always release. Iāve observed that there are sometimes delayed releases, but most times it just wonāt be released at all.
---
**Author:**  [@Wauplin](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FWauplin)%5C
**Post date:** [March 22, 2025, 8:23am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwill-lfs-related-functionality-come-to-hf-api%2F146721%2F6 "2025-03-22T08:23:40Z")
Ok so if I understand it correctly, what you try to achieve is to delete the actual files that are stored on S3 but it does not do it when you delete all the commits with a pointer to the said files, am I right? Untracked LFS files are indeed garbage collected from time to time but not instant and not guaranteed. Can you tell us more why this is a problem on your side and how did you come to realize that some files are garbage collected and others not? Iād like to better understand your needs in order to help you in the good direction.
---
**Author:**  [@larryvrh](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flarryvrh)%5C
**Post date:** [March 22, 2025, 3:44pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwill-lfs-related-functionality-come-to-hf-api%2F146721%2F7 "2025-03-22T15:44:38Z")
Yes, this issue centers on S3 storage management. I can monitor which files are being garbage collected by checking the āStorage Usageā section in each repositoryās settings page. The problem arises because private storage is now a paid service. While Iām comfortable with paying, I frequently upload and delete temporary checkpoints to Hugging Face, causing my storage usage to increase indefinitely since I lack an effective method to clean up the accumulated storage.
---
**Author:**  [@Wauplin](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FWauplin)%5C
**Post date:** [March 24, 2025, 9:38am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwill-lfs-related-functionality-come-to-hf-api%2F146721%2F8 "2025-03-24T09:38:08Z")
Right, I havenāt spot this issue indeed. Iāll ask around internally what can be done in this case. Note that repositories on the Hub are meant to version data and keep the history. And `super_squash_commit` meant to be a power-user method to reduce the number of commits but not thought it term of ādeleting previously uploaded dataā. If you do not need versioning (i.e. if you do not need past checkpoints to be stored) I can advice to store checkpoints in a temporary repository and then delete it once the āfinal checkpointsā are ready. Instead of the
```py
api.delete_files(repo_id=repo_id, revision=branch, delete_patterns='*')
api.super_squash_history(repo_id=repo_id, branch=branch)
api.delete_branch(repo_id=repo_id, branch=branch)
```
you could even do something like
```py
api.delete_repo(repo_id=repo_id)
api.create_repo(repo_id=repo_id)
api.upload_file(...)
```
Of course this would come with some drawbacks (total history is lost, community tab is lost, link to collections is lost etc.) but depending on your use case and workflow it can be a good workaround.
---
**Author:**  [@Wauplin](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FWauplin)%5C
**Post date:** [March 25, 2025, 10:10am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwill-lfs-related-functionality-come-to-hf-api%2F146721%2F9 "2025-03-25T10:10:30Z")
To complete on my answer above, here is some documentation about how to free-up space: [Storage limits](https://huggingface.co/docs/hub/storage-limits#how-can-i-free-up-storage-space-in-my-accountorganization). There is a UI in the repo settings to manually delete some LFS files.
We will also add support for this method in the Python client in the near future.
---
**Author:**  [@Wauplin](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FWauplin)%5C
**Post date:** [March 25, 2025, 12:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwill-lfs-related-functionality-come-to-hf-api%2F146721%2F10 "2025-03-25T12:08:01Z")
PR: [Support permanently deleting LFS files by Wauplin Ā· Pull Request #2954 Ā· huggingface/huggingface\_hub Ā· GitHub](https://github.com/huggingface/huggingface_hub/pull/2954). Expect it to land in next huggingface\_hub release.
---
**Author:**  [@larryvrh](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flarryvrh)%5C
**Post date:** [March 25, 2025, 10:27pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwill-lfs-related-functionality-come-to-hf-api%2F146721%2F11 "2025-03-25T22:27:02Z")
Got it, thanks a lot for helping! š š
---
**Author:**  [@system](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsystem)%5C
**Post date:** [March 26, 2025, 10:27am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fwill-lfs-related-functionality-come-to-hf-api%2F146721%2F12 "2025-03-26T10:27:29Z")
Closed on March 26, 2025, 10:27am UTC
This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.