# Backend low level kernel libraries used in Transformers
**URL:**
**Author:**  [@vineel007](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvineel007)%5C
**Post date:** [October 25, 2024, 6:00am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbackend-low-level-kernel-libraries-used-in-transformers%2F113907%2F1 "2024-10-25T06:00:58Z")
Hello @ArthurZ @RaushanTurganbay ,
What are the backend libraries used for low level kernels operations(matmul, softmax etc) in transformers library?
**Issue:** If I ran same model(say mamba or llama) on x86 machine and aarch64 machine, I observe difference in the model timing. I suspect there are different paths for kernels in x86 and aarch64.
Please specify the backend libraries used for x86 and aarch64
---
**Author:**  [@RaushanTurganbay](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FRaushanTurganbay)%5C
**Post date:** [October 26, 2024, 1:38pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbackend-low-level-kernel-libraries-used-in-transformers%2F113907%2F2 "2024-10-26T13:38:49Z")
In Transformers the attention is by default [`torch.nn.sdpa()`](https://pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html) so the backend libraries would be handled Pytorch
---
**Author:**  [@vineel007](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvineel007)%5C
**Post date:** [October 26, 2024, 5:42pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbackend-low-level-kernel-libraries-used-in-transformers%2F113907%2F3 "2024-10-26T17:42:36Z")
@RaushanTurganbay,
Yes pytorch is evident. But we need to go another level down i.e what libraries do torch calls for operations? As we observe difference in timings on x86 and arm, we can find bottleneck only if we know low level kernel library
---
**Author:**  [@benstokes](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbenstokes)%5C
**Post date:** [October 27, 2024, 9:06am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fbackend-low-level-kernel-libraries-used-in-transformers%2F113907%2F4 "2024-10-27T09:06:36Z")
> [@RaushanTurganbay](#):
>
> In Transformers the attention is by default [`torch.nn.sdpa()`](https://pytorch.org/docs/stable/generated/torch.nn.functional.scaled_dot_product_attention.html) so the backend libraries would be handled Pytorch
I also face this issue. but fortunately i saw your thread. Thanks for giving solution.