# GPU Google Colab not working with langchain

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpu-google-colab-not-working-with-langchain%2F73487%3E%5C
**Category:** Models\
**Created:** [February 16, 2024, 2:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpu-google-colab-not-working-with-langchain%2F73487 "2024-02-16T14:57:18Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Christian0009](https://avatars.discourse-cdn.com/v4/letter/c/5e9695/32.png) [@Christian0009](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FChristian0009)%5C
**Post date:** [February 16, 2024, 2:57pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fgpu-google-colab-not-working-with-langchain%2F73487%2F1 "2024-02-16T14:57:18Z")

</div>

I am trying to peform Retrieval Augmented Generation with langchain and models on HF. It works but it is very slow as GPU seems not to work.

I checked the following:

- GPU on Google Colab is on (I chose T4)
- Some links ask to specify device or n\_gpu\_layers as arguments, but it doesn’t work (see code below). - Specifically for the device argument, a numerical id was mentioned, I tried 0 (as cuda:0 is retrieved)  
the cuda version of ctransformers is installed through !pip install ctransformers[cuda]
- I also tried the “model.to(device)” but it seems not to be possible with my configuration.

Here’s the code but I also tried variations as mentioned above:

> ```
> #Test Llama2
> llm = CTransformers(device="cuda:0",n_gpu_layers = 110,
> n_batch = 512,model="TheBloke/Llama-2-7B-Chat-GGUF", model_type="llama",config={"context_lengrecth" :
> 
> 4096,"max_new_tokens": 1024})
> 
> rag_pipeline = RetrievalQA.from_chain_type(
> llm=llm, chain_type='stuff',
> retriever=retriever
> )
> 
> import time
> start_time = time.time()
> output=rag_pipeline.invoke("Is a license compulsory for triathlon ?")
> print("--- %s seconds ---" % (time.time() - start_time))
> 
> print(output)
> 
> ```