# AutoModelForCausalLM and transformers.pipeline

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautomodelforcausallm-and-transformers-pipeline%2F104343%3E%5C
**Category:** Beginners\
**Created:** [August 28, 2024, 9:38am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautomodelforcausallm-and-transformers-pipeline%2F104343 "2024-08-28T09:38:55Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![alice86](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@alice86](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Falice86)%5C
**Post date:** [August 28, 2024, 9:38am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautomodelforcausallm-and-transformers-pipeline%2F104343%2F1 "2024-08-28T09:38:55Z")

</div>

what is the different?  
which method is good?

> **[meta-llama/Meta-Llama-3-8B · Hugging Face](https://huggingface.co/meta-llama/Meta-Llama-3-8B#use-with-transformers)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

```auto
pipeline = transformers.pipeline(
    "text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto"

```

and

> <https://github.com/PromtEngineer/localGPT/blob/a1dea3becb8b1ae28a87369b1636c4c4a4501c27/load_models.py#L115>

```auto
        model = AutoModelForCausalLM.from_pretrained(model_id,
                                            # quantization_config=quantization_config,
                                            # low_cpu_mem_usage=True,
                                            # torch_dtype="auto",
                                             torch_dtype=torch.bfloat16,
                                             device_map="auto",
                                             cache_dir="./models/")

        tokenizer = AutoTokenizer.from_pretrained(model_id, cache_dir="./models/")

    pipe = pipeline(
        "text-generation",
        model=model,
        tokenizer=tokenizer,
        max_length=MAX_NEW_TOKENS,
        temperature=0.2,
        # top_p=0.95,
        repetition_penalty=1.15,
        generation_config=generation_config,
    )

    local_llm = HuggingFacePipeline(pipeline=pipe)

```

---

<div class="post-metadata">

**Author:** ![mahmutc](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mahmutc/32/64626_2.png) [@mahmutc](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmahmutc)%5C
**Post date:** [August 28, 2024, 10:23am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautomodelforcausallm-and-transformers-pipeline%2F104343%2F2 "2024-08-28T10:23:10Z")

</div>

hi @alice86  
it’s concerned with degrees of abstraction. There is no good or bad. It’s about being easy 🙂

Pipeline takes care of some of the details under the hood for you. It’s better to stick with the simple one until you reach the limits.

> **[Behind the pipeline - Hugging Face NLP Course](https://huggingface.co/learn/nlp-course/chapter2/2)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

---

<div class="post-metadata">

**Author:** ![alice86](https://avatars.discourse-cdn.com/v4/letter/a/c68b51/32.png) [@alice86](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Falice86)%5C
**Post date:** [August 29, 2024, 7:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautomodelforcausallm-and-transformers-pipeline%2F104343%2F3 "2024-08-29T07:37:49Z")

</div>

I already download the model and directly use the path.

```auto
model_id = "./Llama3"

pipeline = transformers.pipeline(
    "text-generation", model=model_id, model_kwargs={"torch_dtype": torch.bfloat16}, device_map="auto"
)

```

It show the error

```auto
ValueError: You are trying to offload the whole model to the disk. Please use the `disk_offload` function instead.

```

Q1.  
What is the define for `You are trying to offload the whole model to the disk`  
If I use the `model_id=meta-llama/Meta-Llama-3-8B` .  
It will automatically download the folder models–meta-llama–Meta-Llama-3-8B on ./cache

So the download model is also the case that offload the whole model to the disk ?

Q2. Because I saw the offload the whole model.  
what is the onlineload the model code