# Chapter 1 questions
**URL:**
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [June 14, 2021, 2:50pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F1 "2021-06-14T14:50:50Z")
Use this topic for any question about [Chapter 1](https://huggingface.co/course/chapter1) of the course.
---
**Author:**  [@severo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsevero)%5C
**Post date:** [June 16, 2021, 9:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F2 "2021-06-16T09:46:49Z")
Small mismatch?
From the widget in [roberta-large-mnli · Hugging Face](https://huggingface.co/roberta-large-mnli?text=I+like+you.+%3C%2Fs%3E%3C%2Fs%3E+I+love+you.), I see the classification is between “CONTRADICTION”, “NEUTRAL”, “ENTAILMENT”
---
**Author:**  [@severo](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsevero)%5C
**Post date:** [June 16, 2021, 9:47am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F3 "2021-06-16T09:47:25Z")
while the quizz mentions “entailment”, “hypothesis”, “neutral”
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [June 16, 2021, 11:44am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F4 "2021-06-16T11:44:24Z")
good catch! will push a fix this afternoon 🙂
---
**Author:**  [@jon-fernandes](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjon-fernandes)%5C
**Post date:** [June 17, 2021, 5:09am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F5 "2021-06-17T05:09:14Z")
At the end of the colab notebook, you need to install sentencepiece to complete the translation task.
!pip install sentencepiece
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [June 17, 2021, 12:35pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F6 "2021-06-17T12:35:48Z")
thanks for reporting this! i’ll push a fix this afternoon
---
**Author:**  [@dk-crazydiv](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdk-crazydiv)%5C
**Post date:** [June 17, 2021, 7:23pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F7 "2021-06-17T19:23:44Z")
Hi. I wanted to understand a bit deeper on difference between encoder vs encoder-decoder models for question-answering. ModelHub currently has both BERT and T5 models for question answering. The [encoder lecture](https://youtu.be/MUqNwgPjJvQ?t=145) mentions BERT is good for QA tasks. Could someone please point me with more reading material to understand the nuances/differences between the two.
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [June 17, 2021, 7:33pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F8 "2021-06-17T19:33:44Z")
Hi there! So Ecoder-decoders models will be very good at “generative question answering”, which is generating the answer to the question given the context. It’s different from what an encoder can do (which is “extractive question answering”, which just says the answer to this question is from word xxx to word yyy) in the sense the model answers the question in its own words instead of trying to get a part of the context.
---
**Author:**  [@dk-crazydiv](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fdk-crazydiv)%5C
**Post date:** [June 17, 2021, 8:11pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F9 "2021-06-17T20:11:53Z")
Understood. Thanks for the explanation.
---
**Author:**  [@ghosh-r](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fghosh-r)%5C
**Post date:** [June 18, 2021, 9:49am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F10 "2021-06-18T09:49:56Z")
[I cannot download translation models in Colab.](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fcannot-download-translation-models-in-colab%2F6952)
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [June 18, 2021, 1:21pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F11 "2021-06-18T13:21:29Z")
I replied on the topic!
---
**Author:**  [@LJay](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FLJay)%5C
**Post date:** [June 30, 2021, 12:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F12 "2021-06-30T00:48:41Z")
Beginner Beginner query:
I’m working my way through to try and use the transformers for classification, and potentially question answering, and sentence forming. Is it possible to use say of the bert models to encode the text for zero-shot classification, and use the same model to interact with responses and formulate the stored data based on interaction. I’m only up to chapter 2 so no doubt (hopefully) it will come up. But I’m looking ahead to optimise so I don’t use more than the necessary models, and encoders/decoders.
edit: as an example cause on re-read it might not be clear.
I have a bot that classifies context in a conversation chain i.e. “we are going to use bootstrap”. From this I think using zero shot and my labels it will classify it. I then would like to interact with Questions i.e. “did you want to store this” followed by creating a sentence to store the data. I’ll need more than one model I think…
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [June 30, 2021, 12:46pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F13 "2021-06-30T12:46:04Z")
hi @LJay welcome to the 🤗 forum!
your use case sounds quite tricky to solve end-to-end with a single model, so you’re right to think that you’ll need multiple components / models.
for instance, BERT models are not typically designed for text generation, so a decoder-only model like GPT-2 would be a better choice for sentence generation - although it’s worth noting that generating high-quality text is a form of art 🙂
my suggestion would be to first build a simple end-to-end pipeline based on rules (where possible) so you can get a feel for how the inputs are converted to outputs. you might find you don’t need machine learning for all the components (which is quite nice if you plan to deploy this in a production environment!)
---
**Author:**  [@bhagerty](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbhagerty)%5C
**Post date:** [March 25, 2022, 2:26am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F14 "2022-03-25T02:26:04Z")
There seems to be an editing error in the explanation for answer 1 to Question 11 in the Chapter 1 quiz. The explanation says (emphasis added):
> Correct! When applying Transfer Learning, the bias in the pretrained model used **perspires** in the fine-tuned model.
I think “perspires” should be something like “persists.” I’m pretty sure bias cannot sweat. Unless “perspire” is being used in some metaphorical way I am not familiar with (which would surprise me, as a pretty educated native English speaker), this looks like a mistake.
---
**Author:**  [@verderog](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fverderog)%5C
**Post date:** [September 24, 2022, 1:35pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F15 "2022-09-24T13:35:35Z")
I’m going through Chapter 1 using a Python virtual environment. The examples work until I get to the Text-Generation section, when I see the following:
```auto
Python 3.8.10 (default, Sep 15 2021, 10:14:58)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from transformers import pipeline
>>> generator = pipeline("text-generation")
No model was supplied, defaulted to gpt2 (https://huggingface.co/gpt2)
>>> generator("In this course, we will teach you how to")
Setting `pad_token_id` to `eos_token_id`:50256 for open-end generation.
Traceback (most recent call last):
File "", line 1, in
File "/home/user/.pyenv/versions/3.8.10/lib/python3.8/site-packages/transformers/pipelines/text_generation.py", line 175, in __call__
return super(). __call__ (text_inputs, **kwargs)
File "/home/user/.pyenv/versions/3.8.10/lib/python3.8/site-packages/transformers/pipelines/base.py", line 1026, in __call__
return self.run_single(inputs, preprocess_params, forward_params, postprocess_params)
File "/home/user/.pyenv/versions/3.8.10/lib/python3.8/site-packages/transformers/pipelines/base.py", line 1033, in run_single
model_outputs = self.forward(model_inputs, **forward_params)
File "/home/user/.pyenv/versions/3.8.10/lib/python3.8/site-packages/transformers/pipelines/base.py", line 943, in forward
model_outputs = self._forward(model_inputs, **forward_params)
File "/home/user/.pyenv/versions/3.8.10/lib/python3.8/site-packages/transformers/pipelines/text_generation.py", line 213, in _forward
generated_sequence = self.model.generate(input_ids=input_ids, **generate_kwargs) # BS x SL
File "/home/user/.pyenv/versions/3.8.10/lib/python3.8/site-packages/torch/autograd/grad_mode.py", line 28, in decorate_context
return func(*args, **kwargs)
File "/home/user/.pyenv/versions/3.8.10/lib/python3.8/site-packages/transformers/generation_utils.py", line 1310, in generate
return self.sample(
File "/home/user/.pyenv/versions/3.8.10/lib/python3.8/site-packages/transformers/generation_utils.py", line 1963, in sample
next_tokens = torch.multinomial(probs, num_samples=1).squeeze(1)
RuntimeError: Inplace update to inference tensor outside InferenceMode is not allowed.You can make a clone to get a normal tensor before doing inplace update.See https://github.com/pytorch/rfcs/pull/17 for more details.
```
Any ideas what may be causing this?
Here’s some version info from `pip freeze` for my install:
```auto
$ grep -E "transform|torch" pip-freeze.txt
pytorch-lightning==1.4.2
-e git+https://github.com/CompVis/taming-transformers.git@24268930bf1dce879235a7fddd0b2355b84d7ea6#egg=taming_transformers
torch==1.9.0+cu111
torch-fidelity==0.3.0
torchaudio==0.9.0
torchmetrics==0.6.0
torchvision==0.10.0+cu111
transformers==4.19.2
```
---
**Author:**  [@verderog](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fverderog)%5C
**Post date:** [October 5, 2022, 6:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F16 "2022-10-05T18:56:28Z")
> [@verderog](#):
>
> `generator("In this course, we will teach you how to")`
FWIW, I solved this by upgrading torch by executing `pip install --upgrade torch torchvision torchaudio`. This bumped me from 1.9.0+cu111, to 1.12.1:
```auto
$ pip freeze | grep -E "transform|torch"
pytorch-lightning==1.4.2
-e git+https://github.com/CompVis/taming-transformers.git@24268930bf1dce879235a7fddd0b2355b84d7ea6#egg=taming_transformers
torch==1.12.1
torch-fidelity==0.3.0
torchaudio==0.12.1
torchmetrics==0.6.0
torchvision==0.13.1
transformers==4.19.2
```
---
**Author:**  [@tomjam](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftomjam)%5C
**Post date:** [October 12, 2022, 3:46am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F18 "2022-10-12T03:46:03Z")
I had a few issues on the “Introduction” page for the course with the set up listed for a python virtual environment.
“pip install transformers[sentencepiece]” did not work.
“pip install transformers sentencepiece” did work.
“source .env/bin/deactivate” did not work.
“deactivate” did work.
---
**Author:**  [@tomjam](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ftomjam)%5C
**Post date:** [October 12, 2022, 3:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F19 "2022-10-12T03:48:39Z")
I had an issue on the “Transformers are everywhere” page using a python virtual environment.
“classifier = pipeline(“sentiment-analysis”)” did not work.
“classifier = pipeline(“sentiment-analysis”, model=“distilbert-base-uncased-finetuned-sst-2-english”)” did work.
---
**Author:**  [@luojianguang](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fluojianguang)%5C
**Post date:** [November 8, 2022, 8:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F20 "2022-11-08T08:37:38Z")
[https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/transformers.svg](https://huggingface.co/datasets/huggingface-course/documentation-images/resolve/main/en/chapter1/transformers.svg)
There must be a mistake in the picture. At the encoder side, it should be “input embedding” instead of “output embedding”.
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [November 22, 2022, 8:49am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797%2F21 "2022-11-22T08:49:19Z")
Hi @tomjam thanks for reporting these issues! Just for clarity:
> [@tomjam](#):
>
> “pip install transformers[sentencepiece]” did not work.
> “pip install transformers sentencepiece” did work.
were you running this from the terminal? I think for some operating systems, one needs quotes like:
```auto
pip install 'transformers[sentencepiece]'
```
In any case, I’ll fix this in the content.
Edit: I see we already indicate one should use quotes [here](https://huggingface.co/course/chapter0/1?fw=pt#installing-dependencies). Let me know if you’re still having problems 🙂
[Next page](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-1-questions%2F6797.md%3Fpage%3D2)