# Chapter 2 questions
**URL:**
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [June 14, 2021, 2:51pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F1 "2021-06-14T14:51:48Z")
Use this topic for any question about [Chapter 2](https://huggingface.co/course/chapter2) of the course.
---
**Author:**  [@amitness](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Famitness)%5C
**Post date:** [June 15, 2021, 11:15am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F2 "2021-06-15T11:15:02Z")
1. In the **Handling multiple sequences** page of Chapter 2, there is a bug in the code under **Attention masks** section.
2. There is a typo on [https://huggingface.co/course/chapter2/6?fw=pt](https://huggingface.co/course/chapter2/6?fw=pt)
3. Isn’t **wordpiece** a subword algorithm as well?
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [June 15, 2021, 12:07pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F3 "2021-06-15T12:07:16Z")
Thanks for flagging all of this, will push a fix in the morning!
---
**Author:**  [@harish3110](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fharish3110)%5C
**Post date:** [June 16, 2021, 2:48am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F4 "2021-06-16T02:48:29Z")
- In Chapter 2, under the **Putting it all together** page, the above code snippet should include padding=True 🙂
- Under **Wrapping up: From tokenizer to model** , the last line of code snippet should be changed to `output = model(**tokens)`
Page: [Putting it all together](https://huggingface.co/course/chapter2/6?fw=pt)
@sgugger I’m absolutely loving this course! A great refresher to the library with really intuitive videos and tutorials to wade through and understand the Hugging Face Library. I honestly wished I had this resource when I started out. Can’t wait for the next part of the course! 🙂
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [June 16, 2021, 11:54am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F5 "2021-06-16T11:54:52Z")
thanks for reporting the bugs and suggested fixes @harish3110! will push a fix this afternoon 🙂
---
**Author:**  [@realjanpaulus](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Frealjanpaulus)%5C
**Post date:** [June 24, 2021, 9:20am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F6 "2021-06-24T09:20:52Z")
I found a small typo in the section [Behind the pipeline - Postprocessing the output](https://huggingface.co/course/chapter2/2?fw=pt).
- **error** : `[0.9946, 0.0544]`
- **correction** : `[0.9995, 0.0005]`
The following block gives a little bit more context (see the section **Postprocessing the output** )
```auto
tensor([[4.0195e-02, 9.5980e-01],
[9.9946e-01, 5.4418e-04]], grad_fn=)
```
_Now we can see that the model predicted `[0.0402, 0.9598]` for the first sentence and `[0.9946, 0.0544]` for the second one. These are recognizable probability scores._
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [June 24, 2021, 12:08pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F7 "2021-06-24T12:08:04Z")
Thanks for reporting, will fix this morning!
---
**Author:**  [@d4niel92](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fd4niel92)%5C
**Post date:** [June 24, 2021, 2:45pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F8 "2021-06-24T14:45:17Z")
In the **Handling multiple sequences** notebook (for Tensorflow) of Chapter 2, there is a bug in the code under **Tokenization** section.
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [June 24, 2021, 2:47pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F9 "2021-06-24T14:47:15Z")
No that is not a bug, the course explicitly says this doesn’t work and explains why.
---
**Author:**  [@d4niel92](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fd4niel92)%5C
**Post date:** [June 24, 2021, 2:48pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F10 "2021-06-24T14:48:23Z")
Okay sorry for that, have to go over it again. Thanks!
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [June 24, 2021, 2:50pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F11 "2021-06-24T14:50:53Z")
No worries, I understand why you’d be suprised. The notebooks are auto-generated, so I can’t add some comments in Markdown cells but I can add comments in the code!
---
**Author:**  [@lincht](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flincht)%5C
**Post date:** [August 8, 2021, 2:20pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F12 "2021-08-08T14:20:43Z")
In _Handling multiple sequences - Attention masks - Try it out_, there is a caveat that may be good to mention in case someone encounters the same question.
Here, by manually tokenizing the two sentences, creating attention masks, and passing them through the model, we should be able to reproduce the same logits as in section 2, which are:
```auto
```
However, if you do it this way:
```auto
import tensorflow as tf
from transformers import AutoTokenizer, TFAutoModelForSequenceClassification
checkpoint = "distilbert-base-uncased-finetuned-sst-2-english"
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = TFAutoModelForSequenceClassification.from_pretrained(checkpoint)
raw_inputs = [
"I've been waiting for a HuggingFace course my whole life.",
"I hate this so much!",
]
batched_tokens = [tokenizer.tokenize(r) for r in raw_inputs]
batched_ids = [tokenizer.convert_tokens_to_ids(t) for t in batched_tokens]
# Add padding
batched_ids = [ids + [tokenizer.pad_token_id] * (max(map(len, batched_ids))-len(ids))
for ids in batched_ids]
attention_mask = [[0 if x == tokenizer.pad_token_id else 1 for x in ids]
for ids in batched_ids]
print(model(tf.constant(batched_ids), attention_mask=tf.constant(attention_mask)).logits)
```
The results will be different from section 2:
```auto
tf.Tensor(
[[-2.7276204 2.878937]
[3.1930914 -2.668523]], shape=(2, 2), dtype=float32)
```
This is because `tokenizer.tokenize` does not add the special tokens `[CLS]` and `[SEP]` by default, whereas the high-level API `tokenizer()` does.
If we change this line in the code to:
```auto
batched_tokens = [tokenizer.tokenize(r, add_special_tokens=True) for r in raw_inputs]
```
the result is indeed the same:
```auto
tf.Tensor(
[[-1.5606964 1.6122813]
[4.169231 -3.3464475]], shape=(2, 2), dtype=float32)
```
---
**Author:**  [@fredguth](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Ffredguth)%5C
**Post date:** [October 6, 2021, 1:41pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F13 "2021-10-06T13:41:40Z")
Just following the code (for PyTorch) for the course leads to a small error:
```python
outputs = model(**inputs)
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
...
TypeError: forward() got an unexpected keyword argument 'token_type_ids'
```
I removed the problematic `token_type_ids` key and it worked:
```python
outputs = model(** { k: inputs[k] for k in ['input_ids', 'attention_mask'] })
```
But, then, the result is not what we expected:
```python
import torch
from transformers import AutoModelForSequenceClassification
checkpoint = "distilbert-base-uncased-finetuned-sst-2-english"
model = AutoModelForSequenceClassification.from_pretrained(checkpoint)
raw_inputs = [
"I've been waiting for a HuggingFace course my whole life.",
"I hate this so much!",
]
inputs = tokenizer(raw_inputs, padding=True, truncation=True, return_tensors="pt")
outputs = model(**{ k: inputs[k] for k in ['input_ids', 'attention_mask'] })
predictions = torch.nn.functional.softmax(outputs.logits, dim=-1)
print(predictions)
-----
tensor([[0.9618, 0.0382],
[0.9350, 0.0650]], grad_fn=)
```
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [October 6, 2021, 6:33pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F14 "2021-10-06T18:33:20Z")
Are you sure you are using the right tokenizer? It doesn’t seem so since you have those `token_type_ids` added.
---
**Author:**  [@StoriedPerson](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FStoriedPerson)%5C
**Post date:** [September 21, 2022, 7:14am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F16 "2022-09-21T07:14:44Z")
There’s a typo in this illustration:
---
**Author:**  [@makram](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmakram)%5C
**Post date:** [October 16, 2022, 12:50am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F17 "2022-10-16T00:50:50Z")
I want to use behind the pipeline as a way to use batches of dataset, is there a way? and is it possible to apply batches using pipeline as we can specify batch size and give the model a dataset instead of examples?
---
**Author:**  [@pallavi176](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fpallavi176)%5C
**Post date:** [October 26, 2022, 2:06pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F19 "2022-10-26T14:06:25Z")
from transformers import TFAutoModel
checkpoint = “distilbert-base-uncased-finetuned-sst-2-english”
model = TFAutoModel.from\_pretrained(checkpoint)
outputs = model(inputs)
print(outputs.last\_hidden\_state.shape)
The output generated from above step is not being used in the next step.
from transformers import TFAutoModelForSequenceClassification
checkpoint = “distilbert-base-uncased-finetuned-sst-2-english”
model = TFAutoModelForSequenceClassification.from\_pretrained(checkpoint)
outputs = model(inputs)
print(outputs.logits.shape)
Then why are we generating output from TFAutoModel
---
**Author:**  [@vsrinivas](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fvsrinivas)%5C
**Post date:** [November 2, 2022, 8:07am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F20 "2022-11-02T08:07:20Z")
Hi, I regret to mention that I am finding it very difficult to follow Mr.Sylvia’s pronunciation. The subtitles seem to be only in French; if they are made available in English too, it will be easier to follow/ understand.
---
**Author:**  [@anon92148384](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fanon92148384)%5C
**Post date:** [November 15, 2022, 4:56am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F21 "2022-11-15T04:56:38Z")
Hi Sylvain,
There is a typo in the mask output in _ **Preprocessing with a tokenizer** _, _ **Behind the pipeline** _.
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [November 15, 2022, 2:03pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799%2F22 "2022-11-15T14:03:52Z")
You should make a PR with your fix!
[Next page](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-2-questions%2F6799.md%3Fpage%3D2)