# Chapter 7 questions
**URL:**
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [November 15, 2021, 2:13pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F1 "2021-11-15T14:13:26Z")
Use this topic for any question about [Chapter 7](https://huggingface.co/course/chapter7) of the course.
---
**Author:**  [@emvy03](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Femvy03)%5C
**Post date:** [November 22, 2021, 12:34pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F2 "2021-11-22T12:34:44Z")
Hi - this is a relatively simple question but i’m totally new to HuggingFace so apologies in advance but on section 3 you discuss domain adaption.
I’m just experimenting with the task at the end of the section i.e. “To quantify the benefits of domain adaptation, fine-tune a classifier on the IMDb labels for both the pretrained and fine-tuned MiniLM checkpoints…”
Can you use the ‘Fill-Mask’ domain-adapted checkpoint you generated in the course (huggingface-course/distilbert-base-uncased-finetuned-imdb) for a classification task? Or do you have to adapt the original distilbert-base-uncased to the domain specifically for classification?
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [November 22, 2021, 12:51pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F3 "2021-11-22T12:51:20Z")
No, you would need to fine-tune it on the classification task next. It’s just that the fine-tuned masked language model might do better, since it’s more specialized on your corpus.
Hope that makes sense!
---
**Author:**  [@emvy03](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Femvy03)%5C
**Post date:** [November 22, 2021, 1:15pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F4 "2021-11-22T13:15:52Z")
Thank you!! So I should just follow the standard fine-tuning methodology for classification as per Ch. 3 but use the ‘fine-tuned masked model’ as the starting checkpoint? i.e.
```auto
from transformers import TFAutoModelForSequenceClassification, AutoTokenizer
checkpoint = 'huggingface-course/distilbert-base-uncased-finetuned-imdb'
tokenizer = AutoTokenizer.from_pretrained(checkpoint)
model = TFAutoModelForSequenceClassification.from_pretrained(checkpoint, num_labels=2)
...
```
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [November 22, 2021, 2:56pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F5 "2021-11-22T14:56:29Z")
That’s correct indeed!
---
**Author:**  [@emvy03](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Femvy03)%5C
**Post date:** [November 22, 2021, 7:17pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F6 "2021-11-22T19:17:30Z")
Thank you very much!
---
**Author:**  [@ducatyb](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fducatyb)%5C
**Post date:** [December 26, 2021, 11:45pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F7 "2021-12-26T23:45:38Z")
Perhaps this is a typo? This sentence in the Question answering section
> The `answers` field is a bit trickier as it **comports** a dictionary with two fields that are both lists.
I guess here it means **comprises**
---
**Author:**  [@jon-fernandes](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjon-fernandes)%5C
**Post date:** [December 29, 2021, 11:56am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F8 "2021-12-29T11:56:01Z")
This is with regards to the translation section.
I don’t understand what is the purpose of adding a padding token at the start of the decoder\_input\_ids.
I understand that the decoder\_input\_ids is the labels shifted by one.
Does the labels have the ground truth, and when we check the next token for decoder\_input\_ids, we then compare it to the labels as part of the training?
i.e.
```auto
batch['labels'] = tensor([[83, 7471, 23, ...]])
batch['decoder_input_ids'] = tensor([[59513, 83, 7471, 23, ...]])
where 59513 is the pad token.
```
Many thanks
---
**Author:**  [@sgugger](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsgugger)%5C
**Post date:** [December 29, 2021, 3:26pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F9 "2021-12-29T15:26:24Z")
The decoder is generating an output by predicting each token one after the other with:
- the encoder hidden state form the inputs
- the previously predicted tokens of the outputs.
But for the very first token, there is no already previously predicted tokens, so we feed it a special token, which might be the pad token, or a special “beginning of stream” (bos) token. This part depends on the exact model.
That’s why the decoder inputs are the labels shifted by one with this special token at the start.
---
**Author:**  [@jon-fernandes](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjon-fernandes)%5C
**Post date:** [December 29, 2021, 5:47pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F10 "2021-12-29T17:47:23Z")
In this code snippet, what is eval\_preds? I can see it is the argument to the compute\_metrics function, but I don’t know what it is, and hence why we know we can split it as a tuple.
```auto
def compute_metrics(eval_preds):
preds, labels = eval_preds
```
I can see it is one of the arguments for Seq2SeqTrainer:
```auto
trainer = Seq2SeqTrainer(
model,
args,
train_dataset=tokenized_datasets["train"],
eval_dataset=tokenized_datasets["validation"],
data_collator=data_collator,
tokenizer=tokenizer,
compute_metrics=compute_metrics,
```
But where would is it getting it’s argument, eval\_preds from? The purpose of the compute\_metrics function is to compare the predicted values with the actual values.
Are the labels (actual values) - data\_collator[‘labels’] ?
Where do we get the predicted values from?
Many thanks.
---
**Author:**  [@jon-fernandes](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjon-fernandes)%5C
**Post date:** [December 29, 2021, 5:53pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F11 "2021-12-29T17:53:55Z")
I have seen “labels” used in translation
```auto
model_inputs["labels"] = labels["input_ids"]
```
and in token classification
```auto
tokenized_inputs["labels"] = new_labels
```
Is “labels” always used to hold the ground-truth, please?
Many thanks
---
**Author:**  [@jon-fernandes](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fjon-fernandes)%5C
**Post date:** [December 30, 2021, 3:25pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F12 "2021-12-30T15:25:30Z")
In the translation section, what is the difference between
AutoModelForSeq2SeqLM and AutoModelForCausalLM please?
Is it:
AutoModelForSeq2SeqLM is used for language translation tasks
AutoModelForCausalLM is only for text generation (e.g. GPT+)
Many thanks
---
**Author:**  [@Abirate](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAbirate)%5C
**Post date:** [January 6, 2022, 9:45pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F13 "2022-01-06T21:45:20Z")
In the Chapter7:
Task : Question answering
When Running this code :
```python
tf_train_dataset = train_dataset.to_tf_dataset(
columns=[
"input_ids",
"start_positions",
"end_positions",
"attention_mask",
"token_type_ids",
],
dummy_labels=True,
shuffle=True,
batch_size=16,
)
tf_eval_dataset = validation_dataset.to_tf_dataset(
columns=["input_ids", "attention_mask", "token_type_ids"],
shuffle=False,
batch_size=16,
)
```
I get this error :
TypeError: to\_tf\_dataset() missing 1 required positional argument: ‘collate\_fn’
How to do it, especially as the validation and train datasets already include a padding to the max length!
---
**Author:**  [@Rocketknight1](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FRocketknight1)%5C
**Post date:** [January 10, 2022, 5:29pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F14 "2022-01-10T17:29:26Z")
Hi @Abirate, thank you for this bug report! This is our fault - we recently changed the `to_tf_dataset` method to always require a `collate_fn`. I’m working on updating the course materials right now, and I’ll let you know as soon as a fixed version is available.
---
**Author:**  [@Abirate](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FAbirate)%5C
**Post date:** [January 10, 2022, 6:47pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F15 "2022-01-10T18:47:10Z")
@Rocketknight1 Ok, thanks
---
**Author:**  [@mbateman](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmbateman)%5C
**Post date:** [January 27, 2022, 5:00pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F16 "2022-01-27T17:00:06Z")
Hi,
I had this problem running the evaluation on Colab. Any ideas?
## \*\*\*\*\* Running Evaluation \*\*\*\*\* Num examples = 21018 Batch size = 64 [329/329 21:25]
TypeError Traceback (most recent call last)
in ()
----\> 1 trainer.evaluate(max\_length=max\_target\_length)
2 frames
/usr/local/lib/python3.7/dist-packages/transformers/trainer.py in evaluation\_loop(self, dataloader, description, prediction\_loss\_only, ignore\_keys, metric\_key\_prefix)
2407
2408 if all\_losses is not None:
→ 2409 metrics[f"{metric\_key\_prefix}_loss"] = all\_losses.mean().item()
2410
2411 # Prefix all keys with metric\_key\_prefix + '_’
TypeError: ‘NoneType’ object does not support item assignment
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [January 27, 2022, 10:42pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F17 "2022-01-27T22:42:20Z")
Hey @mbateman in which section in chapter 7 do you find this error? I’d like to run the relevant Colab notebook myself to see if I can reproduce the error 🙂
---
**Author:**  [@mbateman](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmbateman)%5C
**Post date:** [January 28, 2022, 9:06am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F18 "2022-01-28T09:06:55Z")
Hi @lewtun thanks for getting back to me. This was in the fine tuning subsection of the translation section:
trainer.evaluate(max\_length=max\_target\_length)
trainer.train()
trainer.evaluate(max\_length=max\_target\_length)
Doesn’t happen when run locally.
Hope that helps.
Michael
---
**Author:**  [@PyaePK](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FPyaePK)%5C
**Post date:** [February 14, 2022, 4:06pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F19 "2022-02-14T16:06:11Z")
I encountered the same problem and it seem problem is `compute_metrics` does not return anything and also `metric.compute` is not used inside that function. Since return value is missing, this result in `metrics = None` and then `NoneType` item assignment error. Adding `return metric.compute(predictions=decoded_preds, references=decoded_labels)` to `compute_metrics` solve the problem for me.
```auto
def compute_metrics(eval_preds):
preds, labels = eval_preds
# In case the model returns more than the prediction logits
if isinstance(preds, tuple):
preds = preds[0]
decoded_preds = tokenizer.batch_decode(preds, skip_special_tokens=True)
# Replace -100s in the labels as we can't decode them
labels = np.where(labels != -100, labels, tokenizer.pad_token_id)
decoded_labels = tokenizer.batch_decode(labels, skip_special_tokens=True)
# Some simple post-processing
decoded_preds = [pred.strip() for pred in decoded_preds]
decoded_labels = [[label.strip()] for label in decoded_labels]
return metric.compute(predictions=decoded_preds, references=decoded_labels)
```
---
**Author:**  [@lewtun](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Flewtun)%5C
**Post date:** [February 18, 2022, 3:55pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746%2F20 "2022-02-18T15:55:42Z")
Thanks for catching this bug @PyaePK ! I’ll post a fix to the website and notebooks 🤗
[Next page](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fchapter-7-questions%2F11746.md%3Fpage%3D2)