# Dataset map method - how to pass argument to the function

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-map-method-how-to-pass-argument-to-the-function%2F16274%3E%5C
**Category:** Beginners\
**Created:** [March 30, 2022, 6:36pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-map-method-how-to-pass-argument-to-the-function%2F16274 "2022-03-30T18:36:37Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![sssingh](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sssingh/32/19900_2.png) [@sssingh](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsssingh)%5C
**Post date:** [March 30, 2022, 6:36pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-map-method-how-to-pass-argument-to-the-function%2F16274%2F1 "2022-03-30T18:36:37Z")

</div>

Hi, just started using the Huggingface library. I am wondering how can I pass `model` and `tokenizer` to my processing function along with the batch when using the `map` method.

def my\_processing\_func(batch, model, tokenizer):  
–code–

I am using map like this…  
new\_dataset = my\_dataset.map(my\_processing\_func, model, tokenizer, batched=True)

when I do this it does not fail but instead of passing the dictionary with input\_ids and attention\_mask, it passes a `list` of just input\_ids as the batch to my\_processing\_func. When I remove the model and tokenizer argument then it sends the dictionary as expected.

Where am I going wrong?

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmariosasko)%5C
**Post date:** [March 31, 2022, 11:24am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-map-method-how-to-pass-argument-to-the-function%2F16274%2F2 "2022-03-31T11:24:25Z")

</div>

Hi! You can use `fn_kwargs` to pass the arguments to the map function:

```python
new_dataset = my_dataset.map(my_processing_func, batched=True, fn_kwargs={"model": model, "tokenizer": tokenizer})

```

Or you can use `partial`:

```python
from functools import partial
new_dataset = my_dataset.map(partial(my_processing_func, model=model, tokenizer=tokenizer), batched=True)

```

---

<div class="post-metadata">

**Author:** ![BramVanroy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bramvanroy/32/47360_2.png) [@BramVanroy](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FBramVanroy)%5C
**Post date:** [March 31, 2022, 11:51am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-map-method-how-to-pass-argument-to-the-function%2F16274%2F3 "2022-03-31T11:51:59Z")

</div>

Is there any downside to using either options? If I remember correctly (?) lambdas are not picklable. So my assumption would be that if you do something like

```auto
new_dataset = my_dataset.map(lambda batch: my_processing_func(batch, model, tokenizer), batched=True)

```

it won’t be cached. Is that correct?

---

<div class="post-metadata">

**Author:** ![sssingh](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sssingh/32/19900_2.png) [@sssingh](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fsssingh)%5C
**Post date:** [March 31, 2022, 1:05pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-map-method-how-to-pass-argument-to-the-function%2F16274%2F4 "2022-03-31T13:05:33Z")

</div>

Super!! this works for me … thanks 🙏

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmariosasko)%5C
**Post date:** [April 5, 2022, 12:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdataset-map-method-how-to-pass-argument-to-the-function%2F16274%2F5 "2022-04-05T12:16:51Z")

</div>

1. There shouldn’t be a significant difference in speed between these two approaches.
2. We use `dill`, which knows how to pickle lamdas in most situations.
