# Parallel HPO when using \`trainer.hyperparameter\_search()\`

**URL:** </static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fparallel-hpo-when-using-trainer-hyperparameter-search%2F13226%3E%5C
**Category:** 🤗Transformers\
**Created:** [December 30, 2021, 2:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fparallel-hpo-when-using-trainer-hyperparameter-search%2F13226 "2021-12-30T02:37:23Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![MrRobot](https://avatars.discourse-cdn.com/v4/letter/m/b38774/32.png) [@MrRobot](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FMrRobot)%5C
**Post date:** [December 30, 2021, 2:37am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fparallel-hpo-when-using-trainer-hyperparameter-search%2F13226%2F1 "2021-12-30T02:37:23Z")

</div>

One of the features of `optuna` is its support for asynchronous parallelization of trials across multiple devices (see its [doc](https://optuna.readthedocs.io/en/stable/tutorial/10_key_features/004_distributed.html)) . But from my experiences of using `trainer.hyperparameter_search()`, it seems that different trials are executed one after another. So even though I do have access to multiple devices, I could not leverage them for parallel HPO.

I am wondering if it is possible to use the parallelization feature of `optuna`. Any input is appreciated.