# Decision Transformer for Discrete action
**URL:**
**Author:**  [@Jeon87](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2FJeon87)%5C
**Post date:** [March 29, 2024, 1:47pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdecision-transformer-for-discrete-action%2F79316%2F1 "2024-03-29T13:47:44Z")
I’m studying the decision transformer referring to [Train your first Decision Transformer](https://huggingface.co/blog/train-decision-transformers).
In the post, the example is for “halfcheetah” ( action space is continuous) and
the following model code is used.
I’m trying to apply this to the discrete action space.
I added the logit layer for the discrete action and changed the loss function as below.
( red color: removed , blue color: added )
Is this the right approach?
---
**Author:**  [@xian-lee](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fxian-lee)%5C
**Post date:** [April 26, 2024, 4:31am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdecision-transformer-for-discrete-action%2F79316%2F2 "2024-04-26T04:31:30Z")
Hey,
I’m also trying out something similar recently. Based on my implementation, your implementation looks almost similar to mine, except for the fact that I did not use an additional linear layer and directly used the outputs as the logit. Also, I had encode my actions into one-hot-encoding, so I had to do some reshaping with the action targets, but otherwise, I think this is pretty much spot on.
I usually just print out the shapes and the intermediate variables at least once for a sanity check to make sure everything looks right and nothing it broadcasted incorrectly.
---
**Author:**  [@carlosgcg](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcarlosgcg)%5C
**Post date:** [December 4, 2024, 5:20pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdecision-transformer-for-discrete-action%2F79316%2F3 "2024-12-04T17:20:19Z")
Hey!
and in the original\_forward, why did you define the action\_targets? Because I understand that the logits are the action\_preds, no?
---
**Author:**  [@xian-lee](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fxian-lee)%5C
**Post date:** [December 4, 2024, 7:52pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdecision-transformer-for-discrete-action%2F79316%2F4 "2024-12-04T19:52:03Z")
Yeah, it does seem like the action\_targets are technically not needed in the original\_forward() function which is used during test mode, and I don’t see it being used in the code snippet either.
---
**Author:**  [@carlosgcg](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fcarlosgcg)%5C
**Post date:** [December 7, 2024, 11:36am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdecision-transformer-for-discrete-action%2F79316%2F5 "2024-12-07T11:36:57Z")
Thank you for your quick response! 😄 I’m using DT for my bachelor degree final thesis and I’m a bit lost. Seeing that you have been using it for a longer time, I hope you don’t mind if I ask you some questions.
I have a case very similar to the one you mentioned, with discrete actions and using one-hot encoding. So, does this code look good in your opinion?
I’m also at a bit of a loss as to which activation function best fits this type of problem and certain model configuration parameters. How do you see these?
Thanks in advanced!!!
---
**Author:**  [@benstokes](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fbenstokes)%5C
**Post date:** [December 7, 2024, 7:20pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fdecision-transformer-for-discrete-action%2F79316%2F6 "2024-12-07T19:20:23Z")
Thanks for sharing. It helps me a lot[.](https://sassagrantstatuscheck.co.za/)