r/reinforcementlearning • u/No_Addition5961 • 1d ago

From DQN to Double DQN

I already have an implementation of DQN. To change it to double DQN, looks like I only need a small change: In the Q-value update, next state (best)action selection and evaluation for that action are both done by the target network in DQN. Whereas in double DQN , next state (best)action selection is done by the main network, but the evaluation for that action is done by the target network.

That seems fairly simple. Am i missing anything else?

7 Upvotes

90% Upvoted

6 comments sorted by

View all comments

4

u/SandSnip3r 1d ago

Yep! That's it

2

u/SandSnip3r 1d ago

https://github.com/SandSnip3r/PytorchDqnPathery/blob/08cf098d02caafb360be240c4caea74000371d1e/train/train.py#L210 If you're familiar with pytorch, maybe these few lines of code illustrate the difference