question

Hey Willie,

Yes, it's definitely possible to convert a MultiDiscrete action space into a Discrete one by combining OP1 and OP2 into a single value. Basically, you treat each combination of OP1 and OP2 as a unique action in a Discrete space.

For example, if OP1 has 10 (1,2,3...) possible values and OP2 has 5 possible values, you can map each pair (OP1, OP2) to a single discrete index using:

//MultiDiscrete to Discrete
int Op1 = 7;
int Op2 = 4;
int nOp2 = 5; // Num of posible actions of Op2
return Op1 * nOp2 + Op2; //the Discret input
 

//Discrete to MultiDiscrete 
int answer = 39; 
int nOp2 = 5; 
 
int OP1 = Math.floor(answer / nOp2 );
int OP2 = Math.fmod(answer ,nOp2);
return OP1 + "" + OP2; 

This makes it possible to use DQN, but to be honest, it’s not the most efficient way. DQN treats each action as an independent choice, so combining multiple actions into one number can make it harder for the model to recognize patterns. This often leads to longer training times and more complex learning.

Since PPO is already working well for you, I'd recommend sticking with it if possible. But if DQN is a must, this method will get the job done—it just might take more effort to train.

Hope this helps!