Reducing the training time of an RL agent

I am trying to develop an rl agent using DQN algorithm.During training, the agent interacts with environment which is a simulated one.Each episode takes around 10 mins to run. This way if want my agent to train for some 1000000(to achieve convergence) episodes, its becoming computationally infeasible, Is there a way anyone is aware to speed up my training process, like using parallel threading or using cuda. Or is it something because of the algorithm?

my episode here basically is of a one day long with actions taken at every 15 mins. I am using FMIs for the simulation and to finish this 1 episode it takes 10 mins. I can reduce my episode time to few seconds by using reduced order models/equations. This decreases the episode length(time). But my question is are there ways to speed up the training process. Or there better algorithms than DQN which donot reqire this much of training.

Thank you.