Can the 'Rainbow Algorithm' be scaled up and sped up?

What's the proper way to train the algorithm with bigger batches or otherwise speed it up?

The 'Rainbow Algorithm' is a Deep Q, Reinforcement Learning algorithm with two neural networks that I would like to speed up or scale up during training.

You can read the paper here.

Training is fairly slow because the observations have to be converted to tensors and updated to the model after each step. It's kind of a special and unique model, so I hope someone special can answer.

I haven't gotten any results skipping the back prop at each step so I know I can increase the batch size in my case but I'm wondering if this does any good, or might even harm the results at each step. If I increase the batch size should I increase something else, like the buffer size?

STARTING_BALANCE = 10_000

GAMMA = 0.99
BATCH_SIZE = 8192
BUFFER_SIZE = 80_000
MIN_REPLAY_SIZE = 1000
EPSILON_START = 0.59
EPSILON_END = 0.001
EPSILON_DECAY = 90_000
TARGET_UPDATE_FREQ = 1_000