Is "nb_steps_warmup" set for each episode or globally?

When I configure a DQN agent, nb_steps_warmup can be set. Is this parameter set for each episode or once globally?

What I am trying to ask is, imaging I have a game environment which takes about 3000 max. steps per episode. The DQN is fitted as follows:

dqn.fit(env, nb_steps=30000, visualize=True, verbose=2)

So, as I understand it, the fitting will run approximately 10 episodes (nb_steps / max. steps per episode).

If I set nb_steps_warmup = 5000, what actually happens?

A) nb_steps_warmup=5000, so 5000 out of nb_steps=30000 are used for warming up

B) within each episode the first nb_steps_warmup=5000 steps are used for warm up. As nb_steps_warmup max. steps per episode, the complete episode is used for warming up, which essentially means the fitting never leaves the warming up phase.

C) sth. else?

Topic keras-rl keras reinforcement-learning python

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.