Transferring the hidden state of a RNN to another RNN

I am using Reinforcement Learning to teach an AI an Austrian Card Game with imperfect information called Schnapsen. For different states of the game, I have different neural networks (which use different features) that calculate the value/policy. I would like to try using RNNs, as past actions may be important to navigate future decisions.

However, as I use multiple neural networks, I somehow need to constantly transfer the hidden state from one RNN to another one. I am not quite able to do that, especially during training I don't know how to make backpropagation through time work. I am grateful for any advice or links to related papers/blogs!

I am currently working with Flux in Julia, but I am also willing to switch to Tensorflow or Pytorch in Python.

Topic backpropagation rnn reinforcement-learning

Category Data Science

1
answered at

These are a few approaches I found in the research field that combines both RNN and Reinforcement Learning that looks promising

  • Reinforcement learning with LSTM networks
  • Reinforcement learning with RNN
  • Hybrid RNN approach

Research paper links

  1. A Reinforcement Learning and Recurrent Neural Network Based Dynamic User Modeling System
  2. Supervised Reinforcement Learning with Recurrent Neural Network for Dynamic Treatment Recommendation
  3. The state of mind: Reinforcement learning with recurrent neural networks
  4. Reinforcement learning by backpropagation through an LSTM model/critic
  5. Hybrid RNN

Note: The following paper seems to be the only one verified in the industry and applied to an industrial problem D. Prokhorov, Toward effective combination of off-line and on-line training in ADP framework, in: Proceedings of the IEEE International Symposium on Approximate Dynamic Programming and Reinforcement Learning (ADPRL), Honolulu, HI, pp. 268–271, 2007

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.