Strategy for achieving a “hybrid” GAN

I have worked with a few GAN-like algorithms, but always with similar inputs and outputs. Being only a novice in deep learning, I often work by adapting an already existing Notebook, but today I have a more complex problem and I cannot find a similar example.

For my algorithm I need to predict a curve, which is a list of floats (e.g. dimension 20x1), from two types of input data; a 2D image (e.g. a 16x16 normalized float array) and a set of some parameters. Currently I know how to introduce an image into a generator to output another or a set of parameters to predict a value, but I don't know how I can combine the image and the set of parameters. I'm afraid that the pixel values of the image have too much weight.

Example of the schematic model:

A good method could be to convolve the image and pass my parameters through a neural network to obtain two dense layers and concatenate them. Once that is done, a 1D upsampling should be done and then convoluted.

And also, can an architecture like this be implemented in Keras? I wonder if it is planned to separate the inputs in my generator to do two different treatments, or if I have to do that separately?

Topic gan keras tensorflow deep-learning machine-learning

Category Data Science

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.