Extracting component means and convariances from mixture model

I am currently trying to write a simple multivariate gaussian mixture model using tensorflow probability. Specifically, I have some 2-dimensional input and 2-dimensional output data and am looking to produce a probabilistic model that best fits this data by using a neural network. I am utilizing tfp.layers.MixtureSameFamily to generate this model, which is working perfectly well as expected. Here is my code for doing so:

Notes: I am using tensorflow 2.4.1 and tensorflow-probability 0.12.1

x = some 2-d input data
y = some 2-d output data

event_shape = y.shape[-1]
num_components = 5
params_size = tfp.layers.MixtureSameFamily.params_size(
num_components,
component_params_size=tfp.layers.MultivariateNormalTriL.params_size(event_shape))

model = tf.keras.Sequential([
tf.keras.layers.Dense(params_size),
tfp.layers.MixtureSameFamily(num_components, tfp.layers.MultivariateNormalTriL(event_shape)),
])

model.compile(optimizer=tf.optimizers.Adam(learning_rate=0.01),
loss=lambda y, model1: -model1.log_prob(y),
metrics=[])
batch_size = 250
model.fit(x, y,
batch_size=batch_size,
epochs=500,
steps_per_epoch=n // batch_size,
verbose=False,
shuffle=True)

yhat_mix=model(x_tst)

However, once the network has been trained and the model produced, I do not know how to extract the specific information that I am actually after, i.e. the means and covariances of the component 2-dimensional gaussians that make up the mixture model. Is there an easy method to extract this information? I've been looking for something akin to .mean() and .covariance() but which give the individual component means and covariances, instead of for the entire distribution. I thought using .mode() might help since I'm partly looking for the modes of the entire distribution but this just returns the following error:

NotImplementedError Traceback (most recent call last)
in
---- 1 yhat_mix.mode()

~\anaconda3\envs\py36\lib\site-packages\tensorflow_probability\python\distributions\distribution.py in mode(self, name, **kwargs)
1394 Mode.
1395 with self._name_and_control_scope(name):
- 1396 return self._mode(**kwargs)
1397
1398 def _cross_entropy(self, other):

~\anaconda3\envs\py36\lib\site-packages\tensorflow_probability\python\distributions\distribution.py in _mode(self, **kwargs)
1389 def _mode(self, **kwargs):
1390 raise NotImplementedError('mode is not implemented: {}'.format(
- 1391 type(self).name))
1392
1393 def mode(self, name='mode', **kwargs):

NotImplementedError: mode is not implemented: MixtureSameFamily

Ideally, I can get the parameters that define all the component multivariate distributions though (their means and covariances) and not just mode locations of the mixture model. Does anyone know how to get this information? Any help would be appreciated, thanks!

Topic gaussian tensorflow implementation neural-network python

Category Data Science


The values you are looking for are the weights of your dense layer. model.weights[1][...] should do the job. Take a look at this tutorial: https://www.richard-stanton.com/2020/07/14/tfp-fit-normal.html

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.