Training a Graph model like an Artificial Neural Network

I currently have a Graph model whereby I am mapping connections of different types between entities and attributing a weight to these connections based upon my own personal experience. Also, I would like to understand the connections between these entities in relation to a particular outcome. Looking at this problem, I can't help but notice it's similarity to a typical Artificial Neural Network (ANN) and am wondering if/how I can bake some of the theory there into my model.

Let me explain further; I have a set of entities for which I have sets of connections. Each different type of connection has it's own individual weight, as shown in this diagram:

The weights in this diagram are estimated based on my own experience and can then be summed to look at the strength of a relationship between these entities. It is this strength of relationship that I want to understand in relation to a particular Output/Outcome.

I can't help but notice when looking at this particular application of Graph Theory that it bears a large resemblance to an Artificial Neural Network (ANN), as shown in this diagram;

The difference here being that whilst in the first diagram the nodes represent entities, in the second diagram the nodes represent input values. In the ANN, input values, randomized weights and the process of back-propagation are are used to help determine what the weights need to be whilst in the first diagram, these values are estimated. What I want to be able to do is to use randomized weights and the process of back-propagation to determine the weights in my Graph model that maps the connections between entities.

Does anyone have any experience of doing anything like this or could maybe at lest discuss the possibility of doing something like this?

Thanks!

Topic graphical-model machine-learning

Category Data Science

1
answered at

First, a note: don't let the graph structure make you think that this can be addressed with neural networks; this structure only suggests that this is a hierarchical model.

Second, let's take into account the following:

  • In your first figure, you don't connect A, B or C to the output. I assume that there is a relationship from A,B,C to output.
  • You mention that you have expert a priori knowledge on the relationships from A to B and from A to C.
  • I also assume that A, B, C and the output are numeric variables.
  • We don't know whether A, B, C and the output can be observed or not.
  • We don't know why there are 3 lines from A to C and 2 lines from A to B, so I will not take this into account.

You may want to provide more information if any of the assumptions above is not correct.

Now:

If only A and the output are observable, we are talking about a latent variable model. Latent variables are best suited for Hierarchical Bayesian methods. If the relations between variables are linear, you can model this as a bayesian linear model, e.g. hierarchical bayesian regression to model your variables and inject your prior knowledge as prior distributions over the weights. Here you can see the application of hierarchical bayesian regression to a latent variable model. This kind of methods are simple and interpretable, they handle prior knowledge graciously and they handle uncertainty.

If all variables are observable, you can define a combination of different extremely simple "neural network" models to approach your problem: Network 1 would receive A and output C; Network 2 would receive A and output B; Network 3 would receive A, B and C and generate the output. If the relationships among variables are linear, Network 1 and Network 2 are simple multiplications. You can then initialize their weights with your prior knowledge. The training would adjust the weights for everything to fit, but will not give you insight about the relationships except the fitted weights.

I suggest you think about whether bayesian hierarchical models can fit your problem.

About

Geeks Mental is a community that publishes articles and tutorials about Web, Android, Data Science, new techniques and Linux security.