Simple Markov Chains Memoryless Property Question

Question

Simple Markov Chains Memoryless Property Question

mlgal55

2022年2月19日 11:01

I have a sequential data from time T1 to T6. The rows contain the sequence of states for 50 customers. There are only 3 states in my data. For example, it looks like this:

        T1  T2  T3  T4  T5  T6
Cust1   C   B   C   A   A   C

My transition matrix X looks like this:

    A   B   C
A   0.3 0.6 0.1
B   0.5 0.2 0.3
C   0.7 0.1 0.2

Now, we see that at time T6 the state is at C which corresponds to c=[0 0 1] vector. I am now predicting T7 by doing the matrix multiplication: c * X which gives me [0.7 0.1 0.2]. Based on this, I decide that the state at T7 would be A (highest prob. value).

For T8, I use the result of the probability vector I got above and do [0.7 0.1 0.2]*X = [0.4 0.44 0.14] and decide that the state is B.

My question is: Am I doing something wrong? Am I contradicting the memoryless property of the Markov Chains?

Topic markov markov-process

Category Data Science

christianb93 · Accepted Answer · 2018年4月8日 19:46

I am not sure what you actually want to do, but if you want to simulate the Markov chain, than you really have to bring probabilities into play.

Thus, in each step, you would use the transition matrix to determine the probability for each of the possible target states - A, B and C in your example. You would then draw a value - let me call it U - from a uniform distribution on [0,1] and determine the next state based on that value.

In your example, you would proceed from C to A if U is between 0 and 0.7 (and therefore with probability 0.7), to B if U is between 0.7 and 0.8 (and therefore with probability 0.8 - 0.7 = 0.1) and stay at C if U is between 0.8 and 1.0 (with probability 1.0 - 0.8 = 0.2)

Frankstr · Accepted Answer · 2018年4月7日 11:07

I think you are not doing anything wrong, the markov property is satisfied when the prediction can be solely based on the present state. I do not think you are contradicting this "memoryless" condition (which applies to the whole chain) when you predict what could probaly happen after a last state that is already just predicted.

Simple Markov Chains Memoryless Property Question

About