Markov Process and transition matrix

Your problem is unusual in two ways:

• Apparently the states are known, not hidden. Afaik it's much more common that the states are hidden, and only observations are known. This is what Hidden Markov Models deal with.
• There's a single sequence. It can be a problem if the sequence is not long enough to show a representative sample of all the transitions. Also it's unknown whether there are any other possible initial states.

Apart from this, you can easily estimate a transition matrix: just count how many times each pair of states appear next to each other. For example in 1,2,4,1,3,4,2,4,2,2,4,1 we have:

• 1 -> 2: 1
• 1 -> 3: 1
• 2 -> 2: 1
• 2 -> 4: 3
• 3 -> 4: 1
• 4 -> 1: 2
• 4 -> 2: 2

Note: the total number of transitions should be equal to the length of the sequence minus 1.

From this we can calculate every transition probability,it's just the conditional probability of arriving in state $x$ given starting point $y$, i.e.

$$p(x|y)=\frac{\#(x,y)}{\sum_{y'}\#(x,y')}$$

Here $\#(x,y)$ means "number of times that y -> x happens". In the example:

• 1 -> 2: 1/2
• 1 -> 3: 1/2
• 2 -> 2: 1/4
• 2 -> 4: 3/4
• 3 -> 4: 1
• 4 -> 1: 1/2
• 4 -> 2: 1/2