What is it?
In Probability and Reinforcement Learning, the Markov property states that the conditional probability distribution of the future state of a random process depends only on its present state, and not on any past states. In other words, the future is independent from the past, given the present. However, this assumes a stochastic process.
The Markov property can be written as:
Which equals the probability of going to state given and the probability of going to state given all past states. In short, the information of previous states does not contribute to predict the future state: only the present state is significant.
Not all environments and processes follows this property.
However, for those who do, it’s then possible to apply the Markov Decision Process to predict favorable future states.