Artificial Intelligence and Soft Computing: Behavioral ... - Arteimi.info

Let us now assume that the game is being played between a computer
and a person and the computer keeps a record of the moves it has chosen in all
its turns in a game. This is recorded in a matrix, where the (i, j)-th element of
the matrix stands for the probability of success, if the computer in its turn
makes a change from j-th to i-th state. It is to be noted that the sum of all
elements under each column of the above matrix is one. This directly follows
intuitively, since the next state could be any of the possible states under one
column. The structure of the matrix is presented in fig. 13.12.
It should be added that the system learns from a reward-penalty
mechanism. This is realized in the following manner. After completion of a
game, the computer adjusts the elements of the matrix. If it wins the game,
then the elements corresponding to all its moves are increased by δ and the
rest of the elements under each column are decreased equally, so that the
column-sum remains one. On the other hand, if the computer loses the game,
then the elements corresponding to its moves are decreased by δ and the
remaining elements under each column are increased equally, so that column
sum remains one.
After a large number of such trials, the matrix becomes invariant and the
computer in its turn selects the state with highest probability under a given
column.
13.4.2 Adaptive Dynamic Programming
The reinforcement learning presumes that the agent receives a response from
the environment but can determine its status (rewarding/punishable) only at
the end of its activity, called the terminal state. We also assume that initially
the agent is at a state S0 and after performing an action on the environment, it
moves to a new state S1. If the action is denoted by a 0, we say
a0
S0 → S1,
i.e., because of action a0, the agent changes its state from S0 to S1. Further, the
reward of an agent can be represented by a utility function. For example, the
points of a ping-pong agent could be its utility.
The agent in reinforcement learning could be either passive or active. A
passive learner attempts to learn the utility through its presence in different
states. An active learner, on the other hand, can infer the utility at unknown
states from its knowledge, gained through learning.