Actor Critic Method: Maze Example - FIAS

More documents

Recommendations

Info

Policy Evaluation <strong>Example</strong> w ( u) → w( u) + εδ with є=0.5 and δ = ( u) + v( u') − v( u) r a Thick lines are running average of weight values; figure taken from Dayan&Abbott
Note: Sutton and Barto book uses “p” for action preferences Q a' δ = r a Policy Improvement (using a so-called direct actor rule) ( u) → Qa' ( u) + ε ( δ aa' − p( a'; u)) δ ( u) + v( u') − v( u) positive if a’ was chosen, else negative , where positive if outcome better than expected, else negative p(a ’;u) is the softmax probability of choosing action a’ in state u as determined by Q a’ (u) This term is not strictly necessary but has the advantage that actions which are already chosen with very high probability only increase their Q value very slowly
Page 1 and 2: Actor Critic Method: Maze Example
Page 3 and 4: Actor-Critic Method Policy Agent Ac
Page 5 and 6: Policy Iteration • Two Observatio
Page 7: Temporal Difference Learning Idea:
Page 11: Policy Improvement Example •learn

Actor Critic Method: Maze Example - FIAS

Create successful ePaper yourself

Delete template?

Save as template?