Neeraj Jaggi - Witchita State University - Wichita State University

Perfect State Information
• Markov Decision Process (Finite K)
◦ State Space = {(L, E); 0 ≤ L ≤ K, E є [0, 1]}
L – Current Energy Level, E – On/Off period
Reward r– one if event detected; zero otherwise
Action u є [0, 1]; Transition probabilities p
10
• Optimality equations (average reward criteria)
◦ h* – state variables
◦ λ* – optimal reward
Neeraj Jaggi Dept of EECS Wichita State University