Reinforcement Learning

Add to your list(s) Download to your calendar using vCal

• Kin K. Leung, EEE Department, Imperial College
• Tuesday 11 January 2022, 14:00-16:00
• Via Teams.

If you have a question about this talk, please contact Kin K Leung.

This is the second 2-hour seminar on “Online Learning and Reinforcement Learning,” aiming to introduce reinforcement learning and its application to communication systems to our post-graduate students, although everyone is welcome to attend.

Reinforcement learning (RL) has successfully been applied to many application domains ranging from control and management of communication and computer systems, navigation of driverless vehicles, robots and flying drones, to guiding medical imaging and surgery, to name a few. We describe the Markov Decision Process (MDP) as the mathematical foundation of RL. To solve the MDP , the goal is to derive the optimal (action) policy that decides the optimal action for every given state of the system in order to maximize the long-term reward.

As the complexity of the conventional solution techniques increase for large systems, we describe how neural networks can be used to approximate the expected rewards (i.e., Q-values) as a function of system states and actions. This has become a feasible approach to overcoming the complexity issue, leading to deep Q-learning (where the neural-network parameters are “learned” using techniques discussed in the first companion seminar). Furthermore, one can also use a neural network to represent the optimal policy. The latter results into the well-known actor-critic RL where both the Q-values and policies are approximated by two different neural networks.

For illustration purposes, deep RL are used to manage and control communication and computer infrastructures. Such application reveals a very challenging issue for deep RL. That is, the huge state and action spaces increase the model complexity and training time beyond practical feasibility. To address this issue, we shall highlight several new techniques, including the state-action separable, embedding and state-space decomposition techniques, with a common goal of reducing model complexity and training time for deep RL. Open research issues such as distributed multi-agent RL, hierarchical RL, relationship between distributed RL and distributed optimization, and non-Markovian RL will also be highlighted.

A Teams link to attend this seminar is given below:

https://teams.microsoft.com/l/meetup-join/19%3ameeting_ZDEyOTNmMmEtYmI3MC00MDQ0LTg3OTAtOTg4MjAxOTAwNGQ4%40thread.v2/0?context=%7b%22Tid%22%3a%222b897507-ee8c-4575-830b-4f8267c3d307%22%2c%22Oid%22%3a%229ddee623-e996-457c-8025-c76d4162f7f7%22%7d

This talk is part of the Featured talks series.

This talk is included in these lists:

• Andrea Picciau's list
• COMMSP & CP list
• COMMSP Seminar
• Featured talks
• SAP Talks
• Via Teams

Note that ex-directory lists are not shown.