# GetWiki

*reinforcement learning*

ARTICLE SUBJECTS

being →

database →

ethics →

fiction →

history →

internet →

language →

linux →

logic →

method →

news →

policy →

purpose →

religion →

science →

software →

truth →

unix →

wiki →

ARTICLE TYPES

essay →

feed →

help →

system →

wiki →

ARTICLE ORIGINS

critical →

forked →

imported →

original →

reinforcement learning

[ temporary import ]

**please note:**

- the content below is remote from Wikipedia

- it has been imported raw for GetWiki

**Reinforcement learning**(

**RL**) is an area of machine learning concerned with how software agents ought to take actions in an environment so as to maximize some notion of cumulative reward. Reinforcement learning is one of three basic machine learning paradigms, alongside supervised learning and unsupervised learning.It differs from supervised learning in that labelled input/output pairs need not be presented, and sub-optimal actions need not be explicitly corrected. Instead the focus is finding a balance between exploration (of uncharted territory) and exploitation (of current knowledge).JOURNAL, Kaelbling, Leslie P., Littman, Michael L., Michael L. Littman, Moore, Andrew W., Andrew W. Moore, 1996, Reinforcement Learning: A Survey,weblink yes, Journal of Artificial Intelligence Research, 4, 237â€“285, 10.1613/jair.301,weblink 2001-11-20, harv, Leslie P. Kaelbling, cs/9605103, The environment is typically formulated as a Markov decision process (MDP), as many reinforcement learning algorithms for this context utilize dynamic programming techniques.Dimitri P. Bertsekas and John N. Tsitsiklis. "Neuro-Dynamic Programming", Athena Scientific, 1996,weblinkDimitri P. Bertsekas. "Dynamic Programming and Optimal Control: Approximate Dynamic Programming, Vol.II", Athena Scientific, 2012,weblinkBOOK, Reinforcement learning and markov decision processes, van Otterlo, M., Wiering, M., Reinforcement Learning, 12, 3â€“42, 2012, 10.1007/978-3-642-27645-3_1, Adaptation, Learning, and Optimization, 978-3-642-27644-6, The main difference between the classical dynamic programming methods and reinforcement learning algorithms is that the latter do not assume knowledge of an exact mathematical model of the MDP and they target large MDPs where exact methods become infeasible.{{toclimit|3}}

## Introduction

(File:Reinforcement learning diagram.svg|thumb|right|250px| The typical framing of a Reinforcement Learning (RL) scenario: an agent takes actions in an environment, which is interpreted into a reward and a representation of the state, which are fed back into the agent.)Reinforcement learning, due to its generality, is studied in many other disciplines, such as game theory, control theory, operations research, information theory, simulation-based optimization, multi-agent systems, swarm intelligence, statistics and genetic algorithms. In the operations research and control literature, reinforcement learning is called*approximate dynamic programming,*or

*neuro-dynamic programming.*The problems of interest in reinforcement learning have also been studied in the theory of optimal control, which is concerned mostly with the existence and characterization of optimal solutions, and algorithms for their exact computation, and less with learning or approximation, particularly in the absence of a mathematical model of the environment. In economics and game theory, reinforcement learning may be used to explain how equilibrium may arise under bounded rationality.Basic reinforcement is modeled as a Markov decision process:

- a set of environment and agent states, {{mvar|S}};
- a set of actions, {{mvar|A}}, of the agent;
- P_a(s,s')=Pr(s_{t+1}=s'mid s_t=s, a_t=a) is the probability of transition from state s to state s' under action a.
- R_a(s,s') is the immediate reward after transition from s to s' with action a.
- rules that describe what the agent observes

*full observability*). If not, the agent has

*partial observability*. Sometimes the set of actions available to the agent is restricted (a zero balance cannot be reduced. For example, if the current value of the agent is 3 and the state transition reduces the value by 4, the transition will not be allowed).A reinforcement learning agent interacts with its environment in discrete time steps. At each time {{mvar|t}}, the agent receives an observation o_t, which typically includes the reward r_t. It then chooses an action a_t from the set of available actions, which is subsequently sent to the environment. The environment moves to a new state s_{t+1} and the reward r_{t+1} associated with the

*transition*(s_t,a_t,s_{t+1}) is determined. The goal of a reinforcement learning agent is to collect as much reward as possible. The agent can (possibly randomly) choose any action as a function of the history.When the agent's performance is compared to that of an agent that acts optimally, the difference in performance gives rise to the notion of

*regret*. In order to act near optimally, the agent must reason about the long term consequences of its actions (i.e., maximize future income), although the immediate reward associated with this might be negative.Thus, reinforcement learning is particularly well-suited to problems that include a long-term versus short-term reward trade-off. It has been applied successfully to various problems, including robot control, elevator scheduling, telecommunications, backgammon, checkers{{Sfn|Sutton|Barto|p=|loc=Chapter 11}} and go (AlphaGo).Two elements make reinforcement learning powerful: the use of samples to optimize performance and the use of function approximation to deal with large environments. Thanks to these two key components, reinforcement learning can be used in large environments in the following situations:

- A model of the environment is known, but an analytic solution is not available;
- Only a simulation model of the environment is given (the subject of simulation-based optimization);{{sfn|Gosavi|2003}}
- The only way to collect information about the environment is to interact with it.

## Exploration

The exploration vs. exploitation trade-off has been most thoroughly studied through the multi-armed bandit problem and for finite state space MDPs in Burnetas and Katehakis (1997){{citation | last1 = Burnetas|first1 = Apostolos N.|last2 = Katehakis|first2 = Michael N.|authorlink2 = Michael N. Katehakis|year = 1997|title = Optimal adaptive policies for Markov Decision Processes|url =weblink|journal = Mathematics of Operations Research|volume = 22|pages = 222--255|ref = harv}}. Reinforcement learning requires clever exploration mechanisms. Randomly selecting actions, without reference to an estimated probability distribution, shows poor performance. The case of (small) finite Markov decision processes is relatively well understood. However, due to the lack of algorithms that scale well with the number of states (or scale to problems with infinite state spaces), simple exploration methods are the most practical.One such method is epsilon-greedy, when the agent chooses the action that it believes has the best long-term effect with probability 1-epsilon. If no action which satisfies this condition is found, the agent chooses an action uniformly at random. Here, 0 < epsilon < 1 is a tuning parameter, which is sometimes changed, either according to a fixed schedule (making the agent explore progressively less), or adaptively based on heuristics.{{citation | last1 = Tokic | first1 = Michel | last2 = Palm | first2 = GÃ¼nther | chapter = Value-Difference Based Exploration: Adaptive Control Between Epsilon-Greedy and Softmax | pages = 335â€“346 | publisher = Springer | series = Lecture Notes in Computer Science | title = KI 2011: Advances in Artificial Intelligence | volume = 7006 | year = 2011 | chapter-url =weblink | isbn = 978-3-642-24455-1}}## Algorithms for control learning

Even if the issue of exploration is disregarded and even if the state was observable (assumed hereafter), the problem remains to use past experience to find out which actions are good.### Criterion of optimality

#### Policy

The agent's action selection is modeled as a map called*policy*:

pi: A times S rightarrow [0,1]
pi(a,s) = Pr(a_t = amid s_t =s)

The policy map gives the probability of taking action a when in state s.BOOK,weblink Reinforcement learning: An introduction, {{Rp|61}} There are also non-probabilistic policies.#### State-value function

Value function V_pi(s) is defined as the*expected return*starting with state s, i.e. s_0 = s, and successively following policy pi. Hence, roughly speaking, the value function estimates "how good" it is to be in a given state.{{Rp|60}}

V_pi(s) = operatorname E[R] = operatorname Eleft[sum_{t=0}^infty gamma^t r_tmid s_0 = sright],

where the random variable R denotes the **return**, and is defined as the sum of future discounted rewards{{clarify|reason=What is a discounted reward?|date=January 2019}}

R=sum_{t=0}^infty gamma^t r_t,

where r_t is the reward at step t, gamma in [0,1] is the discount-rate{{clarify|reason=What is discount rate?|date=January 2019}}.The algorithm must find a policy with maximum expected return. From the theory of MDPs it is known that, without loss of generality, the search can be restricted to the set of so-called *stationary*policies. A policy is

*stationary*if the action-distribution returned by it depends only on the last state visited (from the observation agent's history). The search can be further restricted to

*deterministic*stationary policies. A

*deterministic stationary*policy deterministically selects actions based on the current state. Since any such policy can be identified with a mapping from the set of states to the set of actions, these policies can be identified with such mappings with no loss of generality.

### Brute force

The brute force approach entails two steps:- For each possible policy, sample returns while following it
- Choose the policy with the largest expected return

### Value function

Value function approaches attempt to find a policy that maximizes the return by maintaining a set of estimates of expected returns for some policy (usually either the "current" [on-policy] or the optimal [off-policy] one).These methods rely on the theory of MDPs, where optimality is defined in a sense that is stronger than the above one: A policy is called optimal if it achieves the best expected return from*any*initial state (i.e., initial distributions play no role in this definition). Again, an optimal policy can always be found amongst stationary policies.To define optimality in a formal manner, define the value of a policy pi by

V^{pi} (s) = E[Rmid s,pi],

where R stands for the return associated with following pi from the initial state s. Defining V^*(s) as the maximum possible value of V^pi(s), where pi is allowed to change,
V^*(s) = max_pi V^pi(s).

A policy that achieves these optimal values in each state is called *optimal*. Clearly, a policy that is optimal in this strong sense is also optimal in the sense that it maximizes the expected return rho^pi, since rho^pi = E[ V^pi(S) ], where S is a state randomly sampled from the distribution mu{{clarify |date=July 2018 |reason= mu no previously defined}}.Although state-values suffice to define optimality, it is useful to define action-values. Given a state s, an action a and a policy pi, the action-value of the pair (s,a) under pi is defined by

Q^pi(s,a) = operatorname E[Rmid s,a,pi],,

where R now stands for the random return associated with first taking action a in state s and following pi, thereafter.The theory of MDPs states that if pi^* is an optimal policy, we act optimally (take the optimal action) by choosing the action from Q^{pi^*}(s,cdot) with the highest value at each state, s. The *action-value function*of such an optimal policy (Q^{pi^*}) is called the

*optimal action-value function*and is commonly denoted by Q^*. In summary, the knowledge of the optimal action-value function alone suffices to know how to act optimally.Assuming full knowledge of the MDP, the two basic approaches to compute the optimal action-value function are value iteration and policy iteration. Both algorithms compute a sequence of functions Q_k (k=0,1,2,ldots) that converge to Q^*. Computing these functions involves computing expectations over the whole state-space, which is impractical for all but the smallest (finite) MDPs. In reinforcement learning methods, expectations are approximated by averaging over samples and using function approximation techniques to cope with the need to represent value functions over large state-action spaces.

#### Monte Carlo methods

Monte Carlo methods can be used in an algorithm that mimics policy iteration. Policy iteration consists of two steps:*policy evaluation*and

*policy improvement*.Monte Carlo is used in the policy evaluation step. In this step, given a stationary, deterministic policy pi, the goal is to compute the function values Q^pi(s,a) (or a good approximation to them) for all state-action pairs (s,a). Assuming (for simplicity) that the MDP is finite, that sufficient memory is available to accommodate the action-values and that the problem is episodic and after each episode a new one starts from some random initial state. Then, the estimate of the value of a given state-action pair (s,a) can be computed by averaging the sampled returns that originated from (s,a) over time. Given sufficient time, this procedure can thus construct a precise estimate Q of the action-value function Q^pi. This finishes the description of the policy evaluation step.In the policy improvement step, the next policy is obtained by computing a

*greedy*policy with respect to Q: Given a state s, this new policy returns an action that maximizes Q(s,cdot). In practice lazy evaluation can defer the computation of the maximizing actions to when they are needed.Problems with this procedure include:

- The procedure may spend too much time evaluating a suboptimal policy.
- It uses samples inefficiently in that a long trajectory improves the estimate only of the
*single*state-action pair that started the trajectory. - When the returns along the trajectories have
*high variance*, convergence is slow. - It works in episodic problems only;
- It works in small, finite MDPs only.

#### Temporal difference methods

The first problem is corrected by allowing the procedure to change the policy (at some or all states) before the values settle. This too may be problematic as it might prevent convergence. Most current algorithms do this, giving rise to the class of*generalized policy iteration*algorithms. Many

*actor critic*methods belong to this category.The second issue can be corrected by allowing trajectories to contribute to any state-action pair in them. This may also help to some extent with the third problem, although a better solution when returns have high variance is Sutton's{{sfn|Sutton|1984}}{{sfn|Sutton|Barto|1998|loc=Â§6. Temporal-Difference Learning}} temporal difference (TD) methods that are based on the recursive Bellman equation. Note that the computation in TD methods can be incremental (when after each transition the memory is changed and the transition is thrown away), or batch (when the transitions are batched and the estimates are computed once based on the batch). Batch methods, such as the least-squares temporal difference method,{{Sfn|Bradke|Barto|1996|p=}} may use the information in the samples better, while incremental methods are the only choice when batch methods are infeasible due to their high computational or memory complexity. Some methods try to combine the two approaches. Methods based on temporal differences also overcome the fourth issue.In order to address the fifth issue,

*function approximation methods*are used.

*Linear function approximation*starts with a mapping phi that assigns a finite-dimensional vector to each state-action pair. Then, the action values of a state-action pair (s,a) are obtained by linearly combining the components of phi(s,a) with some

*weights*theta:

Q(s,a) = sum_{i=1}^d theta_i phi_i(s,a).

The algorithms then adjust the weights, instead of adjusting the values associated with the individual state-action pairs. Methods based on ideas from nonparametric statistics (which can be seen to construct their own features) have been explored.Value iteration can also be used as a starting point, giving rise to the Q-learning algorithm and its many variants. {{Sfn|Watkins|1989|p=}}The problem with using action-values is that they may need highly precise estimates of the competing action values that can be hard to obtain when the returns are noisy. Though this problem is mitigated to some extent by temporal difference methods. Using the so-called compatible function approximation method compromises generality and efficiency. Another problem specific to TD comes from their reliance on the recursive Bellman equation. Most TD methods have a so-called lambda parameter (0le lambdale 1) that can continuously interpolate between Monte Carlo methods that do not rely on the Bellman equations and the basic TD methods that rely entirely on the Bellman equations. This can be effective in palliating this issue.### Direct policy search

An alternative method is to search directly in (some subset of) the policy space, in which case the problem becomes a case of stochastic optimization. The two approaches available are gradient-based and gradient-free methods.Gradient-based methods (*policy gradient methods*) start with a mapping from a finite-dimensional (parameter) space to the space of policies: given the parameter vector theta, let pi_theta denote the policy associated to theta. Defining the performance function by

rho(theta) = rho^{pi_theta},

under mild conditions this function will be differentiable as a function of the parameter vector theta. If the gradient of rho was known, one could use gradient ascent. Since an analytic expression for the gradient is not available, only a noisy estimate is available. Such an estimate can be constructed in many ways, giving rise to algorithms such as Williams' REINFORCE{{sfn|Williams|1987}} method (which is known as the likelihood ratio method in the simulation-based optimization literature).{{Sfn|Peters|Vijayakumar|Schall|2003|p=}} Policy search methods have been used in the robotics context.{{sfn|Deisenroth|Neumann|Peters|2013}} Many policy search methods may get stuck in local optima (as they are based on local search).A large class of methods avoids relying on gradient information. These include simulated annealing, cross-entropy search or methods of evolutionary computation. Many gradient-free methods can achieve (in theory and in the limit) a global optimum.Policy search methods may converge slowly given noisy data. For example, this happens in episodic problems when the trajectories are long and the variance of the returns is large. Value-function based methods that rely on temporal differences might help in this case. In recent years, *actorâ€“critic methods*have been proposed and performed well on various problems.WEB,weblink Simple Reinforcement Learning with Tensorflow Part 8: Asynchronous Actor-Critic Agents (A3C), Juliani, Arthur, 2016-12-17, Medium, 2018-02-22,

## Theory

Both the asymptotic and finite-sample behavior of most algorithms is well understood. Algorithms with provably good online performance (addressing the exploration issue) are known.Efficient exploration of large MDPs is largely unexplored (except for the case of bandit problems).{{Clarify|reason=What are bandit problems?|date=August 2017}} Although finite-time performance bounds appeared for many algorithms, these bounds are expected to be rather loose and thus more work is needed to better understand the relative advantages and limitations.For incremental algorithms, asymptotic convergence issues have been settled. Temporal-difference-based algorithms converge under a wider set of conditions than was previously possible (for example, when used with arbitrary, smooth function approximation).## Research

Research topics include- adaptive methods that work with fewer (or no) parameters under a large number of conditions
- addressing the exploration problem in large MDPs
- large-scale empirical evaluations
- learning and acting under partial information (e.g., using predictive state representation)
- modular and hierarchical reinforcement learning
- improving existing value-function and policy search methods
- algorithms that work well with large (or continuous) action spaces
- transfer learning
- lifelong learning
- efficient sample-based planning (e.g., based on Monte Carlo tree search).
- bug detection in software projectsWEB,weblink On the Use of Reinforcement Learning for Testing Game Mechanics : ACM - Computers in Entertainment, cie.acm.org, en, 2018-11-27,

- Actor-critic reinforcement learning

## Comparison of reinforcement learning algorithms {| class"wikitable"

| Sample-means |

Q-learning > | | Q-value |

Stateâ€“actionâ€“rewardâ€“stateâ€“action>SARSA | Stateâ€“actionâ€“rewardâ€“stateâ€“action | Model-Free | On-policy | Discrete | Discrete | Q-value |

Q-learning - Lambda > | | Q-value |

Stateâ€“actionâ€“rewardâ€“stateâ€“action>SARSA - Lambda | Stateâ€“actionâ€“rewardâ€“stateâ€“action with eligibility traces | Model-Free | On-policy | Discrete | Discrete | Q-value |

| Q-value |

| Q-value |

| Advantage |

| Advantage |

| Advantage |

Proximal Policy Optimization>PPO | Proximal Policy Optimization | Model-Free | On-policy | Continuous | Continuous | Advantage |

### Deep reinforcement learning

This approach extends reinforcement learning by using a deep neural network and without explicitly designing the state space.JOURNAL, Vincent, etal, Francois-Lavet, 2018, An Introduction to Deep Reinforcement Learning,weblink Foundations and Trends in Machine Learning, 11, 3â€“4, 219â€“354, 10.1561/2200000071, 1811.12560, The work on learning ATARI games by Google DeepMindJOURNAL, Volodymyr, etal, Mnih, 2015, Human-level control through deep reinforcement learning, Nature, 518, 7540, 529â€“533, 10.1038/nature14236, 25719670, 2015Natur.518..529M, increased attention to deep reinforcement learning or end-to-end reinforcement learning.### Inverse reinforcement learning

In inverse reinforcement learning (IRL), no reward function is given. Instead, the reward function is inferred given an observed behavior from an expert. The idea is to mimic observed behavior, which is often optimal or close to optimal.Ng, A. Y., & Russell, S. J. (2000, June). Algorithms for inverse reinforcement learning. In Icml (pp. 663-670).### Apprenticeship learning

In apprenticeship learning, an expert demonstrates the target behavior. The system tries to recover the policy via observation.## See also

- Temporal difference learning
- Q-learning
- Stateâ€“actionâ€“rewardâ€“stateâ€“action (SARSA)
- Fictitious play
- Learning classifier system
- Optimal control
- Dynamic treatment regimes
- Error-driven learning
- Multi-agent system
- Distributed artificial intelligence

## Footnotes

{{Reflist|30em}}## References

- JOURNAL, Auer, Peter, Jaksch, Thomas, Thomas Jaksch, Ortner, Ronald, Ronald Ortner, 2010, Near-optimal regret bounds for reinforcement learning,weblink Journal of Machine Learning Research, 11, 1563â€“1600, harv, Peter Auer,
- BOOK

, Bertsekas, Dimitri P., Dimitri P. Bertsekas

, Tsitsiklis, John, John Tsitsiklis

, Neuro-Dynamic Programming

, Athena Scientific

, 1996

, Nashua, NH

, 978-1-886529-10-6

,weblink

, harv,

, Tsitsiklis, John, John Tsitsiklis

, Neuro-Dynamic Programming

, Athena Scientific

, 1996

, Nashua, NH

, 978-1-886529-10-6

,weblink

, harv,

- BOOK

, Bertsekas, Dimitri P., Dimitri P. Bertsekas

, Dynamic Programming and Optimal Control: Approximate Dynamic Programming, Vol.II

, Athena Scientific

, 2012

, Nashua, NH

, 978-1-886529-44-1

,weblink

, harv,

, Dynamic Programming and Optimal Control: Approximate Dynamic Programming, Vol.II

, Athena Scientific

, 2012

, Nashua, NH

, 978-1-886529-44-1

,weblink

, harv,

- BOOK,weblink Reinforcement Learning and Dynamic Programming using Function Approximators, Busoniu, Lucian, Babuska, Robert, De Schutter, Bart, Ernst, Damien, Taylor & Francis CRC Press, 2010, 978-1-4398-2108-4, harv, Lucian Busoniu, Robert Babuska, Bart De Schutter, Damien Ernst,
- BOOK, A Survey on Policy Search for Robotics, Deisenroth, Marc Peter, Neumann, Gerhard, Peters, Jan, NOW Publishers, 2013, Foundations and Trends in Robotics, 2, 1â€“142, harv, Marc Peter Deisenroth, Gerhard Neumann, Jan Peters (researcher), 10044/1/12051, JOURNAL

, 10.1023/A:1018056104778

, Bradtke, Steven J., Steven J. Bradtke

, Barto, Andrew G., Andrew G. Barto

, Learning to predict by the method of temporal differences

, Machine Learning

, 22

, 33â€“57

, 1996

, harv, 10.1.1.143.857,

, Bradtke, Steven J., Steven J. Bradtke

, Barto, Andrew G., Andrew G. Barto

, Learning to predict by the method of temporal differences

, Machine Learning

, 22

, 33â€“57

, 1996

, harv, 10.1.1.143.857,

- BOOK,weblink Simulation-based Optimization: Parametric Optimization Techniques and Reinforcement, Gosavi, Abhijit, Springer, 2003, 978-1-4020-7454-7, harv, Abhijit Gosavi, Operations Research/Computer Science Interfaces Series,
- CONFERENCE

, Peters, Jan, Jan Peters (researcher)

, Vijayakumar, Sethu, Sethu Vijayakumar

, Schaal, Stefan, Stefan Schaal

, Reinforcement Learning for Humanoid Robotics

, IEEE-RAS International Conference on Humanoid Robots

, 2003

,weblink

, harv,

, Vijayakumar, Sethu, Sethu Vijayakumar

, Schaal, Stefan, Stefan Schaal

, Reinforcement Learning for Humanoid Robotics

, IEEE-RAS International Conference on Humanoid Robots

, 2003

,weblink

, harv,

- BOOK

, Powell, Warren

, Approximate dynamic programming: solving the curses of dimensionality

, 2007

, Wiley-Interscience

, 978-0-470-17155-4

,weblink

, harv,

, Approximate dynamic programming: solving the curses of dimensionality

, 2007

, Wiley-Interscience

, 978-0-470-17155-4

,weblink

, harv,

- BOOK,weblink Reinforcement Learning: An Introduction, Sutton, Richard S., Barto, Andrew G., MIT Press, 1998, 978-0-262-19398-6, harv, Richard S. Sutton, Andrew Barto,
- JOURNAL

, 10.1007/BF00115009

, Sutton, Richard S., Richard S. Sutton

, Learning to predict by the method of temporal differences

, Machine Learning

, 3

, 9â€“44

, 1988

,weblink

, harv,

, Sutton, Richard S., Richard S. Sutton

, Learning to predict by the method of temporal differences

, Machine Learning

, 3

, 9â€“44

, 1988

,weblink

, harv,

- THESIS, Sutton, Richard S., Temporal Credit Assignment in Reinforcement Learning, PhD, University of Massachusetts, Amherst, MA,weblink Richard S. Sutton, 1984, harv,
- CONFERENCE, Szita, Istvan, Szepesvari, Csaba, Csaba Szepesvari, 2010, Model-based Reinforcement Learning with Nearly Tight Exploration Complexity Bounds,weblink Omnipress, 1031â€“1038, harv, Istvan Szita, ICML 2010, yes,weblink" title="web.archive.org/web/20100714095438weblink">weblink 2010-07-14,
- CONFERENCE

, Williams, Ronald J., Ronald J. Williams

, A class of gradient-estimating algorithms for reinforcement learning in neural networks

, Proceedings of the IEEE First International Conference on Neural Networks

, 1987

, harv, 10.1.1.129.8871,

, A class of gradient-estimating algorithms for reinforcement learning in neural networks

, Proceedings of the IEEE First International Conference on Neural Networks

, 1987

, harv, 10.1.1.129.8871,

- THESIS

, Watkins, Christopher J.C.H., Christopher J.C.H. Watkins

, PhD

, Learning from Delayed Rewards

, 1989

, Kingâ€™s College, Cambridge, UK

,weblink

, harv,

, PhD

, Learning from Delayed Rewards

, 1989

, Kingâ€™s College, Cambridge, UK

,weblink

, harv,

- JOURNAL

, 10.1561/2200000071

, FranÃ§ois-Lavet, Vincent

, Henderson, Peter

, Islam, Riashat

, Bellemare, Marc G.

, Pineau, Joelle

, An Introduction to Deep Reinforcement Learning

, Foundations and Trends in Machine Learning

, 11

, 3â€“4, 219â€“354

, 2018

,weblink

, harv, 1811.12560,

, FranÃ§ois-Lavet, Vincent

, Henderson, Peter

, Islam, Riashat

, Bellemare, Marc G.

, Pineau, Joelle

, An Introduction to Deep Reinforcement Learning

, Foundations and Trends in Machine Learning

, 11

, 3â€“4, 219â€“354

, 2018

,weblink

, harv, 1811.12560,

## Literature

### Conferences, journals

Most reinforcement learning papers are published at the major machine learning and AI conferences (ICML, NIPS, AAAI, IJCAI, UAI, AI and Statistics) and journals (JAIR, JMLR, Machine learning journal, IEEE T-CIAIG). Some theory papers are published at COLT and ALT. However, many papers appear in robotics conferences (IROS, ICRA) and the "agent" conference AAMAS. Operations researchers publish their papers at the INFORMS conference and, for example, in the Operation Research, and the Mathematics of Operations Research journals. Control researchers publish their papers at the CDC and ACC conferences, or, e.g., in the journals IEEE Transactions on Automatic Control, or Automatica, although applied works tend to be published in more specialized journals. The Winter Simulation Conference also publishes many relevant papers. Other than this, papers also published in the major conferences of the neural networks, fuzzy, and evolutionary computation communities. The annual IEEE symposium titled Approximate Dynamic Programming and Reinforcement Learning (ADPRL) and the biannual European Workshop on Reinforcement Learning (EWRL) are two regularly held meetings where RL researchers meet.## External links

- Website for
*Reinforcement Learning: An Introduction*(1998), by Rich Sutton and Andrew Barto, MIT Press, including a link to an html version of the book. - Reinforcement Learning Repository
- Reinforcement Learning and Artificial Intelligence (RLAI, Rich Sutton's lab at the University of Alberta)
- A Beginner's Guide to Deep Reinforcement Learning
- Autonomous Learning Laboratory (ALL, Andrew Barto's lab at the University of Massachusetts Amherst)
- Hybrid reinforcement learning
- Real-world reinforcement learning experiments at Delft University of Technology
- Stanford University Andrew Ng Lecture on Reinforcement Learning
- Dissecting Reinforcement Learning Series of blog post on RL with Python code
- An Introduction to Deep Reinforcement Learning

**- content above as imported from Wikipedia**

- "

- time: 1:38am EDT - Sat, Jul 20 2019

- "

__reinforcement learning__" does not exist on GetWiki (yet)- time: 1:38am EDT - Sat, Jul 20 2019

[ this remote article is provided by Wikipedia ]

LATEST EDITS [ see all ]

GETWIKI 09 JUL 2019

GETWIKI 09 MAY 2016

GETWIKI 18 OCT 2015

GETWIKI 20 AUG 2014

GETWIKI 19 AUG 2014

© 2019 M.R.M. PARROTT | ALL RIGHTS RESERVED