Dynamic optimisation and reinforcement learning

This course provides an introduction to deterministic and stochastic dynamic optimisation and reinforcement learning, with applications in economics. It combines theory with hands-on implementation, focusing on both analytical and numerical methods.

The aims are (i) to motivate the use of dynamic optimisation techniques (including reinforcement learning) in theoretical and applied micro- and macroeconomic research, (ii) to learn the analytical and numerical methods required to solve the optimisation problems, (iii) to apply the techniques to selected empirical applications. As the methods are computationally intensive, the exercises and applications will be done in R.

By the end of the course, participants will: have a solid understanding of the theoretical background of dynamic optimisation; be able to program the methods covered in the course to investigate empirical dynamic optimisation problems; understand the related techniques they encounter in the literature; appreciate the advantages of reinforcement learning; be able to use the R software.

Content

• Deterministic dynamic optimisation: problem setup and motivation in discrete time; Lagrange approach; dynamic programming; Bellman equations; numeric solution strategies; numerical tools for function representations and uni- and multivariate optimisation; infinite horizon case.
• Stochastic dynamic optimisation: introduction to stochastic processes; Markov processes; numerical tools for integration; problem setup and motivation in discrete time; dynamic programming; Bellman equations; infinite horizon case.
• Reinforcement learning: introduction to the terminology and its relation to classical dynamic optimisation; basics of neural networks for representing value functions in high-dimensional spaces.

Schedule

The course takes place on Tuesdays 14-16 and Thursdays 10-12 in room STA 1 (Am Stadtgraben 9).

Course format

There is no strict separation between lectures and classes. Students are expected to solve (programming) exercises alone or in small groups.

Prerequisites

The course requires intermediate knowledge of economics and basic understanding of statistics and time series analysis. Prior experience with programming concepts (e.g., loops) is helpful; basic working knowledge of R or a similar programming language is necessary.

Administration

• Maximum number of participants: 15
• The course belongs to the "Methods" module of the PhD program but is also open to Master students.
• The course (including the exam) is eligible for 6 credit points.
• The exam consists of a one-week take-home exercise and a 30-minute presentation of the results.