Dynamic Optimization Winter 2023/24, Simon Haastert, Jörg Lingens, Mark Trede

Dynamic optimization and reinforcement learning

This course will provide participants with an introduction to deterministic and stochastic dynamic optimization as well as to the basics of reinforcement learning.

The aims are (i) to motivate the use of the dynamic optimization techniques (including reinforcement learning) in theoretical and applied micro- and macroeconomic research, (ii) to learn the analytical and numerical methods required to solve the optimization problems, (iii) to apply the techniques to selected empirical applications. As the methods are computationally intensive, the exercises and applications will be done in R.

After the course, participants will: have a solid understanding of the theoretical background of dynamic optimization; be able to program the methods covered in the course to investigate empirical dynamic optimization problems; understand the related techniques they encounter in the literature; appreciate the advantages of reinforcement learning; be able to use the R software.

Content

• Deterministic dynamic optimization: problem setup and motivation in discrete time; Lagrange approach; dynamic programming; Bellman equations; numeric solution strategies; numerical tools for function representations and uni- and multivariate optimization; infinite horizon case.
• Stochastic dynamic optimization: introduction to stochastic processes; Markov processes; numerical tools for integration; problem setup and motivation in discrete time; dynamic programming; Bellman equations; infinite horizon case.
• Reinforcement learning: introduction to the terminology and its relation to classical dynamic optimization; basics of neural networks for representing value functions in high-dimensional spaces.