An Introduction to Optimal Control Theory: The Dynamic Programming Approach

This book provides a concise and systematic presentation of optimal control theory for large families of deterministic and stochastic systems with discrete or continuous time parameters. It focuses on the dynamic programming technique and is suitable for undergraduate students with basic calculus and linear algebra knowledge, as well as advanced undergraduates and graduate students with mathematical analysis and stochastic processes expertise.

Format: Hardback
Length: 273 pages
Publication date: 23 February 2023
Publisher: Springer International Publishing AG

This comprehensive book delves into the realm of optimal control problems, catering to a wide array of deterministic and stochastic systems with discrete or continuous time parameters. Spanning various disciplines such as Economics, Engineering, Operations Research, and Management Science, among others, it aims to provide a concise, systematic, and self-contained presentation of key topics in optimal control theory. The foundation of the analyses is rooted in the dynamic programming (DP) technique, a versatile tool applicable to almost all control problems encountered in theory and applications.

The book begins with a general introduction to control problems, dividing the topic into four distinct sections, each focusing on different dynamical systems:

Control of discrete-time deterministic systems: This section introduces the fundamentals of optimal control for discrete-time systems, covering topics such as state feedback control, optimal stopping, and dynamic programming. It assumes a basic understanding of elementary calculus, linear algebra, and some concepts from probability theory (random variables, expectations, and so forth).

Discrete-time stochastic systems: Here, the book explores the control of stochastic systems with discrete time, employing techniques such as stochastic differential equations, Markov chains, and optimal control theory. It assumes a working knowledge of mathematical analysis (derivatives, integrals, and so on) and stochastic processes.

Ordinary differential equations: This section delves into the control of ordinary differential equations (ODEs), employing methods such as Pontryagin's maximum principle, Lyapunov's stability theory, and optimal control theory. It assumes a strong foundation in mathematical analysis and differential equations.

General continuous-time MCP with applications for stochastic differential equations: The final section introduces the concept of a general continuous-time MCP, which encompasses a wide range of systems with continuous time parameters. It discusses applications of optimal control theory to stochastic differential equations, including models of financial markets, chemical reactions, and biological systems.

Throughout the book, the authors employ a clear and concise writing style, making the material accessible to undergraduate students with a solid foundation in elementary calculus, linear algebra, and some concepts from probability theory. However, the third and fourth sections are designed for advanced undergraduates or graduate students who possess a deeper understanding of mathematical analysis (derivatives, integrals, and so on) and stochastic processes.

By presenting a comprehensive coverage of optimal control problems across diverse systems, this book serves as a valuable resource for students, researchers, and practitioners in the fields of control theory, mathematics, and applied sciences. It provides a solid foundation for understanding and applying optimal control techniques to real-world problems, enabling individuals to make informed decisions and optimize system performance.

Weight: 610g
Dimension: 160 x 243 x 21 (mm)
ISBN-13: 9783031211386
Edition number: 1st ed. 2023