Optimal Control Discussion

Led by: Dr. Anil Rao
Taken: FS 2015
Text: Optimal Control Theory: An Introduction
by: D.E. Kirk

Table of Contents

Minimum Principle

Functions vs. Functionals

\Delta vs. \delta

Calculus of Variations

Simple Example Analogous to Calculus of Variations

Simple Calculus of Variations Example (same as previous example)

Advanced Calculus of Variations

First Example

  • To begin working through the optimality conditions of the first example listed above we start with the problem statement
    • J = \int^{t_{f}}_{t_{0}} L \left[ x(t), \dot{x}(t), t \right] dt where x(t_{0}), t_{0} and x(t_{f}) are fixed and t_{f} is free
    • min J at \delta J = 0
    • There are two quantities that vary (t_{f} and x(t)) and these two quantities are independent of each other
  • Taking the variation of the function just like before gives
    • \delta J = \delta \int^{t_{f}}_{t_{0}} L \left[ x(t), \dot{x}(t), t  \right] dt = 0
    • Unlike in the simple calculus of variations example the variation can not simply be moved inside the integral because t_{f} is now allowed to vary also
  • Taking into account the ability of t_{f} to vary the expression becomes
    • \delta J = \frac{\partial J}{\partial t_{f}} \delta t_{f} +  \int^{t_{f}}_{t_{0}} \delta L \left[ x(t), \dot{x}(t), t]dt
  • The left term of the right hand side of the above expression can be simplified with the following result
    • \frac{\partial J}{\partial t_{f}} = L \left[ x(t), \dot{x}(t), t  \right] |_{t_{f}} = L \left[ x(t_{f}), \dot{x}(t_{f}), t_{f}  \right]
  • The second term of the \delta J expression needs to be simplified using integration by parts just like it was in the simple calculus of variations example
    • \int^{t_{f}}_{t_{0}} \delta L \left[ x(t), \dot{x}(t), t \right] dt = \int^{t_{f}}_{t_{0}} \left[ \frac{\partial L}{\partial x} \delta x  + \frac{\partial L}{\partial \dot{x}} \delta \dot{x} \right] dt
    • \int^{t_{f}}_{t_{o}} \frac{\partial L}{\partial \dot{x}} \delta  \dot{x} dt = \left[ \frac{\partial L}{\partial \dot{x}} \delta x  \right]^{t_{f}}_{t_{0}} - \int^{t_{f}}_{t_{0}} \frac{d}{dt}  (\frac{\partial L}{\partial \dot{x}})\delta xdt = \frac{\partial  L}{\partial \dot{x}}|_{t_{f}} \delta x(t_{f}) -  \int^{t_{f}}_{t_{0}} \frac{d}{dt} (\frac{\partial L}{\partial  \dot{x}})\delta xdt
  • Since in this problem \delta x_{f} = 0 the term \delta x(t_{f}) =  -\dot{x}(t_{f}) \delta t_{f}
  • Arranging all of the terms back into the expression for the variation of the action integral yields
    • \delta J = L \left[ x(t_{f}), \dot{x}(t_{f}), t_{f} \right] \delta  t_{f} - \frac{\partial L}{\partial \dot{x}}|_{t_{f}} \dot{x}(t_{f})  \delta t_{f} + \int^{t_{f}}_{t_{0}} \left[ \frac{\partial  L}{\partial x} - \frac{d}{dt}\frac{\partial L}{\partial \dot{x}}  \right] \delta x dt = 0
    • \delta J = \left( L \left[ x(t_{f}), \dot{x}(t_{f}), t_{f} \right] -  \frac{\partial L}{\partial \dot{x}}|_{t_{f}} \dot{x}(t_{f}) \right)  \delta t_{f} + \int^{t_{f}}_{t_{0}} \left[ \frac{\partial  L}{\partial x} - \frac{d}{dt}\frac{\partial L}{\partial \dot{x}}  \right] \delta x dt = 0
  • Keeping in mind that \delta t_{f} and \delta x are independent of one another, for the above expression to be valid for any varitiation in t_{f} or x(t) both terms have to equal zero
    • L \left[ x(t_{f}), \dot{x}(t_{f}), t_{f} \right] -  \frac{\partial L}{\partial \dot{x}}|_{t_{f}} \dot{x}(t_{f}) = 0 (refered to as the natural boundary condition)
    • \frac{\partial L}{\partial x} - \frac{d}{dt}\frac{\partial  L}{\partial \dot{x}} = 0
  • We now have optimality conditions for the case with a non-fixed end time t_{f}

Second Example

  • Now for the second advanced calculus of variations example we will have fixed start position and time and a fixed end time but not a fixed end position
    • x(t_{0}), t_{0} and t_{f} are fixed
    • x(t_{f}) is free
  • We will again begin by taking the variation of functional J and noting that this time because the start and end times are fixed the variation can be moved inside the integral
    • \delta J = \delta \int^{t_{f}}_{t_{0}} L \left[ x(t), \dot{x}(t), t  \right] dt
    • \delta J = \int^{t_{f}}_{t_{0}} \delta L \left[ x(t), \dot{x}(t), t  \right] dt
    • \delta J = \int^{t_{f}}_{t_{0}} \left[ \frac{\partial L}{\partial x}  \delta x + \frac{\partial L}{\partial \dot{x}} \delta \dot{x}  \right]dt
  • The next step moving forward will be to integrate the second term in the integral by parts as has been done in the previous examples
    • \int^{t_{f}}_{t_{0}} \frac{\partial L}{\partial \dot{x}} \delta  \dot{x}(t)dt = \left[ \frac{\partial L}{\partial \dot{x}} \delta x  \right]^{t_{f}}_{t_{0}} - \int^{t_{f}}_{t_{0}} \frac{d}{dt}  (\frac{\partial L}{\partial \dot{x}}) \delta x dt
    • \int^{t_{f}}_{t_{0}} \frac{\partial L}{\partial \dot{x}} \delta  \dot{x}(t)dt = \frac{\partial L}{\partial \dot{x}}|_{t_{f}} \delta  x(t_{f}) - \int^{t_{f}}_{t_{0}} \frac{d}{dt} (\frac{\partial  L}{\partial \dot{x}}) \delta x dt
  • Putting this result back in the previous equation yields the following expression
    • \delta J = \frac{\partial L}{\partial \dot{x}}|_{t_{f}} \delta  x(t_{f}) + \int^{t_{f}}_{t_{0}} \left[ \frac{\partial L}{\partial  x} - \frac{d}{dt} (\frac{\partial L}{\partial \dot{x}}) \right]\delta  x dt = 0
  • Now considering that the expression has to hold for any variation \delta  x(t_{f}) and \delta x both of the terms have to independently equal zero
    • \frac{\partial L}{\partial \dot{x}}|_{t_{f}} = 0
    • \frac{\partial L}{\partial x} - \frac{d}{dt} (\frac{\partial  L}{\partial \dot{x}}) = 0
  • We now have optimality conditions for the case with a non-fixed end position x(t_{f})

Dynamical Systems

Simple Example Problem

Alternative Differential Equation Solving Method