# Optimal Control Discussion

**Led by: Dr. Anil Rao**

**Taken: FS 2015**

**Text: Optimal Control Theory: An Introduction**

*by: D.E. Kirk*

## Table of Contents

## Minimum Principle

- The most general form of finding a minimum value for a function can be stated as
- will return the minimum value of over the input interval [a, b]
- will find the input value that resulted in the minimum value

- The minimum principle can be used to determine the optimal solution, , to the function
- This form is more general of a condition than that provided by the first and second derivative tests
- This would be considered direct optimization because the function is actually evaluated when finding the optimal input

## Functions vs. Functionals

- Functions map real numbers to real numbers whereas functionals map functions to real numbers
- An example of a functional is an integral. It takes a function as an input and maps it to a real number

- When optimizing functions only ordinary calculus is required but when optimizing functionals variational calculus is required

vs.

- is the difference between points in real space
- is variation in a function

- is itself a function

## Calculus of Variations

Simple Example Analogous to Calculus of Variations

- Suppose we want to determine a function that minimizes an integral that is a function of and

- is refered to as the
action integral- and are fixed
- and are fixed
- Let be the optimal solution and the function be determined as follows

- where is arbitrary and is chosen at will
- and because those points are fixed, therefore
- Rewriting with the new definition of gives the following expression

- is now solely a function of and so the critical point can be found at

- is set to zero because that is what results in the optimal solution ()
- Using the definition of the critical point determined above the followin expression emerges

- The following results from taking the derivative of x as given above with respect to

- When substituted into the above expression the following is the result

- Note, however, that is not independent of
- In order to finish solving the equation we will integrate the second term in the integral by parts

- Keeping in mind from earlier that y evaluated at the boundary points is zero the first term in the result goes to zero ()
- We can now rewrite the expression for the critical point as:

- Since the choice of y is arbitrary the portion inside the brackets has to equal zero in order to meet the equality

- This is the result of the analysis, however, note that it does not give a solution, just a condition for optimality

Simple Calculus of Variations Example (same as previous example)

- This will be the same example as the previous one but variational calculus will be used this time
- The problem statement remains the same: suppose we want to determine a function that minimizes an integral that is a function of and

- is refered to as the
action integral- and are fixed
- and are fixed
- The first step this time will be to take the variation of J as the extremal solution is obtained by setting the first variation equal to zero ()

- Now due to the start and end times being fixed the variation can be moved inside the integral as the integral itself will be providing no variation

- The next step is to find the variation of L with respect to its components

- Note that t has dropped from the function L. This is because the integral is definite and therefore t will be integrated out meaning J is not a function of t
- This is the same form of equation that was found above where y is now equivalent to x and is now equivalent to
- Using the same integration by parts method as above and keeping in mind that the boundary conditions are still fixed the following expression can be derived

- Therefore for the condition to hold for any variational function the part in brackets must equal zero

- This result is the same as was previously obtained and is still not a solution but rather a condition for optimality

Advanced Calculus of Variations

- More advanced calculus of variations problems arise when portions of the boundary conditions are no longer fixed (ex. is free or is free)
- One example of this would be a specified start time and position and a specified end position but no specified end time

- , and are fixed
- is free
- Another example would be a specified start time and position and a specified end time but no specified end position

- , and are fixed
- is free

- Before beginning the process to solve the two above examples a differentiation first needs to be made at the boundary of the function

- is the difference in the final value of the variation and the function
- is the difference between the value of the variation at (original not ) and the function
- These two values are related by the expression

First Example

- To begin working through the optimality conditions of the first example listed above we start with the problem statement

- where , and are fixed and is free
- min at
- There are two quantities that vary ( and ) and these two quantities are independent of each other
- Taking the variation of the function just like before gives

- Unlike in the simple calculus of variations example the variation can not simply be moved inside the integral because is now allowed to vary also
- Taking into account the ability of to vary the expression becomes

- The left term of the right hand side of the above expression can be simplified with the following result

- The second term of the expression needs to be simplified using integration by parts just like it was in the simple calculus of variations example

- Since in this problem the term
- Arranging all of the terms back into the expression for the variation of the action integral yields

- Keeping in mind that and are independent of one another, for the above expression to be valid for any varitiation in or both terms have to equal zero

- (refered to as the natural boundary condition)
- We now have optimality conditions for the case with a non-fixed end time

Second Example

- Now for the second advanced calculus of variations example we will have fixed start position and time and a fixed end time but not a fixed end position

- , and are fixed
- is free
- We will again begin by taking the variation of functional and noting that this time because the start and end times are fixed the variation can be moved inside the integral

- The next step moving forward will be to integrate the second term in the integral by parts as has been done in the previous examples

- Putting this result back in the previous equation yields the following expression

- Now considering that the expression has to hold for any variation and both of the terms have to independently equal zero

- We now have optimality conditions for the case with a non-fixed end position

- Notice in both of the advanced calculus of variations examples making a boundary point free resulted in another optimality condition

## Dynamical Systems

- A given system can be defined in each point in time by its state
- The state of the system is affected by the input control on the system
- Note that the components of the state and control are not refered to as the states or controls but rather the component of the state or control
- The states or controls refers to the state or control at multiple time instances

- It is desired to control the motion of the system in a manner that optimizes some performance criteria
- The optimization of a performance criteria is going to be translated into minimizing a cost functional J
- The cost functional can be written as
- The first term of the cost functional is refered to as the Mayer cost or endpoint cost and it represents the constraints that are enforced at the start and terminus
- The second term of the cost functional is refered to as the Lagrange cost, running cost or integrated cost and it represents the constraints that are enforce along the path during motion

- The first term of the cost functional is refered to as the Mayer cost or endpoint cost and it represents the constraints that are enforced at the start and terminus
- The general form of the optimal control problem can then be stated as
- Minimize
- Subject to:

Simple Example Problem

- Minimize

- Subject to:

- fixed
- fixed
- fixed/given
- The first step will be to get the term in the integral in terms of only , and so that we can use the calculus of variations results that have already been derived

- From we find that
- Now from calculus of variations we know the necessary condition for optimality is

- Where
- Solving for the components of the above expression individually yields

- Combining the two components back into the necessary condition for optimality gives the following expression for optimality for this example

- This linear, second-order, constant coefficient, homogeneous differential equation can be solved by making a guess at the solution and solving forward

- Guess for solution x:
- Now substituing the guessed solution back into the original expression and solving for yeilds:

- Keeping in mind that will never equal zero for any value of t the first term must equal zero

- Therefore the general solution of the differential equation is the following expression

- where and are constant coefficients
- Now we will use the boundary conditions given in the problem to solve for the coefficients and

Alternative Differential Equation Solving Method