close

Вход

Забыли?

вход по аккаунту

?

Practical Optimization Methods with Mathematica Applications. 2000

код для вставкиСкачать
M. Asghar Bhatti
P r a c t i c a l O p t i m i z a t i o n M e t h o d s
W i t h M a t h e m a t i c a l A p p l i c a t i o n s
M. A s g h a r B h a t t i
P r a c t i c a l O p t i m i z a t i o n
M e t h o d s------------
W i t h M a t h e m a t i c a ® A p p l i c a t i o n s
CD-ROM
INCLUDED
M. Asghar Bhatti Department of Civil and Environmental Engineering University of Iowa Iowa City, IA 52242 USA
mabhatti@uiowa.edu
Library of Congress Cataloging-in-Publication Data Bhatti, M. Asghar
Practical optimization methods : with Mathematica applications / M. Asghar Bhatti. p. cm.
Includes bibliographical references and index.
ISBN 0-387-98631-6 (alk. paper)
1. Mathematical optimization—Data processing. 2. Mathematica (Computer file) I. Title QA402.5.B49 1998
519.3—dc2l 98-31038
Printed on acid-free paper.
Mathematica is a registered trademark of Wolfram Research, Inc.
© 2000 Springer-Verlag New York, Inc.
TELOS®, The Electronic Library of Science, is an imprint of Springer-Verlag New York, Inc.
This Work consists of a printed book and a CD-ROM packaged with the book, both of which are protected by federal copyright law and international treaty. The book may not be translated or copied in whole or in part without the written permission of the publisher (Sprmger-Verlag New York, Inc., 175 Fifth Avenue, New York, NY 10010, USA), except for brief excerpts in connection with reviews or scholarly analysis. For copyright information regarding the CD-ROM, please consult the printed information packaged with the CD-ROM in the back of this publication, and which is also stored as a “readme" file on the CD-ROM. Use of the printed version of this Work in connection with any form of information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known, or hereafter developed, other than those uses expressly granted in the CD-ROM copyright notice and disclaimer information, is forbidden.
The use of general descriptive names, trade names, trademarks, etc., in this publication, even if the former are not especially identified, is not to be taken as a sign that such names, as understood by the Trade Marks and Merchandise Marks Act, may accordingly be used freely by anyone. Where those designations appear in the book and Springer-Verlag was aware of a trademark claim, the designations follow the capitalization style used by the manufacturer.
Production managed by Steven Pisano; manufacturing supervised by Jerome Basma.
Typeset by Integra Technical Publishing Co., Inc., Albuquerque, NM.
Printed and bound by Hamilton Printing Co., Rensselaer, NY.
Printed in the United States of America.
987654321
ISBN 0-387-98631-6 Springer-Verlag New York Berlin Heidelberg SPIN 10693839
Preface
The goal of this book is to present basic optimization theory and modem computational algorithms in a concise manner. The book is suitable for un­
dergraduate and graduate students in all branches of engineering, operations research, and management information systems. The book should also be use­
ful for practitioners who are interested in learning optimization and using these techniques on their own.
Most available books in the field tend to be either too theoretical or present computational algorithms in a cookbook style. An approach that falls some­
where in between these two extremes is adopted in this book. Theory is pre- sented in an informal style to make sense to most undergraduate and graduate students in engineering and business. Computational algorithms are also de­
veloped in an informal style by appealing to readers’ intuition rather than mathematical rigor.
The available, computationally oriented books generally present algorithms alone and expect readers to perform computations by hand or implement these algorithms by themselves. This obviously is unrealistic for a usual introductory optimization course in which a wide variety of optimization algorithms are discussed. There are some books that present programs written in traditional computer languages such as Basic, FORTRAN, or Pascal. These programs help with computations, but are of limited value in developing understanding of the algorithms because veiy little information about the intermediate steps
P r e f a c e
is presented. The user interface for these programs is primitive at best and typically requires creating user-defined functions and data files defining a specific problem.
This book fully exploits Mathematica’s symbolic, numerical, and graphical capabilities to develop thorough understanding of optimization algorithms. The problems are defined in the form of usual algebraic expressions. By using Mathematica's symbolic manipulation capabilities, appropriate expressions for intermediate results are generated and displayed in customary style. Mathe­
matical graphical capabilities are used in presenting the progress of numerical algorithms towards the optimum, thus creating a powerful environment for visual comparison of different optimization algorithms.
However, a knowledge of Mathematica is not a prerequisite for benefiting from this book. With its numerous examples and graphical illustrations, the book should be useful to anyone interested in the subject. Those familiar with Mathematica can further benefit by msing the software on the accompanying CD to solve a variety of problems on their own to further develop their under­
standing. An appendix in included is the book for those interested in a concise introduction to Mathematica.
The arrangement of the book is fairly typical of other books covering sim­
ilar topics. Methods applicable to special classes of problems are presented before those that are more general but computationally expensive. The meth­
ods presented in earlier chapters are more developed than those in the later chapters.
General guidelines for formulating problems in the form suitable for an op­
timization algorithm are presented in Chapter 1. The ideas are illustrated by several examples. Since cost plays an important role in many optimum design formulations, the chapter also contains a brief introduction to the time value of money. Optimization problems involving two variables can be solved very effectively using a graphics method. This topic is covered in detail in Chapter
2. A unique Mathematica implementation for a graphics solution is presented in this chapter also. Some of the basic mathematical concepts useful in de­
veloping optimization theory are presented in Chapter 3. Chapter 4 presents necessaiy and sufficient conditions for optimality of unconstrained and con­
strained problems. A Mathematica function is developed that uses necessary conditions to solve small-scale optimization problems. Chapter 5 presents nu­
merical methods for solving unconstrained optimization problems. Several one-dimensional line-search methods are also discussed. All methods are im­
plemented in the form of Mathematica functions. Chapter 6 presents a well- known simplex method for solving linear programming problems. Both the tableau form and the revised forms are discussed in detail. These problems
P r e f a c e
can be solved using built-in Mathematica functions; however, equivalent new functions are implemented to show all the computational details. The chapter also includes a discussion of post-optimality analysis and presents Mathemat­
ica functions implementing these ideas. Interior point methods have gained popularity for solving linear and quadratic programming problems. Chapter 7 presents two simple but fairly successful methods belonging to this class for solving linear programming problems. Chapter 8 presents extensions of interior point methods for solving convex quadratic programming problems. This chapter also includes solution methods for special forms of quadratic programming problems that are generated as intermediate problems when solving more general constrained optimization problems. Chapter 9 presents numerical algorithms for solving general nonlinearly constrained problems. TWo methods, generally considered among the best in their class, the aug­
mented Lagrangian penalty function method, and the sequential quadratic programming method, are implemented as Mathematica functions and several numerical examples are presented.
The accompanying CD contains the implementation of all Mathematica functions discussed in the book. Tbgether, these functions constitute what is referred to as the Optimizationlbolbox in the text. The functions are written to follow the general steps presented in the text. Computational efficiency was not a major goal in creating these functions. Options are built in many func­
tions to show intermediate calculations to enhance their educational value. The CD also contains additional examples showing the use of these Mathemat­
ica functions.
All chapters contain problems for homework assignments. Solutions to most homework problems can be generated using the functions defined in the Op- timizationlbolbox.
The author has been involved in teaching optimization methods to under- graduate engineering students for the past fifteen years. The book grew out of this experience. The book includes more material than what can be covered in a one-semester course. By selecting the most suitable material from this book, and possibly supplementing it with other material, a wide variety of courses can be taught. The first two chapters of the book can be incorporated into any introductory undergraduate engineering design course to give students a quick introduction to optimum design concepts. A one-semester undergradu­
ate course on optimum design can be developed to cover Chapters 1, 2, 3, and 4 (excluding the last section), and parts from Chapters 5 through 9. A grad­
uate course on optimum design can cover the entire book in one semester. A course suitable for management information and operations research stu- dents can cover material in Chapters 1 through 4 and 6 through 8 of the book.
P r e f a c e
Practicing engineers can become proficient users of optimization techniques in everyday design by studying the first two chapters followed by Chapters 5 and 9.
Installing OptimizationTbolbox
The OptimizationTbolbox consists of the following Mathematica packages.
1. CommonFunctions.m
2. EconomicFactors.m
3. GraphicalSolution.m
4. Chap3Tbols.m
5. OptimalityConditions.m
6. Unconstrained.m
7. LPSimplex.m
8. InteriorPoint.m
9. QuadraticProgramming.m 10. ConstrainedNLP.m
The CommonFunctions.m package contains functions that are common to sev­
eral other packages. The other nine packages implement functions described in each of the nine chapters of the book.
All these files must be placed in a folder (directory) called OptimizationTbol- box. The best place to put this folder is inside the Mathematica 3.0/Add0ns/ Applications folder. These packages can then be imported (loaded) into any Mathematica notebook by simply executing a Needs command. There is no need to explicitly load the CommonFunctions.m package. It is loaded auto­
matically when any of the other packages are loaded. For example, to import all functions defined in the OptimalityConditions.m package, one needs to execute the following command:
Meeds["O p t im iz a t io n T o o lb o x'O p t im a lit y C o n d it lo n s ‘;
P r e f a c e
Using OptimizationTbolbox
The CD includes Mathematica notebooks containing additional examples for each chapter. These notebooks are identified as ChaplCD.nb, Chap2CD.rib, and so forth. Each notebook contains a line for loading appropriate packages for that chapter. The rest of the notebook is divided into sections in the same way they appear in the book. If applicable, for each section, solutions of sample problems are provided using the appropriate OptimizationTbolbox functions. Homework problems or any other problem can be solved by simply copying/pasting these examples and making appropriate modifications to define new problems.
M . Asghar Bhatti
Contents
Preface v
1 Optimization Problem Formulation 1
1.1 Optimization Problem Formulation........................................ 2
1.2 The Standard Form of an Optimization Problem..................... 13
1.3 Solution of Optimization Problems........................................ 16
1.4 Time Value of Money............................................................ 18
1.5 Concluding Remarks............................................................ 31
1.6 Problems.............................................................................. 32
2 Graphical Optimization 47
2.1 Procedure for Graphical Optimization.................................. 48
2.2 G r a p h i c a l S o l u t i o n F u n c t i o n ............................................................... 57
2.3 Graphical Optimization Examples.......................................... 59
2.4 Problems.............................................................................. 70
3 Mathematical Preliminaries 75
3.1 Vectors and Matrices............................................................ 75
3.2 Approximation Using the T&ylor Series.................................. 89
3.3 Solution of Nonlinear Equations............................................. 100
C o n t e n t s
3.4 Quadratic Forms ................................................................. 106
3.5 Convex Functions and Convex Optimization Problems........... 112
3.6 Problems.............................................................................. 125
4 Optimality Conditions 131
4.1 Optimality Conditions for Unconstrained Problems................ 132
4.2 The Additive Property of Constraints..................................... 144
4.3 Karush-Kuhn-TUcker (KT) Conditions..................................... 147
4.4 Geometric Interpretation of KT Conditions—........................... 165
4.5 Sensitivity Analysis............................................................... 175
4.6 Optimality Conditions for Convex Problems........................... 181
4.7 Second-Order Sufficient Conditions........................................ 187
4.8 Lagrangian Duality............................................................... 199
4.9 Problems.............................................................. 208
5 Unconstrained Problems 227
5.1 Descent Direction................................................................. 229
5.2 Line Search Techniques—Step Length Calculations................ 231
5.3 Unconstrained Minimization Techniques . .................. 253
5.4 Concluding Remarks............................................................ 302
5.5 Problems.............................................................. 303
6 Linear Programming 315
6.1 The Standard LP Problem.................................................... 316
6.2 Solving a Linear System of Equations..................................... 319
6.3 Basic Solutions of an LP Problem.......................................... 334
6.4 The Simplex Method............................................................ 339
6.5 Unusual Situations Arising During the Simplex Solution .... 365
6.6 Post-Optimality Analysis....................................................... 376
6.7 The Revised Simplex Method................ 387
6.8 Sensitivity Analysis Using the Revised Simplex Method .... 402
6.9 Concluding Remarks............................................................ 420
6.10 Problems.............................................................................. 421
7 Interior Point Methods 437
7.1 Optimality Conditions for Standard LP.................................. 438
7.2 The Primal Affine Scaling Method....................................... 445
7.3 The Primal-Dual Interior Point Method ................................ 464
7.4 Concluding Remarks............................................................ 481
C o n t e n t s
7.5 Appendix—Null and Range Spaces....................................... 481
7.6 Problems.............................................................................. 486
8 Quadratic Programming 495
8.1 KT Conditions for Standard Q P............................................. 495
8.2 The Primal Affine Scaling Method for Convex QP.................... 502
8.3 The Primal-Dual Method for Convex QP................................ 520
8.4 Active Set Method................................................................. 535
8.5 Active Set Method for the Dual QP Problem.......................... 552
8.6 Appendix—Derivation of the Descent Direction Formula for
the PAS Method...................................................... 563
8.7 Problems.............................................................................. 573
9 Constrained Nonlinear Problems 581
9.1 Normalization...................................................................... 582
9.2 Penalty Methods ................................................................. 585
9.3 Linearization of a Nonlinear Problem..................................... 608
9.4 Sequential Linear Programming—SLP.................................. 614
9.5 Basic Sequential Quadratic Programming— SQP..................... 620
9.6 Refined SQP Methods ... *.................................................. 645
9.7 Problems.............................................................................. 660
Appendix An Introduction to Mathematica 677
A.l Basic Manipulations in Mathematica..................................... 678
A.2 Lists and Matrices................................................................. 682
A.3 Solving Equations................................................................. 689
A.4 Plotting in Mathematica......................................................... 691
A.5 Programming in Mathematica............................................... 695
A.6 Packages in Mathematica....................................................... 702
A.7 Online Help......................................................................... 703
B i b l i o g r a p h y 7 0 5
I n d e x
7 0 9
CHAPTER ONE
Optimization Problem Formulation
Optimization problems arise naturally in many different disciplines. A struc­
tural engineer designing a multistory building must choose materials and pro­
portions for different structural components in the building in order to have a safe structure that is as economical as possible. A portfolio manager for a large mutual fund company must choose investments that generate the largest possible rate of return for its investors while keeping the risk of major losses to acceptably low levels. A plant manager in a manufacturing facility must sched­
ule the plant operations such that the plant produces products that maximize company's revenues while meeting customer demands for different products and staying within the available resource limitations. A scientist in a research laboratory may be interested in finding a mathematical function that best describes an observed physical phenomenon.
All these situations have the following three things in common.
1. There is an overall goal, or objective, for the activity. For the structural engineer, the goal may be to minimize the cost of the building, for the portfolio manager it is to maximize the rate of return, for the plant manager it is to maximize the revenue, and for the scientist, the goal is to minimize the difference between the prediction from the mathematical model and the physical observation.
C h a p t e r 1 O p t i m i s a t i o n P r o b l e m F o r m u l a t i o n
2. In addition to the overall goal, there usually are other requirements, or constraints, that must he met. The structural engineer must meet safety requirements dictated by applicable building standards. The portfolio man­
ager must keep the risk of major losses below levels determined by the company's management. The plant manager must meet customer demands and work within available work force and raw material limitations. For the laboratory scientist, there are no other significant requirements.
3. Implicit in all situations is the notion that there are choices available that, when made properly, will meet the goals and requirements. The choices are known as optimization or design variables. The variables that do not affect the goals are clearly not important. For example, from a structural safety point of view, it does not matter whether a building is painted purple or pink, and therefore the color of a building would not represent a good optimization variable. On the other hand, the height of one story could be a possible design variable because it will determine overall height of the building, which is an important factor in ascertaining structural safety.
1.1 Optimization Problem
Formulation of an optimization problem involves taking statements, defining general goals and requirements of a given activity, and transcribing them into a series of well-defined mathematical statements. More precisely, the formulation of an optimization problem involves:
1. Selecting one or more optimization variables,
2. Choosing an objective function, and
3. Identifying a set of constraints.
The objective function and the constraints must all be functions of one or more optimization variables. The following examples illustrate the process.
1.1.1 B u i l d i n g D e s i g n
Tb save eneigy costs for heating and cooling, an architect is considering de­
signing a partially buried rectangular building. The total floor space needed is 20,000 m2. Lot size limits the building plan dimension to 50 m. It has already
1.1 O p t i m i z a t i o n P r o b l e m
been decided that the ratio between the plan dimensions must be equal to the golden ratio (1.618) and that each story must be 3.5 m high. The heating and cooling costs are estimated at $100 per m2 of the exposed surface area of the building. The owner has specified that the annual energy costs should not exceed $225,000. Formulate the problem of determining building dimensions to minimize cost of excavation.
Optimization Variables
Fiom the given data and Figure 1.1, it is easy to identify the following variables associated with the problem:
n = Number of stories d = Depth of building below ground h = Height of building above ground i = Length of building in plan w = Width of building in plan
FIGURE 1.1 Partially buried building.
O b j e c t i v e F u n c t i o n
The stated design objective is to minimize excavation cost. Assuming the cost of excavation to be proportional to the volume of excavation, the objective function can be stated as follows:
Minimize dtw
Constraints________________________________________________
All optimization variables are not independent. Since the height of each story is given, the number of stories and the total height are related to each other as follows:
tl
Also, the requirement that the ratio between the plan dimensions must be equal to the golden ratio makes the two plan dimensions dependent on each other as follows:
--------------------------------------1 = 1.618w--------------------------------------
The total floor space is equal to the area per floor multiplied by the number of
stories. Thus, the floor space requirement can be expressed as follows:
;
niw > 20,000
The lot size places the following limits on the plan dimensions:
£ <50 w < 50
The energy cost is proportional to the exposed building area, which includes the areas of the exposed sides and the roof. Thus, the energy budget places the following restriction, on the design:
100(2fc£ + 2hw + tw) < 225,000
Tb make the problem mathematically precise, it is also necessary to explicitly state that the design variables cannot be negative.
i,w,h,d>0 n > 1, must be an integer
The complete optimization problem can be stated as follows:
Find (η, I, w, h, d) in order to Minimize dlw
Subject to
^ = 3.5 ί = 1.618w ntw > 20,000 t < 50 w < 50
100(2W + 2hw + iw) < 225,000
\
n> 1 £, w,h, d > 0
1,1.2 P l a n t O p e r a t i o n
A tire manufacturing plant has the ability to produce both radial and bias-ply automobile tires. During the upcoming summer months, they have contracts to deliver tires as follows.
Date
Radial tires
Bias-ply tires
June 30 July 31 August 31 Tbtal
5.000
6.000 4,000
15,000
3.000
3.000
5.000
11.000
The plant has two types of machines, gold machines and black machines, with appropriate molds to produce these tires. The following production hours are available during the summer months:
Month
Gold machines
Black machine
June
700
1,500
July
August
300
1,000
400
300
The production rates for each machine type and tire combination, in terms of hours per tire, are as follows:
Type
Gold machines
Black machines
Radial
0.15
0.16
Bias-ply
0.12
0.14
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
The labor costs of producing tires are $10.00 per operating hour, regardless of which machine type is being used or which tire is being produced. The material costs for radial tires are $5.25 per tire, and those for bias-ply tires are $4.15 per tire. Finishing, packing, and shipping costs are $0.40 per tire. The excess tires are carried over into the next month but are subjected to an inventory carrying charge of $0.15 per tire. Wholesale prices have been set at $20 per tire for radials and $15 per tire for bias-ply.
How should the production be scheduled in order to meet the delivery requirements while maximizing profit for the company during the three-month period?
Optimization Variables
From the problem statement, it is clear that the only variables that the pro­
duction manager has control over are the number and type of tires produced on each machine type during a given month. Thus, the optimization variables are as follows:
*1
Number of radial tires produced in June on the gold machines
*2
Number of radial tires produced in July on the gold machines
*3
Number of radial tires produced in August on the gold machines
*4
Number of bias-ply tires produced in June on the gold machines
*5
Number of bias-ply tires produced in July on the gold machines
*6
Number of bias-ply tires produced in August on the gold machines
*7
Number of radial tires produced in June on the black machines
*8
Number of radial tires produced in July on the black machines
*9
Number of radial tires produced in August on the black machines
*10
Number of bias-ply tires produced in June on the black machines
*11
Number of bias-ply tires produced in July on the black machines
*12
Number of bias-ply tires produced in August on the black machines
Objective Function
The objective of the company is to maximize profit. The profit is equal to the total revenue from sales minus all costs associated with the production,
storing, and shipping.
Revenue from sales
= $20(χι + X2 + *3 + *7 + *8 + *9) + $15(*4 + *5 + *6 + *10 + *11 + *12) Material costs
= $5.25(xi + *2 + *3 + *7 + *8 + *9)
+ $4.15(*4 + * 5 + xq + *10 + *11 + *12)
Labor costs
= $10(0.15(*i + *2 + *3) + 0.16(*7 + *8 + * 9) + 0.12(x4 + x5 + x&)
+ 0.14 (*10 + *11 +*12))
Finishing, packing, and shipping costs
= $0.40(*i + *2 + *3 + *4 + *5 + *6 + *7 + *8 + *9 + *10 + *11 + *12)
The inventory-carrying charges are a little difficult to formulate. Assum­
ing no inventory is carried into or out of the three summer months, we can determine the excess tires produced as follows:
Excess tires produced by June 30
= (x\ +x? — 5,000) + (*4 +*io — 3,000)
Excess tires produced by July 31
= (*i + *2 + x? + *8 ~ 11,000) + (X4 + * 5 -f- *10 + *11 — 6,000)
By assumption, there are no excess tires left by the end of August 31. At $0.15 per tire, the total inventory-carrying charges are as follows:
Inventory cost
= $0.15((χι + χγ — 5,000) + (*4 + *io — 3,000)
+ (*1 + *2 + Xrj + *8 - 11,000) + (*4 + *5 + *10 + *11 ~ 6,000))
By adding all costs and subtracting from the revenue, the objective function to be maximized can be written as follows:
Maximize 3,750 + 12.55*i + 12.7*2 + 12.85x3 + 8.95*4 + 9.1*5 + 9.25x6
+ 12.45*7 + 12.6*8 + 12.75*9 + 8.75xio + 8.9xn + 9.05*12
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
Constraints
In a given month, the company must meet the delivery contracts. During July and August, the company can also use the excess inventory to meet the demand. Thus, the delivery contract constraints can be written as follows:
*1 + *7 5,000 *4 + *io ^ 3,000
ί ~ 1 . I · . ,
χ\ + *2 +X7 +*8 > 11,000 X4 +X5 +*lo + *11 > 6,000 Xl + X2 + *3 + *7 + XQ + *9 = 15,000 *4 +*5 +*8 +*10 +*11 +*12 = 11,000
Note that the last two constraints are expressed as equalities to stay consistent with the assumption that no inventory is carried into September.
The production hours for each machine are limited. Using the time that it takes to produce a given tire on a given machine type, these limitations can be expressed as follows:
O.lSxj + 0.12*4 < 700
0.15*2 +0.12*5 — 300
0.15*3 + 0.12*6 < 1,000
1 U>1 1Λ|Π 1
0.16*8 + 0.14*ii <400
0.16*9 +0.14*12 < 300
The only other requirement is that all optimization variables must be positive. The complete optimization problem can be stated as follows:
Find (*1 ,X2,*12) in order to
Maximize 3,750 + 12.55*i + 12.7*2 + 12.85*3 + 8.95*4 + 9.1*5 + 9.25*8 + 12.45*7 + 12.6*8 + 12.75*9 + 8.75*io + 8.9*n + 9.05*12
1.1 O p t i m i z a t i o n P r o b l e m
x\ + * 7 > 5,000 *4 +*10 ^ 3,000 *1 + *2 + *7 + *8 > 11,000 *4 + *5 +*10 +*11 > 6,000
*1 + *2 +* 3 +* 7 +* 8 + * 9 = 15,000 *4 + *5 + *8 + *10 + *11 + *12 = 11,000 Subject to 0.15*1 + 0.12x4 < 700
0.15*2+0.12x5 < 300 0.15*3 + 0.12*6 < 1,000 O.I6X7 + 0.14*io < 1,500 0.16*8 + 0.14*ii < 400 0.16*9+0.14*12 < 300 *i,*2, ...,*12 > 0
1.1.3 P o r t f o l i o M a n a g e m e n t
A portfolio manager for an investment company is looking to make investment decisions such that investors will get at least a 10 percent rate of return while minimizing the risk of major losses. For the past six years the rates of return in four major investment types are as follows:
Type
Annual rates of return
/ Year
1
2
3
4
5
6
Average
Blue chip stocks
18.24
12.12
15.23
5.26
2.62
10.42
10.6483
Technology stocks
12.24
19.16
35.07
23.46
-10.62
-7.43
11.98
Real estate
8.23
8.96
8.35
9.16
8.05
7.29
8.34
Bonds
8.12
8.26
8.34
9.01
9.11
8.95
8.6317
Optimization Variables
The portfolio manager must decide what percentage of the total capital to invest in each investment type. Thus, the optimization variables are as follows:
XI
Portion of capital invested in blue-chip stocks
*2
Portion of capital invested in technology stocks
*3
Portion of capital invested in real estate
X4
Portion of capital invested in bonds
Objective Function
The objective is to minimize risk of losses. A measure of this risk is the amount of fluctuation in the rate of return from its average value. The variance of investment j is defined as follows:
1 ”
. k=i
where tt = total number of observations, iy* = rate of return of investment j for the kth observation (year in the example), and μ-j is the average value of the investment j. Using the numerical data given for this example, the variances are computed as follows:
vn = -[(18.24 - 10.6483)2 + · · · + (10.42 - 10.6483)2] = 29.0552 6
Similarly,
V22 = 267.344 V33 = 0.3759 t/44 = 0.1597
From the definition, it is clear that the variance measures the risk within one investment type. Tb measure risk among different investment types, we define covariance between two investments i and j as follows:
1 "
vij = ~ ^ ,irik ~ {M)(.rjk ~ A6/)
” k=l
Usi ng t he numeri cal data gi ven for t hi s exampl e, t he covari ances are comput ed as fol l ows:
ΙΊ2 = t;[(18.24 - 10.6483) ( 12.24 - 11.98) + · · ■
6
- I - ( 1 0.4 2 - 1 0.6 4 8 3 ) ( - 7.4 3 - 1 1.9 8 ) ]
= 4 0.3 9 0 9
1.1 O p t i m i z a t i o n P r o b l e m
Similarly, other covariances can easily be computed. All variances and covari­
ances can be written in a covariance matrix as follows:
V\\ v\%V1 3 VU
/ 29.0S.S2 4 0.3 9 0 Q - ( 1 7.8 7 8 8 3 - 1.QS 3 7. \
V ~
Vz\ V21 V23 ^24
1/31 i>32 ^33 I/34
Vf41 1^42 ^43 ^44 /
40.3909 267.344 6.83367 -3.69702
-0.287883 6.83367 0.375933 -0.0566333
V -1.9532 -3.69702 -0.0566333 0.159714
Using the covariance matrix, the investment risk is written as the following function:
Risk = Ol X2 *3 *4)
ί 29.0552 40.3909· -0.287883 V -1.9532
40.3909
267.344
6.83367
-0.287883
6.83367
0.375933
-3.69702 -0.0566333
-1.9532 N -3.69702 -0.0566333 0.159714
Carrying out the matrix products, the objective function can be written explic­
itly as follows.
Minimize / = 29.0552xf + 80.7818x2*1 — 0.575767X3X1 - 3.90639x^*1 + 267.344*2 + 0.375933x3 + 0.159714x4 + 13.6673X2*3 — 7.39403X2X4 — 0.113267X3X4
{ χΐ \ X2 X3 V * 4/
= xTVx
Constraints
Since the selected optimization variables represent portions of total invest- ment, their sum must equal the entire investment. Thus,
Xl +*2+*3+X4 = l
The second constraint represents the requirement that the desired average rate of return must be at least 10 percent. It can be written as follows:
10.6483X1 + 11.98X2 + 8.34x3 + 8.6317X4 > 10
All optimization variables must also be positive.
_______________________ Xi > 0 t = 1........4
The complete optimization problem can be stated as follows:
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
Find (χι, X2, · · ·, *4) in order to
Minimize 29.0552x ^ +80.7818x2*1 —0.575767*3X1 — 3.90639X4X1+ 267.344x2 + 0.375933x3 + 0.159714*4 + 13.6673x2xs - 7.39403x2x4 - 0.113267x3x4
(
x i + X2 + X3 + X4 = 1 \
10.6483xi + 11.98x2 + 8.34x3 + 8.63l7x4 > 10 J xi,x2, ...,*4 > 0 /
1.1.4 D a t a F i t t i n g
Finding the best function that fits a given set of data can be formulated as an optimization problem. As an example, consider fitting a surface to the data given in the following table:
Point
X
y
^observed
I
0
1
1.26
2
0*25
1
2*19
3
0.5
1
0.76
4
0,75
1
1.26
5
1
2
1.86
6
1.25
2
1.43
7
1.5
2
1.29
8
1.75
2
0.65
9
2
2
1.6
The form of the function is first chosen based on prior knowledge of the overall shape of the data surface. For the example data, consider the following general form
zcomputed = Cl*2 + C2y2 + C$Xy
T h e g o a l n o w i s t o d e t e r m i n e t h e b e s t v a l u e s o f c o e f f i c i e n t s c\t C2, and C3 in order to minimize the sum of squares of error between the computed z yalues and the observed ones.
Optimization Variables
Values of coefficients c\, 02, and c3
1.2 T h e S t a n d a r d F o r m o f a n O p t i m i z a t i o n P r o b l e m
Objective Function
Minimize f = [z0bserved(*ij Yi) ~ ^computed (xii yi)]2
Using the given numerical data, the objective function can be written as fol­
lows:
/ = (1.26 — C2)2 + (2.19—0.0625ci - c2 - 0.25c3)2 + · ■.
+ ( 1.6 - 4ci - 4 q - 4c3)2
or
/ = 18.7 - 32.8462ci + 34.2656^ - 65.58c2 + 96.75cic2 + 84c\ - 43.425c3 + 79.875cic3 -I- 123.c2c3 + 48.375c2 The complete optimization problem can be stated as follows:
Find (ci, ci, and C3) in order to
Minimize 18.7—32.8462ci -I- 34.2656^—65.58c2 +96.75cic2+84c2 — 43.425c3+ 79.875^x^3 -f- I23.C2C3 48.375^3
This example represents a simple application from a wide field known as Regression Analysis. For more details refer to many excellent books on the subject, e.g., Bates and Watts [1988].
1.2 The Standard Form o f an Optimization Problem
As seen in the last section, a large class of situations involving optimization can be expressed in the following form:
Find a vector of optimization variables, x = (x\, x2,..., xn)T in order to Minimize an objective (or cost) function, / (jc)
Subject to
gi(x) < 0 i = 1, 2,..., m Less than type inequality constraints (LE)
hi(x) = 0 i = l,2,...,p Equality constraints (EQ)
xn < Xi < Xiu i = 1, 2,..., n Bounds on optimization variables
The bounds constraints are in fact inequality constraints. They are sometimes kept separate from the others because of their simple form. Certain numerical optimization algorithms take advantage of their special form in making the
computational procedure efficient.
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
An equality constraint represents a given relationship between optimization variables. If the constraint expression is simple, it maybe possible to solve for one of the variables from the equality constraint equation in terms of the remaining variables. After this value is substituted in all remaining constraints and the objective function, we have a problem with one less optimization variable. Thus, if the number of equality constraints is equal to the number of optimization variables, i.e., p — rt, the problem is not of an optimization form. We just need to solve constraint equations to get the desired solution. If ρ > n, the problem formulation is not correct because some of the constraints must be dependent on others and therefore are redundant. Thus, for a general optimization problem, p must be less than n. Since the inequality constraints do not represent a specific relationship among optimization variables, there is no limit on the number of inequality constraints in a problem.
The problem as stated above will be referred to as the standard form of an optimization problem in the following chapters. Note that the standard form considers only minimization problems. It will be shown later that a maximiza­
tion problem can be converted to a minimization problem by simply multi­
plying the objective function by a negative sign. Furthermore, the standard form considers only the less than type of inequality constraints. If a constraint is actually of the greater than type (GE), it can be converted to a less than type by multiplying both sides by a negative sign. Thus, the standard form is more general than it appears at first glance. In fact, it can handle all situations considered in this text.
1.2.1 M u l t i p l e O b j e c t i v e F u n c t i o n s
Sometimes there is more than one objective to be minimized or maximized. For example, we may want to maximize profit from an automobile that we are designing and at the same time minimize the possibility of damage to the car during a collision. These types of problems are difficult to handle because the objective functions are often contradictory.
One possible strategy is to assign weights to each objective function de­
pending on their relative importance and then define a composite objective function as a weighted sum of all these functions, as follows:
/(jc) - wifijx) + W2 fi( * ) H-------
where W\,W2>... are suitable weighting factors. The success of the method clearly depends on a clever choice of these weighting factors.
1.2 T h e S t a n d a r d F o r m o f a n O p t i m i z a t i o n P r o b l e m
Another possibility is to select the most important goal as the single objective function and treat others as constraints with reasonable limiting values.
1.2.2 C l a s s i f i c a t i o n o f O p t i m i z a t i o n P r o b l e m s
The methods for solving the general form of the optimization problem tend to be complex and require considerable numerical effort. Special, more efficient methods are available for certain specials forms of the general problem. For this purpose, the optimization problems are usually classified into the following types.
Unconstrained Problems
These problems have an objective function but no constraints. The data-fitting problem, presented in the first section, is an example of an unconstrained optimization problem. The objective function must be nonlinear (because the minimum of an unconstrained linear objective function is obviously —oo). Problems with simple bounds on optimization variables can often be solved first as unconstrained. After examining different options, one can pick a solu­
tion that satisfies the bounds on the variables.
Linear Programming (LP) Problems
If the objective function and all the constraints are linear functions of opti­
mization variables, the problem is called a linear programming problem. The tire plant management problem, presented in the first section, is an example of a linear optimization problem. An efficient and robust algorithm, called the Simplex method, is available for solving these problems.
Quadratic Programming (QP) Problems
If the objective function is a quadratic function and all constraint functions are linear functions of optimization variables, the problem is called a quadratic programming problem. The portfolio management problem, presented in the first section, is an example of a quadratic optimization problem. It is possible to solve QP problems using extensions of the methods for LP problems.
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
Nonlinear Programming (NLP) Problems
The general constrained optimization problems, in which one or more func­
tions are nonlinear, are called nonlinear programming problems. The building design problem, presented in the first section, is an example of a general nonlinear optimization problem.
1.3 Solution o f Optimization Problems
Different solutions discussed in this text are as follows:
1.3.1 G r a p h i c a l O p t i m i z a t i o n
For problems involving two optimization variables, it is possible to obtain a so­
lution by drawing contours of constraint functions and the objective function. This is a general and powerful method and also gives a great deal of insight into the space of all feasible choices. However, since the results are to be read from graphs, the method cannot give very precise answers. Furthermore, it is obviously not possible to use a graphical method for problems involving more than two optimization variables. Chapter 2 presents details of the graphical optimization procedure.
1.3.2 O p t i m a l i t y C r i t e r i a M e t h o d s
By extending the ideas of minimization of a function of a single variable — treated in elementary calculus textbooks—it is possible to develop necessary conditions for unconstrained and constrained optimization problems. When these necessary conditions are applied to a given problem, the result is a sys­
tem of equations. A solution of this system of equations is a candidate forbeing minimum. Sufficient conditions are also available that can be used to select the optimum solution from all those that satisfy the necessary conditions. The method is suitable for problems with only a few optimization variables and constraints. For large problems, setting up and solving the system of equa­
1.3 S o l u t i o n o f O p t i m i z a t i o n P r o b l e m s
tions becomes impractical. Chapter 3 presents the mathematical background necessary to understand optimality conditions and other methods discussed in later chapters in the book. Solutions based on the optimality conditions are discussed in Chapter 4.
1.3.3 N u m e r i c a l M e t h o d s f o r U n c o n s t r a i n e d P r o b l e m s
There are several numerical methods that are available for solving uncon­
strained optimization problems. Some of the methods are Steepest descent, Conjugate gradient, and Newton's method. Different methods have their strengths and weaknesses. Chapter 5 presents several methods for uncon­
strained problems.
1.3.4 T h e S i m p l e x M e t h o d
This is an efficient and robust algorithm for solving linear programming prob­
lems. The method can easily handle problems with thousands of variables and constraints. An extension of the basic algorithm can also handle quadratic programming problems. Chapter 6 presents a detailed treatment of the Sim­
plex method. Both the standard tableau form and the revised matrix form are discussed.
1.3.5 I n t e r i o r P o i n t M e t h o d s f o r L P a n d Q P P r o b l e m s
These relatively recent methods are appealing because of their superior the­
oretical convergence characteristics. In practice, the Simplex method is still more widely used, however. Several methods for solving general nonlinear programming problems generate QP problems as intermediate problems. In­
terior point methods are becoming important for solving these intermediate QP problems. Chapters 7 and 8 are devoted to interior point methods for LP and QP problems, respectively.
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
1.3.6 N u m e r i c a l M e t h o d s f o r G e n e r a l N o n l i n e a r P r o g r a m m i n g P r o b l e m s
Numerical methods for solving general nonlinear programming problems are discussed in Chapter 9. The so-called direct methods are based on the idea of finding a search direction by linearizing the objective and constraint functions and then taking a suitable step in this direction. Currently one of the most popular methods, known as the Sequential Quadratic Programming method, is based on this idea.
A fundamentally different approach is adopted by the so-called Penalty methods. In these methods, constraint functions are multiplied by suitable penalty functions and are added to the objective function. The result is an unconstrained optimization problem that can be solved by methods suit­
able for such problems. A penalty function is defined such that near con­
straint boundaries, a large positive value is added to the objective function. Since we are trying to minimize this function, the process leads to satisfac­
tion of constraints and eventually to the minimum of the constrained prob­
lem.
1.4 Time Value o f Money
As seen from the examples in section I, many optimization problems involve minimizing cost or maximizing profit associated with an activity. Therefore, an understanding of economic factors—such as interest rates, depreciation, inflation, and taxes—is important for proper formulation of these problems. Long-term projects, such as the design of a large dam that may take sev­
eral years to plan, design, and construct, obviously are directly influenced by these economic factors. Even for smaller projects, one may have to con­
sider economic factors in order to properly assess life cycle cost (which in­
cludes initial cost plus cost of any repairs over its useful service life) of such projects.
Several compound interest formulas are derived in this section to gain an understanding of the time value of money. Simple examples explaining the use of these factors are also presented. Using these formulas, it is possible to compare between different alternatives, based on economic factors alone. The second subsection presents a few such examples. For additional details refer to books on engineering economy, e.g., Degarmo, Sullivan, Bontadelli, and Wicks [1997],
1.4 T i m e V a l u e o f M o n e y
1.4.1 I n t e r e s t F u n c t i o n s
Interest rates are the main reason why the value of money changes over time. This section reviews some of the basic relationships that are useful in computing changes in the value of money over time, assuming that the interest rates are known.
Single Payment Compound Amount Factor—spcaf
Consider the simplest case of determining the future value of a fixed amount of money invested:
P = Amount invested i = Interest rate per period
The appropriate period depends on the investment type. Most commercial banks base their computations assuming daily compounding. Some of the large capital projects may be based on monthly or annual compounding.
Tbtal investment at the end of first period = P + iP = (1 + i)P Tbtal investment at the end of second period = (I + i)P + i(l + i)P
= ( l + i ) 2P
Cont i nui ng t hi s pr o c e s s f or n peri ods, i t i s e a s y t o s e e t hat
I bt a l i nv e s t me nt a f t e r t t peri ods, S„ = (1 + i ) nP = s pcaf f i, n] P
Ex a mp l e 1.1 A f at her de pos i t s $2,000 i n hi s daught e r ’s ac c ount o n he r s i xt h bi rt hday. I f t he ba nk pays an annual i nt e r e s t rat e o f 8%, compounded monthly, how much money will be there when his daughter reaches her sixteenth birthday?
i = 0.08/12 tt ~ 120 spcaf[0.08/12,120] = (1 + 0.08/12)120 = 2.21964
$120 = 2000 x 2.21964 = $4,439.28
Single Payment Present Worth Factor— sppwf
The inverse of the spcaf will give the present value of a future sum of money. That is,
P = (1 + 0 nS„ = sppwf [I, ttjSn
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
F.yample 1.2 A person will need $10,000 in exactly five ypars from today. How much money should she deposit in a bank account, that pays an annual interest rate of 9%, compounded monthly?
i = 0.09/12 « = 60 sppwf[0.09/12, 60] = (1 + 0.09/12)"60 = 0.6387
P = 10.000 x 0.6387 = $6,387
Uniform Series Compound Amount Factor—uscaf
Now consider a more complicated situation in which a series of payments (or investments) are made at regular intervals:
R = Uniform series of amounts invested per period i = Interest rate per period
The first payment earns interest over n — 1 periods, and thus is equal to (1 + i)n~l R.
The second payment earns interest over n — 2 periods, and thus is equal to (1 + i)n~2R.
Continuing this process, the total investment after n periods is as follows:
Sn =' (1 + t)n~l R + (1 + i)n~2R + · · · + (1 +i ) R + R
This represents a geometric series whose sum, derived in most calculus text­
books, is as follows:
e (1 + Q" — 1 „ r r.
Sn ----- . R = uscaf [t, ti]R
ι
Example 1.3 A mother deposits $200 every month in her daughter's account starting on her sixth birthday. If the bank pays an annual interest rate of 8 %, compounded monthly, how much money will be there when her daughter reaches her sixteenth birthday?
i = 0.08/12 n - 120
(1 + 0.08/12)120 - 1
uscaf [0.08/12,120] = ^ — --------= 182.946
0.08/12
Si20 = 200 x 182.946 = $36,589.2
1.4 T i m e V a l u e o f M o n e y
Sinking Fund Deposit Factor—sfdf
The inverse of the uscaf can be used to compute uniform series of payments from a given future amount:
R = L Sn = sfdf[i, tl\Sn
(1 + j)n - 1
Example 1.4 A person will need $10,000 in exactly five years from today. How much money should he save every month in a bank account that pays an annual interest rate of 9%, compounded monthly?
0.08/12 (1 + 0.08/12)bu - 1
R = 10,000 χ Ό.0136097 = $136.1
i = 0.09/12 rt = 60 sfclf[0.09/12, 60] = w'nn ^ 6Q— 7 - 0.0136097
Capital Recovery Factor—erf
The above formulas can be combined to give a simple formula to convert a given present amount to a series of uniform payments:
R ~ ( l + 0” - 1 S ” - ( i + 0” - 1 (1 + ΐ) P ~ l - ( l + i ) “"P
or
R = ------- -P = crf[i, n]P
1 — (1 + i)
Example 1.5 A car costs $15,000. You put $5,000 down and finance the rest for three years at 10% annual interest rate compounded monthly. What are your monthly car payments?
Cost = $15,000 Down payment = $5,000 Amount financed = $10,000
0.1/12 (1 + O J/12)30 - 1
R = 10,000 x 0.0322672 = $322.67
/ = 0.1/12 rt = 36 erf[0.1/12, 36] = ^ — 7 = 0.0322672
Uniform Series Present Worth Factor—uspwf
The inverse of the erf can be used to compute the present worth of a uniform series of payments:
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
1 _ (1 + ,·)-»
P = --------:------- R = uspwf[i, «]i?
I
Note that when n is large (1 + t)~n goes to zero. Thus,
uspwffj, oo] = τ and erf [ί, oo] = i
Example 1.6 A car costs $15,000. You put $5,000 down and finance the rest for three years at 10% annual interest rate compounded monthly. Your monthly payments are $322.67. Compute the amount needed to pay off the loan after making twelve payments.
One way to compute this is to figure out the principal and interest portions of each of the payments. Obviously, the principal left after the twelve payments will be the required amount. The computations are tedious by hand but can be easily handled with a loop structure in Mathematica.
We start by initializing variables for i, monthly Payment, and remainingLoan to their appropriate values. From each payment, the amount of interest is equal to the interest rate times the remaining loan. This is identified as interest in the following computation. The remaining portion of the payment (identified as principal) goes to reduce the remaining loan. By repeating this calculation twelve times, we get the following table:
i = . 1/1 2; m o n t h l y P a y m e n t = 3 2 2.6 7; r e m a i n i n g L o a n = 10000;
t b l = ( ("n", "P a y m e n t", "I n t e r e s t", "P r i n c i p a l", "R e m a i n i n g l o a n" } };
Do [ i n t e r e s t = i * r e m a i n i n g L o a n; p r i n c i p a l = m o n t h l y P a y m e n t - i n t e r e s t; r e m a i n i n g L o a n = r e m a i n i n g L o a n - p r i n c i p a l;
A p p e n d T o [ t b l, { n, m o n t h l y P a y m e n t, i n t e r e s t, p r i n c i p a l, r e m a i n i n g L o a n } ] ,
{ n, 1, 1 2 } ];
TableForm [tbl]
n
Payment
Interest
Principal
Remaining loan
1
322.67
83.3333
239.337
9760.66
2
322.67
81.3389
241.331
9519.33
3
322.67
79.3278
243.342
9275.99
4
322.67
77.2999
245.37
9030.62
5
322.67
75.2552
247.415
8783.21
6
322.67
73.1934
249.477
8533.73
7
322.67
71.1144
251.556
8282.17
8
322.67
69.0181
253.652
8028.52
9
322.67
66.9043
255.766
7772.76
10
322.67
64.773
257.897
7514.86
11
322.67
62.6238
260.046
7254.81
12
322.67
60.4568
262.213
6992.6
1.4 T i m e V a l u e o f M o n e y
If we are not interested in all the details, and would only like to know how much we owe the bank after making twelve payments, a much easier way is to convert the remaining twenty-four payments to an equivalent single amount using uspwf as follows:
i as 0.1/12 n = 24 uspwf[0.1/12, 24] = * ~ (1 =21.6709
U.l / 1Z
Remaining loan after 12 payments = P = 322.67 x 21.6709 = $6,992.53
Summary and Mathematica Implementation
The interest formulas are summarized in the following table for easy reference.
Given
Tb find
Multiply given value by
P
Sn
spcaff/, tt] = (1 + i)n
Sn
P
sppwf[t, n] = (1 + 0-"
R
Sn
uscaf[i, tt] —
$tl
R
sfdf[i, n] =
P
R
crf[i, tt] = crf[i, oo] = i
R
P
uspwf[i, n] = - —, uspwf[i, oo] = γ
All these compound interest formulas have been implemented into simple Mathematica functions and are included in the OptimizationTbolbox 'Economic- Factors' package. All functions take interest per period i and number of periods n as arguments and return the appropriate interest factor. The following line computes all these factors with i = 0.08/12 and n = 12.
i = 0.0 8/1 2; n = 12;
{uspw f [ i, η] , e r f [ i, η ] , s f d f [ i, η] , u s c a f [ i, η ] , sppwf [ i, η ], s p c a f [ i, n ] > {11.4958, 0.0869884, 0.0803218, 12.4499, 0.923361, 1.083}
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
1.4.2 C o m p a r i s o n o f A l t e r n a t i v e s B a s e d o n E c o n o m i c F a c t o r s
The compound interest formulas derived in the previous section can be used to compare different alternatives based on economic factors. The main idea is to bring all costs/revenues associated with different options to a common reference point The two commonly used approaches are as follows:
Annual-cost comparisons—for each alternative, all costs/re venues are ex­
pressed in terms of equivalent annual amounts.
Present-worth comparisons—for each alternative, all costs/revenues are ex­
pressed in terms of equivalent present amounts.
It becomes easy to make these comparisons if all financial transactions re­
lated to a particular alternative are represented on a time line showing the amount of the transaction and the time when the transaction takes place. Such diagrams are called cash flow diagrams and are used in the following exam­
ples. The following function, included in the OptimizationTbolbox 'Economic- Factors' package, is useful in drawing these cash flow diagrams.
N e e d s ["O p t i m i z a t i o n T o o l b o x'E c o n o m i c F a c t o r s'* ] j ?C a s h F l o w D i a g r a m
C a s h F l o w D i a g r a m [ d a t a, o p t s ]. D ra w s a c a s h f l o w d i a g r a m. The d a t a i s e x p e c t e d i n a two d i m e n s i o n a l l i s t. E a c h s u b l i s t c o n s i s t s o f e i t h e r 2 o r 3 e n t r i e s. T he 2 e n t r i e s a r e i n t e r p r e t e d a s { v a l u e, p e r i o d }. T he t h r e e e n t r i e s a r e assumed a s a ύ η ϊ ΐ ο π η s e r i e s { v a l u e, s t a r t P e r i o d, e n d P e r i o d }. T h e v a l u e may b e e i t h e r a n u m e r i c a l v a l u e o r i n t h e f o r m o f 'l a b e l'- > v a l u e. T he n u m e r i c a l v a l u e o r t h e l a b e l, i f s p e c i f i e d, i s p l a c e d on t h e d i a g r a m. T h e o p t s may b e a n y v a l i d o p t i o n f o r t h e g r a p h i c s Show f u n c t i o n.
Example 1.7 Purchase versus rent A company is considering the purchase of a new piece of testing equipment that is expected to produce $8,000 additional profit at the end of the first year of operation; this amount will decrease by $500 per year for each additional year of ownership. The equipment costs $18,000 and will have a salvage value of $3,000 after four years of use.
The same equipment could also be rented for $6,500 per year, payable in advance on the first day of each rental year. Use a present worth analysis to compare the alternatives. What is your recommendation to the company— purchase, rent, neither? The interest rate is 25% compounded annually.
(a) Option A— Purchase If we choose this option, we need to spend $18,000 right away. The additional profits per year are $8,000, $7,500, $7,000, and $6,500. The salvage value of the equipment after four years produces an in-
1.4 T i m e V a l a e o f M o n e y
come of $3,000. Thus, the cash flow diagram for this option is as follows:
c o s t = 180 00; p i — 8000; p2 · 7500; p3 = 7000; p4 “ 6500; s a lv a g e = 3000; i = .2 5;
CashFlow D iagram [ { { - c o s t, 0 }, { p i/ 1 }, {p 2, 2 }, {p 3, 3 }, {p 4, 4 } , { s a l v a g e, 4 } }, p lo tR a n g e -> A l l ]
8000
7500
7000
6500
3000
©-
-18000
FIGURE 1.2 Cash flow diagram for purchase option.
The —$18,000 is already a present amount. The first income of $8,000 comes after one year (one period "because annual compounding is assumed). Tb con­
vert this amount to the present, we simply need to multiply it with sppwf with n = 1. Similarly, all other amounts can be brought to the present. Thus, the present worth of this option is as follows:
PWA = - c o s t + p i * s p p w f [ i,1] + p2*sppwf [ i,2] + p3*sppwf [ i,3]
+ p4*sppwf [ i,4 ] + s a lv a g e * s p p w f [ i#4]
675.2
Thus, in terms of present dollars, this option results in additional profit of $675.20.
(b) Option B—Rent The costs associated with this option are annual rents. The rent for the following year is paid in advance. Since the company does not own the equipment, there is no salvage value. Thus, the cash flow diagram for this option is as follows:
r e n t = 6 5 0 0;
C a s h F l o w D i a g r a m [ { { - r e n t, 0, 3 }, { p i, 1 }/ { p 2, 2}, {p 3, 3>, {p 4, 4>>, P l o t R a n g e -> A l l ]
8000
-6500
FIGURE 1.3 Cash flow diagram for purchase option.
The profits are converted to the present amount by multiplying them by appropriate sppwf, as in the case of option A. The rent is a uniform series of payments and thus can be converted to the present amount by using the uspwf. There is one subtle point to note, however. The derivation of uspwf assumes that the first payment is at the end of the first period. Thus, we can use the uspwf on the three rent payment, excluding the first one. The present worth of this option is as follows:
PWB = - r e n t - r e n t * u s p w f [ i,3 J + p l * s p p w f [ i,1] + p 2 * s p p w f [ i,2] +
p 3 * s p p w f [ i,3] + p 4 * s p p w f [ i,4]
-1741.6
Thus in terms of present dollars, this option results in a loss of $1741.60.
(c) Conclusion The rent option results in a net loss. The company should purchase the equipment.
Example 1.8 Water treatment facilities A city is considering following two proposals for its water supply.
1.4 T i m e V a l u e o f M o n e y
proposal 1:
First dam and treatment plant—Construction cost = $500,000, annual op­
erations cost = $36,000. The capacity of this facility will be enough for the community for twelve years.
Second dam and treatment plant—Construction cost = $480,000, additional annual operations cost = $28,000. The capacity of the expanded facility will be enough for the foreseeable future.
proposal 2:
Large dam and first treatment plant—Construction cost = $700,000, annual operations cost = $37,000. The capacity of this facility will be enough for the community for fifteen years.
Second treatment plant—Construction cost = $100,000, additional annual operations cost = $26,000. The capacity of the expanded facility will be enough for the foreseeable future.
Use a present worth analysis to compare the alternatives. Assume annual interest rate of 10% and annual compounding.
(a) Proposal 1 The cash flow diagram for this option is as follows.
CaehFlowDiagram[{{"let dam + plant·-> - 1, 0}, {"Operations* -> -1/4,1,12}, {"2nd dam + plant" -> - 1, 12}, {"Operations'* -> - 1/2, 13, 16},
{" -> o o" -> - 1/2, 17}}, PlotRange -> All]
Operations
Ist dam + plant
I f i - B
Operations
-±i7
2nd dam + plant
FIGURE 1.4 Cash flow diagram for the water supply problem—proposal 1.
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
The cost of the first dam and the treatment plant is already in terms of present dollars. Annual operations costs for twelve years can he brought to the present by multiplying by uspwf[i,12]. The cost of the second dam and the treatment plant can be brought to the present by multiplying by sppwf[i,12]. Starting from year thirteen, the operation costs increase and stay the same for the foreseeable future. If we multiply these costs by uspwf[i,oo]t this will bring all these costs to a single amount at the year twelve. Tb bring it to the present, this single amount needs to be multiplied by sppwf[i,12]. Thus, the complete calculations for the present worth are as follows:
cl = 500000; al = 36000; a2 = al+28000; c2 = 480000; i = 0.1:
PW = cl + al * uspwf[i, 12] + c2 * sppwf [i, 12] + a2 * uspwf [i, oo] * sppwf[i, 12]
1.10216 χ
106
(b) Proposal 2 The cash flow diagram for this option is as follows:
cashFlowDiagraxn[{{"Dai& + let plant" -> - 1, 0},
{“Operations’1 -> - 1/4, 1, 15}, {"2nd plant" -> - 1, 15},
{"Operations" -> 1/2, 16, 20}, {"-»»■->- 1/2, 21}}, PlotRange -> All]
0-
Operations
Dam + 1st plant
O] >erations
-2|1
2nd )lant
FIGURE 1.5 Cash flow diagram for the water supply problem—proposal 2.
1.4 T i m e V a l u e o f M o n e y
The cost of the dam and the treatment plant is already in terms of present dollars. Annual operation costs for fifteen years can be brought to the present by multiplying by uspwf[i,15]. The cost of the second treatment plant can be brought to the present by multiplying by sppwf[i,15]. Starting from year sixteen, the operation costs increase and stay the same for the foreseeable future. If we multiply these costs by uspwf[i,oo], this will bring all these costs to a single amount at year fifteen. Ib bring it to the present, this single amount needs to be multiplied by sppwf[i,15]. Thus, the complete calculations for the present worth are as follows:
cl = 700000; al ■ 37000; a2 = al+26000; c2 = 100000; i = 0.1;
PW = cl + al * uspwf [i, 15] + c2 * sppwf[i# 15] + a2 * uspwf[i# oo] * sppwf [i, 15]
1.15618 xIQ6
is 1.10216 χ 106 and that of the second proposal is 1.15618 χ 106. The first proposal is slightly cheaper for the city and should be adopted.
Example 1.9 Optimization problem involving annual cost A company requires open-top rectangular containers to transport material. Using the following data, formulate an optimum design problem to determine the container dimensions for minimum annual cost.
Construction costs
Sides = $65/m2 Ends = $80/m2 Bottom = $120/m2
Useful life
10 years
Salvage value
20% of the initial construction cost
Yearly maintenance cost
$12/m2 of the outside surface area
Minimum required volume of the container
1200 m3
Nominal interest rate
10% (Annual compounding)
As seen from Figure 1.6, the design variables are the three dimensions of the container.
b = Width of container £ = Length of container h = Height of container
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
FIGURE 1.6 Open-top rectangular container
In terms of these design variables, the different costs and container volume are as follows:
Construction cost = 120ib + 2 x 65ih + 2 x QObh = I20ib + 130ih -I- 160i>/i Yearly maintenance cost = 12(ib + 2ih + 2bh)
Salvage value = 0.2(120ib +130ih + 160Wi)
Container volume = bht > 1200
lb determine the annual cost, it is useful to draw the cash flow diagram.
0-
VJaintenanci:
Construction
Salvage
FIGURE 1.7 Cash flow diagram for the container design problem.
1.5 C o n c l u d i n g R e m a r k s
CaehFlowDiagram [ {{"construction" -> -1,0},
{"Maintenance" -> - 1/4, 1, 10}, {"Salvage" >1/2, 10}},
PlotRange -> All]
Using this cash flow diagram, it is easy to see that to convert all transactions to annual cost, we need to multiply the construction cost by crf[i,10] and the salvage value by sfdf[i,10]. Thus, the annual cost of the project is expressed as follows:
Annual cost = crf[0.1,10](120ib -I- I30ih + 160Wi) + 12(ib -I- 2ih + 2bh)
- sfdf[0.1,10] x 0.2(120ib + 130ih + 160Wj)
Annual cost = 48.0314bh + 30.0236W + 43.5255h£
The optimization problem can now be stated as follows:
Find b, h, and i to
Minimize annual cost = 48.0314Wi + 30.0236M + 43.5255hi Subject to bhi > 1,200 and b, h, and i > 0
1.5 C o n c l u d i n g R e m a r k s
The importance of careful problem formulation cannot be overemphasized. A sophisticated optimization method is useless if the formulation does not capture all relevant variables and constraints. Unfortunately, it is difficult, if not impossible, to teach problem formulation in a textbook on optimization. The examples presented in textbooks such as this must introduce a variety of assumptions to make complex real-world situations easy to understand to read- ers from different backgrounds. Real-world problems can only be understood and properly formulated by those who are experts in the relevant fields.
The examples presented in this book are intended to provide motivation for studying and applying the optimization techniques. Little attempt has been made to use actual field data for the examples. Readers interested in seeing more engineering design examples formulated as optimization prob­
lems should consult Arora [1989], Ertas and Jones [1996], Hayhurst [1987], Hymann [1998], Papalambros and Wilde [1988], Pike [1986], Rao [1996], Stark and Nicholls [1972], Starkey [1988], and Suh [1990]. Excellent industrial engi­
neering examples can be found in Ozan [1986] and Rardin [1998]. For a large collection of linear programming examples and discussion of their solution using commercially available software, see the book by Pannell [1997].
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
1.6 P r o b l e m s
Optimization Problem Formulation
1.1. Hawkeye foods owns two types of trucks. Thick type I has a refrigerated capacity of 15 m3 and a nonrefrigerated capacity of 25 m3. Thick type II has a refrigerated capacity of 15 m3 and nonrefrigerated capacity of 10 m3. One of their stores in Gofer City needs products that require 150 m3 of refrigerated capacity and 130 m3 of nonrefrigerated capacity. For the round trip from the distribution center to Gofer City, truck type I uses 300 liters of fuel, while truck type II uses 200 liters. Formulate the problem of determining the number of trucks of each type that the company must use in order to meet the store's needs while minimizing the fuel consumption.
1.2. A manufacturer requires an alloy consisting of 50% tin, 30% lead, and 20% zinc. This alloy can be made by mixing a number of available alloys, the properties and costs of which are tabulated. The goal is to find the cheapest blend. Formulate the problem as an optimization problem.
Available alloys
Properties
A
B
C
D
E
Lead Γ96Ϊ
10
10
40
60
30
Zinc (%)
10
30
50
30
30
Tin (%)
80
60
10
10
40
Cost: ($/lb alloy)
8.2
9.3
11.2
13
17
1.3. Dust from an older cement manufacturing plant is a major source of dust pollution in a small community. The plant currently emits two pounds of dust per barrel of cement produced. The Environmental Protection Agency (EPA) has asked the plant to reduce this pollution by 85% (1.7 lbs/barrel). There are two models of electrostatic dust collectors that the plant can install to control dust emission. The higher efficiency model would reduce emissions by 1.8 lbs/barrel and would cost $0.70/barrel to operate. The lower efficiency model would reduce emissions by 1.5 lbs/barrel and would cost $0.50/barrel to operate. Since the higher effi­
ciency model reduces more than the EPA required amount and the lower efficiency less than the required amount, the plant has decided to install one of each. If the plant has a capacity to produce 3 million barrels of
1.6 P r o b l e m s
cement per year, how many barrels of cement should be produced using each dust control model to meet the EPA requirements at a minimum cost? Formulate the situation as an optimization problem.
1.4. A small firm is capable of manufacturing two different products. The cost of making each product decreases as the number of units produced ;iven by the following empirical relationships:
„ 1,500 „ 2,500
. ci = 5 H C2 = 7 H--------
«1 «2
where n\ and are the number of units of each of the two products produced. The cost of repair and maintenance of equipment used to pro­
duce these products depends on the total number of products produced, regardless of its type, and is given by the following quadratic equation:
(»i + «2)[0.2 + 2.3 x 10"5(«i + n2> + 5.3 χ 10~9(ni -I- rt2>2]
The wholesale selling price of the products drops, as more units are produced, according to the following relationships:
pi = 15 — Ο.ΟΟΙηχ p2 = 25 — 0.0015«2
Formulate the problem of determining how many units of each product the firm should produce to maximize its profit.
1.5. A company can produce three different types of concrete blocks, iden­
tified as A, B, and C. The production process is constrained by facilities available for mixing, vibration, and inspection/drying. Using the data given in the following table, formulate the production problem in order to maximize the profit.
Blocks
A
B
C
Available
Mixing (hours/ batch)
1
3
9
900
Vibration (hours/batch)
2
3
6
1,200
Inspection/drying (hours/batch).
0.7
0.8
1
400
Profit: ($/batch)
7
17
30
1.6. A thin steel plate, 10 in. wide and 1/2 in. thick and carrying a tensile
force of T = 150,000 lbs, is to be connected to a structure through high
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
strength bolts, as shown in Figure 1.8. The plate must extend by a fixed distance L — 120 in. from the face of the structure. The material for a steel plate can resist stress up to Fu = 58,000 lbs/in2. The bolts are chosen as A325 type (standard bolts available from a number of steel manufacturers). The bolts are arranged in horizontal and vertical rows. The spacing between horizontal rows is s = 3d inches, where d is the diameter of the bolt, and that between vertical rows is g > 3d inches. The bolts must be at least 1.5*2 inches from the edges of the plate or the structure.
The design problem is to determine the diameter of bolts, the total number ofbolts required, and the arrangement ofbolts (i.e., the number of horizontal and vertical rows) in order to minimize the total cost of the plate and the bolts. Assuming the cost of a steel plate is $2 per inch and that of a bolt is $5 per bolt, formulate the connection design problem as an optimization problem. The pertinent design requirements are as follows:
(a) The tensile force that one A325 bolt can support is given by the smaller of the following two values, each based on a different failure criteria.
Based on failure of bolt = 12,000πά2 lbs
Based on failure of plate around bolt hole = 2AdtFu lbs
where d = diameter of the bolt, t = thickness of the plate, and F„ = ultimate tensile strength of the plate material = 58,000 Ibs/in2.
1.6 P r o b l e m s
(b) Vertical rows ofbolts (those along the width of the plate) make the plate weaker. Taking these holes into consideration, the maximum tensile force that the plate can support is given by the following equation:
Tfensile load capacity of plate = 0.75Fu(w — ripdh)
where w — width of plate, 1% = number ofbolts in one vertical row across the plate width, and dh = bolt hole diameter (= d + 1/8 in).
(c) Practical requirements. The number ofbolts in each row is the same. Each row must have at least two bolts, and there must be at least two rows ofbolts. The smallest bolt diameter allowed is 1/2 in.
1.7. A mining company operates two mines, identified as A and B. Each mine can produce high.-, medium-, and low-grade iron ores. The weekly demand for different ores and the daily production rates and operating costs are given in the following table. Formulate an optimization problem to determine the production schedule for the two mines in order to meet the weekly demand at the lowest cost to the company.
Weekly Demand
Daily Production
Ore grade
(tons)
Mine A (tons)
Mine B (tons)
High
12,000
2,000
1,000
Medium
8,000
1,000
1,000
Low
24,000
5,000
2,000
Operations cost ($/day)
210,000
170,000
1.8. A company manufactures fragile gift items and sells them directly to its customers through the man. An average product weighs 12 kg, has a volume of 0.85 m3, and costs $60 to produce. The average shipping distance is 120 miles. The shipping costs per mile based on total weight and volume are $0.006/kg plus $0.025/m3. The products are shipped in cartons that are estimated to cost $2.5/m3 and weigh 3.2 kg/m3. The empty space in the carton is completely filled with a packing material to protect the item during shipping. This packing material has negligible weight but costs $0.95/m3. Based on past experience, the company has developed the following empirical relationship between breakage and
the amount of packing material:
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
ι /, Volume of packing material \
% breakage = 85 (1 - —;1 ------------- I
\ Volume or the shippmg carton /
The manufacturer guarantees deliveiy in good condition which means that any damaged item must be replaced at the company's expense. Formulate an optimization problem to determine the shipping carton volume and volume of packing material that will result in the minimum overall cost of packing, shipping, and delivery.
1.9. Assignment of parking spaces for its employees has become an issue for an automobile company located in an area with a harsh climate. There are enough parking spaces available for all employees; however, some employees must be assigned spaces in lots that are not adjacent to the buildings in which they work. The following table shows the distances in meters between parking lots (identified as 1, 2, and 3) and office buildings (identified as A, B, C, and D). The number of spaces in each lot and the number of employees who need spaces are also tabulated. Formulate the parking assignment problem to minimize the distances walked by the employees from their parking spaces to their offices.
Distances from parking lot (m)
Parking lot
Building A
Building B
Building C
Building D
Spaces Available
1
29p
410
260
410
80
2
430
350
330
370
100
3
310
260
290
380
40
# of employees
40
40
60
60
1.10. An investor is looking to make investment decisions such that she will get at least a 10 % rate of return while minimizing the risk of major losses. For the past six years, the rates of return in three major investment types that she is considering are as follows.
Annual rates of return
Stocks
18.24
17.12
22.23
15.26
12.62
15.42
Mutual funds
12.24
11.16
10.07
8.46
6.62
8.43
Bonds
5.12
6.26
6.34
7.01
6.11
5.95
Formulate the problem as an optimization problem.
1.6 P r o b l e m s
1.11. Hawkeye Pharmaceuticals can manufacture a new drug using any one of the three processes identified as A, B, & C. The costs and quantities of ingredients used in one batch of these processes are given in the following table. The quantity of a new drug produced during each batch of different processes is also given in the table.
Ingredients used per batch
Process
Cost ($ per batch)
Ingredient I (tons)
Ingredient II (tons)
Quantity of drug produced (tons)
A
$12,000
3
2
2
B
$25,000·
2
6
5
C
$9,000
7
2
1
The company has a supply of 80 tons of ingredient I and 70 tons of ingredient II at hand and would like to produce 60 tons of the new drug at a minimum cost.
Formulate the problem as an optimization problem. Clearly identify the design variables, constraints, and objective function.
1.12. For a chemical process, pressure measured at different temperatures is given in the following table. Formulate an optimization problem to determine the best values of coefficients in the following exponential model for the data.
Pressure = ae^T
Ifemperature (T°C)
Pressure (mm of Mercury)
20
15.45
25
19.23
30
26.54
35
34.52
40
48.32
50
68.11
60
98.34
70
120.45
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
1.13. Consider the cantilever beam-mass system shown in Figure 1.9. The beam cross-section is rectangular. The goal is to select cross-sectional dimensions (b and h) to minimize weight of the beam while keeping the fundamental vibration frequency (ώ) larger than 8 rad/sec.
Section A-A
FIGURE 1.9 Rectangular cross-section cantilever beam with a suspended mass.
The numerical data and various equations for the problem are as follows.
Fundamental vibration frequency
ω = y/kg/m radians/sec
Equivalent spring constant,
1 1,I 3 U1 £ - τ + 3ΕΓ
Mass attached to the spring
tn = W/g
Gravitational constant
g = 386 in/sec2
Weight attached to the spring
W = 60 lbs
Length of beam
1 = 15 in
Modulus of elasticity
E = 30 χ 106 lbs/in2
Spring constant
fc = 10 lbs/in2
Moment of inertia
T — W*3 j-,4 1 — TT m
Width of beam cross-section
0.5 in < fc < 1 in
Height of beam cross-section
0.2 in < ft < 2 in
Unit weight of beam material
0.286 lbs/in3
1.14. Modem aircraft structures and other sheet metal construction require stiffeners that are normally of I-, Z-, or C-shaped cross sections. An I
1.6 P r o b l e m s
Ρ
shaped cross section, shown in Figure 1.10, has been selected for the present situation. Since a large number of these stiffeners are employed in a typical aircraft, it is important to optimally proportion the dimen- sions, b, t, and h.
For a preliminary investigation, a stiffener is treated as an axially loaded column, as shown in the figure. The goal is to have a least volume design while meeting the following yield stress and buckling load limits.
Yield stress limit
P/A < Cy
Overall buckling load
x2EImin p
Flange buckling load
°·4 3ϊ55Ϊ25(τ)
Web buckling load
'
Area of cross-section, A
A — 2 bt + (h — 2t)t
Minimum moment of inertia, Zmin
Min[J*, Iy]
Ix = Ibt3 + \bt(h - t)2 + fa(h - 2f)3
ly = gffr* + —
Applied load, P
2000 lbs
Yield stress, ay
25,000 lbs/in2
Young’s modulus, E
107 lbs/in2
Poisson's ratio, v
0.3
.Length or stutener, L
Z5 in
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m F o r m u l a t i o n
Formulate the problem in the standard optimization problem format. Clearly indicate your design variables and constraints. Use the given numerical data to express your functions in as simple a form as possible.
1.15. A major auto manufacturer in Detroit, Michigan needs two types of seat assemblies during 1998 on the following quarterly schedule:
T^pe 1
Type 2
First quarter
25,000
25,000
Second quarter
35,000
30,000
Third Quarter
35 000
25 000
Fourth quarter
25,000
30,000
Tbtal
120,000
110,000
The excess seats from each quarter are carried over to the next quarter but are subjected to an inventory-carrying charge of $20 per thousand seats. However, assume no inventory is carried over to 1999.
The company has contracted with an auto-seat manufacturer that has two plants: one in Detroit and the other in Waterloo, Iowa. Each plant can manufacture either type of seat; however, their maximum capacities and production costs are different. The production costs per seat and: the annual capacity at each of the two plants in terms of number of seat assemblies is given as follows:
Quarterly capacity
Production cost
Production cost
Ljr pC I
IUi- a
iui type z
Detroit plant
30,000
$225
$240
Waterloo plant
35*000
$165
$180
The packing and shipping costs from the two plants to the auto manu- facturer are as follows:
Cost/100 seats Detroit plant $10
Waterloo plant $80
1.6 P r o b l e m s
Formulate the problem to determine the seat acquisition schedule from the two plants to minimize the overall cost of this operation to the auto manufacturer for the year.
1.16. Consider the optimum design of the rectangular reinforced concrete beam shown in Figure 1.11. There is steel reinforcement near the bottom. Formwork is required on three sides during construction. The beam must support a given bending moment. A least-cost design is required.
The bending strength of the beam is calculated from the following formula:
M „ = 0.9Al V ( l ^ 0.5 9 G s ) (/j ) )
where Fy is the specified yield strength of the steel, and f'c is the specified compressive strength of the concrete. The ductility requirements dictate minimum and maximum limits on the steel ratio p = As/bd.
Pmin — P — Pmax
Use the following numerical data.
Maximum steel ratio
Pmax — 0.025
Minimum steel ratio
Pmin = 0.0033
Required moment capacity
Mu > 400 χ 103 N-m
Minimum beam width
b > 300 mm
Concrete cover
c — b5 limit
Maximum beam depth
h < 1200 mm
Concrete cost
$100/m3
Formwork cost
$2/m2
Steel reinforcement cost
$610/ton (1 ton = 907.18 kg)
Density of steel
7850 kg/m3
Yield stress of steel, Fy
420 MPa
Ultimate concrete strength, f'
35 MPa
Formulate the problem of determining the cross-section variables and the amount of steel reinforcement to meet all the design requirements at a minimum cost. Assume a unit beam length for cost computations.
C h a p t e r 1 O p t i m i z a t i o n P r o b l e m E o r m u l a t i o n
Formwork
FIGURE 1.11 Reinforced concrete beam.
Clearly indicate your design variables and constraints. Use the given numerical data to express your functions in as simple a form as possible.
Time Value of Money
1.17. You deposit $5,000 in your savings account that pays an annual interest rate of 9%, compounded daily. How much money will be there in five years?
1.18. Collene plans to put enough money into fixed interest CDs to purchase a car for her son in three years. The car is expected to cost $15,000. How much money should she put into a CD that pays an annual interest rate of 8.5%, compounded daily?
1.19. Nancy is thinking about redecorating her bedroom, which is estimated to cost $9,000. If she can save $200 per month, how long will she have to wait till she has enough money for this project? The annual interest rate is 9.5%, compounded monthly.
1.20. A house costs $150,000. You put down 20% down payment and take out a 15-year mortgage for the rest at a fixed annual interest rate of 10%. Assuming monthly compounding, what will be yniir monthly payments?
1.21. A house costs $200,000. You put 20% down and take out a 20-year mort­
gage for the rest at a fixed annual interest rate of 10%. After five years (60 payments), you discover that the interest rates have fallen to 7% and you would like to refinance. If you refinance exactly the balance that is left after five years, what will be your new monthly payment? Assume
monthly compounding.
1.6 P r o b l e m s
1.22. A company is considering acquiring a new piece of equipment that is expected to produce $12,000 in additional revenue per year for the company. The equipment costs $50,000 and will require $1000 per year maintenance. It has a salvage value of $9,000 after seven years of use. The same equipment could also be leased for $8,500 per year, payable in advance on the first day of each year. The leasing company is responsible for maintenance costs.
Use a present worth analysis to compare the alternatives. What is your recommendation to the company—purchase, lease, neither? Assume that the interest rate is 12% compounded annually.
1.23. A company has designs for two facilities, A and B. Initial cost of the facilities, maintenance and operation costs, and additional profits from the facilities are estimated as follows:
Facility
Initial cost
Maintenance He operation costs per year
Profit/year
A
$1,000,000
$100,000
$250,000
B
$800,000
$80,000
$200,000
In addition to the above expenses, it is anticipated that the facilities will need renovation at the end of the tenth year of operation, costing $150,000 for A and $100,000 for B. Using the present worth method, which facility is more profitable? Assume a 20-year life for each facility with a salvage value of $200,000 for A and $160,000 for B. The prevailing interest rate is 9% per year.
1.24. A salesman is trying to sell a $55,000 piece of equipment to the TUmer Construction Co. of Iowa City. Using this equipment, the company can generate net annual revenue before taxes of $15,000 over the next five years. The equipment has a salvage value of $5,000 at the end of five years. The tax laws allow straight-line depreciation of equipment, which means that the company can deduct $10,000 from its yearly taxable income. The income tax rate for the company is 34%. The company likes to make at least 8% annually on their investments after all taxes are paid. Would you advise the company to go ahead and make the necessary investment from its internal funds?
1 *25. The Hawk City public library needs additional space to accommodate its rapidly expanding multimedia acquisitions. The proposal is to purchase an adjacent property at a cost of $1.2 million to be paid on January 1, 1998. The cost for the new building, including demolition of the existing
C h a p t e r 1 O p t i m i s a t i o n P r o b l e m F o r m u l a t i o n
building on the property, is estimated at $10 million to be paid on January 1, 2002. In addition, semi-annual maintenance is.estimated at $50,000 starting on July 1, 2002.
Because of budget constraints, the city is unable to fund the proposal.
The library board is considering charging a fee for its multimedia patrons to fund the project. Assuming that there are 18,000 patrons and that fees are collected each January 1 and July 1, with the first payment to start on July 1, 2002, what semiannual fee per user would be needed to support this proposal? Assume a nominal annual interest rate of 8% compounded semiannually and a 15-year period for cost recovery.
Optimization Problems Involving Economic Considerations
1.26. A chemical manufacturer requires an automatic reactor-mixer. The mix- ing time required is related to the size of the mixer and the stirring power as follows:
y/S
T = 1,000-=- ΡΔ
where S = capacity of the reactor-mixer, kg, P — power of the stirrer, k- Watts and T is the time taken in hours per batch. The cost of building the reactor-mixer is proportional to its capacity and is given by the following empirical relationship:
Cost = $60,000VS
The cost of electricity to operate the stirrer is $0.05/k-W-hr and the overhead costs are $137.2 P per year. The total reactor to be processed by the mixer per year is 107 kg. Time for loading and unloading the mixer is negligible. Using present worth analysis, formulate the problem
to minimize cost. Assume a 5-year useful life, 9% annual interest rate compounded monthly, and a salvage value of 10% of the initial cost of the mixer.
1.27. Use the annual cost method in Problem 1.26.
1.28. A multicell evaporator is to be installed to evaporate water from a salt water solution in order to increase the salt concentration in the solu­
tion. The initial concentration of the solution is 5% salt by weight. The desired concentration is 10%, which means that half of the water from
1.6 P r o b l e m s
the solution must be evaporated. The system utilizes steam as the heat source. The evaporator uses 1 lb of steam to evaporate 0.8n lb of water, where n is the number of cells. The goal is to determine the number of cells to minimize cost. The other data are as follows:
The facility will be used to process 500,000 lbs of saline solution per day.
The unit will operate for 340 days per year.
Initial cost of the evaporator, including installation = $18,000 per cell.
Additional cost of auxiliary equipment, regardless of the number of cells = $9,000.
Annual maintenance cost = 5% of initial cost.
Cost of steam = $1.55 per 1,000 lbs.
Estimated life of the unit = 10 years.
Salvage value at the end of 10 years = $2,500 per cell.
Annual interest rate = 11%.
Formulate the optimization problem to minimize annual cost.
1.29. Use the present worth method in problem 1.28.
1.30. A small electronics company is planning to expand two of its manu­
facturing plants. The additional annual revenue expected from the two plants is as follows:
From plant 1: 0.00002xf — X2 From plant 2: 0.00001x1
where x\ and X2 are the investments made to upgrade the facilities. Each plant requires a minimum investment of $30,000. The company can borrow a maximum of $100,000 for this upgrade to be paid back in yearly installments in ten years at an annual interest rate of 12%. The revenue that the company generates can earn interest at an annual rate of 10%. After the 10-year period, the salvage value of the upgrades is expected to be as follows:
For plant 1: O.lxi For plant 2: 0.15x2
Formulate an optimization problem to maximize the net present worth of these upgrades.
CHAPTERTWO
G r a p h i c a l O p t i m i z a t i o n
Graphical optimization is a simple method for solving optimization problems involving one or two variables. For problems involving only one optimization variable, the minimum (or maximum) can be read simply from a graph of the objective function. For problems with two optimization variables, it is possible to obtain a solution by drawing contours of constraint functions and the objective function. The procedure is discussed in detail in the first section. After developing the procedure in detail with hand calculations, a Mathematica function called GraphicalSolution is presented that automates the process of generating the complete rnntmir pints The second section presents solutions of several optimization problems using this function.
Practical optimization problems most likely will involve more than two vari­
ables, and thus, a direct graphical solution may not be feasible. For small-scale problems, it may still be possible to at least get an idea of the optimum by fixing some of the variables to reasonable values based on prior experience and obtaining a graphical solution for two of the most important variables. This is demonstrated in Section 2.1.1 by solving the building design problem formulated in Chapter 1, using the GraphicalSolution function. Perhaps the most important use of graphical optimization is that it enables a clearer un-
C h a p t e r 2 G r a p h i c a l O p t i m i z a t i o n
derstanding of the optimality conditions and the performance of numerical optimization algorithms presented in later chapters. Seyeral options, such as GradientVectors and PlotHistoiy, are built into the GraphicalSolution function to make this process easier. Later chapters make extensive use of GraphicalSo­
lution and relevant options to demonstrate several fundamental concepts. The function is used in generating all contour plots that appear in the textbook. Tb improve readability, the Mathematica code is not shown in all cases. However, the accompanying CD contains complete function calls for a large number of examples.
2.1 P r o c e d u r e f o r G r a p h i c a l S o l u t i o n
Consider an optimization problem involving two optimization variables and written in the standard form as follows:
Find a vector of optimization variables, (xi, x2) in order to Minimize an objective (or cost) function, f ( x 1, *2)
Subject to
gi(xi,x2) < 0 i = 1, 2,..., m Less than type inequality constraints hi(x 1, *2) = 0 i = 1, 2,..., p Equality constraints
A graphical solution of this problem involves the following three steps:
1. Choose an appropriate range for optimization variables.
2. Draw a contour for each constraint function to represent its boundary.
3. Draw several contours of the objective function.
The complete procedure is discussed in detail in this section. Tb make the process clear to all readers, the steps are presented as if the contour plots are to be hand drawn. Experienced Mathematica users undoubtedly are aware of the built-in ContourPlot function that makes most of the computations shown in the section unnecessary. However, it is instructive to go through at least one detailed example by hand before using the more automated tools. In fact, the GraphicalSolution function presented in the following section goes even a step further than the ContourPlot and completely automates the process of generating complete graphical optimization plots.
2.1 P r o c e d u r e f o r G r a p h i c a l S o l u t i o n
2.1.1 C h o i c e o f a n A p p r o p r i a t e R a n g e f o r O p t i m i z a t i o n V a r i a b l e s
The first task is to select a suitable range of values for the two optimization variables. A certain trial-and-error period is usually necessaiy before a suitable range is determined. The range must obviously include the optimum point. A large range may result in very small contour graphs making it difficult to differentiate between different constraints. On the other hand, a vary small range of variable values may not show the region that represents the optimum solution.
For practical problems, a suitable range can be determined based on prior experience with similar designs. For problems for which one has no prior knowledge of the optimum solution at all, the following procedure may be used.
1. Pick lower and upper limits for one of the variables, say for x\. Arbitrary values can be chosen if nothing is known about the solution.
2. Solve for the corresponding X2 values from each of the constraint equations. Set the minimum and maximum values of *2 based on these values.
As an illustration, consider a graphical solution of the following problem:
Minimize f ( x\, x2) = 4xf — 5*1*2 + *2
Subject to ( f ^ ’X\) = X?_X2 + c2- ° )
V Hx 1. *2) = *1 + *2 - 6 = 0 )
Assuming we have no idea of the optimum point, we arbitrarily pick lower and upper limits for x\ such as 0 and 10, respectively. Next, we solve for the corresponding *2 values from each of the constraint equations, treating all constraints as equalities.
g : with x\ =0, Xj — *2 + 2 = 0 gives *2 = 2 with *i = 10, xf — *2 + 2 = 0 gives X2 = 102 h : with xi = 0, xi + X2 — 6 = 0 gives X2 = 6 with xi = 10, xi + X2 — 6 = 0 gives X2 = - 4
Thus for our first trial, we select the range for xias (0,10) and that for X2 as
(-4,102).
C h a p t e r 2 G r a p h i c a l O p t i m i z a t i o n
2.1.2 C o n t o u r s o f C o n s t r a i n t
F u n c t i o n s — F e a s i b l e R e g i o n
The second task is to draw contours for constraint functions and determine the region over which all constraints are satisfied. This region is called the feasible region. Assuming that the constraints are written in the standard form (less than or equal to type with the right-hand side being 0), we need to draw lines representing equations gi(xi,x2) = 0 and hj(xi,x2) = 0. These contour lines can be drawn by selecting several values for one of the variables and solving for the other variable from the constraint function. That is, to plot the contour for constraint gi, select a few different values for x\ over the chosen range. For each selected value of x\, compute the corresponding x2 by solving the equation gi(x\, x2) = 0. A line passing through pairs of such (xi, *2) values represents a contour for the function gi(xi>x2) = 0. If a function is linear, obviously one needs only two points to draw such a contour. For nonlinear functions, several points are needed so that a reasonably smooth contour can be drawn.
Continuing with the example from the previous section, our task is to draw lines passing through points that satisfy the following equations:
Boundary of constraint 1: xf — x2 + 2 = 0 Boundary of constraint 2: xi + x 2 — 6 = 0
Ib generate the data to draw these contours, we take several points along the xi axis and compute the corresponding X2 values by solving the corresponding equation. The computed values are shown in the following table:
Xl
X2 from xf — x2 + 2 = 0
*2 from xi + x2 - 6 = 0
0
2.
6.
2.5
8.25
3.5
5.
27.
1.
7.5
58.25
-1.5
10.
102.
-4.
The graph in Figure 2.1 shows lines that pass through pairs of these points.
For an equality constraint, the contour hi(x\,x2) = 0 represents the feasible line since any point on this line satisfies the given constraint. For an inequality constraint, the contour g;(x1, x2) = 0 represents the boundary of the feasible side of the constraint. On one side of this contour, gi > 0 and on the other side,
2.1 P r o c e d u r e f o r G r a p h i c a l S o l u t i o n
gi < 0. Obviously, the side with g t < 0 is the feasible side of the constraint since any point on this side will satisfy the constraint. The infeasible side of each inequality constraint is shown on the graph by shading that side of the constraint contour.
For the example problem, the graph in Figure 2.1 shows that the region below the g line is infeasible. The second constraint being an equality, the feasible region for the problem is that segment of the h line that is above the intersection with the g line.
*2
After all constraint contours are drawn, the intersection of feasible sides of all constraints represents the feasible region for the optimization problem. The points inside the feasible region satisfy all constraints. Depending on the prob­
lem formulation, the feasible region may be disjoint, bounded, unbounded, empty, or just a single point. An empty feasible region obviously means that the problem as formulated does not have a solution. One must go back and reconsider the formulation. A single feasible point also represents a special situation where no further consideration is necessaiy. There is only one suit- able solution that meets all constraints, and thus there is no opportunity to
C h a p t e r 2 G r a p h i c a l O p t i m i s a t i o n
m i n i m i z e th e o b je ctive f u n c t i o n. I n o t h e r cases, we need to go t o th e n e x t step o f d ra w in g ob je ctive f u n c t i o n contours to d e te rm in e a n.o p tim u m, i f i t exists.
2.1.3 C o n t o u r s o f t h e O b j e c t i v e F u n c t i o n
For a well-posed o p t i m iz a ti o n p r o b le m —one w i t h a n o n e m p ty feasible r e g io n — t h e n e x t step is to select a p o i n t i n th e feasible do m ain t h a t has th e low e st value o f th e ob je ctive f u n c t i o n. T h is is achieved b y d ra w in g a fe w (a t least tw o ) contours f o r th e ob je ctive f u n c t i o n to d e te rm in e th e d ir e c t io n i n w h i c h th e o b je ctive f u n c t i o n is decreasing. V i s u a l l y, one can t h e n select a p o i n t t h a t is i n th e feasible do m ain as w e l l as has th e lo w e s t value o f th e ob je ctive f u n c t i o n.
D ra w in g o b je ctive f u n c t i o n contours poses one d i f f i c u l t y t h a t was n o t th e re i n th e case o f c o n s t r a in t f u n c t i o n contours. I n th e case o f c o n s tr a in t fu n c tio n s, i t was n a t u r a l to draw contours f o r zero f u n c t i o n values because such contours represented c o n s tr a in t boundaries. However, i n th e case o f ob je ctive fu n c tio n s, standard set o f values t h a t can always be used. One m u st p i c k a fe w values o f th e o bjective f u n c t i o n f o r w h i c h to draw th e contours. Values selected t o t a l l y r a n d o m ly u s u a lly w i l l n o t w o r k because th e corresponding contours may l i e outside th e range chosen f o r th e graph. A reasonable strategy is to p i c k tw o d i s t i n c t p o in ts on th e graph and evaluate th e o bjective f u n c t i o n / at these tw o p o in ts. For m ost problems, u sin g / values a t th e t h i r d p o i n t o f th e s o lu t io n do m ain w orks w e l l f o r t h e i n i t i a l ob je ctive f u n c t i o n contours. I n t h i s case, th e tw o values are computed as fo llo w s:
= / Climax — *lmm)> ~ (X2max ~ * 2 m in) ^
_ / 7. 2 \
c2 — f ( “ ( *lmax — *lmin)> ~ ( * 2 m a x — *2min) 1
Most l i k e l y, th e values o f /a t these tw o p o in ts w i l l be d if f e r e n t, say c\ and c%. I f not, d iffe re n t points are chosen u n til two distin ct values o f the objective fu nction are found.
A fte r selecting the contour values, the objective fu nction contours can be drawn in a manner sim ila r to that for the constraint fu nction contours. That is, we p ic k a few a rb itra ry values for one o f the variables, say x\, and solve for corresponding *2 values from / ( x i, *2) = c\. The lin e passing through these pairs o f points represents a contour for the objective fu nction value o f c\. Sim ilarly, a contour corresponding to / (jci , *2) — C2 is drawn. From these two contours, the general shape o f the objective fu n ctio n and the direction in which
i t decreases s h o uld become apparent. One can n o w guess at th e o p t i m u m p o i n t f r o m the graph. I b c o n f i r m t h i s guess, one needs to draw tw o more contours, one c o rresp o n d in g t o th e m i n i m u m value o f th e o bjective f u n c t i o n, and th e o th e r w i t h a s l i g h t l y lo w e r value o f /. T h is l a s t co n to u r should c l e a r l y be outside th e feasible region.
C o n t in u in g w i t h th e example f r o m th e p re v io u s section, th e t h i r d p o in ts o f the chosen o p t i m iz a t i o n space are computed as fo llo w s:
1 ,1 0 1 106 P o i n t l: ^ = - ( 1 0 - 0 ) = — x2 = - (102 - ( - 4 ) ) = ----------------
Point 2: X! = | ( 1 0 - 0) = y *2 = |(1 0 2 - ( - 4 ) ) = ^
Evaluating the objective function values at these two points, we select the follow ing two values fo r the objective fu n c tio n contours:
r fl0 106 \ . /2 0 212\
£ ι =/( τ'χ ) = 700 « -/( t · — ) = 2'810
O u r t a s k n o w i s t o d r a w l i n e s p a s s i n g t h r o u g h p o i n t s t h a t s a t i s f y t h e f o l l o w i n g e q u a t i o n s:
O b j e c t i v e f u n c t i o n c o n t o u r 1: 4xf — 5*ijc2 4- x% = 700 Objective fu n ctio n contour 2: 4xj — 5x1x2 + *2 = 2,810
Ib generate the data to draw these contours, we take several points along the xi axis and compute the corresponding X2 values b y solving the corresponding equation. The computed values are shown in the follow ing table.
Xl
X2 from — 5*1*2 + *2 — 700
*2 from 4xf — 5*1*2 + *2 = 2,810
0
26.4575
53.0094
2.5
32.9719
59.3919
5.
40.
66.0374
7.5
47.5
72.9401
10.
55.4138
80.0908
Plotting these contours together with the constraint contours, we get the graph shown in Figure 2.2.
C h a p t e r 2 G r a p h i c a l O p t i m i y - a t i o n
X2
FIGURE 2.2 First complete graph of the example problem.
2.1.4 S o l u t i o n o f t h e E x a m p l e P r o b l e m
In this section, we continue with the example problem. From Figure 2.2, we see that the feasible region (area where all constraints are satisfied) is the small line near the origin. It is clear that we should adjust ranges for both x\ and X2 to get an enlarged view of the feasible domain. Also, the contour lines for nonlinear functions are not very smooth. Thus, we should use more points in plotting these contours.
Second Attempt
For the second attempt, we repeat the steps used in the previous sections but with the following range:
Xlmin = 0 Xlmax = 3 X2min = 0 *2 max == 15
The third points of this domain, and the corresponding objective function
values, are as follows:
2.1 P r o c e d u r e f o r G r a p h i c a l S o l u t i o n
Point 1: x\ = 1 X2 = 5 f — 4 Point 2: xi = 2 *2 = 10 / = 16
The data points to draw the four functions are shown in the following table:
Χ2 from
XI
■**
II
/ = 16
g = 0
h = 0
0
2. & —2
4. & —4
2.
6.
0.75
4.1697 & -0.4197
6.0302 & -2.28
2.5625
5.25
1.5
6.7604 & 0.7396
8.3394 & -0.8394
4.25
4.5
2.25
9.5481 & 1.7019
10.8586 & 0.3914
7.0625
3.75
3.
12.4244 Sf 2.5756
13.5208 & 1.4792
11.
3.
The resulting contours are shown in Figure 2.3.
0 0.5 1 1.5 2 2.5 3
FIGURE 2.3 Second graph for detailed example.
C h a p t e r 2 G r a p h i c a l O p t i m i z a t i o n
Third Attempt
Figure 2.3 is a big improvement over Figure 2.2 but still shows the feasible region that is a little smaller than the overall graph. Tb get even a better graph, the limits are adjusted again and the process repeated. The resulting graph is shown in Figure 2.4.
*2
Final Graph
Figure 2.4 shows the feasible region very clearly. The objective function con­
tours show that the minimum point is at the intersection of the two constraints. Solving the two constraint equations simultaneously, we get the following co­
ordinates of the intersection point:
Optimum: x* = 1.56 x* = 4.44 f* = —5.2
The final graph shown in Figure 2.5 shows several different objective function contours. Also, more points are used in drawing these contours for a smoother appearance. The contour with f — — 5.2 just touches the feasible region. As the
2.2 G r a p h i c a l S o l u t i o n F u n c t i o n
contour for/ = —10 demonstrates, all contours with smaller objective function values will lie outside of the feasible region. Thus, the minimum point is as shown in the figure.
*2
FIGURE 2.5 Final graph for detailed example.
2,2 G r a p h i c a l S o l u t i o n F u n c t i o n
The procedure for graphical optimization described in section 2.1 is imple­
mented in a Mathematica function called GraphicalSolution. The function internally calls the built-in function ContourPlot with appropriate contour values to be drawn. The function is included in the OptimizationTbolbox 'GraphicalSolution'
Neads["OptimizationToolbox'GraphicalSolution'"];
^GraphicalSolution
GraphicalSolution[f, {x,xmin, xmax}, {y,ymin,ymax}, options]. The function
draws contours of the objective function (f) and optionally those for
C h a p t e r 2 G r a p h i c a l O p t i m i z a t i o n
constraint functions. All contours are labelled. The function returns a graphics object. See Options[GraphicalSolution] .for a description of different options for this function.
The function accepts the following options:
OptionsUsage[GraphicalSolution]
{ObjectiveContours -» 5, Obj ectiveContourLabels -» True, Constraints-» {}, ConstraintLabels -» {}, ShadingOffset 0.1, ShadingThickness-» 0 . 015, EQConstraintThickness 0 .01, PrintPoints -*
False, GradientVectors -*
{}, GradientVectorScale -*
0 .25, PlotHistory -» {}}
ObjectiveContours: Number of objective function contours to be drawn or a list of specific values for which the contours are desired. Default is 0bjectiveContours->5.
ObjectiveContourLabels: Option for GraphicalSolution to put labels on the objective function contours. Default is Obj ectiveContourLabels->True.
Constraints: List of constraint expressions. A constraint can be
specified as a simple expression in which case it is assumed as a '<' constraint with a zero right hand side. A constraint can also be specified as left-hand-side '< , < (both assumed as <)', '>, > (both assumed as £)', or '==' right-hand-side. Such constraints are first converted to standard forn ('<' or '==' with right hand side zero). Because of these re-arrangements, the order of the constraints may be different than that used in the input list of constraints. Default is Constraints->{}.
ConstraintLabels: List of labels to be used to identify the constraints on the graph. Default is gi for LE constraints and hi for EQ constraints.
ShadingOffset: This factor is used to determine the offset for shading to show the infeasible side for inequalities. Default is ShadingOffset->0.1.
ShadingThickness: defines the thickness of shading to show the
infeasible side for inequalities. Default is ShadingThickness->0.015:
EQConstraintThickness: defines the thickness of line to show the equality constraint. Default is EQConstraintThickness->0.01.
PrintPoints: Option for GraphicalSolution to print values of few data points used to draw each contour. Default is PrintPoints->False.
GradientVectors->{{f 1, . . .},{fl_Label, . . .},{{xl,yl}, . . ·}}· Draw gradient vectors at given points. A list of functions, labels, and points
where vectors are drawn is expected. By default no gradient vectors
are drawn.
2.3 G r a p h i c a l O p t i m i z a t i o n E x a m p l e s
GradientVectorScale: Scale factor by which the gradient vectors are multiplied before plotting. Default is GradientVectorScale->0.25.
PlotHistory->{{xl,yl}, . . .}. Shows a joined list of points on the plot.
One use is to show history of search in numerical methods.
The last three options are useful in later chapters. The gradient vectors at selected points are useful to show physical interpretation of optimality conditions discussed in Chapter 4. The PlotHistoiy option superimposes a line joining the given points on the graph. This option is useful in later chapters to show the progress of numerical algorithms towards the optimum.
In addition to the above options, the function accepts all relevant options from standard Mathematica Graphics and ContourPlot functions. Particularly useful are the options to change the text format (using TfextStyle options) and the Epilog option used to place labels and text on the graphs. The Epilog is used frequently in the following examples.
2.3 G r a p h i c a l O p t i m i z a t i o n E x a m p l e s
Several examples are presented in this section to illustrate the graphical opti­
mization procedure and the use of the GraphicalSolution function.
2.3.1 G r a p h i c a l O p t i m i z a t i o n E x a m p l e
In this section, we consider the graphical solution of the following two variable optimization problems involving an inequality and an equality constraint:
Minimize f — e*1 — X\X2 + x2
Subject to ( ^ = ^ + | _ ~ 42f 0° ) ---------------------------------------------------
The following Mathematica expressions define the objective and the constraint functions:
f = ^ - X l x 2 + xL·
9 = 2xr + Xj - 2 Z 0 ;
h = +x2-4==0;
Using GraphicalSolution, we get the left graph shown in Figure 2.6. Since the objective function values for which contours are to be drawn are not specified,
C h a p t e r 2 G r a p h i c a l O p t i m i s a t i o n
five values are automatically selected internally. The feasible domain for this problem is that portion of the dark line (representing the equality constraint) which is on the feasible side of constraint g. From the contours of the objective function, it appears that the optimum will be somewhere in the third quadrant.
We redraw the graph with objective function contours drawn for {—1,1, 3,10}. The number of plot points is increased to 30 to get smoother contours. Also, the ShadingOfFset is reduced to 0.06. This brings the line showing the infeasible side closer to the constraint line. Note that the individual graphs (grl, gr2) arc first generated but are not shown. Using the GraphicsArray function, the two graphs are displayed side-by-side.
grl = GraphicalSolution[£, {X]_, -4, 4}, {x2' *}' Constraints -» {g, h}] ;
gr2 = GraphicalSolution[f, {xir
-4, 4}, {x2' ' Constraints -*
{ff, h},
ObjectiveContours -» {-1, 1, 3, 10}, PlotPoints -► 30, ShadingOffset -» 0.06] ;
Show[GraphicsArray[{{grl, ffr2 }}]];
*2 *2
FIGURE 2.6 First and second graphs for the example.
From Figure 2.6, one can easily see that the minimum is near / = — 1. Precise location of the minimum is difficult to determine from the graph. The following values are read directly from the graph and represent an approximate optimum.
xopt = {xx -» -1.8, x2 ->-0.8};
£/.xopt
- 0.6 3 4 7 0 1
2.3 G r a p h i c a l O p t i m i z a t i o n E x a m p l e s
lb confirm this solution, the final graph is drawn with this value included in the list of ObjectiveContours. Using the Epilog option of Mathematica's built-in graphics functions, we can label the optimum point as well.
GraphicalSolution[£, {Xjv -4/ 4}, {x2, -4, 4}, Constraints -*■ {g, h},
ObjectiveContours -» {-2, -0.64, 2}, PlotPoints -*■ 30, ShadingOf f set -*
0.06, Epilog -> {RGBColor [1/ 0,0], Line [{{-1.8, -0.8}, {-0.8, 0.2}}],
Text["«Optimum" ,{-0.8,0.45}]}];
*2
4
2
n
-2
X1
-4
-4 -3 -2 -1 0 1 2 3
FIGURE 2.7 Final graph for the example.
2.3.2 D i s j o i n t F e a s i b l e R e g i o n
Consider the solution of the following optimization problem involving two variables and one equality constraint.
Minimize f = (x + 2)2 + ( y - 3)2 Subject to 3X2 + 4xy + 6y - 140 = 0
C h a p t e r 2 G r a p h i c a l O p t i m i z a t i o n
The following Mathematica expressions define the objective and the constraint function.
f = (x + 2) 2 + (y-3)2; h = 3x2 + 4xy + 6y - 140 == Oi
Using GraphicalSolution, we get the graph shown in Figure 2.8. The feasible region is disjoint and consists of points that lie on the two dark lines represent­
ing the equality constraint. The optimum is approximately where the / = 32 contour just touches the lower line representing the equality constraint. Note that this contour is still slightly below the upper dark line representing the equality constraint. The optimum solution is as follows:
Approximate optimum: x* = —7 y* = 0.3 f* = 32.2
GraphicalSolution[f, {x, -10, 10}, {y, -10, 10},
Constraints -» {h}, ObjectiveContours -» {20, 32, 40, 50, 80}, PlotPoints -» 50, Epiloff-» {RGBColor[1, 0, 0] , Line[{{-7, 0.3}, {-2, 3}}] ,
Text[Optimum,{-2,3.5}]}];
y
FIGURE 2.8 Graphical solution of example with disjoint feasible domain.
2.3 G r a p h i c a l O p t i m i z a t i o n E x a m p l e s
For a problem with a disjoint feasible region, the optimum solution can change drastically even with a small change in the constraints or the objective function. This example demonstrates this behavior. If the constant 3 in the objective function is changed to 3.5, the solution for the modified problem is as shown in Figure 2.9. The optimum is approximately where the / = 32 contour just touches the upper line representing the equality constraint. The optimum solution is as follows:
Approximate optimum: x* = 3.2 y* = 5.8 f* = 32.4
Comparing this solution to that of the original problem, we see that the ob­
jective function values are close. However, the optimum points have changed drastically.
f = (x + 2)2 + (y-3.5) 2 ; h = 3x2 + 4xy + 6y - 140 = = 0 ;
GraphicalSolution[f, {x, -10, 10}, {y, -10, 10},
Constraints -*
{h}, ObjectiveContours -*
{20, 32, 40, 50, 80}, PlotPoints -» 50, Epilog -> {RGBColor [1, 0,0], Line [{{3.2, 5.8}, {-2, 3}}],
Text["Optimum",{-2,2.5}]}];
-10 -5 0 5 10
FIGURE 2.9 Graphical solution of example with slightly modified constraint.
Consider the building design problem presented in Chapter 1. The problem statement is as follows.
Tb save energy costs for heating and cooling, an architect is considering designing a partially buried rectangular building. The total floor space needed is 20,000 m2. Lot size limits the longer building dimensions to 50 m in plan. The ratio between the plan dimensions must be equal to the golden ratio (1.618) and each story must be 3.5 m high. The heating and cooling costs are estimated at $100 m2, of the exposed surface area of the building. The owner has specified that the annual energy Costs should not exceed $225,000. The objective is to determine building dimensions such that the cost of excavation is minimized.
A formulation for this problem involving five design variables is discussed in Chapter 1.
n = Number of stories d = Depth of building below ground h = Height of building above ground t — Length of building in plan w — Width of building in plan
2.3.3 B u i l d i n g D e s i g n
Find (n, d, h, I and w) in order to
Minimize / = dtw
Subject to
i _ 3 5
t = 1.618w 100(2 hi + 2hw + lw) < 22,5000 ί < 50
w < 50 nlw > 20000 V n > l,d > 0,h > 0,£ > 0 and w > 0 /
For a graphi cal s ol ut i on, we n e e d t o r e duc e t he numbe r o f vari abl es t o t wo. Reduc i ng t he f ormul at i on t o t hr e e vari abl es i s e a s y be c a us e o f t he t wo s i mpl e r e l at i ons hi ps b e t we e n t he opt i mi z at i on var i abl e s ( e qual i t y c ons t rai nt s ). Subs t i t ut i ng n = (d + h)/3.5 and I — 1.618w, we get the following formulation in terms of three variables.
2.3 G r a p h i c a l O p t i m i z a t i o n E x a m p l e s
Find (d, h, and w) in order to Minimize / = l.618dw2
( 100(5.236/iw + 1.618w2) < 225,000 \
Subject to
1.618w < 50 w < 50
0.462286(ii + h)w2 > 20,000 d > 0, h > 0, and w > 0
The third constraint is now redundant and can be dropped. In order to further reduce the number of variables to two, we must fix one of the variables to a reasonable value. For example, if we fix the total number of stories in the building to 25, the total building height is
d + h = 25 x 3.5 = 87.5 m, giving h = 87.5 — d
Thus in terms of two variables, the problem is as follows:
Find (d and w) in order to
Minimize / = 1.618cfw2
(
100(5.236(87.5 - d)w + 1.6l8w2) < 225,000 ^
1.6l8w < 50 40.45*2 , 20,000 d > 0 and w > 0
The following Mathematica expressions define the objective and the con- straint functions:
Clear[d, w] ;
£ = 1 - e ie d w 2 ;
9 = {100(5.236(87.5 - d)w+1.618W2) ί 225000, 1.618w ί 50, 40.45W2 £ 20000, d^0,w&0};
Using GraphicalSolution, we get the graph shown in Figure 2.10. Note that the objective function is divided by a factor of 10,000 to avoid a large number of digits on the labels of the objective function contours. The variable range and the objective function contours drawn obviously have been determined after several trials. The placement of labels has involved a few trials as well.
C h a p t e r 2 G r a p h i c a l O p t i m i z a t i o n
GraphicalSolution[j^^, {w, 10, 50}, {d, 50, 100}, Constraints -» ff,
AspectRatio-» 1, PlotPoints -» 25, ObjectiveContours {3, 6, 9, 12, 18}, Epiloff -*
{RGBColor [ 1, 0,0], Line [ { {22 .24, 75.05}, (15, 80}}] ,
Text [ -Optimum", {15, 81}], Text ["Feasible", (26, 90}]}];
Objectivefunction -»■ 0 . OOOlfilSdw2
g x -225000 + 100 (5.236 (87 .5 - d ) w + 1.618W2) s 0 ' g2 -» -50 + 1.618w < 0 LEConstraints -» g3 -» 20000 - 40.45W2 < 0
g4 -d < 0
g5 -> -w < 0
d
FIGURE 2.10 Graphical solution of building design example.
The optimum is clearly at the intersection of the first and the third con­
straint. The precise optimum can be located by computing the intersection of these two constraints as follows:
2.3 G r a p h i c a l O p t i m i z a t i o n E x a m p l e s
sol = Solve[{100 (5.236(87.5-d)w+ 1.618W2) == 225000, 40.45W2 == 20000},
{Wr d>]
{{d-> 75.0459, w-y
22.236}, {d-> 99.9541, w-► -22 .236} }
£/.solHID
60036 .7____________________________________________________________
The first solution corresponds to the desired intersection point. Thus, the optimum solution is as follows:
Optimum: w* = 22.24 m d* = 75.05 m f* = 60,036.7 m3
As you probably expected, this optimum solution indicates that we keep most of the building below ground. The height above ground is h = 87.5 - 75.05 = 12.45 m. The length in plan is i = 1.618 x 22.24 — 35.98 m.
2.3.4 P o r t f o l i o M a n a g e m e n t
Consider the graphical solution of a simple two-variable portfolio management problem. An investor would like to invest in two different stock types to get at least a 15% rate of return while minimizing the risk of losses. For the past six years, the rates of return in the two stock types are as follows:
Type
Annual rate of return
Blue chip stocks
15.24
17.12
12.23
10.26
12.62
10.42
Tfechnology stocks
12.24
19.16
26.07
23.46
5.62
7.43
The optimization variables are as follows:
X
Portion of capital invested in Blue chip stocks
y
Portion of capital invested in Tfechnology stocks
The given rates of return are used to define a two-dimensional list called returns as follows:
blueChipstocks = {15.24, 17.12, 12.23, 10.26, 12.62, 10.42}; techStocks = {12.24, 19.16, 26.07, 23.46, 5.62, 7.43};
returns = {blueChipstocks,techStocks}
C h a p t e r 2 G r a p h i c a l O p t i m i z a t i o n
{{15.24, 17.12, 12.23, 10.26, 12.62, 10.42},
{12.24, 19.16, 26.07, 23.46, 5.62, 7.43}}
Using Mathematica's Apply and Map functions (for a description of these functions, see the chapter on introduction to Mathematica), the average returns for each investment type are computed as follows:
averageRetums ■ Map [Apply [Plus, #] /Length[#]&, returns]
{12.9817,15.6633}
In order to define the objective function, we need to compute the covariance matrix, V. Using the formula given in Chapter 1, the following function is defined to generate the covariance coefficient between any two investment types.
covariance [x_, y_] i = Module [ {xb, yb, n = Length [x] },----------------------------------
xb = Apply [Plus, x] /n; yb = Apply [Plus, y] /n;
Apply [Plus, (x - xb) (y - yb) ] /n
];
Using the coVariance function and Mathematica’s built-in Outer function, the complete covariance matrix V is generated by the following expression:
Vtaat = Outer [covariance, returns, returns, 1] ; Mat rixFonn [Vtaat]
/ 6.14855 0.403411 \
\θ.403411 60.2815 j
The objective function and the constraints can now be generated by the following expressions:
vars = {x, y};
£ = Expand[vars.Vtaat.vars]
6.14855x2 + 0 . 806822xy +■ 60.2815Y2
g = {Apply [Plus, vars] == 1, average Re turns, vars £ 15}
{x + y == 1, 12.9817x + 15.6633y> 15}
The graphical solution is obtained by using GraphicalSolution as follows:
GraphicalSolution[£, {x, 0., 0.5}, {y, 0.5, 1},
Constraints -» g, Plot Points -» 30,
Epilog -» {RGBColor [ 1, 0, 0] , Line [{{.247, .753}, {.4, .8}}],
Text [Optimuin, { .4, .82}] }] ;
2.3 G r a p h i c a l O p t i m i z a t i o n E x a m p l e s
y
_____FIGURE 2.11 Graphical solution of portfolio management example.______
The optimum is clearly at the intersection of the two constraints. The precise optimum canbe located by computing the intersection of these two constraints as follows;
sol = Solve [ {x + y == 1, 12.982x + 15.663y == 15), {x, y>]
{{x-> 0.247296, y-> 0.752704}}
£/·soljlj
34,6795
Thus, the optimum solution is as follows:_____________________________
Optimum strategy: Blue chip stocks = 25% Tbchnology stocks = 75%
Risk= 35
C h a p t e r 2 G r a p h i c a l O p t i m i z a t i o n
2.4 P r o b l e m s
Use graphical methods to solve the following two variable optimization prob­
lems.
2.1. Minimize /(xi, x2) = 3xi - *2
Subject to ( Xl + *2 ~ 3 \
\ 2 *i + 4x2 < 2 /
2.2. Minimize /(*i, * 2) = 6xi + *2
S— tto ( - - - - )
2.3. Ma x i m i z e /( χ ι, X2) = — 6x + 9^
/ x - ^ > 2
Subject to I 3x + y > 1
\ 2x — 3>* > 3
2.4. Minimize / (xi, x2) = x2 + 2>»2 Subject to ( * + ^ 01 )
2.5. Minimize / (xi, x2) = x2 + 2>>2 - 24x - 20><
/ x + 2>> > 0 \ x + 2>> < 9 x + y < 8
Subject to
χ + y > 0
2.6. M i n i m i z e /( x1, Χ2) = x i + Q + ^
* 2 Xl
Subject to / χι + χ2 > 2 \
V Xl,X2>0 J
2.7. Maximize /f a, x2) - fa - 2)2 + (x2 - 10)2
xf + *2 < 50 Subject to [ xf+x% + 2xix2 - x\ - xi + 20 > 0
* i, *2 > 0
2.4 P r o b l e m s
2.8. Minimize f i x, y) = x2 + 2>*x + y2 — I5x — 20 y
, . / x2 + >*2 < 20 \
Subj ect t o ^ ^ ^ 1Q j
2.9. Minimize f i x, y) = ^
Subject to ( * + )
2.10. Minimize /(x, y) =
Subject to ( eX^ eyyf 12° )
2.11. Hawkeye foods owns two types of trucks. 'Ituck type I has a refrigerated capacity of 15 m3 and a nonrefrigerated capacity of 25 m3. Ttnck type II has a refrigerated capacity of 15 m3 and a nonrefrigerated capacity of 10 m3. One of their stores in Gofer City needs products that require 150 m3 of refrigerated capacity and 130 m3 of nonrefrigerated capacity. For the round trip from the distribution center to Gofer City, truck type I uses 300 liters of fuel, while truck type II uses 200 liters. Formulate the problem of determining the number of trucks of each type that the company must use in order to meet the store's needs while minimizing the fuel consumption. Use graphical methods to determine the optimum solution.
2.12. Dust from an older cement manufacturing plant is a major source of dust pollution in a small community. The plant currently emits 2 pounds of dust per barrel of cement produced. The Environmental Protection Agency (EPA) has asked the plant to reduce this pollution by 85% (1.7 lbs/barrel). There are two models of electrostatic dust collectors that the plant can install to control dust emission. The higher efficiency model would reduce emissions by 1.8 lbs/barrel and would cost $0.70/barrel to operate. The lower efficiency model would reduce emissions by 1.5 lbs/barrel and would cost $0.50/barrel to operate. Since the higher effi­
ciency model reduces more than the EPA required amount and the lower efficiency less than the required amount, the plant has decided to install one of each. If the plant has the capacity to produce 3 million barrels of cement per year, how many barrels of cement should be produced using each dust control model to meet the EPA requirements at a minimum cost? Formulate the situation as an optimization problem. Use graphical
methods to determine the optimum solution.
C h a p t e r 2 G r a p h i c a l O p t i m i z a t i o n
2.13. For a chemical process, pressure measured at different temperatures is given in the following table. Formulate an optimization problem to determine the best values of coefficients in the following exponential model for the data.
Pressure = ae@T
Tfemperature (T°C)
Pressure (mm of Mercury)
20
15.45
25
19.23
30
26.54
35
34.52
40
48.32
50
68.11
60
98.34
70
120.45
Use graphical methods to determine the optimum solution.
2.14. Consider the cantilever beam-mass system shown in Figure 2.0. The beam cross-section is rectangular. The goal is to select cross-sectional dimensions (b and H) to minimize weight of the beam while keeping the fundamental vibration frequency (ω) larger than 8 rad/sec. Use graphical methods to determine the optimum solution.
Section A-A
FIGURE 2.12 Rectangular cross-section cantilever beam with a suspended mass.
2.4 P r o b l e m s
The numerical data and various equations for the problem are as follows:
Fundamental vibration frequency
ω = y/ke/m radians/sec
Equivalent spring constant, ke
1 _ 1 . I3 Κ ~ τ + ΊΕ1
Mass attached to the spring
m = W/g
Gravitational constant
g = 386 in/sec2
Weight attached to the spring
W = 60 lbs
Length of beam
I = 15 in
Modulus of elasticity
E = 30 χ 106 lbs/in2
Spring constant.
fc = 10lbs/in2
Moment of inertia
J=!£in<
Width of beam cross-section
0.5 in < b < 1 in
Height of beam cross-section
0.2 in < h < 2 in
Unit weight of beam material
0.286 lbs/in3
2.15. A small electronics company is planning to expand two of its manu­
facturing plants. The additional annual revenue expected from the two plants is as follows.
From plant 1: 0.00002xf — *2 From plant 2: O.OOOOlx^
where X\
and x%
are the investments made into upgrading the facilities. Each plant requires a minimum investment of $30,000. The company can borrow a maximum of $100,000 for this upgrade to be paid back in yearly installments in ten years at an annual interest rate of 12%. The revenue that the company generates can earn interest at an annual rate of 10%. After the 10-year period, the salvage value of the upgrades is expected to be as follows:
For plant 1: O.lxi For plant 2: 0.15x2
Formulate an optimization problem to maximize the net present worth of these upgrades. Use graphical methods to determine the optimum solution.
CHAPTER THREE
M a t h e m a t i c a l P r e l i m i n a r i e s
This chapter presents some of the basic mathematical concepts that are used frequently in the later chapters. A review of matrix notation and some of the basic concepts from linear algebra are presented in the first section. Ikylor series approximation, which plays an important role in developing various optimization techniques, is presented in section 2. Definitions of the gradient vector and the Hessian matrix of functions of n variables are introduced in this section also. As an application of the Tkylor series, and also since nonlinear equations are encountered frequently in solving optimization problems, the Newton-Raphson method for the numerical solution of nonlinear equations is presented in the third section. A discussion of quadratic functions and convex functions and sets is included in the last two sections.
3.1 V e c t o r s a n d M a t r i c e s
3.1.1 N o t a t i o n a n d B a s i c O p e r a t i o n s
Matrix notation is a convenient way of organizing computations when dealing with a function of more than one variable. A vector with n components is
C h a p t e r 3 M a t h e m a t i c a l P r e l i m i n a r i e s
written as an η χ 1 column vector, as follows:
x =
( xi \
X2
\ χ» }
Vectors and matrices are denoted by bold letters in this text. The components of a vector are denoted by using subscripts. A vector is always defined as a column vector. Since the matrix transpose operation moves columns into rows, if a computation requires a row vector, it will be defined with a transpose symbol, as follows.
XT
= (xi Xl
Xn)
An m χ n matrix A and its transpose are written as follows:
( «11.
ai2
Clin
\
( an
021 · * ·
ami \
A =
a21
a22
... C2n
AT =
ai2
022 - ■ ·
am2
(1mxn)
\ aml
am2
amn
)
(nxm)
\ ai„
a2w
&mn J
The entries an, «23,, am„ are called the main diagonal of a matrix. If the number of rows and columns are the same, the matrix is called a square matrix; otherwise, it is called a rectangular matrix. If the matrix after a transpose operation is the same as the original matrix, then the matrix is known as a symmetric matrix.
An identity matrix is a square matrix in which all entries on the main diagonal are equal to 1, and the rest of the entries are all 0. An identity matrix is usually denoted by I.
I
(nxn)
/ 1 0 0 1
. 0 \ . 0
l o 0 ... 1/
Multiplication of a matrix by a scalar means all entries in the matrix are multiplied by the scalar. Similarly, matrix addition and subtraction operations apply to each element. Clearly, matrix sizes must be the same for these op­
erations to make sense. The product of two matrices is generated by taking rows of the first matrix and multiplying by the columns of the second matrix.
3.1 V e c t o r s a n d M a t r i c e s
Therefore, for a matrix product to make sense, the number of columns in the first matrix must be the same as the number of rows in the second matrix. The result of multiplying an m χ ti matrix with an n x p matrix is a matrix of size m x p -
The following relationships, involving matrices of compatible sizes, are use­
ful to remember. Their proofs are given in most elementary books on linear algebra.
1. IA = A J = A
2. A B φ BA
3. A B C = A ( B C ) = (A B ) C
4. (AT) T= A
5. ( ABC) T = C TB TA T
6. Gi ve n A as a re ct angul ar mat ri x, A TA and A A T are s quare s ymme t r i c mat ri ­
ces.
Exampl e 3.1 In this example, some of the matrix operations are illustrated through numerical examples involving the following matrices. Notice that the matrix A is a rectangular matrix, matrix β is a square symmetric matrix, and c and d are column vectors.
1 2 3 4
A = I 5 6 7 8
9 10 11 12
B =
d =
(a) Addition, subtraction, and scalar multiplication
8
c + d =
2A = I 10 12 14 16 18 20 22 24
Obviously, operations such as A + B and B — c do not make sense because of incompatible sizes of the matrices involved.
(b) Transpose of A
( \ 5 9 \
2 6 10
3 7 11
V 4 8 12 y
(c) Matrix product AB The matrix product AB is not possible since the number of columns of A (= 4) is not the same as the. number of rows in B (= 3 )
(d) Matrix product BA
(
38 44 50 56 \
67 78 89 100 I
82 96 110 124 )
Note that the first entry'is obtained by multiplying the first row of matrix B with the first column of matrix A.
All other entries are computed in a similar way.
(e) Matrix square For square matrices, the following operation makes sense:
f 14 25 31 \
Br=BB-\ 25 45 56 I \ 31 56 70 /
(f) Matrix products A TA andAAr
(1, 2,3) x (1, 5, 9) = 1 + 10 + 27 = 38
/ 107 122 137 152 \
122 140 158 176
137 158 179 200
^ 152 176 200 224 ,
Notice that even though A is not a symmetric
ATA and
AAT result in symmetric matrices.
(g) Demonstration of (BA)T = A TBT
44 50
78 89
96 110
/ 38 67 82 \
50 89 110
^ 56 100 124 ;
3.1 V e c t o r s a n d M a t r i c e s
ATBT =
/ 1
5
9
\
2
6
10
3
7
11
^ 4
8
12
J
f
38
67
82
\
44
78
96
50
89
110
\
56
100
124
/
3.1.2 V e c t o r N o r m
The norm or length of a vector is defined as follows:
11*11 = y/x t + * i + "- + x%
The dot product or inner product of two n x l vectors x and y is a scalar given as follows:
XTy = Xiyi+ X2y2 H + XnYn
The dot product of two vectors gives the cosine of the angle between the two vectors.
τ
x y
COS# =
ll-vllilvll
The two vectors are orthogonal if their dot product is zero.
Using the dot product, the norm of a vector can be written as follows:
llx ll = + i - ■·· + *£ =
Example 3.2 Compute the length of vector c and the cosine of the angle between vectors c and d.
(a) Length of vector c
llcjl = Vc^c = λ/Ϊ4 = 3.74166
(b) Cosine of angle between vectors c and d
||c|| = 3.74166 \\d\\ = 8.77496
32
cTd = 32 cos^ =------------------------- = 0.974632
3.74166 x 8.77496
C h a p t e r 3 M a t h e m a t i c a l P r e l i m i n a r i e s
3.1.3 D e t e r m i n a n t o f a S q u a r e M a t r i x
The determinant of a 2 x 2 matrix is computed as follows:
_ / «11 «12 \ \ <*21 «22 /
Det [A] = \A\ =
«11 «12 «21 «22
= «11 «22 — «12«21
For larger matrices, the determinant is computed by reducing to a series of smaller matrices, eventually ending up with 2 x 2 matrices.
For a 3 x 3 matrix:
«11 «12 «13
A = I «21 «22 «23
«31 «32 «33
1A| =
«11 « 1 2 « 1 3 <*21 « 2 2 « 2 3
= «11
« 2 2 «23
— «12
«21 «23
+ «13
«31 « 3 2 «33
«21 «22 «3 1 « 3 2
For a 4 x 4 matrix:
( <*11 « 1 2 « 1 3 « 1 4 \
A =
« 2 1 « 2 2 « 2 3 « 2 4
« 3 1 « 3 2 « 3 3 « 3 4
\ « 4 1 « 4 2 « 4 3 « 4 4 /
\A\ = «11
« 2 2 « 2 3 « 2 4
« 3 2 « 3 3 « 3 4
— «12
«21 «23 «24
«31 «33 «34
+ «13
«21 «22 «24 «31 « 3 2 «34
« 4 2 «43 «44
«41 «43 «44
«41 «42 «44
— fl14
«21 « 2 2 «23
«31 « 3 2 «33
«41 «42 « 4 3
The determinants of each of the 3x3 matrices are evaluated as in the previous example. Notice that the sign of odd terms is positive and that of even terms is negative.
E x a m p l e 3.3 C o m p u t e t h e d e t e r m i n a n t o f t h e f o l l o w i n g 4 x 4 m a t r i x:
A =
{ 13 1 2 3 \
4 14 5 6
7 8 15 9
^ 10 11 12 16 /
14 5 6
4 5 6
4 14 6
= 13
8 15 9
—
7 15 9
+ 2
7 8 9
11 12 16
10 12 16
10 11 16
- 3
4 14 5
7 8 1?
10 11 12
14 5 6
8 15 9
11 12 16
= 14
15 9
12 16
- 5
8 9
11 16
+ 6
8 15
11 12
= 1289
Similarly,
4 5 6
4 14 6
4 14 5
7 15 9 10 12 16
= 22
7 8 9 10 11 16
= -210
7 8 15 10 11 12
= 633
Thus,
Det[A] = |A| = 13 x 1289 - 1 x 22 + 2 x -210 - 3 x 633 = 14416
3.1.4 I n v e r s e o f a S q u a r e M a t r i x
Given a square matrix A, its inverse is another square matrix, denoted by A-1,
such that the following relation holds:
AA"1 = A-1A =/
Efficient computation of the inverse of large matrices involves using the so- called Gauss-Jordan form, and will be discussed in chapter 6. For small ma­
trices and hand computations, the inverse can be computed by using the following formula:
A'1 = — <Co[A])T
Det [ A]
C h a p t e r 3 M a t h e m a t i c a l P r e l i m i n a r i e s
where Det[A] is the determinant of matrix A and Co[A] refers to the matrix of cofactors of A. Each term in a cofactor matrix is computed using the following formula:
(Co[A])w = (-Di+' Det[4#]
where Ay is the:
ith row and the /th column
from the A matrix. The following relationships, involving matrices of compat­
ible sizes, are useful to remember. Their proofs are given in most elementary books on linear algebra.
1. (ABC)-1 = C~l B - lA l
2. ( A T 1 - ( A -') T
3. (A + B)-1 φΑ~ χ +B-1
Example 3.4 In this example, some of the matrix operations are illustrated through numerical examples involving the following matrices:
1 2 3 4
A = | 5 6 7 8
9 10 11 12
B =
(a) Inverse of matrix B The cofactor matrix can be computed as follows:
/
Co[B] =
“ · [ « Μ - “ K » ] “ ·[(> 9 ]
Kl Μ M - ~ [ G »)]
— Det
l ( - ) J “ · [ ( - ) ]
— Det
The determinant of the matrix B can easily be seen to be Det[B] = — 1. Thus, the inverse of matrix β is as follows:
3.1 V e c t o r s a n d M a t r i c e s
We can easily verify that this inverse is correct by evaluating the matrix product BB~l =
(b) Computation of [(AAr + B)B)]
- l
AAT + B =
30 70 110
70 174 278
110 278 446
31 72 113
(AAT + B) B= | 72 178 283
113 283 452
31 72 113
72 178 283
113 283 452
514 915 1,131
= I 1,277 2,271 2,804
2,035 3,618 4,466
The cofactors and the determinant of the resulting matrix are as follows:
-2,586
Co [(AAT + B) B] =\ 5,568
-2,841
D et[(AAT + B)B] = -303
3,058 -1,299
-6,061 2,373
3,031 -1,161
Taking the transpose of the cofactors matrix and dividing each term by the determinant, we get the following inverse matrix:
514 915 1,131
[(AAT+B)B]~1 = [ 1,277 2,271 2,804
2,035 3,618 4,466
- l
862 1,856 947
101 101 101
3,058 6,061 3,031
303 303 303
433 791 387
101 101 101
From the relationship of the product of inverses,
[(AAT + B)B]~l = B~l (AAT + B)_1 We can demonstrate the validity of this as follows:
31 72 113 v -1
(AAT+B)'1 = | 72 178 283
113 283 452
367
565
262
303
303
303
565
1?243
637
303
303
303
262
637
334
303
303
303
1 - 3
B l = I - 3
2 - 1
C h a p t e r 3 M a t h e m a t i c a l P r e l i m i n a r i e s
B~1(AAT + B) 1 =
1
-3
2 >
-3
3
-1
2
-1
0 J
862
1,856
367
565
262
303
303
303
565
1,243
637
303
303
303
262
637
334
303
305
303
947
101 101 101 3,058 6,061 3,031
303 303 303
433 791 387
101 101 101
which is the same as before.
3.1.5 E i g e n v a l u e s o f a S q u a r e M a t r i x
If A is an η χ n matrix and x is an η χ 1 nonzero vector such that Ar = λχ, then λ is called an eigenvalue of a square matrix A. Eigenvalues of a matrix play an important role in several key optimization concepts. The eigenvalues are obtained by solving the following equation, known as a characteristic equation of a matrix.
Det[A - λ/] = 0
where / is an identity matrix. For a nnx n matrix, the characteristic equation is a polynomial of nth degree in terms of λ. Solutions of this equation give n eigenvalues of matrix A.
Example 3.5 Compute eigenvalues of the following 3 x 3 matrix:
/ 13 1 2 \
A = I 4 14 5 I
\ 7 8 15 /
ί 13
1
2 >
1 / 1
0
0 \
13-λ 1 2
\Α-λΙ\ =
4
14
5
1
>*
ο
1
0 ]
=
4 14-λ 5
V 7
8
15 ,
' V °
0
7 8 15 - λ
Eval uat i ng t he de t e r mi nant, we g e t t he f o l l o wi ng charact e ri s t i c equat i on:
( 13 - λ) [ ( 14 - λ ) ( 15 - λ ) - 40] - [ 4( 15 - λ) - 35] + 2[ 32 - 7( 14 - λ) ] = 0
or
2,053 - 529λ + 42λ2 - λ 3 = 0
3.1 V e c t o r s a n d M a t r i c e s
This nonlinear equation has the following three solutions:
λι = 7.9401 λ2 = 11.4212 λ3 = 22.6387
Mathematica has a built-in function, called Eigenvalues, that gives the eigen­
values for any square matrix.
Eigenvalues[{{13.,1,2}, {4,14,5}, {7, 8, 15}}]
{22.6387,11.4212,7.9401}
3.1.6 P r i n c i p a l M i n o r s o f a S q u a r e M a t r i x
Principal minors of a matrix are the determinants of square submatrices of the matrix. The first principal minor is the first diagonal element of the matrix. The second principal minor is the determinant of the 2 x 2 matrix obtained from the first two rows and columns. The third principal minor is the determinant of the 3x3 matrix obtained from the first three rows and columns. The process is continued until the last principal minor, which is equal to the determinant of the entire matrix. As a specific example, consider Λ to be the following 3x3 matrix:
« 1 1 « 1 2 « 1 3
A = \ «21 « 2 2 « 2 3
0 3 1 « 3 2 « 3 3
This matrix has the following three principal minors:
A i = a n
A2 =
A 3 =
«11
«12
«21
«22
«11
«12
« 1 3
«21
«22
« 2 3
« 3 1
« 3 2
« 3 3
Example 3.6 Compute the principal minors of the following 3 x 3 matrix:
A =
C h a p t e r 3 M a t h e m a t i c a l P r e l i m i n a r i e s
Ai s= 13 A2 =
M =
13
4
1
14
= 178
13
1
2
4
14
5
= 2,053
7
8
15
3.1.7 R a n k o f a M a t r i x
The rank of a matrix is equal to the size (dimension) of the largest nonsingular square submatrix that can be found by deleting any rows and columns of the given matrix, if necessaiy. Since the determinant must be nonzero for a matrix to be nonsingular, to determine the rank of a matrix, one must find a square submatrix with a nonzero determinant. The dimension of the largest such submatrix gives the rank of the matrix.
A matrix of full rank is a square matrix whose determinant is nonzero.
Example 3.7 Compute the rank of the following 3x3 matrix.
A =
Since the matrix is already a square matrix, we tty computing the determinant of the entire matrix.
Det[A] =
13 1 2
4 14 5
7 8 15
= 2,053
Since the determinant is nonzero, the rank of the matrix is 3. Thus, the given matrix is of full rank.
E x a m p l e 3.8 C o m p u t e t h e r a n k o f t h e f o l l o w i n g 3 x 4 m a t r i x:
3.1 V e c t o r s a n d M a t r i c e s
The largest square submatrix that can be found from the given matrix is a 3 x 3. There are four possible submatrices depending upon which column is deleted:
2 3 4
1 3 4
6 7 8
= 0
5 7 8
= 0
10 11 12
9 11 12
1
2
4
1
2
3
5
6
8
= 0
5
6
7
9
10
12
9
10
11
The determinants of all possible 3x3 submatrices are 0. Therefore, the matrix is not of rank 3. Now we must try all 2 x 2 submatrices. The number of possibilities is now large, however, we don't need to try all possible combinations. We just need to find one submatrix that has a nonzero determinant, and we are done.
We see that the first 2x2 submatrix has a nonzero determinant. Thus, we have found a largest possible submatrix with a nonzero determinant and hence, the rank of given matrix A is equal to 2.
A function called Rank is included in the OptimizationTbolbox 'Common- Functions' package for computing the rank of matrices.
?Rank
Rank[a_] , returns rank of matrix a.
Rank[{{l, 2, 3,4}, {5,6, 7,8}, {9,10,11,12}}]
2
3.1.8 L i n e a r I n d e p e n d e n c e o f V e c t o r s
A given set of tt χ 1 vectors is linearly independent if any other n x l vector can be represented by a suitable linear combination of that set. As an example, consider the following set of two 2 x 1 vectors:
a=0) *=0 )
C h a p t e r 3 M a t h e m a t i c a l P r p l i m i T i a r i c s
These two vectors are linearly independent because any 2 x 1 vector can be represented by a combination of these two. For example,
c = ^ _ f 7 ^=10e-7&
A given set of vectors can be tested for linear independence by assembling a matrix A whose columns are the given vectors. The rank of this matrix de­
termines the number of linearly independent vectors in the set. If the rank is equal to the number of vectors in the given set, then they are linearly independent.
Since the rank of a matrix can never be greater than the number of rows in the matrix, and since the vectors are arranged as columns of this matrix, it should be obvious that the maximum number of linearly independent vectors is equal to the dimension of the vector.
Example 3.9 Check to see if the following vectors are linearly independent:
2 \
/ 1 \
/ - 9
- 5
h
2
8
2
Ό —
3
C —
4
- i
I 4 )
^ - i /
The vectors are arranged as columns of the following 4 x 3 matrix:
2 1 -9 '
-5 2 8
A = 2 3 4
,-1 4 -1 ,
Rank [A]
3
The rank of the matrix A is 3. Therefore, these vectors are linearly independent. Example 3.10 Check to see if the following vectors are linearly independent:
/ 2 \
( 1 ^
( ~ 4 )
- 5
h —
2
19
2
V —
3
c =
0
\ "I )
< 4 J
I 11 J
The vectors are arranged as columns of the following 4 x 3 matrix:
3.2 A p p r o x im a t i o n Using t he Thylor Series
2 1 - 4 '
- 5 2 19
A = 2 3 0 1
- 1 4 1 1 ,____________________________________________________________________________________________
R a n k [λ]
2
T h e r a n k o f t h e m a t r i x A i s 2. T h e r e f o r e, o n l y t w o o f t h e t h r e e v e c t o r s a r e l i n ­
e a r l y i n d e p e n d e n t. I t i s e a s y t o s e e t h a t t h e t h i r d v e c t o r i s a l i n e a r c o m b i n a t i o n o f t h e f i r s t t w o a n d c a n b e o b t a i n e d a s f o l l o w s:
- 3 a + 2 b
{ - 4,1 9,0,1 1 }
3.2 A p p r o x i m a t i o n U s i n g t h e T a y l o r S e r i e s
A n y d i f f e r e n t i a b l e f u n c t i o n c a n b e a p p r o x i m a t e d b y a p o l y n o m i a l u s i n g t h e T k y l o r s e r i e s. W e c o n s i d e r t h e f u n c t i o n o f a s i n g l e v a r i a b l e f i r s t a n d t h e n g e n e r a l i z e e x p r e s s i o n s f o r f u n c t i o n s o f t w o o r m o r e v a r i a b l e s.
3.2.1 F u n c t i o n s o f a S i n g l e V a r i a b l e
C o n s i d e r a f u n c t i o n / ( x ) o f a s i n g l e v a r i a b l e x. W e c a n a p p r o x i m a t e t h i s f u n c ­
t i o n a b o u t x u s i n g t h e T & y l o r s e r i e s a s f o l l o w s:
f ( x ) w f ( x ) + ~ x ) + \d - x ) 2 + -"
A l i n e a r a p p r o x i m a t i o n o f t h e f u n c t i o n i s o b t a i n e d i f w e r e t a i n t h e f i r s t t w o t e r m s i n t h e s e r i e s w h i l e a q u a d r a t i c a p p r o x i m a t i o n i s o b t a i n e d b y i n c l u d i n g t h e s e c o n d d e r i v a t i v e t e r m a s w e l l.
A M a t h e m a t i c a f u n c t i o n T a y l o r S e r i e s A p p r o x i s c r e a t e d t o p e r f o r m t h e n e c e s s a r y c o m p u t a t i o n s t o g e n e r a t e l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s. T h e
f u n c t i o n g r a p h i c a l l y c o m p a r e s l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s w i t h t h e o r i g i n a l f u n c t i o n. I n o r d e r t o s t a y c o n s i s t e n t w i t h t h e n o t a t i o n u s e d f o r f u n c ­
t i o n s o f m o r e t h a n o n e v a r i a b l e, t h e f i r s t d e r i v a t i v e o f / i s i n d i c a t e d b y V/, a n d t h e s e c o n d d e r i v a t i v e b y V 2/.
Chap ter 3 Mathematical PrRlitninaxies
N e e d s ["O p t i m i z a t i o n T o o l b o x'C h a p 3 T o o l s'■ ];
?T a y l o r S e r i e e A p p r o x
T a y l o r S e r i e s A p p r o x [ f, v a r s _ L i s t, p t - L i s t ], g e n e r a t e s l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s o f a f u n c t i o n 'f ( v a r s )' a r o u n d g i v e n p o i n t 'p t' u s i n g T a y l o r s e r i e s. I t r e t u r n s f u n c t i o n v a l u e a t g i v e n p o i n t a n d i t s l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s. F o r f u n c t i o n s o f o n e a n d t wo v a r i a b l e s, it- c a n a l s o g e n e r a t e g r a p h c o m p a r i n g a p p r o x i m a t i o n s w i t h t h e o r i g i n a l f u n c t i o n. To g e t g r a p h s t h e a r g u m e n t s m u s t b e i n t h e f o l l o w i n g f o r m. F o r f ( x ): T a y l o r S e r i e s A p p r o x [ f, { x,x m i n,x m a x }, p t ] . F o r f ( x,y ): T a y l o r S e r i e s A p p r o x [ f, {{x,xmi n,xma x}, {y,y mi n,y ma x } }, p t l
M a t h e m a t i c a h a s a m u c h m o r e p o w e r f u l b u i l t - i n f u n c t i o n c a l l e d S e r i e s t h a t g e n e r a t e s a T & y l o r s e r i e s a p p r o x i m a t i o n o f a n y d e s i r e d o r d e r. T h e m a i n r e a s o n f o r c r e a t i n g t h e T a y l o r S e r i e s A p p r o x f u n c t i o n i s t o s h o w t h e i n t e r m e d i a t e c a l c u l a t i o n s a n d t o a u t o m a t e t h e p r o c e s s o f g r a p h i c a l c o m p a r i s o n.
S e r i e s [ f, {x, xO, n}] g e n e r a t e s a p o w e r s e r i e s e x p a n s i o n f o r f a b o u t t h e p o i n t x = xO t o o r d e r ( χ - χ Ο ) Λη. S e r i e s [ f, {x, xO, nx}, {y, yO, ny}] s u c c e s s i v e l y f i n d s s e r i e s e x p a n s i o n s w i t h r e s p e c t t o y,t h e n x.
E x a m p l e 3.1 1 D e t e r m i n e l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s o f t h e f o l l o w ­
i n g f u n c t i o n a r o u n d t h e g i v e n p o i n t u s i n g t h e T a y l o r s e r i e s.
T h e l i n e a r a n d q u a d r a t i c T & y l o r s e r i e s a p p r o x i m a t i o n s a r e a s f o l l o w s:
L i n e a r a p p r o x i m a t i o n ->■ - 0.0 1 1 7 1 8 8 + 0.1 0 9 3 7 5 x
Q u a d r a t i c a p p r o x i m a t i o n - » - 0.0 0 2 9 2 9 6 9 + 0.0 3 9 0 6 2 5 x + 0.1 4 0 6 2 5 x 2
T h e o r i g i n a l a n d t h e a p p r o x i m a t e f u n c t i o n s a r e c o m p a r e d i n t h e f o l l o w i n g p l o t. N o t e t h a t a r o u n d x, t h e a p p r o x i m a t i o n i s v e r y g o o d. H o w e v e r, a s w e m o v e a w a y f r o m x, t h e a c t u a l f u n c t i o n a n d i t s l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s b e c o m e q u i t e d i f f e r e n t.
?S e r i e s
1
T h e f u n c t i o n a n d i t s d e r i v a t i v e s a t t h e g i v e n p o i n t x a r e a s f o l l o w s:
3.2 A p p r o x i m a t i o n Using the I j i y l o r Series
F I G U R E 3.1 C o m p a r i s o n o f l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s f o r a f u n c t i o n o f a s i n g l e v a r i a b l e.
E x a m p l e 3.1 2 D e t e r m i n e l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s o f t h e f o l l o w ­
i n g f u n c t i o n a r o u n d t h e g i v e n p o i n t u s i n g t h e T & y l o r s e r i e s.
f ( x ) = x 2 c o s p y ) x = ~
T a y l o r S e r i e s A p p r o x [ x 2 C o s [ χ ] , { x, - 2, 2 } , π/4 ];
f - > x 2 C o s [ x ] f { ^ } 0.4 3 6 1 7 9
V f -» ( x ( 2 C o s [ x ] - x S i n [ x ] ) ) V f { - » ( 0.6 7 4 5 4 2 )
V2 f - » ( - ( - 2 + x 2 ) C o s [ x ] - 4 x S i n [ x ] ) V2 f [ f } -» ( - 1.2 4 3 4 1 )
Δχ^ ( - j + x)
L i n e a r a p p r o x i m a t i o n - » - 0.0 9 3 6 0 4 8 + 0.6 7 4 5 4 2 x
Q u a d r a t i c a p p r o x i m a t i o n - » - 0 .4 7 7 1 0 3 + 1.6 5 1 1 1 x - 0.6 2 1 7 0 3 x 2
F I G U R E 3.2 C o m p a r i s o n o f l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s f o r a f u n c t i o n o f a
s i n g l e v a r i a b l e.
Ch ap te r 3 Mathematical Preliminaries
3.2.2 F u n c t i o n s o f T t o o V a r i a b l e s
C o n s i d e r a f u n c t i o n / ( * i, X 2) o f t w o v a r i a b l e s. W e c a n a p p r o x i m a t e t h i s f u n c t i o n a b o u t a k n o w n p o i n t { x i, * 2} u s i n g t h e T a y l o r s e r i e s a s f o l l o w s:
- χ , Γ 3/( * ΐ · * 2 ),_ , 3f &,x ), _ J
f i x 1, x a ) » /( X i - * 2) + I — ^ --------( x\ ~ x i ) + ~ d x 2 ( X 2 ~ X 2)\
1 1 -,,ι ·,·,» g ^ i, - W C - « J
T h i s e x p r e s s i o n c a n b e w r i t t e n i n a m o r e c o m p a c t f o r m b y u s i n g m a t r i x n o t a ­
t i o n, a s f o l l o w s:
/d 2 f i x 1,X 2 ) d 2 f i x l,x 2 )\
+ 2 “ * 1 χ 2 - * 2 )
^ d x\ 3 x 2
D e f i n e a 2 χ 1 v e c t o r c a l l e d a g r a d i e n t v e c t o r a n d a 2 x 2 m a t r i x c a l l e d a H e s s i a n m a t r i x a s f o l l o w s:
G r a d i e n t v e c t o r, v/( x i,x 2) =
H e s s i a n m a t r i x, V 2/ ( χ χ, * 2) =
/ d f ( x 1,X 2 ) \
3xi
d f ( X!,X 2 )
\ 3x2
f 32/( x i,x 2) 32/( x i,x 2) ^
3 x j 3 x i 3 x 2
d 2 f ( x 1,X 2 ) 3 2/( X l,X 2 )
V 3 x i 3 x 2
N o t i n g t h a t
(
3/ ( x i, x 2 ) 3/ ( x i, x 2 )
3 x i
9X2
)■
a 4
v f ( x i,x 2 y
3.2 A p p r o x im a t i o n Using t he Tfaylor Series
t h e t r a n s p o s e o f t h e g r a d i e n t v e c t o r a n d
( ί:ί ) - ( ϊ Μ ϊ ) - < * - * > -"
v e c t o r o f v a r i a b l e s s h i f t e d b y x, t h e I k y l o r s e r i e s f o r m u l a c a n b e w r i t t e n a s f o l l o w s:
/ ( * i, *2) * f ( x i ,X 2 ) + V/( x i, x2)TAx + ^ A x T V 2 f ( χ ι, x 2 ) A x + ■ ·.
E x a m p l e 3.1 3 D e t e r m i n e l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s o f t h e f o l l o w ­
i n g f u n c t i o n a r o u n d t h e g i v e n p o i n t u s i n g t h e T a y l o r s e r i e s.
f ( x,y ) ~ x y ( x, y ) = ( 2,2 )
y
F I G U R E 3.3 C o m p a r i s o n o f l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s f o r a f u n c t i o n o f t w o v a r i a b l e s.
T a y l o r S e r i e s A p p r o x [ x y, 1.5, 3}, 1.5, 3} }, { 2, 2 > ] ;
f { 2, 2 } - » 4.
V f j U O x ] ) « ί 2 - 2 » - ( s.7 7 2 5 9 )
V 2 f - ) ( 1X ~ 2 + Y ( - 1 + y ) y x ~ 1 + Y ( l + y L o g [ x ] ) \
\ x'1 + y ( l + y L o g [ x ] ) x y L o g [ x ] 2 1
C h ap te r 3 Mathematical P reliminaries
2 l r, I 2. 4.7 7 2 5 9 \
V -* | 4 _ 7 7 259 1.9 2 1 8 1 J
Δ χ - » ( - 2 + x - 2 + y )
L i n e a r a p p r o x i m a t i o n - » - 9.5 4 5 1 8 + 4,x + 2.7 7 2 5 9 y
Q u a d r a t i c a p p r o x i m a t i o n - » ■ 1 7.3 8 8 8 - 9.5 4 5 1 8 x + 1 .x 2 - 1 0.6 1 6 2 y
+ 4.7 7 2 5 9 x y + 0.9 6 0 9 0 6 Y 2
E x a m p l e 3.1 4 D e t e r m i n e l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s o f t h e f o l l o w ­
i n g f u n c t i o n a r o u n d t h e g i v e n p o i n t u s i n g t h e I & y l o r s e r i e s.
/( *, y ) = 2X2 + y 2 + 1/(2** + y 2 ) ( x, y ) = (2, 2)
y
F I G U R E 3.4 C o m p a r i s o n o f l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s f o r a f u n c t i o n o f t w o v a r i a b l e s.
f = 2x 2 + y 2 + 1/ ( 2x 2 + y 2 ) ;
T a y l o r S e r i e s A p p r o x [ f, { { x, 1, 3 }, { y, 1, 3 } }, { 2,2 } ];
3.2 A p p ro x im a ti o n Using the Taylor Series
f -> 2 x 2 + y 2 + 2 χ ^ + f { 2, 2} -> 1 2.0 8 3 3
Vf
'
i 4
A
{2x2 + ) j
0
to
>
Γ ( 2 x 2 + y 2 ) 1! j
j
V f { 2,2}
7.9 4 4 4 4
3.9 7 2 2 2
1 6 x y
( 2 x 2 + y 2 ) ( 2 x 2 + y 2 )'
1 6 x y
( 2 x 2 + y 2 )
{ 4.0 4 6 3 0.1
7 f { 2, 2} -» ^ 0 3 7 0 3 7 2.
2 +
(2 X2 +Y2 )
8V2 __________ 2_
( 2 x 2 + y 2 ) ( 2 x 2 + y 2 )
4.0 4 6 3 0.0 3 7 0 3 7
00463
Λχ -> ( - 2 + x - 2 + y )
L i n e a r a p p r o x i m a t i o n -> - 1 1.7 5 + 7.9 4 4 4 4 x + 3 .9 7 2 2 2 y
Q u a d r a t i c a p p r o x i m a t i o n -» 0.5 - 0 - 2 2 2 2 2 2 x + 2 .0 2 3 1 5 x 2 - 0. l l l l l l y
+ 0.0 3 7 0 3 7 x y + 1.0 0 2 3 1 Y 2
3.2.3 F u n c t i o n s o f n V a r i a b l e s
F o r a f u n c t i o n o f n v a r i a b l e s, x = [ x\,x 2,.. .] T, t h e T k y l o r s e r i e s f o r m u l a c a n b e w r i t t e n b y d e f i n i n g a n η χ 1 g r a d i e n t v e c t o r a n d a n n x n H e s s i a n m a t r i x a s f o l l o w s:
G r a d i e n t v e c t o r, V f ( λ ) =
0
° 2l
3 x i 3x2
d2f \ 3 x i 3x„
H e s s i a n m a t r i x, V 2/ =
a 2/
3x 23x i
3 2f 3 x 2 3x„
d2f \ dx»dx2
d2f
dx„dx2
3 2f
/
T h e H e s s i a n m a t r i x i s a l s o d e n o t e d b y H f. F o r a c o n t i n u o u s l y d i f f e r e n t i a b l e f u n c t i o n, t h e m i x e d s e c o n d p a r t i a l d e r i v a t i v e s a r e s y m m e t r i c, t h a t i s
a 2/ a 2/
d x i d x j d x j d x i
Chapter 3
H e n c e, t h e H e s s i a n m a t r i x i s a l w a y s s y m m e t r i c. T h e T & y l o r s e r i e s a p p r o x i m a ­
t i o n a r o u n d a k n o w n p o i n t x c a n b e w r i t t e n a s f o l l o w s:,
/ ( x ) « f ( x ) - t - V/( x ) r A x + - A x T V 2/( x ) A x H---------
z
T h e f u n c t i o n s G r a d a n d H e s s i a n, i n c l u d e d i n t h e O p t i m i z a t i o n T b o l b o x 'C o m - m o n F u n c t i o n s' p a c k a g e, c o m p u t e t h e g r a d i e n t v e c t o r a n d H e s s i a n m a t r i x o f a f u n c t i o n o f a n y n u m b e r o f v a r i a b l e s.
N e e d s ["O p t l m l z a t i o n T o o l b o x'C o u u n o n F U i i c t i o n e' ” ]; ?G r a d
G r a d [ f, v a r s ], c o m p u t e s g r a d i e n t o f f u n c t i o n f ( o r a l i s t o f f u n c t i o n s ) w i t h r e s p e c t t o v a r i a b l e s v a r s.
?H e e s i a n
H e s s i a n [ f,v a r s ] C o m p u t e s H e s s i a n m a t r i x o f f u n c t i o n f ( o r a l i s t o f
f u n c t i o n s ).
E x a m p l e 3.1 5 D e t e r m i n e l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s o f t h e f o l l o w ­
i n g f u n c t i o n a r o u n d t h e g i v e n p o i n t u s i n g t h e T k y l o r s e r i e s.
f (x, y,z ) = x * + 1 2 x y 2 + 2 y 2 + 5 z 2 + x z 4 ( x, y, z ) = ( 1, 2, 3 ) ----------------
f = s t3 + 1 2 X Y 2 + l y 2 + 5 z 2 + x z *; T a y l o r S e r i e s A p p r o x [ f, { x, y, z }, { 1,2,3 } ];
f - » x 3 + 2 y 2 + 1 2 x y 2 + 5 z 2 + x z 4 f { 1, 2, 3 } - > 1 8 3.
3 x 2 + 1 2 y ^ + z * '
1 3 2. '
V f -»
4 ( y + 6 x y )
V f { l, 2, 3 } ->
5 6.
, 2 z ( 5 + 2 x z 2 ) ,
1 3 8. ,
’ 6 x
2 4 y
4 z 3 i
' 6.
4 8.
1 0 8.
2 4 y
4 + 2 4 x
0
V2 f { l, 2, 3 }
4 8.
2 8.
0
4 z 3
0
2 ( 5 + 6 x z 2 ) j
.1 0 8.
0
1 1 8.
Δ χ -> ( - 1 + x - 2 + y - 3 + z )
L i n e a r a p p r o x i m a t i o n - » - 4 7 5. + 1 3 2.x + 5 6.y + 1 3 8.z
Q u a d r a t i c a p p r o x i m a t i o n -»
5 3 5. - 2 9 4.x + 3.x 2 - 4 8 .y + 4 8 .x y + 1 4.y 2 - 3 2 4.z + 1 0 8. x z + 5 9.z 2
3.2.4 S u r f a c e s a n d T h e i r T a n g e n t P l a n e s
T h e T k y l o r s e r i e s a p p r o x i m a t i o n g i v e s a n i m p o r t a n t g e o m e t r i c a l i n t e r p r e t a t i o n t o t h e g r a d i e n t v e c t o r. I b s e e t h i s, c o n s i d e r t h e e q u a t i o n o f a s u r f a c e i n n
3.2 A p p r o x im a t i o n Using t h e Taylor Series
d i m e n s i o n s w r i t t e n a s f o l l o w s.
w h e r e c i s a g i v e n c o n s t a n t. A t a n a r b i t r a r y p o i n t, j c = ( * i,..., x n ), t h e l i n e a r a p p r o x i m a t i o n o f / i s a s f o l l o w s:
f i x 1, ... , Xn) & f i x 1, ...,* „ ) + [ V/( X 1, ..., Xn)Y
Xn — Xn
I f t h e p o i n t x i s o n t h e s u r f a c e, t h e n f i x ) — f i x ) = 0 a n d t h e r e f o r e, f o r a n y p o i n t o n t h e s u r f a c e
V f ( x ) T ( x — x ) = 0 o r V/( x ) T d — 0
w h e r e d — x — x.
T h i s e q u a t i o n s a y s t h a t t h e d o t p r o d u c t o f t h e g r a d i e n t v e c t o r w i t h a n y v e c t o r o n t h e s u r f a c e i s z e r o. F r o m t h e d e f i n i t i o n o f t h e v e c t o r d o t p r o d u c t, t h i s m e a n s t h a t t h e g r a d i e n t v e c t o r i s n o r m a l t o t h e s u r f a c e. F u r t h e r m o r e, i t c a n b e s h o w n t h a t a l o n g t h e g r a d i e n t v e c t o r l o c a l l y, t h e f u n c t i o n v a l u e i s i n c r e a s i n g t h e m o s t r a p i d l y.
T h e e q u a t i o n o f t h e t a n g e n t p l a n e t o t h e s u r f a c e a t j c i s g i v e n b y t h e e q u a t i o n
T k n g e n t p l a n e: V/( x ) T d = 0
T h e f o l l o w i n g t w o - a n d t h r e e - d i m e n s i o n a l e x a m p l e s i l l u s t r a t e t h e s e c o n c e p t s g r a p h i c a l l y:
E x a m p l e 3.1 6 C o n s i d e r t h e f o l l o w i n g f u n c t i o n o f t w o v a r i a b l e:
f i x, y ) = ^ + y 2
F o r s p e c i f i c f u n c t i o n v a l u e s, t h e c o n t o u r s a r e c i r c l e s c e n t e r e d a r o u n d t h e o r i ­
g i n. A s a n e x a m p l e, c o n s i d e r t h e c o n t o u r f o r / = 2. T h e e q u a t i o n f o r t h i s c o n t o u r i s
x 2 + y 2 = 2
T h e p o i n t ( 1,1 ) s a t i s f i e s t h i s e q u a t i o n a n d i s t h u s o n t h i s c o n t o u r. T h e g r a d i e n t ° f t h e f u n c t i o n a t ( 1,1 ) i s a s f o l l o w s.
Chap ter 3 Mathematical Preliminaries
T h e f t q i i a t i o n o f t h e t a n g e n t l i n e t o t h e c o n t o u r a t ( 1; 1 ) i s
( 2 2 ) ^ * “ J ^ = 0 o r 2 x + 2 y - 4 = 0
T h e c o n t o u r / = 2, t h e g r a d i e n t v e c t o r, a n d t h e t a n g e n t l i n e a t ( 1,1 ) a r e i l l u s t r a t e d i n t h e f o l l o w i n g f i g u r e. T h e f i g u r e c l e a r l y s h o w s t h e r e l a t i o n s h i p a m o n g t h e m.
y
F I G U R E 3.5 I k n g e n t l i n e a n d g r a d i e n t v e c t o r f o r a f u n c t i o n o f t w o v a r i a b l e s.
E x a m p l e 3.1 7 C o n s i d e r t h e f o l l o w i n g f u n c t i o n o f t h r e e v a r i a b l e s:
f ( x, y ) = x 2 + y 2 + z 2
F o r s p e c i f i c f u n c t i o n v a l u e s, t h e c o n t o u r s a r e s p h e r e s c e n t e r e d a r o u n d t h e o r i g i n. A s a n e x a m p l e, c o n s i d e r t h e c o n t o u r f o r / = 3. T h e e q u a t i o n f o r t h i s
T h e p o i n t ( 1,1,1 ) s a t i s f i e s t h i s e q u a t i o n a n d i s t h u s o n t h i s c o n t o u r. T h e g r a d i e n t o f t h e f u n c t i o n a t ( 1,1,1 ) i s a s f o l l o w s:
_______________________________ V/= ^ 2 y j V/{ 1,1,1 } = ^ 2
T h e e q u a t i o n o f t h e t a n g e n t p l a n e t o t h e c o n t o u r a t ( 1,1,1 ) i s
/* -!\
( 2 2 2 ) I y - 1 I = 0 o r 2 x + 2 ^ + 2 2 — 6 = 0
T h e c o n t o u r / = 3, t h e g r a d i e n t v e c t o r, a n d t h e t a n g e n t l i n e a t ( 1,1,1 ) a r e i l l u s t r a t e d i n t h e f o l l o w i n g f i g u r e. T h e f i g u r e c l e a r l y s h o w s t h e r e l a t i o n s h i p a m o n g t h e m.
c o n t o u r i s
x2+y2+z2~ 3
Chapter 3 Mathematical Preliminaries
3.3 S o l u t i o n o f N o n l i n e a r E q u a t i o n s
A p r a c t i c a l a p p l i c a t i o n o f t h e I k y l o r s e r i e s a p p r o x i m a t i o n i s t h e d e v e l o p m e n t o f t h e s o - c a l l e d N e w t o n - R a p h s o n m e t h o d f o r s o l v i n g t h e f o l l o w i n g n x n s y s t e m o f n o n l i n e a r e q u a t i o n s:
f l ( X l,X 2, = 0
f l i x i i X l i ...,* » ) = o
f n ( X l,X 2, ...,X n ) = 0
T h e m e t h o d s t a r t s f r o m a n i n i t i a l g u e s s a n d i t e r a t i v e l y t r i e s t o m o v e i t c l o s e r t o t h e s o l u t i o n o f t h e e q u a t i o n s. T h e b a s i c i t e r a t i o n i s w r i t t e n i n t h e f o l l o w i n g f o r m:
= x k + A x k k = 0,1,...
w i t h x 0 b e i n g t h e i n i t i a l g u e s s. T h e i m p r o v e m e n t A x k i s o b t a i n e d b y e x p a n d i n g t h e e q u a t i o n s a r o u n d x k u s i n g t h e T & y l o r s e r i e s, a s f o l l o w s:
f i ( x k+1) & f i ( x k) + V f 1 ( x k) T A x k = 0 f 2( x k + 1 ) * f i ( x k ) + V/2 ( x k ) T A x k = 0
W r i t i n g a l l n e q u a t i o n s i n a m a t r i x f o r m, w e h a v e
/ h t f ) \
/i t » * )
d f 2 ( x k )/d x i
d f 2 ( x k )/d x 2
· · ■ d f i ( ^ ) f d X n \ ■ * · d f 2 ( x k )/d x n
/ 0 \
0
V /» ( * * ) )
4 -
K d f n ( X k )/S X l
d/„ ( x k )/d x 2
· ·. d f n ( x k )/d x „ )
A x =
k 0 /
o r
f ( x k) + j ( x k) A x k = 0
T h e m a t r i x J o f p a r t i a l d e r i v a t i v e s o f t h e f u n c t i o n s i s k n o w n a s t h e J a c o b i a n m a t r i x. T h e i m p r o v e m e n t A x k c a n n o w b e o b t a i n e d b y i n v e r t i n g t h e J a c o b i a n m a t r i x, g i v i n g
3.3 S o lu ti o n o f Nonlinear Equations
W i t h e a c h i t e r a t i o n, t h e s o l u t i o n i n g e n e r a l w i l l i m p r o v e. T h a t i s, t a k e n t o ­
g e t h e r, f i ( x k + 1 ) s h o u l d b e c l o s e r t o z e r o t h a n /,· ( j c fc). T h u s, t h e c o n v e r g e n c e o f t h e a l g o r i t h m c a n b e d e f i n e d i n t e r m s o f t h e n o r m o f t h e v e c t o r f ( x k + 1 ).
Λ
1=1
w h e r e t o l i s a s m a l l t o l e r a n c e.
T h e N e w t o n - R a p h s o n m e t h o d c o n v e r g e s f a i r l y r a p i d l y w h e n s t a r t e d f r o m a p o i n t t h a t i s c l o s e t o a s o l u t i o n. H o w e v e r, t h e m e t h o d i s k n o w n t o d i v e r g e i n s o m e c a s e s a s w e l l b e c a u s e o f t h e J a c o b i a n m a t r i x b e i n g n e a r l y s i n g u l a r. A n o t h e r d r a w b a c k o f t h e m e t h o d i s t h a t i t g i v e s o n l y o n e s o l u t i o n t h a t i s c l o s e s t t o t h e s t a r t i n g p o i n t. S i n c e n o n l i n e a r e q u a t i o n s, i n g e n e r a l, m a y h a v e s e v e r a l s o l u t i o n s, t h e o n l y w a y t o g e t o t h e r s o l u t i o n s i s t o t r y d i f f e r e n t s t a r t i n g p o i n t s. M o r e s o p h i s t i c a t e d m e t h o d s f o r s o l v i n g n o n l i n e a r e q u a t i o n s c a n b e f o u n d i n t e x t b o o k s o n n u m e r i c a l a n a l y s i s.
T h e a b o v e a l g o r i t h m i s i i
T h e m a i n r e a s o n f o r c r e a t i n g t h i s f u n c t i o n i s t o s h o w a l l i n t e r m e d i a t e c o m p u ­
t a t i o n s. T h e s t a n d a r d M a t h e m a t i c a f u n c t i o n s N S o l v e a n d F i n d R o o t a r e m u c h m o r e f l e x i b l e a n d e f f i c i e n t f o r s o l v i n g g e n e r a l s y s t e m s o f n o n l i n e a r e q u a t i o n s, a n d s h o u l d b e u s e d i n p r a c t i c e.
N e e d s ["O p t i i n i z a t i o n T o o l b o x'C h a p 3 T o o l e'"];
?N e w t o n R a p h s o n
N e w t o n R a p h s o n [ f c n s, v a r s, s t a r t, m a x l t e r a t i o n s ], c o m p u t e s z e r o o f a
s y s t e m o f n o n l i n e a r f u n c t i o n s u s i n g N e w t o n - R a p h s o n m e t h o d, v a r s = l i s t o f v a r i a b l e s, f e n s = l i s t o f f u n c t i o n s, s t a r t = s t a r t i n g v a l u e s o f v a r i a b l e s, m a x l t e r a t i o n s = maximum n u m b e r o f i t e r a t i o n s a l l o w e d ( o p t i o n a l, D e f a u l t i s 1 0 ).
E x a m p l e 3.1 8 D e t e r m i n e a s o l u t i o n o f t h e f o l l o w i n g n o n l i n e a r e q u a t i o n u s ­
i n g t h e N e w t o n - R a p h s o n m e t h o d.
x + 2 = e*
( a ) S t a r t f r o m x ° = 2 U s i n g N e w t o n R a p h s o n f u n c t i o n c a l c u l a t i o n s f o r a l l i t e r a t i o n s a r e a s s h o w n b e l o w:
f = x + 2 - Exp [ χ ] ;
{ s o l, h i s t } = N e w t o n R a p h s o n [ f, x, 2] ;
J a c o b i a n m a t r i x - » (1 - Ex )
Chapter 3 Mathematical Preliminaries
*********** i t e r a t i o n 1 * * * * * * * * * * *
x k - > ( 2.) F ( x k ) -* ( - 3.3 8 9 0 6 ) | | F ( x k ) | | -» 3 .3 8 9 0 6
J ( x k ) ->■ ( - 6.3 8 9 0 6 )
I n v e r s e [ J ( x k ) ] -* ( - 0.1 5 6 5 1 8 )
Δ χ -> ( - 0.5 3 0 4 4 7 ) N e w x k -» ( 1.4 6 9 5 5 )
* * * * * * * * * * * i t e r a t i o n 2 * * * * * * * * * * *
x k - > ( 1.4 6 9 5 5 ) F ( x k ) ( - 0.8 7 7 7 3 8 ) | | F ( x k ) | | —> 0.8 7 7 7 3 8
J ( x k ) -» ( - 3.3 4 7 2 9 )
I n v e r s e [ J ( x k ) ] -» ( - 0.2 9 8 7 4 9 )
Δ χ -» ( - 0.2 6 2 2 2 3 ) N e w x k - » ( 1.2 0 7 3 3 )
* * * * * * * * * * * i t e r a t i o n 3 * * * * * * * * * * *
x k - » ( 1.2 0 7 3 3 ) F ( x k ) ( - 0.1 3 7 2 1 2 ) | | F ( x k ) | | - > 0.1 3 7 2 1 2
J ( x k ) -> ( - 2.3 4 4 5 4 )
I n v e r s e [ J ( x k ) ] -» ( - 0.4 2 6 5 2 3 )
Δ χ -»■ ( - 0.0 5 8 5 2 3 9 ) N e w x k - > ( 1.1 4 8 8 1 )
I t e r a t i o n 4
x k ^ ( 1.1 4 8 8 1 ) F ( x k ) -» ( - 0.0 0 5 6 1 7 4 8 ) | | F ( x k ) | | -> 0.0 0 5 6 1 7 4 8
J ( x k ) ->■ ( - 2.1 5 4 4 2 )
I n v e r s e [ J ( x k ) ] -» ( - 0.4 6 4 1 6 1 )
Δ χ - » ( - 0.0 0 2 6 0 7 4 2 ) N e w x k -» ( 1.1 4 6 2 )
* * * * * * * * * * * i t e r a t i o n 5 * * * * * * * * * * *
x k - > ( 1.1 4 6 2 ) F ( x k ) -» ( - 0.0 0 0 0 1 0 7 1 3 5 ) | | F ( x k ) | | -> 0.0 0 0 0 1 0 7 1 3 5
J ( x k ) -> ( - 2.1 4 6 2 1 )
I n v e r s e [ J ( x k ) ] - > ( - 0.4 6 5 9 3 8 )
Δ χ -*■ ( - 4.9 9 1 8 5 X 1 0"6 ) N e w x k -» ( 1.1 4 6 1 9 )
U s i n g t h e P l o t f u n c t i o n, t h e f u n c t i o n i s p l o t t e d a s s h o w n i n t h e f o l l o w i n g f i g u r e. T h e N e w t o n - R a p h s o n i t e r a t i o n s a r e a l s o s h o w n o n t h e p l o t.
P l o t [ f, { x, 1,2.1 }, P l o t R a n g e A l l, P l o t S t y l e { { G r a y L e v e l [ 0.7 ] } },
A x e s L a b e l -*■ {"x", "f"}, T e x t S t y l e { F o n t F a m i l y -> "T i m e s", F o n t s i z e 1 0 }, E p i l o g -» { R G B C o l o r [ 1, 0, 0 ] , L i n e [ ( {#1 [ [ 1 ] ], f/,x - » #1 [ [ 1 ] I } & } /© h i s t ] ,
T e x t ["S t a r t", { 2, - 3.4 }, { - 1, 0} ], T e x t ["S o l u t i o n", { 1.1 4 6, 0.2 } ] } ];
3.3 S olu ti on o f Nonlinear Equations
F I G U R E 3.7 G r a p h o f f u n c t i o n f ( x ) = x + 2 — e * a n d h i s t o r y o f N e w t o n - R a p h s o n i t e r a t i o n s w i t h = 2.
( b ) S t a r t f r o m x ° = 0.1 U s i n g t h e N e w t o n R a p h s o n f u n c t i o n w i t h t h i s s t a r t i n g p o i n t, t h e m e t h o d r u n s i n t o d i f f i c u l t i e s. T h e h i s t o r y o f p o i n t s c o m p u t e d b y t h e m e t h o d a r e a s f o l l o w s:
x
l i m n
0.1
0.9 9 4 8 2 9
9.5 5 9 1 7
0.9 9 4 8 2 9
8.5 5 9 9 1
1 4 1 6 2.4
7.5 6 1 7 4
5 2 0 7.6 5
6.5 6 6 2
1 9 1 3.6 3
5.5 7 6 8 6
7 0 2.0 9 6
4.6 0 1 8 4
2 5 6.6 6 3
3.6 5 8 6 2
9 3.0 6 6
2.7 8 1 8 4
3 3.1 4 9
2.0 3 1 4 8
1 1.3 6 6 8
1.4 8 9 0 4
3.5 9 3 9 2
1.2 1 4 1 1
0.9 4 3 7 9 7
1.1 4 9 4
0.1 5 3 1 8
1.1 4 6 2
0.0 0 6 8 9 9 8
1.1 4 6 1 9
0.0 0 0 0 1 6 1 4 1 4
Chap ter 3 Mathematical Preliminaries
N e a r * = 0.1, t h e f u n c t i o n i s v e r y f l a t a n d i t s f i r s t d e r i v a t i v e i s v e r y c l o s e t o z e r o. T h i s c a u s e s t h e m e t h o d t o a c t u a l l y g o v e i y f a r f r p m t h e s o l u t i o n i n t h e f i r s t f e w i t e r a t i o n s. F o r t u n a t e l y, t h e s l o p e o f t h e f u n c t i o n i s w e l l b e h a v e d i n t h i s r e g i o n a n d t h e m e t h o d e v e n t u a l l y c o n v e r g e s t o t h e s o l u t i o n. T h e f o l l o w i n g g r a p h i l l u s t r a t e s t h e b e h a v i o r:
E x a m p l e 3.1 9 S t a r t i n g f r o m * ° = ( 2, 2, 2 ), d e t e r m i n e a s o l u t i o n o f t h e f o l ­
l o w i n g s y s t e m o f n o n l i n e a r e q u a t i o n s u s i n g t h e N e w t o n - R a p h s o n m e t h o d.
—xf + *2 + *1* 2*3 = 1 - 3 4 *1*2 ~ *3 = 0.0 9 e * 1 - fr*2 + * 3 = 0.4 1
U s i n g t h e N e w t o n R a p h s o n f u n c t i o n, t h e f o l l o w i n g s o l u t i o n i s o b t a i n e d a f t e r f i v e i t e r a t i o n s.
0.9 0 2 2 1 8
* = | 1.1 0 0 3 4
0.9 5 0 1 3 2
C a l c u l a t i o n s f o r a l l i t e r a t i o n s a r e a s s h o w n b e l o w.
2 2 2 t = { x 1 x 2 s t 3 “ x l + * 2 “ 1 * 3 4, x ^ x j - * 3 - 0 .0 9,
E s t p E ^ ] - E x p [ x 2 ] + x 3 - 0.4 1 };
N e w t o n R a p h s o n [ f, { x 1# x 2, x 3 } „ { 2, 2, 2 } ];
J a c o b i a n m a t r i x - »
- 2 x x + x 2 x 3 2 x 3 + x x x 3 χ χ χ 2
- 2 x 3
X 1
- e X2
3.3 S o lu ti o n o f Nonlinear Equations
t * * * * * * * * * i t e r a t i o n 1 **********
x k -»
'2. 1 2.
F (xk) ->
'6.66 1 - 0.0 9
| | F ( x k ) | | - > 6.8 4 7 7 6
[ 2. J
[ 1.5 9
J ( x k )
0. 8. 4. ’
2. 2. - 4.
[ Ί. 3 8 9 0 6 - 7.3 8 9 0 6 1. ,
I n v e r s e [ J ( x k ) ]
f O.0 7 4 3 4 0 7 0.1 0 1 3 1 9 0.0 8 5 1 3 1 9 0.0 7 9 7 3 6 3
0.0 7 9 7 3 6 3
0.1 0 7 9 1 1 - 0.0 2 1 5 8 2 3
- 0.1 5 9 4 7 3 0.0 4 3 1 6 4 5
r - 0.6 5 7 5 7
1.3 4 2 4 3 1
Δ χ - >
- 0.5 2 5 4 8 6
N e w x k - »
1.4 7 4 5 1
- 0.6 1 4 0 2 8
1.3 8 5 9 7 j
* * * * * * * * * * * o n 2 * * * * * * * * * * *
i l.3 4 2 4 3
1.7 7 5 5 1
x k ->
1.4 7 4 5 1
F ( x k ) -»
- 0.0 3 1 4 8 6 5
[ l.3 8 5 9 7
[> 0.4 3 5 3 9 7 ,
J J F ( x k ) I I -» 1.8 2 8 3 9
J ( x k )
- 0.6 4 1 2 2 6 4.8 0 9 6 1.9 7 9 4 3
1.4 7 4 5 1 1.3 4 2 4 3 - 2.7 7 1 9 4
3.8 2 8 3 4 - 4.3 6 8 9 1 1.
I n v e r s e [ J ( x k ) ] -*
( 0.1 4 5 2 1 7 0.1 8 1 4 8 9 0.2 1 5 6 3 1
0.1 6 2 9 9 8 0.1 1 0 8 4 4 - 0.0 1 5 3 9 1 1
0.1 5 6 1 8 6 - 0.2 1 0 5 3 5 0.1 0 7 2 4 9
Λχ -»
U · J f t U U U j
- 0 * 2 7 9 2 1 4
U .
1 1953
- 0,3 3 0 6 3 5 ,
,1.0 5 5 3 4 ,
** * * * * * * * * * i t e r a t i o n 3 * * * * * * * * * * *
f0.9 9 6 4 2 5
r 0.3 5 2 8 1 3
x k -»
1.1 9 5 3
F ( x k ) ->
- 0.0 1 2 7 0 9 9
„ 1.0 5 5 3 4
,0.0 4 9 3 7 1 1
| F ( x k )
0.3 5 6 4 7 8
J (xk)
- 0.7 3 1 4 0 6 3.4 4 2 1 6 1.1 9 1 0 3
1.1 9 5 3 O'. 9 9 6 4 2 5 - 2.1 1 0 6 7
, 2.7 0 8 5 8 - 3.3 0 4 5 5 1.
I n v e r s e [ J ( x k ) ]
0.2 1 8 6 7 4 0.2 6 9 8 6 6 0.3 0 9 1 5 4
0.2 5 2 8 3 1 0.1 4 4 7 5 1 0.0 0 4 3 9 3 8 9 0.2 4 3 1 9 6 - 0.2 5 2 6 1 8 0.1 7 7 1 5 2
- 0.0 8 8 9 8 4 4 '
0.9 0 7 4 4 1 '
Δχ_»
- 0.0 8 7 5 7 9 4
New x k ->
1.1 0 7 7 2
,- 0.0 9 7 7 5 9 8
,0.9 5 7 5 7 7
Ch ap te r 3 Mathematical Preliminaries
***********
I t e r a t i o n 4 *** * * *
* * * * *
0.9 0 7 4 4 1 1
1 0.0 2 6 1 4 3 7 '
x k ->
1.1 0 7 7 2
F (xk)
- 0.0 0 1 7 6 3 7 9
,0.9 5 7 5 7 7 )
,- 0.0 0 1 8 9 8 7 2 ,
J F ( x k ) | | - > 0.0 2 6 2 7 1 8
J ( xk) ->
- 0.7 5 4 1 5 4 3.0 8 4 3 9 1.0 0 5 1 9
1.1 0 7 7 2 0.9 0 7 4 4 1 - 1.9 1 5 1 5
2.4 7 7 9 7 - 3.0 2 7 4 5 1.
I n v e r s e [ J ( xk) ] ->
Ό.2 4 4 5 6 4 0.3 0 6 4 2 0.3 4 1 0 0 9 '
0.2 9 2 7 1 2 0.1 6 2 2 7 2 0.0 1 6 5 4 4 8
,0.2 8 0 1 4 8 - 0.2 6 8 0 3 0.2 0 5 0 7 8 ,
- 0.0 0 5 2 0 5 8 5 '
0.9 0 2 2 3 5 ’
Δχ ->
- 0.0 0 7 3 3 4 9 4
New x k ->
1,1 0 0 3 9
,- 0.0 0 7 4 0 7 4 6 ,
, 0.9 5 0 1 7 ,
* * * * * * * * * * * i t e r a t i o n 5 * * * * * * * * * * *
Ό.9 0 2 2 3 5 '
1 0.0 0 0 1 5 5 0 0 3 '
xk- >
1.1 0 0 3 9
F (xk) ->
- 0.0 0 0 0 1 6 6 8 5 9
,0.9 5 0 1 7 j
- 0.0 0 0 0 4 7 7 2 2 1 ,
i 1F (xk) | | -> 0.0 0 0 1 6 3 0 3 9
J (xk) ->
- 0.7 5 8 9 1 7 3.0 5 8 0 5 0.9 9 2 8 0 6
1.1 0 0 3 9 0.9 0 2 2 3 5 - 1.9 0 0 3 4
2.4 6 5 1 1 - 3.0 0 5 3 2 1.
I n v e r s e [ J (xk) ] ->
0.2 4 6 2 0 2 0.3 0 9 3 2 0.3 4 3 3 8 3
0.2 9 6 1 7 1 0.1 6 4 1 5 3 0.0 1 7 9 0 5 2
i 0.2 8 3 1 7 7 - 0.2 6 9 1 7 5 0.2 0 7 3 3 6
- 0.0 0 0 0 1 6 6 1 3 7 '
0.9 0 2 2 1 8 '
Δχ ->
- 0.0 0 0 0 4 2 3 1 3 8
New xk- >
1.1 0 0 3 4
,- 0.0 0 0 0 3 8 4 9 0 2 ,
,0.9 5 0 1 3 2
3.4 Q u a d r a t i c F o r m s
A q u a d r a t i c f o r m i s a f u n c t i o n o f n v a r i a b l e s i h w h i c h e a c h t e r m i s e i t h e r a s q u a r e o f o n e v a r i a b l e o r i s t h e p r o d u c t o f t w o d i f f e r e n t v a r i a b l e s. C o n s i d e r t h e f o l l o w i n g f u n c t i o n s:
1. / ( * ι, x 2, x 3 ) = x? + X 1X 2 ~ 2x 1*3 - 1 7 * 2 ~ * 2*3 + 7 * §
2. f ( x 1, * 2. * 3) = *1 + * 1* 2*3 — 2 j c i * 3 — 1 7 * 2 — * 2*3 + 7 * §
3. / ( χ ι, X2, X3) = x f x 2 + * 1*2 — 2X1X3 — 1 7 * 2 — * 2 * 3 + 7 * §
O u t o f t h e s e t h r e e, o n l y t h e f i r s t f u n c t i o n i s a q u a d r a t i c f o r m. T h e s e c o n d t e r m i n t h e s e c o n d f u n c t i o n a n d t h e f i r s t t e r m i n t h e t h i r d f u n c t i o n a r e t h e r e a s o n s w h y t h e o t h e r t w o a r e n o t q u a d r a t i c f u n c t i o n s.
3.4 Quadratic Eorms
A l l q u a d r a t i c f o r m s c a n b e w r i t t e n i n t h e f o l l o w i n g m a t r i x f o r m:
fix) = \χ Τ Α χ
w h e r e jc i s t h e v e c t o r o f v a r i a b l e s a n d Λ i s a s y m m e t r i c m a t r i x o f c o e f f i c i e n t s o r g a n i z e d a s f o l l o w s.
( i ) T h e d i a g o n a l e n t r i e s o f m a t r i x A a r e t w i c e t h e c o e f f i c i e n t s o f t h e s q u a r e t e r m s. T h e f i r s t r o w d i a g o n a l i s t w i c e t h e c o e f f i c i e n t o f t h e x f t e r m, t h e s e c o n d r o w d i a g o n a l t e r m i s t w i c e t h e c o e f f i c i e n t o f t h e t e r m, a n d s o o n.
( i i ) T h e o f f - d i a g o n a l t e r m s c o n t a i n t h e c o e f f i c i e n t s o f t h e c r o s s - p r p d u c t t e r m s. T h e t e r m s i n t h e f i r s t r o w a r e c o e f f i c i e n t s o f X\X 2, * 1* 3,..x i x n - T h e s e c o n d r o w c o n t a i n s c o e f f i c i e n t s o f X 2X 1, * 2* 3, · · ·, * 2X n, a n d s o o n.
E x a m p l e 3.2 0 W r i t e t h e f o l l o w i n g f u n c t i o n a s a q u a d r a t i c f o r m:
2 2 2 2 f = - 4 x i - x ±x 2 - 5 x 2 + 2X 2X3 + 4 x 2x 3 - 6x 3 - 3 χ χ χ 4 - 5 x 2x 4 +■ 6X3X4 ~ 7 χ 4>
v a r s = { x 1# x 2, x 3, x 4 };
T h e s y m m e t r i c m a t r i x A a s s o c i a t e d w i t h f u n c t i o n f ( x\, x 2, * 3, * 4) i s a s f o l l o w s:
-8
-1
2
- 3
-1
-10
4
- 5
2
4
-12
6
i-3
- 5
6
- 1 4 ,
T h e f o l l o w i n g d i r e c t c o m p u t a t i o n s h o w s t h a t / ( j c ) = j x TA x: l/2 v a r s.λ.v a r s//E x p a n d
- 4 x f - x - l Xj - 5X2 + 2x ^ 3 + 4x 2x 3 - 6x 3 - 3X^ X4 - 5x 2X4 + 6X3X4 “ ^ x 4
3.4.1 D i f f e r e n t i a t i o n o f a Q u a d r a t i c F o r m
I t c a n e a s i l y b e v e r i f i e d b y d i r e c t c o m p u t a t i o n s t h a t t h e g r a d i e n t v e c t o r a n d H e s s i a n m a t r i x o f a q u a d r a t i c f o r m / (jc) = \x TA x c a n b e w r i t t e n a s f o l l o w s:
V/( j c ) - A x V 2/( x ) = A
E x a m p l e 3.2 1 V e r i f y t h e a b o v e g r a d i e n t a n d H e s s i a n e x p r e s s i o n s f o r t h e f o l l o w i n g q u a d r a t i c f o r m:
r i t a p t e r 3 Mathematical Preliminaries
f = -4X1 - ΧχΧ 2 - 5 X2 + 2 X l X 3 + 4 x 2 x 3 - 6 x 3 - 3x 1 * 4 - 5 x 2x 4 + 6X3X4 - 7 X^ ! v a r s = { x 2 , x 2 , x 3, x 4 };
B y d i r e c t l y d i f f e r e n t i a t i n g t h e g i v e n f u n c t i o n, t h e g r a d i e n t a n d H e s s i a n a r e a s f o l l o w s:
G r a d [ f, v a r s ] //M a t r i x F o r m
- 8X]_ - X2 + 2x 3 - 3x 4 - X! - 1 0 x 2 + 4x 3 - 5 x 4 2 χ χ + 4 x 2 - 1 2 x 3 + 6X4
— j X 2 + j
H e s s i a n [ f, v a r s ] //H a t r i x F o r m
i -8
-1
2
- 3 )
-1
-10
4
- 5
2
4
-12
6
1-3
- 5
6
- 1 4 ,
W e c a n s e e t h a t t h e H e s s i a n m a t r i x i s t h e s a m e a s t h e s y m m e t r i c m a t r i x A a s s o c i a t e d w i t h t h e q u a d r a t i c f o r m:
■-8
- 1
2
- 3 i
- 1
- 1 0
4
- 5
λ =
2
4
- 1 2
6
/
- 3
- 5
6
- 1 4 j
T h e f o l l o w i n g d i r e c t c o m p u t a t i o n v e r i f i e s t h e g r a d i e n t e x p r e s s i o n:
λ.v a r s//E x p a n d
{ - δ χ -L - x 2 + 2 x 3 - 3 χ 4 , - X! - 1 0 x 2 + 4 x 3 - 5 x 4,
2 χ χ + 4 x 2 - 1 2 x 3 + 6X4, - 3 χ ^ - 5 x 2 + 6x 3 - 1 4 x 4 }
3.4.2 S i g n o f a Q u a d r a t i c F o r m
D e p e n d i n g o n t h e s i g n o f t h e p r o d u c t x TA x f o r a l l p o s s i b l e v e c t o r s x, q u a d r a t i c f o r m s a r e c l a s s i f i e d a s f o l l o w s:
1. P o s i t i v e d e f i n i t e i f x T A x > 0 f o r a l l v e c t o r s j c.
2. P o s i t i v e s e m i d e f i n i t e i f x TA x > 0 f o r a l l v e c t o r s j c.
3. N e g a t i v e d e f i n i t e i f j c r A x < 0 f o r a l l v e c t o r s j c.
4. N e g a t i v e s e m i d e f i n i t e i f x T A x < 0 f o r a l l v e c t o r s x.
5. I n d e f i n i t e i f x T A x < 0 f o r s o m e v e c t o r s a n d x TA x > 0 f o r o t h e r v e c t o r s.
N o t e t h a t t h e s e d e f i n i t i o n s s i m p l y t e l l u s w h a t i t m e a n s f o r a m a t r i x t o b e p o s ­
i t i v e d e f i n i t e, s e m i d e f i n i t e, a n d s o o n. U s i n g t h e d e f i n i t i o n s a l o n e, w e c a n n o t
3.4 Quadratic Forms
d e t e r m i n e t h e s i g n o f a q u a d r a t i c f o r m b e c a u s e i t w o u l d i n v o l v e c o n s i d e r a t i o n o f a n i n f i n i t e n u m b e r o f v e c t o r s x. T h e f o l l o w i n g t w o t e s t s g i v e u s a p r a c t i c a l WSLy t o d e t e r m i n e t h e s i g n o f a q u a d r a t i c f o r m.
I f a l l e i g e n v a l u e s, λ (· i = 1 o f t h e η χ n s y m m e t r i c m a t r i x A i n t h e q u a d r a t i c f o r m / ( x ) = \x TA x a r e k n o w n, t h e n t h e s i g n o f t h e q u a d r a t i c f o r m i s d e t e r m i n e d a s f o l l o w s.
1. P o s i t i v e d e f i n i t e i f λ; > 0 i — 1,..., n.
2. P o s i t i v e s e m i d e f i n i t e i f λ; > 0 t = 1,..., n.
3. N e g a t i v e d e f i n i t e i f λ (· < 0 i = 1,..., n.
4. N e g a t i v e s e m i d e f i n i t e i f λ,· < 0 i = 1,..., n.
5. I n d e f i n i t e i f λ,· < 0 f o r s o m e i a n d λ,· > 0 f o r o t h e r s.
P r i n c i p a l M i n o r s T e s t f o r D e t e r m i n i n g t h e S i g n o f a Q u a d r a t i c F o r m
T h i s t e s t u s u a l l y r e q u i r e s l e s s c o m p u t a t i o n a l e f f o r t t h a n t h e e i g e n v a l u e t e s t. I f a l l p r i n c i p a l m i n o r s, A,· i = 1,..., rt, o f « x rt s y m m e t r i c m a t r i x A i n t h e q u a d r a t i c f o r m / ( x ) = ^ x T A x a r e k n o w n, t h e n t h e s i g n o f t h e q u a d r a t i c f o r m i s d e t e r m i n e d a s f o l l o w s:
1. P o s i t i v e d e f i n i t e i f A,· > 0 i = 1,..., rt.
2. P o s i t i v e s e m i d e f i n i t e i f A i > 0 i = 1,..., n.
ι j λ ·* - r ί -A* < 0 i = 1, 3, 5,... ( o d d i n d i c e s )
3. N e g a t i v e d e S m t e r f { ^ > Q J = 2 4,6 _ ( e v e n i n d i c e s ) ·
4. N e g a t i v e s e m i d e f i n i t e i f ί ^ ^ _ τ ’ c' k ^ ^ e s )
[ A{ > 0 t = 2, 4, 6,... ( e v e n i n d i c e s )
5. I n d e f i n i t e i f n o n e o f t h e a b o v e c a s e s a p p l i e s.
T h e f o l l o w i n g t w o f u n c t i o n s, c o n t a i n e d i n t h e O p t i m i z a t i o n T b o l b o x 'C o m m o n - F u n c t i o n s' p a c k a g e, r e t u r n p r i n c i p a l m i n o r s o f a m a t r i x a n d i t s s i g n:
C hap ter 3 Mathematical Preliminaries
N e e d s ["O p t i m i z a t i o n T o o l b o x'C o i m n o n F u n c t i o n s'"];
?P r i n c i p a l M i n o r s
? P x i n c i p a I H i n o r s
P r i n c i p a l M i n o r s [ A ] , r e t u r n s P r i n c i p a l m i n o r s o f a m a t r i x A. ?Q u a d r a t i c K o r m S i f l n
Q u a d r a t i c F o r m S i g n [ A ], d e t e r m i n e s t h e s i g n o f a q u a d r a t i c f o r m i n v o l v i n g s y m m e t r i c m a t r i x A u s i n g p r i n c i p a l m i n o r s t e s t. I t r e t u r n s t h e s i g n ( p o s i t i v e d e f i n i t e, e t c.) a n d t h e p r i n c i p a l m i n o r s o f A.
E x a m p l e 3.2 2 W r i t e t h e f o l l o w i n g f u n c t i o n a s a q u a d r a t i c f o r m a n d d e t e r ­
m i n e i t s s i g n u s i n g b o t h t h e e i g e n v a l u e t e s t a n d t h e p r i n c i p a l m i n o r s t e s t:
2 2 2 2 f = - 4 x i - x ± x 2 - 5X2 + 2 X i X3 + 4 x 2x 3 - 6x 3 - 3 x 1x 4 - 5 x 2x 4 + 6X3X4 - 7x 4 ;
3CV = {Χχ ,X 2,X 3,X 1 ) i
T h e s y m m e t r i c m a t r i x A a s s o c i a t e d w i t h t h e f u n c t i o n f ( x 1, * 2, * 3, * 4) i s a s f o l l o w s:
-8
-1
2
- 3
-1
-10
4
- 5
2
4
-12
6
1 - 3
- 5
6
- 1 4 \
T h e e i g e n v a l u e s o f m a t r i x A a r e a s f o l l o w s. T h e C h o p f u n c t i o n i s u s e d t o r o u n d o f f r e s u l t s r e t u r n e d b y t h e E i g e n v a l u e f u n c t i o n:
C h o p [ N[ E i g e n v a l u e s [ a ] ] ]
{ - 7. , - 6. , - 2 3.2 6 2 1,- 7.7 3 7 9 1 }
T h e q u a d r a t i c f o r m i s n e g a t i v e d e f i n i t e, s i n c e a l l e i g e n v a l u e s a r e n e g a t i v e. T h e f o l l o w i n g p r i n c i p a l m i n o r s t e s t r e t u r n s t h e s a m e c o n c l u s i o n ( o d d p r i n c i p a l m i n o r s a r e n e g a t i v e, a n d e v e n a r e p o s i t i v e ).
Q u a d r a t i c F o r m S i g n [ A ];
P r i n c i p a l m i n o r s -
- 8 7 9 7 9 6 ^ 7 5 6 0
Q u a d r a t i c f o r m - > N e g a t i v e P e f i n i t e
3.4 Quadratic Forms
E x a m p l e 3.2 3 F o r t h e f o l l o w i n g f u n c t i o n s o f t w o v a r i a b l e s, d e t e r m i n e t h e q u a d r a t i c f o r m s a n d t h e i r s i g n s. C o m p a r e t h e c o n t o u r p l o t s o f t h e s e f u n c t i o n s.
f ( x, y) = x2 + x y + 2y 2 g( x, y ) = x2 + x y - 2y 2
T h e s y m m e t r i c m a t r i c e s i n t h e q u a d r a t i c f o r m f o r t h e s e f u n c t i o n s a r e a s f o l ­
l o w s!
T h e p r i n c i p a l m i n o r s o f t h e t w o m a t r i c e s a r e
P M a = ( 2, 7 ) P M b = ( 2, — 9 )
T h u s, t h e f u n c t i o n / i s p o s i t i v e d e f i n i t e, w h i l e g i s i n d e f i n i t e. T h e t h r e e - d i m e n s i o n a l a n d c o n t o u r p l o t s o f t h e s e f u n c t i o n s a r e s h o w n i n t h e f o l l o w i n g f i g u r e. N o t e t h a t t h e p o s i t i v e d e f i n i t e f u n c t i o n h a s a w e l l - d e f i n e d m i n i m u m ( a t ( 0,0 ) f o r /), w h e r e a s t h e i n d e f i n i t e f u n c t i o n c h a n g e s s i g n b u t d o e s n o t h a v e a m i n i m u m.
F I G U R E 3.9 T h r e e - d i m e n s i o n a l a n d c o n t o u r p l o t s o f f u n c t i o n s / a n d g.
Chap ter 3 Mathematical Preliminaries
3.5 C o n v e x F u n c t i o n s a n d C o n v e x O p t i m i z a t i o n P r o b l e m s
O p t i m i z a t i o n p r o b l e m s t h a t a r e d e f i n e d i n t e r m s o f c o n v e x f u n c t i o n s a r e v e r y s p e c i a l i n o p t i m i z a t i o n l i t e r a t u r e. T h e r e a r e m a n y t h e o r e t i c a l r e s u l t s t h a t a r e a p p l i c a b l e o n l y t o s u c h p r o b l e m s. C o n v e r g e n c e o f m a n y o p t i m i z a t i o n a l g o ­
r i t h m s a s s u m e s c o n v e x i t y o f o b j e c t i v e a n d c o n s t r a i n t f u n c t i o n s. T h i s s e c t i o n b r i e f l y r e v i e w s t h e b a s i c d e f i n i t i o n s o f c o n v e x s e t s a n d f u n c t i o n s. A t e s t i s a l s o d e s c r i b e d t h a t c a n b e u s e d t o d e t e r m i n e w h e t h e r a f u n c t i o n i s c o n v e x o r n o t.
3.5.1 C o n v e x F u n c t i o n s
A f u n c t i o n / ( x ) i s c o n v e x i f f o r a n y t w o p o i n t s x ^ a n d x ^ 2\ t h e f u n c t i o n v a l u e s s a t i s f y t h e f o l l o w i n g i n e q u a l i t y:
/( a x ^ + ( 1 — α ) χ ^ ) < a/( x ^ ) + ( 1 — a )/( x ^ ) f o r a l l 0 < a < 1
F o r a f u n c t i o n o f a s i n g l e v a r i a b l e, t h i s m e a n s t h a t t h e f u n c t i o n i s c o n v e x i f t h e g r a p h o f / ( x ) l i e s b e l o w t h e l i n e j o i n i n g a n y t w o p o i n t s o n t h e g r a p h, a s i l l u s t r a t e d i n F i g u r e 3.1 0. A s o - c a l l e d c o n c a v e f u n c t i o n i s d e f i n e d s i m p l y b y c h a n g i n g t h e d i r e c t i o n o f t h e i n e q u a l i t y i n t h e a b o v e e x p r e s s i o n.
f ( x ) f (x )
F I G U R E 3.1 0 C o n v e x a n d n o n c o n v e x f u n c t i o n s.
U s i n g t h e d e f i n i t i o n, i t i s d i f f i c u l t t o c h e c k f o r c o n v e x i t y o f a g i v e n f u n c t i o n b e c a u s e i t w o u l d r e q u i r e c o n s i d e r a t i o n o f i n f i n i t e l y m a n y p o i n t s. H o w e v e r, u s i n g t h e s i g n o f t h e H e s s i a n m a t r i x o f t h e f u n c t i o n, w e c a n d e t e r m i n e t h e c o n v e x i t y o f a f u n c t i o n a s f o l l o w s:
3.5 Convex Funr.Hnr^ and Con ve x Op t im iz at i o n Problems
( a ) A f u n c t i o n / (jc) i s c o n v e x i f i t s H e s s i a n V 2/ (j c) i s a t l e a s t p o s i t i v e s e m i d e f ­
i n i t e.
( b ) A f u n c t i o n / (jc) i s c o n c a v e i f i t s H e s s i a n V 2/ ( χ ) i s a t l e a s t n e g a t i v e s e m i d e f ­
i n i t e.
( c ) A f u n c t i o n /( x ) i s n o n c o n v e x i f i t s H e s s i a n V 2/( x ) i s i n d e f i n i t e.
F o r a l i n e a r f u n c t i o n, t h e H e s s i a n m a t r i x c o n s i s t s o f a l l z e r o s, a n d h e n c e a l i n e a r f u n c t i o n i s b o t h c o n v e x a n d c o n c a v e a t t h e s a m e t i m e.
A s s e e n i n t h e p r e v i o u s s e c t i o n, t h e H e s s i a n o f a q u a d r a t i c f u n c t i o n i s e q u a l t o t h e c o e f f i c i e n t m a t r i x i n t h e q u a d r a t i c f o r m, a n d i s t h u s a m a t r i x i n v o l v i n g n u m b e r s. I t i s f a i r l y e a s y t o u s e t h e p r i n c i p a l m i n o r t e s t t o d e t e r m i n e t h e s i g n o f t h e H e s s i a n.
F o r m o r e c o m p l i c a t e d f u n c t i o n s, t h e H e s s i a n m a y b e a m e s s y m a t r i x i n ­
v o l v i n g v a r i a b l e s, a n d i t m a y n o t b e e a s y t o d e t e r m i n e i t s s i g n f o r a l l p o s s i b l e v a l u e s o f v a r i a b l e s. T h e f o l l o w i n g r e s u l t s, p r o v e d i n b o o k s o n c o n v e x a n a l y s i s, a r e o f t e n u s e f u l i n d e a l i n g w i t h m o r e c o m p l e x s i t u a t i o n s:
1. I f / ( x ) i s a c o n v e x f u n c t i o n, t h e n a f ( x ) i s a l s o a c o n v e x f u n c t i o n f o r a n y a > 0.
2. T h e s u m o f c o n v e x f u n c t i o n s i s a l s o a c o n v e x f u n c t i o n. T h a t i s, i f /,( x ), i —
1,..., k a r e a l l c o n v e x f u n c t i o n s, t h e n / ( x ) = Σ * = 1 /, ( x ) i s a c o n v e x f u n c t i o n.
3. I f / ( x ) i s a c o n v e x f u n c t i o n, a n d g ( y ) i s a n o t h e r c o n v e x f u n c t i o n w h o s e v a l u e i s c o n t i n u o u s l y i n c r e a s i n g t h e n t h e c o m p o s i t e f u n c t i o n g ( f ( x ) ) i s a l s o a c o n v e x f u n c t i o n.
S o m e f u n c t i o n s m a y n o t b e c o n v e x o v e r t h e i r e n t i r e d o m a i n; h o w e v e r, t h e y m a y b e c o n v e x o v e r a s p e c i f i e d s e t. T h e s e c o n c e p t s a r e i l l u s t r a t e d t h r o u g h t h e f o l l o w i n g e x a m p l e s:
E x a m p l e 3.2 4 D e t e r m i n e t h e c o n v e x i t y o f t h e f o l l o w i n g f u n c t i o n o f a s i n g l e v a r i a b l e:
f (x) = x + l/x x > 0
T h e H e s s i a n i s s i m p l y t h e s e c o n d d e r i v a t i v e o f t h i s f u n c t i o n:
■> 2 V / = — > 0 f o r x > 0.
Xr
T h e r e f o r e, t h e g i v e n f u n c t i o n i s c o n v e x o v e r t h e s p e c i f i e d d o m a i n.
T h e f o l l o w i n g p l o t c o n f i r m s t h i s g r a p h i c a l l y:
Chapter 3
P l o t [ x + 1/x, { x, 0.0 0 1, 2 },
T e x t S t y l e - > { F o n t F a m i l y - > “T i m e s", F o n c S i z e - > 10}] ;
F I G U R E 3.1 1 P l o t o f x + 1/x.
E x a m p l e 3.2 5 D e t e r m i n e t h e c o n v e x i t y o f t h e f o l l o w i n g f u n c t i o n o f a s i n g l e v a r i a b l e:
f ( x ) = x s i n [ x ]
T h e H e s s i a n i s s i m p l y t h e s e c o n d d e r i v a t i v e o f t h i s f u n c t i o n.
V 2/ = 2 c o s f x ] — x s i n [ x ]
F o r s o m e v a l u e s o f x, t h i s i s p o s i t i v e a n d f o r o t h e r s, i t i s n e g a t i v e. T h e r e f o r e, t h i s i s a n o n c o n v e x f u n c t i o n. T h e f o l l o w i n g p l o t c o n f i r m s t h i s g r a p h i c a l l y.
x S i n [ x ] & ( x S i n [ x ] )
FIGURE 3.12 Plot of x sin[x] and its second derivative o f x sin[x].
3.5 Convex F u n c ti o n « a n H Convex O p d m i z a t io n Problems
p r o m t h e g r a p h, o n e m a y b e t e m p t e d t o c o n c l u d e t h a t t h e f u n c t i o n i s c o n v e x o v e r — 2 < x < 2. H o w e v e r, t h e s e c o n d d e r i v a t i v e a c t u a l l y c h a n g e s s i g n a t x = ± 1.0 7 7, a s t h e g r a p h o f t h e s e c o n d d e r i v a t i v e o f t h e f u n c t i o n s h o w s. T h u s, b e t w e e n ( - π, π ), t h e f u n c t i o n i s c o n v e x o n l y o v e r - 1.0 7 7 < x < 1.0 7 7.
E x a m p l e 3.2 6 D e t e r m i n e t h e c o n v e x i t y o f t h e f o l l o w i n g f u n c t i o n o f t w o v a r i a b l e s:
f ( x,y ) = 5 — 5 x — 2 y + 2 x? + 5 x y - I- 6 y 2
H e s s i a n ■
5 \ -■ · f 4
5 1 2 ) P r i n c i P a l M i n o r s -* 2 3
S t a t u s C o n v e x
T h e f o l l o w i n g c o n t o u r p l o t c l e a r l y s h o w s t h a t t h e f u n c t i o n h a s a w e l l - d e f i n e d m i n i m u m:
G r a p h i c a l S o l u t i o n [ 5 - 5 x - 2 y + 2 x 2 + 5 x y + B y 2, { x, - 5, 5 }, { y, - 5, 5 }, O b j e c t i v e C o n t o u r s -* { 2, 5, 1 0, 3 0 }, P l o t P o i n t s -» 4 0 ] ;
y
F I G U R E 3.1 3 C o n t o u r p l o t o f / = 5 — 5 x — 2 y + l x 2 + 5 x y + 6 y 2.
Ch ap te r 3 Mathematical P reliminaries
E x a m p l e 3.2 7 D e t e r m i n e t h e .c o n v e x i t y o f t h e f o l l o w i n g f u n c t i o n o f t w o v a r i a b l e s:
f ( x, y ) = e (-x"+y ^ + e ^ x+2y^
T h e H e s s i a n m a t r i x f o r t h i s f u n c t i o n i s a s f o l l o w s:
V 2/
_ / 4 e * 2 +'2x 2 + e + 2 y + 4 e * * + y 2 x y - (- 2 e x + 2 y \
~ \ 4 e * * + y 2 x y + 2 e x + 2 y 4 e * * + y 2 y 2 + 4 e x + 2 y + 2 ^ + y l J
I t i s o b v i o u s l y h o p e l e s s t o t r y t o e x p l i c i t l y c h e c k t h e s i g n o f t h e p r i n c i p a l m i n o r s o f t h i s m a t r i x. H o w e v e r, w e c a n p r o c e e d i n t h e f o l l o w i n g m a n n e r t o s h o w t h a t t h e f u n c t i o n i s i n d e e d c o n v e x e v e r y w h e r e.
F i r s t, w e r e c o g n i z e t h a t t h e g i v e n f u n c t i o n i s t h e s u m o f t w o f u n c t i o n s. T h u s, w e n e e d t o s h o w t h a t t h e t w o t e r m s i n d i v i d u a l l y a r e c o n v e x. B o t h t e r m s a r e o f t h e f o r m e 1, w h e r e z i s a n y r e a l n u m b e r. A s t h e f o l l o w i n g p l o t d e m o n s t r a t e s, t h i s f u n c t i o n i s c o n t i n u o u s l y i n c r e a s i n g a n d i s c o n v e x. U s i n g t h e c o m p o s i t e f u n c t i o n r u l e, t h e r e f o r e, w e j u s t n e e d t o s h o w t h a t t h e p o w e r s o f e a r e c o n v e x f u n c t i o n s.
f l ( x, y ) = x 2 + y 2 V 2/i = ^ q 2 ) ===* <"'o n v e x * T h u 8, i s c o n v e x,
f 2 (x, y ) = x + 2 y L i n e a r = > C o n v e x. T h u s e ^ x,y ^ i s c o n v e x.
T h u s, e a c h t e r m i n d i v i d u a l l y i s c o n v e x a n d h e n c e, f i x, y ) = g ( * 2 + >'2 ) + e^ x + 2 ^ i s a c o n v e x f u n c t i o n.
P l o t [ e z, ( z, - 5/ 1 }, T e x t S t y l e -» { F o n t F a m i l y - * "T i m e s", F o n t s i z e -» 1 0 } ];
- 5 - 4 - 3 - 2 - 1 1
F I G U R E 3.1 4 P l o t o f e 2 s h o w i n g i t i s a c o n t i n u o u s l y i n c r e a s i n g c o n v e x f u n c t i o n.
3.5 Convex Func tion s and Convex Op t imizat ion Problems
T h e f o l l o w i n g c o n t o u r p l o t c l e a r l y s h o w s t h a t t h e f u n c t i o n h a s a w e l l - d e f i n e d m i n i m u m:
Gr a p h A c a l S o l u t i o n [ e 1* + e * * ^/ {*# - 2, 2 }, { y v ~ 2, 2 } t
O b j e c t i v e C o n t o u r s -* { 2, 5, 1 0, 2 0, 4 0 ), P l o t P o i n t s -► 4 0 ] ;
_ , V + 2 v 4- V^
O b j e c t i v e f u n c t i o n -> e y + e 1_________________________________________________________________________
F I G U R E 3.1 5 C o n t o u r p l o t o f / = e * + 2 y + ^ + y 2,
3.5.2
A s e t i s c o n v e x i f a s t r a i g h t l i n e j o i n i n g a n y t w o p o i n t s i n t h e s e t l i e s c o m p l e t e l y i n s i d e t h e s e t. O t h e r w i s e, i t i s n o n c o n v e x. T h e f o l l o w i n g f i g u r e s h o w s a n e x a m p l e o f a c o n v e x a n d a n o n c o n v e x s e t:
N o n c o n v e x s e t F I G U R E 3.1 6 C o n v e x a n d n o n c o n v e x s e t s.
Chapter 3 Mathematical Preliminaries
/
{ x - i ) 2 4 - ( y - 5 ) 2 < 1 0 2 X2 + 3 y 2 < 3 5 x > 0
y > 0 -----------------------
\
I t c a n e a s i l y b e v e r i f i e d t h a t n o n l i n e a r c o n s t r a i n t s g i a n d g 2 a r e b o t h c o n v e x f u n c t i o n s. T h e f e a s i b l e r e g i o n i s s h o w n i n F i g u r e 3.1 9 a n d c l e a r l y r e p r e s e n t s a c o n v e x s e t.
y
F I G U R E 3.1 9 T h e f e a s i b l e r e g i o n f o r a p r o b l e m i n v o l v i n g c o n v e x n o n l i n e a r i n e q u a l ­
i t y c o n s t r a i n t s.
E x a m p l e 3.3 1 O p t i m i z a t i o n p r o b l e m i n v o l v i n g n o n l i n e a r e q u a l i t y c o n s t r a i n t s T h e f e a s i b l e r e g i o n f o r a n o p t i m i z a t i o n p r o b l e m i n v o l v i n g n o n l i n e a r e q u a l i t y c o n s t r a i n t s i s n o t a c o n v e x s e t r e g a r d l e s s o f w h e t h e r t h e e q u a l i t y c o n s t r a i n t i s c o n v e x o r n o t. C o n s i d e r a p r o b l e m i n t w o v a r i a b l e s i n v o l v i n g t h e f o l l o w i n g f o u r c o n s t r a i n t s.
/ h \
( * ~ i ) + O' - 5 ) 2 = i o ^
2 X 2 + 3 y 2 < 3 5 x > 0 y > 0
g l S2
\ £3 /
3.5 Convex Func tion s «nd Convex Op timization Problems
T h e f u n c t i o n s a r e t h e s a m e a s t h o s e u s e d i n t h e p r e v i o u s e x a m p l e a n d h e n c e a r e a l l c o n v e x. H o w e v e r, t h e f e a s i b l e r e g i o n, l i n e s e g m e n t b e t w e e n c o n s t r a i n t s g j a n d g 2, i s n o t a c o n v e x s e t, a s s h o w n i n F i g u r e 3.2 0. T h i s i s b e c a u s e t h e l i n e j o i n i n g a n y t w o p o i n t s o n t h i s l i n e i s n o t i n t h e f e a s i b l e l i n e.
- 1 0 1 2 3 4 5 6
F I G U R E 3.2 0 T h e f e a s i b l e r e g i o n f o r a p r o b l e m i n v o l v i n g a n o n l i n e a r e q u a l i t y c o n ­
s t r a i n t.
3.5.3 C o n v e x O p t i m i z a t i o n P r o b l e m
A n o p t i m i z a t i o n p r o b l e m i n w h i c h t h e o b j e c t i v e f u n c t i o n i s c o n v e x a n d t h e f e a s i b l e r e g i o n i s a c o n v e x s e t i s k n o w n a s a c o n v e x p r o g r a m m i n g p r o b l e m. I t i s d e a r f r o m t h e d i s c u s s i o n i n t h e p r e v i o u s s e c t i o n t h a t a l i n e a r p r o g r a m ­
m i n g p r o b l e m i s a l w a y s a c o n v e x p r o g r a m m i n g p r o b l e m. A n o n l i n e a r p r o g r a m ­
m i n g p r o b l e m w i t h a c o n v e x o b j e c t i v e f u n c t i o n a n d i n v o l v i n g l i n e a r ( e q u a l i t y o r i n e q u a l i t y ) o r c o n v e x i n e q u a l i t y c o n s t r a i n t s i s a l s o a c o n v e x o p t i m i z a t i o n P r o b l e m. A p r o b l e m i n v o l v i n g e v e n a s i n g l e n o n l i n e a r e q u a l i t y c o n s t r a i n t i s a
n o n c o n v e x o p t i m i z a t i o n p r o b l e m.
Chapter 3 Mathematical Preliminaries
T h e m o s t i m p o r t a n t p r o p e r t y o f a c o n v e x p r o g r a m m i n g p r o b l e m i s t h a t a n y l o c a l m i n i m u m p o i n t x * i s a l s o a g l o b a l m i n i m u m. W e c a n p r o v e t h i s, b y c o n t r a d i c t i o n, a s f o l l o w s:
S u p p o s e x * i s a l o c a l m i n i m u m. I f i t i s n o t a g l o b a l m i n i m u m, t h e n t h e r e m u s t b e a n o t h e r l o c a l m i n i m u m x s u c h t h a t
f ( x ) < f ( x * )
C o n s i d e r t h e l i n e j o i n i n g t h e p o i n t s x * a n d ic a s f o l l o w s:
x = a x + f l — a ) x * 0 < a < 1
S i n c e t h e f e a s i b l e r e g i o n i s a c o n v e x s e t, a l l p o i n t s x o n t h i s l i n e a r e i n t h e f e a s i b l e s e t. F u r t h e r m o r e, s i n c e f i x ) i s a c o n v e x f u n c t i o n, w e h a v e
/ i x ) < < */( * ) + ( 1 — o c ) f i x * ) 0 < a < 1
o r
f { x ) < f i x * ) + <*[fix) - f i x * ) ]
F r o m t h e s t a r t i n g a s s e r t i o n, t h e t e r m i n t h e s q u a r e b r a c k e t m u s t b e l e s s t h a n z e r o, i n d i c a t i n g t h a t
f i x ) < / ( χ * ) f o r a l l 0 < a < 1
T h i s m e a n s t h a t e v e n i n t h e s m a l l n e i g h b o r h o o d o f x *, w e h a v e o t h e r p o i n t s w h e r e t h e f u n c t i o n v a l u e i s l o w e r t h a n t h e m i n i m u m p o i n t. C l e a r l y, t h e n χ * c a n n o t b e a l o c a l m i n i m u m, w h i c h i s a c o n t r a d i c t i o n. H e n c e, t h e r e c a n n o t b e a n o t h e r l o c a l m i n i m u m, p r o v i n g t h e a s s e r t i o n t h a t x * m u s t b e a g l o b a l m i n i m u m.
M o s t o p t i m i z a t i o n a l g o r i t h m s d i s c u s s e d i n l a t e r c h a p t e r s a r e d e s i g n e d t o f i n d o n l y a l o c a l m i n i m u m. I n g e n e r a l, t h e r e i s n o g o o d w a y t o f i n d a g l o b a l m i n i m u m e x c e p t f o r t o t r y s e v e r a l d i f f e r e n t s t a r t i n g p o i n t s w i t h t h e h o p e t h a t o n e o f t h e s e p o i n t s w i l l l e a d t o t h e g l o b a l m i n i m u m. H o w e v e r, f o r c o n v e x p r o b l e m s, o n e c a n u s e a n y o p t i m u m s o l u t i o n o b t a i n e d f r o m t h e s e m e t h o d s w i t h c o n f i d e n c e, k n o w i n g t h a t t h e r e i s n o o t h e r s o l u t i o n t h a t h a s a b e t t e r o b j e c t i v e f u n c t i o n v a l u e t h a n t h i s o n e.
T h e f o l l o w i n g f u n c t i o n, c o n t a i n e d i n t h e O p t i m i z a t i o n'I b o l b o x 'C o m m o n - F u n c t i o n' p a c k a g e, c h e c k s c o n v e x i t y o f o n e o r m o r e f u n c t i o n s o f a n y n u m b e r o f v a r i a b l e s:
N e e d s ["O p t i m i z a t i o n T o o l b o x ’ C o m m o n F u n c t i o n s'"];
?C o n v e x i t y C h e c k
C o n v e x i t y C h e c k [ f,v a r s ], c h e c k s t o s e e i f f u n c t i o n f ( o r a l i s t o f f u n c t i o n s f ) i s c o n v e x.
3.5 Convex Functions and Convex O p t im iz at i o n Problems
E x a m p l e 3.3 2 D e t e r m i n e i f t h e f o l l o w i n g o p t i m i z a t i o n p r o b l e m i s a c o n v e x p r o g r a m m i n g p r o b l e m:
M i n i m i z e f ( x, y ) = ( s - 5 ) 2 + ( y - l ) 2 + x y __________________________________
T h e p r o b l e m i n v o l v e s a n o n l i n e a r e q u a l i t y c o n s t r a i n t; t h e r e f o r e, i t i s n o t a c o n v e x p r o g r a m m i n g p r o b l e m.
E x a m p l e 3.3 3 D e t e r m i n e i f t h e f o l l o w i n g o p t i m i z a t i o n p r o b l e m i s a c o n v e x p r o g r a m m i n g p r o b l e m:
M i n i m i z e f ( x, y ) = ( x — 5 ) 2 + ( y — l ) 2 + x y
T h e p r o b l e m d o e s n o t i n v o l v e a n o n l i n e a r e q u a l i t y c o n s t r a i n t; t h e r e f o r e, i t i s c o n v e x i f a l l f u n c t i o n s i n t h e p r o b l e m a r e c o n v e x. W e u s e t h e C o n v e x i t y - C h e c k f u n c t i o n t o d e t e r m i n e t h e c o n v e x i t y s t a t u s o f o b j e c t i v e a n d c o n s t r a i n t f u n c t i o n s.
f = ( x - 5 ) 2 + ( y - I ) 2 + x y; g = { x + y 5 i,x J + y 2 t 2 x s 1 6, x 2 j y <. 1};
C o n v e x i t y C h e c k [ f, { x, y } ] ι
-------------------- F u n c t i o n - » ( - 5 + x ) 2 + ( - 1 + y ) 2 + x y
S t a t u s - » C o n v e x
C o n v e x i t y C h e c k [ g, { x, y } ];
F u n c t i o n - 4 + x + y
S t a t u s -> C o n v e x -----------------------------------------------
F u n c t i o n -» - 1 6 + 2 x + x 2 + y 2
Chap ter 3 Mathematical P reliminaries
S t a t u s
C o n v e x ■ F u n c t i o n ■
- 1 +
H e s s i a n ■
2
Y
2 x
2 x
Y
P r i n c i p a l M i n o r s
(!)
s t a t u s -»U n d e t e r m i n e d
N o t e t h a t s i n c e t h e p r i n c i p a l m i n o r o f t h e t h i r d c o n s t r a i n t c o n t a i n s a v a r i a b l e, t h e f u n c t i o n i s u n a b l e t o d e t e r m i n e i t s c o n v e x i t y a n d r e t u r n s "U n d e t e r m i n e d'’ s t a t u s. H o w e v e r, w e c a n s e e t h a t t h e f i r s t p r i n c i p a l m i n o r i s p o s i t i v e f o r y > 0. T h u s, t h e p r o b l e m i s c o n v e x w i t h t h e a d d i t i o n a l c o n s t r a i n t t h a t y m u s t b e p o s i t i v e. W i t h o u t t h i s c o n s t r a i n t, t h e p r o b l e m i n n o n c o n v e x. T h e o p t i m u m s o l u t i o n i n t h e f i r s t q u a d r a n t i s s h o w n i n t h e f o l l o w i n g f i g u r e.
G r a p h i c a l S o l u t i o n [ f, { x, 0.1, 5 ), { y, 0.1, 5 },
C o n s t r a i n t s -* g, O b j e c t i v e C o n t o u r s -» { 5, 1 0, 1 5, 2 2, 3 0 ),
S h a d i n g O f f s e t -» 0.0 5, P l o t P o i n t s -» 4 0,
E p i l o g - » { L i n e [ { { 1.5 6, 2.4 4 }, { 3, 3 } } ], T e x t [ "O p t i m u m" , { 3, 3.2 } ] } ];
4
0
0 2 3 4 5
F I G U R E 3.2 1 G r a p h i c a l s o l u t i o n.
3.6 Problems
3.6 P r o b l e m s
V e c t o r s a n d M a t r i c e s
U s e t h e f o l l o w i n g m a t r i c e s i n P r o b l e m s 3.1 t h r o u g h 3.8.
/ 1 - 2 3 - 4 \/ 1 2 -3 \
A = [ 5 6 — 7 8 j B = I 2 4 - 5 1
\ 9 - 1 0 1 1 1 2 / \ - 3 - 5 6 /
3.1. P e r f o r m t h e f o l l o w i n g m a t r i x c o m p u t a t i o n:
( A A T — 2 B ) c
3.2. P e r f o r m t h e f o l l o w i n g m a t r i x c o m p u t a t i o n:
A T B ~ 1 c
3.3. C o m p u t e t h e d e t e r m i n a n t o f t h e m a t r i x r e s u l t i n g f r o m t h e f o l l o w i n g c o m p u t a t i o n:
A A T - B/2
3.4. C o m p u t e t h e c o s i n e o f t h e a n g l e b e t w e e n v e c t o r d a n d t h e o n e r e s u l t i n g f r o m t h e c o m p u t a t i o n B d.
3.5. C o m p u t e t h e i n v e r s e o f t h e m a t r i x r e s u l t i n g f r o m t h e f o l l o w i n g c o m p u ­
t a t i o n:
A A T + B
3.6. C o m p u t e e i g e n v a l u e s o f t h e m a t r i x r e s u l t i n g f r o m t h e f o l l o w i n g c o m p u ­
t a t i o n:
A A T
3 - 7. C o m p u t e p r i n c i p a l m i n o r s o f t h e m a t r i x r e s u l t i n g f r o m t h e f o l l o w i n g c o m p u t a t i o n:
A T A
'T i a p t e r 3 Mathematical P reliminaries
S t a t u s C o n v e x
F u n c t i o n -» - 1 + —
y
H e s s i a n -»
S t a t u e U n d e t e r m i n e d
N o t e t h a t s i n c e t h e p r i n c i p a l m i n o r o f t h e t h i r d c o n s t r a i n t c o n t a i n s a v a r i a b l e, t h e f u n c t i o n i s u n a b l e t o d e t e r m i n e i t s c o n v e x i t y a n d r e t u r n s “U n d e t e r m i n e d" s t a t u s. H o w e v e r, w e c a n s e e t h a t t h e f i r s t p r i n c i p a l m i n o r i s p o s i t i v e f o r y > 0. T h u s, t h e p r o b l e m i s c o n v e x w i t h t h e a d d i t i o n a l c o n s t r a i n t t h a t y m u s t b e p o s i t i v e. W i t h o u t t h i s c o n s t r a i n t, t h e p r o b l e m i n n o n c o n v e x. T h e o p t i m u m s o l u t i o n i n t h e f i r s t q u a d r a n t i s s h o w n i n t h e f o l l o w i n g f i g u r e.
O r a p h i c & l S o l u t i o n [ f , { x,0.1, 5 }, { y, 0.1,5 ),
C o n s t r a i n t s -» g, O b j e c t i v e C o n t o u r s { 5, 1 0, 1 5, 2 2, 3 0 },
S h a d i n g O f f s e t -» 0.0 5, P l o t P o i n t s -» 4 0,
E p i l o g -* { L i n e [{ { 1.5 6, 2.4 4 }, { 3,3 } } ], T e x t ["O p t i m u m", { 3,3.2 } ] } ];
y
o
1
4
3
5
1
x
0 1
%
3
4
5
F I G U R E 3.2 1 G r a p h i c a l s o l u t i o n.
3.6 Problems
3.6 P r o b l e m s
V e c t o r s a n d M a t r i c e s
U s e t h e f o l l o w i n g m a t r i c e s i n P r o b l e m s 3.1 t h r o u g h 3.8.
/ 1 - 2 3 - 4 \ / 1 2 - 3 \
A = I 5 6 - 7 8 5 = 1 2 4 - 5 )
V 9 - 1 0 1 1 1 2 / \ - 3 - 5 6 /
3.1. P e r f o r m t h e f o l l o w i n g m a t r i x c o m p u t a t i o n:
( A A T - 2 B ) c
3.2. P e r f o r m t h e f o l l o w i n g m a t r i x c o m p u t a t i o n:
A T B ~ 1 c
3.3. C o m p u t e t h e d e t e r m i n a n t o f t h e m a t r i x r e s u l t i n g f r o m t h e f o l l o w i n g c o m p u t a t i o n:
A A T — B/2
3.4. C o m p u t e t h e c o s i n e o f t h e a n g l e b e t w e e n v e c t o r d a n d t h e o n e r e s u l t i n g f r o m t h e c o m p u t a t i o n B d.
3.5. C o m p u t e t h e i n v e r s e o f t h e m a t r i x r e s u l t i n g f r o m t h e f o l l o w i n g c o m p u ­
t a t i o n:
A A T + B
3.6. C o m p u t e e i g e n v a l u e s o f t h e m a t r i x r e s u l t i n g f r o m t h e f o l l o w i n g c o m p u ­
t a t i o n:
A A T
3.7. C o m p u t e p r i n c i p a l m i n o r s o f t h e m a t r i x r e s u l t i n g f r o m t h e f o l l o w i n g c o m p u t a t i o n:
A T A
Ch ap te r 3 Mathematical Preliminaries
3.8. C o m p u t e r a n k s o f m a t r i c e s A, B a n d c.
3.9. C h e c k t o s e e i f t h e f o l l o w i n g t h r e e v e c t o r s a r e l i n e a r l y i n d e p e n d e n t:
a =\ 2
b =
3.1 0. S h o w t h a t o n l y t w o o f t h e f o l l o w i n g t h r e e v e c t o r s a r e l i n e a r l y i n d e p e n ­
d e n t. F i n d t h e l i n e a r r e l a t i o n s h i p b e t w e e n t h e t h i r d v e c t o r a n d t h e t w o l i n e a r l y i n d e p e n d e n t o n e s.
a = I 2
3.1 1. C h e c k t o s e e i f t h e f o l l o w i n g f o u r v e c t o r s a r e l i n e a r l y i n d e p e n d e n t: / 2 \ / 1 \ / - 9 \ / - 1 \
8 4 \ 1
a —
- 1
1
b =
4 \ 1
c —
4
1
/
A p p r o x i m a t i o n U s i n g t h e H i y l o r S e r i e s
D e t e r m i n e l i n e a r a n d q u a d r a t i c a p p r o x i m a t i o n s o f t h e f o l l o w i n g f u n c t i o n s a r o u n d t h e g i v e n p o i n t s u s i n g t h e I & y l o r s e r i e s. F o r o n e - a n d t w o - v a r i a b l e p r o b l e m s, g r a p h i c a l l y c o m p a r e t h e a p p r o x i m a t i o n s w i t h t h e o r i g i n a l f u n c t i o n.
3.1 2. /( jc) = jc2 e “ x2 jc° — 1.5
3.1 3. f { x ) = x + 1/x — 5 x ° ~ 2
3.1 4. f i x ) = e - *2 c o s [ x/7 r ] - t - s i n l x 3 ] — 1 x ° = π/3
3.1 5. f i x, y ) — x 2 - j - 4 y x + y 2 — 4 ( x °, y ° ) = ( 1,1 )
3.1 6. f i x, y ) ~ δ χ 2 - 4 y x + 2 y 2 - 8 ( x °, y ° ) = ( 1,1 )
3.1 7. f i x,y ) = e ^ y'> - x + y - 5 ( x °, y ° ) = ( 1, 2 )
3.1 8. f i x, y ) = 3 + 2 x + 2 J y ( x °, y ° ) = ( 1, 2 )
3.1 9. / ( x, y, z ) — 2 y f z + x + 2 ( x °, y °, z ° ) = ( 1, 2,3 )
3.2 0. f i x, y, z ) = x 2 + y 2 + z 2 - 1 ( x °, y °, z ° ) = ( 1, 2, 3 )
3.2 1. /( X 1,X 2,X 3,* 4 ) = 1/4- 1- X 1/X 2 “ X2/X 3 + X 3A 4 - ^4/^1 ( x f, x §, x §, x j ) = ( 1, - 1,1, - 1 )
3.6 Problems
S o l u t i o n o f N o n l i n e a r E q u a t i o n s
S t a r t i n g f r o m g i v e n p o i n t s, s o l v e t h e f o l l o w i n g n o n l i n e a r e q u a t i o n s u s i n g t h e N e w t o n - R a p h s o n m e t h o d.
3.22. = 0 x ° = 1.5
3.2 3. x + 1/x = 5 x ° = 2
3.2 4. e ~ * * c o s [ x/7r ] + s i n t x 3 ] = 1 x ° = π/3
3.2 5. x2 + 4 y x + y 2 - 4 = 0 δ χ2 - 4 y x + 2 y 2 - 8 = 0 ( x °, y ° ) = ( 1,1 )
3.2 6. e ( x ~ - x + y - 5 = 0 3 + 2 x + 2/y = 0 ( x °,y ° ) = ( 1, 2 )
3.2 7. x 2 + y 2 + z 2 = 1 2x + z = 0 2y/z + x +2 = 0 ( x °, y °, z ° ) = ( 1, 2,3 )
Q u a d r a t i c F o r m s
F o r t h e f o l l o w i n g p r o b l e m s, w r i t e t h e f u n c t i o n s i n t h e i r q u a d r a t i c f o r m s a n d d e t e r m i n e t h e i r s i g n s u s i n g t h e e i g e n v a l u e t e s t. F o r f u n c t i o n s o f t w o v a r i a b l e s, d r a w c o n t o u r p l o t s o f t h e f u n c t i o n s.
3.2 8. f i x ι, * 2) = 6 x f + 3 * 1 X 2 + 2 x ^
7 7X2
3.29. f i x ι, X2) = 2 x f — X1X2 + '22,
3.3 0. f i x 1, x 2, x 3 ) = x f - X1X2 + ^ + 3x 1x 3 - X2*3 + 2x 3
3.31. /( χ ι,Χ 2, X3) = 6x f - 4 x f - 3x 1x 3 + 2 x |
x2 9 13*2 7
3.32. /( X l, X2, X3, X4) = ~2 + *2 ~~ + “2^ + 2x 1x 4 + X2*4 + 7 x ^
3.3 3. f i x 1, X2, X3, X4) = — — x | + 3X1X3 — 5^3 - 2X1X4 — X 2 * 4 + X3X4 — 15X4
F o r t h e f o l l o w i n g p r o b l e m s, w r i t e t h e f u n c t i o n s i n t h e i r q u a d r a t i c f o r m s a n d d e t e r m i n e t h e i r s i g n s u s i n g t h e p r i n c i p a l m i n o r s t e s t. F o r f u n c t i o n s o f t w o v a r i a b l e s, d r a w c o n t o u r p l o t s o f t h e f u n c t i o n s.
3.3 4. f i x 1,x 2 ) = 6 x f + 3 x i x 2 + 2 x |
F o r t h e f o l l o w i n g p r o b l e m s, d e t e r m i n e t h e c o n v e x i t y s t a t u s o f t h e g i v e n f u n c ­
t i o n s. I f p o s s i b l e, d e t e r m i n e t h e d o m a i n o v e r w h i c h a f u n c t i o n i s c o n v e x a n d p l o t t h e f u n c t i o n ( c o n t o u r p l o t f o r a f u n c t i o n o f t w o v a r i a b l e s ) o v e r t h i s r e g i o n,
3.4 0. f ( x ) = s i n O O/C *2 + 1) 0 < χ < 2 π
3.4 1. f i x ) = χ 7 + χ 5 + χ 3 + x
3.4 2. f i x ) = e ~ * 2 c o s f x/π ] -f- s i n [ x 3 ] — 1 0 < χ < 2π________________________
3.4 3. f i x, y ) = x 2 4- 4 y x + y 2 — 4
3.4 4. /( x, y ) = S x 2 — 4 y x + 2 y 2 — 8
3.4 5. f ( x, y ) = e ^ x ~ y ) — x + y — 5
3.4 6. f i x, y) = 3 + 2 x + 2/y
3.4 7. / ( x, y, z ) = 2 y/z + x + 2
3.4 8. f ( x, y, z) = x 2 + y 2 + z 2 - 1
3.4 9. /( χ ι, x 2, X3, X4) = I/4 + X1/X 2 - X2/X 3 + Χ3Λ 4 ~ X4/X 1
Convex Functions
C o n v e x O p t i m i z a t i o n P r o b l e m s
D e t e r m i n e i f t h e f o l l o w i n g p r o b l e m s a r e c o n v e x o p t i m i z a t i o n p r o b l e m s. U s e g r a p h i c a l m e t h o d s t o s o l v e t h e s e p r o b l e m s. V e r i f y t h e g l o b a l o p t i m a l i t y o f c o n v e x p r o b l e m s.
3.50. M a x i m i z e f i x 1, X2) = —6 x + 9 y
x - y > 2 S u b j e c t t o I 3 x + y > 1 2 x — 3 y > 3
3.5 1. M i n i m i z e / ( x i, X2) — x 2 + 2 y 2 S u b j e c t t o ( x + )
3.5 2. M i n i m i z e f i x 1, x 2 ) = x 2 + 2 y 2 - 2 4 x - 2 0 y
/ x + 2 y > 0 \ x + 2 y < 9 x + y < 8 \ x + r > 0 /
S u b j e c t t o
3.6 Problems
3.5 3. M a x i m i z e f i x 1, * 2) = *1 + ;| + g ·
( * + *2> 0Z )
S u b j e c t t o
)
3.5 5. M a x i m i z e f ( x 1, * 2) — ( x\ - 2 ) 2 4 - ( x 2 - 1 0 ) 2
(
x f + x f ^ 5 0 Xj - I - * 2 + 2X1X2 — Χ ι — X2 + 2 0 > 0 X l, * 2 > 0
3.5 7. M i n i m i z e / ( x, y ) = x 2 4 - y 2 S u b j e c t t o ( y — l ) 3 > x 2
3.5 8. S h o w t h a t t h e f o l l o w i n g i s a c o n v e x o p t i m i z a t i o n p r o b l e m.
M i n i m i z e f ( x,y ) = x 2 + y 2 — l o g [ x 2 ^ 2 ]
S u b j e c t t o x < l o g [ y ] x > 1 y > 1 U s e g r a p h i c a l m e t h o d s t o d e t e r m i n e i t s s o l u t i o n.
3.5 9. S h o w t h a t t h e f o l l o w i n g i s a c o n v e x o p t i m i z a t i o n p r o b l e m.
M i n i m i z e f ( x, y,z ) = x + y + z S u b j e c t t o x - 2 + x ~ 2 y ~ 2 4 - x ~ 2 y ~ 2 z ~ 2 < 1
3.5 6. M i n i m i z e /( x, y ) =
S u b j e c t t o
S u b j e c t t o
)
Optimality Conditions
c h a p t e r f o u r
T h i s c h a p t e r d e a l s w i t h m a t h e m a t i c a l c o n d i t i o n s t h a t m u s t b e s a t i s f i e d b y t h e s o l u t i o n o f a n o p t i m i z a t i o n p r o b l e m. A t h o r o u g h u n d e r s t a n d i n g o f t h e c o n c e p t s p r e s e n t e d i n t h i s c h a p t e r i s a p r e r e q u i s i t e f o r u n d e r s t a n d i n g t h e m a ­
t e r i a l p r e s e n t e d i n l a t e r c h a p t e r s. T h e s u b j e c t m a t t e r o f t h i s c h a p t e r i s q u i t e t h e o r e t i c a l a n d r e q u i r e s u s e o f s o p h i s t i c a t e d m a t h e m a t i c a l t o o l s f o r r i g o r o u s t r e a t m e n t. S i n c e t h e b o o k i s n o t i n t e n d e d t o b e a t h e o r e t i c a l t e x t o n m a t h e ­
m a t i c a l o p t i m i z a t i o n, t h i s c h a p t e r i s k e p t s i m p l e b y a p p e a l i n g t o i n t u i t i o n a n d a v o i d i n g p r e c i s e m a t h e m a t i c a l s t a t e m e n t s. T h u s, i t i s i m p l i c i t i n t h e p r e s e n t a ­
t i o n t h a t a l l f u n c t i o n s a r e w e l l b e h a v e d a n d h a v e t h e n e c e s s a r y c o n t i n u i t y a n d d i f f e r e n t i a b i l i t y p r o p e r t i e s. T h e b o o k b y P e r e s s i n i, S u l l i v a n a n d U h l, J r. [ 1 9 8 8 ] i s r e c o m m e n d e d f o r t h o s e i n t e r e s t e d i n a m a t h e m a t i c a l l y r i g o r o u s, y e t v e r y r e a d a b l e, t r e a t m e n t o f m o s t o f t h e c o n c e p t s p r e s e n t e d i n t h i s c h a p t e r. M o r e d e t a i l s c a n a l s o b e f o u n d i n B e v e r i d g e, G o r d o n, a n d S c h e c h t e r [ 1 9 7 0 ], J a h n [ 1 9 9 6 ], J e t e r [ 1 9 8 6 ], M c A l o o n a n d T r e t k o f f [ 1 9 9 6 ], P i e r r e a n d L o w e [ 1 9 7 5 ], a n d S h o r [ 1 9 8 5 ].
T h e f i r s t s e c t i o n p r e s e n t s n e c e s s a r y a n d s u f f i c i e n t c o n d i t i o n s f o r o p t i m u m o f u n c o n s t r a i n e d p r o b l e m s. S e v e r a l e x a m p l e s a r e p r e s e n t e d t h a t u s e t h e s e c o n d i t i o n s t o d e t e r m i n e s o l u t i o n s o f u n c o n s t r a i n e d p r o b l e m s. T h e r e m a i n ­
i n g s e c t i o n s i n t h e c h a p t e r c o n s i d e r c o n s t r a i n e d o p t i m i z a t i o n p r o b l e m s. S e c ­
t i o n 2 e x p l a i n s t h e a d d i t i v e p r o p e r t y o f t h e l e s s t h a n t y p e i n e q u a l i t y ( < ) a n d e q u a l i t y ( = ) c o n s t r a i n t s t h a t i s u s e f u l i n d e v e l o p i n g o p t i m a l i t y c o n d i t i o n s f o r c o n s t r a i n e d p r o b l e m s. T h e f i r s t - o r d e r n e c e s s a r y c o n d i t i o n s f o r c o n s t r a i n e d p r o b l e m s, c o m m o n l y k n o w n a s K a r u s h - K u h n - T U c k e r ( K T ) c o n d i t i o n s, a r e p r e ­
s e n t e d i n s e c t i o n 3. U s i n g t h e s e c o n d i t i o n s, a p r o c e d u r e i s d e v e l o p e d t o f i n d
Chap ter 4 Op t im al it y Conditions
c a n d i d a t e m i n i m u m p o i n t s ( K T p o i n t s ) f o r c o n s t r a i n e d p r o b l e m s. K T c o n d i ­
t i o n s h a v e a s i m p l e g e o m e t r i c i n t e r p r e t a t i o n t h a t i s d i s c u s s e d i n s e c t i o n 4. T h e s o - c a l l e d L a g r a n g e m u l t i p l i e r s o b t a i n e d d u r i n g a s o l u t i o n u s i n g K T c o n d i t i o n s a r e u s e f u l i n e s t i m a t i n g t h e e f f e c t o f c e r t a i n c h a n g e s i n c o n s t r a i n t s o n t h e o p t i m u m s o l u t i o n. T h i s i s k n o w n a s s e n s i t i v i t y a n a l y s i s a n d i s d i s c u s s e d i n s e c t i o n 5. S e c t i o n 6 s h o w s t h a t f o r c o n v e x p r o b l e m s t h e K T c o n d i t i o n s a r e n e c ­
e s s a r y a n d s u f f i c i e n t f o r t h e o p t i m u m. S e c o n d - o r d e r s u f f i c i e n t c o n d i t i o n s f o r a g e n e r a l c o n s t r a i n e d o p t i m i z a t i o n p r o b l e m a r e d i s c u s s e d i n s e c t i o n 7. T h e l a s t s e c t i o n b r i e f l y c o n s i d e r s t h e t o p i c o f d u a l i t y i n o p t i m i z a t i o n. T h e c o n c e p t o f d u a l i t y i s u s e f u l i n d e v e l o p i n g e f f e c t i v e c o m p u t a t i o n a l a l g o r i t h m s f o r s o l v i n g l i n e a r a n d q u a d r a t i c p r o g r a m m i n g p r o b l e m s d i s c u s s e d i n l a t e r c h a p t e r s.
4.1 O p t i m a l i t y C o n d i t i o n s f o r
T h e u n c o n s t r a i n e d o p t i m i z a t i o n p r o b l e m c o n s i d e r e d i n t h i s s e c t i o n i s s t a t e d a s f o l l o w s:
F i n d a v e c t o r o f o p t i m i z a t i o n v a r i a b l e s x = ( x i x 2 ... X n ) T i n o r d e r t o M i n i m i z e f ( x )
4.1.1 N e c e s s a r y C o n d i t i o n f o r O p t i m a l i t y o f U n c o n s t r a i n e d P r o b l e m s
T h e f i r s t - o r d e r o p t i m a l i t y c o n d i t i o n f o r t h e m i n i m u m o f / ( x ) c a n b e d e r i v e d b y c o n s i d e r i n g l i n e a r e x p a n s i o n o f t h e f u n c t i o n a r o u n d t h e o p t i m u m p o i n t x * u s i n g t h e T U y l o r s e r i e s, a s f o l l o w s.
f ( % ) % /( * * ) + V/( X * ) T ( X - X * )
o r
/( x ) — / ( x * ) = V/( x * ) r d
w h e r e V/ ( x * ) i s a g r a d i e n t o f f u n c t i o n / ( x ) a n d d ξ x — x *.
4.1 Op t im al it y Condition» f o r Unconstrained Problems
I f x * i s a m i n i m u m p o i n t, t h e n f ( x ) — f ( x * ) m u s t b e g r e a t e r t h a n o r e q u a l t o z e r o i n a s m a l l n e i g h b o r h o o d o f x *. B u t s i n c e d c a n e i t h e r b e n e g a t i v e o r p o s i t i v e, t h e o n l y w a y t o e n s u r e t h a t t h i s c o n d i t i o n i s a l w a y s s a t i s f i e d i s i f
V f ( x * ) = 0
T h u s, t h e f i r s t - o r d e r n e c e s s a r y c o n d i t i o n f o r t h e m i n i m u m o f a f u n c t i o n i s t h a t i t s g r a d i e n t i s z e r o a t t h e o p t i m u m. N o t e t h a t a l l t h i s c o n d i t i o n t e l l s u s i s t h a t t h e f u n c t i o n v a l u e i n t h e s m a l l n e i g h b o r h o o d o f x * i s n o t i n c r e a s i n g. T h i s c o n d i t i o n i s t r u e e v e n f o r a m a x i m u m p o i n t a n d a t a n y o t h e r p o i n t w h e r e l o c a l l y t h e s l o p e i s z e r o ( i n f l e c t i o n p o i n t s ). T h e r e f o r e, t h i s i s o n l y a n e c e s s a r y c o n d i t i o n a n d i s n o t a s u f f i c i e n t c o n d i t i o n f o r t h e m i n i m u m o f f. A l l p o i n t s t h a t s a t i s f y t h i s c o n d i t i o n a r e k n o w n a s s t a t i o n a r y p o i n t s.
A p l o t o f a f u n c t i o n o f a s i n g l e v a r i a b l e o v e r t h e i n t e r v a l { — 2 π, 2 π ], s h o w n i n F i g u r e 4.1, i l l u s t r a t e s t h a t i t h a s f o u r p o i n t s w h e r e i t s f i r s t d e r i v a t i v e i s z e r o. A s s h o w n o n t h e p l o t, t w o a r e l o c a l m i n i m a, o n e i s a l o c a l m a x i m u m, a n d t h e f o u r t h i s s i m p l y a n i n f l e c t i o n p o i n t.
F I G U R E 4.1 G r a p h o f a f u n c t i o n o f s i n g l e v a r i a b l e / = x ( —C o s [ l ] — S i n [ l ] + S i n [ x ] ) a n d i t s f i r s t d e r i v a t i v e.
4.1.2 S u f f i c i e n t C o n d i t i o n f o r O p t i m a l i t y o f U n c o n s t r a i n e d P r o b l e m s
T h e s e c o n d - o r d e r o p t i m a l i t y c o n d i t i o n f o r t h e m i n i m u m o f / ( x ) c a n b e d e r i v e d b y c o n s i d e r i n g q u a d r a t i c e x p a n s i o n o f t h e f u n c t i o n a r o u n d t h e o p t i m u m p o i n t
C hap ter 4 Op t im al it y Conditions
x * u s i n g t h e T U y l o r s e r i e s, a s f o l l o w s:
/ ( X ) - / ( x * ) = V f ( x * ) T d + i d T V 2 f ( x * ) d
w h e r e V 2 f ( x * ) i s a H e s s i a n m a t r i x o f f u n c t i o n /( x ) a n d d = x — x *.
F o r x* t o b e a l o c a l m i n i m u m p o i n t, / (x) — / (x*) m u s t b e g r e a t e r t h a n o r e q u a l t o z e r o i n a s m a l l n e i g h b o r h o o d o f x*. F r o m t h e f i r s t - o r d e r n e c e s s a r y c o n d i t i o n V f (x*) = 0 a n d t h e r e f o r e f o r / (x) - / (x*) > 0, w e m u s t h a v e
T h i s i s a q u a d r a t i c f o r m, d i s c u s s e d i n C h a p t e r 3, a n d t h e r e f o r e t h e s i g n o f t h e p r o d u c t d e p e n d s o n t h e s t a t u s o f t h e H e s s i a n m a t r i x V 2/( x * ). I f t h e H e s ­
s i a n i s p o s i t i v e d e f i n i t e, t h e n | d r V 2/( x * ) d > 0 f o r a n y v a l u e s o f d, m a k i n g x * c l e a r l y t h e m i n i m u m p o i n t. I f t h e H e s s i a n i s p o s i t i v e s e m i d e f i n i t e, t h e n j d T V 2/( x * ) d > 0 a n d t h e r e f o r e, a l l w e c a n s a y i s t h a t t h e f u n c t i o n i s n o t i n ­
c r e a s i n g i n t h e s m a l l n e i g h b o r h o o d o f x *, m a k i n g i t j u s t a n e c e s s a r y c o n d i t i o n f o r t h e m i n i m u m.
T h u s, t h e s e c o n d - o r d e r s u f f i c i e n t c o n d i t i o n f o r t h e m i n i m u m o f a f u n c t i o n i s t h a t i t s H e s s i a n m a t r i x i s p o s i t i v e d e f i n i t e a t t h e o p t i m u m.
P r o c e e d i n g i n a s i m i l a r m a n n e r, i t c a n e a s i l y b e s h o w n t h a t a s u f f i c i e n t c o n d i t i o n f o r t h e m a x i m u m o f a f u n c t i o n i s t h a t i t s H e s s i a n m a t r i x i s n e g a t i v e d e f i n i t e a t t h e o p t i m u m.
4.1.3 F i n d i n g t h e O p t i m u m U s i n g O p t i m a l i t y C o n d i t i o n s
S i n c e a l l m i n i m u m p o i n t s m u s t s a t i s f y t h e f i r s t - o r d e r n e c e s s a r y c o n d i t i o n s, w e c a n f i n d t h e s e p o i n t s b y s o l v i n g t h e f o l l o w i n g s y s t e m o f e q u a t i o n s r e s u l t i n g f r o m s e t t i n g t h e g r a d i e n t o f t h e f u n c t i o n t o z e r o:
- d r V 2/( x * ) d > 0 2
T h i s s y s t e m o f e q u a t i o n s i s g e n e r a l l y n o n l i n e a r a n d h e n c e, m a y h a v e s e v e r a l p o s s i b l e s o l u t i o n s. T h e s e s o l u t i o n s a r e k n o w n a s s t a t i o n a r y p o i n t s.
A f t e r c o m p u t i n g a l l s t a t i o n a r y p o i n t s, t h e s e c o n d - o r d e r s u f f i c i e n t c o n d i t i o n s a r e u s e d t o f u r t h e r c l a s s i f y t h e s e p o i n t s i n t o a l o c a l m i n i m u m, m a x i m u m, o r s i m p l y a n i n f l e c t i o n p o i n t a s f o l l o w s:
1. A s t a t i o n a r y p o i n t a t w h i c h t h e H e s s i a n m a t r i x i s p o s i t i v e d e f i n i t e i s a t l e a s t a l o c a l m i n i m u m.
2. A s t a t i o n a r y p o i n t a t w h i c h t h e H e s s i a n m a t r i x i s n e g a t i v e d e f i n i t e i s a t l e a s t a l o c a l m a x i m u m.
3. A s t a t i o n a r y p o i n t a t w h i c h t h e H e s s i a n m a t r i x i s i n d e f i n i t e i s a n i n f l e c t i o n p o i n t.
4. I f t h e H e s s i a n m a t r i x i s s e m i d e f i n i t e ( e i t h e r p o s i t i v e o r n e g a t i v e ) a t a s t a ­
t i o n a r y p o i n t, t h e n e v e n t h e s e c o n d - o r d e r c o n d i t i o n s a r e i n c o n c l u s i v e. O n e m u s t d e r i v e e v e n h i g h e r - o r d e r c o n d i t i o n s o r u s e p h y s i c a l a r g u m e n t s t o d e c i d e t h e o p t i m a l i t y o f s u c h a p o i n t.
E x a m p l e 4.1 F i n d a l l s t a t i o n a r y p o i n t s f o r t h e f o l l o w i n g f u n c t i o n. U s i n g s e c o n d - o r d e r o p t i m a l i t y c o n d i t i o n s, c l a s s i f y t h e m a s m i n i m u m, m a x i m u m, o r i n f l e c t i o n p o i n t s. V e r i f y t h e s o l u t i o n u s i n g g r a p h i c a l m e t h o d s.
f = x 2 - x y +· l y 2 - 2x;
O b j e c t i v e f u n c t i o n - 2 x + x 2 - x y + 2 y 2
P o s s i b l e s o l u t i o n s ( s t a t i o n a r y p o i n t s ) -» ( x -» 1.1 4 2 8 6 y - > 0.2 8 5 7 1 4 )
G r a d i e n t v e c t o r -»
* * * * * *
F i r s t o r d e r o p t i m a l i t y c o n d i t i o n s * * * * * *
* * * * * *
S e c o n d o r d e r o p t i m a l i t y c o n d i t i o n s * * * * * * P o i n t - » { x - > 1.1 4 2 8 6, y -» 0.2 8 5 7 1 4 }
S t a t u s M i n i m u m P o i n t F u n c t i o n v a l u e -» - 1.1 4 2 8 6
Chapter 4 o p t i m a l i t y r e n d i t i o n s
S i n c e t h e H e s s i a n m a t r i x i s p o s i t i v e d e f i n i t e, w e k n o w t h a t t h e f u n c t i o n i s c o n v e x a n d h e n c e, a n y m i n i m u m i s a g l o b a l m i n i m u m. T h e c o m p u t a t i o n s c o n ­
f i r m t h i s f a c t b e c a u s e t h e r e i s o n l y o n e p o s s i b l e s o l u t i o n a n d h e n c e, o b v i o u s l y i s a g l o b a l m i n i m u m.
T h e p l o t s s h o w n i n F i g u r e 4.2 c o n f i r m t h i s s o l u t i o n a s w e l l. T h e G r a p h i ­
c a l S o l u t i o n f u n c t i o n d e s c r i b e d i n C h a p t e r 2 i s u s e d f i r s t t o g e n e r a t e s e v e r a l c o n t o u r s o f t h e o b j e c t i v e f u n c t i o n. T h e b u i l t - i n M a t h e m a t i c a f u n c t i o n P l o t 3 D i s t h e n u s e d t o g e n e r a t e a t h r e e - d i m e n s i o n a l s u r f a c e p l o t o f t h e f u n c t i o n. F i n a l l y, t h e t w o p l o t s a r e s h o w n s i d e - b v - s i d e u s i n g t h e G r a p h i c s A r r a y f u n c t i o n.
g r l = G r a p h i c a l S o l u t i o n!f, { χ, 0, 2 }, { y, 0, 1 },
P l o t P o i n t s 2 0, O b j e c t i v e C o n t o u r s - » { - l, - .5,0, .5, 1 },
E p i l o g -*■ {RGBCol or [ 1, 0, 0] , P o i n t S i z e [.0 2 ],
P o i n t [ { ®, j } ], T e x t [ "Mi n i mu m", { 1.1 7, ^ >, { - 1, 0}] }] ;
g r 2 = S h o w [{
G r a p h i c s 3 D [ S u r f a c e G r a p h i c a [ P l o t 3 D [ f, { x, 0, 2), { y, 0, 1},
D i s p l a y F u n c t i o n -+ I d e n t i t y ] ] ] ,
G r a p h i c s 3 D [ { R G B C o l o r [ 1, 0,0 ], P o i n t S i z e [ .0 2 ], P o i n t [ { 1.1 7, j, - 1.1 } ] } ]
}, D i s p l a y F u n c t i o n- * $ D i s p l a y F u n c t i o n ];
S h o w [ G r a p h i c s A r r a y [ { { g r l, g r 2 } } ] ];
y
0 0.5— i rs 2
F I G U R E 4.2 G r a p h i c a l s o l u t i o n o f f = x 2 — x y + l y 2 — 2 x.
I m p o r t a n t O b s e r v a t i o n s
T h e f o l l o w i n g o b s e r v a t i o n s s h o u l d b e n o t e d f r o m t h e o p t i m a l i t y c o n d i t i o n s.
1. T h e m i n i m u m p o i n t x * d o e s n o t c h a n g e i f w e a d d a c o n s t a n t t o t h e o b j e c t i v e f u n c t i o n. T h e r e a s o n i s t h a t a c o n s t a n t t e r m d o e s n o t c h a n g e t h e g r a d i e n t
4.1 Op t im al it y Conditions f o r Unconstrained Problems
a n d h e n c e, t h e s y s t e m o f e q u a t i o n s b a s e d o n t h e n e c e s s a i y c o n d i t i o n s r e m a i n s u n c h a n g e d.
2. B y t h e s a m e r e a s o n i n g a s a b o v e, t h e m i n i m u m p o i n t x * d o e s n o t c h a n g e i f w e m u l t i p l y t h e o b j e c t i v e f u n c t i o n b y a p o s i t i v e c o n s t a n t.
3. T h e p r o b l e m c h a n g e s f r o m a m i n i m i z a t i o n p r o b l e m t o a m a x i m i z a t i o n p r o b ­
l e m, a n d v i c e v e r s a, i f w e m u l t i p l y t h e o b j e c t i v e f u n c t i o n b y a n e g a t i v e s i g n. T h i s i s b e c a u s e i f x * i s a m i n i m u m p o i n t o f /( x ), t h e n i t s H e s s i a n V 2/( x * ) i s p o s i t i v e d e f i n i t e. T h e H e s s i a n o f — /( x ) w i l l b e — V 2/( x ), w h i c h c l e a r l y w i l l b e n e g a t i v e d e f i n i t e, m a k i n g x * a m a x i m u m p o i n t o f — f i x ). T h u s,
M i n/( x ) · < = = » M a x — /( x )
4. S i n c e t h e r e a r e n o c o n s t r a i n t s, t h e u n c o n s t r a i n e d p r o b l e m i s a c o n v e x p r o ­
g r a m m i n g p r o b l e m i f t h e o b j e c t i v e f u n c t i o n i s c o n v e x. R e c a l l t h a t f o r a c o n v e x c a s e, a n y l o c a l m i n i m u m i s a l s o a g l o b a l m i n i m u m.
T h e f o l l o w i n g f u n c t i o n, i n c l u d e d i n t h e O p t i m i z a t i o n l b o l b o x O p t i m a l i t y C o n - d i t i o n s' p a c k a g e, i m p l e m e n t s a s o l u t i o n p r o c e d u r e b a s e d o n o p t i m a l i t y c o n d i ­
t i o n s f o r u n c o n s t r a i n e d p r o b l e m s. F r o m t h e n e c e s s a r y c o n d i t i o n s, t h e f u n c t i o n g e n e r a t e s a s y s t e m o f e q u a t i o n s. T h e s e e q u a t i o n s a r e t h e n s o l v e d u s i n g t h e b u i l t - i n M a t h e m a t i c a f u n c t i o n s N S o l v e ( d e f a u l t ) o r F i n d R o o t. N S o l v e t r i e s t o f i n d a l l p o s s i b l e s o l u t i o n s b u t c a n n o t h a n d l e c e r t a i n t y p e s o f e q u a t i o n s, p a r t i c ­
u l a r l y t h o s e i n v o l v i n g t r i g n o m e t r i c f u n c t i o n s. F i n d R o o t g i v e s o n l y o n e s o l u t i o n t h a t i s c l o s e s t t o t h e g i v e n s t a r t i n g p o i n t. S e e t h e s t a n d a r d M a t h e m a t i c a d o c u ­
m e n t a t i o n t o f i n d m o r e d e t a i l s o n t h e s e t w o f u n c t i o n s.
N e e d s ["O p t i m i z a t i o n T o o l b o x'O p t i m a l i t y C o n d i t i o n s'"];
?U n c o n s t r a i n e d Q p t i m a l i t y
U n c o n s t r a i n e d O p t i m a l i t y [ f, v a r s, o p t s ], f i n d s a l l s t a t i o n a r y p o i n t s o f f u n c t i o n f o f v a r s. U s i n g s e c o n d o r d e r s u f f i c i e n t c o n d i t i o n s i t t h e n c l a s s i f i e s t h e m i n t o mi ni mum, maximum o r i n f l e c t i o n p o i n t s. O p t i o n s c a n b e u s e d t o s p e c i f y a M a t h e m a t i c a f u n c t i o n t o b e u s e d f o r s o l v i n g s y s t e m o f e q u a t i o n s. I f t h i s f u n c t i o n i t s e l f h a s o p t i o n s, t h e y c a n a l s o b e s p e c i f i e d.
O p t i o n s U s a g e [ U n c o n s t r a i n e d O p t i m a l i t y ]
{ S o l v e E q u a t i o n s U s i n g N S o l v e, S t a r t i n g S o l u t i o n - *{}}
S o l v e E q u a t i o n s U s i n g -* M a t h e m a t i c a f u n c t i o n u s e d t o s o l v e s y s t e m o f
o n t o t h a t m e t h o d.
C h ap te r 4
S t a r t i n g S o l u t i o n u s e d w i t h t h e F i n d R o o t f u n c t i o n i n M a t h e m a t i c a.. I t i s u s e d o n l y i f t h e m e t h o d s p e c i f i e d i s F i n d R o o t.
E x a m p l e 4.2 T h e f o l l o w i n g f u n c t i o n o f t w o v a r i a b l e s h a s t w o d i s t i n c t l o c a l m i n i m u m p o i n t s a n d a n i n f l e c t i o n p o i n t:
f = ( x - 2) 2 + ( x - i y 2 ) 2 }
s o l n a u n c o x i f i t r a i n e d O p t i m a l i t y [ £, { x, y } ];
O b j e c t i v e f u n c t i o n s ( - 2 + x ) 2 + ( x - 2 y 2 ) 2
/- 4 + 4 x - 4 Y 2 \
G r a d i e n t v e c t o r s | _ 8 x y + 1 6 y 3 j
[ 4 - 8 y \
H e s s i a n m a t r i x ^ ^ _ g y _ 8 x + 4 3 y 2 j
* * * * * * F i r s t o r d e r o p t i m a l i t y c o n d i t i o n s * * * * * *
- 4 + 4 x - 4 y 2 = = 0 ^
- 8 x y + 1 6 y 3 = = 0
N e c e s s a r y c o n d i t i o n s - » | 0___ * 3 _ _
P o R R i h l f t R n l n t -.i n n s ( s t a t i o n a r y p o i n t s?) - >
x - » 1 y - » 0 1 x -4 2 y -» - 1
tx - » 2 y - » l j
* * * * * * s econ(3 o r d e r o p t i m a l i t y c o n d i t i o n s * * * * * * ----------------- P o i n t -» { x ^ y -» 0 }
( 4 0 \ / 4
H e s s i a n - » I P r i n c i p a l m i n o r s - » I ^
S t a t u s - » I n f l e c t i o n P o i n t F u n c t i o n v a l u e -» 2 ----------------- P o i n t -» { x -* 2, y -* - 1 }
1 4 8 \ . I 4
H e s s i a n - » K j P r i n c i p a l m i n o r s - » I
S t a t u s -* M i n i m u m P o i n t F u n c t i o n v a l u e -» 0 -------------------- P o i n t -» { x -» 2, y 1 }
H e s s i a n - » | ^ P r i n c i p a l m i n o r s -»
S t a t u s -> M i n i m u m P o i n t F u n c t i o n v a l u e -» 0
T h e f o l l o w i n g p l o t s, g e n e r a t e d u s i n g t h e G r a p h i c a l S o l u t i o n a n d P l o t 3 D f u n c ­
t i o n s a s e x p l a i n e d i n a n e a r l i e r e x a m p l e, c o n f i r m t h e s e s o l u t i o n s. I n p a r t i c u l a r, n o t e t h e i n f l e c t i o n p o i n t, w h i c h i s j u s t a f l a t s p o t b e t w e e n t h e t w o m i n i m u m s.
4.1 Op t im al it y Conditions f o r Unconstrained Problems
- 2 - 1 0 1 2 3 4 5
F I G U R E 4.3 G r a p h i c a l s o l u t i o n o f f = ( x — 2 ) 2 + ( x — 2 y 2 ) 2.
E x a m p l e 4.3 T h e f o l l o w i n g f u n c t i o n o f t w o v a r i a b l e s h a s o n l y o n e m i n i m u m p o i n t b u t t w o d i s t i n c t l o c a l m a x i m u m p o i n t s a n d t w o i n f l e c t i o n p o i n t s.
f = 2 5 x 2 - 1 2 x 4 - 6x y + 2 5 y 2 - 2 4 χ 2 ^ - 1 2 y 4;
s o l n = U H c o n s t r a i n e d O p t i m & l i t y [ f, { x, y > ];
O b j e c t i v e f u n c t i o n 2 5 x 2 - 1 2 x ^ - 6 x y + 2 5 y 2 - 2 4 x 2 y 2
_ ,, / 5 0 x - 4 8 x 3 - 6 y - 4 8 x 1 ^ \
G r a d i e n t _ 6 x , 5 0 y . 4 β χ 2 γ . 4 S y 3 j
50 - 1 4 4 x 2 - 4 8 y 2 - 6 - 9 6 x y \
- 6 - 9 6 x y 50 - 4 8 x 2 - 1 4 4 ^ ]
I 2 y
H e s s i a n m a t r i x
****** F i r s t o r d e r o p t i m a l i t y c o n d i t i o n s *** * * *
5 Ox - 4 8 x 3 - 6y - 4 8 x y 2 == 0 ~ 6 x + 5 0 y - 4 8 x 2y - 4 8 y 3 =- 0
N e c e s s a r y c o n d i t i o n s
P o s s i b l e s o l u t i o n s ( s t a t i o n a r y p o i n t s ) -»
x- > - 0.7 6 3 7 6 3 x -» - 0. 677003 x 0. x- > 0.6 7 7 0 0 3 x - » 0.7 6 3 7 6 3
y - » 0.7 6 3 7 6 3 y - » - 0.6 7 7 0 0 3 y -» 0. y - » 0.6 7 7 0 0 3 y -» - 0 .7 6 3 7 6 3/
H e s s i a n ■
****** s e c o n d o r d e r o p t i m a l i t y c o n d i t i o n s *** * * * ----------------- P o i n t - » { x - » - 0.7 6 3 7 6 3, y - » 0.7 6 3 7 6 3 }
( - 6 2. 5 0. \ . I - 6 2.
j·_ „ P r i n c i p a l m i n o r s - » _ _ ..
5 0. - 6 2./ ^ 1 3 4 4.
S t a t u s -» M a x i mu mP o i n t F u n c t i o n v a l u e 1 6.3 3 3 3 ----------------- P o i n t -» {x-> - 0.6 7 7 0 0 3, y - » - 0.6 7 7 0 0 3 }
He<==.· / - 3 8. - 5 0.
H e s s i a n ■
P r i n c i p a l m i n o r s
/ - 3 8. \- 1 0 5 6.
- 5 0.
-38.
C h ap te r 4 Op t im al it y Conditions
H e s s i a n
H e s s i a n -»
S t a t u s -> I n f l e c t i o n P o i n t F u n c t i o n v a l u e -» 1 0. 0833 ----------------- P o i n t - » { x - * 0.,y - » 0.}
50 - - 6.\ _ . . . . β I 5 0.
- 6. 5 0.) « “ ° * · - ( 3 4 M.
S t a t u s -» M i n i m u m P o i n t F u n c t i o n v a l u e -» 0 .
----------------- p o i n t -» {x-» 0. 6 7 7 0 0 3, y 0 . 677003}
- 3 8. - 5 0.\ . . η . / - 3 8.
- 5 0. - 3 8.) P r i n c i p a l m n o r e - * | _ 1 0 5 6 >
S t a t u s -* I n f l e c t i o n P o i n t F u n c t i o n v a l u e -» 1 0 .0 8 3 3 ----------------- P o i n t -* { x -» 0.7 6 3 7 6 3, y -» - 0.7 6 3 7 6 3 }
6 2. 5 0. \ , , ,______ ^ I - 6 2.
H e s s i a n ■
P r i n c i p a l m i n o r s
5 0. - 6 2./ —' \1 3 4 4.
S t a t u s -* M a x i m u m P o i n t F u n c t i o n v a l u e -» 1 6 .3 3 3 3
T h e f o l l o w i n g g r a p h c o n f i r m s t h e s e s o l u t i o n s.
y
F I G U R E 4.4 G r a p h i c a l s o l u t i o n o f / = 2 5 X2 - 1 2 x 4 - 6x y + 2 5 y 2 - 2 4 - 1 2 y 4.
E x a m p l e 4.4 T h e f o l l o w i n g f u n c t i o n o f f o u r v a r i a b l e s s h o w s t h r e e d i f f e r ­
e n t l o c a l m i n i m u m p o i n t s. I f t h i s f u n c t i o n w a s f r o m a p r a c t i c a l o p t i m i z a t i o n p r o b l e m, f i n d i n g a g l o b a l m i n i m u m w o u l d b e i m p o r t a n t.
4 2 2 2 4 22
E = X l + X l (1 - 2x 2 ) - 2x 2 + 2X2 - 2x 3 + X3 + X3 + 2»! (-2 + x 4 ) - 4 x d - 2X3X4 + 2x 4; s o l n = T J n c o n s t r a i n e d O p t i m a l i t y [ f , { x ^, x 2, x 3, x 4 } ];
O b j e c t i v e f u n c t i o n -> x f + x f ( 1 - 2 x 2 ) - 2 x 2 + 2 x 2 - 2 x 3 + X 3 + x f + 2 x j ( - 2 + x 4 ) - 4 x 4 - 2x 3X4 + 2 x 4
G r a d i e n t v e c t o r -»
-4 + 2 χ ^ + 4 x i - 4 x i x 2 + 2x 4' - 2 - 2 x i + 4 x 2 - 2 + 2 x 3 + 4 x | - 4X3X4 - 4 + 2 x i - 2x 3 + 4x 4
4.1 O p t im al it y Conditions f o r Unconstrained Problems
H e s s i a n m a t r i x -
1 2 x i ~ 4 x 2
- 4 Xl
0
2
- 4 x i
4
0
0
0
0
2 + I 2 x 3 - 4 x 4
- 4 x 3
2
0
- 4 x 3
4
****** F i r s t o r d e r o p t i m a l i t y c o n d i t i o n s ******
N e c e s s a r y c o n d i t i o n s
- 4 + 2 x 1 + 4 x i - 4 x 1 x 2 + 2 x 4 == 0 - 2 - 2 x | + 4 x 2 = = 0 - 2 + 2 x 3 + 4 x | - 4 x 3x 4 = - 0 - 4 + 2 χ χ - 2 x § + 4 x 4 == 0
p o s s i b l e s o l u t i o n s ( s t a t i o n a r y p o i n t s ) -»
x 2 -> 0.5 0 9 3 5 2 x 4 -> 2.0 0 2 5 6 x 3 -» 1.3 6 6 8 8 -> - 0.1 3 6 7 6 1 Ϊ
x 2 -» 0.8 9 6 0 5 3 x 4 -> 2.7 0 4 9 8 x 3 -» 1.5 8 7 4 4 χ λ - 0.8 9 0 0 0 3 x 2 -> 1. x 4 -> 1. x 3 -> 1. X]^ - » 1.
* * * * * * S e c o n d o r d e r o p t i m a l i t y c o n d i t i o n s * * * * * *
--------------------P o i n t -> { x g -» 0.5 0 9 3 5 2, x 4 -> 2.0 0 2 5 6, x 3 -> 1.3 6 6 8 8, X! -» - 0.1 3 6 7 6 l }
H e s s i a n -»
0.1 8 7 0 3 5
0.5 4 7 0 4 3
0
2
0.5 4 7 0 4 3
4
0
0
0
0
1 6.4 1
- 5.4 6 7 5 1
2
0
- 5.4 6 7 5 1
4
p r i n c i p a l m i n o r s
0.1 8 7 0 3 5'
0.4 4 8 8 8 4 7.3 6 6 2 - 2 4 6.5 1 4,
S t a t u s -> I n f l e c t i o n P o i n t F u n c t i o n v a l u e - » - 5 . 3 4 7 9 0 -----------------P o i n t -» { x 2 -> 0.8 9 6 0 5 3, x 4 -» 2 .7 0 4 9 8, x 3 -» 1.5 8 7 4 4, x ^
7.9 2 1 0 6 3.5 6 0 0 1 0 2
3.56001400
- 0.8 9 0 0 0 3 }
H e s s i a n -»
0
0
2 1.4 1 9 5
- 6.3 4 9 7 4
- 6.3 4 9 7 4
4
P r i n c i p a l m i n o r s -*
7.9 2 1 0 6'
1 9.0 1 0 6 4 0 7.1 9 6 5 1 9.5 8 3,
S t a t u s -» M i n i m u m P o i n t F u n c t i o n v a l u e -* - 5.5 6 4 8 4
----------------- P o i n t -» | x 2
-> 1.,
χ 4 -* l
( 1 0. - 4.
0
2
H e s s i a n ->
- 4. 4 0 0
0
1 0.
0
- 4.
2 0
- 4.
4
’ 1 0.
P r i n c i p a l
m i n o r s ->
2 4.
2 4 0.
.4 1 6.
Ch ap te r 4 Op t im al it y Conditions
S t a t u s -»M i n i m u m P o i n t F u n c t i o n v a l u e - » - 6.
E x a m p l e 4.5 D a t a f i t t i n g C o n s i d e r t h e d a t a - f i t t i n g p r o b l e m d i s c u s s e d i n C h a p t e r 1. T h e g o a l i s t o f i n d a s u r f a c e o f t h e f o r m ^ c o m p u t e d = ^ i * 2 + c 2 y 2 + c 3* y t o b e s t a p p r o x i m a t e t h e d a t a i n t h e f o l l o w i n g t a b l e:
P o i n t
X
y
^ o b s e r v e d
1
0
1
1.26
2
0.2 5
1
2.1 9
3
0.5
1
0.7 6
4
0.7 5
1
1.26
5
1
2
1.8 6
6
1.2 5,
2
1.4 3
7
1.5
2
1.29
8
1.7 5
I
0.6 5
9
2
2
1.6
T h e b e s t v a l u e s o f c o e f f i c i e n t s c\, C2, a n d a r e d e t e r m i n e d t o m i n i m i z e t h e s u m o f s q u a r e s o f e r r o r b e t w e e n t h e c o m p u t e d z v a l u e s a n d t h e o b s e r v e d v a l u e s.
M i n i m i z e f = Σ ϊ =1 O b s e r v e d (■*■«» V t ) ~ ^ c o m p u t e d C * i > Υ ί ) ΐ ί ^
U s i n g t h e g i v e n n u m e r i c a l d a t a, t h e o b j e c t i v e f u n c t i o n c a n b e w r i t t e n a s f o l ­
l o w s:
x y D a t a = {
{ 0,1 },{ 0.2 5,1 }/{ 0.5,1 },{ 0.7 5,1 },{ 1,2 ),
{ 1.2 5,2 }, { 1.5,2 },{ 1.7 5,2 ),{ 2,2 } }; z o = { 1.2 6, 2.1 9, .7 6, 1.2 6, 1.8 6, 1.4 3, 1.2 9, .6 5, 1.6 }; z c r M a p t i C i X 2 + C2 y 2 + c 3 x y )/. { x -» # [ [ 1 ] ], y -» # [ [ 2 ] ] } &, a ^ y D a t a ] ;
£ = E x p a n d [ A p p l y [ P l u s, ( z o - z c ) 2 ] ]
1 8.7 - 3 2.8 4 6 2 c! + 3 4.2 6 5 6 c i - 6 5.5 8 c 2 + 9 6.7 5 c i c 2 + 84c2 - 43 .4 2 5 c 3 + 7 9.8 7 5 c 1c 3 + 1 2 3.c 2 c 3 + 4 8.3 7 5 C 3
T h e m i n i m u m o f t h i s f u n c t i o n i s c o m p u t e d a s f o l l o w s: s o l n = t J U c o n s t r a i n e d O p t i m a l i t y [ £, { c 1# c 2, c 3 } ];
O b j e c t i v e f u n c t i o n ^ 1 8.7 - 3 2.8 4 6 2 c! + 3 4.2 6 5 6 c f - 6 5.5 8 c 2 + 9 6.7 5 ^ 2 + 8 4 c 2 - 4 3.4 2 5 c 3 + 7 9.8 7 5 c 1 c 3 + 1 2 3.c 2 c 3 + 4 8.3 7 5 c 3
G r a d i e n t v e c t o r - »
H e s s i a n m a t r i x - »
'- 3 2.8 4 6 2 + 6 8.5 3 1 3 c! + 9 6.7 5 c 2 + 7 9.8 7 5 c 3' - 6 5.5 8 + 9 6.7 5 c 1 + 1 6 8 c 2 + 1 2 3.c 3 - 4 3.4 2 5 + 7 9. 8 7 5 ^ + 123 . c 2 + 9 6.7 5 c 3
( 6 8.5 3 1 3 9 6.7 5 7 9.8 7 5
9 6.7 5 16 8 1 2 3.
7 9.8 7 5 1 2 3. 9 6.7 5
*** * * * F i r s t o r d e r o p t i m a l i t y c o n d i t i o n s *** * * *
- 3 2.8 4 6 2 + 6 8.5 3 1 3 c! + 9 6.7 5 c 2 + 7 9.8 7 5 c 3 == 0' - 6 5 .5 8 + 9 6.7 5 c! + 1 6 8 c 2 + 1 2 3,c 3 = = 0 - 4 3.4 2 5 + 7 9.8 7 5 c! + 1 2 3.c 2 + 9 6.7 5 ^ = = 0
N e c e s s a r y c o n d i t i o n s
p o s s i b l e s o l u t i o n s ( s t a t i o n a r y p o i n t s ) -»
(C l -> 2.0 9 1 0 8 c 2 -» 1.7 5 4 6 5 c 3 -» - 3.5 0 8 2 3 )
****** s e c o n d o r d e r o p t i m a l i t y c o n d i t i o n s *** * * * -------------------- P o i n t -» { c j -» 2.0 9 1 0 8, c 2 -» 1.7 5 4 6 5, c 3 -» - 3.5 0 8 2 3 )
6 8.5 3 1 3
9 6.7 5
7 9.8 7 5
6 8.5 3 1 3 ’
H e s s i a n -»
9 6.7 5
168
1 2 3.
P r i n c i p a l m i n o r s -»
2 1 5 2.6 9
,7 9.8 7 5
1 2 3.
9 6.7 5 ,
,6 8 5.5 4 7,
S t a t u s M i n i m u m P o i n t F u n c t i o n v a l u e -» 2 .9 9 5 5 6
T h u s, t h e s u r f a c e o f t h e f o r m Z c o m p u t e d = c i x 2 + c 2 y z + c $ x y t h a t b e s t f i t s t h e g i v e n d a t a i s a s f o l l o w s:
z = 2.0 9 1 0 8 * 2 + 1.7 5 4 6 5 y 2 - 3.5 0 8 2 3 j < y
T h e e r r o r b e t w e e n t h e o b s e r v e d z v a l u e s a n d t h o s e c o m p u t e d f r o m t h e a b o v e f o r m u l a i s s h o w n i n t h e f o l l o w i n g t a b l e. C o n s i d e r i n g t h a t w e u s e d o n l y a f e w
P o i n t
X
y
2o b s e r v e d
^ c o m p u t e d
^ ^ o b s e r v e d W p u t e d l 1 Q ( ) % L z o b s e r v e d J
1
0 ,
1
1.2 6
1.7 5 4 6 5
3 9.3
2
0.2 5
1
2.1 9
1.0 0 8 2 8
5 4.
;
3
0.5
1
0.7 b
0.5 2 3 3 0 5
3 1.1
4
0.7 5
1
1.2 6
0.2 9 9 7 1
7 6.2
5
1
2
1.8 6
2.0 9 3 2 2
1 2.5
6
1.2 5
2
1.4 3
1.5 1 5 3 4
5.9 7
7
1.5
2
1.2 9
1.1 9 8 8 4
7.0 7
8
1.7 5
2
0.6 5
1.1 4 3 7 2
7 6.
9
2
2
1.6
1.3 5
1 5.6
Chapter 4 Optimali t y Conditions
d a t a p o i n t s, t h e r e s u l t s a r e r e a s o n a b l e. U s i n g m o r e d a t a p o i n t s o r a d i f f e r e n t f u n c t i o n a l f o r m c o u l d i m p r o v e r e s u l t s.
4.2 T h e A d d i t i v e P r o p e r t y o f C o n s t r a i n t s
C o n s i d e r a g e n e r a l c o n s t r a i n e d o p t i m i z a t i o n p r o b l e m w i t h t h e f o l l o w i n g i n - e q u a l i t v a n d e q u a l i t y c o n s t r a i n t s:
h j ( x ) = 0, i = 1,..., p
T h e p u r p o s e o f t h i s s e c t i o n i s t o d e m o n s t r a t e t h a t i f a p o i n t x s a t i s f i e s a l l t h e s e c o n s t r a i n t s i n d i v i d u a l l y, t h e n i t a l s o s a t i s f i e s t h e f o l l o w i n g a g g r e g a t e c o n s t r a i n t ( o b t a i n e d b y a l i n e a r c o m b i n a t i o n o f t h e c o n s t r a i n t s )
m P
g a ■ Σ + Σ ^ ( X ) < 0
i = l i = l
w h e r e u x > 0 a n d Vi a r e a n y a r b i t r a r y s c a l a r m u l t i p l i e r s. N o t e t h a t t h e m u l ­
t i p l i e r s f o r i n e q u a l i t y c o n s t r a i n t s a r e r e s t r i c t e d t o b e p o s i t i v e, b u t t h o s e f o r e q u a l i t y c o n s t r a i n t s h a v e n o r e s t r i c t i o n s. T h i s p r o p e r t y o f c o n s t r a i n t s i s c a l l e d t h e a d d i t i v e p r o p e r t y o f c o n s t r a i n t s a n d i s u s e f u l i n u n d e r s t a n d i n g o p t i m a l i t y c o n d i t i o n s f o r c o n s t r a i n e d p r o b l e m s p r e s e n t e d i n l a t e r s e c t i o n s. T h e f o l l o w i n g e x a m p l e s g r a p h i c a l l y i l l u s t r a t e t h i s p r o p e r t y o f c o n s t r a i n t s.
E x a m p l e 4.6 C o n s i d e r t h e f o l l o w i n g e x a m p l e w i t h t w o i n e q u a l i t y c o n ­
s t r a i n t s:
1 1 C o n s t r a i n t 1 : x 2 y 2 < 5 C o n s t r a i n t 2 : x — 2 y 4 - - y 2 < 0
---------------------------------------------------------------- 4 ------------------------------------------------------------------------- 4 -----------------------------------
= * 2 - J y 2 - 5 s 0; g 2 = x - 2y + J y 2 s 0;
L e t's d e f i n e a n a g g r e g a t e c o n s t r a i n t b y a r b i t r a r i l y c h o o s i n g m u l t i p l i e r s 3 a n d 5. = B x p a a d [ 3 F i r e t [ f f x ] + S F i r s t [ b 2 ] ] i 0
ο Y 2
- 1 5 + 5 x + 3 x ^ - l O y + — s 0
2
4.2 T h e Additive P r o p e rt y o f Constraints
T h e o r i g i n a l a n d t h e a g g r e g a t e c o n s t r a i n t s a r e s h o w n i n t h e f o l l o w i n g g r a p h. I t i s c l e a r t h a t t h e f e a s i b l e d o m a i n o f t h e o r i g i n a l c o n s t r a i n t s i s e n t i r e l y o n t h e f e a s i b l e s i d e o f t h e a g g r e g a t e c o n s t r a i n t. I n o t h e r w o r d s, a n y f e a s i b l e p o i n t f o r t h e o r i g i n a l c o n s t r a i n t s i s a l s o f e a s i b l e f o r t h e a g g r e g a t e c o n s t r a i n t.
A c t u a l c o n s t r a i n t s g a w i t h m u l t i p l i e r s 3 & 5
A s a n o t h e r i l l u s t r a t i o n, a n e w a g g r e g a t e c o n s t r a i n t i s d e f i n e d u s i n g a m u l ­
t i p l i e r 1 f o r b o t h c o n s t r a i n t s. A g a i n, t h e p l o t c l e a r l y s h o w s t h a t t h e f e a s i b l e d o m a i n o f t h e o r i g i n a l c o n s t r a i n t s i s e n t i r e l y o n t h e f e a s i b l e s i d e o f t h e a g g r e ­
g a t e c o n s t r a i n t.
0a = E x p a n d [ F i r s t + F i r s t [ g 2 ] ] £ 0
- 5 + x + x 2 - 2 y < 0
M u l t i p l i e r s 1 & 1 M u l t i p l i e r s - 3 & 5
C h ap te r 4 O p t im al it y Conditions
I b d e m o n s t r a t e t h a t a n e g a t i v e m u l t i p l i e r f o r a n i n e q u a l i t y c o n s t r a i n t d o e s n o t y i e l d a c o r r e c t a g g r e g a t e c o n s t r a i n t, c o n s i d e r t h e f o l l o w i n g c a s e i n w h i c h t h e m u l t i p l i e r s - 3 a n d 5 a r e u s e d. T h e g r a p h s h o w s t h a t t h e a g g r e g a t e c o n s i s t s o f t w o d i s j o i n t l i n e s l a b e l e d ga- L o o k i n g a t t h e i n f e a s i b l e s i d e s o f t h e s e l i n e s, i t i s c l e a r t h a t n o f e a s i b l e s o l u t i o n i s p o s s i b l e. T h e f e a s i b l e r e g i o n i s e m p t y. T h u s, u s i n g a n e g a t i v e m u l t i p l i e r f o r a n i n e q u a l i t y c o n s t r a i n t d o e s n o t w o r k.
g a = E x p a n d [ - 3 F i r e t [ g t ] + 5 F i r e t [ g 2 ] ] s 0
15 + 5 x - 3 x 2 - l O y + 2 y 2 ί 0
E x a m p l e 4.7 A s a s e c o n d e x a m p l e, c o n s i d e r a p r o b l e m w i t h a n e q u a l i t y a n d a n i n e q u a l i t y c o n s t r a i n t:
C o n s t r a i n t 1: x 2 + 2 y 2 < 1 C o n s t r a i n t 2: x + y = 0
g -—1 + x 2 + 2y 2 i 0; h = x + y == 0 ;
L e t's d e f i n e a n a g g r e g a t e c o n s t r a i n t b y a r b i t r a r i l y c h o o s i n g m u l t i p l i e r s 3 a n d 5:
g a = E x p a n d [ 3 F i r s t [ g ] + S F i r s t [ h ] ] £ 0
- 3 + 5 x + 3 x 2 + 5 y + 6 y 2 s 0
T h e o r i g i n a l a n d t h e a g g r e g a t e c o n s t r a i n t s a r e s h o w n i n t h e f o l l o w i n g g r a p h. T h e f e a s i b l e r e g i o n f o r t h e o r i g i n a l c o n s t r a i n t s i s t h e d a r k l i n e i n s i d e t h e g c o n t o u r. C l e a r l y, t h i s e n t i r e l i n e i s s t i l l i n s i d e t h e a g g r e g a t e c o n s t r a i n t g a -
E q u a l i t y m u l t i p l i e r 5 E q u a l i t y m u l t i p l i e r - 5
F I G U R E 4.7 G r a p h s h o w i n g f e a s i b l e r e g i o n s f o r o r i g i n a l c o n s t r a i n t s a n d a g g r e g a t e c o n s t r a i n t.
4.3 Kargsh-Kuhn-Ttacker (KT) Conditions
T b d e m o n s t r a t e t h a t t h e s i g n o f a m u l t i p l i e r f o r a n e q u a l i t y c o n s t r a i n t d o e s n o t m a t t e r, c o n s i d e r t h e f o l l o w i n g c a s e i n w h i c h t h e m u l t i p l i e r s 3 a n d — 5 a r e u s e d. T h e o r i g i n a l f e a s i b l e r e g i o n i s s t i l l w i t h i n t h e f e a s i b l e r e g i o n o f t h e a g g r e g a t e c o n s t r a i n t.
g a = E x p a n d [ 3 F i r s t [ f f ] - 5 F i r s t [ h ] ] £ 0
- 3 - 5x + 3 x 2 - 5 y + 6 y 2 < 0
4.3 K a r u s h - K u h n - l U c k e r ( K T ) C o n d i t i o n s
K T c o n d i t i o n s a r e f i r s t - o r d e r n e c e s s a i y c o n d i t i o n s f o r a g e n e r a l c o n s t r a i n e d m i n i m i z a t i o n p r o b l e m w r i t t e n i n t h e f o l l o w i n g s t a n d a r d f o r m:
F i n d x i n o r d e r t o M i n i m i z e / ( x )
N o t e t h a t i n t h i s s t a n d a r d f o r m, t h e o b j e c t i v e f u n c t i o n i s o f m i n i m i z a t i o n t y p e, a l l c o n s t r a i n t r i g h t - h a n d s i d e s a r e z e r o, a n d t h e i n e q u a l i t y c o n s t r a i n t s a r e o f t h e l e s s t h a n t y p e. B e f o r e a p p l y i n g t h e K T c o n d i t i o n s, i t i s i m p o r t a n t t o m a k e s u r e t h a t t h e p r o b l e m h a s b e e n c o n v e r t e d t o t h i s s t a n d a r d f o r m.
4.3.1 B a s i c I d e a
T h e K T c o n d i t i o n s c a n b e d e r i v e d b y c o n s i d e r i n g l i n e a r e x p a n s i o n o f t h e o b j e c t i v e a n d t h e c o n s t r a i n t f u n c t i o n s a r o u n d t h e o p t i m u m p o i n t x * u s i n g t h e I k y l o r s e r i e s. T h e o b j e c t i v e f u n c t i o n i s a p p r o x i m a t e d a s f o l l o w s:
F o r x * t o b e a m i n i m u m p o i n t, w e m u s t h a v e V f ( x * ) r d > 0 w h e r e t h e s m a l l c h a n g e s d m u s t b e s u c h t h a t t h e c o n s t r a i n t s a r e s a t i s f i e d. T h e e q u a l i t y c o n ­
s t r a i n t s a r e e x p a n d e d a s f o l l o w s:
/( x ) « /( x * ) + V/( x * ) r ( x - x * ) o r /( x ) - /( x * ) = V f ( x * ) r d
f c i ( x ) f c,( x * ) + V h i ( x * ) T d = 0
Chap ter 4 Op t im al it y Conditions
S i n c e t h e c o n s t r a i n t s m u s t b e s a t i s f i e d a t t h e o p t i m u m p o i n t, h,( x*) = 0. T h u s f o r s u f f i c i e n t l y s m a l l d, w e h a v e
V h j ( x * ) T d = 0, i = 1, 2,..., ρ
F o r e a c h i n e q u a l i t y c o n s t r a i n t, w e n e e d t o c o n s i d e r t h e f o l l o w i n g t w o p o s s i b i l -
1. I n a c t i v e i n e q u a l i t y c o n s t r a i n t s: c o n s t r a i n t s f o r w h i c h #( x * ) < 0. T h e s e c o n s t r a i n t s d o n o t d e t e r m i n e t h e o p t i m u m a n d h e n c e, a r e n o t n e e d e d i n d e v e l o p i n g o p t i m a l i t y c o n d i t i o n s.
2. A c t i v e i n e q u a l i t y c o n s t r a i n t s: c o n s t r a i n t s f o r w h i c h g,( x * ) = 0. T h e s e c o n ­
s t r a i n t s a r e c a l l e d a c t i v e c o n s t r a i n t s a n d a r e e x p a n d e d a s f o l l o w s:
g,- ( x ) « #( x * ) + V g;( x * ) T d < 0 o r V g j ( x * ) r d < 0, i € A c t i v e
I n s u m m a r y, f o r x * t o b e a l o c a l m i n i m u m p o i n t, t h e f o l l o w i n g c o n d i t i o n s m u s t b e s a t i s f i e d s i m u l t a n e o u s l y i n a s m a l l n e i g h b o r h o o d ( i.e., w i t h s u f f i c i e n t l y s m a l l d ).
( a ) V/( x * ) r d > 0
( b ) V h i ( x * ) r d « 0, i = 1, 2,..., p
( c ) V g j ( x * ) r d < 0, i e A c t i v e
U s i n g t h e a d d i t i v e p r o p e r t y o f c o n s t r a i n t s d i s c u s s e d i n t h e p r e v i o u s s e c t i o n, t h e c o n s t r a i n t c o n d i t i o n s c a n b e c o m b i n e d b y m u l t i p l y i n g t h e e q u a l i t y c o n s t r a i n t s b y V i a n d t h e a c t i v e i n e q u a l i t y c o n s t r a i n t s b y «,· > 0. N o t e t h a t t h e m u l t i p l i e r s f o r t h e i n e q u a l i t y c o n s t r a i n t s m u s t b e g r e a t e r t h a n 0; o t h e r w i s e, t h e d i r e c t i o n o f t h e i n e q u a l i t y w i l l b e r e v e r s e d. T h u s, w e h a v e
i t i e s:
P
τ
.i e
d < 0
o r
p
T
d > 0
,ie Active
i=l
C o m p a r i n g t h i s w i t h ( a ) a b o v e, w e c a n s e e t h a t a l l c o n d i t i o n s a r e m e t i f t h e f o l l o w i n g e q u a t i o n i s s a t i s f i e d:
V/( x * ) = -
Σ “ΐν»(χ*)+Σ'’· ^ ( χ*)
_t€Active i=l
F o r a d e t a i l e d m a t h e m a t i c a l j u s t i f i c a t i o n o f w h y t h i s c o n d i t i o n i s n e c e s s a i y f o r t h e m i n i m u m, s e e t h e b o o k b y P e r e s s i n i, S u l l i v a n a n d U h l. T h u s, t h e f i r s t - o r d e r n e c e s s a r y c o n d i t i o n f o r o p t i m a l i t y i s t h a t t h e r e e x i s t m u l t i p l i e r s m > 0, i € A c t i v e, a n d v u i = 1, 2,..., p ( k n o w n a s L a g r a n g e m u l t i p l i e r s ) s u c h t h a t
v/( x * ) + · £ Uiv ^ x * ) + = °
leActive
i = l
I n o t h e r w o r d s, a n e c e s s a r y c o n d i t i o n f o r o p t i m a l i t y i s t h a t t h e g r a d i e n t o f t h e o b j e c t i v e f u n c t i o n i s e q u a l a n d o p p o s i t e t o t h e l i n e a r c o m b i n a t i o n o f g r a d i e n t s o f e q u a l i t y a n d a c t i v e i n e q u a l i t y c o n s t r a i n t s. T h i s p h y s i c a l i n t e r p r e t a t i o n i s e x p l a i n e d i n m o r e d e t a i l i n a l a t e r s e c t i o n.
4.3.2 T h e R e g u l a r i t y C o n d i t i o n
O n e o f t h e k e y a s s u m p t i o n s i n t h e d e r i v a t i o n o f t h e o p t i m a l i t y c o n d i t i o n i s t h a t i t i s p o s s i b l e t o f i n d s u i t a b l e m u l t i p l i e r s s u c h t h a t
V/( x * ) = - £ u,T g i ( x * ) + £ l/,■?*,■(**>
ie Active i=l
M a t h e m a t i c a l l y t h i s m e a n s t h a t t h e g r a d i e n t o f t h e o b j e c t i v e f u n c t i o n c a n b e r e p r e s e n t e d b y a l i n e a r c o m b i n a t i o n o f t h e g r a d i e n t s o f a c t i v e i n e q u a l i t y a n d e q u a l i t y c o n s t r a i n t s. A f u n d a m e n t a l r e q u i r e m e n t t o m a k e i t p o s s i b l e i s t h a t t h e g r a d i e n t s o f a c t i v e i n e q u a l i t y a n d a l l e q u a l i t y c o n s t r a i n t s m u s t b e l i n e a r l y i n d e p e n d e n t. T h i s i s k n o w n a s r e g u l a r i t y c o n d i t i o n.
T h e o p t i m a l i t y c o n d i t i o n s m a k e s e n s e o n l y a t p o i n t s t h a t a r e r e g u l a r, i.e. w h e r e t h e g r a d i e n t s o f a c t i v e i n e q u a l i t y a n d e q u a l i t y c o n s t r a i n t s a r e l i n e a r l y i n d e p e n d e n t. I t s h o u l d b e u n d e r s t o o d t h a t w e a r e n o t s a y i n g t h a t a l l m i n i m u m p o i n t s m u s t b e r e g u l a r p o i n t s a s w e l l. I n f a c t i t i s e a s y t o c o m e u p w i t h e x a m p l e s o f i r r e g u l a r p o i n t s t h a t c a n b e v e r i f i e d t o b e m i n i m u m p o i n t s t h r o u g h o t h e r m e a n s, s u c h a s a g r a p h i c a l s o l u t i o n. T h e r e q u i r e m e n t o f r e g u l a r i t y i s t i e d t o t h e u s e o f t h e o p t i m a l i t y c o n d i t i o n s d e r i v e d h e r e. S i n c e r e g u l a r i t y a s s u m p t i o n i s m a d e d u r i n g t h e d e r i v a t i o n o f t h e s e c o n d i t i o n s, t h e c o n c l u s i o n s d r a w n f r o m t h e s e c o n d i t i o n s a r e v a l i d o n l y a t r e g u l a r p o i n t s.
Chapter 4 Op t im al it y Conditions
P r o c e d u r e f o r c h e c k i n g l i n e a r i n d e p e n d e n c e o f a g i v e n s e t o f v e c t o r s i s e x p l a i n e d i n c h a p t e r 3. I t i n v o l v e s d e t e r m i n i n g r a n k o f t h e m a t r i x w h o s e c o l u m n s a r e t h e g i v e n v e c t o r s. I f t h e r a n k o f t h i s m a t r i x i s e q u a l t o t h e n u m b e r o f v e c t o r s, t h e n t h e g i v e n s e t o f v e c t o r s i s l i n e a r l y i n d e p e n d e n t.
C l e a r l y, i f t h e r e i s o n l y o n e a c t i v e c o n s t r a i n t, t h e n a n y p o i n t i s a r e g u l a r p o i n t, s i n c e t h e q u e s t i o n o f l i n e a r d e p e n d e n c e d o e s n o t e v e n a r i s e. W h e n t h e r e a r e m o r e t h a n o n e a c t i v e c o n s t r a i n t s, o n l y t h e n i t i s n e c e s s a r y t o c h e c k f o r l i n e a r i n d e p e n d e n c e o f g r a d i e n t v e c t o r s o f a c t i v e c o n s t r a i n t s.
4.3.3 T h e L a g r a n g i a n F u n c t i o n
I t i s p o s s i b l e t o d e f i n e a s i n g l e f u n c t i o n ( c a l l e d t h e L a g r a n g i a n f u n c t i o n ) t h a t c o m b i n e s a l l c o n s t r a i n t s a n d t h e o b j e c t i v e f u n c t i o n. T h e f u n c t i o n i s w r i t t e n i n s u c h a w a y t h a t t h e n e c e s s a r y c o n d i t i o n s f o r i t s m i n i m u m a r e t h e s a m e a s t h o s e f o r t h e c o n s t r a i n e d p r o b l e m. I n o r d e r t o d e f i n e t h i s f u n c t i o n, i t i s c o n v e n i e n t t o c o n v e r t a l l i n e q u a l i t i e s t o e q u a l i t i e s. T h i s c a n b e d o n e b y a d d i n g a s u i t a b l e p o s i t i v e n u m b e r ( o b v i o u s l y u n k n o w n a s y e t ) t o t h e l e f t - h a n d s i d e o f e a c h i n e q u a l i t y, a s f o l l o w s:
g i W + s f = 0, * = l,2,...,m
T h e s,'s a r e k n o w n a s s l a c k v a r i a b l e s. T h e y a r e s q u a r e d t o m a k e s u r e t h a t o n l y a p o s i t i v e n u m b e r i s a d d e d. T h e L a g r a n g i a n f u n c t i o n c a n n o w b e d e f i n e d a s
w h e r e x i s a v e c t o r o f o p t i m i z a t i o n v a r i a b l e s, s i s a v e c t o r o f s l a c k v a r i a b l e s, u > 0 i s a v e c t o r o f L a g r a n g e m u l t i p l i e r s f o r i n e q u a l i t y c o n s t r a i n t s, a n d v i s a v e c t o r o f L a g r a n g e m u l t i p l i e r s f o r e q u a l i t y c o n s t r a i n t s.
S i n c e t h e L a g r a n g i a n f u n c t i o n i s u n c o n s t r a i n e d, t h e n e c e s s a r y c o n d i t i o n s f o r i t s m i n i m u m a r e t h a t d e r i v a t i v e s w i t h r e s p e c t t o a l l i t s v a r i a b l e s m u s t b e e q u a l t o z e r o. T h a t i s,
f o l l o w s:
t n
I ( X, U, V, 8 ) = / ( X ) + ^ Ui [#( χ ) + s i ] + Σ V i h i ^
/
m
P
\
4.3 Karush-Kuhn-TUcker ( K T ) Conditions
T h e f i r s t s e t i s k n o w n a s t h e g r a d i e n t c o n d i t i o n. A l l i n e q u a l i t y c o n s t r a i n t s a r e i n c l u d e d i n t h e f i r s t s u m. H o w e v e r, f r o m t h e l a s t s e t o f c o n d i t i o n s, k n o w n a s c o m p l e m e n t a r y s l a c k n e s s
o r s w i t c h i n g c o n d i t i o n s, w e c a n s e e t h a t e i t h e r «,· = 0 o r s,· = 0. I f a s l a c k v a r i a b l e i s z e r o, t h e n t h e c o r r e s p o n d i n g c o n s t r a i n t i s a c t i v e w i t h a p o s i t i v e L a g r a n g e m u l t i p l i e r. I f a c o n s t r a i n t i s i n a c t i v e, t h e n t h e c o r r e s p o n d i n g L a g r a n g e m u l t i p l i e r m u s t b e z e r o. H e n c e, i n r e a l i t y t h e f i r s t s u m i s o v e r a c t i v e i n e q u a l i t y c o n s t r a i n t s o n l y. T h e r e f o r e, t h e s e c o n d i t i o n s a r e t h e s a m e a s t h e o p t i m a l i t y c o n d i t i o n s d e r i v e d i n t h e p r e v i o u s s e c t i o n. T h e s e c o n d a n d t h i r d s e t s o f c o n d i t i o n s s i m p l y s t a t e t h a t t h e c o n s t r a i n t s m u s t b e s a t i s f i e d. T h u s, t h e n e c e s s a r y c o n d i t i o n s f o r t h e m i n i m u m o f t h e L a g r a n g i a n f u n c t i o n a r e t h e s a m e a s t h o s e f o r t h e c o n s t r a i n e d m i n i m i z a t i o n p r o b l e m.
T h e c o m p l e t e s e t o f n e c e s s a r y c o n d i t i o n s f o r o p t i m a l i t y o f a c o n s t r a i n e d m i n - i m i z a t i o n p r o b l e m, k n o w n a s K a r u s h - K u h n - I U c k e r ( K T ) c o n d i t i o n s, a r e a s f o l ­
l o w s:
F o r a p o i n t x * t o b e m i n i m u m o f / ( x ) s u b j e c t t o h,( x ) = 0, i = 1, 2,..., p a n d g;( x ) < 0, i = 1, 2,..., m t h e f o l l o w i n g c o n d i t i o n s m u s t b e s a t i s f i e d.
1. R e g u l a r i t y c o n d i t i o n: x * m u s t b e a r e g u l a r p o i n t
2. G r a d i e n t c o n d i t i o n s: V/( x * ) + « i V g,- ( x * ) + Σ ί = ι t > » V/i;( x * ) = 0
4. C o m p l e m e n t a r y s l a c k n e s s o r s w i t c h i n g c o n d i t i o n s: u j S i = 0, i = 1, 2,..., m
5. F e a s i b i l i t y c o n d i t i o n s: s f > 0, ί = 1, 2,..., m
6. S i g n o f i n e q u a l i t y m u l t i p l i e r s: > 0, i — 1, 2,..., m
T h e K T c o n d i t i o n s r e p r e s e n t a s e t o f e q u a t i o n s t h a t m u s t b e s a t i s f i e d f o r a l l l o c a l m i n i m u m p o i n t s o f a c o n s t r a i n e d m i n i m i z a t i o n p r o b l e m. S i n c e t h e y a r e o n l y n e c e s s a r y c o n d i t i o n s, t h e p o i n t s t h a t s a t i s f y t h e s e c o n d i t i o n s a r e o n l y c a n d i d a t e s f o r b e i n g a l o c a l m i n i m u m, a n d a r e u s u a l l y k n o w n a s K T p o i n t s. O n e m u s t c h e c k s u f f i c i e n t c o n d i t i o n s ( t o b e d i s c u s s e d l a t e r ) t o d e t e r m i n e i f a g i v e n K T p o i n t a c t u a l l y i s a l o c a l m i n i m u m o r n o t.
E x a m p l e 4.8 O b t a i n a l l p o i n t s s a t i s f y i n g K T c o n d i t i o n s f o r t h e f o l l o w i n g
o p t i m i z a t i o n p r o b l e m:
4.3.4 S u m m a r y o f K T C o n d i t i o n s
Chapter 4 Op t im al it y Conditions
M i n i m i z e f i x, y, z ) = 1 4 — 2 x 4 - x 2 - 4 y 4 - y 2 — 6z 4 - z 2 S u b j e c t t o x 2 + y 2 + z ~ l = 0
f = 1 4 - 2 x + x 2 - 4 y + y 2 - 6 z + z 2 ; h = x 2 + y 2 + z - 1 = = 0 ; v a r s = { x, y, z >;
M i n i m i z e f 1 4 - 2 x + x 2 - 4 y + y 2 - 6 z + z 2
- 2 + 2 x ]
V f - 4
- 4 + 2 y
- 6 + 2 Z;
* * * * * e q c o n s t r a i n t s & t h e i r g r a d i e n t s
/2 x'
I *! - * - 1 + x 2 + y 2 + z = = 0 V h ^ - » 2 y
U )
* * * * * L a g r a n g i a n - * 1 4 - 2 x + x 2 - 4 y + y 2 - 6 z + z 2 + ( - 1 + x 2 + y 2 + ζ ) ν χ
V L = 0 - *
2 + 2 x + 2 x v! = = 0 ’ 4 + 2 y + 2 y v i = = 0 - 6 + 2 z + v! - = 0 i - l + x 2 + y 2 + z = = 0,
T h e g r a d i e n t o f L a g r a n g i a n y i e l d s f o u r n o n l i n e a r e q u a t i o n s. T h e f o l l o w i n g i s t h e o n l y p o s s i b l e s o l u t i o n t o t h e s e e q u a t i o n s:
f - > 8.0 3 4 8 x - * 0.1 8 6 9 3 5 y - * 0.3 7 3 8 7 z - > 0.8 2 5 2 7 6 V i - > 4.3 4 9 4 5
S i n c e t h e r e i s o n l y o n e a c t i v e c o n s t r a i n t, t h e r e g u l a r i t y c o n d i t i o n i s t r i v i a l l y s a t i s f i e d. S i n c e a l l r e q u i r e m e n t s a r e m e t, t h i s i s a v a l i d K T p o i n t.
4.3.5 S o l u t i o n o f O p t i m i z a t i o n P r o b l e m s U s i n g K T C o n d i t i o n s
S i n c e t h e r e a r e e x a c t l y a s m a n y e q u a t i o n s a s t h e n u m b e r o f u n k n o w n s, i n p r i n c i p l e, i t i s p o s s i b l e t o s o l v e t h e K T e q u a t i o n s s i m u l t a n e o u s l y t o o b t a i n c a n d i d a t e m i n i m u m p o i n t s ( K T p o i n t s ). H o w e v e r, c o n v e n t i o n a l s o l u t i o n p r o ­
c e d u r e s u s u a l l y g e t s t u c k b e c a u s e o f m a n y d i f f e r e n t s o l u t i o n p o s s i b i l i t i e s a s a r e s u l t o f s w i t c h i n g c o n d i t i o n s. A s p o i n t e d o u t e a r l i e r, t h e s w i t c h i n g c o n d i t i o n s
4.3 Karush-Kuhn-Tucker (ΚΊΓ) Conditions
s a y t h a t a n i n e q u a l i t y c o n s t r a i n t i s e i t h e r a c t i v e (s,· = 0) o r i n a c t i v e ( u,· = 0). A s o l u t i o n c a n b e o b t a i n e d r e a d i l y i f o n e c a n i d e n t i f y t h e c o n s t r a i n t s t h a t a r e a c t i v e a t t h e o p t i m u m. F o r p r a c t i c a l p r o b l e m s, i t m a y b e p o s s i b l e t o i d e n t i f y t h e c r i t i c a l c o n s t r a i n t s b a s e d o n p a s t e x p e r i e n c e w i t h s i m i l a r p r o b l e m s. F o r a g e n e r a l c a s e, o n e m a y c o n s i d e r a t r i a l - a n d - e r r o r a p p r o a c h. A g u e s s i s m a d e f o r t h e a c t i v e a n d i n a c t i v e c o n s t r a i n t s, t h u s s e t t i n g e i t h e r u\ o r s f e q u a l t o z e r o f o r e a c h i n e q u a l i t y c o n s t r a i n t. T h e v a l u e s f o r u n k n o w n u, a n d s f a r e o b t a i n e d f r o m t h e o t h e r K T c o n d i t i o n s. I f a l l c o m p u t e d u\ a n d s f v a l u e s a r e g r e a t e r t h a n o r e q u a l t o z e r o, w e h a v e i d e n t i f i e d t h e c o r r e c t c a s e o f a c t i v e a n d i n a c t i v e c o n s t r a i n t s. T h e s o l u t i o n c o r r e s p o n d i n g t o t h i s c a s e r e p r e s e n t s a K T p o i n t. O f c o u r s e, a f t e r f i n d i n g a s o l u t i o n, o n e m u s t v e r i f y t h a t t h e c o m p u t e d p o i n t i s a r e g u l a r p o i n t, b e c a u s e a s p o i n t e d o u t e a r l i e r, t h e K T c o n d i t i o n s a r e v a l i d o n l y f o r r e g u l a r p o i n t s.
N u m b e r o f P o s s i b l e C a s e s t o b e E x a m i n e d
I f t h e r e a r e m i n e q u a l i t y c o n s t r a i n t s, t h e n t h e s w i t c h i n g c o n d i t i o n s i m p l y t h a t t h e r e a r e a t o t a l o f 2 m d i f f e r e n t c a s e s t h a t a r e p o s s i b l e. F o r e x a m p l e, a p r o b l e m w i t h t h r e e i n e q u a l i t y c o n s t r a i n t s h a s t h e f o l l o w i n g 8 ( = 23 ) p o s s i b i l i t i e s:
1. N o n e o f t h e c o n s t r a i n t s a c t i v e u\ = u 2 = «3 = 0
2. g i A c t i v e s i = u 2 = «3 = 0
3. g 2 A c t i v e u i = s 2 = «3 = 0
4. g 3 A c t i v e «1 = «2 = 53 = 0
5. g i A c t i v e, g 2 A c t i v e s i = s 2 = «3 = 0
6. g 2 A c t i v e, £3 A c t i v e u i = s 2 = S3 = 0
8. A l l t h r e e a c t i v e s i = s 2 = S3 = 0
T h e a c t u a l n u m b e r o f c a s e s t o b e c o n s i d e r e d w i l l i n g e n e r a l b e s m a l l e r t h a n t h i s t h e o r e t i c a l m a x i m u m. T h i s i s b e c a u s e, f o r a w e l l - f o r m u l a t e d p r o b l e m, t h e n u m b e r o f a c t i v e c o n s t r a i n t s m u s t b e l e s s t h a n o r a t m o s t e q u a l t o t h e n u m b e r o f o p t i m i z a t i o n v a r i a b l e s. I f t h i s i s n o t t h e c a s e, t h e n t h e r e g u l a r i t y c a n n o t
b e s a t i s f i e d. M o r e e q u a t i o n s t h a n t h e n u m b e r o f v a r i a b l e s m e a n s t h a t e i t h e r
s o m e o f t h e e q u a t i o n s a r e r e d u n d a n t, o r t h e r e i s n o s o l u t i o n f o r t h e s y s t e m o f e q u a t i o n s. I n t h i s c a s e, w e m u s t r e f o r m u l a t e t h e p r o b l e m. I n t h e a b o v e e x a m p l e, i f t h e p r o b l e m h a s t w o v a r i a b l e s t h e n o b v i o u s l y t h e l a s t c a s e o f a l l t h r e e c o n s t r a i n t s b e i n g a c t i v e i s i m p o s s i b l e. H e n c e, t h e n u m b e r o f c a s e s t o b e
e x a m i n e d r e d u c e s t o s e v e n.
Chapter 4
T h e p r e s e n c e o f e q u a l i t y c o n s t r a i n t s f u r t h e r r e d u c e s t h e n u m b e r o f p o s s i ­
b l e c a s e s s i n c e e q u a l i t y c o n s t r a i n t s a r e a l w a y s a c t i v e b y d e f i n i t i o n. T h u s, i f a p r o b l e m h a s t w o v a r i a b l e s, t h r e e i n e q u a l i t y c o n s t r a i n t s, a n d o n e e q u a l i t y c o n ­
s t r a i n t, o n l y o n e o f t h e i n e q u a l i t i e s c a n b e a c t i v e. P a i r s o f g i c a n n o t b e a c t i v e, a n d t h e r e g u l a r i t y c o n d i t i o n s a t i s f i e d a t t h e s a m e t i m e, s i n c e t h a t w o u l d i m p l y m o r e c o n s t r a i n t s t h a n t h e v a r i a b l e s a t t h e m i n i m u m p o i n t. T h u s, t h e n u m b e r o f p o s s i b l e c a s e s t o b e e x a m i n e d r e d u c e s t o f o u r, a s f o l l o w s.
1. N o n e o f t h e g's a c t i v e
2. g i A c t i v e
3. g 2 A c t i v e
4. g 3 A c t i v e
E v e n w i t h t h i s r e d u c t i o n i n t h e n u m b e r o f c a s e s, t h e c o m p u t a t i o n a l e f f o r t i n c r e a s e s d r a m a t i c a l l y a s t h e n u m b e r o f v a r i a b l e s a n d i n e q u a l i t y c o n s t r a i n t s i n c r e a s e s. T h u s, t h i s m e t h o d s h o u l d b e u s e d o n l y f o r p r o b l e m s w i t h f e w v a r i ­
a b l e s a n d a m a x i m u m o f t h r e e o r f o u r i n e q u a l i t y c o n s t r a i n t s. M o r e e f f i c i e n t s o l u t i o n m e t h o d s f o r l a r g e r p r o b l e m s a r e p r e s e n t e d i n l a t e r c h a p t e r s.
O b t a i n a l l p o i n t s s a t i s f y i n g K T c o n d i t i o n s f o r t h e f o l l o w i n g
o p t i m i z a t i o n p r o b l e m:
M i n i m i z e / ( χ, y,z ) = 1 /( x 2 + y 2 )
/x 2 + y 2 < 5\
S u b j e c t t o
\x + 2 y = 4/
f = l/( x 2 +Y2 ) /
c o n s = { x 2 + y2 £ 5,x £ l 0,y £ 4,x + 2 y = = 4 };
Mi n i mi z e f ->
2 y
***** LE c o n s t r a i n t s & t h e i r g r a d i e n t s
-* - 5 + x 2 + y 2 <, 0 g 2 -* - 1 0 + x < 0
4.3 KaT-ush-Kuhn-Tuckcr ( K T ) Conditions
vg2
Vg3
***** e q c o n s t r a i n t s & t h e i r g r a d i e n t s
h 1 - * - 4 + x + 2 y = = 0 Vhx -> . 2
1
***** L a g r a n g i a n -* ( - 4 + x + 2 y ) V i
x 2 +Y2
+ ( - 5 +sc2 + y 2 + s i ) u i + ( - 1 0 + x + S 2) u 2 + ( - 4 + y + S 3) u 3 +
2 s'y + 2 x u 1 + u 2 + v i = = 0 { x * + y ^ ) *
2 y
"Π 272 + 2^ 1 + u 3 + 2 v l =* 0
( x + y )
- 5 + x 2 + y 2 + s f == 0 -10 + x + s i == 0
VL = 0 -*
- 4 + y + s 3 = = 0 - 4 + x + 2 y == 0
2s^uj _ == 0
2 s 2 u 2 == 0 2 s 3u 3 == 0
T h e g r a d i e n t o f L a g r a n g i a n y i e l d s s i x n o n l i n e a r e q u a t i o n s a n d t h r e e s w i t c h ­
i n g c o n d i t i o n s. S i n c e t h i s i s a t w o - v a r i a b l e p r o b l e m, a n d t h e r e i s o n e e q u a l i t y c o n s t r a i n t a l r e a d y, o n l y o n e o f t h e i n e q u a l i t i e s c a n p o s s i b l y b e a c t i v e. T h u s, t h e r e a r e a t o t a l o f f o u r c a s e s t h a t m u s t b e e x a m i n e d.
^ * * * * * * * * * * * *__________________________________________________________________
A c t i v e i n e q u a l i t i e s - >N o n e
Kn o wn v a l u e s -> f u ^ -> 0, u 2 -> 0, u 3 -» 0 }
2 x
(x'
v x == 0
2v-i == 0
E q u a t i o n s f o r t h i s c a s e -*
7 x W > 2 1
- 5 + x 2 + y 2 + s i == 0
- 1 0 + x + S 2 == 0 2
- 4 + y + s 3 = = 0 - 4 + x + 2 y == 0
S o l u t i o n 1
Ch ap te r 4 Op t im al it y Conditions
( 2\
Si
fl.81
2
S2
*4
9.2
2
S3 J
[2. 4j
'1*1) f ° x u 2 -» 0 \U 3/ W
(V l ) -> ( 0.1 5 6 2 5 ) _________________________________________________________________________________
KT S t a t u s -*■ V a l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e -> 0.3 1 2 4 9 9 9
A c t i v e i n e q u a l i t i e s -* {1}
Kn o wn v a l u e s -> { u 2 -* 0, U 3 -» 0, S j -» 0}
2x
+ 2 x u 1 + V! == 0
E q u a t i o n s f o r t h i s c a s e -»
( x 2 + Y2 ) 2
- ^ w + 2 y u i * 2 v i"° |
- 5 + x 2 + y 2 = = 0
-10 + x + s 2 == 0
2
- 4 + y + S 3 = = 0 - 4 + x + 2 y == 0
f c ) ^ f c
S o l u t i o n 1-
- 0.4 Ϊ
2
U!
« 2
1 0.0 4 0
0
S l'
ί 0
s 2
->
1 0.4
l»iJ
U - 8
(vx) - (0)
C o n s t r a i n t g r a d i e n t m a h r - i y
^ | 0.8 l j p a r,v ^?
4.4
R e g u l a r i t y s t a t u s -» R e g u l a r P o i n t
KT S t a t u s -» V a l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e - > 0.2
/x\ [ 2.
S o l u t i o n 2- 2.1
h i
0
0
s f
0
u 2
0
3 2
8.
l u 3 J
0 J
s i
I 3
( V i ) ( 0 )
C o n s t r a i n t g r a d i e n t m a t r i x
4. 1 2. 2
R a n k -> 2
R e g u l a r i t y s t a t u s - » R e g u l a r P o i n t
KT S t a t u s - » V a l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e - > 0.2
4.3 Kargsh-Kul m-Hicker ( K T ) Conditions
* * * * * * * * * * * * c a s e 3 * * * * * * * * * * * * A c t i v e i n e q u a l i t i e s -> {2}
Known v a l u e s -> {u^ - * 0,U 3 - » 0,s 2 - » 0}
E q u a t i o n s f o r t h i s c a s e
2x \
+ u 2 + Vi == 0
+ 2v^ == 0
{x 2 + y 2}2 2y ( x 2 + y 2) 2 - 5 + x 2 + y 2 + s i == 0
-10 + x == 0
. 2 - 4 + y + S 3 = = 0
- 4 + x + 2 y = = 0
- S o l u t i o n 1 - x l ( 1 0.
Y I "* 1 - 3.
i u i ]
°
u 2
0.0 0 1 9 3 5 8 6
\u 3
0 j
( - 1 0 4 0
7 -
( v i ) -» ( - 0.0 0 0 2 5 2 5 0 4 ) C o n s t r a i n t g r a d i e n t m a t r i x ■
Ca
R a n k —t 2
R e g u l a r i t y s t a t u s - » R e g u l a r P o i n t
KT S t a t u s - » I n v a l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e - * 0.0 0 9 1 7 4 3 1 1
T h e s o l u t i o n i s i n v a l i d b e c a u s e t h e s l a c k v a r i a b l e f o r t h e f i r s t c o n s t r a i n t i s n e g a t i v e.
* * * * * * * * * * * * 4 * * * * * * * * * * * *
A c t i v e i n e q u a l i t i e s - * { 3 }
Kn o wn v a l u e s -+ -> 0, u 2 -» 0, S 3 -> θ}
2x
E q u a t i o n s f o r t h i s c a s e -►
( x ^ Y 2) 2 2y
V l == 0
+ u-j + 2 v! — — 0
( x ^ Y 2 ) 2 3
- 5 + x 2 + y 2 + s 2 = = 0 - 1 0 + x + s 2 = = 0 - 4 + y == 0 - 4 + x + 2 y == 0
- S o l u t i o n 1- x l I - i.'
V I * 4.
Chapter 4 O p t im a l it y Conditions
'V
u 2
u 3
(Vi)
0 0
0.0 2 3 4 3 7 5, ( - 0.0 0 7 8 1 2 5 )
S l'
1
to
'J
s 2
->
1 4.
,
, 0 ,
C o n s t r a i n t g r a d i e n t m a t r i x - *
Ra nk
R e g u l a r i t y s t a t u s -» R e g u l a r P o i n t
KT S t a t u s -» I n v a l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e -» 0.0 3 1 2 5
A g a i n, t h i s s o l u t i o n i s i n v a l i d b e c a u s e t h e s l a c k v a r i a b l e f o r t h e f i r s t c o n s t r a i n t
i s n e g a t i v e. T h u s, w e g e t t h e f o l l o w i n g t h r e e v a l i d K T p o i n t s:
***** v a l i d KT P o i n t ( s ) *****
f -» 0.2 x - 0 . 4 y -* 2 .2 u i -» 0.0 4 u 2 0 u-j -+ 0
s i -* U
s f -* 1 0.4 s f -+ 1.8
V i -+ 0
f -> 0.3 1 2 5 x -» 0 . 8 y -* 1.6
Ui -» 0
u 2 -» 0 u 3 -+ 0 s f -♦ 1.8 s i - * 9.2 S 3 - * 2.4 V i -* 0.1 5 6 2 5
f -» 0.2 x -» 2.
y - » i.
u i -* 0.0 4 u 2 0 u 3 -* 0 s i -» 0
8.
s |
S 3 3
V i -» 0
4.3.6 T h e K T S o l u t i o n F u n c t i o n
T h e s o l u t i o n p r o c e d u r e b a s e d o n e x a m i n i n g a l l p o s s i b l e c a s e s i s i m p l e m e n t e d i n a f u n c t i o n c a l l e d t h e K T S o l u t i o n. T h i s f u n c t i o n a n d o t h e r a u x i l i a r y f u n c t i o n s t h a t i t n e e d s a r e i n c l u d e d i n t h e O p t i m i z a t i o n T b o l b o x O p t i m a l i t y C o n d i t i o n s' p a c k a g e. S i m i l a r t o t h e U n c o n s t r a i n e d O p t i m a l i t y f u n c t i o n, t h e e q u a t i o n s c a n b e s o l v e d e i t h e r b y u s i n g t h e t h e N S o l v e ( d e f a u l t ) o r F i n d R o o t f u n c t i o n s. N S o l v e t r i e s t o f i n d a l l p o s s i b l e s o l u t i o n s a n d s h o u l d b e t r i e d f i r s t. I f i t f a i l s, t h e n F i n d R o o t c a n b e u s e d t o g e t a s o l u t i o n c l o s e s t t o a u s e r - g i v e n s t a r t i n g p o i n t.
N e e d s ["O p t i m i z a t i o n T o o l b o x'O p t i m a l i t y C o n d i t i o n s'"];
?K T S o l u t i o n
K T S o l u t i o n [ f, c o n, v a r s, o p t s ], f i n d s c a n d i d a t e m i n i m u m p o i n t s b y s o l v i n g KT c o n d i t i o n s, f i s t h e o b j e c t i v e f u n c t i o n, c o n i s a l i s t o f c o n s t r a i n t s. T h e f u n c t i o n a u t o m a t i c a l l y c o n v e r t s t h e c o n s t r a i n t s t o s t a n d a r d f o r m b e f o r e p r o c e e d i n g, v a r s i s a l i s t o f p r o b l e m v a r i a b l e s. S e v e r a l o p t i o n s c a n b e u s e d w i t h t h e f u n c t i o n. S e e O p t i o n s [ K T S o l u t i o n ]. B y d e f a u l t a l l p o s s i b l e c a s e s o f i n e q u a l i t i e s b e i n g a c t i v e a r e e x a m i n e d. A c t i v e i n e q u a l i t i e s c a n b e e x p l i c i t l y
s p e c i f i e d t h r o u g h A c t i v e C a s e s o p t i o n.
O p t i o n s U s a g e [ K T S o l u t i o n ]
{ P r i n t L e v e l -» 1, A c t i v e C a s e s -» {} , KTVarNames -» {u, s, ν } ,
S o l v e E q u a t i o n s U s i n g ->N S o l v e, S t a r t i n g S o l u t i o n {}}
P r i n t L e v e l i s a n o p t i o n f o r m o s t f u n c t i o n s i n t h e O p t i m i z a t i o n T o o l b o x.
I t i s s p e c i f i e d a s a n i n t e g e r. T h e v a l u e o f t h e i n t e g e r i n d i c a t e s how much i n t e r m e d i a t e i n f o r m a t i o n i s t o b e p r i n t e d. A P r i n t L e v e l - » 0 s u p p r e s s e s a l l p r i n t i n g. D e f a u l t f o r m o s t f u n c t i o n s i s s e t t o 1 i n w h i c h c a s e t h e y p r i n t o n l y t h e i n i t i a l p r o b l e m s e t u p. H i g h e r i n t e g e r s p r i n t m o r e i n t e r m e d i a t e r e s u l t s.
A c t i v e C a s e s - » L i s t o f a c t i v e LE c o n s t r a i n t s fco b e c o n s i d e r e d i n t h e s o l u t i o n. D e f a u l t i s t o c o n s i d e r A l l p o s s i b l e c a s e s.
KTVarNames-» V a r i a b l e n a m e s u s e d f o r g ('<' c o n s t r a i n t s ) m u l t i p l i e r s, s l a c k v a r i a b l e s a n d h c o n s t r a i n t s ) m u l t i p l i e r s. D e f a u l t i s
{u, s, v }.
S o l v e E q u a t i o n s U s i n g i s a n o p t i o n f o r O p t i m a l i t y c o n d i t i o n b a s e d m e t h o d s. M a t h e m a t i c a f u n c t i o n u s e d t o s o l v e s y s t e m o f e q u a t i o n s i s s p e c i f i e d w i t h t h i s o p t i o n. T h e c h o i c e i s b e t w e e n N S o l v e ( d e f a u l t ) a n d F i n d R o o t.
S t a r t i n g S o l u t i o n u s e d w i t h t h e F i n d R o o t f u n c t i o n i n M a t h e m a t i c a. i t i s u s e d o n l y i f t h e m e t h o d s p e c i f i e d i s F i n d R o o t.
E x a m p l e 4.1 0 O b t a i n a l l p o i n t s s a t i s f y i n g K T c o n d i t i o n s f o r t h e f o l l o w i n g o p t i m i z a t i o n p r o b l e m:
M i n i m i z e f ( x, y, z ) = 1 /( x 2 + y 2 + z 2 )
S u b j e c t t o ( * + & + * - ■ ■
\ x + y + z = 0 )
U s i n g t h e K T S o l u t i o n f u n c t i o n, t h e s o l u t i o n i s o b t a i n e d a s f o l l o w s:
f = l/( x 2 + y 2 + z 2 ) i h = { x 2 + 2 y 2 + 3 z 2 == 1, x + y + z = = 0 }; v a r s = { x, y, z };
K T S o l u t i o n [ f, h, v a r s ] ;
M i n i m i z e f ~ ~
x . + y ^ + z
2 x
(X2
♦ y 2 ^ 2 ) 2
2 y
(X2
+ y 2 + z 2 ) 2
2 z
(X2
+ y 2 + z 2 ) 2
* * * * * EQ c o n s t r a i n t s & t h e i r g r a d i e n t s h i -► - l + x 2 + 2-y2 + 3 z 2 == 0 h 2 - + x + y + z = = 0
Chapter 4 Op t im al it y Conditions
f2 x l
f l\
Vhi -»
4 y
v h 2 -*
1
LezJ
U J
***** L a g r a n g i a n - 2x
x 2 + y 2 + z
( x 2 +Y2 + z 2)2
2^ + ( - 1 + x + 2y* + 3 z ) vx + ( x + y + z ) v 2 + 2xv^ + v 2 == 0
2y
VL = 0 -»
+ 4yv^ + v 2 == 0 0
( x 2 + y 2 + z 2 ) 2 2z
— =------- =— 5· + 6z v x + v 2
( x 2 + y 2 + z 2) 2
- 1 + x 2 + 2Y 2 + 3 z 2 = = 0
x + y + z == 0
T h e g r a d i e n t o f L a g r a n g i a n y i e l d s f i v e n o n l i n e a r e q u a t i o n s. T h e r e a r e f o u r p o s s i b l e s o l u t i o n s t o t h e s e e q u a t i o n s.
— S o l u t i o n 1
f - 0.6 6 1 2 2 5 ^ -> 0.4 8 4 0 5
0.1 7 7 1 7 5
1.4 2 2 6 5 - 0.7 9 5 1 6 6 }
C o n s t r a i n t g r a d i e n t m a t r i x ■
J v i j ^ / 1.4 2 2 6 5 \
( - 1.3 2 2 4 5 1\
1.9 3 6 2 1
R a n k -> 2
.1.0 6 3 0 5l j R e g u l a r i t y s t a t u s - » R e g u l a r P o i n t
KT S t a t u s -» V a l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e -» 1.42264
S o l u t i o n 2 —
x\ f - 0.131633' y -> - 0.3 5 9 6 2 7 z j 0.4 9 1 2 6 ;
v x\ I 2.5 7 7 3 5 \ v 2/ ^ - 1.0 7 0 2 8 ]
C o n s t r a i n t g r a d i e n t m a t r i x -
Ra n k -» 2
( - 0.2 6 3 2 6 5 l 1 - 1.4 3 8 5 1 1
. 2.9 4 7 5 6 I j ------------------------------------------------
R e g u l a r i t y s t a t u s -+ R e g u l a r P o i n t
KT S t a t u s -» V a l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e -» 2.5 7 7 3 5
S o l u t i o n 3-
'x\ ( 0.1 3 1 6 3 2 3 ^
y | -> 0.3 5 9 6 2 7
\z ) - 0.4 9 1 2 6
4.3 Karush-Kuhn-IUcker ( K T ) Conditions
v i
v 2
\ /2.5 7 7 3 5\
r l » ·
0 7 0 2 8/
( 0.2 6 3 2 6 5 1
C o n s t r a i n t g r a d i e n t m a t r i x
1.4 3 8 5 1 1 Ra n k - » 2
^ - 2.9 4 7 5 6 l j
R e g u l a r i t y s t a t u s - >R e g u l a r P o i n t
KT S t a t u s -> Va l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e -> 2 .5 7 7 3 5
- S o l u t i o n 4-
X
' 0.6 6 1 2 2 5 '
y
- 0.4 8 4 0 5
- 0.1 7 7 1 7 5,
\ / 1.4 2 2 6 5 ^
i ^ I 0 - 7
.7 9 5 1 6 6/
C o n s t r a i n t g r a d i e n t m a t r i x ->
1.3 2 2 4 5 1\
Ra n k -> 2
- 1.9 3 6 2 1
[ - 1.0 6 3 0 5 1,
R e g u l a r i t y s t a t u s -> R e g u l a r P o i n t
KT S t a t u s -» V a l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e -» 1.4 2 2 6 4
* **** V a l i d KT P o i n t ( s ) *****
f - » 1.4 2 2 6 5 x- > - 0.6 6 1 2 2 5 y - » 0.4 8 4 0 5 z -> 0.1 7 7 1 7 5 Vi - * 1.4 2 2 6 5 v 2 -> - 0.7 9 5 1 6 6
f - > 2.5 7 7 3 5 x- > - 0.1 3 1 6 3 3 y - > - 0.3 5 9 6 2 7 z - > 0.4 9 1 2 6 v x - > 2.5 7 7 3 5 v 2 -> —1.0 7 0 2 8
f -> 2 r 57735 x - > 0.1 3 1 6 3 3 y - > 0.3 5 9 6 2 7 z -> - 0.4 9 1 2 6 v x - > 2.5 7 7 3 5 Vj -> 1.0 7 0 2 8
f - > 1.4 2 2 6 5 x -> 0.6 6 1 2 2 5 y -> - 0.4 8 4 0 5 z -> - 0.1 7 7 1 7 5 v x ^ 1.4 2 2 6 5 v 2 -> 0.7 9 5 1 6 6
E x a m p l e 4.1 1 B u i l d i n g d e s i g n C o n s i d e r t h e b u i l d i n g d e s i g n p r o b l e m p r e ­
s e n t e d i n C h a p t e r 1. T h e p r o b l e m s t a t e m e n t i s a s f o l l o w s.
T b s a v e e n e r g y c o s t s f o r h e a t i n g a n d c o o l i n g, a n a r c h i t e c t i s c o n s i d e r i n g d e s i g n i n g a p a r t i a l l y b u r i e d r e c t a n g u l a r b u i l d i n g. T h e t o t a l f l o o r s p a c e n e e d e d i s 2 0,0 0 0 m 2. L o t s i z e l i m i t s t h e l o n g e r b u i l d i n g d i m e n s i o n s t o 5 0 m i n p l a n. T h e r a t i o b e t w e e n t h e p l a n d i m e n s i o n s m u s t b e e q u a l t o t h e g o l d e n r a t i o ( 1.6 1 8 ), a n d e a c h s t o r y m u s t b e 3.5 m h i g h. T h e h e a t i n g a n d c o o l i n g c o s t s a r e e s t i m a t e d a t $ 1 0 0 m 2 o f t h e e x p o s e d s u r f a c e a r e a o f t h e b u i l d i n g. T h e o w n e r h a s s p e c i f i e d t h a t t h e a n n u a l e n e r g y c o s t s s h o u l d n o t e x c e e d $ 2 2 5,0 0 0. T h e o b j e c t i v e i s t o d e t e r m i n e b u i l d i n g d i m e n s i o n s s u c h t h a t t h e c o s t o f e x c a v a t i o n i s m i n i m i z e d.
A f o r m u l a t i o n f o r t h i s p r o b l e m i n v o l v i n g f i v e d e s i g n v a r i a b l e s i s d i s c u s s e d i n C h a p t e r 1.
n = N u m b e r o f s t o r i e s
d = D e p t h o f b u i l d i n g b e l o w g r o u n d
h — H e i g h t o f b u i l d i n g a b o v e g r o u n d ί = L e n g t h o f b u i l d i n g i n p l a n w = W i d t h o f b u i l d i n g i n p l a n
F i n d ( r t, d, h, i, a n d w ) i n o r d e r t o
M i n i m i z e / = d l w
/ d + h
= 3.5
\
S u b j e c t t o
S u b j e c t t o
rt
ί = 1.6 1 8 i v 1 0 0 ( 2 U + 2 h w + i w ) < 2 2 5,0 0 0 ί < 5 0 w < 5 0 n l w > 2 0,0 0 0 \n > 1, d > 0, h > 0, ί > 0 a n d w > 0/
U s i n g t h e e q u a l i t y c o n s t r a i n t s ( s u b s t i t u t i n g n = ( d + h )/3.5 a n d I = 1.6 1 8 w ), w e g e t t h e f o l l o w i n g f o r m u l a t i o n i n t e r m s o f t h r e e v a r i a b l e s:
F i n d ( d, h, a n d i v ) i n o r d e r t o
M i n i m i z e / = 1.6 1 8 0 0 i f t v 2
/1 0 0 ( 5.2 3 6 0 0 J i i v + 1.6 1 8 0 0 w 2 ) < 2 2 5,0 0 0\
1.6 1 8 0 0 i v < 5 0 0.4 6 2 2 8 5 7 ( d + f c ) i v 2 > 2 0,0 0 0 y d > 0, h > 0, a n d w > 0 /
I n o r d e r t o s o l v e i t u s i n g K T c o n d i t i o n s, t h e r e w i l l b e 2 6 = 6 4 c a s e s t o b e e x a m i n e d. I t i s n o t p r a c t i c a l t o l i s t t h e s o l u t i o n s o f a l l t h e s e c a s e s. A f t e r a f e w t r i a l s, i t w a s d e t e r m i n e d t h a t t h e c a s e t h a t p r o d u c e s a v a l i d K T p o i n t i s w h e n t h e f i r s t a n d t h e t h i r d c o n s t r a i n t s a r e a c t i v e. T h e s o l u t i o n f o r o n l y t h i s c a s e i s p r e s e n t e d b e l o w.
C l e a r [ d, h, w] ; v a r s = {d, h, w }; f = l.e i e d w 2;
c o n s = {100 ( 5.2 3 6 h w + 1.618W2 ) S 2 2 5,0 0 0, 1.6 1 8 w S 5 0, 0 .4 6 2 2 8 5 6 ( d + h ) * 2 *
2 0,0 0 0, d i 0, h i 0,W 2 0 };
K T S o l u t i o n [ f, c o n s, v a r s, P r i n t L e v e l - » 2, A c t i v e C a s e s { { 1/3 } } ] }
M i n i m i z e f - » 1.618dw2
1.6 1 8 W2'
Vf -» 0
,3.236dw(
4.3 Kani «h-Kuhn-TUcker (KT) Conditions
***** LE c o n s t r a i n t s & t h e i r g r a d i e n t s
g ^ -► - 2 2 5,0 0 0 + 1 0 0 ( 5 .2 3 6 h w + 1.6 1 8 W2 ) 5 0 g 2 -+ - 5 0 + 1. 6 1 8 w < 0 g 3 -► 2 0,0 0 0 - 0 .4 6 2 2 8 6 ( d + h ) w2 < 0 g 4 -» - d < 0 g 5 -» - h s 0 g 6 -> - w s 0
0
>
' 0 ’
v g i -»
523.
6 w
V g 2 -*
0
l523.6h +
323. 6w,
1.6 1 8,
' - 0.4 6 2 2 8 6 W 2
'- V
V g 3 -*
- 0.4 6 2 2 8 6 W 2
V g 4
0
- 0.9 2 4 5 7 1 d w - 0.9 2 4 5 7 1 h w;
.0 j
f 0
' 0 '
Vg5 ->
- 1
Vg6
0
,o J
-1,
* * * * * L a g r a n g i a n -* 1.6 1 8 d w 2 + ( - 2 2 5,0 0 0 + 1 0 0 ( 5 .2 3 6 h w + 1.618W2 ) + s 2 ) u ^ +- ( - 5 0 + 1.6 1 8 w + s 2 ) u 2 + ( 2 0,0 0 0 - 0.4 6 2 2 8 6 ( d + h ) v ^ + s f ) u 3 + ( - d + s 2 ) u 4 + ( - h + s 2 ) u 5 + ( - t o + s f ) u g
* * * * * * * * * * * * Ca s © X * * * * * * * * * * * *
A c t i v e i n e q u a l i t i e s * * { 1, 3}
Kn o wn v a l u e s -> { u 2 -> 0, u 4 -» 0, u 5 -* 0, u 6 -» 0, S i -» 0, s 3 -> θ}
E q u a t i o n s f o r t h i s c a s e
1.6 1 8 W 2 - 0 .4 6 2 2 86w2 u 3 == 0 5 2 3 . 6wu^ - 0 . 4 6 2 2 8 6 w 2 u 3 == 0 3.2 3 6 d w + 5 2 3.6 h u! + 3 2 3.6 w u x - 0.9 2 4 5 7 1 d w u 3 - 0.9 2 4 5 7 1 h w u 3 == 0 - 2 2 5,0 0 0 + 1 0 0 ( 5.2 3 6 h w + 1.6 1 8 W 2 ) = = 0 - 5 0 + 1.6 1 8 w + s 2 == 0 2 0,0 0 0 - 0.4 6 2 2 8 6 ( d + h ) w2 == 0 - d + S 4 = = 0
- h + s 2 = = 0
I - w + Sg == 0
- S o l ut i o n 1-
rd'
'8 0.0 2 7 3'
h
-*
1 3.3 0 6 1
2 1.5 2 9 9,
U i ’
Ό.0 6 6 5 3 0 4'
s i
S2
0 '
u 2
0
15.1647
u 3
- V
3.5
S 3
0
u 4
0
sl
80.0273
u 5
0
s i
13.3061
0
* 1
i21.5299.
C o n s t r a i n t g r a d i e n t m a t r i x - »
0 - 2 1 4.2 8 6
1 1 2 7 3. - 2 1 4.2 8 6
^ 1 3 9 3 4.1 - 1 8 5 7.8 8 )
Rank
R e g u l a r i t y s t a t u s - »R e g u l a r P o i n t
KT S t a t u s -»V a l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e -» 6 0 0 2 0 .4
- S o l u t i o n 2-
fd]
( 1 0 6.6 3 9
h
- 1 3.3 0 6 1
- 2 1.5 2 9 9
S 1
s 2
s f
s l
S5
\s |;
C o n s t r a i n t g r a d i e n t m a t r i x ■
[u i'
( - 0.0 6 6 5 3 0 4 1
u 2
0
u 3
3.5
u 4
0
u 5
0
U»6 J
0
8 4.8 3 5 3
0
1 0 6.6 3 9
- 1 3.3 0 6 1
- 2 1.5 2 9 - 9/
0
- 1 1 2 7 3.
^ - 1 3 9 3 4.1
- 2 1 4.2 8 6 - 2 1 4.2 8 6 1 8 5 7.8 8 ,
Ra n k -» 2
R e g u l a r i t y s t a t u s -» R e g u l a r P o i n t
KT S t a t u s - » I n v a l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e
7 9 9 7 9.6
* **** v a l i d KT P o i n t ( s ) *****
f - » 6 0 0 2 0.5 d- » 8 0.0 2 7 3 h -> 1 3.3 0 6 1 w-* 2 1.5 2 9 9 Ui -» 0.0 6 6 5 3 0 4 u 2 0 u 3 -» 3 .5 u 4 -» 0 u 5 0 u 6 -» 0 s i -> 0
s\ -» 1 5.1 6 4 7 S 3 -♦ 0
s l -» 8 0.0 2 7 3 s s -> 1 3.3 0 6 1 s l -» 2 1.5 2 9 9
T h u s, t h e K T s o l u t i o n i s a s f o l l o w s:
d * = 8 0.0 3 m h * = 1 3.3 1 m w * = 2 1,5 3 m /* = 6 0 0 2 0.5 m 3
4.4 Int e rpre tat io n o f KT Conditions
T h i s s o l u t i o n i s c o m p a r a b l e t o t h e o n e o b t a i n e d u s i n g g r a p h i c a l m e t h o d s i n C h a p t e r 1.
4.4 G e o m e t r i c I n t e r p r e t a t i o n o f K T C o n d i t i o n s
F o r p r o b l e m s w i t h t w o v a r i a b l e s, i t i s p o s s i b l e t o d r a w g r a d i e n t v e c t o r s a n d g e o m e t r i c a l l y i n t e r p r e t t h e K T c o n d i t i o n s. T h e s a m e c o n c e p t s a p p l y t o t h e g e n e r a l c a s e o f n v a r i a b l e s, b u t o b v i o u s l y p i c t u r e s c a n n o t b e d r a w n i n t h a t c a s e. C o n s i d e r i n g o n l y t h e a c t i v e i n e q u a l i t y c o n s t r a i n t s, t h e L a g r a n g i a n f u n c t i o n i s
P
L = /( x ) + Σ K t f i O O + Σ v M * )
ie Active i=l
F r o m t h e a d d i t i v e p r o p e r t y o f c o n s t r a i n t s, t h e t e r m s ^ i e A c t i v e u;g;( x ) 4 - Σ ί = ι v i h i ( x ) r e p r e s e n t a n a g g r e g a t e c o n s t r a i n t. T h i s i s t r u e f o r a n y a r b i t r a r y m u l t i p l i e r s ( u j > 0 t h o u g h ). T h e s p e c i f i c m u l t i p l i e r s f o r w h i c h t h e m i n i m u m o f t h e u n c o n s t r a i n e d L a g r a n g i a n f u n c t i o n i s t h e s a m e a s t h a t o f t h e o r i g i n a l c o n s t r a i n e d p r o b l e m a r e t h o s e t h a t s a t i s f y t h e f o l l o w i n g e q u a t i o n s:
V/( x * ) = -
Ρ
Σ + £ > V l n ( x · )
ie Active i=l
I f t h e r e a r e n o e q u a l i t y c o n s t r a i n t s a n d n o n e o f t h e i n e q u a l i t i e s a r e a c t i v e, t h e n t h e p r o b l e m i s e s s e n t i a l l y u n c o n s t r a i n e d, a t l e a s t i n a s m a l l n e i g h b o r h o o d o f x *, a n d t h e a b o v e c o n d i t i o n s s i m p l y s t a t e t h a t t h e g r a d i e n t o f t h e o b j e c t i v e f u n c t i o n m u s t b e z e r o. T h e s e a r e o b v i o u s l y t h e s a m e a s t h e n e c e s s a r y c o n d i t i o n s f o r t h e m i n i m u m o f u n c o n s t r a i n e d p r o b l e m s.
I f t h e r e i s o n l y o n e c o n s t r a i n t ( e i t h e r a n e q u a l i t y o r a n a c t i v e i n e q u a l i t y ), t h e n t h e a b o v e e q u a t i o n s s a y t h a t a t a K T p o i n t, t h e g r a d i e n t o f t h e o b j e c t i v e f u n c t i o n m u s t l i e a l o n g t h e s a m e l i n e, b u t o p p o s i t e i n d i r e c t i o n, a s t h e g r a d i e n t o f c o n s t r a i n t. T h e L a g r a n g e m u l t i p l i e r i s s i m p l y a s c a l e f a c t o r t h a t m a k e s t h e l e n g t h o f t h e c o n s t r a i n t g r a d i e n t v e c t o r t h e s a m e a s t h a t o f t h e o b j e c t i v e f u n c t i o n g r a d i e n t.
I f t h e r e a r e t w o c o n s t r a i n t s ( e i t h e r e q u a l i t y o r a c t i v e i n e q u a l i t i e s ), t h e n t h e r i g h t - h a n d s i d e r e p r e s e n t s t h e v e c t o r s u m ( r e s u l t a n t ) o f c o n s t r a i n t g r a d i e n t s. T h e L a g r a n g e m u l t i p l i e r s a r e t h o s e s c a l e f a c t o r s t h a t m a k e t h e l e n g t h o f t h e
Chapter 4 Op t im al it y Conditions
r e s u l t a n t v e c t o r t h e s a m e a s t h a t o f t h e o b j e c t i v e f u n c t i o n g r a d i e n t. I n t e r m s o f t h e a g g r e g a t e c o n s t r a i n t, t h i s m e a n s t h a t m u l t i p l i e r s a r e d e f i n e d s u c h t h a t t h e r e s u l t a n t v e c t o r c o i n c i d e s w i t h t h e g r a d i e n t o f t h e a g g r e g a t e c o n s t r a i n t. T h e f o l l o w i n g e x a m p l e s i l l u s t r a t e t h e s e p o i n t s.
E x a m p l e 4.1 2 C o n s i d e r t h e s o l u t i o n o f t h e f o l l o w i n g t w o v a r i a b l e m i n i m i z a ­
t i o n p r o b l e m s.
f = - x - y;
ff = {x + y 2 - 5 s 0/x - 2 s 0 } > v a r s = { x, y };
T h e s o l u t i o n u s i n g t h e K T c o n d i t i o n s i s a s f o l l o w s:
K T S o l u t i o n [ f, g, v a r s, P r i n t L e v e l -» 2 ];
M i n i m i z e f -» - x - y
S i n c e t h e r e a r e t w o i n e q u a l i t y c o n s t r a i n t s, w e n e e d t o e x a m i n e f o u r p o s s i b l e c a s e s.
* * * * * * * * * * * * Ca.se X * * * * * * * * * * * *
Ac t i v e i n e q u a 1 i t i e s -> None
* **** LE c o n s t r a i n t s & t h e i r g r a d i e n t s
g 1 -» - 5 +.x + y 2 < 0 g 2 -> - 2 t x ί 0
2 2 2 L a g r a n g i a n -* - x - y + ( - 5 + x + y ^ + s i ) ^ + ( - 2 + x + s 2 ) u 2
VL = 0 -*
- 1 + u- l + u 2 = = 0 - 1 + 2 y u! == 0 - 5 + x + y 2 + s i =ψ 0 - 2 + x + s | = = 0
2 s 1 u 1 = = 0
2 s 2 u 2 = = 0
K n o w n v a l u e s -» { u! -* 0, u 2 -» θ }
E q u a t i o n s f o r t h i s c a s e -»
F a l s e F a l s e - 5 + x + y 2 + s f == 0 ι - 2 + x + s | = = 0
N o s o l u t i o n f o r t h i s c a s e
N o t e t h a t t h e F a l s e i n t h e l i s t o f e q u a t i o n s i n d i c a t e s t h a t b y s u b s t i t u t i n g k n o w n v a l u e s f o r t h e c a s e, t h e c o r r e s p o n d i n g e q u a t i o n c a n n o t b e s a t i s f i e d. I n
4.4 Geometric Int e r pr e tat io n o f KT Conditions
t h i s c a s e, b y s e t t i n g u\ = « 2 = 0, o b v i o u s l y t h e f i r s t t w o e q u a t i o n s b e c o m e n o n s e n s e.
C a s e 2 * * * * * * * * * * * *
A c t i v e i n e q u a l i t i e s ->{1} Known v a l u e s -> {u2 -> 0, SJ -> 0 }
E q u a t i o n s f o r t h i s c a s e -
- 1 + U j == 0
- 1 + 2 y u i == 0 - 5 + x + = = 0
- 2 + x + s 2 = = 0
e j
» 1
U,
- S o l u t i o n 1 -
( 4.7 5 Ϊ 0.5
1.
2 \ S l 2
- 2.7 5
s 2
KT S t a t u s -> I n v a l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e - > - 5.2 5
T h i s s o l u t i o n i s i n v a l i d b e c a u s e t h e s l a c k v a r i a b l e f o r t h e s e c o n d c o n s t r a i n t i s n e g a t i v e.
* * * * * * * * * * * * C a s e 3 * * * * * * * * * * * *
A c t i v e i n e q u a l i t i e s -» { 2 }
K n o w n v a l u e s -» { u i -> 0, s 2 - 0 }
E q u a t i o n s f o r t h i s c a s e
- 1 + u 2 = = 0 F a l s e - 5 + x + y 2 + S i = = 0
- 2 + x
N o s o l u t i o n f o r t h i s c a s e
* * * * * * * * * * * * C a s e 4 * * * * * * * *
A c t i v e i n e q u a l i t i e s - > { 1,2 }
K n o w n v a l u e s -> { s i -> 0, s 2 -> θ }
E q u a t i o n s f o r t h i s c a s e ->
- 1 + u j + u 2 = = 0' - 1 + 2 y u! == 0 - 5 + x + y 2 = = 0 - 2 + x = = 0
- S o l u t i o n 1 -
2 z \
■ 1.7 3 2 0 5/
- 0.2 8 8 6 7 5
1.2 8 8 6 8
( 3
U1
u 2
C hap ter 4 O p t im a l it y Conditions
C o n s t r a i n t g r a d i e n t m a t r i x - »
1 1 - 3.4 6 4 1 0
R a n k - » 2
R e g u l a r i t y s t a t u s - » R e g u l a r P o i n t
KT S t a t u s - » I n v a l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e - » - 0.2 6 7 9 4 9
T h i s s o l u t i o n i s i n v a l i d b e c a u s e t h e L a g r a n g e m u l t i p l i e r f o r t h e f i r s t c o n s t r a i n t i s n e g a t i v e.
S o l u t i o n 2 ------------
R e g u l a r i t y s t a t u s - » R e g u l a r P o i n t
KT S t a t u s - » V a l i d KT P o i n t O b j e c t i v e f u n c t i o n v a l u e - » - 3.7 3 2 0 5
T h e r e a r e t h r e e p o i n t s t h a t s a t i s f y t h e e q u a t i o n s r e s u l t i n g f r o m s e t t i n g t h e g r a d i e n t o f L a g r a n g i a n t o z e r o. H o w e v e r, t w o o f t h e s e p o i n t s a r e r e j e c t e d. T h e f i r s t o n e ( x = 4.7 5, y = 0.5 ) w a s r e j e c t e d b e c a u s e t h e s l a c k v a r i a b l e f o r t h e s e c o n d c o n s t r a i n t w a s n e g a t i v e. T h e s e c o n d o n e [ x = 2, y = — 1.7 3 2 = -\/3 ) w a s r e j e c t e d b e c a u s e t h e L a g r a n g e m u l t i p l i e r f o r t h e f i r s t c o n s t r a i n t w a s n e g a t i v e a t t h i s p o i n t. O n l y o n e p o i n t ( x = 2, y = 1.7 3 2 = \/3 ) s a t i s f i e s a l l r e q u i r e m e n t s o f t h e K T c o n d i t i o n s.
I b u n d e r s t a n d e x a c t l y w h a t i s g o i n g o n, a l l t h e s e p o i n t s a r e l a b e l e d o n t h e g r a p h i c a l s o l u t i o n i n F i g u r e 4.8. F r o m t h e g r a p h, i t i s c l e a r t h a t t h e g l o b a l m i n i m u m i s a t t h e u p p e r i n t e r s e c t i o n o f c o n s t r a i n t s g\ a n d g 2 ( x i = 2 a n d *2 = V S ). T h i s i s t h e s a m e v a l i d K T p o i n t c o m p u t e d b y t h e K T S o l u t i o n. T h e p o i n t l a b e l e d I n v a l i d K T -1 c l e a r l y v i o l a t e s t h e s e c o n d c o n s t r a i n t. D u r i n g t h e K T s o l u t i o n, t h i s v i o l a t i o n w a s i n d i c a t e d b y t h e n e g a t i v e s l a c k v a r i a b l e f o r t h i s c o n s t r a i n t. T h e r e a s o n w h y t h i s p o i n t s a t i s f i e d t h e g r a d i e n t c o n d i t i o n i s a l s o c l e a r f r o m t h e s e c o n d g r a p h t h a t s h o w s g r a d i e n t s o f a c t i v e c o n s t r a i n t ( g i ) a n d t h e o b j e c t i v e f u n c t i o n. T h e g r a d i e n t o f t h e o b j e c t i v e f u n c t i o n a n d t h e f i r s t c o n s t r a i n t a r e p o i n t i n g e x a c t l y i n t h e o p p o s i t e d i r e c t i o n s a t t h i s p o i n t.
T h e g r a d i e n t v e c t o r s s h o w n i n t h e g r a p h s i n F i g u r e 4.9 c l a r i f y w h y t h e l o w e r i n t e r s e c t i o n o f g i a n d g 2 d o e s n o t s a t i s f y t h e K T c o n d i t i o n s. T h e r e s u l t a n t v e c t o r o f g r a d i e n t v e c t o r s V g i a n d V g 2 w i l l o b v i o u s l y l i e i n t h e p a r a l l e l o g r a m f o r m e d b y t h e s e v e c t o r s. T h e m u l t i p l i e r s ( w i t h p o s i t i v e v a l u e s ) w i l l s i m p l y i n c r e a s e o r d e c r e a s e t h e s i z e o f t h i s p a r a l l e l o g r a m. F r o m t h e d i r e c t i o n o f t h e V f v e c t o r a t t h e u p p e r i n t e r s e c t i o n, i t i s c l e a r t h a t f o r s o m e v a l u e s o f t h e m u l t i p l i e r s,
l l.7 3 2 0 5
2.
C o n s t r a i n t g r a d i e n t m a t r i x - »
1 1 3.4 6 4 1 0
R a n k - » 2
- 6 - 4 - 2 0 2 4 6 0 1 2 3 4 5 6 7
F I G U R E 4.8 G r a p h s h o w i n g a l l p o i n t s c o m p u t e d d u r i n g t h e K T s o l u t i o n a n d g r a d i e n t s a t o n e p o i n t.
t h e r e s u l t a n t v e c t o r c a n b e m a d e t o l i e a l o n g t h e s a m e l i n e ( b u t o p p o s i t e i n d i r e c t i o n ) t o t h e V/ v e c t o r. H o w e v e r, t h i s i s i m p o s s i b l e t o d o a t t h e l o w e r i n t e r s e c t i o n. T h u s, e v e n t h o u g h b o t h p o i n t s s a t i s f y t h e s a m e s e t o f c o n s t r a i n t s, o n l y t h e u p p e r o n e i s a K T p o i n t.
F i n a l l y, w e c a n a l s o o b s e r v e t h a t t h e r e s u l t a n t o f t h e a c t i v e c o n s t r a i n t g r a d i ­
e n t v e c t o r i s e x a c t l y t h e s a m e a s t h a t o f t h e a g g r e g a t e c o n s t r a i n t i f w e u s e t h e m u l t i p l i e r s o b t a i n e d f r o m t h e K T s o l u t i o n. F r o m t h e K T s o l u t i o n, w e g e t t h e f o l l o w i n g v a l u e s o f t h e L a g r a n g e m u l t i p l i e r s f o r t h e t w o a c t i v e c o n s t r a i n t s:
u i - 0.2 8 8 6 7 5 u 2 - 0.7 1 1 3 2 5
W i t h t h e s e m u l t i p l i e r s, t h e a g g r e g a t e c o n s t r a i n t i s d e f i n e d a s f o l l o w s:
g a = E x p a n d [ 0.2 8 8 6 7 5 F i r s t [ g [ [ l ] ] ] + 0.7 1 1 3 2 5 F i r s t [ g [ [ 2 ] ] ] ] * 0
-2.86603 + 1.x+ 0.288675y2 < 0
T h e g r a d i e n t o f t h i s a g g r e g a t e c o n s t r a i n t a n d t h a t o f t h e o b j e c t i v e f u n c t i o n a r e a s f o l l o w s:
g r d = G r a d [ { £, F i r s t [ g a ] }, v a r s ]
{{-1, -1} , {1., 0.57735y}}
A t t h e t w o i n t e r s e c t i o n p o i n t s, t h e s e g r a d i e n t s a r e a s f o l l o w s:
{ g r d/. {x-» 2,y - * V 3 >/ O r d/. {*-» 2,y - »
{{{-1, -1} , {1., 1.}} , {{-1, -1} , {1., -1.}}}
Chap ter 4 O p t im a l it y Conditions
N o t e t h a t a t t h e u p p e r i n t e r s e c t i o n p o i n t, t h e t w o a r e e q u a l a n d o p p o s i t e b u t n o t a t t h e l o w e r i n t e r s e c t i o n. T h e r e f o r e, o n l y t h e u p p e r i n t e r s e c t i o n p o i n t s a t i s f i e s t h e K T c o n d i t i o n s. T h e s a m e t h i n g i s s h o w n g r a p h i c a l l y b y p l o t t i n g t h e a g g r e g a t e c o n s t r a i n t, t o g e t h e r w i t h t h e a c t u a l c o n s t r a i n t s.
4
2
0
- 2
- 4
V f - 7
/'■' i
"»«®Vg2 ^
4
2
0
- 2
- 4
V T
^ g l
ryga
^ v g l
0
0 1
F I G U R E 4.9 G r a p h s s h o w i n g a c t i v e c o n s t r a i n t s, a g g r e g a t e c o n s t r a i n t, a n d o b j e c t i v e f u n c t i o n g r a d i e n t s.
E x a m p l e 4.1 3 O b t a i n a l l p o i n t s s a t i s f y i n g K T c o n d i t i o n s f o r t h e f o l l o w i n g o p t i m i z a t i o n p r o b l e m:
M i n i m i z e / ( x, y ) = —x 2 + y 2
S u b j e c t t o jc2 + y 2 — 1 — 0
U s i n g t h e K T S o l u t i o n f u n c t i o n, t h e s o l u t i o n i s o b t a i n e d a s f o l l o w s. T h e g r a - d i e n t o f L a g r a n g i a n y i e l d s t h r e e n o n l i n e a r e q u a t i o n s. T h e r e a r e f o u r p o s s i b l e s o l u t i o n s t o t h e s e e q u a t i o n s:
f = - x 2 + y 2 ; h = x 2 + y 2 - l = = 0; v a r s = { x, y}
K T S o l u t i o n [ £, h, v a r s ];
M i n i m i z e f -» - x 2 + y 2 '- 2 x\
2 y
EQ c o n s t r a i n t s & t h e i r g r a d i e n t s
-» - 1 + χ 2 +Y 2 == 0 Vhi -> ( g )
4.4 Geometric I nt e r pr e tat io n o f K T Conditions
***** L a g r a n g i a n - x 2 + y 2 + ( - 1 + x 2 + y 2 ) V! VL = 0 ->
- 2 x + 2 x v i == 0 1 2 y + 2 y v i = = 0 ,- l + x 2 + y 2 == 0;
***** V a l i d KT P o i n t ( s )
x -* - 1. y -» 0 v i -»· 1.
f -> 1. x - * 0 y - » - l.
V j _ — 1.
f - > 1.
x- > 0
y - » i -
v^ -» - 1.
f —>— 1. x - » 1. y 0
Vj_ - » 1.
T h e f o u r K T p o i n t s a r e s h o w n o n t h e f o l l o w i n g g r a p h. T h e c o n s t r a i n t a n d t h e o b j e c t i v e f u n c t i o n g r a d i e n t s a r e a l s o s h o w n a t t h e s e f o u r p o i n t s. A g a i n, w e n o t i c e t h a t t h e s e a r e t h e o n l y f o u r p o i n t s w h e r e t h e o b j e c t i v e a n d t h e c o n s t r a i n t g r a d i e n t s a r e a l o n g t h e s a m e l i n e. T h e t w o p o i n t s w h e r e t h e g r a d i e n t s p o i n t i n t h e o p p o s i t e d i r e c t i o n s a r e t h e m i n i m u m p o i n t s, w h i l e t h e o t h e r t w o p o i n t s i n f a c t a r e t h e m a x i m u m p o i n t s. T h e g r a d i e n t c o n d i t i o n i s s a t i s f i e d a t t h e t o p a n d t h e b o t t o m p o i n t s b e c a u s e o f t h e n e g a t i v e m u l t i p l i e r s. S i n c e t h e s i g n o f L a g r a n g e m u l t i p l i e r s f o r e q u a l i t y c o n s t r a i n t s i s n o t r e s t r i c t e d, K T c o n d i t i o n s c a n n o t a u t o m a t i c a l l y d i s t i n g u i s h a m a x i m u m f r o m a m i n i m u m.
1.5
1
0.5
;.............................................................. ■1........................ ....... ί..w · τ ·
0
V h — + 4?f - ^ · 2 ~ 4 2 v f < £ 4 - - V h
- 0.5
- 1
V// w \\v
- 1.5
■ ■ ■ . ■ ................................................................, ■ - . . ■ ■ I
- 1.5 - 1 - 0.5 0 0.5 1 1.5
F I G U R E 4.1 0 G r a p h s h o w i n g f o u r K T p o i n t s w h e r e t h e g r a d i e n t c o n d i t i o n i s s a t i s f i e d.
C h ap te r 4 O p t im a l it y Conditions
T b f u r t h e r d e m o n s t r a t e t h a t t h e s i g n o f t h e L a g r a n g e m u l t i p l i e r f o r a n e q u a l ­
i t y c o n s t r a i n t h a s n o s i g n i f i c a n c e, w e m u l t i p l y t h e e q u a l i t y c o n s t r a i n t b y a n e g a t i v e s i g n ( o b v i o u s l y t h e r e i s n o t h i n g w r o n g i n d o i n g t h i s b e c a u s e e q u a l i t y s h o u l d s t i l l h o l d ) a n d c o m p u t e t h e K T s o l u t i o n a g a i n.
f = - X 2 + y 2;
h = - ( x 2 + y 2 - l ) == 0; v a r s = { x, y };
K T S o l u t i o n [ f, h, v a r s ];
* * * * * V a l i d KT P o i n t ( s ) * * * * *
f -» - 1. f - » 1. f - » 1. f -» - 1.
x -> - 1. x 0 x -» 0 x - » l.
y -» 0 y - » - l. y - » l. y -* o
W e g e t e x a c t l y t h e s a m e s o l u t i o n a s b e f o r e e x c e p t f o r t h e o p p o s i t e s i g n s f o r t h e m u l t i p l i e r s. T h e g r a d i e n t s n o w a r e a s s h o w n i n t h e f o l l o w i n g g r a p h. N o w t h e g r a d i e n t s a t t h e m i n i m u m p o i n t a r e i n t h e s a m e d i r e c t i o n. H o w e v e r, s i n c e t h e i r m u l t i p l i e r s a r e n e g a t i v e, t h e y s a t i s f y t h e g r a d i e n t c o n d i t i o n.
- 1.5 - 1 - 0.5
0
0.5
1
1.5
F I G U R E 4.1 1 G r a p h s h o w i n g g r a d i e n t s w i t h e q u a l i t y c o n s t r a i n t m u l t i p l i e d b y a n e g - a t i v e s i g n.
E x a m p l e 4.1 4 A s a f i n a l e x a m p l e, c o n s i d e r a p r o b l e m t h a t d e m o n s t r a t e s a n a b n o r m a l c a s e b e c a u s e t h e o p t i m u m p o i n t i s n o t a r e g u l a r p o i n t:
f = - X i
g = {y - (1 - x) 3 ί 0, - x i 0,- y i 0 }; v a r s = { x, y };
W e f i r s t c o n s i d e r a g r a p h i c a l s o l u t i o n. F r o m t h e g r a p h s h o w n i n F i g u r e 4.1 2, i t i s c l e a r t h a t t h e p o i n t, x = 1 a n d y = 0, i s t h e g l o b a l m i n i m u m. A l s o, c o n s t r a i n t s g i a n d g z a r e a c t i v e a t t h i s p o i n t.
N o w c o n s i d e r t h e K T s o l u t i o n w i t h t h e k n o w n a c t i v e c a s e.
K T S o l u t i o n [ £, g, v a r s, A c t i v e C a s e s - * { { 1, 3 > >, P r i n t L e v e l -» 2 ];
M i n i m i z e f -» - x
* **** l e c o n s t r a i n t s & t h e i r g r a d i e n t s g-L -» - (1 - x) 3 + y < 0 g 2 -> - x < 0
v g 3 -»
L a g r a n g i a n - » - x + ( - ( 1 - x ) 3 + y + s i ) ^ + ( - x + S 2) u 2 + ( - y + s § ) u 3
- 1 + 3 u! - 6 x u -l + 3 x 2U! - u 2 == 0 1 Uj - u 3 == 0 - ( 1 - x ) 3 + y + S i == 0
VL = 0 -*
- x + S 2 == 0 - y + s§ == 0 2 8 ] ^ == 0 2 s 2u 2 == 0 2 s 3u 3 —- — 0
* * * * * * * * * * * * c a s e 1 * * * * * * * * * * * *
A c t i v e" i n e q u a l i t i e s - » { 1, 3}
Kn o wn v a l u e s -» ( u 2 - » 0,s 1 - » 0,s 3 - » 0 }
- 1 + 3 u! - 6XU! + 3 x 2 U i == 0' U1 " u 3 =~ 0
E q u a t i o n s f o r t h i s c a s e - » - ( l - x ) 3 + y = = 0
- x + s | == 0 - y = = 0
No s o l u t i o n f o r t h i s c a s e
* **** No v a l i d KT P o i n t s F o u n d *****
C h ap te r 4 Opt i m a l i t y Conditions
T h e f i r s t t w o e q u a t i o n s f o r t h i s c a s e g i v e t h e s o l u t i o n x = 1 a n d y = 0. H o w e v e r, s u b s t i t u t i n g x = 1 i n t h e f o u r t h e q u a t i o n g i v e s t h e n o n s e n s e — 1 = 0, T h u s t h e e q u a t i o n s d o n o t h a v e a s o l u t i o n. T h i s i s o b v i o u s l y s u r p r i s i n g b e c a u s e w e k n o w t h a t t h i s c a s e g i v e s t h e o p t i m u m s o l u t i o n. T h e a n s w e r l i e s i n t h e r e g u l a r i t y r e q u i r e m e n t. T h e p o i n t ( 1, 0 ) i s n o t a r e g u l a r p o i n t. I b s e e t h i s, w e c o m p u t e g r a d i e n t s o f g\ a n d g i a t ( 1, 0 ) a s f o l l o w s:
d g = G r a d [ { g [ [ I s 1]] , g [ [ 3, 1 ] ] }, v a r s ]/.{ x ·* 1, y -* 0}
{ { 0, 1} , { 0, - 1 } }
R a n k [ d g ]
1
T h e r a n k o f t h i s m a t r i x i s 1, i n d i c a t i n g t h a t t h e g r a d i e n t v e c t o r s a r e n o t l i n e a r l y i n d e p e n d e n t a n d t h u s, t h e p o i n t i s n o t a r e g u l a r p o i n t. B y s h o w i n g t h e g r a d i e n t s o f a c t i v e c o n s t r a i n t s a t t h e m i n i m u m p o i n t, t h e f o l l o w i n g g r a p h d e m o n s t r a t e s t h a t a t a n i r r e g u l a r p o i n t, i t i s i m p o s s i b l e t o e x p r e s s t h e g r a d i ­
e n t o f t h e o b j e c t i v e f u n c t i o n a s a l i n e a r c o m b i n a t i o n o f c o n s t r a i n t g r a d i e n t s. T h e r e f o r e, a t s u c h p o i n t s t h e g r a d i e n t c o n d i t i o n c a n n o t b e s a t i s f i e d.
F I G U R E 4.1 2 G r a p h i c a l s o l u t i o n a n d g r a d i e n t s a t t h e m i n i m u m p o i n t s h o w i n g t h a t i t i s n o t a r e g u l a r p o i n t.
T h i s e x a m p l e d e m o n s t r a t e s t h a t K T c o n d i t i o n s g i v e v a l i d a n s w e r s o n l y f o r r e g u l a r p o i n t s. A n i r r e g u l a r p o i n t m a y o r m a y n o t b e o p t i m u m, b u t w e c a n n o t u s e K T c o n d i t i o n s t o d e t e r m i n e t h e s t a t u s o f s u c h a p o i n t.
4.5 S e n s it i v i t y An a l ys i s
4.5 S e n s i t i v i t y A n a l y s i s
A f t e r a n o p t i m u m s o l u t i o n i s o b t a i n e d, t h e L a g r a n g e m u l t i p l i e r s a t t h e o p ­
t i m u m p o i n t c a n h e u s e d t o d e t e r m i n e t h e e f f e c t o f s m a l l c h a n g e s i n t h e r i g h t - h a n d s i d e s ( r h s ) o f c o n s t r a i n t s o n t h e o p t i m u m s o l u t i o n. I n d i c a t i n g t h e c h a n g e i n t h e r i g h t - h a n d s i d e o f t h e r t h c o n s t r a i n t b y Δ b j, i t c a n b e s h o w n t h a t
p m
fL·,* r
i= 1 i= 1
w h e r e /* i s t h e o p t i m u m o b j e c t i v e f u n c t i o n v a l u e o f t h e o r i g i n a l p r o b l e m, a n d m a n d V i a r e t h e a s s o c i a t e d L a g r a n g e m u l t i p l i e r s.
T h i s e q u a t i o n g i v e s a r e a s o n a b l e e s t i m a t e o f t h e n e w o b j e c t i v e f u n c t i o n v a l u e a s a r e s u l t o f s m a l l c h a n g e s i n t h e r i g h t - h a n d s i d e s o f c o n s t r a i n t s. I t s h o u l d c l e a r l y b e n o t e d t h a t t h e e q u a t i o n m e r e l y g i v e s a n e s t i m a t e o f t h e n e w o b j e c t i v e f u n c t i o n v a l u e. I t d o e s n o t t e l l u s t h e n e w v a l u e s o f t h e o p t i m i z a t i o n v a r i a b l e s. T h e o n l y w a y t o f i n d t h e c o m p l e t e s o l u t i o n i s t o r e p e a t t h e s o l u t i o n p r o c e d u r e w i t h t h e m o d i f i e d c o n s t r a i n t.
I t i s a l s o i m p o r t a n t t o n o t e t h a t t h i s e q u a t i o n i s a p p l i c a b l e o n l y i f t h e c o n ­
s t r a i n t s a r e e x p r e s s e d a s t h e l e s s t h a n o r e q u a l t o t y p e. I n o r d e r t o d e t e r m i n e t h e a p p r o p r i a t e s i g n f o r t h e c h a n g e, i t i s b e s t t o w r i t e t h e o r i g i n a l a n d m o d i f i e d c o n s t r a i n t s s o t h a t t h e i r l e f t - h a n d s i d e s a r e e x a c t l y t h e s a m e. T h e a l g e b r a i c d i f f e r e n c e i n t h e r i g h t - h a n d s i d e ( r h s ) c o n s t a n t s ( c h a n g e = n e w r h s — o r i g i n a l r h s ) t h e n r e p r e s e n t s t h e c h a n g e.
F o r e x a m p l e, c o n s i d e r a p r o b l e m w i t h t h e o r i g i n a l a n d m o d i f i e d c o n s t r a i n t s a s f o l l o w s.
O r i g i n a l: x\ 4 - 2x% < —3 M o d i f i e d: x i + 2 x f + 3.1 < 0
T h e o r i g i n a l c o n s t r a i n t i s w r i t t e n i n t h e s t a n d a r d f o r m a s x\ + 2 x ^ + 3 < 0. T h e r e f o r e, w e w r i t e t h e m o d i f i e d c o n s t r a i n t a s x\ 4 - 2 x | + 3 < — 0.1. T h u s, t h e c h a n g e i s A b = — 0.1.
D e r i v a t i o n o f t h e S e n s i t i v i t y E q u a t i o n
F o r s i m p l i c i t y, c o n s i d e r t h e o r i g i n a l p r o b l e m w i t h o n l y a s i n g l e e q u a l i t y c o n ­
s t r a i n t w r i t t e n a s f o l l o w s:
F i n d x t h a t m i n i m i z e s / ( x ) s u b j e c t t o h ( x ) = 0
T h e o p t i m u m s o l u t i o n o f t h e p r o b l e m i s d e n o t e d a s f o l l o w s:
V a r i a b l e s = x * L a g r a n g e m u l t i p l i e r s = v O b j e c t i v e f u n c t i o n = /*
L(x, v ) = /( x ) + v h ( ± )
N o w s u p p o s e t h e r i g h t - h a n d s i d e o f t h e c o n s t r a i n t i s m o d i f i e d a s f o l l o w s:
M o d i f i e d c o n s t r a i n t: h ( x ) = e o r h ( ± ) — e = 0
B e c a u s e o f t h i s m o d i f i c a t i o n, o n e w o u l d e x p e c t t h e o p t i m u m s o l u t i o n t o
c h a n g e. L e t x(e) a n d v ( e ) d e n o t e h o w t h e s o l u t i o n a n d t h e L a g r a n g e m u l t i ­
p l i e r c h a n g e s a s e c h a n g e s. T h e L a g r a n g i a n f o r t h e m o d i f i e d p r o b l e m i s a s f o l l o w s:
L(x, v, e ) = /( x ) + v [ h ( n.) - e ]
A t t h e o p t i m u m, t h e c o n s t r a i n t i s s a t i s f i e d a n d t h e r e f o r e, / ( x * ( e ) ) = L ( x * ( e ), v ( e ), € ). U s i n g t h e c h a i n r u l e
d f (x*) „ - * ,r 9x d L d v d L
= V*L(x , v ) T ~ ~ + — — + — d e d e d v d e d e
N o t i n g t h a t = — v a n d f r o m t h e o p t i m a l i t y c o n d i t i o n s V x L ( x *, v ) — 0 a n d = 0, w e g e t t h e f o l l o w i n g r e l a t i o n s h i p:
9/0 0
T h e L a g r a n g i a n is w r i t t e n a s f o ll o w s:
de
= —v
T h u s, t h e L a g r a n g e m u l t i p l i e r f o r a c o n s t r a i n t r e p r e s e n t s t h e r a t e o f c h a n g e o f t h e o p t i m u m v a l u e o f t h e o b j e c t i v e f u n c t i o n w i t h r e s p e c t t o c h a n g e i n t h e r h s o f t h a t c o n s t r a i n t. I f t h e L a g r a n g e m u l t i p l i e r o f a c o n s t r a i n t h a s a l a r g e m a g n i t u d e, i t m e a n s t h a t c h a n g i n g t h a t c o n s t r a i n t w i l l h a v e a g r e a t e r i n f l u e n c e o n t h e o p t i m u m s o l u t i o n.
T r e a t i n g t h e o b j e c t i v e f u n c t i o n a s a f u n c t i o n o f t h e c h a n g e i n t h e r i g h t - h a n d s i d e, w e c a n u s e t h e T & y l o r s e r i e s t o a p p r o x i m a t e t h e n e w o b j e c t i v e f u n c t i o n v a l u e a s
/£ » * / ( * * > + = / ( * * ) - «
F o r a g e n e r a l c a s e, i n d i c a t i n g t h e c h a n g e s i n t h e r i g h t - h a n d s i d e o f t h e i t h c o n s t r a i n t b y Δ&,· ( c h a n g e = n e w r h s — o r i g i n a l r h s ), w e h a v e
p m
^ - Σ V i A b i ~ Σ U i A b i
» = i ;= i
4.5 S e n sit i vi ty An a l ys i s
T h e N e w F U s i n g S e n s i t i v i t y F u n c t i o n
T h e f o l l o w i n g f u n c t i o n, i n c l u d e d i n t h e O p t i m i z a t i o n T b o l b o x 'O p t r m a l i t y C o n - d i t i o n s' p a c k a g e, c o m p u t e s t h e n e w o b j e c t i v e f u n c t i o n v a l u e u s i n g t h e s e n s i - t i v i t y t h e o r e m.
N e e d s ["O p t i m i z a t i o n T o o l b o x'O p t i m a l i t y C o n d i t i o n s'"];
?N e w F U s i n g S e n s i t i v i t y
N e w F U s i n g S e n s i t i v i t y [ v a r s, f, s o l, g, mg] r e t u r n s t h e new v a l u e o f t h e o b j e c t i v e f u n c t i o n wh e n c o n s t a n t s i n o n e o r m o r e c o n s t r a i n t s a r e c h a n g e d, v a r s i s a l i s t o f o p t i m i z a t i o n v a r i a b l e s,f i s t h e o b j e c t i v e f u n c t i o n, s o l i s t h e l i s t o f o p t i m u m v a l u e s o f v a r i a b l e s f o l l o w e d b y t h e L a g r a n g e m u l t i p l i e r s, g i s a l i s t o f o r i g i n a l c o n s t r a i n t s, a n d mg i s a l i s t o f m o d i f i e d c o n s t r a i n t s.
E x a m p l e 4.1 5 C o n s i d e r t h e f o l l o w i n g o p t i m i z a t i o n p r o b l e m: f = - * - y 3;
g = {x + y 2 - 10 == 0, - x + 2 ϋ 0,y - 5 S 0 }; v a r s = { x, y} ;
( a ) S o l u t i o n o f t h e o r i g i n a l p r o b l e m A l l K T p o i n t s c a n r e a d i l y b e c o m ­
p u t e d u s i n g t h e K T S o l u t i o n a s f o l l o w s:
s o l n = K T S o l u t i o n [ f / g, v a r s ];
-3 2 2 2
***** L a g r a n g i a n - > - x - y + (2 - x + s i ) u ^ + ( - 5 + y + S 2) u 2 + { - 1 0 + x + ) v i
- 1 - uj_ + vj_ == 0
- 3 y 2 + u 2 + 2 y v i == 0 2 - x + s f = = 0
VL— 0 -> ------------- 5 + y + S 2 == 0
- 1 0 + χ + Y2 == 0
2s^U]_ == 0
2 s 2 u 2 — — 0
***** v a l i d KT P o i n t ( s ) *****
f - > - 2 4.6 2 7 4
f - > - 9.8 5 1 8 5
f - > - 1 0.
x -> 2 ■
x -» 9.5 5 5 5 6 y -> 0. 6 66 6 6 7
u i -> 0
x -> 1 0.
y -» 0
y - »2 .8 2 8 4 3 u -l - > 3.2 4 2 6 4 u 2 0
u 2 -> 0
s i - > 7.5 5 5 5 6 sl ->4.33333 v ^ —> 1.
s | -> 2 .1 7 1 5 7 Vi -> 4 .2 4 2 6 4
v^ -> 1.
T h e o b j e c t i v e f u n c t i o n a t t h e f i r s t K T p o i n t i s t h e l o w e s t a n d t h e r e f o r e i s a c c e p t e d a s t h e o p t i m u m s o l u t i o n. T h i s s o l u t i o n i s v e r i f i e d g r a p h i c a l l y a s f o l l o w s:
C h ap te r 4 O p t im a l it y Conditions
F I G U R E 4.1 3 G r a p h i c a l s o l u t i o n o f t h e o r i g i n a l p r o b l e m.
( T b ) S o l u t i o n o f a m o d i f i e d p r o b l e m N o w c o n s i d e r t h e s o l u t i o n o f t h e p r o b l e m i f t h e c o n s t a n t 1 0 i n t h e f i r s t c o n s t r a i n t i s c h a n g e d t o 1 1. T h e o t h e r c o n s t r a i n t s r e m a i n t h e s a m e. T h u s, t h e m o d i f i e d l i s t o f c o n s t r a i n t s i s a s f o l l o w s:
mg = { x + y 2 - 1 1 ^ = o,- x + 2 ί 0,y - 5 i 0>; s o l = { 2., 2.8 2 8 4,4.2 4 2 6 4,3.2 4 2 6 4, 0} ;
U s i n g t h e s e n s i t i v i t y t h e o r e m, t h e n e w v a l u e o f t h e o b j e c t i v e f u n c t i o n i s
c o m p u t e d a s f o l l o w s:
N e w F U s i n g S e n s i t i v i t y [ v a r s, £, e o l, g, mg] ;
- 1 0 + x + y 2 = = 0'
O r i g i n a l c o n s t r a i n t s - »
M o d i f i e d c o n s t r a i n t s -»
2 - x < 0 - 5 + y i 0
f - 1 1 -t-x + y 2 == 0^ 2 - χ ί 0 - 5 -t- y ^ 0
A b - »
O r i g i n a l f -» - 2 4.6 2 6 7 N e w f - > - 2 8.8 6 9 4
4.5 S e n s it i v i ty An al y s i s
I b v e r i f y t h i s s o l u t i o n, t h e m o d i f i e d p r o b l e m i s s o l v e d a g a i n u s i n g t h e K T ­
S o l u t i o n, a s f o l l o w s:
C T B o l n t i o n r f » mg, v a r s ];
-a 2 2 9
L a g r a n g i a n -» - x - y + ( 2 - x + s i ) u i + > ( - 5 + y + S 2 ) u 2 + ( - l l + x + y ) v i
—1 — u 1 + v i == 0 - 3 y 2 + u 2 + 2 y v ^ = = 0
2 - x + s i == 0 - 5 + y + s 2 = = 0 - 1 1 + x + y 2 = = 0
2 s ] _ U i == 0
2 s 2u 2 == 0
* * * * * V a l i d KT P o i n t ( s ) * * * * *
f -» - 2 9 . f -> - 1 0 . 8 5 1 9 f - > - l l.
x - > 2. x - » 1 0.5 5 5 6 x - » l l.
y - » 3. u i -»■ 3 .5 u 5 -* 0 s i - » 0
s 2 ^ 2.
y -> 0.6 6 6 6 6 7
u i -» 0 u 2 -»■ 0
s i -» 8 .5 5 5 5 6 S2 -> 4.3 3 3 3 3
V i -> 4 . 5 V i -> 1,
U i -*■ 0
u 9 -> 0
5 1 - » 9.
5 2 - * 5.
v i -> 1.
T h e a p p r o x i m a t e o p t i m u m v a l u e o f — 2 8.8 7 c o m p a r e s w e l l w i t h t h e e x a c t n e w o p t i m u m v a l u e o f — 2 9.
( c ) S o l u t i o n o f a n o t h e r m o d i f i e d p r o b l e m A s a n o t h e r e x a m p l e, c o n s i d e r t h e s o l u t i o n o f t h e p r o b l e m i f t h e c o n s t a n t s i n a l l t h r e e c o n s t r a i n t s a r e c h a n g e d, a s f o l l o w s:
m g = { x + y 2 - 9.5 = = 0, - x + 2.5 ί 0,y - 4.5 i 0 };
U s i n g t h e s e n s i t i v i t y t h e o r e m, t h e n e w v a l u e o f t h e o b j e c t i v e f u n c t i o n i s c o m p u t e d a s f o l l o w s:
N e w F U s i n g S e n s i t i v i t y [ v a r s, £, s o l, g, m g ];
- 1 0 + x + y 2 = = O'
O r i g i n a l c o n s t r a i n t s -» 2 - χ ύ 0
- 5 + y < 0
'- 9.5 + x + y 2 == O'
M o d i f i e d c o n s t r a i n t s -» 2 .5 - χ ί 0
- 4.5 + y < 0
Ab
0.5 0.5 1 - 0.5/
Chapter 4 O p t im a l it y Conditions
O r i g i n a l f - » - 2 4.6 2 6 8 New f - > - 2 0.8 8 4 1
I b v e r i f y t h i s s o l u t i o n, t h e m o d i f i e d p r o b l e m i s s o l v e d a g a i n u s i n g t h e K T S o ­
l u t i o n, a s f o l l o w s:
K T S o l u t i o n [ £, mff, v a r s ];
L a g r a n g i a n -» - x - y 3 + (2 .5 - x +- s i ) U i + ( - 4.5 + y + S 2) u 2 + ( - 9.5 +-x + y 2 ) Vi
- 1 - u j + Vj == 0
- 3 y 2 + u 2 + 2yv-L == 0
2.5 - x + s i = = 0 - 4.5 + y + s | == 0 - 9 .5 + x + y 2 = = 0 2 3 ] ^! == 0 2 s 2u 2 == 0
VL = 0 -»
* **** V a l i d
KT P o i n t ( s ) *****
f -> - 2 1.0 2 0 3
f -> - 9.3 5 1 8 5
f -» 9.5
x -» 2 .5
x- > 9.0 5 5 5 6
x ^ 9.5
y - > 2.6 4 5 7 5
y - > 0.6 6 6 6 6 7
y - » 0
u^ -» 2 .9 6 8 6 3
u^ -> 0
Ui -> 0
u 2 -> 0
n 2 -» 0
u 5 -» 0
s i -» 0
si.-» 6.5 5 5 5 6
2 - s i - »7.
S2 - > 1.8 5 4 2 4
s i - »3.8 3 3 3 3
s l - > 4.5
V! - > 3.9 6 8 6 3
v j -» 1.
Vi -> 1.
A g a i n, t h e a p p r o x i m a t e o p t i m u m v a l u e o f — 2 0.8 8 c o m p a r e s w e l l w i t h t h e e x a c t n e w o p t i m u m v a l u e o f —2 1.0 2.
E x a m p l e 4.1 6 B u i l d i n g d e s i g n C o n s i d e r t h e b u i l d i n g d e s i g n p r o b l e m p r e ­
s e n t e d i n E x a m p l e 4.1 1. T h e f o l l o w i n g o p t i m u m s o l u t i o n w a s o b t a i n e d u s i n g t h e K T c o n d i t i o n s.
d * = 8 0.0 3 m h * = 1 3.3 1 m w * = 2 1.5 3 m /* = 6 0,0 2 0.5 m 3
T h e c o n s t r a i n t s ( i n s t a n d a r d f o r m ) a n d a s s o c i a t e d L a g r a n g e m u l t i p l i e r s w e r e a s f o l l o w s:
C o n s t r a i n t
L a g r a n g e m u l t i p l i e r
E n e r g y b u d g e t
1 0 0 ( 5.2 3 6 0 0/i w + 1.6 1 8 0 0 w 2 ) - 2 2 5,0 0 0 < 0
0.0 6 6 5 3 0 4 1
P l a n d i m e n s i o n
1.6 1 8 0 0 w < 5 0
0
F l o o r s p a c e
—0.4 6 2 2 8 5 7 (<f + h ) w 2 + 2 0,0 0 0 < 0
3.5 0 0 0 0
P r a c t i c a l d i m e n s i o n s
d > 0, h > 0, a n d w > 0
0
U s i n g s e n s i t i v i t y a n a l y s i s, w e s t u d y t h e e f f e c t o f t h e f o l l o w i n g t w o c h a n g e s o n t h e o p t i m u m s o l u t i o n.
( a ) T h e e n e r g y b u d g e t i s i n c r e a s e d t o $ 2 5 0,0 0 0. I n t h i s c a s e, t h e f i r s t c o n s t r a i n t i s m o d i f i e d a s f o l l o w s:
1 0 0 ( 5.2 3 6 0 0/n v + 1.6 1 8 0 0 w 2 ) - 2 5 0,0 0 0 < 0
M a k i n g t h e l e f t - h a n d s i d e i d e n t i c a l t o t h e o r i g i n a l c o n s t r a i n t
1 0 0 ( 5.2 3 6 0 0/j w + 1.6 1 8 0 0 w 2 ) - 2 2 5,0 0 0 < 2 5,0 0 0
w e s e e t h a t t h e c h a n g e i s A b i = 2 5,0 0 0. A s s u m i n g t h i s c h a n g e i s s m a l l, t h e n e w v a l u e o f t h e o b j e c t i v e f u n c t i o n c a n b e e s t i m a t e d a s f o l l o w s:
N e w f * = O l d f * - u j A b j = 6 0,0 2 0.5 - 0.0 6 6 5 3 0 4 1 x 2 5,0 0 0 = 5 8,3 5 7.2
B y s o l v i n g t h e p r o b l e m w i t h t h e m o d i f i e d f i r s t c o n s t r a i n t a l l o v e r a g a i n u s i n g t h e K T S o l u t i o n, w e g e t t h e e x a c t/* = 5 8,3 1 1.8 i n d i c a t i n g t h a t t h e a p p r o x i m a t e v a l u e u s i n g s e n s i t i v i t y a n a l y s i s i s f a i r l y g o o d.
( b ) T h e t o t a l f l o o r s p a c e r e q u i r e m e n t i s r e d u c e d t o 1 9,5 0 0 m 2. I n t h i s c a s e, t h e t h i r d c o n s t r a i n t i s m o d i f i e d a s f o l l o w s:
— 0.4 6 2 2 8 5 7 0 * + h ) w 2 + 1 9,5 0 0 < 0
M a k i n g t h e l e f t - h a n d s i d e i d e n t i c a l t o t h e o r i g i n a l c o n s t r a i n t
— 0.4 6 2 2 8 5 7 ( < i + h ) w 2 + 2 0,0 0 0 < 5 0 0
w e s e e t h a t t h e c h a n g e i s A b s = 5 0 0. A s s u m i n g t h i s c h a n g e i s s m a l l, t h e n e w v a l u e o f t h e o b j e c t i v e f u n c t i o n c a n b e e s t i m a t e d a s f o l l o w s:
N e w f * = O l d f * - u 3A b 3 = 6 0,0 2 0.5 - 3.5 x 5 0 0 = 5 8,2 7 0.5
B y s o l v i n g t h e p r o b l e m w i t h t h e m o d i f i e d t h i r d c o n s t r a i n t a l l o v e r a g a i n u s i n g t h e K T S o l u t i o n, w e g e t t h e e x a c t f * = 5 8,2 7 0.5, w h i c h i s t h e s a m e a s t h e o n e c o m p u t e d b y s e n s i t i v i t y a n a l y s i s.
4.6 O p t i m a l i t y C o n d i t i o n s f o r C o n v e x P r o b l e m s
A s d i s c u s s e d i n C h a p t e r 3, a n o p t i m i z a t i o n p r o b l e m i s c o n v e x i f t h e r e a r e n o n o n l i n e a r e q u a l i t y c o n s t r a i n t s a n d t h e o b j e c t i v e f u n c t i o n a n d a l l c o n s t r a i n t
__________________________ 4.6 Op t i m a l i t y Con ditions f o r Convex Problems
C h ap te r 4 Op t im al it y Conditions
f u n c t i o n s a r e c o n v e x ( i.e. t h e i r H e s s i a n m a t r i c e s a r e a t l e a s t p o s i t i v e s e m i ­
d e f i n i t e ). F o r c o n v e x p r o b l e m s, a n y l o c a l m i n i m u m i s a l s o a g l o b a l m i n i m u m. T h e r e f o r e, a p o i n t t h a t s a t i s f i e s K T c o n d i t i o n s i s a g l o b a l m i n i m u m p o i n t f o r t h e s e p r o b l e m s. T h u s, f o r c o n v e x p r o b l e m s, t h e K T c o n d i t i o n s a r e n e c e s s a r y a s w e l l a s s u f f i c i e n t f o r t h e o p t i m u m.
E x a m p l e 4.1 7 C o n v e x c a s e C o n s i d e r t h e f o l l o w i n g o p t i m i z a t i o n p r o b l e m:
f = ( x - 2) 2 + (y - 3 ) 2 f g = ( x - 4 ) 2 + ( y - 5 ) 2 - 6 ^ 0; v a r s = { m, y };
A s t h e f o l l o w i n g c o m p u t a t i o n s s h o w, t h i s i s a c o n v e x p r o g r a m m i n g p r o b l e m.
C o n v e x i t y C h e c k [ { f, g }, v a r s ];
-------------- F u n c t i o n - » ( - 2 + x ) 2 + ( - 3 + y ) 2
/2 θ\..Ί H e s s i a n - » L · 2 P r i n c i p a l M i n o r s - » I
S t a t u s - » C o n v e x
-------------- F u n c t i o n -» - 6 + ( - 4 + x ) + ( - 5 + y )
12 θ\ . . 1 2\
H e s s i a n - » L· ^ P r i n c i p a l M i n o r s - » I I
S t a t u s - » C o n v e x
T h e K T S o l u t i o n c a n b e o b t a i n e d a s f o l l o w s:
K T S o l u t i o n [ f, g, v a r s ];
M i n i m i z e f -» ( - 2 + x ) 2 + ( - 3 + y ) 2
. /- 4 + 2 x
V f ->
- 6 + 2 y
* * * * * L E c o n s t r a i n t s & t h e i r g r a d i e n t s 9 1 -» “ 6 + ( - 4 + x ) 2 + (-5 + y ) 2 < 0 v g i -» | '^ ^ γ
***** Lagrangian -» (-2 + x)2 + (-3 +y) 2 -t- (-6 + (-4 + x)2 + (-5 -t-y)2 + s i ) ^ -4 + 2x - 8u! -t- 2xu! == 0
VL = 0 -»
-6 + 2y - 10ui + 2yUi == 0 -6 + (-4 + x)2 +■ (-5 +y)2 -t- sf == 0
2 si Ui == 0
***** Valid KT Point(s) ***** f -» 0.143594
x-»2.26795 y 3 .26795 ui -» 0.154701
si -» 0
4.6 QprimaHty Conditions f o r Convex Problems
A s e x p e c t e d, t h e r e i s o n l y o n e p o i n t t h a t s a t i s f i e s a l l K T c o n d i t i o n s a n d t h u s i s a g l o b a l m i n i m u m. T h e f o l l o w i n g g r a p h i c a l s o l u t i o n c o n f i r m s t h i s p o i n t.
F I G U R E 4.1 4 G r a p h i c a l s o l u t i o n.
E x a m p l e 4.1 8 N o n c o n v e x c a s e I f t h e c o n s t r a i n t i n t h e p r e v i o u s e x a m p l e i s c h a n g e d t o a n e q u a l i t y, t h e p r o b l e m b e c o m e s n o n - c o n v e x ( s i n c e n o w w e h a v e a n o n l i n e a r e q u a l i t y c o n s t r a i n t ).
t = l x - 2 ) 2 + ( v - 3 ) 2:
h = ( x - 4 ) 2 + ( y - 5) 2 - 6 == 0; v a r s = {χ, y } ;
S o l v i n g t h i s p r o b l e m u s i n g K T c o n d i t i o n s p r o d u c e s t w o v a l i d K T p o i n t s.
K T S o l u t i o n [ £, h, v a r s ];
M i n i m i z e f -» ( - 2 + x ) 2 + ( - 3 + y ) 2
v f
- 4 + 2 x - 6 + 2 y
BQ c o n s t r a i n t s & t h e i r g r a d i e n t s
- » - 6 + ( - 4 + x) + ( - 5 + y) = = 0
- 8 + 2 x - 1 0 + 2 y
***** L a g r a n g i a n -» ( - 2 + x ) 2 + ( - 3 + y) 2 + ( - 6 + (-4 + x ) 2 + ( - 5 + y ) 2 ) v i
Chap ter 4 O p t im a l it y Conditions
VL = 0 -»
- 4 + 2x - 8V! -t- 2xv^ == 0 - 6 + 2 y - 1 0 v i + 2 y v ^ == 0 -6 + ( - 4 + x ) 2 + (-5 + y ) 2 = = 0
* * * * * v a l i d KT P o i n t ( s ) * * * * *
f - » 0.1 4 3 5 9 4 f -» 2 7. 8 5 6 4
x - * 2.2 6 7 9 5 x - > 5.7 3 2 0 5
y -* 3 .2 6 7 9 5 y - » 6.7 3 2 0 5
V l -> 0.1 5 4 7 0 1 V i - > - 2.1 5 4 7
A s s e e n f r o m t h e f o l l o w i n g g r a p h i c a l s o l u t i o n, o n e o f t h e s e p o i n t s c o r r e s p o n d s t o t h e g l o b a l m i n i m u m, w h i l e t h e o t h e r i s i n f a c t t h e g l o b a l m a x i m u m.
0 1 2 3 4 5 6 7
F I G U R E 4.1 5 G r a p h i c a l s o l u t i o n.
E x a m p l e 4.1 9 C o n s i d e r t h e f o l l o w i n g o p t i m i z a t i o n p r o b l e m:
f = x i + 2x 2 + 2x 3 ;
2 2 2 c o n = { x j + x 2 + X3 £ i, x i - x 2 + 2x 3 £ 2 } ;
v a r s = { x j, x 2/ x 3 >;
T h e p r o b l e m i s c o n v e x a s s e e n f r o m t h e f o l l o w i n g c o m p u t a t i o n s: C o n v e x i t y C h e c k [ { f , c o n } , v a r s ];
4
--------------F u n c t i o n - » x i + 2 x 2 + 2 x 3
4.6 OptitnaTity Con dit ions f o r Convex Problems
1 2 x i
0
°)
1 2 x i
H e s s i a n - »
0
0
0
P r i n c i p a l M i n o r s -*
0
, 0
0
0,
0
S t a t u s -» C o n v e x
2 2 F u n c t i o n - » - 4 + x i -t- x 2 + X3
He s s i a n ->
/2 0 0 0
O'!
J i
(0 0 S t a t u s -» C o n v e x
P r i n c i p a l M i n o r s
0
F u n c t i o n -> - 2 + x i - x 2 + 2 x 3
2
0
O'
2'
H e s s i a n
0
0
0
p r i n c i p a l M i n o r s -»
0
.0
0
.0,
S t a t u s -* C o n v e x
Th e KT s o l u t i o n i s as f ol l ows:
K T S o l u t i o n [ f, c o n, v a r s ];
4
M i n i m i z e f -* x i + 2 x 2 + 2x 3
( a 3\
4 x ^
V f
2
2
* * * * * L E c o n s t r a i n t s & t h e i r g r a d i e n t s
2 2 2 g i -» - 4 + x i + x 2 + X3 < 0 g 2 -» - 2 + x i - x 2 + 2 x 3 ί 0
’2 x i'
2 x i'
V g i —♦
1
v g 2 - »
- 1
,2x 3,
, 2 ,
4 2 2 2 2 2
L a g r a n g i a n -* x i + 2 x 2 + 2x 3 + u j (~2 + s 2 + * i - x 2 + 2 x 3 ) + Ui ( - 4 + s i + χ ι + x 2 +X 3)
2uiXi + 2u2Xi + 4xi = = 0
2 + ui - u2 ==0
2 + 2u2 + 211^X3 - — 0
VL = 0 -*
- 4 + s i + X l 4- x 2 + X3 = = 0 2 2
- 2 + Ξ2 + x i - x 2 + 2x 3 = = 0
2 s i U! = = 0
2 s 2u2 == 0
* * * * * v a l i d KT P o i n t ( s ) * * * * * f - 2 5.8 7 4 5
X l -> 0
x 2 -» - 9.2 9 1 5 x 3 -» - 3.6 4 5 7 5 u i - » 1.1 3 3 8 9 u 2 -» 3.1 3 3 8 9
s i 0
Ch ap te r 4 Op t im al it y Conditions
A s e x p e c t e d, t h e r e i s o n l y o n e p o i n t t h a t s a t i s f i e s a l l K T c o n d i t i o n s a n d t h u s, i s a g l o b a l m i n i m u m.
I f t h e o b j e c t i v e f u n c t i o n w e r e a c t u a l l y t h e n e g a t i v e o f t h e o n e g i v e n, t h e p r o b l e m b e c o m e s n o n c o n v e x. S e v e r a l d i f f e r e n t p o i n t s s a t i s f y t h e K T c o n d i t i o n s i n t h i s c a s e.
C o n v e x!t y C h e c k [ - £, v a r s j ;
F u n c t i o n -> - χ χ - 2 x 2 - 2 x 3
- 12x 1
υ
o j
f- 1 2 x i
H e s s i a n ->
0
0
0
P r i n c i p a l M i n o r s ->
0
0
0
0
i 0
K T S o l u t i o n [ - f, c o n, v a r s ];
M i n i m i z e f -> - χ χ - 2 x 2 - 2 x 3
V f ->
. 3 - 4 x j
- 2
- 2
* * * * * L E c o n s t r a i n t s & t h e i r g r a d i e n t s
2 2 2 -> - 4 + x i t X; t X3 i 0 g 2 -» - 2 -t- x i — x 2 ■» 2x 3 £ 0
2xj
l2xA
Vgi ->
1
Vg2 ->
-1
2x3.
, 2 ,
4 2 2 2 2 2
* * * * * L a g r a n g i a n -> - x i - 2 x 2 - 2 x 3 + u 2 ( - 2 + S 2 + x i - x 2 + 2 x 3 ) + U i ( - 4 4- s i + χ ι + x 2 + X 3)
VL = 0 ->
2 u 1 x 1 4- 2 u 2 X x - 4 x i == 0 - 2 + u i - u 2 == 0 - 2 -t- 2 u 2 -t- 2 u j X 3 = = 0
2 2 2 - 4 4- S l + X l 4- x 2 4- X3 = = 0
2 2
- 2 + S 2 + X l - X 2 + 2 x 3 == 0 2 == 0 ----------------2S 2U2 == 0----------------
* * * * * V a l i d KT
P o i n t ( s )
f - > - 1 1.3 0 5 1
f -> - 7.5
f - > - 8.5
H\
1
-j
Ul
f -> - 1 1.3 0 5 1
x i -* - 1.8 0 3 0 5
X l -> - 1.
Xl -> 0
X l -> 1.
X l - > 1.8 0 3 0 5
x 2 -> 0.6 6 2 4 1 9
x 2 - > 2.7 5
x 2 -* 3 .7 5
X2 - > 2.7 5
x 2 -> 0.6 6 2 4 1 9
x 3 -> - 0.2 9 4 2 8
x 3 -> 0.5
x 3 - > 0.5
X3 -> 0.5
X3 -> - 0.2 9 4 2 8
u i -> 4.2 5 0 9 8
u i -> 2.
u i -> 2.
Ui -> 2.
u i -> 4.2 5 0 9 8
u 2 —> 2.2 5 0 9 8
u 2 -> 0
u 2 -* 0
u 2 -> 0
u 2 -> 2.2 5 0 9 8
s i -> 0
s i -> 0
s i -» 0
s i -> 0
s i -> 0
s i -> 0
s i -> 2.7 5
s i -> 4.7 5
s i - > 2.7 5
s | - > 0
4.7 Second-Order Su f fi c i e n t Conditions
4.7 S e c o n d - O r d e r S u f f i c i e n t C o n d i t i o n s
T h e K T c o n d i t i o n s, e x c e p t f o r c o n v e x p r o b l e m s, a r e o n l y n e c e s s a r y c o n d i t i o n s f o r t h e m i n i m u m o f a c o n s t r a i n e d o p t i m i z a t i o n p r o b l e m. S i n c e t h e L a g r a n g i a n f u n c t i o n i s e s s e n t i a l l y u n c o n s t r a i n e d, i t i s e a s y t o s e e t h a t t h e s u f f i c i e n t c o n ­
d i t i o n t h a t a g i v e n K T p o i n t i x *. u *. v * l i s a c t u a l l y a l o c a l m i n i m u m i s t h a t
d > 0
w h e r e V 2 s y m b o l s t a n d s f o r t h e H e s s i a n o f a f u n c t i o n. N o t e t h a t o n l y a c t i v e i n e q u a l i t i e s a r e c o n s i d e r e d, a n d t h e g r a d i e n t s a n d H e s s i a n s a r e e v a l u a t e d a t a k n o w n K T p o i n t.
I n t h e u n c o n s t r a i n e d c a s e, t h e c h a n g e s d w e r e a r b i t r a r y. T h i s r e s u l t e d i n t h e c o n d i t i o n t h a t H e s s i a n m u s t b e p o s i t i v e d e f i n i t e. F o r a c o n s t r a i n e d p r o b l e m, w e m u s t c o n s i d e r o n l y t h o s e c h a n g e s t h a t d o n o t v i o l a t e c o n s t r a i n t s. T h u s, t h e f e a s i b l e c h a n g e s d m u s t s a t i s f y t h e f o l l o w i n g l i n e a r i z e d v e r s i o n s o f t h e c o n s t r a i n t s.
V g,( x * ) T d = 0, i e A c t i v e . V 7 t i ( x * ) T d = 0, i = 1,..., p
B e c a u s e o f t h e n e e d t o f i r s t d e t e r m i n e f e a s i b l e c h a n g e s, c h e c k i n g s u f f i c i e n t c o n d i t i o n s f o r a g e n e r a l c a s e i s q u i t e t e d i o u s. F o r a s p e c i a l c a s e w h e n
P
V2/( x * ) + Σ
+ 2 > ν2ί,·'<χ *>
ieActive i—1
i s a p o s i t i v e d e f i n i t e m a t r i x, t h e n t h e c o n d i t i o n i s t r u e f o r a n y d a n d h e n c e, t h e r e i s n o n e e d t o c o m p u t e f e a s i b l e c h a n g e s.
E x a m p l e 4.2 0 C o n s i d e r t h e f o l l o w i n g o p t i m i z a t i o n p r o b l e m:
£ = - x 2 + y; h = - x 2 - y 2 + 1 = = 0 ? v a r s = { x, y } ;
T h e p o i n t s s a t i s f y i n g t h e K T c o n d i t i o n s a r e a s f o l l o w s:
s o l n = K T S o l u t i o n [ £, h, v a r s ];
M i n i m i z e f -*■ - x 2 + y
Ch ap te r 4 Op t im al it y Conditions
* **** e q c o n s t r a i n t s & t h e i r g r a d i e n t s hx -* 1 - x 2 - y 2 = = 0 vhx -*· ί
***** Lagrangian -x 2 + y+ ( l - x 2 - y 2)v1
VL = 0 -»
2x - 2xv1 == 0' 1 - 2yvx == 0
U - χ 2 -y2 ==0,
***** v a l i d KT P o i n t (s ) *****
f -*-1.25 f - » - l. f - * l. f-» -1.2 5
x - 4 -0.8660255 x 0 x-»0 x-» 0.866025
y -»-0.5 y - » - l. y - » l · y -* -0.5
v ^ - 1. V j -* - 0.5 v i -» 0.5 V i -4 - 1.
F o r t h i s p r o b l e m, t h e s u f f i c i e n t c o n d i t i o n f o r x * t o b e m i n i m u m i s t h a t
d T [ V 2/( x * ) + i/V 2/i ( x * ) ] d > 0 w h e r e t h e f e a s i b l e c h a n g e s d m u s t s a t i s f y t h e f o l l o w i n g e q u a t i o n:
V h ( x * ) T d = 0 T h e H e s s i a n m a t r i c e s a r e a s f o l l o w s:
:) m? -,)
( i ) S e c o n d - o r d e r c o n d i t i o n c h e c k a t p o i n t 1
x = - 0.8 6 6 0 2 5 y = - 0.5 v = - 1 —
T h u s, t h e f e a s i b l e c h a n g e s m u s t s a t i s f y t h e f o l l o w i n g e q u a t i o n:
1.7 3 2 0 5 t i i + d 2 = 0
S o l v i n g t h i s e q u a t i o n, w e g e t d\ = —0.5 7 7 3 5 ^ 2. T h e v e c t o r o f f e a s i b l e c h a n g e s i s
d T = ( - 0.5 7 7 3 5 ^ 2 d i ) w i t h d i a r b i t r a r y
4.7 Second-Order Suffi ci e nt Conditions
S u b s t i t u t i n g i n t o t h e e x p r e s s i o n f o r s u f f i c i e n t c o n d i t i o n s, w e h a v e
'—0.5 7 7 3 5 d 2\
( — 0.5 7 7 3 5 d 2 d 2 )
T h i s e x p r e s s i o n e v a l u a t e s t o 2d\. T h i s i s g r e a t e r t h a n 0 f o r a n y d 2 φ 0. T h u s, t h e s u f f i c i e n t c o n d i t i o n i s s a t i s f i e d a t t h i s p o i n t a n d t h e r e f o r e, i t i s a l o c a l m i n i m u m.
a t p o i n t 2
x = 0 y = —1 v = — 0.5
V h
T h u s, t h e f e a s i b l e c h a n g e s m u s t s a t i s f y t h e f o l l o w i n g e q u a t i o n:
2 d 2 = 0
T h e v e c t o r o f f e a s i b l e c h a n g e s i s
d r = ( d i 0 ) w i t h d i a r b i t r a r y S u b s t i t u t i n g i n t o t h e e x p r e s s i o n f o r s u f f i c i e n t c o n d i t i o n s, w e h a v e
( d i 0 )
T h i s e x p r e s s i o n e v a l u a t e s t o — d f. T h i s i s n e v e r g r e a t e r t h a n 0. T h u s, t h e s u f f i c i e n t c o n d i t i o n i s n o t s a t i s f i e d a t t h i s p o i n t a n d t h e r e f o r e, i t i s n o t a l o c a l m i n i m u m.
( i i i ) S e c o n d - o r d e r c o n d i t i o n c h e c k a t p o i n t 3
x = 0 y = — 1 v — 0.5
V h
T h u s, t h e f e a s i b l e c h a n g e s m u s t s a t i s f y t h e f o l l o w i n g e q u a t i o n:
Chapter 4 Op t im al it y Conditions
T h e v e c t o r o f f e a s i b l e c h a n g e s i s
d T = ( d\ 0 ) w i t h d i a r b i t r a r y
S u b s t i t u t i n g i n t o t h e e x p r e s s i o n f o r s u f f i c i e n t c o n d i t i o n s, w e h a v e
( d i 0 )
T h i s e x p r e s s i o n e v a l u a t e s t o — 3 d f. T h i s i s n e v e r g r e a t e r t h a n 0. T h u s, t h e s u f f i c i e n t c o n d i t i o n i s n o t s a t i s f i e d a t t h i s p o i n t a n d t h e r e f o r e, i t i s n o t a l o c a l m i n i m u m.
( i v ) S e c o n d - o r d e r c o n d i t i o n c h e c k a t p o i n t 4
x = 0.8 6 6 0 2 5 y = — 0.5 v = — 1
T h u s, t h e f e a s i b l e c h a n g e s m u s t s a t i s f y t h e f o l l o w i n g e q u a t i o n:
— 1.7 3 2 0 5 d i + d 2 = 0 S o l v i n g t h i s e q u a t i o n, w e g e t d\ = 0.5 7 7 3 5 ^. T h e v e c t o r o f f e a s i b l e c h a n g e s i s d T — ( 0.5 7 7 3 5 ^ 2 d 2 ) w i t h d 2 a r b i t r a r y S u b s t i t u t i n g i n t o t h e e x p r e s s i o n f o r s u f f i c i e n t c o n d i t i o n s, w e h a v e
T h i s e x p r e s s i o n e v a l u a t e s t o 2 d $. T h i s i s g r e a t e r t h a n 0 f o r a n y d 2 φ 0. T h u s, t h e s u f f i c i e n t c o n d i t i o n i s s a t i s f i e d a t t h i s p o i n t a n d t h e r e f o r e, i t i s a l o c a l m i n i m u m.
V h
^ - 2 x ^ _ ^ - 1.7 3 2 0 5 ^
( v ) G r a p h i c a l s o l u t i o n A l l f o u r K T p o i n t s a r e s h o w n o n t h e f o l l o w i n g g r a p h. I t i s c l e a r f r o m t h e g r a p h t h a t t h e p o i n t s 1 a n d 4 a r e a c t u a l l y t h e m i n i m u m p o i n t s, c o n f i r m i n g t h e c o n c l u s i o n s d r a w n f r o m t h e s u f f i c i e n t c o n d i t i o n s.
4.7 Second-Order Suf fi ci e nt Conditions
FIGURE 4.1 6 G r a p h i c a l s o l u t i o n. E x a m p l e 4.2 1 C o n s i d e r t h e f o l l o w i n g o p t i m i z a t i o n p r o b l e m:
3 2
f = xi + 2x2x 3 + 2x3 ;
, 2 2 . n = x i + x 2 + * 3 = = »; 2
g = χχ - x 2 + 2x3 ί 2 ; var s = { x x , x 2 , x 3 };
T h e p o i n t s s a t i s f y i n g t h e K T c o n d i t i o n s a r e a s f o l l o w s:
s o l n = K T S o l u t i o n [ f, { g, h >, v a r s ];
3 2
M i n i m i z e f -» x i + 2 x 3 + 2 x 2 X 3
Vf ->
3xi
4 x 2 x 3
12 + 2 X 2
* * *** le c o n s t r a i n t s & t h e i r gr adi ent s
g i - 2 + x i - x2 + 2x3 < 0 Vgi -*
2 x i'l
- 1
2
* * * * * eq c o n s t r a i n t s & t h e i r gr adi ent s
C h ap te r 4 O p t im al it y Conditions
2 2 h j -* - 4 + χ χ + x 2 + X3 == 0 Vh;L -*
2 χ χ
1
2 x j
3 2 2 2 2 2
* * * * * L a g r a n g i a n ~ * χ χ + 2x 3 + 2x 2x 3 '1'u l f “ 2 + Sj + χ χ - x 2 + 2 x 3 ) + ν χ ( - 4 + χ χ + χ 2 + X3 )
2^ X 31 + 2v 1x 1 + 3 x i == 0 - U j + Vj + 4 x 2x 3 == 0
2 +
VL = 0 -»
+ 2 ν!Χ3 == Q
2 2
-2 + S l + X l - X2 + 2 x 3 == 0
2 2
-4 + χχ + X 2 + X 3 == 0
2 3 ] ^ == 0
***** V a lid KT P oin t(s) ***** £ —> —6*74974 f - 3.9 6 4 4 7
X! - » - 1.6 2 4 1 3 x 2 - * 0.8 5 6 1 6 8 x 3 - 4 - 0.7 1 1 3 6 5
U j ~ * 0
s i - * 1. 6 4 1 1 1 V } ~ * 2.4 3 6 1 9
X l - » - 0.3 4 2 7 1 8 x 2 - * 0 .0 6 5 7 8 4 x 3 -4 - 1.9 5 3 6 5
U j - » 0
S x - 4 5.8 5 5 6 3 v x ~ * 0.5 1 4 0 7 7
£ - » 8.7 8 1 6 9 X l - * - 0.0 3 7 2 2 1 7 x 2 - * 1.2 9 1 8 4 x 3 - * 1.6 4 5 2 3 u x - * 4.2 7 8 6 6
s f - * 0
£ ~ * - 6 3 6 .7 8 2
X j ^ 0
x 2 - * - 9.2 9 1 5 x 3 - » - 3 . 6 4 5 7 5 - » 2 1 9.7 2 s f ~* 0
-4 —4 « 2 2 2 8 3 -4 8 4 * 2 2 1 8
F o r t h i s p r o b l e m, t h e s u f f i c i e n t c o n d i t i o n f o r x * t o b e m i n i m u m i s t h a t d T [ V 2/( x * ) + v V 2h ( x * ) + u V 2g ( x * ) ] d > 0 w h e r e t h e f e a s i b l e c h a n g e s d m u s t s a t i s f y t h e f o l l o w i n g e q u a t i o n s:
V h ( x * ) T d = 0 V g ( x * ) r d = 0 T h e H e s s i a n m a t r i c e s a r e a s f o l l o w s:
f 6 x i
0
0
0\
2
0
0
4 x 3
4X2
V 2 ft = 0
0
0
4X2
0/
V o
0
2
'2 0 0> V2# = [ ο ο o
l0 0 0;
( a ) S e c o n d - o r d e r c o n d i t i o n c h e c k a t p o i n t 1
X l = - 1.6 2 4 1 3 x 2 = 0.8 5 6 1 6 8 x 3 = - 0.7 1 1 3 6 5 u = 0 v = 2.4 3 6 1 9
'—9.7 4 4 7 6 0 0
72 £ Q - 2.8 4 5 4 6 3.4 2 4 6 7
0
V 2/ =
3.4 2 4 6 7
0
T h e i n e q u a l i t y c o n s t r a i n t i s i n a c t i v e; t h e r e f o r e, t h e f e a s i b l e c h a n g e s n e e d t b s a t i s f y l i n e a r i z e d e q u a l i t y c o n s t r a i n t a l o n e.
( 2x i
V h = ( 1 I =
< 2 * 3,
— 3.2 4 8 2 5 ^ 1
,— 1.4 2 2 7 3;
4.7 Second-Order Su f fi c i e n t Conditions
T h u s, t h e f e a s i b l e c h a n g e s m u s t s a t i s f y t h e f o l l o w i n g e q u a t i o n:
— 3.2 4 8 2 5 d i + d 2 - 1.4 2 2 7 3 i 2 3 = 0
S o l v i n g t h i s e q u a t i o n, w e g e t d\ = 0.3 0 7 8 5 7 5 ( ^ 2 — 1.4 2 2 7 3 i 2 3 ). T h e v e c t o r o f f e a s i b l e c h a n g e s i s
d r = ( 0.3 0 7 8 5 7 5 ( ί Ϊ 2 — 1.4 2 2 7 3 ^ ) d 2 d s ) w i t h d 2 a n d a r b i t r a r y
S u b s t i t u t i n g i n t o t h e e x p r e s s i o n f o r s u f f i c i e n t c o n d i t i o n s, w e h a v e
T h i s e x p r e s s i o n e v a l u a t e s t o
—3.3 0 7 2 4 ^ 2 + 8.1 6 3 3 3 ^ 2 ^ 3 + 3.9 3 7 6 5 i i |
W e c a n e a s i l y s e e t h a t t h i s e x p r e s s i o n i s n o t a l w a y s p o s i t i v e. ( F o r e x a m p l e, c h o o s e ^3 = 0 a n d t h e n f o r a n y v a l u e o f d 2, t h e r e s u l t i s n e g a t i v e.) T h e r e f o r e, t h i s i s n o t a m i n i m u m p o i n t.
( b ) S e c o n d - o r d e r c o n d i t i o n c h e c k a t p o i n t 2
x i = - 0.3 4 2 7 1 7 7 x 2 = 0.0 6 5 7 8 4 0 2 x 3 = - 1.9 5 3 6 5 u = 0 v = 0.5 1 4 0 7 6 6
T h e i n e q u a l i t y c o n s t r a i n t i s i n a c t i v e; t h e r e f o r e, t h e f e a s i b l e c h a n g e s n e e d t o s a t i s f y t h e l i n e a r i z e d e q u a l i t y c o n s t r a i n t a l o n e.
T h u s, t h e f e a s i b l e c h a n g e s m u s t s a t i s f y t h e f o l l o w i n g e q u a t i o n:
— 0.6 8 5 4 3 5 5 ( i i + d 2 - 3.9 0 7 3 0 ί ϊ 3 = 0
S o l v i n g t h i s e q u a t i o n, w e g e t d\ = 1.4 5 8 9 2 ( ^ 2 - 3.9 0 7 3 0 ^ ). T h e v e c t o r o f f e a s i b l e c h a n g e s i s
V h =
- 0.6 8 5 4 3 5 5'
- 3.9 0 7 3 0
d T = ( 1.4 5 8 9 2 ( i i 2 - 3.9 0 7 3 0 d 3 ) d 2 d s ) w i t h d 2 a n d d 3 a r b i t r a r y
C h ap te r 4 Op t im al it y Conditions
2.0 5 6 3 0 0 0 \ /2 0 0>
0 - 7.8 1 4 6 1 0.2 6 3 1 3 6 1 I + 0.5 1 4 0 7 6 6 I 0 0 0
0 0.2 6 3 1 3 6 1 0 / \0 0 2;
S u b s t i t u t i n g i n t o t h e e x p r e s s i o n f o r s u f f i c i e n t c o n d i t i o n s, w e h a v e d T
T h i s e x p r e s s i o n e v a l u a t e s t o
- 1 0.0 0 3 0d\ + 1 7.6 2 7 6 ^ 2 ^ 3 - 3 2.3 8 2 0 i i f I t i s n o t a l w a y s p o s i t i v e. T h e r e f o r e, t h i s i s n o t a m i n i m u m p o i n t.
( c ) S e c o n d - o r d e r c o n d i t i o n c h e c k a t p o i n t 3
X! = - 0.0 3 7 2 2 1 7 3 x 2 = 1.2 9 1 8 4 x 3 = 1.6 4 5 2 2 « = 4.2 7 8 6 6 v = - 4.2 2 2 8 2
(
- 0.2 2 3 3 3 0 4 0 0
0 6.5 8 0 9 1 5.1 6 7 3 6
0 5.1 6 7 3 6 0
T h e i n e q u a l i t y c o n s t r a i n t i s a c t i v e; t h e r e f o r e, t h e f e a s i b l e c h a n g e s n e e d t o s a t i s f y b o t h t h e l i n e a r i z e d e q u a l i t y a n d i n e q u a l i t y c o n s t r a i n t s.
- 0.0 7 4 4 4 3 4 6 ^
1
3.2 9 0 4 5
T h u s, t h e f e a s i b l e c h a n g e s m u s t s a t i s f y t h e f o l l o w i n g e q u a t i o n s:
— 0.0 7 4 4 4 3 4 6 i i i - d 2 + 2 d 3 = 0 - 0.0 7 4 4 4 3 4 6 i i i + d 2 + 3.2 9 0 4 5 d 3 - 0
S o l v i n g t h e s e e q u a t i o n s, w e g e t d i = 3 5.5 3 3 3 ^ d 2 = - 0.6 4 5 2 2 7 6 ^ · T h e v e c t o r o f f e a s i b l e c h a n g e s i s
d 7 = ( 3 5.5 3 3 3 ^ 3 — 0.6 4 5 2 2 7 6 ^ 3 d 3 ) w i t h d 3 a r b i t r a r y
S u b s t i t u t i n g i n t o t h e e x p r e s s i o n f o r s u f f i c i e n t c o n d i t i o n s, w e h a v e
/—0.2 2 3 3 3
0
“ ° 1
d T
°
6.5 8 0 9 1
5.1 6 7 3 6
Λ o
5.1 6 7 3 6
o I
( 2 0 0\ /2 0 0>
4.2 2 2 8 2 | 0 0 0 1 + 4.2 7 8 6 6 1 0 0 0
.0 0 2/ \0 0 0 >
— 1 5 3.3 6 4 ^ 1
I t i s n e v e r p o s i t i v e. T h e r e f o r e, t h i s i s n o t a m i n i m u m p o i n t.
( d ) S e c o n d - o r d e r c o n d i t i o n c h e c k a t p o i n t 4
x i = 0 x 2 = - 9.2 9 1 5 0 * 3 = - 3.6 4 5 7 5 u = 2 1 9.7 1 9 v = 8 4.2 2 1 8
T h i s e x p r e s s i o n e v a l u a t e s to
f 0
0
0
V 2/ = 0 - 1 4.5 8 3 0 - 3 7.1 6 6 0
,0 - 3 7.1 6 6 0
0
T h e i n e q u a l i t y c o n s t r a i n t i s a c t i v e; t h e r e f o r e, t h e f e a s i b l e c h a n g e s n e e d t o s a t i s f y b o t h t h e l i n e a r i z e d e q u a l i t y a n d i n e q u a l i t y c o n s t r a i n t s.
/2 * Λ / 0
V h = 1 = 1
\2x 3 ) V - 7'2 9 1 5 0 >
T h u s, t h e f e a s i b l e c h a n g e s m u s t s a t i s f y t h e f o l l o w i n g e q u a t i o n s:
~ d 2 + 2 i * 3 = 0 d 2 — 7.2 9 1 5 0 ί ί 3 = 0
S o l v i n g t h e s e e q u a t i o n s, w e g e t d 2 = 0 d 3 = 0. T h e v e c t o r o f f e a s i b l e c h a n g e s i s
d T = ( d i 0 0 ) w i t h d i a r b i t r a r y
S u b s t i t u t i n g i n t o t h e e x p r e s s i o n f o r s u f f i c i e n t c o n d i t i o n s, w e h a v e
Ό 0 0
0 - 1 4.5 8 3 0 - 3 7.1 6 6 0
,0 - 3 7.1 6 6 0
0
(
2 0 0\ /2 0 0>
0 0 0 1 + 2 1 9.7 1 9 0 0 0
0 0 2/ \0 0 0 j
T h i s e x p r e s s i o n e v a l u a t e s t o
6 0 7.8 8 3 d 2
I t i s a l w a y s p o s i t i v e. T h e r e f o r e, t h i s i s a m i n i m u m p o i n t.
Chapter 4 Op t im al it y Conditions
E x a m p l e 4.2 2 O p e n - t o p c o n t a i n e r C o n s i d e r t h e s o l u t i o n o f t h e o p e n - t o p r e c t ­
a n g u l a r c o n t a i n e r p r o b l e m f o r m u l a t e d i n C h a p t e r 1. T h e p r o b l e m s t a t e m e n t i s a s f o l l o w s.
A c o m p a n y r e q u i r e s o p e n - t o p r e c t a n g u l a r c o n t a i n e r s t o t r a n s p o r t m a t e r i a l. U s i n g t h e f o l l o w i n g d a t a, f o r m u l a t e a n o p t i m u m d e s i g n p r o b l e m t o d e t e r m i n e t h e c o n t a i n e r d i m e n s i o n s f o r t h e m i n i m u m a n n u a l c o s t.
C o n s t r u c t i o n c o s t s
S i d e s = $ 6 5/m 2 E n d s = $ 8 0/m 2 B o t t o m = $ 1 2 0/m 2
U s e f u l l i f e
1 0 y e a r s
S a l v a g e v a l u e
2 0 % o f t h e i n i t i a l c o n s t r u c t i o n c o s t
Y e a r l y m a i n t e n a n c e c o s t
$ 1 2/m 2 o f t h e o u t s i d e s u r f a c e a r e a
M i n i m u m r e q u i r e d v o l u m e o f t h e c o n t a i n e r
■ 1 2 0 0 m 3
N o m i n a l i n t e r e s t r a t e
1 0 % ( A n n u a l c o m p o u n d i n g )
T h e d e s i g n v a r i a b l e s a r e t h e d i m e n s i o n s o f t h e b o x.
b = W i d t h o f c o n t a i n e r I — L e n g t h o f c o n t a i n e r h = h e i g h t o f c o n t a i n e r
C o n s i d e r i n g t i m e v a l u e o f m o n e y, t h e a n n u a l c o s t i s w r i t t e n a s t h e f o l l o w i n g f u n c t i o n o f d e s i g n v a r i a b l e s ( s e e C h a p t e r 1 f o r d e t a i l s ):
A n n u a l c o s t = 4 8.0 3 1 4 h h + 3 0.0 2 3 5 b i 4 - 4 3.5 2 5 5 h i
T h e o p t i m i z a t i o n p r o b l e m i s s t a t e d a s f o l l o w s:
F i n d b, h, a n d i t o
M i n i m i z e a n n u a l c o s t = 4 8.0 3 1 4 f c/j + 3 0.0 2 3 6 M + 4 3.5 2 5 5/i i S u b j e c t t o b h i > 1 2 0 0 a n d b, h, a n d I > 0
C l e a r [ b, h, /] ι v a r s = { b, h, / };
f = 48 . 0 3 1 4 b h + 3 0.0 2 3 6 b/ + 4 3.5 2 5 5 h/f g = { - b h/ + 1200 £ 0};
A v a l i d K T p o i n t i s o b t a i n e d w h e n t h e v o l u m e c o n s t r a i n t i s a c t i v e.
s o l = K T S o l u t i o n [ £, g, v a r s, A c t i v e C a s e s -» { { 1 } } ];
M i n i m i z e f -+ 4 8.0 3 1 4 b h + 3 0.0 2 3 6 b/+ 43 .5 2 5 5 h/
4.7 Second-Order Su f fi c i e nt Conditions
v f
4 8, 03 1 4 h + 3 0.0 2 3 6/' 4 8.0 3 1 4 b + 4 3.5 2 5 5/ ^ 3 0.0 2 3 6 b + 4 3.5 2 5 5 h,
* * * * * l e c o n s t r a i n t s & t h e i r g r a d i e n t s
g- L 1 2 0 0 - b h/ < 0 V g!
'- T o/'
- b f
,- b h,
* * * * * L a g r a n g i a n 4 8 . 0 3 1 4 b h + 3 0.0 2 3 6 b/ + 4 3 . 5 2 5 5 h/ + ( 1 2 0 0 - b h/+ s i ) u i
4 8.0 3 1 4 h + 3 0.0 2 3 6/ - h/u x = = 0 '
4 8 ,0 3 1 4 b + 4 3 .5 2 5 5/ - b/u -. = = <
VL· = 0 -»
3 0.0 2 3 6 b + 4 3.5 2 5 5 h - b h u 1 1 2 0 0 - b h/ + s f = = 0 2 s 1 u 1 =■= 0
= = 0
* * * * * V a l i d KT P o i n t ( s ) * * * * * f - > 1 3 4 6 3.3
b - » 1 1.6 3 84
h -» 8.0 2 8 1
/- * 1 2.8 4 3 3
u i 7.4 7 9 6 3 2 „
S l - 4 0
K T p o i n t -----------------------------------------------------------------------------------------------------------------------------------------
b = 1 1.6 3 8 4 h = 8.0 2 8 1 I = 1 2.8 4 3 3 « ι = 7.4 7 9 6 F o r t h e s u f f i c i e n t c o n d i t i o n, w e n e e d t o s h o w t h a t
_____________________________________d T [ V 2/ ( x * > + Ml v 2 a ( x * ) ] d > 0 -------------------------
f o r a l l f e a s i b l e c h a n g e s d t h a t s a t i s f y t h e f o l l o w i n g e q u a t i o n
V g,( x * ) Td = 0
T h e H e s s i a n m a t r i c e s a r e a s f o l l o w s: ________
(
0 4 8.0 3 1 3 3 0.0 2 3 5 N
4 8.0 3 1 3 0 4 3.5 2 5 4
3 0.0 2 3 5 4 3.5 2 5 4 0
A t t h e K T p o i n t, t h e s e m a t r i c e s a r e
V 2* i =
0 4 8.0 3 1 3 3 0.0 2 3 5 ^ V 2/ = [ 4 8.0 3 1 3 0 4 3.5 2 5 4
,3 0.0 2 3 5 4 3.5 2 5 4 0,
Chapter 4 Op t im al it y Conditions
0 - 1 2.8 4 3 2 —8.0 2 8 0''
V g i = I - 1 2.8 4 3 2 - 8.0 2 8 0
0 - 1 1.6 3 8 4
- 1 1.6 3 8 4 0>
T h e f e a s i b l e c h a n g e s n e e d t o s a t i s f y t h e l i n e a r i z e d i n e q u a l i t y c o n s t r a i n t.
— 1 0 3.1 0 6 V|
v* ■ (:E ) - (
1 4 9.4 7 5 —9 3.4 3 4 2,
T h u s, t h e f e a s i b l e c h a n g e s m u s t s a t i s f y t h e f o l l o w i n g e q u a t i o n:
— 1 0 3.1 0 7 i 2 i - 1 4 9.4 7 5 ^ 2 - 9 3.4 3 4 3 ^ = 0
S o l v i n g t h i s e q u a t i o n, w e g e t d i = 0.0 0 9 6 9 8 6 7 3 ( — 1 4 9.4 7 5 ^ 2 — 9 3.4 3 4 2 ^ ). T h e v e c t o r o f f e a s i b l e c h a n g e s i s
dT = ( 0.0 0 9 6 9 8 6 7 3 ( — 1 4 9.4 7 5 ^ 2 - 9 3.4 3 4 2 ^ ) d 2 d 3 )
w i t h d 2 a n d d $ a r b i t r a r y
S u b s t i t u t i n g i n t o t h e e x p r e s s i o n f o r s u f f i c i e n t c o n d i t i o n s, w e h a v e q = d T
0 4 8.0 3 1 3 3 0.0 2 3 5 >
4 8.0 3 1 3 0 4 3.5 2 5 4
,3 0 1 0 2 3 5 4 3.5 2 5 4 0
0 - 1 2.8 4 3 2 — 8.0 2 8 0 9\
+ 7.4 7 9 6 3 | - 1 2.8 4 3 2 0 - 1 1.6 3 8 4 I
- 8.0 2 8 0 9 - 1 1.6 3 8 4
T h i s e x p r e s s i o n e v a l u a t e s t o
q = 1 3 9.2 6 3 ^ 2 + 8 7.0 5 0 9 ^ 4 + 5 4.4 1 4 0 ^
T h i s i s a q u a d r a t i c f u n c t i o n i n ( d 2, d 3 ). T b d e t e r m i n e t h e s i g n o f t h i s t e r m, w e w r i t e i t i n a q u a d r a t i c f o r m a n d d e t e r m i n e t h e p r i n c i p a l m i n o r s.
, . Z 2 7 8.5 2 6 8 7 « = - ( * y 8 7.
2 V87.0509 1 0 8 T h e p r i n c i p a l m i n o r s o f t h e m a t r i x a r e
.5 2 6 8 7
.0 5 0 9\ /d 2\ 8.8 2 8 ) V d 3 )
M i = 2 7 8.5 2 6 M 2 = D e t
Y 2 7 8.5 2 6 8 7.0 5 0 9 γ
\8 7.0 5 0 9 1 0 8.8 2 8/
- 2 2 7 3 3.5
4.8 I^agrangian D u al it y
S i n c e b o t h p r i n c i p a l m i n o r s a r e p o s i t i v e, t h e m a t r i x i s p o s i t i v e d e f i n i t e, a n d h e n c e t h e s i g n o f q i s a l w a y s p o s i t i v e. T h i s s h o w s t h a t t h e c o m p u t e d K T p o i n t i s a m i n i m u m p o i n t.
4.8 L a g r a n g i a n D u a l i t y
F o r a n y o p t i m i z a t i o n p r o b l e m, t h e L a g r a n g i a n f u n c t i o n a n d t h e K T c o n d i t i o n s c a n b e u s e d t o d e f i n e a d u a l o p t i m i z a t i o n p r o b l e m. T h e v a r i a b l e s i n t h e d u a l p r o b l e m a r e t h e L a g r a n g e m u l t i p l i e r s o f t h e o r i g i n a l p r o b l e m ( c a l l e d t h e p r i m a l p r o b l e m ). T h e d u a l p r o b l e m s p l a y a n i m p o r t a n t r o l e i n t h e o r e t i c a l d e v e l o p ­
m e n t a n d i m p l e m e n t a t i o n o f s e v e r a l c o m p u t a t i o n a l m e t h o d s t h a t a r e d i s c u s s e d i n l a t e r c h a p t e r s.
I t i s i m p o r t a n t t o p o i n t o u t t h a t t h e d u a l i t y c o n c e p t s p r e s e n t e d i n t h i s s e c t i o n a p p l y o n l y t o c o n v e x p r o b l e m s. T h i s d e c i s i o n i s m a d e i n o r d e r t o k e e p t h e p r e s e n t a t i o n s i m p l e a n d a l s o r e c o g n i z i n g t h a t, i n p r a c t i c e, d u a l i t y i s u s e d m o s t o f t e n i n l i n e a r a n d q u a d r a t i c p r o g r a m m i n g p r o b l e m s, w h i c h a r e b o t h c o n v e x. I t i s p o s s i b l e t o e x t e n d t h e i d e a s t o a g e n e r a l n o n c o n v e x c a s e i f w e a s s u m e t h a t l o c a l l y a r o u n d a n o p t i m u m, t h e f u n c t i o n s a r e c o n v e x. I n t h i s c a s e, t h e c o n c e p t i s c a l l e d l o c a l d u a l i t y. F o r d e t a i l e d t r e a t m e n t o f d u a l i t y c o n c e p t s, s e e b o o k s b y B a z a r a a, S h e r a l i, a n d S h e t t y [ 1 9 9 5 ]; N a s h a n d S o f e r [ 1 9 9 6 ]; a n d B e r t s e k a s [ 1 9 9 5 ].
I b k e e p n o t a t i o n s i m p l e, t h e b a s i c c o n c e p t s o f t h e d u a l i t y t h e o r y a r e p r e - s e n t e d b y c o n s i d e r i n g p r o b l e m s w i t h i n e q u a l i t y c o n s t r a i n t s a l o n e. T h e o n l y d i f f e r e n c e f o r t h e e q u a l i t y c o n s t r a i n t s i s t h a t t h e i r L a g r a n g e m u l t i p l i e r s a r e f r e e i n s i g n.
T h e p r i m a l p r o b l e m w i t h i n e q u a l i t y c o n s t r a i n t s a l o n e i s s t a t e d a s f o l l o w s:
M i n i m i z e /( x )
S u b j e c t t o g j ( x ) < 0. i = 1 m
I n t r o d u c i n g t h e L a g r a n g e m u l t i p l i e r v e c t o r u > 0 a n d t h e s l a c k v a r i a b l e v e c t o r s, t h e L a g r a n g i a n f u n c t i o n i s
m
L ( x, a, s ) = /( x ) + u > k'W + 5?]
i =l ________________________________________________________
A s s e e n f r o m m a n y e x a m p l e s p r e s e n t e d i n t h e p r e v i o u s s e c t i o n s, t h e s l a c k v a r i ­
a b l e s g i v e r i s e t o s w i t c h i n g c o n d i t i o n s t h a t s i m p l y s t a t e t h a t e i t h e r a c o n s t r a i n t i s a c t i v e ( i n w h i c h c a s e, «» > 0 a n d s, = 0 ) o r i n a c t i v e ( i n w h i c h c a s e, u; = 0
Chapter 4 Op t im al it y Conditions
a n d s f > 0 ). T h u s, k e e p i n g t h e s w i t c h i n g c o n d i t i o n s i n m i n d, w e c a n d r o p t h e s l a c k v a r i a b l e t e r m f r o m t h e L a g r a n g i a n f u n c t i o n a n d w r i t e'i t a s f o l l o w s.
m
L ( x, u ) = /( x ) + y; U i g i ( x ) s u b j e c t t o u,· > 0. i = 1 m
i = 1
T h e L a g r a n g i a n f u n c t i o n i s a f u n c t i o n o f o p t i m i z a t i o n v a r i a b l e s a n d t h e L a ­
g r a n g e m u l t i p l i e r s. I f a g i v e n p o i n t i s i n s i d e t h e f e a s i b l e r e g i o n, t h e n u\ — 0, i = 1,..., t n, a n d t h e s e c o n d t e r m d o e s n o t c o n t r i b u t e a n y t h i n g t o t h e L a ­
g r a n g i a n f u n c t i o n. I f a c o n s t r a i n t i s v i o l a t e d, t h e n i t s m u l t i p l i e r m u s t b e i n ­
c r e a s e d s o t h a t a m i n i m i z a t i o n w i t h r e s p e c t t o x w i l l f o r c e t h e c o n s t r a i n t t o b e s a t i s f i e d. T h u s, m i n i m i z a t i o n o f L a g r a n g i a n f u n c t i o n c a n b e t h o u g h t o f a s a t w o - s t e p p r o c e s s:
P r i m a l p r o b l e m: M i n i m i z e [ M a x i m i z e L ( x, u ) ]
x u > 0
T h a t i s, w e f i r s t m a x i m i z e t h e L a g r a n g i a n f u n c t i o n w i t h r e s p e c t t o v a r i a b l e s a a n d t h e n m i n i m i z e t h e r e s u l t i n g f u n c t i o n w i t h r e s p e c t t o x. T h i s i s t h e p r i m a l f o r m.
F o r c o n v e x p r o b l e m s, w e c a n w r i t e a n e q u i v a l e n t d u a l f o r m b y i n t e r c h a n g i n g t h e o r d e r o f m a x i m i z a t i o n a n d m i n i m i z a t i o n a s f o l l o w s.
D u a l p r o b l e m: M a x i m i z e [ M i n i m i z e L ( x, t i ) ] u > o χ
A s s u m i n g t h a t w e c a n m i n i m i z e t h e L a g r a n g i a n w i t h r e s p e c t t o o p t i m i z a t i o n v a r i a b l e s, a d u a l f u n c t i o n i s d e f i n e d a s f o l l o w s:
M ( u ) = M i n i m i z e L ( x, α ) ξ M i n
m
f ( x > + y ] ^ ( x )
X X
i=l
T h e f u n c t i o n M ( u ) i s c a l l e d t h e d u a l f u n c t i o n. T h e m a x i m u m o f t h e d u a l f u n c t i o n g i v e s L a g r a n g e m u l t i p l i e r s t h a t s a t i s f y K T c o n d i t i o n s f o r t h e p r i m a l p r o b l e m. T h e d u a l p r o b l e m i s
M a x i m i z e M ( i i )
S u b j e c t t o U{ > 0, i = 1,... m
N o t e t h a t i t i s i m p l i c i t i n t h e a b o v e d i s c u s s i o n t h a t b o t h p r i m a l a n d d i ^ a l p r o b l e m s h a v e f e a s i b l e s o l u t i o n s. O n l y i n t h i s c a s e, t h e m a x i m u m o f t h e d u a l p r o b l e m i s t h e s a m e a s t h e m i n i m u m o f t h e p r i m a l p r o b l e m. C h a p t e r s 7 a n d 8 c o n t a i n s o m e a d d i t i o n a l e x a m p l e s t o c l a r i f y t h i s p o i n t.
T h e f o l l o w i n g e x a m p l e s i l l u s t r a t e t h e b a s i c d u a l i t y c o n c e p t s.
E x a m p l e 4.2 3 C o n s i d e r t h e s o l u t i o n o f t h e f o l l o w i n g t w o v a r i a b l e s m i n i ­
m i z a t i o n p r o b l e m:
4.8 Lagrangian D u a l i t y
f = x i + X 2 - 2x x + 3 x 2 ; g = { x i + x 2 + 5 £ 0,x 1 + 2 s 0 >; v a r s = { x!, x 2 >;
T h e L a g r a n g i a n f u n c t i o n f o r t h e p r o b l e m i s a s f o l l o w s:
L = £ + { u x, u 2 > - M a p [ F i r s t, g ]
- 2 x i + x f + u 2 ( 2 + x 1 ) + 3 x 2 + x i + u i ( 5 + X! + x 2 )
N o t e t h a t f o r e a c h c o n s t r a i n t f u n c t i o n, o n l y t h e l e f t - h a n d s i d e i s n e e d e d t o d e f i n e t h e L a g r a n g i a n f u n c t i o n. I n M a t h e m a t i c a, i t i s c o n v e n i e n t l y d o n e b y a p p l y i n g F i r s t t o e a c h e l e m e n t o f l i s t g u s i n g t h e M a p f u n c t i o n.
W e c a n w r i t e t h e d u a l f u n c t i o n e x p l i c i t l y b y m i n i m i z i n g t h e L a g r a n g i a n f u n c t i o n w i t h r e s p e c t t o x\ a n d x 2, a s f o l l o w s. S i n c e t h i s i s a n u n c o n s t r a i n e d s i t u a t i o n, t h e n e c e s s a r y c o n d i t i o n s f o r t h e m i n i m u m a r e t h a t t h e g r a d i e n t o f L a g r a n g i a n w i t h r e s p e c t t o x i s z e r o.
e q n s = T h r e a d [ G r a d [ L, v a r s ] = = 0 ]
[ - 2 + Hi + UI2 + 2x-l =— 0,3 + Ui + 2x2 — " θ}
s o l = S o l v e [ e q n s, v a r s ]
S u b s t i t u t i n g t h e s e v a l u e s i n t o t h e L a g r a n g i a n, t h e d u a l f u n c t i o n i s w r i t t e n a s f o l l o w s:
H u = E x p a n d [ I i/. s o l [ [!] ] ]
S i n c e t h e r e a r e o n l y t w o v a r i a b l e s i n t h e d u a l f u n c t i o n, w e c a n g r a p h i c a l l y d e t e r m i n e t h e m a x i m u m o f M u, s u b j e c t t o c o n d i t i o n s t h a t u\ a n d u 2 > 0, a s f o l l o w s:
G r a p h i c a l S o l u t i o n [Mu, { u i, 0, 5 }, {u2, 0, 5 }, C o n s t r a i n t s { u i i 0,u 2 ΐ: 0 }, O b j e c t i v e C o n t o u r s -* { 4,5, 6, 7, 7.5, 7.9 }, P l o t P o i n t s ■+ 3 0,
E p i l o g - » { R G B C o l o r [ 1, 0, 0 ],D i s k [ { 3, 3 }, 0.0 5 ],
T e x t ["M a x i m u m",{ 3,3.2 } ] } ];
T h e g r a p h s h o w s t h a t t h e m a x i m u m o f M u i s a t u\ = 3, u 2 = 3. T h u s, w e g e t t h e f o l l o w i n g m i n i m u m p o i n t f o r o u r e x a m p l e.
s o l/. {ux -» 3, u 2 -* 3}
{{x l x 2 ~3}}
Ch ap te r 4 Op t im al it y Conditions
“ 2
F I G U R E 4.1 7 G r a p h i c a l s o l u t i o n o f t h e d u a l p r o b l e m.
T h i s s o l u t i o n c a n b e v e r i f i e d b y s o l v i n g t h e p r o b l e m d i r e c t l y u s i n g K T c o n d i ­
t i o n s.
K T S o l u t i o n [ £, g, v a r s ] ;
2 2
M i n i m i z e f - 2 x 1 + + 3x2 + x 2
Vf ->
1 - 2 +2X-L
\ 3 + 2 x 2
* * * * * LE c o n s t r a i n t s & t h e i r g r a d i e n t s
g-l -» 5 + + x 2 < 0 g 2 -* 2 + x ^ < 0
W l - ( ΐ ) ^ 2 - ( j )
* * * * * L a g r a n g i a n - > ~ 2 x x + x f + u 2 ( 2 + s! + X i ) + 3 x 2 + x i + ( 5 + s i + X i + x 2 )
■2 + + u 2 + 2 X i — — O'
3 + U! + 2 x 2 s = 0 _ 2 _
5 + S i + X! + x 2 = = 0 2
2 + s 2 + x-^ = = 0 0
VL — 0 —^
2 s 1u 1 2 s 2u 2 == 0
4.8 Lagrangian D u al it y
* **** v a l i d KT P o i n t ( s ) ***** f -»■ 8.
Xl - * - 2.
x 2 -> - 3.
Ui -» 3 · u 2 -» 3.
s i -» 0 S2 -*0
E x a m p l e 4.2 4 L i n e a r p r o b l e m This example illustrates the special form taken by the dual o f a linear programming problem.
f = 5x + 9 y j
g = {-x - y + 3 £ 0, * - y - 4 ί 0 ); v a r s = { x, y };
A s o l u t i o n o f t h i s p r o b l e m u s i n g K T c o n d i t i o n s c a n r e a d i l y b e o b t a i n e d a s f o l l o w s:
K T S o l u t i o n [ f, g, v a r s ];
f -» 5 x + 9 y
* * * * * S t a n d a r d i z e d L E ( g < 0 ) c o n s t r a i n t s g i -» 3 - x - y g 2 - » - 4 + x - y
7 g i ( - l ) 7 g 2 ( - 1
2 2 L a g r a n g i a n - > 5 x + 9 y + ( 3 - x - y + s i ) u j + ( - 4 + x - y + S 2) u 2
5 - u -l + u 2 = = 0
VL = 0
9 - u -l - u 2 = = 0
3 - x - y + s i = = G 2
- 4 + x - y + s 2 ==
2 s 1u l == 0 2 s 2u 2 == 0
***** v a l i d KT P o i n t ( s ) *****
f - » 1 3. x ^ 3.5 y -» - 0.5
uj _ -> 7.
u 2 - » 2.
2
Si -* 0 8 2 - * 0
T h e La g r a n g i a n f u n c t i o n f o r t h e p r o b l e m i s a s f o l l o ws:
L = f + { u x , u 2 } .M a p [ F i r s t, g ]
5 x + 9 y +. ( 3 - χ - y ) U l + ( - 4 + x - y ) u 2
C o l l e c t [ L, { x, y } ]
3 u i + y ( 9 - u -l - u 2 ) - 4 u 2 + x ( 5 - U! + u 2 )
Chapter 4 Op t im al it y Conditions
We can write the dual function explicitly by minimizing the Lagrangian with respect to x and y, as follows. Since this is an unconstrained situation, the necessary condition for the minimum is that the gradient of Lagrangian with respect to x is zero.
e q n s = T h r e a d [ G r a d [ L, v a r s ] == 0]
( 5 - u 1 +u2 == 0, 9 - u - l - u 2 == θ}
T h e s e e q u a t i o n s d o n o t i n v o l v e t h e a c t u a l p r o b l e m v a r i a b l e s. Ho w e v e r, w e n o t i c e t h a t t h e s e e q u a t i o n s a r e e x a c t l y t h e c o e f f i c i e n t s o f x a n d y i n t e r ms i n t h e L a g r a n g i a n f u n c t i o n. T h u s, u s i n g t h e s e c o n d i t i o n s, w e c a n e l i m i n a t e p r i ma l p r o b l e m v a r i a b l e s f r o m t h e L a g r a n g i a n f u n c t i o n.
H u = I ·/. { x - + 0,y - + 0 }
3 U! - 4 u 2
T h e d u a l p r o b l e m c a n n o w b e s t a t e d a s f o l l o ws:
Ma x i mi z e 3 u i — 4 u 2
(
5 — u\ + u 2 = O''
9 - u\ - w2 = 0 u i, u 2 > 0
The solution of this dual problem is obtained using KT conditions, as follows:
K T S o l u t i o n [ - H u, e q n s, {u1,u 2 }]
M i n i m i z e f -* - 3 u ^ + 4 u 2
('43'
***** EQ c o n s t r a i n t s & t h e i r g r a d i e n t s h-L -> 5 - u -l + u 2 == 0 h 2 -» 9 - Uj - u 2 = = 0
f-ll
(?)
v h i ^ 1 -, I Vha -» .
* * * * *
L a g r a n g i a n - 3 u a + 4 u 2 + (5 - + u 2 ) νχ + (9 - ^ - u 2 ) v 2
VL = 0 -*
- 3 - Vi - v 2 == 0\
4 + v-l - v 2 = = 0
5 - u -l + u 2 = = 0 9 -uj_ - u 2 == 0
***** v a l i d KT P o i n t ( s ) *****
f -> - 1 3.
u x -> 7.
u 2 -* 2.
v x - 3 .5
v, -> 0.5
4.8 Lagrangian D u a l i t y
C o m p a r i n g t h i s s o l u t i o n w i t h t h e s o l u t i o n o f t h e p r i m a l p r o b l e m, w e s e e t h a t b o t h p r o b l e m s g i v e t h e s a m e s o l u t i o n, e x c e p t t h a t t h e r o l e o f v a r i a b l e s i s r e v e r s e d. I t i s a l s o i n t e r e s t i n g t o n o t e t h a t f o r l i n e a r p r o b l e m s, t h e p r i m a l a n d d u a l p r o b l e m s h a v e a v e r y s i m p l e r e l a t i o n s h i p. T h e d u a l f o r a g e n e r a l l i n e a r p r o g r a m m i n g p r o b l e m w i l l b e p r e s e n t e d i n C h a p t e r 7.
E x a m p l e 4.2 5 N o e x p l i c i t e x p r e s s i o n f o r d u a l f u n c t i o n I n t h e p r e v i o u s e x a m ­
p l e s, t h e o b j e c t i v e a n d t h e c o n s t r a i n t f u n c t i o n w e r e s u c h t h a t i t w a s p o s s i b l e t o g e t e x p l i c i t e x p r e s s i o n s f o r t h e d u a l f u n c t i o n. F o r m o s t n o n l i n e a r p r o b l e m s, i t i s u s u a l l y d i f f i c u l t t o w r i t e d u a l f u n c t i o n s e x p l i c i t l y. T h e d u a l p r o b l e m c a n s t i l l b e d e f i n e d, b u t i t r e m a i n s i n t e r m s o f b o t h t h e a c t u a l o p t i m i z a t i o n v a r i a b l e s a n d t h e L a g r a n g e m u l t i p l i e r s. T h i s e x a m p l e i l l u s t r a t e s t h i s s i t u a t i o n:
2 2 f = x i - 4 χ χ χ 2 + 5 x 2 - L o g [ x i x 2 ] 1
g = { - Χχ + 2 ^ 0,— x 2 + 2 ύ 0 } ;
v a r s = { χ χ, x 2 };
I t c a n e a s i l y b e v e r i f i e d t h a t t h e p r o b l e m i s c o n v e x. T h e L a g r a n g i a n f u n c t i o n f o r t h e p r o b l e m i s a s f o l l o w s:
L = f + { u -l, u 2 } .M a p [ F i r s t, g ]
- L o g [ x 1 x 2 ] +U]_ ( 2 - x x ) + x i + u 2 ( 2 - x 2 ) - 4 χ χ χ 2 + 5 x 2
W e c a n w r i t e t h e d u a l f u n c t i o n b y m i n i m i z i n g t h e L a g r a n g i a n w i t h r e s p e c t t o x\ a n d X 2, a s f o l l o w s:
e q n s = T h r e a d [Or a d [ L, v a r s ] == 0]
A n e x p l i c i t s o l u t i o n o f t h e s e e q u a t i o n s f o r x\ a n d x 2 i s d i f f i c u l t. T h u s, t h e d u a l p r o b l e m c a n o n l y b e w r i t t e n s y m b o l i c a l l y, a s f o l l o w s:
M a x i m i z e — L o g f x ^ ] 4 - « i ( — 2 + x i ) + x f + « 2( — 2 + x 2 ) — 4x\x 2 + 5 x $
/ —« i — τ τ + 2 x i — 4X2 = 0 \
T h e s o l u t i o n o f t h i s d u a l p r o b l e m i s o b t a i n e d b y u s i n g K T c o n d i t i o n s, a s f o l l o w s. N o t e t h e u s e o f — L t o c o n v e r t t h e m a x i m i z a t i o n p r o b l e m t o t h e m i n i m i z a t i o n f o r m e x p e c t e d b y K T S o l u t i o n. A l s o n o t e t h a t d e f a u l t K T v a r i a b l e s ’ n a m e s m u s t
S u b j e c t t o
- « 2 - 4 x i — + 1 0 X 2 = 0
Ml > 0
\ U2 > 0
Chapter 4 O p t im a l it y Conditions
be changed; otherwise, they will conflict with the names of the problem vari­
ables.
K T S o l u t i o n!- L, J o i n [ e q n s, { u x i 0,u 2 i 0 } ], { x x, x 2, « i# u 2 >, K T V a r N a m e s -» { U, S, V} ];
2 2 M i n i m i z e f -> L o g t X ] ^ J - u x ( 2 - X j ) - x i - u 2 ( 2 - x 2 ) + 4 X i X 2 - 5 x 2
' U 1 + s t - 2 x l + 4 x 2 u 2 + 4 x l + h - 1 0 x 2
- 2 + Χχ
- 2 + x 2
* * * * *
L E
c o n s t r a i n t s
9 i - l
J 1 *
' 0
0 g 2 -
u2 ί
' 0 i
V g i ->
o '"J* O
Vg 2 ->
U
0
- 1.
* * * * *
EQ
c o n s t r a i n t s
1 1
h x -> - u!------+ 2 x x - 4x 2 == 0 h 2 -» - u 2 - 4 x 1 ------------+ 1 0 x 2 == 0
X 1 x 2
Vh i -»
l 2 + M
1 - 4 ’
χ ι
v h 2
10 + - i f
- 4
X2
- 1
0
> 0 ,
, - 1 ,
* * * * * L a g r a n g i a n -» L o g [ x 1 x 2 ] + ( S i + ( S 2 - u 2 ) U 2 - U i ( 2 - x ^ - x? + V i ( - ^
1 o 1
— + 2 x j - 4 x 2 ) - u 2 ( 2 - x 2 ) + 4 x j ^x 2 - 5 x 2 + V 2 ( - u 2 - 4 X!---------- + l 0 x 2 )
X1 x 2
U1 + 2 V i - 4 V 2 + ^ - 2 ^ 1 + 4 x
VL = 0 -»
X l
:= 0
u 2 - 4 V i + 1 0 V 2 + 4 x ^ + —£ + - 1 0 x 2 = = 0
x 2
- 2 - % - V i + X! = = 0 - 2 ~ U2 - V2 + X2 == 0 S? - U i == 0
S 2 - u 2 == 0 -U- l - + 2 x x - 4 x 2 = = 0
- u 2 - 4 X l - i + 10x2 == 0 2 S 1 U 1 = = 0 2 S 2 U 2 = = 0
* * * * * v a l i d KT P o i n t ( s ) * * * * *
4.8 Lagrangian D u a l i t y
f - ^ - 1.9 0 5 4 Xl - > 4.1 2 1 3 2 X2 ·* 2 *
U l -> 0
u 2 - * 3 .0 1 4 7 2 U i - > 2.1 2 1 3 2 U2 - * 0
51 - > o
5 2 - > 3.0 1 4 7 2
0
V2 -> 0
T h e s a m e s o l u t i o n i s o b t a i n e d b y s o l v i n g t h e p r o b l e m d i r e c t l y u s i n g K T c o n d i t i o n s. O b v i o u s l y, i n t h i s c a s e t h e r e i s n o a d v a n t a g e t o d e f i n i n g a d u a l a n d t h e n s o l v i n g i t. I n f a c t, s i n c e t h e d u a l p r o b l e m h a s f o u r v a r i a b l e s, i t i s m o r e d i f f i c u l t t o s o l v e t h a n t h e p r i m a l p r o b l e m. T h e m a i n r e a s o n f o r p r e s e n t i n g t h i s
e x a m p l e i s t o s h o w t h e p r o c e s s o f c r e a t i n g a d u a l p r o b l e m c o r r e s p o n d i n g t o a g i v e n p r i m a l p r o b l e m.
K T S o l u t i o n [ f, g, v a r s ];
2 2
M i n i m i z e f -> - L o g [ X1X2 ] + x i - 4 x i x 2 + 5 x 2
V f ■
-34 + 2 x l
4 x,
'4 x l - 5 ^ + 1 0 x 2
* * * * * LE c o n s t r a i n t s & t h e i r g r a d i e n t s g i -> 2 - X i < 0 g 2 -> 2 - x 2 < 0
i - l )
0
v g x
V g 2 ->
( - 1 )
2 2 2 2 L a g r a n g i a n -> - L o g [ x i x 2 ] + u t ( 2 + s i - X i ) + x i + u 2 ( 2 + s 2 - x 2 ) - 4 x j x 2 + 5 x 2
* * * * *
VL = 0
- U1 ".i t + 2 χ ι
■ 4 x o = = 0
- u 2 - 4 x i - j i - + 1 0 x 2 == 0
2 + S i - x j = = 0 2 + S2 - X2 == 0 2 s 1 u 1 =5 0
2s2u2 0
*****
f - » 1. X l -> 4 X9 -»2
V a l i d KT P o i n t ( s )
9054
12132
Ui —> 0
u2 -> 3 s i - >2 s 2 -» 0
.01472
.12132
Chapter 4 O p t im a l it y Conditions
4.9 P r o b l e m s
O p t i m a l i t y C o n d i t i o n s f o r U n c o n s t r a i n e d P r o b l e m s
F i n d a l l s t a t i o n a r y p o i n t s f o r t h e f o l l o w i n g f u n c t i o n s. U s i n g s e c o n d - o r d e r o p t i ­
m a l i t y c o n d i t i o n s, c l a s s i f y t h e m a s m i n i m u m, m a x i m u m, o r i n f l e c t i o n p o i n t s.
F o r o n e a n d t w o v a r i a b l e p r o b l e m s, v e r i f y s o l u t i o n s g r a p h i c a l l y.
4.1. f i x ) = x 4 + 1/2 * 2 - x
4.2. f ( x ) = 3/4 X 6 - 1/3 X 3 + 2
4.3. f ( x, y ) = x + 2 * 2 + 2 y - x y + 2 y 2
4.4. f ( x, y ) = 3 x — 2 X 2 + 2 y — 3 x y — ^ r
4.5. /( x, y ) = 2 X 3 — 3 x y + 4 y 3
4.6. /( x, y ) = x® (y + 2 ) - x y 3 + 2
4.7. /( x, y ) = x 3 + x y + L o g t x 5/y 3 ]
4.8. /( x, y, z ) = x y z - 2+χ2l y2+z2
4.9. S h o w t h a t t h e f o l l o w i n g f u n c t i o n i s c o n v e x, a n d c o m p u t e i t s g l o b a l m i n i m u m.
f ( x, y, ζ ) = 2 X 2 - 2 x y + y 2 - z - y z + z 2
4.1 0. S h o w t h a t t h e f o l l o w i n g f u n c t i o n i s c o n v e x, a n d c o m p u t e i t s g l o b a l m i n i m u m.
7 y i 5 ^2
f i x, y, z ) = 3 x + 2 x * + 2 y - 3 x y + — + z + x z + y z + ~ —
Ζ Δ
4.1 1. S h o w t h a t t h e f o l l o w i n g f u n c t i o n o f f o u r v a r i a b l e s ( χ ι, x 2, X3, X 4) i s c o n ­
v e x, a n d c o m p u t e i t s g l o b a l m i n i m u m.
/ = X1 + 4 x ^ + 2 * 2 - X l X2 + 4 X 2 + 3x 3 + 4 *! + 4 x 4 + 2 x j X4 + 2 X 2 X 4 + 2 * 3 X4 +
4.1 2. A s s u m e t h a t t h e p o w e r r e q u i r e d t o p r o p e l a b a r g e t h r o u g h a r i v e r i s p r o p o r t i o n a l t o t h e c u b e o f i t s s p e e d. S h o w t h a t, t o g o u p s t r e a m, t h e m o s t e c o n o m i c a l s p e e d i s 1.5 t i m e s t h e r i v e r c u r r e n t.
4.1 3. A s m a l l f i r m i s c a p a b l e o f m a n u f a c t u r i n g t w o d i f f e r e n t p r o d u c t s. T h e c o s t o f m a k i n g e a c h p r o d u c t d e c r e a s e s a s t h e n u m b e r o f u n i t s p r o d u c e d i n c r e a s e s a n d i s g i v e n b y t h e f o l l o w i n g e m p i r i c a l r e l a t i o n s h i p s:
1,5 0 0 „ 2,5 0 0
Ci = 5 - | C2 = 7 -I----------------
«1 n 2
4.9 Problems
w h e r e η χ a n d n 2 a r e t h e n u m b e r o f u n i t s o f e a c h o f t h e t w o p r o d u c t s p r o d u c e d. T h e c o s t o f r e p a i r a n d m a i n t e n a n c e o f e q u i p m e n t u s e d t o p r o d u c e t h e s e p r o d u c t s d e p e n d s o n t h e t o t a l n u m b e r o f p r o d u c t s p r o ­
d u c e d, r e g a r d l e s s o f i t s t y p e, a n d i s g i v e n b y t h e f o l l o w i n g q u a d r a t i c e q u a t i o n:
(«1 + n 2 ) [ 0.2 + 2.3 χ 1 0 _ 5 ( « i + n 2 ) + 5.3 χ 1 0 - 9 ( « i + n 2 ) 2 ]
T h e w h o l e s a l e s e l l i n g p r i c e o f t h e p r o d u c t s d r o p s a s m o r e u n i t s a r e p r o d u c e d, a c c o r d i n g t o t h e f o l l o w i n g r e l a t i o n s h i p s:
p i = 1 5 — 0.0 0 1 « ι j p 2 = 2 5 — 0.0 0 1 5 « 2
F o r m u l a t e t h e p r o b l e m o f d e t e r m i n i n g h o w m a n y u n i t s o f e a c h p r o d ­
u c t t h e f i r m s h o u l d p r o d u c e t o m a x i m i z e i t s p r o f i t. F i n d t h e o p t i m u m s o l u t i o n u s i n g o p t i m a l i t y c o n d i t i o n s.
4.1 4. F o r a c h e m i c a l p r o c e s s, p r e s s u r e m e a s u r e d a t d i f f e r e n t t e m p e r a t u r e s i s g i v e n i n t h e f o l l o w i n g t a b l e. F o r m u l a t e a n o p t i m i z a t i o n p r o b l e m t o d e t e r m i n e t h e b e s t v a l u e s o f c o e f f i c i e n t s i n t h e f o l l o w i n g e x p o n e n t i a l m o d e l f o r t h e d a t a. F i n d o p t i m u m v a l u e s o f t h e s e p a r a m e t e r s u s i n g o p t i m a l i t y c o n d i t i o n s.
P r e s s u r e = a e ^ T
T b m p e r a t u r e ( r ° C )
P r e s s u r e ( m m o f M e r c u r y )
2 0
1 5.4 5
2 5
1 9.2 3
3 0
2 6.5 4
3 5
3 4.5 2
4 0
4 8.3 2
5 0
6 8.1 1
6 0
9 8.3 4
7 0
1 2 0.4 5
4.1 5. A c h e m i c a l m a n u f a c t u r e r r e q u i r e s a n a u t o m a t i c r e a c t o r - m i x e r. T h e m i x i n g t i m e r e q u i r e d i s r e l a t e d t o t h e s i z e o f t h e m i x e r a n d t h e s t i r ­
r i n g p o w e r a s f o l l o w s:
T= h000j2
w h e r e S = c a p a c i t y o f t h e r e a c t o r - m i x e r, k g, P = p o w e r o f t h e s t i r r e r, k - W a t t s, a n d T i s t h e t i m e t a k e n i n h o u r s p e r b a t c h. T h e c o s t o f b u i l d i n g t h e r e a c t o r - m i x e r i s p r o p o r t i o n a l t o i t s c a p a c i t y a n d i s g i v e n b y t h e f o l l o w i n g e m p i r i c a l r e l a t i o n s h i p:
C o s t = $ 6 0,0 0 0 V S
T h e c o s t o f e l e c t r i c i t y t o o p e r a t e t h e s t i r r e r i s $ 0.0 5/k - W - h r, a n d t h e o v e r h e a d c o s t s a r e $ 1 3 7.2 P p e r y e a r. T h e t o t a l r e a c t o r t o b e p r o c e s s e d b y t h e m i x e r p e r y e a r i s 1 0 7 k g. T i m e f o r l o a d i n g a n d u n l o a d i n g t h e m i x e r i s n e g l i g i b l e. U s i n g p r e s e n t w o r t h a n a l y s i s, f o r m u l a t e t h e p r o b l e m o f d e t e r m i n i n g t h e c a p a c i t y o f t h e m i x e r a n d t h e s t i r r e r p o w e r i n o r d e r t o m i n i m i z e c o s t. A s s u m e a f i v e - y e a r u s e f u l l i f e, 9 p e r c e n t a n n u a l i n t e r e s t r a t e c o m p o u n d e d m o n t h l y, a n d a s a l v a g e v a l u e o f 1 0 p e r c e n t o f t h e i n i t i a l c o s t o f t h e m i x e r. F i n d a n o p t i m u m s o l u t i o n u s i n g o p t i m a l i t y c o n d i t i o n s.
4.1 6. U s e t h e a n n u a l c o s t m e t h o d i n p r o b l e m 4.1 5.
4.1 7. A m u l t i c e l l e v a p o r a t o r i s t o b e i n s t a l l e d t o e v a p o r a t e w a t e r f r o m a s a l t w a t e r s o l u t i o n i n o r d e r t o i n c r e a s e t h e s a l t c o n c e n t r a t i o n i n t h e s o l u t i o n. T h e i n i t i a l c o n c e n t r a t i o n o f t h e s o l u t i o n i s 5 % s a l t b y w e i g h t. T h e d e s i r e d c o n c e n t r a t i o n i s 1 0 %, w h i c h m e a n s t h a t h a l f o f t h e w a t e r f r o m t h e s o l u t i o n m u s t b e e v a p o r a t e d. T h e s y s t e m u t i l i z e s s t e a m a s t h e h e a t s o u r c e. T h e e v a p o r a t o r u s e s 1 1 b o f s t e a m t o e v a p o r a t e 0.8 r i l b o f w a t e r, w h e r e n i s t h e n u m b e r o f c e l l s. T h e g o a l i s t o d e t e r m i n e t h e n u m b e r o f c e l l s t o m i n i m i z e c o s t. T h e o t h e r d a t a a r e a s f o l l o w s:
T h e f a c i l i t y w i l l b e u s e d t o p r o c e s s 5 0 0,0 0 0 l b s o f s a l i n e s o l u t i o n p e r
d a y.
T h e u n i t w i l l o p e r a t e f o r 3 4 0 d a y s p e r y e a r.
I n i t i a l c o s t o f e v a p o r a t o r, i n c l u d i n g i n s t a l l a t i o n = $ 1 8,0 0 0 p e r c e l l.
A d d i t i o n a l c o s t o f a u x i l i a r y e q u i p m e n t, r e g a r d l e s s o f t h e n u m b e r o f
c e l l s = $ 9,0 0 0.
A n n u a l m a i n t e n a n c e c o s t = 5 % o f i n i t i a l c o s t.
C o s t o f s t e a m = $ 1.5 5 p e r 1 0 0 0 l b s.
E s t i m a t e d l i f e o f t h e u n i t = 1 0 y e a r s.
S a l v a g e v a l u e a t t h e e n d o f 1 0 y e a r s = $ 2,5 0 0 p e r c e l l.
A n n u a l i n t e r e s t r a t e = 1 1 %.
4.9 Problems
F o r m u l a t e t h e o p t i m i z a t i o n p r o b l e m t o m i n i m i z e a n n u a l c o s t. F i n d a n o p t i m u m s o l u t i o n u s i n g o p t i m a l i t y c o n d i t i o n s.
4.1 8. U s e t h e p r e s e n t w o r t h m e t h o d i n p r o b l e m 4.1 7.
A d d i t i v e P r o p e r t y n f C o n s t r a i n t s
G r a p h i c a l l y v e r i f y t h e a d d i t i v e p r o p e r t y f o r t h e f o l l o w i n g c o n s t r a i n t s. T r y t w o d i f f e r e n t m u l t i p l i e r s. A l s o d e m o n s t r a t e t h a t a n e g a t i v e m u l t i p l i e r w o r k s f o r a n e q u a l i t y c o n s t r a i n t b u t n o t f o r a n i n e q u a l i t y c o n s t r a i n t.
4.1 9. 2 X 2 + y = 5 2 x — 3 y < 3
4.2 0. 2 x + y = 5 2 x — 3 y < 3
x — y/S > — 2
K a r u s h - K u h n - T l i c k e r ( K T ) C o n d i t i o n s a n d T h e i r G e o m e t r i c I n t e r p r e t a t i o n
S o l v e t h e f o l l o w i n g p r o b l e m s u s i n g K T c o n d i t i o n s. F o r t w o - v a r i a b l e p r o b l e m s, v e r i f y s o l u t i o n s g r a p h i c a l l y a n d s h o w g r a d i e n t s o f a c t i v e c o n s t r a i n t s t o i l l u s ­
t r a t e g e o m e t r i c i n t e r p r e t a t i o n o f K T p o i n t s.
4.2 1. M i n i m i z e /( x, y ) = — x — 3 y
4.2 2. M i n i m i z e / f o, x 2, x % ) = 4 x j + x 2 + X3 ( x\ + x 2 + 2x 3 < 6^
S u b j e c t *. 2* 1 + « - * > = 4 *1 > 1
S u b j e c t t o
4.2 3. M i n i m i z e f f a, x 2, x 3, * 4) = - x j + x 2 + x 3 + 4 x 4 ( x\ — 5X2 + * 5 + 3X4 — 1 9\
V XI > 2 7
Chapter 4 O p t im al it y Conditions
4.2 4. M i n i m i z e /( x i, x 2 ) = 6x 1 + x 2
_ , . /2 x j + 7 x 2 > 3\
S u b j e c t t o ( 2 χ ι _ ^ ^ 2 J
4.2 5. M a x i m i z e f ( x,y ) — - 6 x + 9 y
c — y > z S u b j e c t t o j 3 x + y > 1 ^ 2 x — 3 y > 3 j
4.2 6. M i n i m i z e / ( x, y ) = t f 2 4 - 2 y 2 S u b j e c t t o ( * +/> - „ )
4.2 7. M i n i m i z e /( x, y ) = x 2 4 - 2/2 - 2 4 x — 2 0 y
< x + 2y > 0 > x + 2 y < 9
OUUJCLL LU
x + y < 8 { x + y > 0 J
4.2 8. M i n i m i z e / ( x, y ) = x 2 + y 2 — L o g O ^ y 2 ] S u b j e c t t o x < L o g [ y ] x > 1 y > 1
4.2 9. M i n i m i z e f { x, y, z ) = x + y + z S u b j e c t t o x - 2 + x ~ 2y ~2 + x ~ 2y ~ 2z ~2 < 1
4.3 0. M i n i m i z e / ( x j, x 2 ) = x\ + % + ψ
%2 *1
S u b j e c t t o
/* i + x2 > 2\
\x i,x 2 > 0 J
4.3 1. M a x i m i z e / ( x j, X2) = x\ + % + S i
X2 x\
S u b j e c t t o ( X l %2 — ^
\χ ι ^ 2 > ο;
4.3 2. M a x i m i z e /( χ ι, x 2 ) = ( x i - 2 ) 2 + ( x 2 - 1 0 ) 2
x f + *2 < 5 0 S u b j e c t t o I x f + x | 4 - 2x 1x 2 — *1 — *2 + 2 0 > 0
x i,x 2 > 0
4.9 P r o b l e m s
4.3 3. M i n i m i z e fix
, y)
= x 2 4- 2yx + y2 — I5x - 20y
/x 2 + y2 <
2 0\
S u b j e c t t o ^ 2 — y2 < lo)
4.3 4. M i n i m i z e f
(
χ
, y ) = ™
S u b j e c t t o j 5)
4.3 5. M i n i m i z e /( x i
, x2)
=
(
X l + X 2 = 10 \
x i > 0 I 3 x i — 5x2 < 1 0/
4.3 6. M i n i m i z e/( x i,x 2,x 3 ) = * f + * i/4 4 - * i/9 — 1 S u b j e c t t o X i + X 2 + * f = l
4.3 7. M i n i m i z e / ( x i, X 2, * 3) = *1 4- * i/4 4- * f/9 — 1 S u b j e c t t o X j 4 - ^ + ^ 3 < l
4.3 8. M i n i m i z e / ( χ ι, X 2, * 3) — 1/(1 + * f 4- *2 + * 3)
_ , . (2 —
3 x? — 4x2 — 5 x i = 0\
S u b j e c t t o {
^;7 χ 2;8 ΐ 3 1 0 j
4.3 9. M i n i m i z e/( x i,x 2,x3> = 1/(1 4- * f 4- *2 4- * 3)
. .. _ ,. / X l 4- 2X2 -I- 3X3 = 0 \
4.4 0. S u b j e c t t o {i4 + 54 + e4 = 7)
4.4 1. M a x i m i z e /( x1, X2, X
3
) = x\xi
4- *3
S u b j e c t t o x f 4- 2x% + 3 x f = 4___________________________
4.4 2. M i n i m i z e /( χ ι, X 2, * 3 ) = * 1*2 4- S u b j e c t t o x f 4~ 2 x | 4- 3 x | > 4
4.4 3. M i n i m i z e /( χ ι, x 2, * 3) =
* f 4- 9 x | 4- *3 S u b j e c t t o X 1X2 > 1
4.4 4. M i n i m i z e / ( χ ι, X 2, X 3) = x f 4- 9 x | 4- *3 S u b j e c t t o x i x 2*3 > 1
C h a p t e r 4 O p t i m a l i t y C o n d i t i o n s
4.4 5. M a x i m i z e / f a, X2, * 3) = 7 x 1 0 9xjx2x%
/(xf4)/107
< 0.7\
S u b j e c t t o I xfx2 =
7 0 0 J
\ Xl <7 /
4.4 6. A d e s i g n p r o b l e m i s f o r m u l a t e d i n t e r m s o f s i x o p t i m i z a t i o n v a r i a b l e s a s f o l l o w s:
M a x i m i z e / = 5 * i + e - 2 * 2 — e~*2 +
* 1*3 -(- 4 x 3 4- 6x 4 +
ίχ\ + x2 +
X 3 4- X 4 + X
5
+ X6 <
IO'' x i 4- X 3 4- xa
< 5
A s s u m e o n e i s c o n s i d e r i n g s o l v i n g t h i s p r o b l e m u s i n g K T c o n d i t i o n s. W h a t i s t h e t o t a l n u m b e r o f c a s e s t h a t m u s t b e c o n s i d e r e d? F i n d t h e
G r a p h i c a l l y s h o w t h a t t h e o p t i m u m i s a t ( 2,1 ) a n d b o t h c o n s t r a i n t s a r e a c t i v e. W r i t e d o w n t h e K T c o n d i t i o n s f o r t h e c a s e w h e n b o t h c o n s t r a i n t s a r e a c t i v e a n d s h o w t h a t t h e y d o n o t s u p p o r t t h e g r a p h i c a l s o l u t i o n. E x p l a i n w h y.
4.4 8. A c y l i n d r i c a l v e s s e l, c l o s e d a t b o t h e n d s w i t h f l a t l i d s, i s m a d e o f s h e e t m e t a l. I b m a k e a v e s s e l o f v o l u m e V,
s h o w t h a t t h e l e a s t a r e a o f s h e e t m e t a l w i l l b e u s e d i f t h e r a d i u s i s r = (
V/2n)1^
a n d h e i g h t = 2r.
4.4 9. A n o p e n - t o p c y l i n d r i c a l v e s s e l o f v o l u m e V
i s m a d e o f s h e e t m e t a l. S h o w t h a t t h e l e a s t a r e a o f s h e e t m e t a l w i l l b e u s e d i f t h e r a d i u s i s e q u a l t o t h e h e i g h t.
4.5 0. H a w k e y e f o o d s o w n s t w o t y p e s o f t r u c k s. T t u c k t y p e I h a s a r e f r i g e r a t e d c a p a c i t y o f 1 5 m 3 a n d a n o n r e f r i g e r a t e d c a p a c i t y o f 2 5 m 3. T h i c k t y p e I I h a s a r e f r i g e r a t e d c a p a c i t y o f 15 m 3 a n d n o n - r e f r i g e r a t e d c a p a c i t y o f 10 m 3. O n e o f t h e i r s t o r e s i n G o f e r C i t y n e e d s p r o d u c t s t h a t r e q u i r e 1 5 0 m
3 o f r e f r i g e r a t e d c a p a c i t y a n d 1 3 0 m 3 o f n o n r e f r i g e r a t e d c a p a c i t y. F o r t h e r o u n d t r i p f r o m t h e d i s t r i b u t i o n c e n t e r t o G o f e r C i t y, t r u c k
S u b j e c t t o
Xl - *2 4- χ3 4- * 5 + Xq <
5 x2
4- 2
x
4 4- X
5 4- 0.8 x 6 = 5
* 3 + * 5 + * 6 = 5 \ Xi>0, i= l
............6 /
c o n -
s t r a i n t s a r e a c t i v e.
4.4 7. C o n s i d e r t h e f o l l o w i n g o p t i m i z a t i o n p r o b l e m: M a x i m i z e f = χ
4.9 P r o b l e m s
t y p e I u s e s 3 0 0 l i t e r s o f f u e l, w h i l e t r u c k t y p e I I u s e s 2 0 0 l i t e r s. U s e K T c o n d i t i o n s t o d e t e r m i n e t h e n u m b e r o f t r u c k s o f e a c h t y p e t h a t t h e c o m p a n y m u s t u s e i n o r d e r t o m e e t t h e s t o r e's n e e d s w h i l e m i n i m i z i n g f u e l c o n s u m p t i o n.
4.5 1. D u s t f r o m a n o l d e r c e m e n t m a n u f a c t u r i n g p l a n t i s a m a j o r s o u r c e o f d u s t p o l l u t i o n i n a s m a l l c o m m u n i t y. T h e p l a n t c u r r e n t l y e m i t s 2 p o u n d s o f d u s t p e r b a r r e l o f c e m e n t p r o d u c e d. T h e E n v i r o n m e n t a l P r o ­
t e c t i o n A g e n c y ( E P A ) h a s a s k e d t h e p l a n t t o r e d u c e t h i s p o l l u t i o n b y 8 5 % ( 1.7 l b s/b a r r e l ). T h e r e a r e t w o m o d e l s o f e l e c t r o s t a t i c d u s t c o l ­
l e c t o r s t h a t t h e p l a n t c a n i n s t a l l t o c o n t r o l d u s t e m i s s i o n. T h e h i g h e r - e f f i c i e n c y m o d e l w o u l d r e d u c e e m i s s i o n s b y 1.8 l b s/b a r r e l a n d w o u l d c o s t $ 0.7 0/b a r r e l t o o p e r a t e. T h e l o w e r - e f f i c i e n c y m o d e l w o u l d r e d u c e e m i s s i o n s b y 1.5 I b s/b a r r e l a n d w o u l d c o s t $ 0.5 0/b a r r e l t o o p e r a t e. S i n c e t h e h i g h e r - e f f i c i e n c y m o d e l r e d u c e s m o r e t h a n t h e E P A r e q u i r e d a m o u n t a n d t h e l o w e r - e f f i c i e n c y l e s s t h a n t h e r e q u i r e d, t h e p l a n t h a s d e c i d e d t o i n s t a l l o n e o f e a c h. I f t h e p l a n t h a s a c a p a c i t y t o p r o d u c e 3 m i l l i o n b a r r e l s o f c e m e n t p e r y e a r, h o w m a n y b a r r e l s o f c e m e n t s h o u l d b e p r o d u c e d u s i n g e a c h d u s t c o n t r o l m o d e l t o m e e t t h e E P A r e q u i r e ­
m e n t s a t a m i n i m u m c o s t? F o r m u l a t e t h e s i t u a t i o n a s a n o p t i m i z a t i o n p r o b l e m. F i n d a n o p t i m u m s o l u t i o n u s i n g K T c o n d i t i o n s.
4.5 2. A s m a l l e l e c t r o n i c s c o m p a n y i s p l a n n i n g t o e x p a n d t w o o f i t s m a n u ­
f a c t u r i n g p l a n t s. T h e a d d i t i o n a l a n n u a l r e v e n u e e x p e c t e d f r o m t h e t w o p l a n t s i s a s f o l l o w s:
F r o m p l a n t 1: 0.0 0 0 0 2 X ] — x2
F r o m p l a n t 2: 0.0 0 0 0 1 * 2
w h e r e x\
a n d x2
a r e t h e i n v e s t m e n t s m a d e i n t o u p g r a d i n g t h e f a c i l i t i e s. E a c h p l a n t r e q u i r e s a m i n i m u m i n v e s t m e n t o f $ 3 0,0 0 0. T h e c o m p a n y c a n b o r r o w a m a x i m u m o f $ 1 0 0,0 0 0 f o r t h i s u p g r a d e t o b e p a i d b a c k i n y e a r l y i n s t a l l m e n t s i n 1 0 y e a r s a t a n a n n u a l i n t e r e s t r a t e o f 1 2 %. T h e r e v e n u e t h a t t h e c o m p a n y g e n e r a t e s c a n e a r n i n t e r e s t a t a n a n n u a l r a t e o f 1 0 %. A f t e r t h e 1 0 - y e a r p e r i o d, t h e s a l v a g e v a l u e o f t h e u p g r a d e s i s e x p e c t e d t o b e a s f o l l o w s:
F o r p l a n t 1: O.l x i F o r p l a n t 2: 0.1 5 x 2
F o r m u l a t e a n o p t i m i z a t i o n p r o b l e m t o m a x i m i z e t h e n e t p r e s e n t w o r t h o f t h e s e u p g r a d e s. F i n d a n o p t i m u m s o l u t i o n u s i n g K T c o n d i t i o n s.
4.5 3. A c o m p a n y m a n u f a c t u r e s f r a g i l e g i f t i t e m s a n d s e l l s t h e m d i r e c t l y t o i t s c u s t o m e r s t h r o u g h t h e m a i l. A n a v e r a g e p r o d u c t w e i g h s 1 2 k g, h a s
C h a p t e r 4 O p t i m a l i t y C o n d i t i o n s
a v o l u m e o f 0.8 5 m 3, a n d c o s t s $ 6 0 t o p r o d u c e. T h e a v e r a g e s h i p p i n g d i s t a n c e i s 1 2 0 m i l e s. T h e s h i p p i n g c o s t s p e r m i l e b a s e d o n t o t a l w e i g h t a n d v o l u m e a r e $ 0.0 0 6/k g p l u s $ 0.0 2 5/m 3. T h e p r o d u c t s a r e s h i p p e d i n c a r t o n s t h a t a r e e s t i m a t e d t o c o s t $ 2.5/m 3 a n d w e i g h 3.2 k g/m 3. T h e e m p t y s p a c e i n t h e c a r t o n i s c o m p l e t e l y f i l l e d w i t h a p a c k i n g m a t e r i a l t o p r o t e c t t h e i t e m d u r i n g s h i p p i n g. T h i s p a c k i n g m a t e r i a l h a s n e g l i g i b l e w p.i g b t b u t c o s t s $ 0.9 5/m 3. B a s e d o n t h e p a s t e x p e r i e n c e, t h e c o m p a n y h a s d e v e l o p e d t h e f o l l o w i n g e m p i r i c a l r e l a t i o n s h i p b e t w e e n b r e a k a g e a n d t h e a m o u n t o f p a c k i n g m a t e r i a l.
(
V o l u m e o f p a c k i n g m a t e r i a l \
% b r e a k a g e = 8 5 ( 1 — — ---------— ----— —:---------------)
\ V o l u m e o f t h e s h i p p m g c a r t o n
}
T h e m a n u f a c t u r e r g u a r a n t e e s d e l i v e r y i n g o o d c o n d i t i o n, w h i c h m e a n s t h a t a n y d a m a g e d i t e m m u s t b e r e p l a c e d a t t h e c o m p a n y ’ s e x p e n s e. F o r m u l a t e a n o p t i m i z a t i o n p r o b l e m t o d e t e r m i n e t h e s h i p p i n g c a r t o n v o l u m e a n d v o l u m e o f p a c k i n g m a t e r i a l t h a t w i l l r e s u l t i n t h e m i n i ­
m u m o v e r a l l c o s t o f p a c k i n g, s h i p p i n g, a n d d e l i v e r y. F i n d a n o p t i m u m s o l u t i o n u s i n g K T c o n d i t i o n s.
4.5 4. A n i n v e s t o r i s l o o k i n g t o m a k e i n v e s t m e n t d e c i s i o n s s u c h t h a t s h e w i l l g e t a t l e a s t a 1 0 % r a t e o f r e t u r n w h i l e m i n i m i z i n g t h e r i s k o f m a j o r l o s s e s. F o r t h e p a s t s i x y e a r s, t h e r a t e s o f r e t u r n i n t h r e e m a j o r i n v e s t m e n t t y p e s t h a t s h e i s c o n s i d e r i n g a r e a s f o l l o w s:
T y p e
A n n u a l r a t e s o f r e t u r n
S t o c k s
1 8.2 4
1 7.1 2
2 2.2 3
1 5.2 6
1 2.6 2
1 5.4 2
M u t u a l f u n d s
1 2.2 4
1 1.1 6
1 0.0 7
8.4 6
6.6 2
8.4 3
B o n d s
5.1 2
6.2 6
6.3 4
7.0 1
6.11
5.9 5
F o r m u l a t e t h e p r o b l e m a s a n o p t i m i z a t i o n p r o b l e m. F i n d a n o p t i m u m s o l u t i o n u s i n g K T c o n d i t i o n s.
4.5 5. C o n s i d e r t h e c a n t i l e v e r b e a m - m a s s s y s t e m s h o w n i n F i g u r e 4.1 8. T h e b e a m c r o s s - s e c t i o n i s r e c t a n g u l a r. T h e g o a l i s t o s e l e c t c r o s s - s e c t i o n a l d i m e n s i o n s (b
a n d f t ) t o m i n i m i z e t h e w e i g h t o f t h e b e a m w h i l e k e e p i n g t h e f u n d a m e n t a l v i b r a t i o n f r e q u e n c y (to)
l a r g e r t h a n 8 r a d/s e c. F i n d a n o p t i m u m s o l u t i o n u s i n g K T c o n d i t i o n s.
4.9 P r o b l e m s
h
b
S e c t i o n A - A
F I G U R E 4.1 8 R e c t a n g u l a r c r o s s - s e c t i o n c a n t i l e v e r b e a m w i t h a s u s p e n d e d m a s s.
T h e n u m e r i c a l d a t a a n d v a r i o u s e q u a t i o n s f o r t h e p r o b l e m a r e a s f o l l o w s:
··_ , -■ ,....„ ...
--------
F u n d a m e n t a l v i b r a t i o n f r e q u e n c y
6 ) γ fcg/ /w r d u i d n s/S 6 C
E q u i v a l e n t s p r i n g c o n s t a n t, ke
1 1 , LJ
ε ~ έ + m
M a s s a t t a c h e d t o t h e s p r i n g
m = W/g
G r a v i t a t i o n a l c o n s t a n t
g ~
386 in/s e c 2
W e ig h t a t t a c h e d to t h e s p r in g
W =
60 lb s
L e n g t h o f b e a m
L
= 15 i n
M o d u lu s o f e l a s t i c i t y
E
= 30 χ 106 Ib s/i n 2
S p r in g c o n s t a n t
k
= 10 lb s/i n 2
M o m e n t o f i n e r t i a
1
— ΤΣ
m
W i d t h o f b e a m c ro s s - s e c tio n
0.5 i n < fc < 1 i n
H e ig h t o f b e a m c ro s s - s e c tio n
0.2 i n < h
< 2 i n
U n i t w e ig h t o f b e a m m a t e r ia l
0.286 lb s/ i n 3
4.5 6. C o n s i d e r t h e o p t i m u m d e s i g n o f a r e c t a n g u l a r r e i n f o r c e d c o n c r e t e b e a m s h o w n i n F i g u r e 4.1 9. T h e r e i s s t e e l r e i n f o r c e m e n t n e a r t h e b o t t o m. F o r m w o r k i s r e q u i r e d o n t h r e e s i d e s d u r i n g c o n s t r u c t i o n. T h e b e a m m u s t s u p p o r t a g i v e n b e n d i n g m o m e n t. A l e a s t - c o s t d e s i g n i s r e q u i r e d.
T h e b e n d i n g s t r e n g t h o f t h e b e a m i s c a l c u l a t e d f r o m t h e f o l l o w i n g f o r m u l a:
M u = 0.9 0 A sF y d f
1 - 0.5 9
C h a p t e r 4 O p t i m a l i t y C o n d i t i o n s
w h e r e Fy
i s t h e s p e c i f i e d y i e l d s t r e n g t h o f s t e e l, a n d f'c
i s t h e s p e c i f i e d c o m p r e s s i v e s t r e n g t h o f c o n c r e t e. T h e d u c t i l i t y r e q u i r e m e n t s d i c t a t e m i n i m u m a n d m a x i m u m l i m i t s o n t h e s t e e l r a t i o p
= As/bd.
Pmin — P — P ma x
F o r m w o r k
F I G U R E 4.1 9 R e i n f o r c e d c o n c r e t e b e a m.
U s e t h e f o l l o w i n g n u m e r i c a l d a t a:
M a x i m u m s t e e l r a t i o
Praia. —
0.025
M i n i m u m s t e e l r a t io
Pmin
~ 0.0033
R e q u ir e d m o m e n t c a p a c i t y
Mu
> 400 χ 103 N - m
M i n i m u m b e a m w id t h
b
> 300 m m
C o n c r e t e c o v e r
c
= 65 m m
M a x im u m b e a m d e p t h
h
< 1200 m m
C o n c r e t e c o s t
$ 1 0 0/m 3
F o r m w o r k c o s t
$ 2/m 2
S t e e l r e i n f o r c e m e n t c o s t
$ 6 1 0/to n ( l t o n = 907.18 k g )
D e n s i t y o f s t e e l
7850 k g/m 3
Y i e l d s t r e s s o f s t e e l, Fy
4 2 0 M P a
U l t i m a t e c o n c r e t e s t r e n g t h, f ’c
3 5 M P a
F o r m u l a t e t h e p r o b l e m o f d e t e r m i n i n g t h e c r o s s - s e c t i o n v a r i a b l e s a n d a m o u n t o f s t e e l r e i n f o r c e m e n t t o m e e t a l l d e s i g n r e q u i r e m e n t s a t a
4.9 P r o b l e m s
m i n i m u m c o s t. A s s u m e a u n i t b e a m l e n g t h f o r c o s t c o m p u t a t i o n s. F i n d a n o p t i m u m s o l u t i o n u s i n g K T c o n d i t i o n s.
S e n s i t i v i t y A n a l y s i s
S o l v e t h e f o l l o w i n g p r o b l e m s u s i n g K T c o n d i t i o n s. I n c r e a s e t h e a b s o l u t e v a l u e o f t h e c o n s t a n t s i n t h e c o n s t r a i n t s b y 10%, a n d u s e t h e s e n s i t i v i t y e q u a t i o n t o f i n d t h e e f f e c t o f t h i s c h a n g e o n t h e o p t i m u m. V e r i f y t h e r e s u l t s o b t a i n e d f r o m t h e s e n s i t i v i t y a n a l y s i s b y s o l v i n g t h e m o d i f i e d p r o b l e m e i t h e r g r a p h i c a l l y o r b y u s i n g t h e K T c o n d i t i o n s.
4.5 7. M i n i m i z e f(x\,
x 2) = x\
+ ^ + f j
4.5 8. M a x i m i z e /( x i, x 2) = x\ + jjt
+ ^
4.5 9. M a x i m i z e /
(x\,x2) —
( x i — 2 )2 + (
x2 —
1 0 )2
/ xf
+ *2 —
5 0
S u b j e c t t o I a f + x$
4- 2x i *2 — x i — x 2 + 2 0 > 0 \ xi,x2>0
4.6 0. M i n i m i z e f(x, y)
= x 2 + 2 y x + y2 —
1 5 x — 2 0
y
4.6 1. M i n i m i z e / ( x, y)
= ~
xy
4.6 2. M i n i m i z e /( x i,x 2) =
—4*1+2x2—40
C h a p t e r 4 O p t i m a l i t y C o n d i t i o n s
4.6 3. M i n i m i z e / (x\, x2,
* 3) = *1 + * 1/4 + * |/9 - 1 S u b j e c t t o X i 4 - X 2 + * 3 = l
4 6 4. M i n i m i z e f (xi,x2,
X3) = xf + x
\/4 + * 3/9 — 1 S u b j e c t ϊ ο ^ + λ ^ 4 - λ | < 1
4.6 5. M i n i m i z e /( x i, x 2, X 3) = 1/(1 4- x^ 4- x | 4- x f ) n /2 - 3 x? - 4x2 - 5 x? = 0\
SubjeCt to ( 6x 1 + 7^2 + 8x 3 = 0 j ___________________________________________________
4.6 6. M i n i m i z e / ( χ ι, X2, X3) = 1/( 1 4 - x? 4 - * 1 4 - *3)
e , . , . / x i -I - 2 x2 4 - 3x 3 = 0 \
S u b j e c t t o ^ + 5 ^ + e j 2 = 7 j
4.6 7. M i n i m i z e
f(x1,x2,x3)
- 1/0 + *? + x j + * j ) -------------------------------------
o u-
/ X l + 2 x 2 + 3X3 = 0 \
S u b j e c t * ° ( 4^ + 5^ + 6x| < 7J
4.6 8. M a x i m i z e / ( χ ι, X 2, X 3) = X 1X 2 4- x§
S u b j e c t t o x^ + 2%o + 3x§ — 4
4.6 9. M i n i m i z e /( χ ι, X 2, X 3) = X 1X 2 4- *3 S u b j e c t t o x^ + 2 x | 4- 3x§ > 4
4.7 0. M i n i m i z e /( χ ι, x 2, X 3) = 4- 9 x | 4- x§
S u b j e c t t o X 1X 2 >
1
4.7 1. M i n i m i z e /( χ ι, x2,
* 3) = 4- 9x^ 4- x%
S u b j e c t t o X1X 2X 3 > 1
4.7 2. M a x i m i z e /( χ ι,X2,X 3) — 7 * 1 0 -9x i x 2x §
/( χ?χ § )/1 07 < 0.7\
S u b j e c t t o I x$x2
= 7 0 0 J
V *1 < 7 )
4.7 3. C o n s i d e r t h e H a w k e y e f o o d s E x e r c i s e 4.5 0 a g a i n. A f t e r s o l v i n g t h e o r ig - i n a l p r o b l e m, u s e s e n s i t i v i t y a n a l y s i s t o d e t e r m i n e w h a t w i l l b e t h e n e w o p t i m u m o b j e c t i v e f u n c t i o n v a l u e f o r e a c h o f t h e f o l l o w i n g c h a n g e s:
( i ) T h e d e m a n d f o r r e f r i g e r a t e d c a p a c i t y i n c r e a s e s t o 1 6 0 m 3.
( i i ) T h e d e m a n d f o r n o n r e f r i g e r a t e d c a p a c i t y i n c r e a s e s t o 1 4 0 m 3.
4.9 P r o b l e m s
( i i i ) T h e d e m a n d f o r n o n r e f r i g e r a t e d c a p a c i t y i n c r e a s e s t o 1 4 0 m 3 a n d t h a t f o r r e f r i g e r a t e d c a p a c i t y d e c r e a s e s t o 1 4 0 m 3.
4.7 4. C o n s i d e r t h e i n v e s t m e n t E x e r c i s e 4.5 4 a g a i n. A f t e r s o l v i n g t h e o r i g i n a l p r o b l e m, u s e s e n s i t i v i t y a n a l y s i s t o d e t e r m i n e w h a t t h e n e w o p t i m u m o b j e c t i v e f u n c t i o n v a l u e w i l l b e f o r e a c h o f t h e f o l l o w i n g c h a n g e s:
( i ) T h e m i n i m u m e x p e c t e d r a t e o f r e t u r n i s i n c r e a s e d t o 1 2 %.
( i i ) T h e m i n i m u m e x p e c t e d r a t e o f r e t u r n i s d e c r e a s e d t o 9 %.
4.7 5. C o n s i d e r t h e c a n t i l e v e r b e a m E x e r c i s e 4.5 5 a g a i n. A f t e r s o l v i n g t h e o r i g i n a l p r o b l e m, u s e s e n s i t i v i t y a n a l y s i s t o d e t e r m i n e w h a t t h e n e w o p t i m u m o b j e c t i v e f u n c t i o n v a l u e w i l l b e f o r e a c h o f t h e f o l l o w i n g c h a n g e s.
( i ) T h e l i m i t i n g v a l u e o f t h e v i b r a t i o n f r e q u e n c y i s i n c r e a s e d t o 9 r a d/s e c.
( i i ) T h e l i m i t i n g v a l u e o f t h e v i b r a t i o n f r e q u e n c y i s d e c r e a s e d t o 7 r a d/s e c.
O p t i m a l i t y C o n d i t i o n s f o r C o n v e x P r o b l e m s
F o r t h e f o l l o w i n g p r o b l e m s, f i r s t s h o w t h a t t h e o p t i m i z a t i o n p r o b l e m i s c o n ­
v e x, a n d t h e n o b t a i n a n o p t i m u m s o l u t i o n u s i n g K T c o n d i t i o n s. F o r p r o b l e m s i n v o l v i n g t w o v a r i a b l e s, v e r i f y t h e K T s o l u t i o n b y u s i n g g r a p h i c a l m e t h o d s.
4.7 6. M i n i m i z e / ( x, y) — —x
— 3y
S u b j e c t t o ( ^ +/,=< 64) ___________________________________________________________________________
4.7 7. M i n i m i z e / ( χ ι, χι,
X 3) — 4 x i 4- *2 + *3
(x\
4- X 2 + 2x 3 < 6\
2 x i + X 2 - X 3 = 4
xi > 1 X 3 > 3
S u b j e c t t o
V
/
4.7 8. M i n i m i z e /( χ ι, x2,
X 3, X 4) = —x i -I- X 2 4- X 3 4- 4 x 4
(x\
— 5x2 + X 3 + 3x4 = 1 9\
X l — 4x 2 -I- 2 x 4 = 5 - 4 x 2 — 5x3 + 1 5 x 4 = 1 0
S u b j e c t t o
V
x i > 2
/
C h a p t e r 4 O p t i m a l i t y C o n d i t i c m s
4.7 9. M i n i m i z e f(x\,xz)
= 6x 1 +x2
S u b j e c t t o
2 x i 4- 7x2
> 3
2 x i — Χ2 > 2
)
4.8 0. M a x i m i z e / ( x, y
) = — 6x 4- 9
y
4.8 1. M i n i m i z e f (x,
y ) = x 2 4- 2y2
S u b j e c t t o ( * +/> o )
4.8 2. M i n i m i z e f (x, y)
= x 2 4- 2y2
— 2 4 x — 2 0 y /x + 2y>
0\
„ , . x + 2 y < 9
S u b j e c t t o n
. x + y < 8
\x 4- y > 0/
4.8 3. M i n i m i z e / ( x, y) =
x 2 + y2 —
L o g [
x2y2]
S u b j e c t t o x < L o g [ y ] x > 1 y >
1
4.8 4. M i n i m i z e /( x, y, ζ) = x + y + z
S u b j e c t t o x~2+x~2y~2 +x~2y~2z~2 < 1
S e c o n d - O r d e r S u f f i c i e n t C o n d i t i o n s
U s e K T c o n d i t i o n s a n d s e c o n d - o r d e r s u f f i c i e n t c o n d i t i o n s t o s o l v e t h e f o l l o w i n g o p t i m i z a t i o n p r o b l e m s.
4.8 5. M i n i m i z e /( x i, x2)
— 6x i + x2
S u b j e c t t o
2 x i 4- 7X 2 > 3 2x i — x2 > 2
)
4.8 6. M a x i m i z e /( x, y)
= — 6 x 4- 9 y
/x - y > 2 \
S u b j e c t t o I 3x + y >1
1
2 x — 3 y > 3
4.9 P r o b l e m s
4.8 7. M i n i m i z e /( x, y ) = x 2 4- 2y2
S u b j e c t t o ( ^ C q1 )
4.8 8. M i n i m i z e / ( x, y)
= x 2 4- 2 y 2 — 2 4 x — 2 0
y
4.8 9. M i n i m i z e / ( x, y)
= x 2 - f y2 -
L o g [ x 2^ 2] S u b j e c t t o x < L o g [ y ] x > 1 y >
1
4.9 0. M i n i m i z e f(x,y,z) = x + y + z
S u b j e c t t o x - 2 4- x~2y~2
4- x~2y~2z~2
< 1
4.9 1. M i n i m i z e f (x\,
x 2) = x\ + Q
+ Ψ
*2 1
4.9 2. M a x i m i z e /( x i, x 2 ) = x i 4 - ^ 4 -
4.9 3. M a x i m i z e /( x i, x 2 ) = ( x i — 2 ) 2 4 - ( x 2 — 1 0 ) 2
7 X i 4 - < 5 0
S u b j e c t t o I x f 4 - x | 4 - 2x i x2 - x i - x 2 4 - 2 0 > 0 \ *i,x2>0
4.9 4. M i n i m i z e / ( x, y)
= x 2 4- 2yx + y2 — 15x -
2 0 y
S u b j e c t t o
*1 4- x 2 > 2
*1· *2 > o
)
S u b j e c t t o
x\
4- x 2 = 2 k * 1,* 2 > 0
:)
4.9 5. M i n i m i z e /( x, y ) = ^
S u b j e c t t o
C h a p t e r 4 O p t i m a l i t y C o n d i t i o n s
4.9 6. M i n i m i z e / ( x i, X2> = ^ 4^ - 4
—4*1+2x2—40
/ Xl + X 2 = 1 0 S u b j e c t t o I x i > 0
y 3 x i — 5 x 2 < 1 0/
4.9 7. M i n i m i z e /( x i, X2, X3) = *1 + * i/4 + * i/9 — 1 S u b j e c t t o X i + X 2 + X 3 = l
4.9 8. M i n i m i z e / ( χ ι, X 2, X 3) = * f + « 1/4 + x f/9 - 1
S u b j e c t t o x f 4- *2 + ^3 — 1
4.9 9. M i n i m i z e / ( χ ι, X 2, X 3) = 1/(1 + * f + x%
+ * 3)
4.1 0 0. M i n i m i z e /( χ ι, X 2,X 3) = 1/(1 + x f + x | 4- x | )
4.1 0 1. M i n i m i z e /( χ ι, X 2, x 3) = 1/(1 + x f 4- *2 + * 3)
4.1 0 2. M a x i m i z e /( x i, X 2, X 3) = X 1X 2 4- *3 S u b j e c t t o x f + 2 x | 4- 3 x f = 4
4.1 0 3. M i n i m i z e / ( χ ι, X 2, X 3) = X 1X 2 + *3 S u b j e c t t o x f + 2 x j + 3 x | > 4
4.1 0 4. M i n i m i z e / ( χ ι, X 2, X 3) = x f 4- 9 x | 4- X 3 S u b j e c t t o X 1X 2 > 1
4.1 0 5. M i n i m i z e / ( χ ι, X 2, X 3) = x f 4- 9x^ 4- x§ S u b j e c t t o X 1X 2X 3 > 1
4.1 0 6. M a x i m i z e /( χ ι, X 2,X 3) = 7 x 1 0 _ 9x | x 2x f
S u b j e c t t o
2 — 3 x f — 4*2 — 5X 3 = 0 6x 1 + 7x 2 + 8x 3 = 0
')
( x f x | )/1 0 7 < 0.7' xfx2 =
7 0 0 xi < 7
4.9 P r o b l e m s
C o n s t r u c t d u a l p r o b l e m s f o r t h e f o l l o w i n g o p t i m i z a t i o n p r o b l e m s. W r i t e e x p l i c i t d u a l f u n c t i o n s, i f p o s s i b l e; o t h e r w i s e, s t a t e t h e d u a l o p t i m i z a t i o n p r o b l e m i n t e r m s o f b o t h p r i m a l a n d d u a l v a r i a b l e s. U s e e i t h e r K T c o n d i t i o n s o r g r a p h i c a l m e t h o d s t o v e r i f y t h a t b o t h t h e p r i m a l a n d t h e d u a l p r o b l e m s g i v e t h e s a m e s o l u t i o n.
4.1 0 7. M i n i m i z e f (x, y)
= — x
- 3y
S u b j e c t t o ( * x+/r =< 64 )
4.108. M in im iz e / (χ\,
x 2, X 3) = 4 x i + *2 + *3
S u b j e c t t o
(x\ + X2 + 2 x 3 < 6 ^
2 * i + x 2 - * 3 = 4 x i > 1
* 3 > 3
4.1 0 9.
S u b j e c t t o
/ ( X l, X 2, X 3, X 4) = - X l + X 2 + * 3 + 4 x 4 (x\ -
5 x 2 +X
3
+
3X 4 = 1 9\
X l — 4 x 2 + 2x 4 = 5
—4x2 — 5 x 3 + 1 5 x 4 = 1 0 x i >2
4.1 1 0. M i n i m i z e / ( χ ι, X 2) = 6x 1 + X 2 _ / 2χχ + 7x 2 > 3\
S u b j e c t t o j
4.1 1 1. M a x i m i z e / ( x, y) = -6x +
9
y
x — y
> 2
S u b j e c t t o j 3x + y >1
^2x — 3y > 3/
4.1 1 2. M i n i m i z e fix, y)
= x 2 + 2 y2
^x
+ y
> l'
S u b j e c t t o
/x + y
> 1\ \ x, y >
0 /
4.1 1 3. M i n i m i z e / ( x, y)
= x 2 + 2
y2
— 2 4 x — 2 0 y
(x + 2y >
0\ x + 2y <9
x + y <
8 ^ 0)
S u b j e c t t o
C h a p t e r 4 O p t i m a l i t y C o n d i t i o n s
4.1 1 4. M i n i m i z e fix, y) =
x 2 + y2 — Loglx^y2]
S u b j e c t t o x <
L o g [ y ] x
> 1 y
> 1
4.1 1 5. M i n i m i z e / (x, y, z) — x + y
+ z
S u b j e c t t o x~2 + x~2y~2
+ x~2y~2z~2 <
1
C H A P T E R F I V E
U n c o n s t r a i n e d P r o b l e m s
A s s e e n i n C h a p t e r 4, i t i s p o s s i b l e t o s o l v e o p t i m i z a t i o n p r o b l e m s b y d i - r e c t l y u s i n g t h e o p t i m a l i t y c o n d i t i o n s. H o w e v e r, s e t t i n g u p a n d s o l v i n g t h e r e s u l t i n g n o n l i n e a r s y s t e m o f e q u a t i o n s b e c o m e s v e r y d i f f i c u l t a s t h e p r o b l e m s i z e i n c r e a s e s. F u r t h e r m o r e, a t l e a s t f o r c o n s t r a i n e d p r o b l e m s, c h e c k i n g t h e s e c o n d - o r d e r s u f f i c i e n t c o n d i t i o n s i s u s u a l l y v e r y d i f f i c u l t. I n s u c h c a s e s, o n e m u s t f i n d a l l s t a t i o n a r y p o i n t s i n o r d e r t o b e a b s o l u t e l y s u r e t h a t a m i n i m u m h a s b e e n f o u n d. O t h e r w i s e i n s t e a d o f a m i n i m u m p o i n t, o n e c o u l d a c t u a l l y e n d u p w i t h a m a x i m u m p o i n t. C l e a r l y, f i n d i n g a l l p o s s i b l e s o l u t i o n s f o r l a r g e s y s t e m s o f n o n l i n e a r e q u a t i o n s i s a d a u n t i n g t a s k, i f n o t i m p o s s i b l e.
S t a r t i n g f r o m t h i s c h a p t e r, t h e r e m a i n d e r o f t h e b o o k i s d e v o t e d t o t h e p r e s e n t a t i o n o f n u m e r i c a l l y o r i e n t e d m e t h o d s t h a t a r e s u i t a b l e f o r p r a c t i c a l o p t i m i z a t i o n p r o b l e m s. M e t h o d s f o r s o l v i n g u n c o n s t r a i n e d p r o b l e m s a r e c o n - s i d e r e d i n t h i s c h a p t e r. C h a p t e r s 6 a n d 7 p r e s e n t m e t h o d s f o r l a r g e - s c a l e p r o b ­
l e m s i n v o l v i n g l i n e a r o b j e c t i v e a n d c o n s t r a i n t f u n c t i o n s. C h a p t e r 8 p r e s e n t s m e t h o d s f o r s o l v i n g a n i m p o r t a n t c l a s s o f p r o b l e m s k n o w n a s q u a d r a t i c p r o ­
g r a m m i n g i n w h i c h t h e o b j e c t i v e f u n c t i o n i s a q u a d r a t i c f u n c t i o n b u t a l l c o n ­
s t r a i n t s a r e l i n e a r f u n c t i o n s. T h e l a s t c h a p t e r c o n s i d e r s t h e m o s t g e n e r a l c a s e o f n o n l i n e a r l y c o n s t r a i n e d o p t i m i z a t i o n p r o b l e m s. A s y o u p r o b a b l y e x p e c t, t h e c o m p l e x i t y o f t h e m e t h o d s i n c r e a s e s d r a m a t i c a l l y f r o m u n c o n s t r a i n e d p r o b ­
l e m s t o l i n e a r l y c o n s t r a i n e d a n d f i n a l l y t o n o n l i n e a r l y c o n s t r a i n e d p r o b l e m s. T h e m e t h o d s p r e s e n t e d i n t h i s a n d t h e n e x t c h a p t e r a r e f a i r l y w e l l d e v e l o p e d.
C h a p t e r 5 I T n n o n s t r a i n e d P r o b l e m s
T h a t i s n o t t h e c a s e f o r s o m e o f t h e m e t h o d s p r e s e n t e d i n C h a p t e r s 7, 8, a n d 9. K e e p i n m i n d t h a t n u m e r i c a l m e t h o d s a r e d e s i g n e d t o f i n d a l o c a l m i n i m u m p o i n t. W i t h t h e o b v i o u s e x c e p t i o n o f c o n v e x p r o b l e m s, t h e r e i s n o g u a r a n t e e t h a t a s o l u t i o n r e t u r n e d b y t h e s e m e t h o d s i s a g l o b a l m i n i m u m. T h e o n l y w a y t o e v e n c o m e c l o s e t o a g l o b a l m i n i m u m i s t o t r y s e v e r a l d i f f e r e n t s t a r t i n g p o i n t s a n d c h o o s e t h e b e s t a m o n g t h e r e s u l t i n g s o l u t i o n s.
T h i s c h a p t e r a n d t h e r e m a i n i n g c h a p t e r s i n t h e b o o k s t a r t w i t h r e l a t i v e l y s i m p l e m e t h o d s s u i t a b l e f o r s o l v i n g t h e c l a s s o f p r o b l e m s c o n s i d e r e d i n t h a t c h a p t e r. M e t h o d s g e n e r a l l y b e c o m e m o r e a d v a n c e d a s w e g e t d e e p e r i n t o e a c h c h a p t e r. U n f o r t u n a t e l y, t h e c h o i c e o f m e t h o d m o s t s u i t a b l e f o r a g i v e n p r o b ­
l e m i s u s u a l l y n o t t h a t d e a r c u t. U s i n g a m o r e s o p h i s t i c a t e d m e t h o d d o e s n o t a u t o m a t i c a l l y g u a r a n t e e t h a t i t i s t h e b e s t m e t h o d f o r a g i v e n p r o b l e m. N u ­
m e r i c a l p e r f o r m a n c e o f d i f f e r e n t m e t h o d s d e p e n d s o n t h e t y p e o f f u n c t i o n s i n v o l v e d a n d t h e c h o s e n s t a r t i n g p o i n t. T h e r e f o r e, i t i s i m p o r t a n t t o u n d e r ­
s t a n d s t r e n g t h s a n d w e a k n e s s e s o f d i f f e r e n t m e t h o d s. A l s o, a c e r t a i n a m o u n t o f n u m e r i c a l e x p e r i e n c e w i t h d i f f e r e n t m e t h o d s h e l p s i n m a k i n g t h e r i g h t d e c i s i o n.
N u m e r i c a l m e t h o d s f o r s o l v i n g u n c o n s t r a i n e d o p t i m i z a t i o n p r o b l e m s a r e p r e s e n t e d i n t h i s c h a p t e r. T h e p r o b l e m i s s t a t e d a s f o l l o w s:
F i n d v e c t o r o f o p t i m i z a t i o n v a r i a b l e s x
t h a t m i n i m i z e s f
( x )
T h e b a s i c i t e r a t i o n f o r a l l m e t h o d s p r e s e n t e d i n t h i s c h a p t e r c a n b e w r i t t e n a s f o l l o w s:
xk+1
= x * + a kd fc fc = 0,1,...
w h e r e dk
i s k n o w n a s t h e d e s c e n t d i r e c t i o n, a n d a jt i s a
s c a l a r k n o w n a s t h e s t e p l e n g t h. T h e s t a r t i n g p o i n t x ° i s u s u a l l y c h o s e n a r b i t r a r i l y. A t e a c h i t e r a t i o n, a s t e p l e n g t h a n d a d e s c e n t d i r e c t i o n a r e c h o s e n s u c h t h a t f (xk+1)
< / ( x fc).
T h e i t e r a t i o n i s s t o p p e d w h e n s u i t a b l e c o n v e r g e n c e c r i t e r i a i s s a t i s f i e d. S i n c e t h e n e c e s s a r y c o n d i t i o n f o r t h e m i n i m u m o f a n u n c o n s t r a i n e d p r o b l e m i s t h a t i t s g r a d i e n t i s z e r o a t t h e o p t i m u m, t h e c o n v e r g e n c e c r i t e r i a i s w r i t t e n a s f o l l o w s:
w h e r e t o l i s a s m a l l t o l e r a n c e ( e.g., 1 0 ~ 3).
T h e f i r s t s e c t i o n p r e s e n t s a s i m p l e t e s t t o d e t e r m i n e i f a g i v e n d i r e c t i o n i s a d e s c e n t d i r e c t i o n a l o n g w h i c h t h e f u n c t i o n v a l u e d e c r e a s e s. O n c e a d e s c e n t d i r e c t i o n i s k n o w n, t h e p r o b l e m o f c o m p u t i n g a n a p p r o p r i a t e s t e p l e n g t h i s
5.1 D e s c e n t D i r e c t i o n
r e d u c e d t o f i n d i n g t h e m i n i m u m o f a f u n c t i o n o f a s i n g l e v a r i a b l e. T h i s p r o ­
c e s s i s k n o w n a s a l i n e s e a r c h. I t i s p o s s i b l e t o d e t e r m i n e t h e s t e p l e n g t h u s i n g o p t i m a l i t y c o n d i t i o n s f o r t h e m i n i m u m o f a f u n c t i o n o f a s i n g l e v a r i a b l e. H o w e v e r, s i n c e t h i s m e t h o d r e q u i r e s a n e x p l i c i t e x p r e s s i o n f o r t h e d e r i v a t i v e o f t h e o n e - d i m e n s i o n a l f u n c t i o n, i t i s u s e f u l o n l y f o r s m a l l p r o b l e m s a n d h a n d c a l c u l a t i o n s. T h e m e t h o d i s d e s c r i b e d i n s e c t i o n 2 a n d i s c a l l e d a n a l y t i c a l l i n e s e a r c h. F o l l o w i n g t h i s, s e v e r a l n u m e r i c a l l i n e s e a r c h m e t h o d s t h a t d o n o t r e ­
q u i r e d e r i v a t i v e s a r e p r e s e n t e d. T h e s e m e t h o d s i n c l u d e i n t e r v a l s e a r c h, g o l d e n s e c t i o n s e a r c h, a n d q u a d r a t i c i n t e r p o l a t i o n. I n t h e t h i r d s e c t i o n, l i n e s e a r c h m e t h o d s a r e c o m b i n e d w i t h m e t h o d s f o r d e t e r m i n i n g t h e d e s c e n t d i r e c t i o n t o c o m e u p w i t h n u m e r i c a l m e t h o d s f o r s o l v i n g u n c o n s t r a i n e d o p t i m i z a t i o n p r o b l e m s. T h e m e t h o d s d i s c u s s e d i n c l u d e S t e e p e s t d e s c e n t, C o n j u g a t e g r a d i ­
e n t, M o d i f i e d N e w t o n, a n d Q u a s i - N e w t o n.
A s i m p l e t e s t c a n b e d e r i v e d t o d e t e r m i n e i f a g i v e n d i r e c t i o n i s a d i r e c t i o n o f d e s c e n t. F o r d fc t o b e a d e s c e n t d i r e c t i o n, w e m u s t h a v e
/( x k+1) < / ( x k)
o r
/ ( x * + G f j t d * ) </( x fc)
U s i n g T k y l o r s e r i e s e x p a n s i o n, w e h a v e
/( x fc) -(- oikVf ( x fc) r d fc < / ( x fc)
or
QffcV/Cx*)7^ < 0
I f w e r e s t r i c t t h e s t e p l e n g t h a jt t o p o s i t i v e v a l u e s, t h e n w e g e t t h e f o l l o w i n g c r i t e r i a f o r d fc t o b e a d e s c e n t d i r e c t i o n a t g i v e n p o i n t x fc:
Vf ( x fc) T d fc < 0
Furthermore, the numerical value of the product Vf ( x fc) T d fc indicates how fast the function is decreasing along this direction.
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
T h e f o l l o w i n g f u n c t i o n, i n c l u d e d i n t h e O p t i m i z a t i o n T b o l b o x 'U n c o n ­
s t r a i n e d 1 p a c k a g e, e m p l o y s t h i s c r i t e r i a t o d e t e r m i n e i f a g i v e n d i r e c t i o n i s a d e s c e n t d i r e c t i o n:
N e e d s ["O p t i m i z a t i o n T o o l b o x'U l i c o n s t r a i n e d'■];
?D e s c e n t D i r e c t i o n C h e c k
D e s c e n t D i r e c t i o n C h e c k [ f, p t, d, v a r s ] Checks t o s e e i f a g i v e n d i r e c t i o n i s a d e s c e n t d i r e c t i o n f o r f u n c t i o n f. p t = c u r r e n t d e s i g n p o i n t, d = d i r e c t i o n v e c t o r, and v a r s = l i s t o f v a r i a b l e s. The f u n c t i o n r e t u r n s { s t a t u s, V f.d ) where s t a t u s i s e i t h e r D e s c e n t o r N o t D e s c e n t, V f i s t h e g r a d i e n t a t g i v e n p o i n t and V f.d i s t h e d o t p r o d u c t o f t h e g r a d i e n t v e c t o r and t h e g i v e n d i r e c t i o n v e c t o r.
E x a m p l e 5.1 F o r t h e f o l l o w i n g f u n c t i o n o f t w o v a r i a b l e s, c h e c k i f t h e d i r e c ­
t i o n s d i, d-
2
,
a n d d 3 a r e d i r e c t i o n s o f d e s c e n t o r n o t a t t h e g i v e n p o i n t x *:
f = {χ? + x 2 -11)2 + (x i + *1 - *
vars = x k = { 1,2 };
d i = { 1, l };d 2 = { - 1/1 } ;d 3 = { 3 1,1 2 };
T h e D e s c e n t D i r e c t i o n C h e c k i s u s e d t o s e e i f d\
i s a d e s c e n t d i r e c t i o n a t xk.
D e s c e n t D i r e c t i o n C h e c k [ f, x k, d 1# v a r s ];
V f.d - » - 4 3 S t a t u s - » D e s c e n t
S i m i l a r l y, w i t h d i r e c t i o n s d'i
a n d d3t
w e g e t
D e sc en tDire c tion Che ck [f, xk, d2, -vars];
Vf.d->19 S ta tu s -» NotDescent
- 3 - 2 - 1 0 1 2 3
F I G U R E 5.1 G r a p h ic a l il l u s t r a t i o n o f t h r e e d i r e c t io n s fr o m t h e g i v e n p o in t.
5.2 L i n e S e a r r h T e c h n i q u e s — S t e p L e n g t h C a l c u l a t i o n s
D e s c e n t D i r e c t i o n C h e c k [ f, x k, d 3, v a r s ] ;
V f.d - » - 1 1 0 5 S t a t u s D e s c e n t
N o t i c e t h a t t h e d i r e c t i o n dz
i s t h e n e g a t i v e o f t h e g r a d i e n t a t p o i n t xk.
I t i s a l w a y s a d e s c e n t d i r e c t i o n. F u r t h e r m o r e, t h e p r o d u c t V/Td h a s t h e l a r g e s t n e g a t i v e v a l u e a s w e l l, i n d i c a t i n g t h a t t h e n e g a t i v e g r a d i e n t d i r e c t i o n i s t h e d i r e c t i o n of steepest descent.
T h e c o m p u t e d r e s u l t s a r e s u p p o r t e d b v the, cnntrmr
a n d s u r f a c e p l o t s o f t h e f u n c t i o n s h o w n i n F i g u r e 5.1.
E x a m p l e 5.2 F o r t h e f o l l o w i n g f u n c t i o n o f t h r e e v a r i a b l e s, c h e c k i f t h e d i ­
r e c t i o n s di, d 2, a n d d 3 a r e d i r e c t i o n s o f d e s c e n t o r n o t a t t h e g i v e n p o i n t x *:
f = (xx - 1 ) 4 + ( x 2 - 3 ) 2 + 4 (x 3 + 5 ) 4 ; v a r s = { X i, x 2/ X3 } ; x k = { 4, 2, - 1 );
dj. = { - 1, 2, 1 }; d2 = { - 1, 1 0, - 1 >; d3 = { - 1 0 8, 2, - 1 0 2 4 };
D e s c e n t D i r e c t i o n C h e c k [ f , x k, d 1# v a r s ];
- 4 + 12xj _ - 1 2 x i + 4 Xl
1 0 8
v f
- 6 + 2x 2
Vf { 4 ., 2 ., —1.3",“*
-2
,2 0 0 0 + 1 2 0 0 x 3 + 2 4 0 X 3 + 16x 3,
,1 0 2 4,
Vf .d - > 9 1 2 S t a t u s -» N o t D e s c e n t
D e s c e n t D i r e c t i o n C h e c k [ f , x k, d 2, v a r s ] ;
V f.d - » - 1 1 5 2 S t a t u s D e s c e n t
D e s c e n t D i r e c t i o n C h e c k [ f, x k, d 3, v a r s ];
Vf . d -» - 1 0 6 0 2 4 4 S t a t u s D e s c e n t
A s b e f o r e, t h i s e x a m p l e d e m o n s t r a t e s t h a t t h e n e g a t i v e g r a d i e n t d i r e c t i o n i s t h e s t e e p e s t d e s c e n t d i r e c t i o n:
5.2 L i n e S e a r c h T e c h n i q u e s — S t e p L e n g t h C a l c u l a t i o n s
A t e a c h i t e r a t i o n o f a n u m e r i c a l o p t i m i z a t i o n m e t h o d, w e n e e d t o d e t e r m i n e a d e s c e n t d i r e c t i o n a n d a n a p p r o p r i a t e s t e p l e n g t h. T h e s t e p l e n g t h c a l c u l a t i o n s a r e d i s c u s s e d i n t h i s s e c t i o n. M e t h o d s f o r d e t e r m i n i n g d e s c e n t d i r e c t i o n s a r e d i s c u s s e d i n t h e l a t e r s e c t i o n s.
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
A t t h e (k
+ 1 ) i t e r a t i o n, w i t h a k n o w n d e s c e n t d i r e c t i o n, t h e m i n i m i z a t i o n p r o b l e m r e d u c e s t o
F i n d a
i n o r d e r t o m i n i m i z e /( χ * +1) = f (xk
+ a fcd fc) == φ(α)
w h e r e t h e s u b s c r i p t A: o n a i s d r o p p e d f o r c o n v e n i e n c e. T h e p r o b l e m t h e r e f o r e r e d u c e s t o f i n d i n g m i n i m u m o f a f u n c t i o n o f a s i n g l e v a r i a b l e. T b f i n d t h i s m i n i m u m, a n a n a l y t i c a l a n d s e v e r a l n u m e r i c a l m e t h o d s a r e p r e s e n t e d i n t h e f o l l o w i n g s u b s e c t i o n s. N o t e t h a t s i n c e t h e i n t e n t i o n i s t o u s e t h e s e m e t h o d s f o r c o m p u t i n g s t e p l e n g t h, i t w i l l b e a s s u m e d t h a t w e a r e i n t e r e s t e d i n a p o s i t i v e v a l u e o f a, u s u a l l y i n t h e n e i g h b o r h o o d o f a =
1.
5.2.1 A n a l y t i c a l L i n e S e a r c h
I f a n e x p l i c i t e x p r e s s i o n f o r φ(α)
i s a v a i l a b l e, t h e o p t i m u m s t e p l e n g t h c a n e a s i l y b e f o u n d f r o m t h e n e c e s s a r y a n d s u f f i c i e n t c o n d i t i o n s f o r t h e m i n i m u m o f a f u n c t i o n o f a s i n g l e v a r i a b l e, n a m e l y
άφ ά2φ
— = 0 a n d — 7 > 0 da da*
E x a m p l e 5.3 F o r t h e f o l l o w i n g f u n c t i o n, c o m p u t e t h e o p t i m u m s t e p l e n g t h a l o n g t h e g i v e n d i r e c t i o n u s i n g a n a n a l y t i c a l l i n e s e a r c h:
f = ( x i + X2 - 1 1 ) 2 + ( *! +X2 - 7 ) j
v a r s = { x ^ x 2 >;
x k = { 1, 2 } ; d = ( 1,2 ),
W e f i r s t c h e c k t o s e e i f t h e g i v e n d i r e c t i o n i s a d e s c e n t d i r e c t i o n o r n o t. D e s c e n t D i r e c t i o n C h e c k [ f, x k, d, v a r s ];
Vf
V f.d - » - 5 5 S t a t u s -> D e s c e n t
l - 4 4 x 1 + 4xi + 4 x 1 x 2 | v £ { 1 </2 < } /- 3 1 - 2 2 + 2 x f + 4 x 2 j \- 1 2
W e n o w c o n s t r u c t t h e f u n c t i o n φ
b y s e t t i n g x = x fc + a d fc, a s f o l l o w s:
x k i = xk + ad;
φ -
E x p a n d [ f/.T h r e a d [ v a r s -» x k i ] ]
62 - 55a + 4a2 + 8a3 + a 4
W e c o m p u t e t h e s t e p l e n g t h b y s o l v i n g t h e e q u a t i o n άφ/da
= 0.
5.2 L i n e S e a r c h T e c h n i q u e s — S t e p L e n g t h C a l c u l a t i o n s
s o l = Fi ndRoot [ E v a l u a t e α] == 0] , {a, 1 } ]
{a -> 1.24639 }
W e c h e c k t h e s e c o n d - o r d e r n e c e s s a i y c o n d i t i o n t o m a k e s u r e t h a t t h e c o m ­
p u t e d a
i s r e a l l y a m i n i m u m.
D [Φ, { a,2 } ]/.s o l
86.4687
A p o s i t i v e v a l u e i n d i c a t e s t h a t w e h a v e a m i n i m u m v a l u e o f a. T h u s, t h e f o l l o w i n g i s t h e n e x t p o i n t a l o n g t h e g i v e n d i r e c t i o n:
n e w p t = x k i/,s o l
{2.24639,4.49278}
B y e v a l u a t i n g / a t t h e g i v e n p o i n t a n d t h i s n e w p o i n t, w e s e e t h a t t h e o b j e c t i v e v a l u e i n d e e d i s r e d u c e d.
f/. { T h r e a d [ v a r s -» x k ] , T h r e a d [ v a r s -» n e w p t ] }
{62, 17 .5658}
A s i l l u s t r a t e d i n F i g u r e 5.2, t h e s t e p c o m p u t e d b y t h e l i n e s e a r c h p r o c e d u r e i s a t a p o i n t w h e r e t h e g i v e n d i r e c t i o n i s t a n g e n t t o o n e o f t h e o b j e c t i v e f u n c t i o n c o n t o u r s.
* 2
F I G U R E 5.2 O p t i m u m s t e p l e n g t h f r o m t h e c u r r e n t p o i n t a l o n g a g i v e n d e s c e n t d i r e c t i o n.
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
D i r e c t C o m p u t a t i o n o f ^
A s s e e n f r o m t h e a b o v e e x a m p l e, i n o r d e r t o s o l v e f o r a,
w e n e e d . I n s t e a d o f f i r s t d e v e l o p i n g a n e x p r e s s i o n f o r φ (a)
a n d t h e n d i f f e r e n t i a t i n g i t, i t i s p o s s i b l e to
d e v e l o p a n e x p r e s s i o n d i r e c t l y f o r a s f o l l o w s:
M i n i m i z e /( x fc+1) = / (xk
+ adk) = φ(α)
-------------------------------------------------
U s i n g t h e c h a i n r u l e o f d i f f e r e n t i a t i o n, a n d t r e a t i n g / a s a f u n c t i o n o f a
a n d x = x * + adk,
w e h a v e
f * = K t i + +...
= V f ( V + ’ V d 1
da Bx\ da dx
2
da
V /
T h a t is, ^ i s w r i t t e n d i r e c t l y b y t a k i n g t h e d o t p r o d u c t o f t h e g r a d i e n t o f t h e f u n c t i o n a t t h e n e w p o i n t ( x fc + adk)
w i t h t h e d i r e c t i o n v e c t o r. T h e f o l l o w i n g f u n c t i o n, i n c l u d e d i n t h e O p t i m i z a t i o n T b o l b o x 'U n c o n ­
s t r a i n e d' p a c k a g e, i m p l e m e n t s t h i s p r o c e d u r e t o d e t e r m i n e t h e o p t i m u m s t e p l e n g t h.
N e ed s["Optimiz&tionToolbox'Oticonstr&ined1"];
?AnalyticaliiineSearch
A n a ly t i c a lL i n e S e a r c h [ g r a d f, p t, d, v a r s, prResults:True, opts] computes optimum s t e p l e n g t h u s in g a n a l y t i c a l approach, gradf = g r a d ien t of given fu n c t io n, p t = current d e sig n p o i n t, d = d i r e c t i o n v e c t o r, vars = l i s t of v a r i a b l e s. See O ptions[AnalyticalLineSearch] f o r v a l i d o p t io n s and t h e i r usage.
OptionsUsage[AnalyticalLineSearch]
{SecantPoints {0, 0.1} , StepLengthVar a}
S e c a n t P o i n t s i s a n o p t i o n f o r A n a l y t i c a l L i n e S e a r c h. T h i s o p t i o n i s u s e d t o s p e c i f y t w o i n i t i a l v a l u e s t o s t a r t t h e S e c a n t m e t h o d f o r f i n d i n g r o o t. D e f a u l t { 0,.1 } *
S t e p L e n g t h V a r i s a n o p t i o n f o r s e v e r a l u n c o n s t r a i n e d o p t i m i z a t i o n m e t h o d s. I t s p e c i f i e s t h e s y m b o l u s e d f o r s t e p l e n g t h v a r i a b l e.
D e f a u l t i s S t e p L e n g t h V a r - » a.
E x a m p l e 5.4 F o r t h e f o l l o w i n g f u n c t i o n, c o m p u t e o p t i m u m s t e p l e n g t h a l o n g t h e g i v e n d i r e c t i o n u s i n g a n a n a l y t i c a l l i n e s e a r c h.
f = ( x i + X 2 - 1 1 ) 2 + (x± +x2 - 7 ) ; v a r s = { *!, x3 };
xk= { l,2 };d = { 1,2 );
5.2 L i n e S e a r c h T i p n l i T i i q o e s — S t e p L e n g t h C a l c u l a t i o n s
d£ = G r a d [ f, - v a r s ]; Mat ri xFor m[ df ]
ll - 4 4 x 2 + 4 x i + 4 x i x 2 — 22 + 2 x i + 4x2
D e s c e n t D i r e c t i o n C h e c k [ f, x k, d, v a r s ];
y f ->
3 \
1 - 4 4 x i + 4 x i + 4 x i x 2 2
V f {!., 2.} -»
- 22 + 2 x i + 4 x 2 ]
V f.d - » - 5 5. S t a t u s - » D e s c e n t
s t e p = A n a l y t i c a l L i n e S e a r c h [ d f, x k, d, v a r s ] ;
1 + a
x k i
2 + 2 a
, , 1V I- 3 1 - 1 6 a + 2 0 a 2 + 4 a 3 « < Λ 1 > -.( 3 ( - 6 * 6 α * α 2 )
άφ/dxx ξ v f ( x k i ) .d = 0 -> - 5 5 + 8a + 2 4 a 2 + 4 a 3 == 0 a 1.2 4 6 3 9
T h i s s t e p l e n g t h i s t h e s a m e a s t h a t c o m p u t e d i n t h e p r e v i o u s e x a m p l e. E x a m p l e 5.5 F o r t h e f o l l o w i n g f u n c t i o n, c o m p u t e o p t i m u m s t e p l e n g t h a l o n g t h e g i v e n d i r e c t i o n u s i n g a n a n a l y t i c a l l i n e s e a r c h.
f = ( x 1 - l ) 4 + { x2 - 3 ) 2 + 4 ( x 3 + 5 ) 4?
v a r s = { x j , x 2 , x 3 } ;
x k = { 4, 2, - 1 }; d = { 1,2,- 3 >i
d f = G r a d [ f, v a r s ]; H a t r i x F o r a [ d f ]
- 4 + 12x 2 - 12x 2 + 4 x i -6 + 2x 2 12000 + 1200x 3 + 2 4 0 X3 + I 6X3J
D e s c e n t D i r e c t i o n C h e c k [ f, x k, d, v a r s ];
-4 + 12xx - 12xi + 4xi '
' 108 '
Vf -»
- 6 + 2x2
Vf {4., 2., -1.} -»
-2
,2000 + I 2OOX3 + 240x§ + 16x3,
,1024,
V f.d - > - 2 9 6 8. S t a t u s ^ D e s c e n t
s t e p = A n a l y t i c a l L i n e S e a r c h [ d f, x k, d, v a r s ] ;
1 4 + a ' x k i -> 2 + 2a
- 1 - 3a,
' 4 ( 3 + a ) 3
Vf ( x k i ) -» - 2 + 4 a
- 1 6 ( - 4 + 3 a ) 3;
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
άφ/άα s Vf ( x k i ) .d = 0 4 ( - 7 4 2 + 1 7 5 7 α - 1 2 8 7 a 2 + 3 2 5 a 3 ) == 0
a -» 0 . 7 8 0 9 4
n e w p t = x k + s t e p d
{ 4.7 8 0 9 4,3.5 6 1 8 8,- 3.3 4 2 8 2 }
B y e v a l u a t i n g / a t t h e g i v e n p o i n t a n d a t t h i s n e w p o i n t, w e s e e t h a t t h e o b j e c t i v e v a l u e i n d e e d i s r e d u c e d.
£/. { T h r e a d [ - v a r s -► x k ], T h r e a d [ v a r s -* n e w p t ] >
{ 1 1 0 6,2 3 4.8 4 5 }
T h e a n a l y t i c a l m e t h o d r e q u i r e s a n e x p l i c i t e x p r e s s i o n f o r φ(α)
o r άφ/da.
F o r a p u r e l y n u m e r i c a l s o l u t i o n, s u c h e x p r e s s i o n s a r e n o t a v a i l a b l e a n d h e n c e o n e m u s t r e s o r t to
n u m e r i c a l l i n e s e a r c h t e c h n i q u e s.
T h e s i m p l e s t l i n e s e a r c h t e c h n i q u e i s t h e e q u a l i n t e r v a l s e a r c h. I n t h i s a p p r o a c h, b o u n d s a r e f o u n d f o r t h e m i n i m u m οΐφ(α).
B y s u c c e s s i v e l y r e f i n i n g t h e s e b o u n d s, t h e m i n i m u m i s b r a c k e t e d t o a n y d e s i r e d d e g r e e o f p r e c i s i o n. S t a r t i n g f r o m a g i v e n l o w e r b o u n d ( s a y a =
0 ) a n d a n i n t e r v a l s t e p p a r a m e t e r 8
> 0 ( s a y 8 =
0.5 ), w e c a n c o m p u t e t h e b o u n d s a s f o l l o w s:
1. S e t a\
= i n i t i a l v a l u e o f a.
C o m p u t e φ(α\).
2. S e t a
2
=ai+ 8.
C d m p u t e φ(β
2
).
A s i l l u s t r a t e d i n F i g u r e 5.3, o n e o f t h e f o l l o w i n g t w o s i t u a t i o n s i s p o s s i b l e.
5.2.2 E q u a l I n t e r v a l S e a r c h
Φ
Minimum
M i n i m u m
F I G U R E 5.3 I l l u s t r a t i o n o f t w o s i t u a t i o n s o c c u r r i n g d u r i n g i n t e r v a l s e a r c h.
5.2 L i n e S e a r c h T t e c h n i q i i e s — S t e p L e n g t h C a l c u l a t i o n s
3. I f φ(αι
2) < t h e n t h e f u n c t i o n i s c o n t i n u i n g t o d e c r e a s e. T h e m i n i m u m i s e i t h e r b e t w e e n αχ a n d a 2 o r h a s n o t y e t b e e n b r a c k e t e d. T h e r e f o r e, s e t αχ = a
2 a n d g o t o s t e p 2.
4. I f φ(οί
2) > φ(α
i ), w e h a v e g o n e p a s t t h e m i n i m u m. T h e m i n i m u m m u s t b e i n t h e i n t e r v a l j u s t t r i e d o r i n t h e o n e b e f o r e. T h u s, w e h a v e f o u n d t h e u p p e r a n d l o w e r l i m i t s o f t h e i n t e r v a l i n w h i c h t h e m i n i m u m l i e s.
Ofj = 0L\ — 8 < Gf mi n Ξ: au — &2
T h e i n t e r v a l s t e p p a r a m e t e r 8
i s r e d u c e d t o δ/F,
w h e r e F
i s a r e f i n e m e n t p a ­
r a m e t e r ( s a y F
= 1 0 ) a n d t h e p r o c e s s i s r e p e a t e d. T h e c o n v e r g e n c e i s a c h i e v e d w h e n I
= (au — aj)
i s r e d u c e d t o a s p e c i f i e d t o l e r a n c e. A f t e r c o n v e r g e n c e, t h e m i n i m u m v a l u e i s s e t t o t h e a v e r a g e o f t h e u p p e r a n d l o w e r b o u n d s.
au
+ or/ g m i n — ^
T h e p r o c e d u r e i s i m p l e m e n t e d i n t h e f o l l o w i n g f u n c t i o n t h a t i s i n c l u d e d i n t h e O p t i m i z a t i o n'I b o l b o x 'U n c o n s t r a i n e d' p a c k a g e.
N e e d s ["O p t i m i z a t i o n T o o l b o x'U n c o n e t r a i n e d'■ ];
?E o u a l I n t e r v a l S e a r c h
E q u a l l n t e r v a l S e a r c h[φ, a, { a i, 6, F}, t o l:1 0 - 3 ] ------ D e t e r m i n e s mi n i mu m
o f φ ( α ) u s i n g E q u a l I n t e r v a l S e a r c h. A r g u m e n t s h a v e t h e f o l l o w i n g m e a n i n g, φ = f u n c t i o n o f s i n g l e v a r i a b l e a, { a i, <5, F} i n i t i a l v a l u e o f α, δ = i n i t i a l d e l t a, a n d F = r e f i n e m e n t f a c t o r, r e s p e c t i v e l y, t o l ( o p t i o n a l ) = c o n v e r g e n c e t o l e r a n c e ( d e f a u l t i s 1 0 - 3 ). T h e f u n c t i o n r e t u r n s { E s t i m a t e d mi n i mu m p o i n t. I n t e r v a l c o n t a i n i n g mi n i mu m}.
E x a m p l e 5.6 D e t e r m i n e a m i n i m u m o f t h e f o l l o w i n g f u n c t i o n u s i n g e q u a l i n t e r v a l s e a r c h:
Φ = 1 - 1/( 1- α + 2α2);
A p l o t o f t h e f u n c t i o n φ
i s s h o w n i n F i g u r e 5.4. T h e p l o t s h o w s t h a t a l o c a l m i n i m u m e x i s t s n e a r a =
0.2 5.
W e u s e E q u a l l n t e r v a l S e a r c h f u n c t i o n t o c o m p u t e t h e m i n i m u m, s t a r t i n g w i t h or = 0, 8 =
0.5, F
= 5, a n d t o l = 0.0 1.
Equ alln tervalS earch [φ, a, { 0, 0.5, 5 }, 0.0 1 ]
δ ^ 0.5
a
φ ( α )
0
0
0.5
0.
1.
0.5
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
Φ
Bounds { 0., 1. }
<5->0.1
a
Φ
( a )
0.
0.
0.1
-0.0869565
0.2
-0.136364
0.3
-0.136364
0.4
-0.0869565
Bounds -»
{ 0.2,0.4 }
δ-* 0.0 2
a
φ ( α )
0.2
- 0.1 3 6 3 6 4
0.2 2
- 0.1 4 0 5 1 1
0.2 4
- 0.1 4 2 5 9 6
0.2 6
- 0.1 4 2 5 9 6
0.2 8
- 0.1 4 0 5 1 1
B o u n d s -* { 0.2 4, 0.2 8 }
<5 ->0.004
α φ (a)
0.2 4------------- 0.1 4 2 5 9 6 ---------------------------------------------------------------:---------------------------------------------------------------
0.2 4 4 - 0.1 4 2 7 6 3
0.2 4 8 - 0.1 4 2 8 4 7
0.2 5 2 - 0.1 4 2 8 4 7
0.2 5 6 - 0.1 4 2 7 6 3
B o u n d s -» { 0.2 4 8, 0.2 5 6 }
{ 0.2 5 2,{ 0.2 4 8,0.2 5 6 } }
E x a m p l e 5.7 D e t e r m i n e a m i n i m u m o f t h e f o l l o w i n g f u n c t i o n u s i n g e q u a l i n t e r v a l s e a r c h:
Φ = 2 - 4 a + Exp [ a ] ?
5.2 L i n e S e a r c h T w h n i q u e s — S t e p L e n g t h C a l c u l a t i o n s
A p l o t o f t h e f u n c t i o n φ
i s s h o w n i n F i g u r e 5.5. T h e p l o t s h o w s t h a t a l o c a l m i n i m u m e x i s t s n e a r a =
1.4.
Φ
F IG U R E 5.5 P lo t o f 2 — 4α + E xp[a].
W e u s e E q u a l l n t e r v a l S e a r c h f u n c t i o n t o c o m p u t e t h e m i n i m u m, s t a r t i n g w i t h α = 0, δ = 0.5, F
= 1 0, a n d t o l = 0.0 1.
E3u&lIn tervalSearch[ 0, a, { 0., 0 .5, 1 0.}, .0 1 ]
(5 - » 0 . 5
α φ ( a )
0. 3.
0.5 1.6 4 8 7 2
1. 0.7 1 8 2 8 2
1.5 0.4 8 1 6 8 9
2. 1.3 8 9 0 6
B o u n d s -» { 1., 2 . }
6 - > 0.0 5
α φ (a)
1. 0.7 1 8 2 8 2
1.θ έ 0.6 5 7 6 5 1
1.1 0.6 0 4 1 6 6
1.1 5 0.5 5 8 1 9 3
1.2 0.5 2 0 1 1 7 1.2 5 0.4 9 0 3 4 3
1.3 0.4 6 9 2 9 7
1.3 5 0.4 5 7 4 2 6
1.4 0.4 5 5 2
1.4 5 0.4 6 3 1 1 5
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
Bounds { 1.3 5, 1.4 5 } δ 0.0 0 5
α Φ
(Q )
1.35 0.457426
1.355 0.456761
1.36 0.456193
1.365 0.455723
1.37 0.455351
1.375 0.455077
1.38 0.454902
1.385 0.454826
1.39 0.45485
Bounds-» { 1.3 8, 1.39} {1.385, { 1.3 8, 1.39}}
5.2.3 S e c t i o n S e a r c h
T h e e q u a l i n t e r v a l, s e a r c h i s s t r a i g h t f o r w a r d b u t r e q u i r e s a l a r g e n u m b e r o f f u n c t i o n e v a l u a t i o n s b e f o r e i t l o c a t e s a m i n i m u m. A c o n s i d e r a b l y m o r e e f ­
f i c i e n t p r o c e d u r e i s t o u s e s e c t i o n s e a r c h i n w h i c h t h e i n i t i a l s e t o f b o u n d s ( α
i, au)
a r e c o m p u t e d u s i n g t h e s a m e p r o c e d u r e a s t h e e q u a l i n t e r v a l s e a r c h. H o w e v e r, t h e b o u n d s a r e t h e n r e f i n e d b y t a k i n g t w o p o i n t s, d e n o t e d b y aa
a n d a bt
w i t h i n t h e s e b o u n d s a n d e v a l u a t i n g t h e f u n c t i o n a t t h e s e t w o p o i n t s. I f t h e i n t e r m e d i a t e p o i n t s a r e p l a c e d a t t h e t h i r d p o i n t s b e t w e e n t h e b o u n d s, t h e n
aa
= aj
+ (
au — acj)
/3 ab — aj
+ 2 (
au
— ai)
/3 A s i l l u s t r a t e d i n F i g u r e 5.6, o n e o f t h e f o l l o w i n g t w o s i t u a t i o n s i s p o s s i b l e.
1. I f φ(αα) < φ(ab),
t h e n i t i s c l e a r f r o m t h e f i g u r e t h a t t h e m i n i m u m m u s t b e i n t h e i n t e r v a l (ar/, ab).
2. I f φ(αα)
> Φ(αι,),
t h e n t h e m i n i m u m m u s t b e i n t h e i n t e r v a l (aa, au).
T h u s, w e h a v e f o u n d n e w b o u n d s f o r t h e m i n i m u m. T h e p r o c e s s c a n n o w b e r e p e a t e d w i t h t h e s e n e w l o w e r a n d u p p e r l i m i t s. C l e a r l y, a t e a c h s t e p t h e i n t e r v a l i n w h i c h t h e m i n i m u m l i e s i s r e d u c e d b y 1/3.
A s b e f o r e, t h e c o n v e r g e n c e i s a c h i e v e d w h e n I = (au — a
/) i s r e d u c e d t o a s p e c i f i e d t o l e r a n c e. A f t e r c o n v e r g e n c e, t h e m i n i m u m v a l u e i s s e t t o t h e
5.2 L i n e S e a r c h T W i V n v i q u e s — S t e p L e n g t h C a l c u l a t i o n s
F I G U R E 5.6 I l l u s t r a t i o n o f t w o s i t u a t i o n s o c c u r r i n g d u r i n g s e c t i o n s e a r c h.
a v e r a g e o f t h e u p p e r a n d l o w e r b o u n d s.
a u + ot;
a min = “
T h e p r o c e d u r e i s i m p l e m e n t e d i n t h e f o l l o w i n g f u n c t i o n t h a t i s i n c l u d e d i n t h e O p t i m i z a t i o n T b o l b o x 'U n c o n s t r a i n e d' p a c k a g e.
N e e d s ["O p t i m i z a t i o n T o o l b o x'U n c o n s t r a i n e d'"];
?S e c t i o n S e a r c h
S e c t i o n S e a r c h [ $, a, { a i, <5 },t o l:l 0 - 3 ] D e t e r m i n e s mi n i mu m o f φ
( a ) u s i n g S e c t i o n S e a r c h. A r g u m e n t s h a v e t h e f o l l o w i n g m e a n i n g, φ = f u n c t i o n o f s i n g l e v a r i a b l e a, { a i, δ } i n i t i a l v a l u e o f α, δ = i n i t i a l d e l t a, t o l ( o p t i o n a l ) = c o n v e r g e n c e t o l e r a n c e ( d e f a u l t i s I O- 3 ).
T h e f u n c t i o n r e t u r n s { E s t i m a t e d mi n i mu m p o i n t, I n t e r v a l c o n t a i n i n g mi n i mu m}.
E x a m p l e 5.8 D e t e r m i n e a m i n i m u m o f t h e f o l l o w i n g f u n c t i o n u s i n g s e c t i o n s e a r c h:
Φ- 1 - 1/( 1 - a + 2 a 2 ) ;
W e u s e S e c t i o n S e a r c h f u n c t i o n t o c o m p u t e t h e m i n i m u m s t a r t i n g w i t h a =
0, S
= 0.1, a n d t o l = 0.0 1.
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
S e c t i o n S e a r c h f#, a, { 0, 0.1 }, 0.0 1 ]
******** Boundi ng p h a s e ********
δ ^ 0.1
α φ ( a )
0 0
0.1 - 0.0 8 6 9 5 6 5 0.2 - 0.1 3 6 3 6 4
0.3 - 0.1 3 6 3 6 4
0.4 - 0.0 8 6 9 5 6 5
B o u n d s { 0.2, 0.4 }
* * * * * * * * R e f i n e m e n t p h a s e * * * * * * * *
«L 0 - 2
0.2 0 - 2 0.2 2 9 6 3 0.2 2 9 6 3 0.2 4 2 7 9 8 0.2 4 2 7 9 8 0.2 4 2 7 9 8 0.2 4 6 7
Qu
0.4
0.3 3 3 3 3 3
0.2 8 8 8 8 9
0.2 8 8 8 8 9
0.269136
0.2 6 9 1 3 6
0.2 6 0 3 5 7
0.2 5 4 5 0 4
0.2 5 4 5 0 4
“ a
0.2 6 6 6 6 7
0.2 4 4 4 4 4
0.2 2 9 6 3
0.2 4 9 3 8 3
0.2 4 2 7 9 8
0.2 5 1 5 7 8
0.2 4 8 6 5 1
0.2 4 6 7
0.2 4 9 3 0 1
« b
0.3 3 3 3 3 3 0.2 8 8 8 8 9 0.2 5 9 2 5 9 0.2 6 9 1 3 6 0.2 5 5 9 6 7 0.2 6 0 3 5 7 0.2 5 4 5 0 4 0 .2 5 0 6 0 2 0.2 5 1 9 0 3
Φ
( a a )
- 0.1 4 2 1 3 2 - 0.1 4 2 7 7 7 - 0.1 4 1 7 7 4 - 0.1 4 2 8 5 6 - 0.1 4 2 7 2 2 - 0.1 4 2 8 5 1 - 0.1 4 2 8 5 2 - 0.1 4 2 8 2 9 - 0.1 4 2 8 5 6
Φ
( a b ) - 0.1 2 5 - 0.1 3 8 9 2 - 0.1 4 2 6 3 3 - 0.1 4 1 9 0 1 - 0.1 4 2 7 6 4 - 0.1 4 2 5 7 7 - 0.1 4 2 8 0 4 - 0.1 4 2 8 5 6 - 0.1 4 2 8 4 8
I
0.2
0.1 3 3 3 3 3
0.0 8 8 8 8 8 9
0.0 5 9 2 5 9 3
0.0 3 9 5 0 6 2
0.0 2 6 3 3 7 4
0.0 1 7 5 5 8 3
0.0 1 1 7 0 5 5
0.0 0 7 8 0 3 6 9,
{ 0.2 4 9 3 0 1,{ 0.2 4 6 7,0.2 5 1 9 0 3 } }
E x a m p l e 5.9 D e t e r m i n e t h e m i n i m u m o f t h e f o l l o w i n g f u n c t i o n u s i n g s e c ­
t i o n s e a r c h.
φ = 2 - 4 a + E x p [ a ];
W e u s e S e c t i o n S e a r c h f u n c t i o n t o c o m p u t e t h e m i n i m u m s t a r t i n g w i t h a
= 0, δ = 0.5, a n d t o l = 0.0 1.
Se ctionSearch [φ, a, {0, 0.5 }, 0.01]
******** Bounding phase ********
<5 ->0.5
a
φ(<χ)
0
3
0.5
1.64872
1.
0.718282
1.5
0.481689
2.
1.38906
Bounds -> { 1., 2 .}
******** R e f i n e m e n t p h a s e ********
' a L
aU
a a
«b
Φ
( a a )
Φ
( a b )
I
1.
2.
1.3 3 3 3 3
1.6 6 6 6 7
0.4 6 0 3 3 5
0.6 2 7 8 2 3
1.
1.
1.6 6 6 6 7
1.2 2 2 2 2
1.4 4 4 4 4
0.5 0 5 8 3 4
0.4 6 1 7 1 8
0.6 6 6 6 6 7
1.2 2 2 2 2
1.6 6 6 6 7
1.3 7 0 3 7
1.5 1 8 5 2
0 .4 5 5 3 2 7
0.4 9 1 3 8 2
0.4 4 4 4 4 4
1.2 2 2 2 2
1.5 1 8 5 2
1.3 2 0 9 9
1.4 1 9 7 5
0.4 6 3 1 7
0.4 5 7 0 8 7
0.2 9 6 2 9 6
1.3 2 0 9 9
1.5 1 8 5 2
1.3 8 6 8 3
1.4 5 2 6 7
0.4 5 4 8 2 3
0.4 6 3 8 3 4
0.1 9 7 5 3 1
1.3 2 0 9 9
1.4 5 2 6 7
1.3 6 4 8 8
1.4 0 8 7 8
0.4 5 5 7 3 3
0.4 5 5 8 4 1
0.1 3 1 6 8 7
1.3 2 0 9 9
1.4 0 8 7 8
1.3 5 0 2 5
1.3 7 9 5 2
0.4 5 7 3 9
0.4 5 4 9 1 4
0.0 8 7 7 9 1 5
1.3 5 0 2 5
1.4 0 8 7 8
1.3 6 9 7 6
1.3 8 9 2 7
0.4 5 5 3 6 6
0 .4 5 4 8 4
0.0 5 8 5 2 7 7
1.3 6 9 7 6
1.4 0 8 7 8
1.3 8 2 7 7
1.3 9 5 7 7
0.4 5 4 8 4 7
0.4 5 5 0 0 3
0.0 3 9 0 1 8 4
1.3 6 9 7 6
1.3 9 5 7 7
1.3 7 8 4 3
1.3 8 7 1
0.4 5 4 9 4 6
0.4 5 4 8 2 4
0.0 2 6 0 1 2 3
1.3 7 8 4 3
1.3 9 5 7 7
1.3 8 4 2 1
1.3 8 9 9 9
0.4 5 4 8 3 1
0.4 5 4 8 5
0.0 1 7 3 4 1 5
1.3 7 8 4 3
1.3 8 9 9 9
1.3 8 2 2 9
1.3 8 6 1 4
0.4 5 4 8 5 5
0.4 5 4 8 2 3
0.0 1 1 5 6 1
.1.3 8 2 2 9
1.3 8 9 9 9
1.3 8 4 8 5
1.3 8 7 4 2
0.4 5 4 8 2 6 7
0.4 5 4 8 2 5
0.0 0 7 7 0 7 3 5
1.3 8 7 4 2,
{ 1.3 8 4 8 5,
1.3 8 9 9 9 } }
5.2.4 T h e G o l d e n S e c t i o n S e a r c h
T h e g o l d e n s e c t i o n s e a r c h i s s i m i l a r t o t h e b a s i c s e c t i o n s e a r c h b u t w i t h t w o m a i n d i f f e r e n c e s t h a t m a k e i t a v e r y c l e v e r i n t e r v a l s e a r c h m e t h o d. I n t h e b o u n d i n g p h a s e, i n s t e a d o f a d d i n g a f i x e d 5 t o t h e p r e v i o u s a, t h e a
.2 i s c o m p u t e d a s f o l l o w s: «2 = « ι + r n _ 1 5 w h e r e n
i s a n i t e r a t i o n c o u n t e r ( n = 1,2,...) a n d r i s t h e g o l d e n r a t i o g i v e n b y
1 +
τ
= ----- — % 1.6 1 8
2 S i n c e τ i s g r e a t e r t h a n 1, t h e s t e p s g e t l a r g e r a n d l a r g e r a s t h e i t e r a t i o n p r o ­
g r e s s e s. T h u s, t h e b o u n d s a r e l o c a t e d q u i c k l y r e g a r d l e s s o f h o w s m a l l 5 v a l u e i s s e l e c t e d. T h e c o m p l e t e b o u n d i n g p h a s e a l g o r i t h m i s a s f o l l o w s: G i v e n i n t e r v a l s t e p p a r a m e t e r δ > 0 ( s a y 8
= 0.5 )
1. S e t a\
= i n i t i a l v a l u e o f a.
C o m p u t e φ{α\).
S e t n = 1 a n i t e r a t i o n c o u n t e r.
2. S e t of2 = αχ + r ” _ 1 5. C o m p u t e φ(α
2).
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
3. I f φ(οί
2
) < Φ(α
i ), t h e m i n i m u m h a s n o t b e e n s u r p a s s e d. T h e r e f o r e, s e t a i = a
.2 a n d g o t o s t e p 2. O t h e r w i s e, t h e m i n i m u m h a s b e e n s u r p a s s e d a n d m u s t l i e i n t h e f o l l o w i n g i n t e r v a l:
I n t h e r e f i n e m e n t p h a s e, t h e t w o i n t e r m e d i a t e p o i n t s a r e p l a c e d a s f o l l o w s:
A s w i l l b e s e e n f r o m t h e n u m e r i c a l e x a m p l e s, t h e a d v a n t a g e o f t h i s s c h e m e i s t h a t o n e o f t h e t w o i n t e r m e d i a t e p o i n t s i s a l w a y s t h e s a m e a s t h e o n e u s e d i n t h e p r e v i o u s i t e r a t i o n. T h u s, e a c h i t e r a t i o n r e q u i r e s o n l y o n e n e w f u n c t i o n e v a l u a t i o n.
A s b e f o r e, t h e c o n v e r g e n c e is. a c h i e v e d w h e n I
= (au —
ar/) i s r e d u c e d t o a s p e c i f i e d t o l e r a n c e. A f t e r c o n v e r g e n c e, t h e m i n i m u m v a l u e i s s e t t o t h e a v e r a g e o f t h e u p p e r a n d l o w e r b o u n d s.
T h e p r o c e d u r e i s i m p l e m e n t e d i n t h e f o l l o w i n g f u n c t i o n t h a t i s i n c l u d e d i n
t h e O p t i m i z a t i o n T b o l b o x 'U n c o n s t r a i n e d' p a c k a g e:
Needs["Opt imi zat ionToolbox'Unconstrained'"];
?GoldenSectionsearch
GoldenSectionSearch[0, a, {ai, δ },t o l:1 0 - 3 ] ------ D e t e r m i n e s mi n i mu m
o f φ ( a ) u s i n g G o l d e n S e c t i o n S e a r c h. A r g u m e n t s h a v e t h e f o l l o w i n g m e a n i n g, φ = f u n c t i o n o f s i n g l e v a r i a b l e a, { a i, δ } i n i t i a l v a l u e o f α, δ = i n i t i a l d e l t a, t o l ( o p t i o n a l ) = c o n v e r g e n c e t o l e r a n c e ( d e f a u l t i s 1 0 - 3 ). T h e f u n c t i o n r e t u r n s { E s t i m a t e d mi n i mu m p o i n t, I n t e r v a l c o n t a i n i n g mi n i mu m}.
E x a m p l e 5.1 0 D e t e r m i n e a m i n i m u m o f t h e f o l l o w i n g f u n c t i o n u s i n g t h e
g o l d e n s e c t i o n s e a r c h:
Φ = l - l/( l - a + 2a2 );
W e u s e G o l d e n S e c t i o n S e a r c h f u n c t i o n t o c o m p u t e t h e m i n i m u m s t a r t i n g w i t h a = 0, 8
= 0.1, a n d t o l = 0.0 1.
ai = ai — 8 < «min < au
= ot2
au+a\
amin — ~
GoldenSectionSearch[φ, a, { 0, 0.1 }/0.0 1 ]
5.2 l i n e S e a r c h T V ^ V i n i g u e s — S t e p L e n g t h C a l c u l a t i o n s
******** Boundi ng p h a s e ******** 5 - » 0.1
α Φ(α)
0 0
0.1 - 0.0 8 6 9 5 6 5
0.2 6 1 8 0 3 - 0.1 4 2 4 9 3
0.5 2 3 6 0 7 0.0 2 4 1 2 5
B o un ds ^ { 0.1, 0.5 2 3 6 0 7 }
******** R e f i n e me n t p h a s e ********
“ L
a U
a a
a b
φ ( a a )
Φ ( a b )
I
0.1
0.5 2 3 6 0 7
0.2 6 1 8 0 3
0.3 6 1 8 0 3
- 0.1 4 2 4 9 3
- 0.1 1 1 1 1 1
0.4 2 3 6 0 7
0.1
0.3 6 1 8 0 3
0.2
0.2 6 1 8 0 3
- 0.1 3 6 3 6 4
- 0.1 4 2 4 9 3
0.2 6 1 8 0 3
0.2
0.3 6 1 8 0 3
0.2 6 1 8 0 3
0.3
- 0.1 4 2 4 9 3
- 0.1 3 6 3 6 4
0.1 6 1 8 0 3
0.2
0.3
0.2 3 8 1 9 7
0.2 6 1 8 0 3
- 0.1 4 2 4 9 3
- 0.1 4 2 4 9 3
0.1
0.2 3 8 1 9 7
0.3
0.2 6 1 8 0 3
0.2 7 6 3 9 3
- 0.1 4 2 4 9 3
- 0.1 4 1 0 4
0.0 6 1 8 0 3 4
0.2 3 8 1 9 7
0.2 7 6 3 9 3
0.2 5 2 7 8 6
0.2 6 1 8 0 3
- 0.1 4 2 8 3 7
- 0.1 4 2 4 9 3
0.0 3 8 1 9 6 6
0.2 3 8 1 9 7
0.2 6 1 8 0 3
0.2 4 7 2 1 4
0.2 5 2 7 8 6
- 0.1 4 2 8 3 7
- 0.1 4 2 8 3 7
0.0 2 3 6 0 6 8
0.2 4 7 2 1 4
0.2 6 1 8 0 3
0.2 5 2 7 8 6
0.2 5 6 2 3 1
- 0.1 4 2 8 3 7
- 0.1 4 2 7 5 6
0.0 1 4 5 8 9 8
0.2 4 7 2 1 4
0.2 5 6 2 3 1
0.2 5 0 6 5 8
0.2 5 2 7 8 6
- 0.1 4 2 8 5 6
- 0.1 4 2 8 3 7
0.0 0 9 0 1 6 9 9
{ 0.2 5 1 7 2 2,{ 0.2 4 7 2 1 4,0.2 5 6 2 3 1 } }
N o t i c e t h a t o n e o f t h e t w o i n t e r m e d i a t e p o i n t s (
aa
o r ay)
i s a l w a y s t h e s a m e a s t h e o n e u s e d i n t h e p r e v i o u s i t e r a t i o n. T h u s, e a c h i t e r a t i o n r e q u i r e s o n l y o n e n e w f u n c t i o n e v a l u a t i o n. A l s o, n o t i c e t h a t t h e o v e r a l l n u m b e r o f i t e r a t i o n s i s s m a l l e r t h a n t h e e a r l i e r m e t h o d s.
E x a m p l e 5.1 1 D e t e r m i n e a m i n i m u m o f t h e f o l l o w i n g f u n c t i o n u s i n g t h e g o l d e n s e c t i o n s e a r c h.
Φ = 2 - 4a + E x p [ a ] ;
W e u s e G o l d e n S e c t i o n S e a r c h f u n c t i o n t o c o m p u t e t h e m i n i m u m s t a r t i n g w i t h a =
0, S
= 0.5, a n d t o l = 0.0 1.
GoldenSectionSearch [φ, a, { 0, 0.5 }, 0.0 1 ]
* * * * * * * * B o u n d i n g p h a s e * * * * * * * * δ - > 0.5
α φ ( a )
0 3
0.5 1.6 4 8 7 2
1.3 0 9 0 2 0.4 6 6 4 6 4
2.6 1 8 0 3 5.2 3 6 6 1
Bounds {0.5, 2.61803}
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
******** R e f i n e m e n t p h a s e * * * * * * * *
»L
a U
“ a
a b
Φ
( a a )
Φ
( a b )
I
0.5
2.6 1 8 0 3
1.3 0 9 0 2
1.8 0 9 0 1
0.4 6 6 4 6 4
0.8 6 8 3 7 6
2.1 1 8 0 3
in
o
1.8 0 9 0 2
1.
1.3 0 9 0 1
0.7 1 8 2 8 2
0.4 6 6 4 6 4
1.3 0 9 0 2
1.
1.8 0 9 0 2
1.3 0 9 0 2
1.4 9 9 9 9
0.4 6 6 4 6 4
0.4 8 1 6 8 9
0.8 0 9 0 1 7
1.
1.5
1.1 9 0 9 8
1.3 0 9 0 1
0.5 2 6 3 8 2
0.4 6 6 4 6 4
0.5
1 1 9 0 9 8
1 5
1 . "=*0902
1 r^ R1 9 6
0,4 6 6 4 6 4
0. 4*^486
0.3 0 9 0 1 7
1.3 0 9 0 2
1.5
1.3 8 1 9 7
1.4 2 7 0 5
0.4 5 4 8 6
0.4 5 8 1 9
0.1 9 0 9 8 3
1.3 0 9 0 2
1.4 2 7 0 5
1.3 5 4 1
1.3 8 1 9 6
0.4 5 6 8 7 3
0.4 5 4 8 6
0.1 1 8 0 3 4
1.3 5 4 1
1.4 2 7 0 5
1.3 8 1 9 7
1.3 9 9 1 8
0.4 5 4 8 6
0.4 5 5 1 5 6
0.0 7 2 9 4 9
1.3 5 4 1
1.3 9 9 1 9
1.3 7 1 3 2
1.3 8 1 9 6
0.4 5 5 2 6 9
0.4 5 4 8 6
0.0 4 5 0 8 5
1.3 7 1 3 2
1.3 9 9 1 9
1.3 8 1 9 7
1.3 8 8 5 4
0.4 5 4 8 6
0.4 5 4 8 3 3
0.0 2 7 8 6 4
1.3 8 1 9 7
1.3 9 9 1 9
1.3 8 8 5 4
1.3 9 2 6 0
0.4 5 4 8 3 3
0.4 5 4 9 0 2
0.0 1 7 2 2 0 9
1.3 8 1 9 7
1.3 9 2 6 1
1.3 8 6 0 3
1.3 8 8 5 4
0.4 5 4 8 2 3
0.4 5 4 8 3 3
0.0 1 0 6 4 3 1
1.3 8 1 9 7
1.3 8 8 5 4
1.3 8 4 4 8
1.3 8 6 0 3
0.4 5 4 8 2 9
0.4 5 4 8 2 3
0.0 0 6 5 7 7 8 1,
{ 1.3 8 5 2 5, { 1.3 8 1 9 7, 1.3 8 8 5 4 } }
5.2.5 T h e Q u a d r a t i c I n t e r p o l a t i o n M e t h o d
A U t h e i n t e r v a l s e a r c h m e t h o d s d i s c u s s e d s o f a r c o m p u t e f u n c t i o n v a l u e s a t p r e d e t e r m i n e d l o c a t i o n s w i t h o u t e x p l i c i t l y c o n s i d e r i n g t h e f o r m o f t h e f u n c ­
t i o n i t s e l f, a n d t h u s, c o u l d b e v e r y s l o w t o c o n v e r g e. A n o t h e r c l a s s o f m e t h o d s k n o w n a s i n t e r p o l a t i o n m e t h o d s a r e b a s e d o n f i t t i n g a p o l y n o m i a l f u n c t i o n t h r o u g h a g i v e n n u m b e r o f p o i n t s. A s t h e n a m e i n d i c a t e s, t h e q u a d r a t i c i n t e r ­
p o l a t i o n m e t h o d u s e s t h r e e g i v e n p o i n t s a n d f i t s a q u a d r a t i c f u n c t i o n t h r o u g h t h e s e p o i n t s. T h e m i n i m u m o f t h i s q u a d r a t i c f u n c t i o n i s c o m p u t e d u s i n g n e c ­
e s s a r y c o n d i t i o n s. A n e w s e t o f t h r e e p o i n t s i s s e l e c t e d b y c o m p a r i n g f u n c t i o n v a l u e s a t t h i s m i n i m u m p o i n t w i t h t h e g i v e n t h r e e p o i n t s. T h e d e c i s i o n p r o c e s s i s s i m i l a r t o t h e o n e u s e d i n t h e s e c t i o n s e a r c h. T h e p r o c e s s i s r e p e a t e d w i t h t h e t h r e e n e w p o i n t s u n t i l t h e i n t e r v a l i n w h i c h t h e m i n i m u m l i e s b e c o m e s f a i r l y s m a l l. S i m i l a r t o t h e g o l d e n s e c t i o n s e a r c h, t h i s m e t h o d r e q u i r e s o n l y o n e n e w f u n c t i o n e v a l u a t i o n a t e a c h i t e r a t i o n. A s t h e i n t e r v a l b e c o m e s s m a l l, t h e q u a d r a t i c a p p r o x i m a t i o n b e c o m e s c l o s e r t o t h e a c t u a l f u n c t i o n, w h i c h s p e e d s u p c o n v e r g e n c e.
M a i n S t e p
A s s u m i n g t h a t w e a r e g i v e n t h r e e p o i n t s ar/, arm, a n d oru, a q u a d r a t i c f u n c t i o n 4>q p a s s i n g t h r o u g h t h e c o r r e s p o n d i n g f u n c t i o n v a l u e s φ ι, φ τη, a n d (f>u i s g i v e n
5.2 L i n e S e a r r - > i 'w - r f l n r i g u e s — S t e p L e n g t h C a l c u l a t i o n s
b y t h e f o l l o w i n g e q u a t i o n:
(a
- am) (a - a
u)
( a - a f) ( a - a
u)
T h e n e c e s s a i y c o n d i t i o n f o r t h e m i n i m u m o f t h i s q u a d r a t i c f u n c t i o n i s
S o l v i n g t h i s e q u a t i o n f o r a, w e g e t t h e f o l l o w i n g m i n i m u m p o i n t d e n o t e d b y
K n o w i n g t h e m i n i m u m, t h e n e x t t a s k i s t o d e t e r m i n e w h i c h o n e o f t h e g i v e n t h r e e p o i n t s t o d i s c a r d b e f o r e r e p e a t i n g t h e p r o c e s s. W e h a v e o n e o f t h e f o l ­
l o w i n g t w o s i t u a t i o n s:
1- <*q
< am
I f φ(am)
> 0 ( a?), t h e n t h e m i n i m u m o f t h e a c t u a l f u n c t i o n i s i n t h e i n t e r v a l ( a/, am);
t h e r e f o r e, w e u s e (ai, aq, am)
a s t h e t h r e e p o i n t s f o r t h e n e x t i t e r a t i o n.
I f φ(ατη)
< 0 ( a 9), t h e n t h e m i n i m u m o f t h e a c t u a l f u n c t i o n i s i n t h e i n t e r v a l (aq, au);
t h e r e f o r e, w e u s e (aq, am, au)
a s t h e t h r e e p o i n t s f o r t h e n e x t i t e r a t i o n.
2. a q > a m
I f 0 ( a w ) > φ(α9),
t h e n t h e m i n i m u m o f t h e a c t u a l f u n c t i o n i s i n t h e i n t e r v a l (am, au);
t h e r e f o r e, w e u s e (
am, aq, au)
a s t h e t h r e e p o i n t s f o r t h e n e x t i t e r a t i o n.
I f φ(αη) < φ(α9),
t h e n t h e m i n i m u m o f t h e a c t u a l f u n c t i o n i s i n t h e i n t e r v a l ( a/, aq);
t h e r e f o r e, w e u s e ( a/, am, aq)
a s t h e t h r e e p o i n t s f o r t h e n e x t i t e r a t i o n.
d<bn 2a — am
— 2 a — at — au 2 a — at — a
«
1
0/(<
<*l ~ <*l)
+ 0m (aj
~ af)
+ φ» (af
- a j
Φι
(am - au) + φ„
(
au - aj)
+ φν (a\ - am)
T W o o f t h e s e f o u r s i t u a t i o n s a r e i l l u s t r a t e d i n F i g u r e 5.7. T h e i n t e r p o l a t e d q u a d r a t i c f u n c t i o n i s s h o w n i n t h e l i g h t e r s h a d e.
C h a p t e r 5
Φ
C o n v e r g e n c e
A s t h e i n t e r v a l i n w h i c h t h e m i n i m u m l i e s b e c o m e s s m a l l e r, t h e q u a d r a t i c f u n c t i o n b e c o m e s c l o s e r t o t h e a c t u a l f u n c t i o n. A n i n d i c a t i o n o f t h i s i s t h a t a t t h e m i n i m u m p o i n t o f t h e q u a d r a t i c f u n c t i o n (aq),
b o t h t h e q u a d r a t i c f u n c t i o n a n d t h e a c t u a l f u n c t i o n h a v e n e a r l y i d e n t i c a l v a l u e s. U s i n g t h i s a s c o n v e r g e n c e c r i t e r i a, t h e p r o c e s s i s t e r m i n a t e d w h e n t h e f o l l o w i n g c o n d i t i o n i s s a t i s f i e d:
Abs
<t>q (dq) ~ Φ K ) Φ K )
w h e r e t o l i s a s m a l l c o n v e r g e n c e t o l e r a n c e.
< t o l
C h o o s i n g T h r e e I n i t i a l P o i n t s a n d E s t a b l i s h i n g B o u n d s
T h e p r o c e d u r e i s e s s e n t i a l l y c o m p l e t e, e x c e p t f o r t h e c h o i c e o f t h r e e i n i t i a l p o i n t s. C h o o s i n g a n a r b i t r a r y t h r e e v a l u e s o f a
m a y c a u s e p r o b l e m s i f t h e d e n o m i n a t o r o f t h e aq
e q u a t i o n i s 0. T b s e e t h e s i t u a t i o n m o r e c l e a r l y, a s s u m e t h a t t h e t h r e e p o i n t s a r e c h o s e n a s 0, 5, a n d 2 5, w h e r e 5 i s a c h o s e n p a r a m e t e r ( s a y S
= 1 ). I n t h i s c a s e, t h e e x p r e s s i o n f o r aq
t a k e s t h e f o l l o w i n g f o r m:
α — (30? — 4 φγη + Φν) δ Uq~ 2φι-4φηι + 2φκ
F o r t h e d e n o m i n a t o r t o b e g r e a t e r t h a n 0, w e m u s t h a v e
. (hi -Ι- φι*
2 φι — Αφη
+ 2φν
> 0 o r — ^ > Φ**
5.2 L i n e S e a r r V i i f e ^ h n i g u e s — S t e p L e n g t h C a l c u l a t i o n s
K e e p i n g t h i s c o n d i t i o n i n m i n d, t h e f o l l o w i n g p r o c e d u r e i s u s e d t o e s t a b l i s h t h r e e i n i t i a l p o i n t s. I n a d d i t i o n, t h e p r o c e d u r e e s t a b l i s h e s u p p e r a n d l o w e r b o u n d s f o r t h e m i n i m u m.
1.
C h o o s e i n i t i a l s t e p 5 ( s a y 5 =
1 ). S e t a j = 0. C o m p u t e φ(α\).
S e t αχ — 8.
C o m p u t e φ ( α ι ).
2. I f φ(<*ι) >
t h e n t h e m i n i m u m m u s t b e a t a p o i n t l e s s t h a n α χ
.
T t y αχ a s t h e t h i r d p o i n t, s e t au
— α χ, a n d φ(au)
= φ(α\).
S e t t h e m i d d l e p o i n t a t am
— 5/2, c o m p u t e φ(αη).
G o t o s t e p ( 6 ).
3. I f φ(α\)
< φ(αι),
t h e n t h e m i n i m u m i s n o t b r a c k e t e d. W e c h o o s e αχ a s t h e m i d d l e p o i n t, s e t am = a\,
a n d 0 ( a m) = φ (αχ).
F o r l o c a t i n g t h e t h i r d p o i n t, s e t ct
2
=
2 5, c o m p u t e φ(α
2), a n d c o n t i n u e t o t h e n e x t s t e p.
4. I f φ(α
2> > Φ(ο
f i ), t h e n w e h a v e l o c a t e d t h e c o r r e c t t h i r d p o i n t. S e t au
= ct
2
,
a n d φ(αν)
= φ(ct
2
),
a n d g o t o s t e p ( 6 ).
5. I f φ(<*
2
)
< Φ(<*ι),
t h e n t h e m i n i m u m i s s t i l l n o t b r a c k e t e d. S e t ατχ — « 2, a n d φ(α
χ ) = φ(α
2>, s e t 5 = 2 5, a n d g o b a c k t o s t e p ( 2 ).
6 i f φι-tyu
> w e a r e d o n e. O t h e r w i s e, s e t 5 = 2 5, s e t αχ = 5, c o m p u t e 0 ( α χ ), a n d g o b a c k t o s t e p ( 2 ).
T h e c o m p l e t e s o l u t i o n p r o c e d u r e i s i m p l e m e n t e d i n t h e f o l l o w i n g f u n c t i o n t h a t i s i n c l u d e d i n t h e O p t i m i z a t i o n T b o l b o x 'U n c o n s t r a i n e d' p a c k a g e:
Needs["OptimizationToolbox'Onconstrained1"];
?QuadraticSe&rch
QuadraticSearch[0, a,
5:1, m a x l t e r:2 0, t o l:0.0 0 1, p r R e s u i t s:T r u e ] ----
Determines minimum o f 0(a) u s in g Quadratic I n t e r p o l a t i o n method, 0 = f u n c t i o n o f s i n g l e v a r i a b l e a, δ ( d e f a u l t = 1 ) p a r a m e t e r u s e d t o e s t a b l i s h i n i t i a l t h r e e v a l u e s o f a, m a x l t e r ( o p t i o n a l ) = ma x i mum n u mb e r o f i t e r a t i o n s a l l o w e d ( d e f a u l t = 2 0 ), t o l ( o p t i o n a l ) = c o n v e r g e n c e t o l e r a n c e ( d e f a u l t i s 1 0'3 ).p r R e s u l t s ( o p t i o n a l ) = I f T r u e ( d e f a u l t ) p r i n t s a l l i n t e r m e d i a t e r e s u l t s. T h e f u n c t i o n r e t u r n s { E s t i m a t e d mi n i mu m p o i n t, I n t e r v a l c o n t a i n i n g mi n i mu m}.
E x a m p l e 5.1 2 D e t e r m i n e a m i n i m u m o f t h e f o l l o w i n g f u n c t i o n u s i n g q u a d ­
r a t i c i n t e r p o l a t i o n.
Φ = l - l/( l ~ a + 2a2) !
W e u s e Q.u a d r a t i c S e a r c h f u n c t i o n t o c o m p u t e t h e m i n i m u m, s t a r t i n g w i t h d e ­
f a u l t S
= 1 a n d t o l = 0.0 0 1.
QuadraticSearch[φ, a]
**** I t e r a t i o n 1
-» 0 . φ, -» 0 . a-y - » 1. -* 0.5
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
S i n c e <p\ > Φι,
t h e t e s t i n s t e p ( 2 ) p a s s e s, a n d w e h a v e l o c a t e d s u i t a b l e b o u n d s a n d t h r e e i n i t i a l p o i n t s.
' 0. 1
' 0. 1
I n i t i a l t hree p o i n t s -*
0.5
Function v a l u e s -*
0.
; ϊ · .
,0.5,
W e c a n n o w u s e t h e l o g i c i n t h e m a i n s t e p t o l o c a t e p u t a t i o n s a r e s u m m a r i z e d a s f o l l o w s:
a, «rn ° u a q <t>f
0, 0.5 1. 0.2 5 0. 0. 0.5
0. 0.2 5 0.5 0.2 5 0. - 0.1 4 2 8 5 7 0.
,0.2 5 0.2 5 0.5 — - 0.1 4 2 8 5 7 - 0.1 4 2 8 5 7 0.
{ 0.2 5,{ 0.2 5,0.5 } }
A s s e e n f r o m F i g u r e 5.4, t h e f u n c t i o n i s f a i r l y f l a t a f t e r a = 2
w i t h n o m i n i ­
m u m; t h e r e f o r e, n o b o u n d s c a n b e e s t a b l i s h e d w i t h & >2
u s i n g t h e p r o c e d u r e g i v e n i n t h i s s e c t i o n. I n t h i s s i t u a t i o n, o n e s h o u l d r e d u c e 8
a n d t r y a g a i n.
Quadr at l cSear c h [φ, a, 2, 5 ]
* * * * I t e r a t i o n 1
a, -j 0. Φ/ 0. a i —> 2. —* 0. 8 5 7 1 4 3
* * * * I t e r a t i o n 2
ctf-* 0. 0,- * 0. a!~ » 4. 0 1 0,9 6 5 5 1 7
* * * * I t e r a t i o n 3
ctf —> 0. Φ/ —^ 0. ct]_ —* 8 . 0^ 0.9 9 1 7 3 6
* * * * I t e r a t i o n 4 '
α,- * 0. 0,- * 0. cti - » 1 6. 0 X 0.9 9 7 9 8 8
**** i t e r a t i o n 5
af -* 0. φ( -* 0. α ι -*■ 3 2 . Φί ~* 0.9 9 9 5 0 4
C h a n g e δ ------ No b o u n d s f o u n d i n 5 t r i e s.
$ A b o r t e d _______________________________________________________________________________________________________
T h e f u n c t i o n v a l u e s a t t h e s e c o n d p o i n t a r e a l w a y s l a r g e r t h a n t h e f i r s t p o i n t. T h e t e s t i n s t e p ( 2 ) p a s s e s, b u t t h a t i n s t e p ( 4 ) f a i l s a n d a l a r g e r v a l u e o f 8
i s t r i e d i n e a c h s u c c e s s i v e i t e r a t i o n. T h e p r o c e s s i s f i n a l l y a b o r t e d a f t e r f i v e t r i e s f a i l.
E x a m p l e 5.1 3 D e t e r m i n e t h e m i n i m u m o f t h e f o l l o w i n g f u n c t i o n u s i n g q u a d r a t i c i n t e r p o l a t i o n.
φ = 2 - 4 a + B x p [ a ];
ι. T h e c o m -
Φ,
C o n v. - 0.1 4 2 8 5 7 0.5 6 2 5
- 0.1 4 2 8 5 7 0.
0.
W e u s e Q u a d r a t i c S e a r c h f u n c t i o n t o c o m p u t e t h e m i n i m u m, s t a r t i n g w i t h t h e d e f a u l t 5 = 1 a n d t o l = 0.0 0 1.
5.2 L i n e S e a i y l i i f e r h n i q u e s — S t e p L e n g t h C a l c u l a t i o n s
Q u a d r a ti c S e a r c h [ < £, a ]
* * * * I t e r a t i o n 1
0 * φ/ —* 3 - ct^ 1. 0.718282
a 2 ~* 2. φ2 -» 1.38906
i n i t i a l
t h r e e p o i n t s ->
( 0.1 l. ( 2.,
F u n c t i o n v a l u e s -»
3. \ 0.7 1 8 2 8 2 1.3 8 9 0 6 )
fa'
«u
aq
0,
0m
K
0q
Conv.
0.
1.
2 .
1.2 7 2 8 1
3.
0.7 1 8 2 8 2
1.3 8 9 0 6
0.4 7 9 6 3 2
0.2 6 8 4 9 5
1.
1.2 7 2 8 1
2.
1.3 4 2 2
0 .718282
0.4 7 9 6 3 2
1.3 8 9 0 6
0.4 5 8 6 5 5
0 .023425"
1.2 7 2 8 1
1.3 4 2 2
2.
1.3 7 1 5 3
0.4 7 9 6 3 2
0.4 5 8 6 5 5
1.3 8 9 0 6
0.4 5 5 2 5 6
0 .003002!
1.3 4 2 2
1.3 7 1 5 3
2 .
1.3 8 0 6 6
0.4 5 8 6 5 5
0.4 5 5 2 5 6
1.3 8 9 0 6
0.4 5 4 8 8 6
0.0 0 0368:
,1.37153
1.3 8 0 6 6
2.
—
0.4 5 5 2 5 6
0.4 5 4 8 8 6
1.3 8 9 0 6
- -
0.0003681
{ 1.3 8 0 6 6,{ 1.3 7 1 5 3,2.} } ·
5.2.6 A p p r o x i m a t e L i n e S e a r c h B a s e d o n A r m i j o's R u l e
T h e l i n e s e a r c h t e c h n i q u e s c o n s i d e r e d s o f a r a t t e m p t t o f i n d a n e x a c t m i n ­
i m u m o f f u n c t i o n 0 ( a ). A s s e e n f r o m t h e n u m e r i c a l e x a m p l e s, a l l m e t h o d s r e q u i r e a l a r g e n u m b e r o f f u n c t i o n e v a l u a t i o n s. F o r u s e a s s t e p l e n g t h, i t i s d e ­
s i r a b l e t o o b t a i n a r e a s o n a b l e v a l u e o f a
w i t h o u t too
m a n y c a l c u l a t i o n s. T h u s, t h e a p p r o x i m a t e l i n e s e a r c h m e t h o d s a r e p o p u l a r f o r u s e w i t h o p t i m i z a t i o n a l g o r i t h m s. I n s t e a d o f f i n d i n g t h e t r u e m i n i m u m, t h e s e m e t h o d s t e r m i n a t e w h e n t h e s t e p l e n g t h i s w i t h i n a s p e c i f i e d p e r c e n t a g e o f t h e e x a c t m i n i m u m.
A p o p u l a r r u l e, k n o w n a s A r m i j o ’s r u l e, s a y s t o a c c e p t a
i f t h e f o l l o w i n g t w o c o n d i t i o n s a r e s a t i s f i e d:
Φ
( « ) < Φ
( 0 ) + ct
€φ'
( 0 ) a n d φ
(
ηα
) < φ
( 0 ) + αηέφ'
( 0 )
w h e r e 0 < e < 1 a n d η
> 1 a r e u s e r - s p e c i f i e d p a r a m e t e r s. U s u a l l y e = 0.2 a n d η
= 2 a r e s e l e c t e d. A s i l l u s t r a t e d i n F i g u r e 5.8, t h e u p p e r l i m i t o n t h e s t e p l e n g t h i s w h e r e 0 ( 0 ) l i n e i n t e r s e c t s t h e g r a p h o f φ.
A s s u m i n g t h e m i n i m u m e x is t s, t h e s l o p e o f t h e f u n c t i o n s h o u l d b e n e g a t i v e a t a = 0. T h u s, t h e c o n d i t i o n s d e f i n e a n a c c e p t a b l e s t e p l e n g t h i n t h e r a n g e aa
a n d a &.
B a s e d o n t h i s c r i t e r i a, a n a p p r o x i m a t e l i n e s e a r c h a l g o r i t h m c a n b e d e f i n e d a s f o l l o w s:
1 - S t a r t w i t h a n a r b i t r a r y v a l u e o f a
.
2. C o m p u t e 0 ( a ). I f 0 ( a ) < 0 ( 0 ) + a e 0'( O ), i n c r e a s e a b y s e t t i n g i t t o ηα,
a n d r e p e a t u n t i l t h e t e s t f a i l s. T h e n t h e s t e p l e n g t h i s e q u a l t o t h e p r e v i o u s v a l u e o f a.
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
F I G U R E 5.8 I l l u s t r a t i o n o f A r m i j o's a p p r o x i m a t e l i n e s e a r c h s t r a t e g y.
3. I f t h e t e s t φ(α) < φ(0) + αεηφ'(0)
f a i l s w i t h t h e i n i t i a l v a l u e o f a,
d e c r e a s e a
b y s e t t i n g i t t o α/η
a n d r e p e a t u n t i l a v a l u e o f a
i s f o u n d t h a t m a k e s t h e t e s t p a s s.
T h e p r o c e d u r e i s i m p l e m e n t e d i n t h e f o l l o w i n g f u n c t i o n t h a t i s i n c l u d e d i n t h e O p t i m i z a t i o n T b o l b o x 'U n c o n s t r a i n e d' p a c k a g e:
Ne ed s["OptimizationToolbox'UnconBtrained'*];
?Armi j oLineSearoh
ArmijoLineSearch[<i>, a, e, η, o p t i o n s ] p e r f o r m s l i n e s e a r c h u s i n g
o f s i n g l e v a r i a b l e a. e a n d η a r e t h e t w o p a r a m e t e r s. T h e r e a r e t w o o p t i o n a l p a r a m e t e r s. T h e f i r s t i s p r R e s u l t s ( d e f a u l t T r u e ) t o c o n t r o l p r i n t i n g o f i n t e r m e d i a t e r e s u l t s. T h e s e c o n d i s M a x l t e r a t i o n s a l l o w e d. D e f a u l t i s M a x l t e r a t i o n s s e t f o r F i n d Mi n i m u m f u n c t i o n. T h e f u n c t i o n s r e t u r n s a p p r o x i m a t e s t e p l e n g t h.
E x a m p l e 5.1 4 D e t e r m i n e a n a p p r o x i m a t e m i n i m u m o f t h e f o l l o w i n g f u n c ­
t i o n u s i n g A r m i j o's l i n e s e a r c h:
φ = 1 - 1/( 1 - α + 2 α 2 );
W e u s e A r m i j o L i n e S e a r c h f u n c t i o n t o c o m p u t e t h e m i n i m u m s t a r t i n g w i t h e —
0.2 a n d = 2.
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T f e c h n i q u e s
* r m i j o L i n e S e a r c h [ 0, a, 0.2, 2]
φ (
0) -* 0 φ 1 {0) - 1
α φ ( a ) 0 ( 0) + αφ' ( 0) e 0 ( 0) + a 0'( O )/7e
1. 0.5 - 0.2
0.5 0. - - - 0.2
0.2 5 - 0.1 4 2 8 5 7 — - 0.1
0.2 5
I n t h i s e x a m p l e, t h e m e t h o d i n f a c t g i v e s t h e e x a c t s t e p l e n g t h.
E x a m p l e 5.1 5 D e t e r m i n e a n a p p r o x i m a t e m i n i m u m o f t h e f o l l o w i n g f u n c ­
t i o n u s i n g A r m i j o ’ s l i n e s e a r c h:
0 = 2 - 4 a + E x p [ a ] ;
W e u s e A r m i j o L i n e S e a r c h t o c o m p u t e t h e m i n i m u m s t a r t i n g w i t h e = 0.2 a n d η
= 2.
ArmijoLineSearch[φ, a, .2, 2]
0 ( 0 ) -» 3 φ' ( 0) -> - 3 ----------------------------------------------------------------------------------------------------------------
a 0 (a) 0 ( 0) + αφ'( 0) e 0 ( 0 ) + αφ'(0)ηε
1. 0.7 1 8 2 8 2 2.4
2. 1.3 8 9 0 6 1.8
4. 4 0.5 9 8 2 0.6
2.
T h e e x a c t s t e p l e n g t h f o r t h i s e x a m p l e i s 1.3 8 6. T h e a p p r o x i m a t e s t e p l e n g t h o f 2 i s c o m p u t e d i n o n l y t h r e e f u n c t i o n e v a l u a t i o n s.
5.
T e c h n i q u e s
A t e a c h i t e r a t i o n o f a n u m e r i c a l o p t i m i z a t i o n m e t h o d, w e n e e d t o d e t e r m i n e a d e s c e n t d i r e c t i o n a n d a n a p p r o p r i a t e s t e p l e n g t h:
χ * + 1 = χ * + c t y d * f c = 0,1,...
T h e s t e p - l e n g t h c a l c u l a t i o n s w e r e p r e s e n t e d i n t h e l a s t s e c t i o n. T h e s e a r e c o m b i n e d w i t h s e v e r a l d i f f e r e n t m e t h o d s f o r d e t e r m i n i n g d e s c e n t d i r e c t i o n s t o g e t m e t h o d s f o r s o l v i n g u n c o n s t r a i n e d p r o b l e m s. A n y o f t h e s t e p - l e n g t h s t r a t e g i e s d i s c u s s e d i n t h e p r e v i o u s s e c t i o n c a n b e u s e d. H o w e v e r, f o r c l a r i t y
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
o f p r e s e n t a t i o n, t h e e x a m p l e s g i v e n i n t h i s s e c t i o n u s e t h e a n a l y t i c a l l i n e s e a r c h m e t h o d.
5.3.1 T h e S t e e p e s t D e s c e n t M e t h o d
A s m e n t i o n e d e a r l i e r, t h e g r a d i e n t v e c t o r o f a f u n c t i o n p o i n t s t o w a r d s t h e d i r e c t i o n i n w h i c h l o c a l l y t h e f u n c t i o n i s i n c r e a s i n g t h e m o s t r a p i d l y. T h u s, a n a t u r a l c h o i c e f o r t h e d e s c e n t d i r e c t i o n i s t o u s e t h e n e g a t i v e g r a d i e n t d i r e c t i o n. S i n c e l o c a l l y, t h e f u n c t i o n i s c h a n g i n g m o s t r a p i d l y i n t h i s d i r e c t i o n, i t i s k n o w n a s t h e steepest descent direction.
T h u s, i n t h i s m e t h o d w e c h o o s e t h e d i r e c t i o n a s f o l l o w s:
A s s e e n f r o m t h e f o l l o w i n g e x a m p le s, t h e m e t h o d p r o d u c e s s u c c e s s i v e d i r e c - t i o n s t h a t a r e p e r p e n d i c u l a r t o e a c h o t h e r. T h u s, w h e n t h e p o i n t i s a w a y f r o m t h e o p t i m u m, t h e m e t h o d g e n e r a l l y m a k e s g o o d p r o g r e s s t o w a r d s t h e o p t i ­
m u m. H o w e v e r, n e a r t h e o p t i m u m, b e c a u s e o f zigzagging,
t h e c o n v e r g e n c e i s v e r y s l o w.
T h e p r o c e d u r e i s i m p l e m e n t e d i n t h e f o l l o w i n g f u n c t i o n t h a t i s i n c l u d e d i n t h e O p t i m i z a t i o n T b o l b o x ‘U n c o n s t r a i n e d * p a c k a g e:
Ne ed s["Optimiz&tionToolbox'Unconstrained'"];
?SteepestDescent
S t e e p e s t D e s c e n t [ f, v a r s, xO, o p t s ]. Computes minimum o f f ( v a r s ) s t a r t i n g from xO u s in g S t e e p e s t Descent method. The s t e p le n g t h i s computed u s in g a n a l y t i c a l l i n e search. See O ptions[S teepestD escent] t o se e a l i s t o f o p t i o n s for the f u n ctio n. The f u n c t i o n retu rn s (x, h i s t }. 'x' i s e i t h e r the optimum p o i n t or the next p o in t a f t e r M axlte rations. 'h i s t' c o n t a i n s h i s t o r y o f v a l u e s t r i e d a t d i f f e r e n t i t e r a t i o n s.
O ptions [S tee pe stD esce nt]
{PrintLevel -»1, M axlterations -*· 50,
ConvergenceTolerance -» 0 .01, StepLengthVar -» a}
TPlotSearchPath
P lo tS ea r ch P a th [f, {xl, xlmin, xlmax}, {x2, x2min, x2max}, h i s t, opts] shows complete search path superimposed on a contour p l o t o f the f u n c t i o n f over the s p e c i f i e d range, 'h i s t' i s assumed to be o f the form {p tl, p t 2, ...}, where p t l = {xl,x2} i s f i r s t p o i n t, e t c. The f u n c t i o n a c ce p t s a l l r e l e v a n t option.? o f the standard ContourPlot and Graphics f u n c t i o n s.
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
E x a m p l e 5.1 6 U s e t h e s t e e p e s t d e s c e n t m e t h o d a n d t h e g i v e n s t a r t i n g p o i n t t o f i n d t h e m i n i m u m o f t h e f o l l o w i n g f u n c t i o n:
f = x 4 + y 4 + 2 x 2y 2 - 4 x + 3; v a r s = {X# y }; xO = { 1.2 5, 1.2 5 };
A l l i n t e r m e d i a t e c a l c u l a t i o n s a r e s h o w n f o r t h e f i r s t t w o i t e r a t i o n s.
S t e e p e s t D e s c e n t [ f, v a r s, xO, P r i n t L e v e l 2, M a x l t e r a t i o n s -» 2] ;
f -» 3 - 4x + x 4 + 2 x 2y 2 + y 4
/- 4 + 4 x 3 + 4xi ^ \
\ 4x2y + 4 y 3 J
* * * * * I t e r a t i o n 1 * * * * * C u r r e n t p o i n t -» { 1.2 5, 1.2 5 }
D i r e c t i o n f i n d i n g p h a s e:
_ . 11 1.6 2 5\ , /- 1 1.6 2 5\
V f ( x ) | 1 5 6 2 5 J d - » | _ 1 5 6 2 5 ]
| | v f ( x ) | | -* 1 9.4 7 5 1 f ( x ) -* 7.7 6 5 6 3
S t e p l e n g t h c a l c u l a t i o n p h a s e:
/l.2 5 - 1 1.6 2 5 a\
X “*■ ( l,2 5 - 1 5.6 2 5 a )
V f i x k l l [ ~ 1 7 6 3 6.6 ( - 0.0 3 5 3 3 1 4 + a ) ( 0.0 1 8 6 5 6 - 0.2 5 1 8 1 2 a + a 2 )
( X ' \ - 2 3 7 0 5.G ( - 0.0 8 + a ) ( 0.0 0 8 2 3 9 2 7 - 0.1 7 9 6 1 6 a + a 2 )
ά φ/d a = V f ( x k i ) . d = 0 -»
5 7 5 4 1 7. ( - 0.0 4 7 9 1 3 7 + α) ( 0.0 1 3 7 5 6 9 - 0.2 2 1 5 1 a + a 2 ) = = 0 a - > 0. 0 4 7 9 1 3 7
* * * * * i t e r a t i o n 2 * * * * * C u r r e n t p o i n t - * { 0.6 9 3 0 0 3, 0 .5 0 1 3 4 9 }
D i r e c t i o n f i n d i n g p h a s e:
1.9 7 1 9 8\ ; I 1.9 7 1 9 8
V f ( x ) -» ‘ d - ,
\ 1.4 6 7 1 6 J \- l.4 6 7 1 6
I (V f ( x ) | | - > 2.4 5 7 9 f ( x ) -> 0.7 6 3 2 3 1
S t e p l e n g t h c a l c u l a t i o n p h a s e: '0,6 9 3 0 0 3 + 1.9 7 1 9 8 a\
3 c k l ι I
^ 0.501349 - 1.46716aj
V f ( x k i ) -» ί 4 7.6 5 3 1 ( - 0.1 4 1 0 2 9 + a ) ( 0.2 9 3 4 2 9 + 0.7 0 1 3 6 2 a + a 2 ) \
( - 3 5.4 5 3 9 ( - 0.3 4 1 7 1 5 + a ) ( 0.1 2 1 1 0 1 + 0.2 0 8 9 0 8 a + a 2 )/
d 0/d a = v f ( x k i ) .d = 0 -»
1 4 5.9 8 7 ( - 0 .1 7 9 0 6 6 + α ) ( θ . 2 3 1 1 + 0.4 9 2 4 2 8 a + a 2 ) = = 0 0.1 7 9 0 6 6
N e w P o i n t ( N o n - O p t i m u m ): { 1.0 4 6 1 2, 0 .2 3 8 6 3 1 } a f t e r 2 i t e r a t i o n s
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
T h e p r i n t o u t o f i n t e r m e d i a t e r e s u l t s i s s u p p r e s s e d ( d e f a u l t o p t i o n ), a n d t h e m e t h o d i s a l l o w e d t o c o n t i n u e u n t i l c o n v e r g e n c e. C o m p u t a t i o n h i s t o r y i s s a v e d i n h i s t.
{apt, h i s t } = S t e e p e s t D e s c e n t [ f, v a r s, x O] ;
Op t i mu m: f O.9 9 9 3 6 2, 0.0 0 1 4 3 1 7 1 } a f t e r 9 i t e r a t i o n s
T a b l e F o r m f h i s t ]
X
d
Λ Λ /TO IT
1 | v f ( x ) | |
t n < 7 ζ ΐ
f ( χ )
*7 n cr. en
1.2 5
1 1 » 6m?
1 7 · 4/ΰ ΐ
I · /OJ OJ
1.2 5
- 1 5.6 2 5
0.6 9 3 0 0 3
1.9 7 1 9 8
2.4 5 7 9
0.7 6 3 2 3 1
0.5 0 1 3 4 9
- 1.4 6 7 1 6
1.0 4 6 1 1 2
- 0.8 1 7 6 2
1.3 6 9 7 4
0.1 4 1 0 3 8
0.2 3 8 6 3 1
- 1.0 9 8 9 5
0.9 4 9 3 4 2
0.5 3 2 8 7 4
0.6 6 4 1 7 9
0.0 3 6 2 6 4
0.1 0 8 5 5 5
- 0.3 9 6 4 5 8
1.0 1 4 0 3
- 0.1 8 5 5 6 5
0.3 1 0 8 7 5
0.0 0 8 7 1 4 6 9
0.0 6 0 4 2 5 8
- 0.2 4 9 4 1 6
0.9 8 8 3 1 7
0.1 3 5 9 1 4
0.1 6 9 4 0 5
0.0 0 2 1 1 9 7 4
0.0 2 5 8 6 3 5
- 0.1 0 1 1 2
1.0 0 3 5 5
- 0.0 4 3 5 6 4 6
0.0 7 2 9 8 4 1
0.0 0 0 5 0 1 1 1 4
0.0 1 4 5 3 2 6
- 0.0 5 8 5 5 6
0.9 9 7 2 6 6
0.0 3 2 5 6 6 5
0.0 4 0 5 9 1 4
0.0 0 0 1 1 8 5 4 1
0.0 0 6 0 9 0 5
- 0.0 2 4 2 2 9 9
1.0 0 0 8 5
- 0.0 1 0 2 3 3 7
0.0 1 7 1 2 0 8
0.0 0 0 0 2 7 8 2 7 7
0.0 0 3 4 2 5 5 6
- 0.0 1 3 7 2 5 7
0.9 9 9 3 6 2
0.0 0 7 6 4 7 9 6
0.0 0 9 5 5 0 1
6.5 3 8 7 9 χ 1 0 - 6
0.0 0 1 4 3 1 7 1
- 0.0 0 5 7 1 9 5 4
T h e f i r s t c o l u m n i s e x t r a c t e d f r o m t h e h i s t o r y, a n d t h e h e a d i n g i s r e m o v e d t o g e t a l i s t o f a l l i n t e r m e d i a t e p o i n t s t r i e d b y t h e a l g o r i t h m.
x h i s t = D r o p [ T r a n s p o s e [ h i s t ] [ [ 1 ] ],1 ]
f { 1.2 5, 1.2 5 } , { 0.6 9 3 0 0 3, 0.5 0 1 3 4 4 } , { 1.0 4 6 1 2, 0.2 3 8 6 3 1 } ,
{ 0.9 4 9 3 4 2, 0.1 0 8 5 5 5 } , { 1.0 1 4 0 3, 0.0 6 0 4 2 5 8 } , ( 0.9 8 8 3 1 7, 0.0 2 5 8 6 3 5 ^
{ 1.0 0 3 5 5, 0.0 1 4 5 3 2 6 5 } , { 0.9 9 7 2 6 6, 0.0 0 6 0 9 0 5 } , { 1.0 0 0 8 5, 0.0 0 3 4 2 5 5 6 } , { 0.9 9 9 3 6 2,0.0 0 1 4 3 1 7 1 } }
U s i n g t h e P l o t S e a r c h P a t h f u n c t i o n, t h e s e a r c h p a t h i s s h o w n o n a c o n t o u r m a p o f f u n c t i o n / i n F i g u r e 5.9. T h e d i r e c t i o n i s t h e n e g a t i v e g r a d i e n t d i ­
r e c t i o n, w h i c h i s a l w a y s p e r p e n d i c u l a r t o t h e t a n g e n t t o c o n t o u r a t t h e g i v e n p o i n t. A s m e n t i o n e d i n s e c t i o n 5.2, t h e o p t i m u m s t e p l e n g t h s e l e c t s t h e n e x t p o i n t s u c h t h a t t h e d i r e c t i o n i s t a n g e n t t o t h e c o n t o u r o f / a t t h e n e w p o i n t. F o r t h e s u b s e q u e n t i t e r a t i o n, t h e n t h e n e g a t i v e g r a d i e n t d i r e c t i o n, b e c o m e s
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
p e r p e n d i c u l a r t o t h e p r e v i o u s d i r e c t i o n. T h i s m a k e s t h e s e a r c h p a t h z i g z a g t o w a r d s t h e o p t i m u m. A s a r e s u l t, c o n v e r g e n c e n e a r t h e o p t i m u m b e c o m e s e x t r e m e l y s l o w.
P l o t S e a r c h P a t h f f, { x, - .1, 1.3 }, { y, - .1, 1.3 }, x h i s t, P l o t P o i n t s -» 30,
E p i l o g -► {RGBColorTl, 0, 0] , D i s k [ { l, 0 }, 0.0 1 5 ] ,
Text["Opti mum", { 1.0 1, 0.0 5 }, { - 1, 0}] }] ;
y
F I G U R E 5.9 A c o n t o u r p l o t s h o w i n g t h e s e a r c h p a t h t a k e n b y t h e s t e e p e s t d e s c e n t m e t h o d.
T h e c o l u m n c o n t a i n i n g t h e n o r m o f t h e f u n c t i o n g r a d i e n t i s e x t r a c t e d i n p r e p a r a t i o n f o r a p l o t s h o w i n g h o w t h e | | V/| | i s d e c r e a s i n g w i t h t h e n u m b e r o f i t e r a t i o n s. T h e p l o t s h o w s t h e r a t e o f c o n v e r g e n c e o f t h e m e t h o d.
normGradhist = D r o p [ T r a n s p o se [ h i st ] [ [ 3 ] ], 1]
{19-4751, 2.4 5 7 9, 1.36974, 0.664179, 0.310875,
0.169405, 0.0729841, 0.0405914, 0.0171208, 0.0095501}
l<ietPlot [normGradhist,
P lo t Joined -> True, AxesLabel -» {"I t er a t io n s" , ■ | | Vf | | "} ] ·,
Iivfll
F I G U R E 5.1 0 P l o t s h o w i n g |
\Vf
| | a t d i f f e r e n t i t e r a t i o n s.
E x a m p l e 5.1 7 U s e t h e s t e e p e s t d e s c e n t m e t h o d a n d t h e g i v e n s t a r t i n g p o i n t t o f i n d m i n i m u m o f t h e f o l l o w i n g f u n c t i o n:
f = (X + y) 2 + (2 (x 2 4- y 2 - 1 ) - J ) 2; v a r s = { x, y }; xO = { - 1.2 5, 0.2 5 };
S t e e p e s t D e s c e n t ff, v a r s, x O, P r i n t L e v e l -» 2. M a x l t e r a t i o n s -» 2 1;
£-> ( x + y ) 2 + + 2 ( - 1 + x 2 + y 2 }
r^ 5 0 x + i g x 3 + 2y + l f i x y 2'
2 x - + 1 6 x 2 y + l e y 3
* * * * * I t e r a t i o n 1 * * * * * C u r r e n t p o i n t - * ■ { - 1.2 5, 0.2 5 }
D i r e c t i o n f i n d i n g p h a s e:
V f ( x )
| | V f ( x ) | | -* 1 1.1 6 7 9 f ( x ) -* 1.8 4 0 2 8
V f - »
- 1 1.1 6 6 7 \ / 1 1.1 6 6 7 \
- 0.1 6 6 6 6 7 ) ( θ,1 6 6 6 6 7 )
S t e p l e n g t h c a l c u l a t i o n p h a s e:
x k i - *
V f ( x k i ) -*
- 1.2 5 + 1 1.1 6 6 7 a |
0.2 5 + 0.1 6 6 6 6 7 a j
2 2 2 8 3.7 ( - 0.1 9 8 0 5 9 + a ) ( - 0.1 1 5 0 5 3 + a ) ( - 0.0 2 1 9 9 0 9 + a )
3 3 2.5 9 3 ( - 0.1 8 2 0 9 8 + a ) ( 0.0 0 1 8 8 8 6 7 - t - a ) ( 1.4 5 7 0 5 + a )
άφ/da
s v f ( x k i ) . d = 0 -»
248890.(-0.197979 +a ) (-0.114697 +a ) (-0.0220681 + a)
a-» 0.0220681
***** I t e r a t i o n 2 ***** Current p o i n t { - 1.0 0 3 5 7, 0.253678}
== 0
D i r e c t i o n f i n d i n g phase:
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
, 4 /0.0 2 8 1 4 9 5\
V£(x) ( - 1.88601 J
/- 0.0 2 8 1 4 9 5\
\ 1.8 8 6 0 1 J
I | v f (x) | t -* 1.8 8 6 2 2 f ( X) -*■ 0.5 9 8 5 6 1
S t e p l e n g t h c a l c u l a t i o n p h a s e:
I - l.0 0 3 5 7 - 0.0 2 8 1 4 9 5 a\ x k l"^\ 0.2 5 3 6 7 8 + 1.8 8 6 0 1 a j
f - 1.6 0 2 4 2 ( - 0.0 0 2 2 2 6 7 2 + α ) ( θ.2 2 0 8 7 3 + a ) ( 3 5.7 1 7 8 + a )
V f ( x k l ) -» ^ 1 0 7.3 62· ( - 0.1 4 0 6 2 5 + α ) ( θ. 1 2 4 9 1 9 + 0.5 5 9 9 6 a + a 2 )
άφ/da = V f ( x k i ) . d = 0 -*
2 0 2.5 3 2 ( - 0.1 3 8 8 5 7 + a ) ( 0.1 2 6 5 1 1 + 0.5 6 6 1 0 2 a + a 2 ) == 0
ct -» 0 .1 3 8 8 5 7
Ne w P o i n t ( N o n - O p t i m u m ):{ - 1.0 0 7 4 8, 0.5 1 5 5 6 3 } a f t e r 2 i t e r a t i o n s
{ o p t, h i s t } = S t e e p e s t D e s c e n t [ £, v a r s, x O ];
O p t i m u m:{ - 0.7 6 4 7 0 3, 0.7 6 2 5 8 8 } a f t e r 2 7 i t e r a t i o n s
T h e f o l l o w i n g g r a p h s s h o w t h e p a t h t a k e n b y t h e p r o c e d u r e i n m o v i n g t o w a r d s t h e o p t i m u m. A g a i n, n o t i c e t h e z i g z a g p a t h t a k e n b y t h e m e t h o d. A l s o n o t i c e t h a t t h e f u n c t i o n h a s t w o l o c a l m i n i m a. A s i s t r u e o f m o s t n u m e r i c a l m e t h o d s, t h e a l g o r i t h m r e t u r n s t h e o n e t h a t i s c l o s e s t t o t h e s t a r t i n g p o i n t.
F I G U R E 5.1 1 G r a p h i c a l s o l u t i o n a n d t h e s e a r c h p a t h t a k e n b y t h e s t e e p e s t d e s c e n t m e t h o d.
T h e f o l l o w i n g p l o t o f t h e n o r m o f t h e g r a d i e n t o f t h e o b j e c t i v e f u n c t i o n s h o w s t h e c o n v e r g e n c e r a t e o f t h e m e t h o d. T h e f i r s t f e w i t e r a t i o n s s h o w s u b ­
s t a n t i a l r e d u c t i o n i n t h i s n o r m. H o w e v e r, i t t a k e s m o r e t h a n 2 0 i t e r a t i o n s t o r e d u c e i t f r o m 2 t o t h e c o n v e r g e n c e t o l e r a n c e o f I O - 3.
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
norxnGradhist = D r o p [ T r a n s p o s e [ h i s t ] [ [ 3 ] ], 1 ];
L i s t P l o t [ n o r m G r a d h i s t,
P l o t J o i n e d -» True, Axes Label -► {"I t e r a t i o n s", " | | Vf | | "}] ;
ii v f i i
5 10 15 2 0 25
F I G U R E 5.1 2 P l o t s h o w i n g | | V/| | a t d i f f e r e n t i t e r a t i o n s.
E x a m p l e 5.1 8 U s e t h e s t e e p e s t d e s c e n t m e t h o d a n d t h e g i v e n s t a r t i n g p o i n t t o f i n d t h e m i n i m u m o f t h e f o l l o w i n g f u n c t i o n o f t h r e e v a r i a b l e s:
f = ( »! - l ) 4 + ( x 2 - 3) 2 + 4 ( x 3 + 5 ) 2; v a r s = {xl f x 2, x 3 }; xO = { - 1, - 2, 1 } ;
S t e e p e s t D e s c e n t [ f, v a r s, x O, M a x l t e r a t i o n s -> 2, P r i n t L e v e l -* 2 ];
f ^ ( - 1 + X;l) 4 + ( - 3 + X 2 ) 2 + 4 ( 5 + X3 ) 2
V f ->
- 4 + 1 2 x x - 1 2 x i + 4 x i
- 6 +· 2x 2 4 0 + 8 x 3
* * * * * I t e r a t i o n 1 * * * * * Cur r e n t p o i n t - > { - 1., - 2., 1. }
D i r e c t i o n f i n d i n g p h a s e:
- 3 2 1
' 3 2 '
V f ( x )
- 1 0
d
1 0
.4 8 ,
>- 48,
I | V f ( χ ) I I -» 5 8.5 4 9 1 f ( x ) -* 1 8 5
S t e p l e n g t h c a l c u l a t i o n p h a s e: (-1. + 3 2 a\
x k i
2. + 1 0 a \1. - 4 8 a
V f ( x k i )
1 3 1 0 7 2. ( - 0.0 6 2 5 -t-a) ( - 0.0 6 2 5 + a ) - 1 0. + 2 0 a 4 8 - 3 8 4 a
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n I t e c h n i q u e s
( 1 2.2856Ϊ
[ - 1 2.2 8 5 6 ]
Vf ( x) -»
- 7.8 4 1 5
d -»
7.8 4 1 5
l6.5 5 6 7 7 j
- 6.5 5 6 7 7;
άφ/da = Vf (xki) .d = 0
4.1 9 4 3 x l O 6 ( - 0.1 0 7 9 2 5 + a ) ( 0.0 0 7 5 7 2 8 4 - 0.0 7 9 5 7 4 9 a + a 2 )
0.1 0 7 9 2 5
* * * * * I t e r a t i o n 2 * * * * * C u r r e n t p o i n t - » { 2.4 5 3 6 0,- 0.9 2 0 7 4 9 1, - 4.1 8 0 4 0 } D i r e c t i o n f i n d i n g p h a s e:
== 0
| | v f ( x ) | | 1 5.9 8 1 8 f ( x ) - > 2 2 - 5 2 3 8
S t e p l e n g t h c a l c u l a t i o n p h a s e:
2.4 5 3 6 - 1 2.2 8 5 6 a ' x k i - » - 0.9 2 0 7 4 9 + 7.8 4 1 5 a ■
, - 4.1 8 0 4 - 6.5 5 6 7 7 a ,
- 7 4 1 7.3 9 ( - 0.1 1 8 3 1 6 + a ) ( 0.0 1 3 9 9 9 2 - 0.2 3 6 6 3 6 a + a 2 ) ’ V f ( x k i ) - 7.8 4 1 5 + 1 5. 6 8 3 a
6.5 5 6 7 - 5 2.4 5 4 1 a
άφ/da = V f ( x k i ) . d = 0
9 1 1 2 7.3 { - 0.l 7 9 3 6 6 + a ) ( 0.0 1 5 6 2 6 5 - 0.1 7 5 5 8 7 a + a*) == 0
a - » 0.1 7 9 3 6 6
Ne w P o i n t ( N o n - Op t i mu m) :
{ 0.2 4 9 9 8 6,0.4 8 5 7 4 5,- 5.3 5 6 4 6 } a f t e r 2 i t e r a t i o n s
{ o p t, h i s t } = S t e e p e s t D e s c e n t [ £, v a r s, x O] ;
Op t i mu m: { 0.8 7 5 8 3 1, 3 - 5.0 0 0 6 3 } a f t e r 2 6 i t e r a t i o n s
n o m G r a d h i s t = D r o p [ T r a n s p o s e [ h i s t ] [ [ 3 ] ], 1 ] ι
L i s t P l o t [ n o m G r a d h i s t,
P l o t J o i n e d -> T r u e, A x e s L a b e l -> {"I t e r a t i o n s ·, " | | V f | | ■} ];
j m
1 0 1 5 2 0 2 5
F I G U R E 5.1 3 P l o t s h o w i n g | | V/1 | a t d i f f e r e n t i t e r a t i o n s.
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
5.3.2 C o n j u g a t e G r a d i e n t
T h e c o n j u g a t e g r a d i e n t m e t h o d a t t e m p t s t o i m p r o v e t h e b e h a v i o r o f t h e s t e e p ­
e s t d e s c e n t m e t h o d b y a d d i n g a p o r t i o n o f t h e p r e v i o u s d i r e c t i o n t o t h e c u r r e n t n e g a t i v e g r a d i e n t d i r e c t i o n a s f o l l o w s:
T h e s c a l a r m u l t i p l i e r β
d e t e r m i n e s t h e p o r t i o n o f t h e p r e v i o u s d i r e c t i o n t o b e a d d e d t o d e t e r m i n e t h e n e w d i r e c t i o n. A s p o i n t e d o u t e a r l i e r, w h e n t h e p o i n t i s f a r a w a y f r o m t h e o p t i m u m, m o v i n g a l o n g t h e n e g a t i v e g r a d i e n t d i r e c t i o n i s a g o o d i d e a. N e a r t h e o p t i m u m, h o w e v e r, w e n e e d t o m a k e c h a n g e s i n t h i s d i r e c t i o n. T h u s, β
m u s t b e d e f i n e d i n s u c h a w a y t h a t i t s h o u l d h a v e s m a l l v a l u e s a t p o i n t s a w a y f r o m t h e m i n i m u m a n d h a v e r e l a t i v e l y l a r g e v a l u e s n e a r i t. O n e o f t h e f o l l o w i n g t w o f o r m u l a s f o r β
i s u s e d m o s t o f t e n i n p r a c t i c e:
T h e d e n o m i n a t o r i n b o t h f o r m u l a s i s t h e s a m e a n d i s e q u a l t o t h e s q u a r e o f t h e n o r m o f t h e g r a d i e n t o f/ a t t h e p r e v i o u s p o i n t. I n t h e F l e t c h e r - R e e v e s f o r m u l a, t h e n u m e r a t o r i s t h e s q u a r e o f t h e n o r m o f t h e g r a d i e n t o f/ a t t h e c u r r e n t p o i n t, w h e r e a s i n t h e P o l a k - R i b i e r e f o r m u l a, i t i s s l i g h t l y m o d i f i e d. T h e P o l a k - R i b i e r e f o r m u l a u s u a l l y g i v e s b e t t e r r e s u l t s t h a n t h e F l e t c h e r - R e e v e s f o r m u l a.
A t t h e b e g i n n i n g, w e o b v i o u s l y d o n ’t h a v e a n y p r e v i o u s d i r e c t i o n a n d t h e r e ­
f o r e, t h e s t e e p e s t d e s c e n t d i r e c t i o n i s c h o s e n a t t h e f i r s t i t e r a t i o n. T h e f o l l o w i n g n u m e r i c a l e x a m p l e s s h o w t h e i m p r o v e d p e r f o r m a n c e w i t h t h i s m e t h o d.
T h e p r o c e d u r e i s i m p l e m e n t e d i n t h e f o l l o w i n g f u n c t i o n t h a t i s i n c l u d e d i n t h e O p t i m i z a t i o n T b o l b o x 'U n c o n s t r a i n e d' p a c k a g e:
N eeds["OptimizationToolbox'Uxiconstrained'"];
?ConjugateGradient
ConjugateGradient[f, vars, xO, o p t s ]. Computes minimum o f f ( v a r s )
s t a r t i n g from xO u sin g Conjugate Gradient method. User has the o p t io n o f u s in g e i t h e r 'P o la k R ib ier e' (d e f a u lt ) or 'F le t c h e r R e e v e s' method. A lso s t e p l e n g t h can be computed u s in g e i t h e r the 'Exact' a n a l y t i c a l l i n e se a r ch or an 'Approximate' l i n e search u s in g Armijo's r u l e. See Options[ConjugateGradient] to s e e a l i s t o f o p t i o n s fo r the fu n c t io n. The f u n c t io n retu rn s {x, h i s t }, 'x' i s e i t h e r the optimum p o i n t or the next point- afhpr Mgxjhi=rat'i nps 'h i s 1-1 m n t a i n a hisi-.n-ry nf v a lu e s t r i e d a t d i f f e r e n t i t e r a t i o n s.
Jt—1
F l e t c h e r - R e e v e s f o r m u l a: β
=
P o l a k - R i b i e r e f o r m u l a: β
=
[Vf (xk~Lt - Vf ('xi)]TVf(xk)
[ V/( x * - ] ) ] r V/( x l - 1 )
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
O p t i o n s [ C o n j u g a t e G r a d i e n t ]
{ P r i n t L e v e l -*·1, M a x l t e r a t i o n s -» 50, Co n v e r g e n c e T o l e r a n c e -» 0 . 01, St epLengt hVar -» a, M e t h o d - » P o l a k R i b i e r e, L i n e S e a r c h -» E x a c t,
A r m i j o P a r a m e t e r s { 0.2, 2 } }
E x a m p l e 5.1 9 U s e t h e c o n j u g a t e g r a d i e n t m e t h o d a n d t h e g i v e n s t a r t i n g p o i n t t o f i n d t h e m i n i m u m o f t h e f o l l o w i n g f u n c t i o n:
f = X4 + y 4 + 2 x 2y 2 - 4 x + 3; v a r s = { X/ y >; x 0 = { 1.2 5, 1 - 2 5 };
A l l i n t e r m e d i a t e c a l c u l a t i o n s a r e s h o w n f o r t h e f i r s t t w o i t e r a t i o n s.
C o n j u g a t e G r a d i e n t [ f, v a r s, x O, P r i n t L e v e l -> 2, M a x l t e r a t i o n s -> 2,
M e t h o d -> F l e t c h e r R e e v e s ];
f - » 3 - 4 x + x 4 + 2 x 2 y 2 + y 4
■ - 4 + 4 x 3 + 4 x y 2
Vf -» I 2 λ 3 4 x y + 4 y J
U s i n g F l e t c h e r R e e v e s m e t h o d w i t h E x a c t l i n e s e a r c h * * * * * I t e r a t i o n 1 * * * * * C u r r e n t p o i n t -» { 1.2 5, 1.2 5 }
D i r e c t i o n f i n d i n g p h a s e:
. /l l.6 2 5\ /- l l.6 2 5
[ l 5 .6 2 s ] ( - 1 5.6 2 5
I [ v f ( x ) I I - * 1 9.4 7 5 1 0 - * O. f ( x ) -» 7.7 6 5 6 3
S t e p l e n g t h c a l c u l a t i o n p h a s e:
x k l.i 1 · 2 5 - 1 1 · 6 2 5 ^
\1.2 5 - 1 5.6 2 5 g j
V f ( x k l ) -> f - 1 7 6 3 6 · 6 ( “ ° · 0353314 + a ) ( 0 . 018656 - 0 .2 5 1 8 1 2 a +·a 2 ) \
( - 2 3 7 0 5. l ( - 0.0 8 + a ) ( 0.0 0 8 2 3 9 2 7 - 0 .1 7 9 6 1 6 a + a 2 ) /
dtp/da =
v f ( x k i ) . d = 0 ->
5 7 5 4 1 7. ( - 0.0 4 7 9 1 3 7 + a ) ( 0.0 1 3 7 5 6 9 - 0.2 2 1 5 1 a + a 2 ) == 0
a -» 0.0 4 7 9 1 3 7
* * * * * I t e r a t i o n 2 * * * * * C u r r e n t p o i n t - » { 0.6 9 3 0 0 3, 0.5 0 1 3 4 9 }
D i r e c t i o n f i n d i n g p h a s e:
Vf ( x, /- 1.9 7 1 9 8 j / 1.7 8 6 8 2 \
\ 1.4 6 7 1 6 ] ^ - 1.7 1 6 0 3 J
I | V f ( χ ) I I ^ 2.4 5 7 9 β -* 0 . 0 1 5 9 2 8 2 f ( x ) 0 .7 6 3 2 3 1
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
S t e p l e n g t h c a l c u l a t i o n p h a s e;
/ 0.6 9 3 0 0 3 + 1.7 8 6 8 a \
\0.5 0 1 3 4 9 - 1.7 1 6 0 3 a ]
/ 4 3.8 6 6 3 ( - 0.1 6 2 5 9 1 + a ) ( 0.2 7 6 4 8 8 + 0.6 7 3 5 9 1 a + a 2 ) \
V f ( x k i ) ^ _ 4 2.i 2 8 5 ( - 0.2 9 2 1 5 6 + a ) ( 0.1 1 9 2 0 3 + 0.1 2 3 1 5 8 a + a 2 ) /
άφ/da
s y f ( x k i ) . d = 0 -»
1 5 0.6 7 5 ( - 0.1 9 7 9 5 6 + a ) ( 0.2 0 2 5 4 3 + 0.3 8 2 6 9 3 a + a 2 ) == 0
«- ► 0.1 9 7 9 5 6
Ne w P o i n t ( N o n - O p t i m u m ):{ 1.0 4 6 7 2, 0 .1 6 1 6 4 9 } a f t e r 2 i t e r a t i o n s
T h e p r i n t o u t o f i n t e r m e d i a t e r e s u l t s i s s u p p r e s s e d ( d e f a u l t o p t i o n ) a n d t h e m e t h o d i s a l l o w e d t o c o n t i n u e u n t i l c o n v e r g e n c e. C o m p u t a t i o n h i s t o r y i s s a v e d i n hist.
{ o p t, h i s t } = Co n j u g a t e Gr a d i e n t [ £, v a r s, atO, Method -» F l e t c h e r R e e v e s ] ;
Us i n g F l e t c h e r R e e v e s method w i t h E x a c t l i n e s e a r c h Optimum: { 1.0 0 0 0 4, 0.0 0 0 1 7 4 6 5 7 } a f t e r 6 i t e r a t i o n s
Tabl eForm [ h i s t ]
x
d
| | V f ( x ) 1 I
f (X)
4
1.2 5
- 1 1.6 2 5
1 9.4 7 5 1
7.7 6 5 6 3
1.
1.2 5
- 1 5.6 2 5
0.6 9 3 0 0 3
1.7 8 6 8 2
2.4 5 7 9
0.7 6 3 2 3 1
0.0
0.5 0 1 3 4 9
- 1.7 1 6 0 3
1.0 4 6 7 2
- 0.3 9 7 4 6 8
1.0 0 5 6 4
0.0 7 1 4 4 6 9
0,1
0.1 6 1 6 4 9
- 1,0 1 2 5 8
0.9 8 8 5 7 6
0.1 2 6 5 6 6
0.1 4 4 8 2 3
0.0 0 1 1 3 5 2 2
o
o
0.0 1 3 5 3 4 2
- 0.0 7 3 9 1 6 7
1.0 0 1 1 9
- 0.0 0 9 4 9 3 3 8
0.0 2 8 6 3 8
0.0 0 0 0 8 4 7 5 2 7
0.0
0.0 0 6 1 6 7 4 5
- 0.0 2 7 6 1 9 9
0.9 9 9 3 1 3
0.0 0 7 3 5 7 1
0.0 0 8 7 0 7 6 3
3.8 3 2 2 x 10 “6
0.0
0.0 0 0 7 0 8 5 7 9 - 0.0 0 5 3 8 3 9 3
1.0 0 0 0 4 — 0.0 0 0 8 6 6 7 2 8 7.1 9 7 0 x l 0 -8
0.0 0 0 1 7 4 6 5 7
T h e f i r s t c o l u m n i s e x t r a c t e d f r o m t h e h i s t o r y, a n d t h e h e a d i n g i s r e m o v e d t o g e t a l i s t o f a l l i n t e r m e d i a t e p o i n t s t r i e d b y t h e a l g o r i t h m.
x h i s t * D r o p [ T r a n s p o s e [ h i s t ] [ [ 1 ] ], 1]
{ { 1.2 5, 1.2 5 } , { 0.6 9 3 0 0 3, 0.5 0 1 3 4 9 } , { 1.0 4 6 7 2, 0.1 6 1 6 4 9 } ,
{ 0.9 8 8 5 7 6, 0.0 1 3 5 3 4 2 } , { 1.0 0 1 1 9, 0.0 0 6 1 6 7 4 5 } ,
{ 0.9 9 9 3 1 3, 0.0 0 0 7 0 8 5 7 9 } , { 1.0 0 0 0 4, 0.0 0 0 1 7 4 6 5 7 } }
U s i n g t h e P l o t S e a r c h P a t h f u n c t i o n, t h e s e a r c h p a t h i s s h o w n o n a c o n t o u r m a p o f f u n c t i o n /. N o t e t h a t t h e s u b s e q u e n t d i r e c t i o n s a r e n o t p e r p e n d i c u l a r t o t h e
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
p r e v i o u s o n e s. T h e s e a r c h p a t h s h o w s c o n s i d e r a b l y l e s s z i g z a g g i n g n e a r t h e o p ­
t i m u m, a s c o m p a r e d t o t h e s t e e p e s t d e s c e n t m e t h o d. T h e r a t e o f c o n v e r g e n c e t o t h e o p t i m u m i s t h u s i m p r o v e d c o n s i d e r a b l y.
p l o t S e a r c h P a t h [ f, { x, -.1, 1 - 3 ), { y, -.1, 1.3 }, x h i s t, P l o t P o i n t s -* 3 0,
E p i l o g -» { R G B C o l o r [ 1, 0, 0 ],D i s k [ { l, 0 >, 0.0 1 5 ],
T e x t ["Op t i nt um", { 1.0 1, 0.0 5 }, { - 1, 0 } ] } ] ;
y
F I G U R E 5.1 4 A c o n t o u r p l o t s h o w i n g t h e s e a r c h p a t h t a k e n b y t h e c o n j u g a t e g r a d i e n t m e t h o d.
T h e c o l u m n c o n t a i n i n g t h e n o r m o f t h e f u n c t i o n g r a d i e n t i s e x t r a c t e d i n p r e p a r a t i o n f o r a p l o t s h o w i n g h o w t h e | | V/| | i s d e c r e a s i n g w i t h t h e n u m b e r o f i t e r a t i o n s. T h e p l o t s h o w s t h e r a t e o f c o n v e r g e n c e o f t h e m e t h o d:
n o r m G r a d h i s t = D r o p [ T r a n s p o s e [ h i s t ] [ [ 3 ] ], 1 ]
{ 1 9.4 7 5 1, 2.4 5 7 9, 1.0 0 5 6 4, 0.1 4 4 8 2 3,
0.0 2 8 6 3 8, 0.0 0 8 7 0 7 6 3, 0.0 0 0 8 6 6 7 2 8 }
L i s t P l o t [ n o r m G r a d h i s t,
P l o t J o i n e d -» T r u e, A x e s L a b e l -» {"I t e r a t i o n s", ■ | | V f | | "} ];
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
HVfll
10
15
5
Iterations
2 3 4 5 6 7
F I G U R E 5.1 5 A p l o t s h o w i n g | | V/1 | a t d i f f e r e n t i t e r a t i o n s.
T h e a p p r o x i m a t e l i n e s e a r c h g i v e s t h e s a m e r e s u l t s b u t t a k e s a f e w m o r e i t e r a t i o n s.
C o n j u g a t e G r a d i e n t [ f, v a r s, xO, L i n e S e a r c h -+ Approxi mat e] ■
U s i n g P o l a k R i b i e r e m e t h o d w i t h A p p r o x i m a t e l i n e s e a r c h O p t i m u m: { 1.0 0 0 0 1, 0.0 0 0 3 5 0 5 5 8 } a f t e r 1 0 i t e r a t i o n s
T h e P o l a k - R i b i e r e m e t h o d t a k e s o n e l e s s i t e r a t i o n a s c o m p a r e d t o t h e F l e t c h e r - R e e v e s m e t h o d.
C o n j u g a t e G r a d i e n t [ £, v a r s, x O ];
U s i n g P o l a k R i b i e r e m e t h o d w i t h E x a c t l i n e s e a r c h O p t i m u m: j l., 2 .6 4 1 0 4 x 1 0 ” ®} a f t e r 5 i t e r a t i o n s
E x a m p l e 5.2 0 U s e t h e c o n j u g a t e g r a d i e n t m e t h o d a n d t h e g i v e n s t a r t i n g p o i n t t o f i n d t h e m i n i m u m o f t h e f o l l o w i n g f u n c t i o n:
£ = ( x + y ) 2 + (2 ( x2 + y 2 - 1) - 3 ) 2 f v a r s = { x, y } ; x O = { - 1.2 5, 0 .2 5 } ;
A l l i n t e r m e d i a t e c a l c u l a t i o n s a r e s h o w n f o r t h e f i r s t t w o i t e r a t i o n s. C o n j u g a t e G r a d i e n t [ £, v a r s, x O, P r i n t L e v e l -+ 2, M a x l t e r a t i o n s -» 2 ] f
f -> ( x + y ) 2 + + 2 ( - 1 + x 2 + y 2 ) j
+ i 6x3 + 2y+ 16xy2' 2x - - + 16x^y +· 16y^
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
Us i n g P o l a k R i b i e r e method w i t h Ex a c t l i n e s e a r c h ***** i t e r a t i o n 1 ***** C u r r e n t p o i n t -» {-1.25, 0.25}
D i r e c t i o n f i n d i n g p h a s e:
/ -11.1667 \ , ( 11.1667 \
Vf(χ ) -* [-0.166667J ( θ.166667J
I IV f (x) I I -> 11.1679 0-* 0. f (x) -» 1.84028
S t e p l e n g t h c a l c u l a t i o n p h a s e:
1-1.25 + 1 1.1 6 6 7 a\ x k l _ * ( θ.2 5 + 0.1 6 6 6 6 7 a ]
/22283.7( - 0.198059 +a) ( - 0.115053+a ) (- 0.0219909 + a) \
Vf (xki ) -» ^ 332.593 (-0.182098+a) (0.00188867 +a) (1.45705+ a) J
ά φ/ά α = Vf ( xki ) . d = 0 -+
248890. (-0.197979 + a ) (-0.114697 + a ) (-0 .0220681 + a) == 0
a-> 0.0220681
* * * * * i t e r a t i o n 2 ***** C u r r e n t p o i n t -*{-1.00357,0.253678}
D i r e c t i o n f i n d i n g p h a s e:
. /0.0281495\ . /0.290392\
(X) ( -1.88601 J \ 1.89077 J
| |V f (x) | | -*1.88622 β -» 0.0285261 f (x) -> 0.598561
Step l ength c a l c u l a t i o n phase:
x k i ( - 1 - 0 0 3 5 7 + 0.2 9 0 3 9 2 a ]
\ 0.253678 +1.8907a j
vf [17.0023 (-3.38977 +a) (-0.0103728+ a) (o.0470866+a )\
(X } \ 110.703 (-0.173294+a) ( θ.098311 + 0.41033a + a2) j
άφ/da = Vf (xki) .d = 0 -*
214.251 (-0.188279 + α ) (θ. 0881985 + 0.342583a + a2) == 0
a-» 0.188279
New Point (Non-Optimum): {-0.948898, 0 .60967} a f t e r 2 i t e r a t i o n s
i ° D t, h i s t } = C o n j u g a t e G r a d i e n t [ f, v a r s, x O ];
Using PolakRibiere method with Exact l i n e search Optimum: {-0.76376, 0 .763765} a f t e r 6 i t e r a t i o n s
T a b l e F o r m [h i s t ]
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
X
d
1 | v f { * ) | |
f (X)
β
- 1.2 5
0.2 5
1 1.1 6 6 7
0.1 6 6 6 6 7
1 1.1 6 7 9
1.8 4 0 2 8
0.
- 1.0 0 3 5 7
0.2 5 3 6 7 8
0.2 9 0 3 9 2
1.8 9 0 7 7
1.8 8 6 2 2
0.5 9 8 5 6 1
0.
- 0.9 4 8 8 9 8
0.6 0 9 6 7
2.7 7 2 3 9 2.8 6 0 8 2
2.3 0 5 9 7
0.1 5 9 5 4 4
1.
- 0.7 8 4 1 6 1
0.7 7 9 6 6 1
0.2 5 3 6 1 4
- 1.1 6 5 0
0.9 9 2 9 2
0.0 1 2 6 1 5 2
- 0
- 0.7 7 7 9 6 3
0.7 5 1 1 8 8
0.0 7 8 2 1 2 1
0.0 6 8 7 1 2 8
0.0 9 1 0 3 7 7
0.0 0 0 7 4 9 2 5 8
- 0
- 0.7 6 3 7 4 3
0.7 6 3 6 8 1
- 0.0 0 0 5 8 0 2 5 1
0.0 0 3 0 3 1
0.0 0 2 6 6 0 0 3
9.8 2 3 3 7 χ IO-8
0.
- 0.7 6 3 7 6
—
0.0 0 0 0 1 9 6 3 1 3
3.4 6 1 0 6 x I G -11
—
T h e s e a r c h p a t h s h o w n i n t h e f o l l o w i n g f i g u r e c l e a r l y d e m o n s t r a t e s t h e i m p r o v e m e n t i n c o n v e r g e n c e t h a t t h i s m e t h o d m a k e s a s c o m p a r e d t o t h e s t e e p e s t d e s c e n t m e t h o d.
x h i s t = D r o p [ T r a n s p o s e [ h i s t ] [ [ 1 ] ],1 ]?
P l o t S e a r c h F a t h [ f, { x, - 1.2 5, 0} , { y, 0, 1.2 5 } , x h i s t, P l o t P o i n t s - > 5 0,
E p i l o g -* { RGB C o l o r [ 1, 0,0 ], D i s k [ { -.7 6, .7 6 ), 0.0 1 ],
T e x t ["O p t i mu m", { -.7 4, .7 6 }, { - 1, 0 } ] } ] ;
y
- 1.2 - 1 - 0.8 - 0.6 - 0.4 - 0.2 0
F I G U R E 5.1 6 A c o n t o u r p l o t s h o w i n g t h e s e a r c h p a t h t a k e n b y t h e c o n j u g a t e g r a d i e n t m e t h o d.
5.3 n n r f l i i a t r a i i i e d M i n i m i z a t i o n T e c h n i q u e s
norxuSradhi st = D r o p [ T r a n s p o s e [ h i s t ] [ [ 3 J ] , 1]
{ 1 1.1 6 7 9, 1.8 8 6 2 2, 2.3 0 5 9 7, 0.9 9 2 9 2,
0.0 9 1 0 3 7 7,0.0 0 2 6 6 0 0 3,0.0 0 0 0 1 9 6 3 1 3 }
L i s t P l o t [nor mGr a d hi s t,
P l o t J o i n e d -+ Tr ue, Ax e s La b e l -* {"I t e r a t i o n s" , ■ | | V £ | | ■} ] j
HVfll
F I G U R E 5.1 7 A p l o t s h o w i n g | | V/| | a t d i f f e r e n t i t e r a t i o n s.
E x a m p l e 5.2 1 U s e t h e c o n j u g a t e g r a d i e n t m e t h o d a n d t h e g i v e n s t a r t i n g p o i n t t o f i n d m i n i m u m o f t h e f o l l o w i n g f u n c t i o n:
f = { χ ± - 1 ) 4 + ( x 2 - 3 ) 2 + 4 ( x3 + 5 ) 2 ί v a r s = {xl t x2, x 3}; xO = { - 1, - 2, 1 };
A l l i n t e r m e d i a t e c a l c u l a t i o n s a r e s h o w n f o r t h e f i r s t t w o i t e r a t i o n s.
C o n j u g a t e G r a d i e n t [ £, v a r s,x O, P r i n t L e v e l -> 2, M a x l t e r a t i o n s 2 ];
f -» ( - 1 + x -l ) 4 + ( - 3 + x 2 ) 2 + 4 ( 5 + x 3 ) 2
'**4 + Ι2χχ - 12*i + 4 x i'
V f -> - 6 + 2x 2
, 4 0 + 8x 3
U s i n g P o l a k R i b i e r e m e t h o d w i t h E x a c t l i n e s e a r c h * * * * * I t e r a t i o n 1 * * * * * C u r r e n t p o i n t -> { - 1, - 2, 1 }
D i r e c t i o n f i n d i n g p h a s e:
-32'
r 32 ’
Vf (X) ->
- 10
d -*
10
[ 48 ,
- « J
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
| |V f (x) | | - * 5 8.5 4 9 1 /3-+0 f (x) - » 185
S t e p l e n g t h c a l c u l a t i o n p h a s e:
x k i
(-1 + 3 2 a' -2 + 1 0 a 1 - 4 8 a,
’3 2 ( - 1 + 1 6 a )
V f ( x k i ) ->
- 1 0 + 2 0 a
4 8 - 3 8 4 a ,
ά φ/d a £ Vf ( x k i ) .d = 0 -» 4 ( - 8 5 7 + 1 6 9 4 6 a - 1 9 6 6 0 8 a 2 + 1 0 4 8 5 7 6 a 3 ) == 0 a 0.1 0 7 9 2 5
* * * * * I t e r a t i o n 2 ***** C u r r e n t p o i n t - * { 2.4 5 3 6,- 0.9 2 0 7 4 9,- 4.1 8 0 4 } D i r e c t i o n f i n d i n g p h a s e:
Ί 2.2 8 5 6 1
'- 9.9 0 1 3
V f ( x ) -»
- 7.8 4 1 5
d —*
8.5 8 6 5 9
6.5 5 6 7 7
ι,- Ι Ο. 1 3 3 2
| | V f ( x ) | | - > 1 5.9 8 1 8 0 - * 0 .0 7 4 5 0 9
I
f ( x ) - > 2 2.5 2 3 8
S t e p l e n g t h c a l c u l a t i o n p h a s e:
x k i -»
2.4 5 3 6 - 9.9 0 1 3 3 a - 0.9 2 0 7 4 9 + 8.5 8 6 5 a - 4.1 8 0 4 - 1 0.1 3 3 2 a
[ - 3 8 8 2 .7 6 ( - 0.1 4 6 8 0 8 + a ) ( 0.0 2 1 5 5 2 9 - 0 .2 9 3 6 1 8 a + a 2 ) ]
V f ( x k i ) -»
- 7.8 4 1 5 + 1 7.1 7 3 2 a
6.5 5 6 7 7 - 8 1.0 6 5 6 a \
ά φ/d a s V f ( x k i ) .d = 0
a - > 0.1 3 8 0 9 1
3 8 4 4 4.5 ( - 0.1 3 8 0 9 1 + α;) ( θ. 0 4 8 1 1 1 5 - 0.3 0 2 3 3 5 a + a 2 )' = - 0
N e w P o i n t ( N o n - O p t i m u m ): ( 1.0 8 6 3 2, 0.2 6 4 9 8 2,- 5.1 } a f t e r 2 i t e r a t i o n s { o p t, h i s t } = C o n j u g a t e G r a d i e n t [ f, v a r s, x G ];
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
U s i n g P o l a k R i b i e r e method w i t h E x a c t l i n e s e a r c h Optimum: { 0.9 4 2 9 2 6, 2.9 9 8 4 3,- 5.0 0 0 1 8 } a f t e r 11 i t e r a t i o n s
T a b l e F o r m [ h i s c ]
X
d
1 i v f ( x ) | 1
f (χ)
β
- 1
32
5 8.5 4 9 1
185
0
-2
10
1
- 4 8
2.4 5 3 6
- 9.9 0 1 3 3
1 5.9 8 1 8
2 2.5 2 3 8
0.0 7 4 5
- 0.9 2 0 7 4 9
8.5 8 6 5 9
- 4.1 8 0 4
- 1 0.1 3 3 2
1.0 8 6 3 2
- 1.5 1 1 0 3
7.1 7 1 4 2
8.8 2 4 6 3
0.1 5 2 3
0.2 6 4 9 8 2
6.7 7 8 1 9
- 5.5 7 9 7 1
3.0 9 3 8 8 9
0.6 2 7 8 5 5
- 0 .264824
3.1 8 3 1 3
0.9 9 5 0 1 4
0.3 1 1 6
2.3 2 1 5 5
3.4 6 9 6 2
- 4.6 4 0 9 9
- 1.9 0 7 7
0.5 7 7 5 3 5
0.3 0 0 0 5 7
0.3 0 5 3 0 3
0.0 3 2 2 7
0.0 0 5 8
2.9808^
0.0^85*531
- 5.0 0 3 4 8
0.0 1 6 7 5 2 1
0.8 3 7 2 9 8
0.0 5 4 9 6 2
0.1 0 9 7 4 2
0.0 0 2 1 8 0 3 8
0.1 2 5 7
3.0 3 1 5 3
- 0.0 5 5 6 8 9 6
- 4.9 8 8 9 8
- 0.0 8 6 0 4 4 2
0.8 4 7 2 7 7
0.0 3 1 1 0 4 4
0.0 5 8 2 5 5 1
0.0 0 1 0 8 7 4 3
0.3 0 6 6
3.0 2 1 4 2
- 0.0 5 9 9 1 1 8
- 5.0 0 4 6
0.0 1 0 4 3 5 6
0.8 5 9 9 9 6 7
0.0 1 2 5 7 6 9
0.0 1 2 8 7 3 2
0.0 0 0 3 9 4 1 5 4
0.0 5 1 4
2.9 9 6 9 2
0.0 0 3 0 8 4 3 2
- 5.0 0 0 3 4
0.0 0 3 2 2 1 5 1
0.8 7 5 3 8 9
0.0 7 5 7 1 2
0.0 2 9 9 0 9 8
0.0 0 0 2 9 3 6 3 9
5.4 0 4 5
3.0 0 0 6 9
0.0 1 5 2 8 5 8
- 4.9 9 6 3 9
- 0.0 1 1 4 4 7 1
fi ΟΠΛ1ίΓ
λ m o m r
Λ ΛΊ Λ Π» Ί ^ Λ
Λ Λ ΛΛ Λ
U. to
υ . U1 4 /
U.U U 0 1 2 8 8 3 1
0.3 8 ζ 9
3.0 0 6 5
- 0.0 0 7 1 4 4 5 4
- 5.0 0 0 7 4
0.0 0 1 5 5 0 7 4
0.9 3 9 2 8 9
0.0 2 1 2 3 0 2
0.0 0 7 9 0 8 7 8
0.0 0 0 0 1 8 5 6 1 4
0.6 2 5 3
2 .9 9 8 7 8
- 0.0 0 2 0 2 7 6 7
- 4.9 9 9 0 7
- 0.0 0 6 4 9 9 6 9
0.9 4 2 9 2 6
—
0.0 0 3 5 2 8 4 7
0.0 0 0 0 1 3 1 9 7 5
—
2.9 9 8 4 3
r
- 5.0 0 0 1 8
n o r m G r a d h i s t
= D r o p [ T r a n s p o s e [ h i s t ] [ [ 3 ] ]
, 1 ] j
L i s t p l o t [ n o r m G r a d h i s t,
P l o t J o i n e d - » T r u e, A x e s L a b e l -► {"I t e r a t i o n s", ” | | V f | | * } ];
Iivfll
F I G U R E 5.1 8 A p l o t s h o w i n g | | V/1 | a t d i f f e r e n t i t e r a t i o n s.
5.3.3 T h e M o d i f i e d N e w t o n M e t h o d
I n i t s b a s i c f o r m, t h e N e w t o n m e t h o d i s d e r i v e d b y c o n s i d e r i n g q u a d r a t i c a p p r o x i m a t i o n o f t h e f u n c t i o n u s i n g t h e I k y l o r s e r i e s:
/( x * +1 ) « /( x * ) + V/( x * ) T
dk
+ i ( d * ) T H ( x * ) d fc
z
w h e r e H ( x fc) i s t h e H e s s i a n m a t r i x a t x *. U s i n g t h e n e c e s s a r y c o n d i t i o n f o r t h e m i n i m u m o f t h i s a p p r o x i m a t e f u n c t i o n ( i.e., d i f f e r e n t i a t i n g w i t h r e s p e c t t o d * ), w e g e t
Vf
( x * ) + H ( x * ) d fc = 0
T h e d i r e c t i o n c a n n o w b e o b t a i n e d b y s o l v i n g t h i s s y s t e m o f e q u a t i o n s a s f o l l o w s:
dk =
- [ H ( x * ) ] - 1^/^ )
N o t e t h a t i n a c t u a l c o m p u t a t i o n s t h e i n v e r s e o f H e s s i a n i s n o t c o m p u t e d. I t i s m o r e e f f i c i e n t t o c o m p u t e t h e d i r e c t i o n b y s o l v i n g t h e l i n e a r s y s t e m o f e q u a t i o n s. T h e a b o v e f o r m i s u s e d j u s t f o r s y m b o l i c p u r p o s e s.
I n t h e o r i g i n a l f o r m, t h i s m e t h o d w a s u s e d w i t h o u t a n y s t e p - le n g t h c a l c u l a ­
t i o n s. T h u s, t h e i t e r a t i v e s c h e m e w a s a s f o l l o w s:
x * +1 = x * - [ H ( x * ) r 1V/( x * )
H o w e v e r, i n t h i s f o r m t h e m e t h o d h a s a t e n d e n c y t o d i v e r g e w h e n s t a r t e d f r o m a p o i n t t h a t i s f a r a w a y f r o m t h e o p t i m u m. T h e s o - c a l l e d Modified Newton
method
u s e s t h e d i r e c t i o n g i v e n b y t h e N e w t o n m e t h o d a n d t h e n c o m p u t e s a n a p p r o p r i a t e s t e p l e n g t h a l o n g t h i s d i r e c t i o n. T h i s m a k e s t h e m e t h o d v e r y s t a b l e. T h u s, t h e i t e r a t i o n s a r e a s f o l l o w s:
x fe+1 = χ * + akdk k =
0,1,...
w i t h dk
= — [ H ( x fc) ]"1 V/ ( x * ) a n d a * o b t a i n e d f r o m m i n i m i z i n g / ( x * + a ^ d * ).
T h e m e t h o d h a s t h e f a s t e s t c o n v e r g e n c e r a t e o f a l l t h e m e t h o d s c o n s i d e r e d i n t h i s c h a p t e r. E a c h i t e r a t i o n, h o w e v e r, r e q u i r e s m o r e c o m p u t a t i o n s b e c a u s e o f t h e n e e d t o e v a l u a t e t h e H e s s i a n m a t r i x a n d t h e n t o s o l v e t h e s y s t e m o f e q u a t i o n s f o r g e t t i n g t h e d i r e c t i o n. T h e f o l l o w i n g n u m e r i c a l e x a m p l e s s h o w t h e p e r f o r m a n c e o f t h i s m e t h o d.
T h e p r o c e d u r e i s i m p l e m e n t e d i n t h e f o l l o w i n g f u n c t i o n t h a t i s i n c l u d e d i n t h e O p t i m i z a t i o n T b o l b o x 'U n c o n s t r a i n e d' p a c k a g e.
Heeds["OptimizationToolbox'lfticonstrained'"];
?Modi£iedNewton
Modifie&Newton[f, v a r s, xO, o p t s ]. Computes minimum of f ( v a r s ) s t a r t i n g from xO u s in g Modified Newton m e t h o d.S e e O ptions[M odified Ne wton]to see a l i s t of o p t io n s for th e fu n c t io n. The fun ctio n retu rn s {x, h i s t }, 'x' i s e i t h e r the optimum p o in t or the next p o in t a f t e r M a x lt e r a t i o n s. 'h i s t' c o n t a i n s h i s t o r y of v a l u e s t r i e d a t d i f f e r e n t i t e r a t i o n s.
Options[ModifiedNewton]
{PrintLevel 1, M axlterations -» 50,
ConvergenceTolerance -» 0.0 1, StepLengthVar -+ a }
E x a m p l e 5.2 2 U s e t h e m o d i f i e d N e w t o n m e t h o d a n d t h e g i v e n s t a r t i n g p o i n t t o f i n d t h e m i n i m u m o f t h e f o l l o w i n g f u n c t i o n:
f = x 4 + y 4 + 2x2y2 - 4 x + 3; v a r s = { x, y } ; xO = { 1.2 5, 1.2 5 } ;
A l l i n t e r m e d i a t e c a l c u l a t i o n s a r e s h o w n f o r t h e f i r s t i t e r a t i o n.
M o d i f i e d N e w t o n [ f, v a r s, x O, P r i n t L e v e l -> 2, M a x l t e r a t i o n s -> 1 ];
f -» 3 - 4 x + x 4 + 2 x 2 y 2 + y 4
V f -» ( - 4 + 4 x 3 + 4 x y 2\
\ 4 x ^ y + 4 y J J
v2 f / 1 2x2 + 4 y 2 8xy \
\ 8 x y 4 x 2 + 1 2 y 2 J
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
***** I t e r a t i o n 1
Curr ent p o i n t - * { 1.2 5,1.2 5 }
D i r e c t i o n f i n d i n g p h a s e:
2 (25. 1 2.5
V f · * 1 2.5 2 5.
1 1.6 2 5
1 5.6 2 5
Vf ( x )
| | V f ( x ) | | -> 1 9.4 7 5 1 f ( x ) - > 7.7 6 5 6 3
d - » ( - 0.2 0 3 3 3 3 - 0.5 2 3 3 3 3 )
S t e p l e n g t h c a l c u l a t i o n p h a s e: x k i
1.2 5 - 0.2 0 3 3 3 3 a\ 2 5 - 0.5 2 3 3 3 3 a ] '
(i:
l - 0 * 2 5 6 3 8 1 ( - 1.5 4 7 + a ) ( 2 9.3 1 0 1 - 1 0.3 6 3 7 a + a 2 ) \
V f ( x k i ) -* 6 5 9 8 6 5 ( _ 2.3 8 8 5 + a ) ( 9 .9 1 3 6 4 - 5.7 6 3 1 3 a + a 2 ) J
ά φ/d a s V f ( x k i ) .d = 0 - * 0.3 9 7 4 6 ( - 2.0 1 4 9 4 + a ) ( 1 3 .1 6 1 9 - 6.6 2 9 7 5 a + a 2 ) a - » 2.0 1 4 9 4
Ne w P o i n t ( N o n - O p t i m u m ): { 0.8 4 0 2 9 5, 0 .1 9 5 5 1 4 } a f t e r 1 i t e r a t i o n s
= = 0
T h e p r i n t o u t o f i n t e r m e d i a t e r e s u l t s i s s u p p r e s s e d ( d e f a u l t o p t i o n ), a n d t h e m e t h o d i s a l l o w e d t o c o n t i n u e u n t i l c o n v e r g e n c e. C o m p u t a t i o n h i s t o r y i s s a v e d i n hist.
{ o p t, h i s t } = M o d i f i e d N e w t o n [ f, v a r s, x 0] ;
Optimum: { l 2.6 0 6 4 4 x 1 0 “7 } a f t e r 3 i t e r a t i o n s
T a b l e F o n n [ h i s t ]
x
1.2 5
1.2 5
- 0.2 0 3 3 3 3
- 0.5 2 3 3 3 3
| | Vf (x) | | 1 9.4 7 5 1
f ( x ) 7.7 6 5 6 3
0.8 4 0 2 9 5
0.1 9 5 5 1 4
0.9 9 9 7 5 4 - 0.0 0 0 5 9 9 8 9 8
1.
2.6 0 6 4 4 x l O"7
1.6 0 7 3 1
0.2 1 3 7 3 5 - 0.2 6 2 8 6 7
0.0 0 0 2 4 6 2 0 9 0.0 0 3 8 0 2 0 9
0.0 0 0 6 0 0 1 9 4
0.1 9 2 8 3 4 1.0 8 2 5 2 x l O - 6 2.2 5 3 8 3 x 1 0"^ 3.0 2 4 0 4 χ I O- 1 3
T h e f i r s t c o l u m n i s e x t r a c t e d f r o m t h e h i s t o r y, a n d t h e h e a d i n g i s r e m o v e d t o g e t a l i s t o f a l l i n t e r m e d i a t e p o i n t s t r i e d b y t h e a l g o r i t h m.
x h i s t = D r o p [ T r a n s p o s e [ h i s t ] [ [ 1 ] ],1 ]
{ { 1.2 5, 1.2 5 } , { 0.8 4 0 2 9 5, 0.1 9 5 5 1 4 } ,
{ 0.9 9 9 7 5 4, - 0.0 0 0 5 9 9 8 9 8 } , { l., 2.6 0 6 4 4 χ 1 0 ~ 7 } }
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
Using the PlotSearchPath function, the search path is shown on a contour map of function f. The search path clearly demonstrates that the method eliminates the zigzagging associated with the steepest descent method.
PlotSearchPath[f, {x, 0, 1.3}, {y, 0, 1.3}, xhist, PlotPoints ->
30,
Epilog-► {RGBColor[1, 0, 0] , Diskffl, 0}, 0.015],
Text Γ"Optimum" , f 1.01, 0.05), f-1, 0 } ] } ] ;
0 0.2 0.4 0.6 0.8 1 1.2
FIGURE 5.1 9 A c o n t o u r p l o t s h o w i n g t h e s e a r c h p a t h t a k e n b y t h e M o d i f i e d N e w t o n m e t h o d.
The column containing the norm of the function gradient is extracted in preparation for a plot showing how the ||V/|| is decreasing with the number of iterations. The plot shows the rate of convergence of the method.
normGradhi s t = Drop[Transpose[hist][[3]], 1 ]
{ 1 9.4 7 5 1, 1.6 0 7 3 1, 0.0 0 3 8 0 2 0 9, 2 .2 5 3 8 3 x 1 0 ~6 }
ListPlot[normQradhist,
Plot Joined -> True, AxesLabel -► {"Iterations", " | | Vf | | "}];
Iterations 1.5 2 2.5 3 3.5 4
F I G U R E 5.2 0 A p l o t s h o w in g ||V/|| a t d i f f e r e n t it e r a t io n s.
Example 5.23 Use the modified Newton method and the given starting point to find the minimum of the following function.
vars = {x, y} ; xO = {-1.25, 0.25};
All intermediate calculations are shown for the first two iterations.
ModifiedNewton[f, vars, xO, PrintLevel -» 2, Maxlterations -> 2];
i_S02 + 16x3 + 2y + 16xy2' V f ^ ( 2 x - SOy + i 6 x 2y + i 6y 3 j
***** Iteration 1 ***** Current point-»{-1.25, 0.25}
Direction finding phase:
| |V f(x) | | -»11.1679 f(x) -> 1.84028 d-* (0.21019 0.163075)
Step length calculation phase:
xki-» (-1-25 + 0.21019ΟΛ \0.25 + 0.163075α)
f = (x + y)2 + (2 (x2 ty2 -1) - | ) 2 ;
2^ f-^r + 48x2 + 16y2 2 + 32xy
^ SO 5 0
2 + 32xy + 16x2 + 48y2
-11.1667
-0.166667
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
/ 0.238013(-1.53958+a) (30.4732-10.6801α + α2) \
Vf (xki) -> | 0.184662(-4.58573 +a) (0.196817 - 0.153898a + a2) )
άφ/άα =
Vf (xki) .d = 0-> 0.0801418 (-2.00245 + a) (14.795-7.40655a + a2) == 0 o;—>2.00244
***** iteration 2 ***** Current point -> {-0.829106, 0.57655}
Direction finding phase:
2 / 21.6479 -13.296\ j
1.44283 \
V f ^\-13.2967 10.2876] \- l.85969]
| |v f (x) | | ->2.35377 f(x) ->0.150034
d-> (0.215323 0.459071)
Step length calculation phase:
, 1 /-0.829106+0.215323a\
\ 0.57655+0.459071a ]
/θ.885788(-3.5121+a)(-0.535061 + a) (0.866795 + a) \
Vf(x ) ^ 1.88851 (-0.467862 +a) (2.10476+ 2.39392a + a2) j
άφ/da
= Vf (xki) .d = 0 ->1.05769 (-0.443886 + a) (1.15668 + 1.44912a + a2) == 0 a-» 0.443886
New Point (Non-Optimum): {-0.733527, 0.780325} after 2 iterations
{opt, hist} = ModifiedNewton[f, vars, xO];
Optimum:{-0.7637625,0.763763} after 4 iterations
TableForm[hist]
x
-1.25
0.25
-0.829106
0.57655
-0,733527
d
0.21019
0.163075
0.215323
0.459071
-0.0327749
11vf (x) 11 11.1679
2.35377
0.358729
f(x)
1.84028
0.150034
0.0037421
0.780325
-0.0180651
-0,763825
0.0000626412
0.00205812
9.25196x10"®
0.763625
0.000137636
-0.763763
—
4.88312 χ10-7
9.63452 χIO-15
0.763763
xhi st = Drop[Transpose[hist][[1]],1];
PlotSearchPath[f, {x, -1.25, 0}, {y, 0, 1.25}, xhist, PlotPoints -» 50, Epilog -+
{RGBColor [ 1, 0, 0] , Disk [ {-.76, .7 6 }, 0.01],
Text["Optimum", { -.7 4, .7 6 }, {-1, 0}]}] ;
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
y
F I G U R E 5.2 1 A c o n t o u r p l o t s h o w i n g t h e s e a r c h p a t h t a k e n b y t h e M o d i f i e d N e w t o n m e t h o d.
nomGradhist = Drop [Transpose [hist] [ [3] ], 1] ;
ListPlot[normGradhist,
Plot Joined -» True, AxesLabel -*
{ "Iterations" / " | | Vf | I "} ];
H V f l l
F I G U R E 5.2 2 A p l o t s h o w i n g | | V/ |j a t d i f f e r e n t i t e r a t i o n s.
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
E x a m p l e 5.2 4 U s e t h e m o d i f i e d N e w t o n m e t h o d a n d t h e g i v e n s t a r t i n g p o i n t t o f i n d t h e m i n i m u m o f t h e f o l l o w i n g f u n c t i o n:
f = (xi - 1)4 + (
x2
- 3)2 + 4(x3 + 5)2; vars = {Xi, x2* X3}; xO = i'1/ ~2* 1)'·
A l l i n t e r m e d i a t e c a l c u l a t i o n s a r e s h o w n f o r t h e f i r s t t w o i t e r a t i o n s.
Modif iecLNewton[f, vars, xO, PrintLevel -*
2, Maxlterations -» 2] ;
f -> (-1 + xx) 4 + (-3 + x2) 2 + 4 (5 + x3 ) 2
Vf -»
V2f -»
1 -A
+ 12x^ - 12xf + 4xi - 6 + 2x2 40 + 8x3
12 - 2 4 χ! + 1 2 χ ι θ ’ 0}
0 2 0
0 0 8
***** Iteration 1 ***** Current point {-1,-2,1} Direction finding phase:
v2f -»
'48
0
0
'-32'
0
2
0
Vf (x) -»
-10
, 0
0
8,
. 48 ,
I |V f (x) || -* 58.5491 f (■*)-» 185
d.| - 5
- 6 )
Step length calculation phase: xki ->
i-1 + ψ
-2 + 5a 1 - 6a
Vf (xki) -»
3
27 (-3 + a) 10(-1+ a) -48(-1 + a),
d0/da = Vf (xki) .d= 0-> (-14553 + 14553a- 288a2 + 32a3) == 0
o*L
a-» 1.0182
***** Iteration 2 ***** Current point-► {-0.321203, 3.09098,-5.10917} Direction finding phase:
2
V f
20.9469
0
O'
’-9.22505'
0
2
0
Vf (x) -»
0.181954
, 0
0
8,
,-0.873378,
I |V f (x) I I ->9.26809 f (x) -»3.10299 d-» (0.440401 - 0.0909768 0.109172)
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
S t e p l e n g t h c a l c u l a t i o n p h a s e:
xki
-0.321203 + 0 .44040la'\
3.09098 - 0.0909768a , -5.10917 + 0.109172aj
0.341669 (-2.99998 + a)
(9.00005 - 6.00002a + a2)'
Vf(xki) -» 0.181954 - 0.181954a
-0.873378+0.873378a
dtp/det s
Vf (xki) .d = 0 -► 0.150471 (-2.07259 + a) (13.386 - 6.92741a + a2) == 0 a -» 2.07259
New Point (Non-Optimum) : {0.591567, 2.90242,-4.8829} after 2 iterations
{opt, hist} = ModifiedNewton[f, vars, xO] ;
Optimum: {0 .96445, 2 .99963,— 4 .99956}—after 6 iterations
TableForm [hist]
X
d
1lv f(x)||
f (X)
-1
2
Ϊ
58.5491
185
-2
5
1
-6
-0.321203
0.440401
9.26809
3.10299
3.09098
-0.0909768
-5.10917
0.109172
0.591567
0.136144
0.994942
0.0921967
2.90242
0.0975807
-4.8829
-0.117097
0.738062
0.0873128
0.102274
0.00507961
3.00742
-0.00741852
-5.0089
0.00890222
0.886003
0.037999
0.0508607
0.000348261
2.99485
0.0051513
-4.99382
-0.00618156
0.92QfiQ7
0 r 02^4f>4?
0.007^80^6
0.0000284^67
3.00076
-0.000759837
-5.00091
0.000911804
0.96445
0.0036177
2.51501 xlO-6
2.99963
-4.99956
normGradhist = Drop[Transpose[hist][[3] ] , 1];
ListPlot[normGradhist,
Plot JoinedTrue, AxesLabel{"Iterations", "| |Vf| I"}];
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
HVfll
F I G U R E 5.23 A plot showing || V/ j| at different iterations.
E x a m p l e 5.2 5 T b c o m p a r e t h e p e r f o r m a n c e o f d i f f e r e n t u n c o n s t r a i n e d m e t h o d s, c o n s i d e r t h e f o l l o w i n g e x a m p l e:
£ = Exp [- x—y] + yy + y
2:
vars = {x,y);xO = { - 1.4,1.5 );
U s i n g o p t i m a l i t y c o n d i t i o n s, w e c a n s e e t h a t t h e f u n c t i o n d o e s n o t h a v e a m a x i m u m o r a m i n i m u m. I t o n l y h a s a s t a t i o n a r y p o i n t a t ( — 1,1 ).
unconst rai neci Opt i mal i ty [ f, vars,
Sol veEquati onsUsi ng -> Fi ndRoot, St art i ngSol ut i on xO];
Obj ect i ve f unct i on -» E~x~y + xy + y2
_ E _ x _ y + y \
_E- x-y + χ+ 2y I
Gradient vector Hessian matrix-
ε-*-* i + E-X~y i + E - x - y * 2 + ε - * - *
Necessary conditions -»
****** First order optimality conditions ******
-E~x~y
+ y = = 0 ^-E“ x-y + x + 2y == 0 J
Possible solutions (stationary points) -»(x-»-l. y-»l.)
****** Second order optimality conditions ******
----------Point -» {x -» - 1 y -» 1. }
Hessian-» ^'J Principal minors
Status -»Inf lectionPoint Function value -»1.
(i.
U s i n g t h e m o d i f i e d N e w t o n m e t h o d, w e r e a d i l y g e t t h i s s o l u t i o n i n t w o i t e r a - t i o n s.
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
{sol, hist} = ModifiedNewton[f, vars, xO,
PrintLevel -+ 2, Maxlterations -*
2] ;
f -» E-x-y + xy + y2 _E-*-y + y
^ -E-JC y + x + 2y
2 / E_x_y 1 + E-x-y
v £ ^ l + i r *'* 2 + E‘ x- y
***** Iteration 1 ***** Current point-» {-1.4, 1.5} Direction finding phase:
2 iO
.904837 1.90484\ /θ-·
7 ( 1.90484 2.90484) (χ)_>(θ.Ι
i0.595163] ,695163]
( |V f (x) M -*0.915134 f (x) -*1.05484 d -» (O .404679 - 0.504679)
Step length calculation phase:
/- 1.4 + 0.404679a\
' 1.5- 0.504679a ]
fl.5 -
0.9048374Ε0·ΐ0< - 0.504679a\
Vf (xki) -* | 1>6 _ 0.9048374E0-iO‘- 0.604679a]
άφ/da =
Vf (xki) .d = 0-» -0.200468 + 0.09048374E°-lot + 0.100936a == 0 a -* 0.995782
***** Iteration 2 ***** Current point-»{-0.997028,0.99745}
Direction finding phase:
n f i v\ I '
.00170681/
2 , (0
.999578 1.99958\ v. /-0.002l2859\
[ 1.99958 2.99958J 7£(X> - (-0.!
| Jv f (x) I I -► 0.00272838 f (x) ->0.999999 d-> (-0.00297194 0.00255017)
Step length calculation phase:
0.997028 - 0.00297194a\
ylrl
' ’ 0.99745+ 0.00255017a
/0.99745 - O.999578E0-00042l769a + 0.00255017a\
' ^0.997872 - 0.999578Ε°·000421769α + 0.0021284a]
άφ/da
= Vf (xki) .d = 0 -► -0.000419618 + Ο.ΟΟΟ42159ΐΕ0·000421769α - 2.15117 x 10~6a == 0 a-> 1.
Optimum:{-l.,l.} after 2 iterations
xhist =Drop[Transpose[hist][[1]],1]
{{-1.4, 1.5} , {-0.997028, 0.99745} , {-1., 1.}}
The search path shown in the following figure shows the direct path that the method takes.
5.3 i T n m « » t i a i n e d M i n i m i z a t i o n T e c h n i q u e s
PlotSearchPath[£, {x, -1.5, 0}, {y, o, 1.5}, xhist, PlotPoints 30, Epilog -> { RGBColor[1, 0, 0], Disk[{-.96, .98},0.015],
Line[{{-0.96,0.98},{-.75,1.2}}],
T e x t ["S t a t i o n a r y p o i n t", { -.7 2, 1.2 }, { - 1,0 } ] } ];
- 1.4 - 1.2 - 1 - 0.8 - 0.6 - 0.4 - 0.2 0
F I G U R E 5.2 4 C o n v e r g e n c e t o t h e i n f l e c t i o n p o i n t o f / (x, y) = e x y
+ xy
+ y2
u s in g t h e m o d if ie d N e w t o n m e th o d.
The steepest descent method fails to find a positive step length at the second iteration.
SteepestDescent [f, vars, xO, PrintLevel -> 2, Maxlterations -» 2] ?
f E_x_y + xy + y2
Vf .
-Ε'*-* + y
r E“x-y + x + 2y J ***** Iteration 1 ***** Current point-* {-1.4, 1.5}
Direction finding phase:
. 10.595163\ , ( - 0.595163\
^ \0.695163j [-0.695163]
| | V f (x) | | -» 0.915134 f (x) 1. 05484
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
Step length calculation phase:
/-1.4 - 0.595163a\
\ 1.5 - 0.695163a )
(1.5-
0.904837E1·290330' - 0.695163a\
Vf (xki) -» ^ 1.6 -0.904837E1·2903301 - 1.98549a J
άφ/da s Vf (xki) .d « 0 -» -2 .005 + 1.16753E1· 2903301 + 1.79397a == 0 a-» 0.235606
***** Iteration 2 ***** Current point-»{-1.54022, 1.33622}
Direction finding phase:
/ 0.109909 \ . /- 0.109909\
Vf(x)-» | _o.0940984] (θ.Ό940984]
| | V f (x) | | -» 0.144688 £ (x) -» 0.953709
Step length calculation phase:
/- l.54022 - 0.109909α-ΐΛ ( 1.33622 + 0.0940984αj
/l.33622 - 1.22631Ε°·0158105α + 0.0940984α\
Vf (xki) -► ^l. 13221 - 1.22631Ε°·0158105α + 0.0782874α]
άφ/δα
s Vf (xki) .d = 0-0.0403231 + 0. 0193886E0·0158105α - 0.00297549a == 0 a->-7.79046
Unconstrained:: step: Negative step length (-7.79046) found. Check direction or change line search parameters.
/-1.54022 - 0.109909a\
\l.33622 + 0.0940984a]
/1.33622 - 1.2263IE0·0158105a + 0.0940984a\
(xk ) -» [ i.13221 - 1.22631Ε0· 01 5810501 + 0.0782879a]
άφ/da =
Vf (xki) .d= 0 -*-0.0403231 + 0.0193886E0 ·0158105α - 0.00297549a == 0 a-*
-7.79046
New Point (Non-Optimum): {-0.683983, 0.603147} after 2 iterations
In general, a negative step length is not acceptable because it implies an increase in the function value. However, if we ignore this and let the method continue, it does converge to the inflection point in 35 iterations.
{opt, hist} s SteepestDescent [f, vars, xO];
Optimum:{-0.964315,0.978407} after 35 iterations
The following search path dramatically shows the difficulty the method is having with this function.
xhist = Drop[Transpose[hist][[1]], 1] ;
PlotSearchPath[f, {x, -1.5, 0}, {y, 0, 1.5}, xhist, PlotPoints 30,
Epilog -» {RGBColor[1, 0,0], Disk[{-.96, .98},0.015],
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
Line[{{-0.96,0.98),{-.75,1.2}}],
Text["Stationarypoint·,{-.72, 1,2}, {-1,0}]}];
F I G U R E 5.2 5 C o n v e r g e n c e t o t h e i n f l e c t i o n p o i n t o f /
(x, y) = e~x~? + xy + y2
u s in g t h e s t e e p e s t d e s c e n t m e t h o d.
The conjugate gradient method encounters a negative step length also; however, if we let it continue, the convergence is quite rapid.
ConjugateGradient [£, vars, xO, PrintLevel -> 2, Maxlterations -> 2];
f E'x_y + xy + y2
-E-*-y +y \
E-x-y + x + 2y I
Vf
Using PolakRibiere method with Exact line search ***** Iteration 1 ***** Current point -* {-1.4, 1.5}
Direction finding phase:
-0.595163\
, 1
0.595163\ Λ
X)
\0.695163) \-0.695163)
I |V f (x) | | -* 0 .915134 0-*O. f (x) -*1.05484
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
Step length calculation phase:
/-1.4 - 0.595163a\
X \
1.5- 0.695163a J
/ΐ.5-0.904837Ε1·29033α- 0.695163α\
Vf (xki) -» ^ 1.6 _ 0.904837E1· 290330'- 1.98549a j
άφ/da m
Vf (xki) .d = 0 -2.005 + 1.16753E1'290330 +1.79397a == 0
a -> 0.235605
***** Iteration 2 ***** Current point-»(-1.54022,1.33622}
Direction finding phase:
^ ( I
0.109909 \ J 7-0.124786\
Vf(x) -» 0940984j d~> ^0.0767212)
| |V f (x) | | -> 0.144688 0-> 0.0249973 f(x) ->0.953709
Step length calculation phase:
/- l.54022 - 0.l24786a\
( l.33622+0.0767212a)
/1-33622 - 1.22631E0-0480652a + 0.0767212a\
Vf (xki) ^ 1.13221 - 1.2263IE0·0480652® + 0.028656a ]
άφ/da s Vf (xki) ,d= 0 -» -0.0798772 + 0.0589427E0·048065201 - 0.00737524a == Q a -» -4 .34465
Unconstrained:: step: Negative step length (-4.34465) found. Check direction or change line search parameters.
/-I.54022 - 0.124786a]
\1.33622 + 0.0767212a/
/l.33622 - 1.22631E°-0480652a + 0.0767212a\
\ 1.13221 - 1.22631E0-0480652ot + 0.028656a j
άφ/da = Vf (xki) .d = 0 -» -0.0798772 + 0.0589427Ε0·0480652α - 0.00737524a == 0 a-» -4.34465
New Point (Non-Optimum) : (-0.99807, 1.00289}—after 2 i t erations {opt,
hist) = ConjugateGradient [f, vars, xO] ;
Using PolakRibiere method with Exact line search
Negative step length found.
Negative step length found.
Optimum: {-1., 0.999998} after 4 iterations
T a b l e P O r m [ h i s t ]
X
d
II? fix)11
f (X)
β
-1.4
-0.595163
0.915134
1.05484
0.
1.5
-0.695163
-1.54022
-0.124786
0.144688
0.953709
0.024997
1.33622
0.0767212
-0.99807
-0.0109613
0.0146931
1.0000
0.026161
1.00289
-0.0105088
-1.00067
-0.000171299
0.000192734
1.
0.004173
1.0004
0.000102372
-1.
9.23086 X10“®
1.
—
0.999998
x h i s t = Dr o p [ Tr a n s p o s e [ h i s t ] [ [ 1 ] ], 1 ];
pl o t s e a r c hpa f c hf f, {
χ
, -1.5, 0}, {y, 0, 1-5}, xhist, PlotPoints -*
30, Epilog -» {RGBColor[l, 0, 0] , Disk [{- .96, .98},
0.015] ,
L i n e [ { { - 0.9 6, 0.9 8 }, {-.75, 1.2}}],
Text["Stationarypoint", {-.72, 1.2}, {-1, 0}] }];
y
FIGURE 5.26 Conveigence to the inflection point of / (x, y) = e x y + xy + y2 using the conjugate gradient method.
C h a p t e r 5 U n c o n s t r a in e d P r o b le m s
5 * 3.4 Q u a s i - N e w t o n M e t h o d s
The Modified Newton method has the fastest conveigence rate but has the drawback that it requires computation of the Hessian matrix at each iteration. A class of methods, known as Quasi-Newton methods, has been developed that approaches the convergence rate of the Newton's methodbut without explicitly requiring the Hessian matrix. In place of the Hessian matrix, these methods start with a positive definite matrix (usually an identity matrix), denoted by Q. Using the gradient information at the current and the previous steps, the matrix Q. is updated so that it approximates the inverse of the Hessian matrix. Thus, the basic iteration of all Quasi-Newton methods is as follows:
1. Determine direction: d* = — Q*V/ (x*)
2. Find step length and update current point xk+l = xk + a^dk
3. Update matrix Qfor next iteration Q_fc+1
Different Quasi-Newton methods differ in the way the matrix Q. is updated.
DFP (Davidon, Fletcher, and Powell) Update
The DFP formula starts with an identity matrix and updates it to approximate the inverse of the Hessian matrix as follows.
ο Μ _ ο 1 ( q V k q W
( q W ( q ^ Q.k q *
where
q* = V/( x * +1) - V/( x * ) sk = xk+1 - xk =
a j t d k
It can be shown that Q* is positive definite as long as (q*)7»* > 0. If the step length is computed exactly, this should always be the case. However, with approximate step-length calculations, this is not necessarily the case. Hence, in numerical implementations, the matrix Q* is updated only if (q*)rsfc > 0. Tb avoid problems due to round off, it is desirable to reset the Q. matrix to the identity matrix after a specified number of iterations. Usually the number of iterations after which it is reset is equal to the number of optimization variables.
B F G S ( B r o y d e n, F l e t c h e r, G o l d f a r b, a n d S h a n o n ) U p d a t e
The BFGS formula is similar to DFP in that it also updates the inverse Hes­
sian matrix. However, the numerical experiments show it to be consistently superior to the DFP. The formula is as follows:
The numerical considerations to update and reset Q, are the same as those for DFP.
The procedures are implemented in the following function that is included in the OptimizationTbolbox 'Unconstrained' package.
N e e d s ["O p t i m i z a t i o n T o o l b o x'U n c o n s t r a i n e d'■];
?QuasiNewtonMethod
QuasiNewtonMethodff, vars, xO, opts]. Computes minimum of f(vars) starting from xO using a Quasi-Newton method. User has the option of using either 'BFGS' (default) or 'DFP' method. Also step length can be computed using either the 'Exact' analytical line search or an 'Approximate' line search using Armijo's rule. The number of iterations after which the inverse hessian approximation is reset to identity matrix can be specified using ResetHessian option. The default is Automatic in which case the reset interval is set to the number of variables. See Options[QuasiNewtonMethod] to see a list of options for the function. The function returns {x, hist}, 'x' is either the optimum point or the next point after Maxlterations. 'hist' contains history of values tried at different iterations.
Options[QuasiNewtonMethod]
{PrintLevel -»1, Maxlterations 50, ConvergenceTolerance -» 0 . 01, StepLengthVar -» a,
Method -» BFGS, LineSearch -» Exact,
ArmijoParameters {0 .2, 2} , ResetHessian-» Automatic}
Example 5.26 Use a Quasi-Newton method and the given starting point to find the minimum of the following function:
f = x4 + y4 + 2x2y2 - 4x + 3; vars = {x, y>; xO = {1.25, 1.25};
All intermediate calculations are shown for the first two iterations. QuasiNewtonMethod[f, vars, xO, PrintLevel -+ 2, Maxlterations -» 2];
f 3 - 4x + x4 + 2x2y2 + y4
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
Using BFGS method with approximate inverse heaaian reset after 2 iterations and Exact line search
***** Iteration 1 ***** Current point-»{1.25, 1.25}
Direction finding phase:
| |V £{ac) | | -* 19.4751 f(x)^7.76563 d -* (-11.625 - 15.625)
Step length calculation phase:
/l.25-11.625a\
\l. 25-15.625a)
(-1.7636.5
(-0.0353314 + a) (° .018656 - 0.251812a + a
2) \
V (xk -» ^ -23705.0(-0.08 + a)(0.00823927 - 0.179616a + a2) )
άφ/da = Vf (xki) .d = 0 -»
575417. (-0.04791367+a) (0.0137569 - 0.22151a + a2)
a->0.0479137
Updating inverse hessian:
qT.S-> 18.1728
Q'q_> 1-14.1578
s.sT ->
0.310245 0.416996
0.416996 0.560479
Q. q. sT ->
7.57347 10.1794
7.88587 10.5993
[Q.q.sT]T + Q.q.sT->
15.1469 18.0653
18.0653 21.1986
Iteration 2 ***** Current point->{0.693003, 0 .501349}
Direction finding phase:
Inverse Hessian->
0.545557-0.484603 -0.484603 0.518285
Vf (x) ->
-1.97198
1.46715
| |V f (x) | | ->2.4579 f(x) -> 0.763231 d-> (1.78682 - 1.71603)
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
Step length calculation phaseί
(0.693003
+1.78682α\ xkl“* [θ.501349 - 1.71603a)
I
43.8663 (-0.162591+α) (θ.276488+0.673591a+a2) \
Vf (xki) -* ^_42.1285 (-0.292156 + α) (θ.119203+0.123158a+a2)/
άφ/da
s vf (xki) . d = 0 -»
150.675 (-0.197956 +a) (0.202543 + 0.382693a + a2) == 0
a-» 0 .197956
New Point (Non-Optimum):{1.0467,0.161649} after 2 iterations
The printout of the intermediate results is suppressed (default option), and the method is allowed to continue until convergence. Computation history is saved in hist.
{opt, hist} = QuasiNewtonMethod[f, vars, xO, ResetHessian-» 10];
Using BFGS method with approximate inverse hessian reset after 10 iterations and Exact line search
Optimum: {1.00004, 0.000106729} after 4 iterations
Tabl©Form[hist]
x
1.25
1.25
0.693003
0.501349
1.04672
d
-11.625
-15.625
1.78682
-1.71603
-0.0144283
Il?f(x)I I
19.4751 2 .45789 1.00564
f (*)
7.76563
0.763231
0.0714469
Inverse Hessian 1 0 0 1
0.545557
-0.484603
0.108982
0.161649
1.00144
-0.00120258
1.00004
0.000106729
-0.051902
-0.00231565
0.0021541
0.018013
0.0000154181
-0.0847718
0.119708
-0.0493572
0.000608993 3.06396x10-®
The first column is extracted from the history, and the heading is removed to get a list of all intermediate points tried by the algorithm.
Xhist = Drop[Transpose[hist][[1
]],
1]
{{1.25, 1.25} , f 0.693003, 0.501349} , fl .04672, 0.161649} ,
{1.00144, -0.00120258} , {1.00004, 0.000106729}}
C h a p t e r 5
P r o b l e m s
Using the PlotSearchPath function, the search path is shown on a contour map of function f. The search path clearly demonstrates that the method is behaving similar to the ModifiedNewton's method.
PlotSearchPath[f , {x, 0,
1.3), {y, 0, 1.3}, xhist, PlotPoints ->30,
Epilog-» {RGBColorfl, 0, 0] , Disk[{l, OJ, 0.015] ,
Text ["Optimum", {1.01, 0.05), {-1, 0>] >];
y
F I G U R E 5.27
A c o n t o u r p l o t s h o w in g t h e s e a r c h p a t h t a k e n b y t h e Q u a s i- N e w to n's m e th o d.
The column containing the norm of the function gradient is extracted in preparation for a plot showing how the 11V/11 is decreasing with the number of iterations. The plot shows the rate of convergence of the method.
nomGradhist = Drop[Transpose [hist] [ [3] ] , 1]
{19.4751, 2.4579, 1.00564, 0.018013, 0.000608993}
ListPlot[normGradhist,
PlotJoined -» True, AxesLabel -*
{"Iterations", "| | v f | |"}];
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
HVfll
10
15
5
Iterations
2
3
4
5
F I G U R E 5.2 8 A p l o t s h o w i n g ||Vf
|| a t d i f f e r e n t it e r a t io n s.
The same solution is obtained using the DFP updates.
(opt* hist) = QuasiNewtonMethodtf , vars# xO, Method -» DFP, ResetHessian -» 10] s
Using DFP method with approximate inverse hessian reset after 10
iterations and Exact line search
Optimum:{1.00004,0.000106729} after 4 iterations
Example 5.27 Use a Quasi-Newton method and the given starting point to find the minimum of the following function:
f = (x + y)2 + (2(χ2 +Y2 -1) - | ) 2; vars = {x, y>; xO = {-1.25, 0.25};
All intermediate calculations are shown for the first two iterations.
QuasiNewtonHethod[f, vars, xO, Method -» DFP, PrintLevel -> 2,
Maxlterations -» 2, ResetHessian-» 10];
f (x + y) 2 + + 2 (-1 + x2 + y2) |
Using DFP method with approximate inverse hessian reset after 10 iterations and Exact line search
***** iteration 1 ***** Current point-»{-1.25, 0.25}
Direction finding phase:
Vf (_5?r + 16χ3 + 2y+16xy21 2x - ^ + 16x2y + 16y3 ;
-0.166667
-11.1667
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
f |V f (x) | | -> 11.1679 f(x) -> 1.84028 ά-> (11.1667 0.166667)
Step length calculation phase:
-1.25 + 11.1667a] ^0.25 + 0.166667a]
/22283.7 {-0.198059 + a) (-0.115053 + a)
(-0.0219909 + a) \
Vf (xki) -» ^ 332.593 (-0.182098 + α) (θ.00188867 + a) (l.45705 +a) j
άφ/da = Vf (xki) . d = 0
248890.(-0.197979 + a) (-0.114697 + a)(-0.0220681 + a) == 0 a -> 0. 0220681
Updating inverse hessian:
,11.1948 ] I
0.0607265 0.000906365 ]
Q'g_> \- l.71935) S'S \0.000906365 0.0000135278/
I 2/
i - 0 · ■
„ , _.7587l 0.0411748 \
Q.q.sT-»] n ,423694 -0.00632379
m . 5.51742-0.382519 [Q.q.sT]T + Q. q. sT -> | ,0,382519 _0.0126476
/θ.0457651 0.154654]
\
0.154654 1.00483 j
Iteration 2 ***** Current point -> {-1.00357, 0.253678}
Direction finding phase:
(0
.0457651 Q . 154654\ „ _ , , /0.0281495\
Inverse Hessian-» „ , , Vf (x) -» „ ..
^ 0.154654 1.00483 / \ -1-88601 J
| |V f (x) | | -> 1.88622 f(x) -> 0.598561 d-> (0.290392 1.89077)
Step length calculation phase:
'-1
.00357 + 0.290392a\
γ]ς^ \ ι ι
0.253678+1.8907a f
Vf (xki) -> (17‘0023
(-3.38977 +a) (-0.0103728 + a) (0 .0470866 + a)
110.703 (-0.173294 + α) (θ.098311 + 0 .41033a + a2) άφ/da = Vf (xki) .d = 0 ->
214.251 (-0.188279 + a) (0.0881986+ 0.342583a + a2) ==0
a->0.188279
5.3 r T n r r i m a t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
Updating inverse hessian:
[ 2.23607 )
/0.24022\
Q'q_> ( 1.8 9 0 0 2/ S'S
( V
00298932 0.0194637]
,0194637 0.12673 /
/0.0131339 0.0855162
Q.q.sT-> 0.103336 0.672831
(0
.0262679 0.188852\
[ O.q.s T J T + Q.q.s T ^ [ 0>1βββ52 i.3 4 5 6 6 )
0.035476 0.0610589]
0.061059 0.222211 /
New Point (Non-Optimum):{-0.948898,0.60967} after 2 iterations
{opt/ hist} = QuasiNewtonMethodff, vars, xO, ResetHessi&n -► 10] ;
Using BFGS method with approximate inverse hessian reset after 10 iterations and Exact line search
Optimum: {-0.763746, 0.763679} after 5 iterations TableForm[hist]
X
d
11vf(x)||
f (X)
Inverse Hessian
-1.25
11.1667
11.1679
1.84028
1 0 Λ *1
0.25
0.166667
u 1
r, rs « m ** ι- λ
Λ 1 Γ
-1.00357
0.253678
0.290392
1.89077
1.88622
0.598561
U * 0457651 0.154654
U · I j
1.00
-0.948898
0.60967
0,0594846
0.061382
2.30597
0.159544
0.035476
0.0610589
0.06
0.22
-0.784161
0.779661
0.0164554
-0.075594
0.99292
0.0126152
0.0901217
0.0691822
0.06
0.18
-0.777963
0.751188
0.00961856
0.00845028
0.0910383
0.000749257
0.0923642
0.0724209
0.07
0.10
-0.763746
0.00265284
9.83226x 10“®
—
0.763679
xhist = prop I
[ [!] ], 1]
PlotSearchPath[f, {x, -1.25, 0}, {y, 0, 1.25}, xhist, PlotPoints-» 50, Epilog -*
(RGBColor [1, 0,0], Disk[ {- .76, .76}, 0.01],
Text["Optimum", {-.74, .76}, {-1,0}]}];
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
FIGURE 5.29 A contour plot showing the search path taken by the Quasi-Newton method.
normGradhist = Drop[Transpose[hist][[ 3 ] ], 1] ;
ListPlot[normGradhist#
Plot Joined -» True# AxesLabel -» {"Iterations", n| |Vf | |"}];
HVfll
F I G U R E 5.3 0 A p l o t s h o w i n g || V/1 | a t d i f f e r e n t i t e r a t i o n s.
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
Using the DFP updates the solution as follows:
(opt/ hist} = QuasiNewtonMethod[f, vars, xO, Method -*■ DFP, ResetHessian -» 10] ;
Using DFP method with approximate inverse hessian reset after 10
iterations and Exact line search
Optimum: {-0.763746, 0 .763679} after 5 iterations
Example 5.28 Use a Quasi-Newton method and the given starting point to find the minimum of the following function:
f = (Xi - l ) 4 + (Xa - 3)2 + 4 (x3 + 5)2; vars = {*1, Xa ·
x3 }' = i -^·* -2, 1};
All intermediate calculations are shown for the first two iterations.
QuasiNewtonMethodff, vars, x0, ResetHessian ·> 10, PrintLevel -*
2
,
M a x l t e r a t i o n s -» 2 ];
f -» (-1 + xx) 4 + (-3 + x2)2 + 4 (5 + x3)2 -4 + 12x1 -
l2xi + 4xi
Vf
-6 + 2xi
40 + βχ3
Using BFGS method with approximate inverse hessian reset after 10
iterations and Exact line search
***** iteration 1 ***** Current point-*{-1, -2, 1}
Direction finding phase:
Ί
0
0'
-32'
Inverse Hessian-»
0
1
0
Vf (x) -»
-10
,0
0
lj
, 48,
I |V f (x) I I ->58.5491 f(x)->185 d-» (32 10 - 48)
Step l ength calculation phase:
xki
Vf (xki) ->
1 + 32a\
2 + 10a , 1 - 48a ;
'32(-1+ 16a)3) -10 + 20a 48 - 384a
d0/da = vf (xki) .d = 0 -» 4 (-857 + 16946a - 196608a2 + 1048576a3) == «-> 0.107925
Updating inverse hessian:
44.2856
2.1585 1-41.4432,
qT.s -> 369.967
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
44.2856'
11.9274
3.7273
-17.8911’
Q.q-»
2.1585
s. sT -»
3.7273
1.16478
-5.59096
-41-4432,
-17.8911
-5.59096
26.8366 ,
Q.q.sT-*
152.945
7.45461
-143.128
47.7953
2-32956
-44.7276
-229.417 -11.1819 214.693
[Q.q.sT]T + Q.q.sT-»
305.89 55.2499
55.2499 4.65913
[-372.546 -55.9096
-372.5461 -55 .9096 429.385 j
Q->
0.52641
-0.0389584
0.477151
-0.0389584 0.477151
1.0219 -0.0144481
-0.0144481 0.634124
***** Iteration 2 ***** Current point-»{2.4536, -0.920749,-4.1804} Direction finding phase:
ί
0.52641 -0.0389584
0.477151
12.285’
Inverse Hessian-» -0.0389584 ' 1,0219
-0.0144481
Vf(x) -»
-7.841
[ 0.477151 -0.0144481
0.634124 ,
β
.5567
| |V f (x) | | -»15.9818 f(x) ->22.5238 d-» (-9.90133 8.58659 - 10.1332)
Step length calculation phase:
r 2.45360- 9.90133a ' xki-» -0.920749 + 8.58659a , -4.1804 - 10.1332a \
-3882.76(^0.146809 + a)(0.0215527 - 0.293617a + a
2)’
Vf (xki) -» -7.8415 + 17.1732a
, 6.55677 - 81.0656a
άφ/άαε
Vf (xki) .d = 0 -»
38444.5(-0.138091 +a)(0.0481115 - 0.302335a + a2) ~ 0
a-» 0.138091
Updating inverse hessian:
-12.283
2.37146 -11.1944 J
qT.s -» 35.2708
-11.8997’
'1.86947
-1.62123
1.91325'
Q-q-»
3.06366
s. sT -»
-1.62123
1.40596
-1.6592
,-12.9938,
,1.91325
-1.6592
1.95805J
Q. q. sT -»
/Ί6.2704 -14.1099 16.6514^
-4.1889 3.63268 -4.287
17.7662 -15.4072 18.1823/
[Q.q.sT]T + Q.q.sT-»
32.5407
-18.2988
34.4176
-18.2988
7.26536
-19,6942
34.4176
-19.6942
36.364
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
0.105973 0.0443696 0.0152603'
q.+ 0.0443696 1.1935 0.0982430
[θ.0152603 0.098243 0.129068 ,
New Point(Non-Optimum):{1.08632, 0.264982,- 5.57971?after 2 iterations
{opt, hist} = QuasiNewtonMethodff, vars, xO, ResetHessian -*■ 10];
Using BFGS method with approximate inverse hessian reset after 10 iterations and Exact line search
Optimum: {1. 03202, 2 .99885,-4.99988} after 5 iterations
normGradhist = Drop[Transpose[hist][[3]], 1]
{58.5491,15.9818, 7.17142, 0.990177, 0,0413141, 0.00249873}
ListPlot[normGradhist,
plotJoined-+■ True, AxesLabel {"Iterations", " | ] vf | 1"}];
iivfii
FIGURE 5.31 A plot showing ||V/|| at different iterations.
Using the DFP updates, the solution is as follows:
{opt, hist} = QuasiNewtonMethod[f , vars, xO, Method -+ DFP, ResetHessian -*
10];
Using DFP method with approximate i nverse hessi an r e s e t a f t e r 10 iterations and Exact line search
O p t i m u m:{1.03203,2.99885,-4.99988} a f t e r 5 i t e r a t i o n s
Example 5.29 Data fitting Consider the data-fitting problem first discussed in Chapter 1. The goal is to find a surface of the form ^computed = ^l*2 + ci y 2 + c3xy to best approximate the data in the following table.
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
Point
X
y
^observed
1
0
1
1.26
2
0.25
1
2.1Θ
3
0.5
1
0.76
4
U .75
1
l.Zb
5
1
2
1.86
6
1.25
2
1.43
7
1.5
2
1.2Θ
8
1.75
2
0.65
Θ
2
2
1.6
The best values of coefficients c\,C2, and c3 are determined to minimize the sum of the squares of error between the computed z values and the observed values.
Minimize f = i [-^observed(.Xji yi) ^computed(*i> Yi)^
Using the given numerical data, the objective function can be written as fol­
lows:
xyData= {{0, 1}, {0.25, 1}, {0.5,1}, {0.75, 1},
{1,2}, {1.25,2}, {1.5,2}, {1.75,2}, {2,2}}; zo = {1.26, 2.19, .76, 1.26, 1.86, 1.43, 1.29, .65,1.6}; zc = Map[ (c^2 + Cjy2 + czxy)
/. {x -» # [ [1] ] , y -» # [ [2] ] }&, xyData]; f = Expand [Apply [Plus, (zo - zc)2] ]
18.7 - 32.8462c! + 34.2656ci - 65.58c2 + 96.75c.,c2 + 84c2 -43.425c3 + 79.875^3 + 123.c2c3 + 48.375C3
Using the BFGS method, the minimum of this function is computed as follows. All calculations are shown for the first iteration.
QuasiNewtonMethod[f, {c1#c2, c3}, {1,1,1},
ResetHessian -*
10, Maxlterations -> 1, PrintLevel -* 1];
Using BFGS method with approximate inverse hessian reset after 10
iterations and Exact line search
***** iteration 1 ***** Current point-»{1, 1, 1}
Direction finding phase:
Inverse Hessian-»
Ί
0 0’ 0 1 0
Vf (x)
212.31’
322.17
lo 0 1
,256.2 ,
5.3 U n c o n s t r a i n e d M i n i m i z a t i o n T e c h n i q u e s
I |v f (X) | | -» 463.15 f (x) -* 343 .114 (-212.31 -322.17 -256.2)
Step length calculation phase:
Ί - 212. 31a^
xki ->
1 ~322.17a
1
- 256.2a
'212.31-66183.8a’
Vf(xkl)-*
322 .17 - 106178.a 256.2 - 81372.5a ,
άφ/da
s Vf (xki) .d = 0 -» -214507. + 6.91065 * 107a == a ·*
0.00310401
Updating inverse hessian:
-205. 435\
qT.s -» 665.834
q
-329.578
-252.581
-205.435
Ό.434298
0.659025
0.524078'
Q.q-»
-329.578
s. sT -»
0.659025
1.00004
0 .795263
,-252.581,
,0.524078
0.795263
0.632419,
135.384
205.439
163.3721
Q.q.sT-»
217.196
329.585
262.096
166.454
252.586
200.865;
270.769
422.635
329.826
[Q.q.sT]T + Q.q.sT-»
422.635
659.169
514.682
,329.826
514.682
401.729,
0.804239 -0.314716 -0.240859}
Q-* -0.314716 0.49564 -0.3868
,-0.240859 -0.3868 0.703762
New point (Non-Optimum) :
{0.340987,-0.0000193931,0.204752} after 1 iterations
{sol, hist} = QuasiNewtonMethod[ f, [ c 1, c2, c 3 }, {1, 1, 1 }, ResetHessian -* 10];
Using BFGS method with approximate inverse hessian reset after 10 iterations and Exact line search
Optimum: {2.09108, 1.75465,-3 . 50823} after 3 iterations
Thus, the surface of the form ^computed = ^l*2 + fyy2 + c$xy that best fits the given data is as follows:
ζ
= 2.09108*2 + 1.75465
y2 - 3.50823xy
This is the same solution that was obtained in Chapter 4 using optimality conditions.
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
5.4 Concluding Remarks
By combining different line-search strategies and direction-finding methods, one can come up with a large number of methods for solving unconstrained problems. For practical problems, one has to base the choice on the relative computational effort required by different methods.
Since the conjugate gradient method has superior convergence properties but requires very little additional computational effort as compared to the steepest descent method, clearly there is no reason to use the steepest method for a practical problem. The modified Newton method has the fastest conver­
gence rate. However, each iteration of this method requires evaluation of the gradient vector and the Hessian matrix. Furthermore, the direction must be computed by solving a linear system of equations. Thus, computationally, each iteration of this method is a lot more expensive than the other methods. The Q.uasi-Newton methods avoid solving equations and need modest effort to up­
date the approximate inverse Hessian. For most practical problems, therefore, a Quasi-Newton method with BFGS updates is a good choice.
Regarding the step-length calculations, a method that requires the least number of objective function evaluations during line search is most preferred. The analytical line-search method obviously is a very good choice in this re­
gard. However, it is not suitable for a purely numerical solution. The examples presented in this chapter take advantage of Mathematica1 s symbolic algebra capabilities to implement this procedure. It will be extremely difficult to im­
plement this strategy in a traditional computer programming language, such as C or Fortran. For such implementations, one must choose a numerical line-search method. For exact line search, the choice is usually between the golden section search and the quadratic interpolation method. The quadratic interpolation method usually requires fewer function evaluations and has a slight edge over the golden section search. Most numerical implementations, however, use an approximate line search based on Armijo's rule, since this method frequently needs very few function evaluations. However, because of the approximate nature of the computed step length, the overall conver­
gence rate may be slower as a result. For additional details refer to Dennis and Schnabel [1983], Hestenes [1980], Himmelblau [1972], Polak [1971], Polak and Polak [1997], and Schittkowski [1980, 1987].
All methods presented in this chapter require computation of gradients. Ob­
viously, if an objective function is not differentiable, or if it is very difficult to compute its gradient, none of these methods can be used. For such problems, one must use search methods that require only the function values or gener­
ate an approximation of a gradient vector using finite differences. The search
5.5 P r o b l e m s
methods are essentially of combinatorial nature and are not discussed in this text. A good reference for a study of this subject is the book edited by Colin R. Reeves [1993]. For global optimization refer to Hansen [1992] and Hentenryck, Michel, and Deville [1997]. Recently, genetic algorithms have become popular for solving these problems as well. A recent book by Mitsuo Gen and Run- wei Cheng [1997] is a good starting point for a study of this field. Additional references are Goldberg [1989] and Xie and Steven [1997].
5.5 Problems
Descent Direction
For the following problems, check to see if the given direction is a descent direction at the given point. If so, compute the optimum step length using analytical line search. Compute the new point and compare the function values at the initial and the new point.
5.1. /( χ ι,χ 2) = 2*? +x%- 2 xix2 + 2x$+xi x° = [2, -1} d° = {-2, 3}
5.2. f i x, y) = 2X2 + x Sin[y] x° = [2, -1} d° = {-2,3}
5.3. f i x,y,z ) = 2xZ+xy + y/2? x° = {2,-1,1} d° = {-2, 3,-4}
5.4. f ( xi,x2) = 4 + x\ + x i 4 + 3 x° = {2, -1} d° = {-2, -3}
5-5. f ( xi,x2) = ( 4 - 4?+ 4 + ( xi 4 + 3)x2 x° = {2,-l} d° = {—2, 3}
Equal Interval Search
For the following problems, use the negative gradient direction and determine the function φ(α) for computing the step length. Compute the step length using equal interval search. Compute the new point and compare the function values at the initial and the new point. Start with the default parameters used in the examples. Adjust if necessary.
5·6· f i x i,*2> = 2xf + 4 ~ 2xixi + 2 4 +X* x° = {2, - 1}
5.7. f ( xt y) = 2X2 + xSin[y] x° = {2, -1}
5.8. /(x, y, z) = 2x2 +xy + y/z? x° = {2, -1,1}
5.9. f(x 1, x2) = 4 + x2 + *1*2 + 3 X° = [2, -1}
5.10. f ( x 1, χ2) = i 4 ~ *2>3 + 4 + C*i*2 + 3)*2 x° = (2, -1}
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
5.11. A multicell evaporator is to be installed to evaporate water from a salt water solution in order to increase the salt concentration in the solution. The initial concentration of the solution is 5% salt by weight. The desired concentration is 10%, which means that half of the water from the solution must be evaporated. The system utilizes steam as the heat source. The evaporator uses 1 lb of steam to evaporate 0.8» lb of water, where tt is the number of cells. The goal is to determine the number of cells to minimize cost. The other data are as follows:
The facility will be used to process 500,000 lbs of saline solution per day.
The unit will operate for 340 days per year.
The initial cost of the evaporator, including installation, is $18,000 per cell.
The additional cost of auxiliary equipment, regardless of the number of cells, is $9,000.
The annual maintenance cost is 5% of the initial cost
The cost of steam is $1.55 per 1000 lbs.
The estimated life of the unit is 10 years.
The salvage value at the end of 10 years is $2,500 per cell.
The annual interest rate is 11 %.
Formulate the optimization problem to minimize annual cost. Find the optimum number of cells using equal interval search.
5.12. Use the present worth method to solve problem 5.11.
Section Search
For the following problems, use the negative gradient direction and determine the function φ(α) for computing the step length. Compute the step length using section search. Compute the new point and compare the function values at the initial and the new point. Start with the default parameters used in the examples. Adjust if necessary.
5.13. f ( x i, x2) = 2xf + - 2xi*2 + 2a^ + x\ x° = [2, -1}
5.14. fi x, y) = 2x2 + x Sin[y] x° = {2, -1}
5.15. / (X,y, z) — Zx2 + xy + y/z3 X° = (2, - 1,1}
5.5 P r o b l e m s
5.16. f ( x1,*2) = *1+ * 2 + * i * 2 + 3 x° — {2, —1}
5.17. /(*ι, *2) = (xf ~ 4 f + *2 + (xi*2 + 3)χ2 x° = ί2» “!}
5.18. Same as problem 5.11, except use section search to find an optimum.
5.19. Same as problem 5.12, except use section search to find an optimum.
Golden Section Search
For the following problems, use the negative gradient direction and determine the function φ(ά)
for computing the step length. Compute the step length using golden section search. Compute the new point and compare the function values at the initial and the new point. Start with the default parameters used in the examples. Adjust if necessary.
5.20. f(xj, x2) = 2x{ +x%- 2x1*2 + 2x? + x\ x° = {2, -1}
5.21. f(x, y) = 2x? + x Sin[y] x° = {2, -1}
5.22. f ( x,y,z ) = 2x2‘+ x y + y/z? x° = f2, —1.1}
5.23. /(xi,x2) = xf + xf + *i*2 + 3 x° = {2, —1}
5.24. /(xi, x2) — 0*i - * 2)3 + xf + (xi*! + 3)X2 x° = {2, -1}
5.25. Same as problem 5.11, except use golden section search to find an optimum.
5.26. Same as problem 5.12, except use golden section search to find an optimum.
Quadratic Interpolation
For the following problems, use the negative gradient direction and determine the function φ(α)
for computing the step length. Compute the step length using quadratic interpolation. Compute the new point and compare the function values at the initial and the new point. Start with the default parameters used in the examples. Adjust if necessary.
5-27. f (*i, Xj) — 2xf + x| — 2x1*2 + 2xi + Xi x^ = {2, —1}
5.28. / (x, y) = 2Χ2 + xSin[y] x° = {2, -1}
5.29. f(x, y, ζ) = 2X2 + xy + y/z3 x° = {2, -1,1}
5.30. / ( x i, x 2 ) = x ^ + *2 + * 1*2 + 3 x° = {2,-1}
5.31. /(xi,x2) = (*i - xl)3 + * 2 + (*i*2 + 3)x2 x° = {2, -1}
5.32. Same as problem 5.11, except use quadratic interpolation to find an optimum.
5.33. Same as problem 5.12, except use quadratic interpolation to find an optimum.
Approximate Line Search Armijo's Rule
For the following problems, use the negative gradient direction and determine the function φ{ρϊ) for computing the step length. Compute the step length using Armijo's approximate line search. Compute the new point and compare the function values at the initial and the new point. Start with default parameters used in the examples. Adjust if necessary.
5.34. f i x i, *2) = 2*i + *2 - 2xi*2 + 2xj + x\ x° = [2, -1}
5.35. f ( x,y ) = 2X2 + xSin[y] x° = {2, - 1 }
5.36. f ( x,y,z ) = 2 ^ + x y + y/^ x ° = { 2,- l,l }
5.37. f { x\,x i ) = x f + x 2 + * 1*2 + 3 3t° = { 2,- 1 }
5.38. / ( x i, x 2 ) = ( x f - x % f + x 2 + ( * i *2 + 3)*2 x ° = (2, - 1 }
Steepest Descent Method
For the following problems, use the steepest descent method, with analytical line search, to compute the minimum point. Show complete calculations for the first two iterations. Plot a search path for problems with two variables. Plot the history of the norm of the gradient. Verify the solution using optimality conditions.
5.39. /( x i. x?) = 2xj + *2 ~ 2xix2 + 2xj + xf x° = [2, — 1}
5.40. f i x, y ) = 2X2 + x S i n t y ] x ° = {2, - 1 }
5.41. /( X l,* 2 ) = x?+ X 2 + * l *2 + 3 x° = { 2,- 1 }
5.42. / ( x i, x 2 ) = i x f ~ x 32 ) 3 + x % + C*i*2 + 3)X2 x ° = {2, - 1 }
5.43. f ( x,y ) = x + 2X2 + 2 y — x y + 2y 2 x ° = {2, —1}
i 7v2 n
5.44. / (x, y) = -3x + 2X2 - 2y + 3xy + x° = {2, - 1 }
z
5.5 P r o b l e m s
5.45. /(*, y, z) = xyz - 7-7 2 , 2 ■ 2 x° = { - 0.1, - 0.2,1}
2 + xl + y 2 + z 2
5.46. /(x, y, 2) = 2k2 - 2xy + y2 - z - yz + z2 x° = {1, 2, 3}
2 7y 2 5z2 n
5.47. f ( x,y,z ) = 3x + 2xz + 2y-3^y+-— +z + xz + yz+~— x° = {1, 2, 3}
z 2
5.4 8. / = x i + 4 * j + 2 x2- x i x 2+ 4 x § + 3*3 + 4 x f + 4 x 4+ 2x 1x 4+ 2x 2x 4 + 2x 3x 4+ ^ x ° = { 1, 2, 3, 4 }
5.49. A small firm is capable of manufacturing two different products. The cost of making each product decreases as the number of units produced increases and is given by the following empirical relationships,
1,500 2,500
c\ = 5 Λ C2 = 7 H---------
m «2
where m and «2 are the number of units of each of the two products produced. The cost of repair and maintenance of equipment used to produce these products depends on the total number of products pro­
duced, regardless of its type, and is given by the following quadratic equation:
(«1 + n2) [0.2 + 2.3 * 10 5 (ni + n2) + 5.3 * 10 9 (n 1 + n2)2]
The wholesale selling price of the products drops as more units are produced, according to the following relationships:
pi = 15 — O.OOlni p2 — 25 — 0.0015M2
Determine how many units of each product the firm should produce to maximize its profit. Choose any arbitrary starting point and find an optimum solution using the steepest descent method.
5.50. For a chemical process, pressure measured at different temperatures is given in the following table. Formulate an optimization problem to determine the best values of coefficients in the following exponential model for the data. Choose any arbitrary starting point and find opti- mum values of these parameters using the steepest descent method.
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
Pressure = ae^T
Tfemperature (T° C)
Pressure (mm of Mercury)
20
15.45
25
19.23
30
26.54
35
34.52
40
48.32
50 .
68.11
60
98.34
70
120.45
5.51. A chemical manufacturer requires an automatic reactor-mixer. The mix­
ing time required is related to the size of the mixer and the stirring power, as follows:
V s
Γ = 1,000-^
where S = capacity of the reactor-mixer, kg, P = power of the stirrer, k-Watts, and T is the time taken in hours per batch. The cost of building the reactor-mixer is proportional to its capacity and is given by the following empirical relationship:
Cost = $60,000VS
The cost of electricity to operate the stirrer is $0.05/k-W-hr, and the overhead costs are $137.2 P per year. The total reactor to be processed by the mixer per year is 107 kg. Time for loading and unloading the mixer is negligible. Using present worth analysis, determine the capacity of the mixer and the stirrer power in order to minimize cost. Assume a 5-year useful life, a 9% annual interest rate compounded monthly, and a salvage value of 10 % of the initial cost of the mixer. Find an optimum solution using the steepest descent method.
5.52. Use the annual cost formulation in problem 5.51.
5.5 P r o b l e m s
Conjugate Gradient Method—Polak-Ribiere
For the following problems, use the conjugate gradient method—Polak-Ribiere, ^vith analytical line search, to compute the minimum point. Show complete calculations for the first two iterations. Plot a search path for problems with two variables. Plot the history of the norm of the gradient of functions. Verify the solution using optimality conditions.
5.53. /( *i,x2) = 2x$ +x$ - 2xix2 + 2x$+xj x° = {2, -1}
5.54. f i x, y) = 2x*+x Sinty] x° = {2, -1}
5.55. f i x 1, *2) = *i + *2 + *1*2 + 3 x° = f2’ - 1}
5.56. /(*1, *2) = (*1 - *2)3 + *2 + (*1*2 + 3)*2 x° = {2, -1}
5.57. /(*, y) = * + 2X2 + 2y - xy + 2y2 x° = {2, - 1 }
7v2
5.58. fi x, y) = —3x + 2X2 - 2y + 3xy + —- x° — {2, -1}
z
5.59. /(x, y, z) = xyz - - — ~ - - x° s= {-0.1, -0.2,1}
2 + xl + y2 + zl
5.60. fi x, y,z) = 2x* — 2xy + y2 —z — yz + ζ2, x° = {1, 2, 3}
7’y2 5Z2
5.61. fi x, y, z) = 3x + Zx2 + 2y — Zxy Λ
-------1- z + xz + yz-\ x° = {l,2,3}
2 2
5.6 2. / = * i + 4 X j +2X2—
* 1 * 2 + 4 λ | + 3 Χ 3 + 4Χ3 + 4Χ4 + 2 χ ι Χ 4 + 2 Χ 2 * 4 + 2 * 3 Χ 4 + * 4 x° = {1, 2, 3, 4 }
5.63. Find an optimum solution of the manufacturing problem 5.49, using the conjugate gradient method—Polak-Ribiere.
5.64. Find an optimum solution of the data-fitting problem 5.50, using the conjugate gradient method—Polak-Ribiere.
5.65. Find an optimum solution of the mixer problem 5.51, using the conju­
gate gradient—Polak-Ribiere method.
5.66. Find an optimum solution of the mixer problem 5.52, using the conju­
gate gradient method—Polak-Ribiere.
Conjugate Gradient Method—Fletcher-Reeves
For the following problems, use the conjugate gradient method—Fletcher-Reeves, with analytical line search, to compute the minimum point. Show complete
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
calculations for the first two iterations. Plot a search path for problems with two variables. Plot the history of the norm of the gradient of functions. Verify the solution using optimality conditions.
5.67. f i x i,*2) = 2*ί + * 2 ~ 2xi*2 + 2xJ + xf x° = {2, -1}
5.68. fix, ν) = 2X2 + xSin[y] x° = (2, -1}
5.69. /(xi,x2) = X? +xf +*1*2 + 3 x° = {2, -1}
5.70. f i x 1, %2> = (*1 - *2>3 + * 2 + (*1*2 + 3)*2 X° = (2, -1}
5.71. fi x, y) = x+ 2X2 + 2y - xy + 2y2 x° = {2, -1}
7γ2
5.72. /(x, y) = -3X + 2X2 - 2 y + 3xy + x° = {2, -1}
z
5.73. fi x, y, z) = xyz - ~ 2 Γ 2 x° = ί- 0 ·1’ “ °·2« ^
2 + xl + y l + zl
5.74. f ( x, y, ζ) = 2X2 — 2xy + y2 — z — yz + z2 x° = {1, 2, 3}
7y2 5 z2
5.75. /( x,y,ζ) = 3x+2X2 + 2y — 3xyH-bz + xz + yzH x° = {l,2,3}
2 2
5.76. / =χι+4χι+2χ2-χιχ2+4χ2+3χ3 + 4χ3+4χ4 + 2χιχ4+2χ2χ4 + 2χ3χ4+χ4 x° = {1, 2, 3, 4}
5.77. Find an optimum solution of the manufacturing problem 5.49, using the conjugate gradient method—Fletcher-Reeves.
5.78. Find an optimum solution of the data-fitting problem 5.50, using the conjugate gradient method—Fletcher-Reeves.
5.79. Find an optimum solution of the mixer problem 5.51, using the conju- gate gradient method—Fletcher-Reeves.
5.80. Find an optimum solution of the mixer problem 5.52, using the conju­
gate gradient method—Fletcher-Reeves.
Modified Newton Method
For the following problems, use the modified Newton method, with analytical line search, to compute the minimum point. Show complete calculations for the first two iterations. Plot a search path for problems with two variables. Plot the history of the norm of the gradient of functions. Verify the solution using optimality conditions.
5.5 P r o b l e m s
5.81. /( *i,*2> = 2x? +x% ~ 2X}X2 + 2xf + x j x° = {2, -1}
5.82. fi x, y) = 2X2 + xSin[y] x° = {2, -1}
5.83. / (*i, *2) = *i + *2 + *1*2 + 3 x° = i2» - 1 }
5.84. / (*i, *2) = (*i - *f)3 + *2 + (*1*2 + 3)*2 X° = {2, -1}
5.85. /(*, y) = * + 2X2 + 2y - xy + 2y2 x° = {2, -1}
7y2
5.86. fi x, y) = ~3x + 2x2 - 2 y + 3xy + ~ ~ x° = {2, -1}
z
5.87. /(*, y, ζ) = 2X2 - 2*y + y2 - z - yz + z2 x° = {1, 2, 3}
7y2 δζ2
5.88. /( *,y,z) = 3* + 2*2 + 2y-3xy + — + z + xz + yz+ — x° = {1,2,3}
z z
5.89. / =Xi+4*f + 2*2-*1*2+ 4*^+ 3*3+4*3+ 4*4+ 2*1 *4+2*2*4+ 2*3*4+*4 x° = {l,2, 3, 4}
5.90. Find an optimum solution of the manufacturing problem 5.49, using the modified Newton method.
5.91. Find an optimum solution of the data-fitting problem 5.50, using the modified Newton method.
5.92. Findanoptimumsolutionofthemixerproblem5.51, using the modified Newton method.
5.93. Find an optimum solution of the mixer problem 5.52, using the modified Newton method.
BFGS Method______________________________________________________
For the following problems, use the BFGS method, with analytical line search, to compute the minimum point. Show complete calculations for the first two iterations. Plot a search path for problems with two variables. Plot the history of the norm of the gradient of functions. Verify the solution using optimality conditions.
5.94. /(xi,x2) = 2xf + 4 ~ 2xix2 + 2x? + x\ x° = {2, -1}
5.95. fi x, y) = 2Χ2 + xSin[y] x° = {2, -1}
5.96. /(xi,x2) = xf + *f +*i*| + 3 x° = {2,-1}
5.97. /(xi, x2) = (xf - xf)3 + *2 + (*1*2 + 3)X2 x ° = {2, -1}
C h a p t e r 5 U n c o n s t r a i n e d P r o b l e m s
5.98. /(x, y) = χ
+ 2X2 + 2y — xy + 2y2 x° = {2, —1}
7y2
5.99. f(x, y) = —3x + 2Χ2 - 2y + 3xy + ~ ~ x° = {2, -1}
z
5.100. /(x, y, z) = xyz - x° = {-0.1, -0.2,1}
2 + x^ + y l +
5.101. /( x, y, ζ ) = 2X2 - 2xy + y2 - z - yz + z2 x° = {1, 2, 3}
2 7y2 5z2 0
5.102. /( x,y,z ) = 3x + 2x2 + 2y- 3xy+ ■■ + z |-xz + yz+ — x° ={1,2,3}
z z
5.103. / — X l + 4 X j + 2 x 2 - X 1X2 + 4X^+ 3X3 + 4* 3 + 4X4 +2xiX 4 + 2X2X4 + 2x3x4 +X 2 x° = {1, 2, 3, 4}
5.104. Find an optimum solution of the manufacturing problem 5.49, using the BFGS Quasi-Newton method.
5.105. Find an optimum solution of the data-fitting problem 5.50, using the BFGS Quasi-Newton method.
5.106. Find an optimum solution of the mixer problem 5.51, using the BFGS Quasi-Newton method.
5.107. Find an optimum solution of the mixer problem 5.52, using the BFGS Quasi-Newton method.
ί
DFP Method
For the following problems, use the DFP method, with analytical line search, to compute the minimum point. Show complete calculations for the first two iterations. Plot a search path for problems with two variables. Plot the history of the norm of the gradient of functions. Verify the solution using optimality conditions.
5.108. / (χχ, X2) = 2xi + X2 — 2X}X2 + 2x? + x\ x° — {2, -1}
5.109. f(x, y) = 2x2 +xSin[y] x° = [2, -1}
5.110. /(xi,x2) = xf + x | + *i*f + 3 x° = {2,- 1}
5.111. /(xi,x2) = (*ι -;φ 3 + *2 + + 3)*2 x° = {2,-1}
5.112. /(x, y) = x+2X2 + 2y — xy + 2y2 x° = {2, —1}
7V2
5.113. f (x, y) = — 3x + 2x2 — 2y + 3xy Λ x° = {2, —1}
2
5.5 P r o b l e m s
5.114. f ix, y, z) = xyz — 2+- χ 2 ^ γ 2 + ζ 2 x° = t- 0 1 ’ “ °·2’
5.115. fi x, y, ζ) = 2X2 - 2xy + y 2 - z - yz + z2 x° = (1, 2, 3}
7y2
5.116. fi x, y,z) = 3x + 2x2 + 2 y - 3 x y + — + z + x z + y z + x° = {l,2, 3}
Z 2a
5.117. / ~ χι+4χϊ+2χ2-χιχ2+4*%ί + 3χ3+4χ3+4χ4 + 2χιχ4 + 2χ2χ4 + 2χ3χ4+χ% x° = { 1,2, 3, 4}
5.118. Find an optimum solution of the manufacturing problem 5.49, using the DFP Quasi-Newton method.
5.119. Find an optimum solution of the data-fitting problem 5.50, using the DFP Quasi-Newton method.
5.120. Find an optimum solution of the mixer problem 5.51, using the DFP Qμasi-Newton method.
5.121. Find an optimum solution of the mixer problem 5.52, using the DFP Quasi-Newton method.
CHAPTER SIX
L i n e a r P r o g r a m m i n g
When the objective function and all constraints are linear functions of opti­
mization variables, the problem is known as a linear programming (LP) prob­
lem. A large number of engineering and business applications have been suc­
cessfully formulated and solved as LP problems. LP problems also arise during the solution of nonlinear problems as a result of linearizing functions around a given point.
It is important to recognize that a problem is of the LP type because of the availability of well-established methods, such as the simplex method, for solving such problems. Problems with thousands of variables and constraints can be handled with the simplex method in a routine manner. In contrast, most methods for solving general nonlinear constrained problems, such as those discussed in chapter 9, do not perform as well for large problems.
This chapter presents the well-known simplex method for solving LP prob­
lems. Another class of methods, known as interior point methods, are discussed in the following chapter. The simplex method requires that the LP problem he stated in a standard form that involves only the equality constraints. Con­
version of a given LP problem to this form is discussed in the first section. In the standard LP form, since the constraints are linear equalities, the simplex method essentially boils down to solving systems of linear equations. A review of solving linear systems of equations using the Gauss-Jordan form and the LU
C h a p t e r 6 L i n e a r P r o g r a m m i n g
decomposition is presented in the second section. The basic solutions for an LP problem are presented in section 4. The traditional tableau form of the sim­
plex method is presented in detail in section 5. The procedure for sensitivity analysis based on the simplex method is presented in section 6. The tableau form is convenient for organizing computations for a hand solution; however, it is inefficient for computer implementation. The so-called revised simplex method, in which the computations are organized as a series of matrix opera­
tions, is discussed in section 7. The last section considers the sensitivity of the optimum solution as the objective function and the constraint coefficients are changed based on the revised simplex method.
6.1 The Standard LP Problem
This chapter presents methods for solving linear programming (LP) problems expressed in the following standard form:
Find x in order to Minimize / (x) = cTx Subject to Ax = b and x > 0.
where
Ύ1
x = [x\, x2,..., Xn] vector of optimization variables
rr*
c = [ci, c%,..., cn] vector of objective or cost coefficients
{ *\\ «12 · · · «1 <*21 «22 - · · «2η
A = m χ n matrix of constraint coefficients
V*ml «m2 · · · a mn/
b = [bi, b2,..., bm]T > 0 vector of right-hand sides of constraints
Note that in this standard form, the problem is of minimization type. All constraints are expressed as equalities with the right-hand side greater than or equal to (>) 0. Furthermore, all optimization variables are restricted to be positive.
6.1 T h e S t a n d a r d L P P r o b l e m
6.1.1 C o n v e r s i o n t o S t a n d a r d L P F o r m
At first glance, it may appear that the standard LP form is very restrictive. However, as shown below, it is possible to convert any LP problem to the above standard form.
Maximization Problem
As already mentioned in previous chapters, a maximization problem can be converted to a minimization problem simply by multiplying the objective function by a negative sign. For example,
Maximize z(x) = 3*i + 5x2 is the same as Minimize / (x) = —3*i — 5x2 Constant Term in the Objective Function
From the optimality conditions, it is easy to see that the optimum solution x* does not change if a constant is either added to or subtracted from the objective function. Thus, a constant in the objective function can simply be ignored. After the solution is obtained, the optimum value of the objective function is adjusted to account for this constant.
Alternatively, a new dummy optimization variable can be defined to multiply the constant and a constraint added to set the value of this variable to 1. For example, consider the following objective function of two variables:
Minimize / (x) = 3*i + 5*2 + 7
In standard LP form, it can be written as follows:
Minimize / (x) = 3x\ + 5x2 + 7*3 Subject to *3 = 1
Negative Values on the Right-Hand Sides of Constraints
The standard form requires that all constraints must be arranged such that the constant term, if any, is a positive quantity on the right-hand side. If a constant appears as negative on the right-hand side of a given constraint, multiply the constraint by a negative sign. Keep in mind that the direction of inequality changes (that < becomes >, and vice versa) when both sides are multiplied by a negative sign. For example,
3*] + 5*2 < —7 is the same as —3*i - 5*2 > 7
C h a p t e r 6 L i n e a r P r o g r a m m i n g
Less than Type Constraints
Add a new positive variable (called a slack variable) to convert a < constraint (LE) to an equality. For example, 3xi + 5*2 <7 is converted to 3χι + 5X2+X3 = 7, where X3 > 0 is a slack variable
Greater than Type Constraints
Subtract a new positive variable (called a surplus variable) to convert a > constraint (GE) to equality. For example, 3xi + 5x2 > 7 is converted to 3xi + 5x2 — X3 = 7, where X3 > 0 is a surplus variable. Note that, since the right-hand sides of the constraints are restricted to be positive, we cannot simply multiply both sides of the GE constraints by - 1 to convert them into the LE type, as was done for the KT conditions in Chapter 4.
Unrestricted Variables
The standard LP form restricts all variables to be positive. If an actual optimiza­
tion variable is unrestricted in sign, it can be converted to the standard form by defining it as the difference of two new positive variables. For example, if variable x\ is unrestricted in sign, it is replaced by two new variables y\ and y2 with Xi — yi — Y2- Both the new variables are positive. After the solution is obtained, if yi > 72» then xi will be positive and if yi < y2, then xi will be negative.
Example 6.1 Convert the following problem to the standard LP form. Maximize z — 3xi + 8x2
(
3 x i + 4 x 2 > - 2 0\
^ x H - ^ 3 x 2 ^ > 6 J
*1 > 0 /
Note that X2 is unrestricted in sign. Define new variables (all > 0)
*i = yi x i = y 2 ~ y 3
Subst i tuti ng t hes e and mul t i pl yi ng t he f i rst const rai nt by a negat i ve si gn, t he probl em i s as f ol l ows:
Maxi mi ze z = 3yi + 8 y 2 - 8y$
/—3 y i - 4 y 2 + 4 y a < 2 0 Subject to [ y j + 3 y 2 - 3 y 3 > 6
\ yi, Y2, y3 > 0
6.2 S o l v i n g L i n e a r S y s t e m s o f E q u a t i o n s
Multiplying the objective function by a negative sign and introducing slack/ surplus variables in the constraints, the problem in the standard LP form is as follows:
Minimize / = — 3yi - 8^2 + 873
Since linear functions are always convex, the LP problem is a convex program­
ming problem. This means that if an optimum solution exists, it is a global optimum.
The optimum solution of an LP problem always lies on the boundary of the feasible domain. We can easily prove this by contradiction. Suppose the solution lies inside the feasible domain; then the optimum is an unconstrained point, and hence, the necessary conditions for optimality would imply that df/dxi = q = 0, i = 1,2 which obviously is not possible (because all d — 0 means / as crx — 0). Thus, the solution cannot lie on the inside of the feasible domain for LP problems.
Once an LP problem is converted to its standard form, the constraints rep­
resent a system of n equations in m unknowns. If m — tt (i.e., the number of constraints is equal to the number of optimization variables), then the solution for all variables is obtained from the solution of constraint equations and there is no consideration of the objective function. This situation clearly does not represent an optimization problem. On the other hand, tn > n does not make sense because in this case, some of the constraints must be linearly dependent on the others. Thus, from an optimization point of view, the only meaningful case is when the number of constraints is smaller than the number of variables (after the problem has been expressed in the standard LP form).
6.2 Solving Linear Systems o f Equations
Subject to
6.1.2 T h e O p t i m u m o f L P P r o b l e m s
It should be clear from the previous section that solving LP problems involves solving a system of undetermined linear equations. (The number of equations
C h a p t e r f i L i n e a r P r o g r a m m i n g
is less than the number of unknowns.) A review of the Gauss-Jordan procedure for solving a system of linear equations is presented in this section.
Consider the solution of the following system of equations:
where A is an m χ n coefficient matrix, x is η χ 1 vector of unknowns, and b is an m χ 1 vector of known right-hand sides.
6.2.1 A S o l u t i o n U s i n g t h e G a u s s - J o r d a n F o r m
A solution of the system of equations can be obtained by first writing an (m χ n + 1) augmented matrix as follows:
Augmented matrix: (A | b)
A suitable series of row operations (adding appropriate multiples of rows to­
gether) is then performed on this matrix in an attempt to convert it to the following form:
where I is a ρ χ p identity matrix (p < m), c represents the remaining p x (n + 1 — p) entries in the augmented matrix, and the 0's are appropriate-sized zero matrices. This form is known as the Gauss-Jordan form. Since the variables corresponding to the identity matrix appear only in one of the equations, a general solution of the system of equations can be written directly in terms of these variables.
Ib get the Gauss-Jordan form, we first perform row operations to reduce the system to an upper triangular form in which all entries on the diagonal are 1 and those below the diagonal are 0. We do it systematically by starting in the first column, making the diagonal entry 1, and then using the appropriate multiples of the first row to make all other entries in the first column zero. Then we proceed to the second column and perform the same series of steps. The process is continued until we have reached the diagonal element of the last row. This completes what is known as the forward pass. Next, we start from the last column in which the forward pass ended and perform another series of row operations to zero out entries above the diagonal. This is known as the backward pass.
Ax — b
6.2 S o l v i n g L i n e a r S y s t e m s o f E q u a t i o n s
The following examples illustrate the process of converting a system of equations to an equivalent Gauss-Jordan form and recovering a general solu­
tion from this form.
Example 6.2 As an example, consider the following system of equations:
The augmented matrix foi
2*1 + 5*2 “ * 3 + *4 = -3*i — 8*2 -f· 2*3 + 3*4 = *! + 2*2 + 5*4 =
this system of equations
^ *1 *2 *3 *4 rhs'' 2 5 - 1 1 1 - 3 - 8 2 3 4 ^ 1 2 0 5 6 >
1
4
5
i s a s f o l l o w s:
I b g e t 1 o n t h e f i r s t r o w d i
A d d i n g ( 3 x r o w 1 ) t o r o w e n t r i e s i n t h e f i r s t c o l u m n
a g o n a l, d i v i d e t h e f i r s t r o / * i * 2 * 3 * 4 r h s\
1 5 1 1 1 1 2 2 2 2 -3 - 8 2 3 4
s, 1 2 0 5 6 J
2 and ( —1 x row 1) to row zero, giving
/*i *2 *3 *4 rhs\ ι 5 1 1 1
why 2.
3 will make the remaining
Tb make the diagonal entr Now we use this modified
1 2 2 2 2 n 1 1 9 11 u 2 2 2 2 Π 1 1 9 11 j \u 2 2 2 2/
y in the second column 1
'*1 *2 * 3 *4 rhsN ι 5 1 1 1 1 2 2 2 2 0 1 - 1 - 9 -11
n 1 1 9 11
2 2 2 2 >
row 2 t o zero out al l e nt
, di vi de row 2 by - 1/2. ri e s be l ow t he di agonal o f
t he s e c ond col umn. Addi ng 1/2 x row 2 t o t he t hi rd row wi l l make t he ent ry
C h a p t e r 6 L in p - a i· P m g r a m m i n g
below the diagonal of the second row zero.
Ui
X2
*3
X4
rhs\
1
5
2
1
2
1
2
1
2
0
1
-1
-9
-11
\o
0
0
0
0 /
Since the diagonal element of the third column is already zero, we have reached the end of the forward pass. In the backward pass, starting from the last column of the forward pass, we make the entries above the diagonal go to zero. Wp only have one entry in the second column that needs to be made zero. This is done by adding (—5/2 x row 2) to row 1, giving the complete Gauss-Jordan form for the problem as follows:
fx\
*2
*3
X4
r h s ^
1
0.
2
23
28
0
1
- 1
-9
- 1 1
lo
0
0
0
0 J
We see that we have a 2 x 2 identity matrix and a 2 x 3 matrix c. This means that only two of the three equations are independent, and we can Only solve for two unknowns in terms of the remaining two. The following general solution can readily be written from the above system of equations:
From the first equation: x1 +2x3 + 23*4 = 28, giving x\ = 28 - 2*3 — 23*4
From the second equation: X2 — X3 — 9x4 = —11, giving x2 = —11 + X3 + 9x4
Row Exchanges
A key step in converting a matrix to its Gauss-Jordan form is division by the diagonal elements. Obviously, if a zero diagonal element is encountered during elimination, this step cannot be carried out. In these cases, it maybe necessary to perform row exchanges to bring another row with a nonzero element at the diagonal before proceeding. Since each row represents an equation, a row exchange simply means reordering of equations and hence does not affect the solution. The following example involves a situation requiring a row exchange.
Example 6.3 Find the solution of the following system of equations:
X i + 2x 2 + 3x3 + 4 x 4 = 5 Xl + 2X2 +
4X3 - 9x 4 — 9 —x i — xi
+ X 3 -f- X 4 s= 6
6.2
The augmented matrix for this system of equations is as follows:
ί xi X 2
1 2
1 2
- 1 - 1
X3
3
4 1
*4
4
- 9
1
rhs\ 5 9 6/
The conversi on to t he Gauss-Jordan form proceeds as f ol l ows. ************ Forward pass **********
JJL
2__3 4
(Row 2) - (1) * Row 1 -»
(Row 3) - ( - 1) * Row 1 -»
0 0
- 1 - 1
1
1
-13
1
t l
0
2 3 0 1 1 4
4
-13
5
4
6
5 4 11
We cannot proceed with the second column because of a zero on the diagonal. However, if we interchange rows 2 and 3 (which physically means writing equation 3 before equation 2), we have the following situation:
Rows 2 and 3 interchanged-
Ί
0
2
1
0
3
4 1
4
5
-13
5
11
4
Since the diagonal entry in the second column is already 1 and those below it are 0, we are done with the second column. Next, we proceed to the third column. The diagonal is again already 1, and therefore, we are done with the forward pass.
**********BaCkward pass**********
1
2
0
43
-T
(Row 1)
- (3) * Row 3 -»
0
1
4
5
11
so
0
1
-13
4 (
fl
2
0
43
-7’
(Row 2)
- (4) * Row 3 -»
0
1
0
57
-5
,0
0
1
-13
4 ,
rl
0
0
- 71
3 '
(Row 1)
- ( 2) * Row 2 -»
0
1
0
57
- 5
,0
0
1
- 13
4,
We h a v e a Ga us s - J o r da n f o r m wi t h a 3 x 3 i d e n t i t y ma t r i x a nd a 3 x 2 ma t r i x c. The g e n e r a l s o l u t i o n f r o m e a c h e q u a t i o n c a n b e wr i t t e n a s f o l l o ws. Re me mb e r t he ί * c o l u mn r e p r e s e n t s t h e c o e f f i c i e n t o f t h e I t h v a r i a bl e, a n d t h e l a s t c o l u mn i s t h e r i g ht - ha nd s i d e o f t h e e q u a t i o n s.
X l — 3 + 7 1 * 4
* 2 = — 5 — 5 7 * 4
* 3 = 4 + 1 3 * 4
C h a p t e r 6 L i n e a r P r o g r a m m i n g
The Gaus&JondanForm Function
Given an augmented matrix, the following Mathematica function converts the matrix to a Gauss-Jordan form. The function also considers row exchanges if a zero is encountered on the diagonal. There are built-in Mathematica functions that do the job more efficiently. The main reason for writing the following function is to document the intermediate steps. The function reports details of all operations by printing the augmented matrix after each operation.
Needs["QptimizationToolbox'LPSimplex'”];
?G a u s s J o r d a n F o r m
Gau s sJ ordanF orm[mat] converts the given matrix into a Gauss-Jordan
form, printing results at all intermediate steps. The function perforins row exchanges, if necessary.
Example 6.4 Find the solution of the following system of equations:
3*i + %2 + *3 =8 2xi - X2 — *3 = - 3 x\ 4- 2x2 —Χ3 — 2
The augmented matrix for this system of equations is as follows:
X2 X3 rhs\
1 1 8
-1 -1 ~3 2 - 1 2 /
The conversion to the Gauss-Jordan form proceeds as follows:
(3 1 1 B\
; G a u s s J o r d a n F o r m [ a ];
- 1 - 1 - 3
2 - 1 2
v «
**********Forward pass**********
/-ι 1 1 8 '
1 Ί Ί Ί
-1 -1 -3
2
(Row l)/3
(Row 2) - (2) * Row 1 -»
(Row 3) - (1) * Row 1-»
0
1
I i
o
l o
2
1
1
8
Ί
ϊ
Ί
5
5
25
~ Ί
"Τ
2
-1
2
1
1
8
Ί
J
Ί
5
5
25
~ Ί
"Τ
5
4
2
Τ
"Τ
6.2 S o l v i n g L i n e a r S y s t e m s o f E q u a t i o n s
1
0
0
1
y
5
y
(Row 3) - { —) * Row 2 -»
1
y
4
y
ii
o
! |
5
-§J
1 y
1
y
1
KO cr
/1 1 1 8\
f l y y y
1 5
-3 -9;
( Ro w 3) /- 3 ·**
0 1 1 5
.0 0 1 3/
**********Backward pass**********
0
il -J
(Row 1) - ( —) * Row 3 -»
5
y
0 1 lo 0
1 5 1 3;
(Row 2) - (1) * Row 3
1 1 n 5
1 y 0 y
n 1 n 2
0 0 1 3
Ί 0 0 Ι ­
(Row 1) - ( - ) * Row 2 -*
Ο 1 0 2
,0 0 1 3,
Solution
Xl = l
*2 = 2 *3=3
Inconsi st ent Systems of Equati ons
For some s ys t ems o f equat i ons, us i ng t he previ ous seri es o f st eps, t he Gauss- Jordan f orm comes out as f ol l ows:
era
The l ast equat i on says t hat a nonzero const ant d is equal to 0, which obviously is nonsense. Such a system of equations is inconsistent and has no solution.
Example 6.5 Consider the same system of equations as the previous exam- ple, except for the right-hand side of the third equation:
C h a p t e r β L i n e a r P r o g r a m m i n g
2x\ + 5*2 — X3 +X4 = 1 —3xi — 8x2 + 2x3 + 3x4 = 4 X l + 2 x 2 + 5x 4 = 7
The augmented matrix for this system of equations is as follows:
Xl
*2
*3
X4
rhs''
2
5
-1
1
1
- 3
- 8
2
3
4
.1
2
0
5
7 )
a =
2 5
-3 -8
1 2
-1
2
0
1
3
5
GaussJordanForm [a];
******** * *po3rw&3rd p&ss* *********
1
(Row l)/2-» -3 -8 2 3 4
1 1 1\
1
(Row 2) - (-3) * Row 1 -»
(Row 3) - (1) * Row 1
(1 0
0 -
5
1
1
5
Ί
1
1.
1
Ί
1
~ Ί
1
1
0
1
1
J
1
2
1
?
9
1
9
1
1
1
11
T
7
1
?
11
~T
13
"2”
( Row 2) / -
- 1
1
- 9
1
- 1 1
13
1
f l
(Row 3) - (-—) * Row 2 ->
A
1 1 Ί 1
0 1 -1 -£
,0 0 0 0
fl 0 2 23
1
2
-11
1
(Row 1) - ( —) * Row 2
0 1 - 1 L0 0 0
28
-9 -11
0 1
The first two equations are fine, but the third equation is nonsense because it says that
Ο χ ι + 0X 2 + OX3 + OX4 = 1
6.2 S o l v i n g L i n e a r S y s t e m s o f E q u a t i o n s
Thus, the Gauss-Jordan form is inconsistent. The system of equations has no solution.
Matrix Inversion Using the Gauss-Jordan Form
It jR qlsn possible to find the inverse of a square matrix using fhp, Gflnss-Tnrrifln form. For this purpose, the augmented matrix is defined as follows:
Initial augmented matrix: (A | I)
where A is the matrix whose inverse is to be found, and I is an identity matrix of the same size. After carrying out the operations to convert the matrix to the Gauss-Jordan form, the final matrix is as follows:
Final Gauss-Jordan form: (I | A-1)
Example 6.6 As an example, consider finding an inverse of the matrix
A =
For finding the inverse, we define the augmented matrix as follows:
& =
13 1 1 1 0 o’i
2 - 1 - 1 0 1 0 1 2 - 1 0 0 1
It s Gaus s - Jordan f or m i s c omput e d as f ol l ows:
Gau s BJordanForm [a1;
**********Forward pass**********
( R o w 1 ) /3 ->
1
i
1 1 Ί Ί
-1
-1
( R o w 2 ) - ( 2 ) * R o w 1 -»
( R o w 3 ) - ( 1 ) * R o w 1 -»
1
0
1
il
0
0
0 1
0 0
1
Ί
5
"I
1
Ί
5
- Ί
0
0
lj
1
Ί
5
“J
- 1
1
i
5
- J
1
Ί
2
~1
0
1
1
2 ~1
0
1
0
0
1
0’
0
1,
o'
C h a p t e r 6 L i n e a r P r o g r a m m i n g
(Row 2) / - —
-»
ο !
i1 i
0 1
1
I
1
1
2
1
~ Ί
0\
0
(Row 3)
(-)
* Row 2 -»■
il
0
lo
1
I
1
0
(Row 3) /- 3 -»
1
0
1
Ί
1
1
2
5
1
i
1
~3
1
1
2
-1
0 ‘ 0
0
li
1
I'
1
I
[0 0 1 * * * * * * * * * * £ ^ ^ ^ ( 5 pass* * ** * *****
fi i 0 £
1
(Row 1) - ( — )
* Row 3 -»
1
Ϊ
3
"5
1
Ϊ
(Row 2) - (1) * Row 3-»
υ ' Ί ~1 ~Ί>
1 τ 0 f ί i 0 1 0 ά Ά i 0 0 1 5 -J.
i'i 0 0 1 i 0 |
1
(Row 1) - ( — ) *
ROW 2-»
0 h 0 ts ~A i Μ ι ί -J -i)
Thus, t he i nv e r s e o f t he gi v e n mat ri x i s
1
- - -1
5
5
1
4
15
15
1
1
3
3
A - 1 =
6.2.2 S o l u t i o n U s i n g L U D e c o m p o s i t i o n f o r S q u a r e M a t r i c e s
Cons i de r a s quare s y s t e m o f e quat i ons ( m e quat i ons i n m unknowns )
Ax — b
whe r e A i s an m x m mat ri x. We can obvi ous l y s ol ve t hi s s y s t e m us i ng t he Gaus s - Jordan f orm. However, i t i s mor e e f f i c i e nt t o f i rs t de c o mpo s e t he mat ri x
6.2 S o l v i n g L i n e a r S y s t e m s o f E q u a t i o n s
A into product of a lower triangular matrix L and an upper triangular matrix U. The matrix L has 1 's on the diagonal and all of its entries above the diagonal are 0. All entries below the diagonal of the matrix U are 0.
1
0
0
... 0\
( U n
U\2
U\3 ...
t/l m\
L 2 1
1
0
... 0
0
U 22
U 2 3 · ·'
U 2 m
L 3 1
L 3 2
1
... 0
u =
0
0
U 3 3 ...
U 3 m
•
•
■·. 0
•
•
j
\L m l
L m 2
L m 3
... 1/
\ U
0
0
U m m/
Such a decomposi t i on i s al ways possi bl e for nonsi ngul ar square matri ces. As­
sumi ng t hi s decomposi t i on i s known, we can sol ve t he s ys t em o f equat i ons i n t wo st eps as f ol l ows:
Ll Jx = b
If we def i ne Ux = y, t hen
Ly = b
or
( J. 0
L 2 1 1
L 31 L32
0
0
1
V^ml L m2 Lm3
...
0\
( y i )
... 0
y2
b2
... 0
ys
—
h
■·. 0
... 1/
\ymj
\bm)
Since L is a lower triangular matrix, the first row gives the solution for the first variable as y\ — b\. Knowing this, we can solve for y2 = &2 ~ £2171 from the second equation. Thus, proceeding forward from the first equation, we can solve for all intermediate variables y,·. This is known as forward elimination.
Now we can solve for X{ as follows:
< l 7 n U\2 U 1 3 .. . U l m ^
/x i >
( y i >
0 U 2 2 U2 3 . . . U 2 m
X 2
y 2
Ux = y or
0 0 U 3 3 ... u 3 m
X3
—
y s
K 0 0 0 ... U m m )
\x m )
U/
In this system, the last equation has only one unknown, giving Xm = ymf Umm. The second to last equation is used next to get Xm- ι. Thus, working backward
from the last equation, we can solve for all unknowns. This is known as backward substitution.
The procedure is especially efficient if solutions for several different right- hand sides are needed. Since in this case, the major computational task of decomposing the matrix into the LU form is not necessary for every right-hand side (rhs), we simply need to perform a forward elimination and a backward substitution for each different rhs.
An Algorithm for LU Decomposition
Given a nonsingular square matrix A, the matrices L and U are determined by applying a series of steps similar to those used for generating the Gauss- Jordan form. The main difference is that each elimination step is expressed as a matrix multiplication operation. At the end, the product of these elimination matrices gives the required decomposition.
For the following discussion, entries in matrix A are denoted by a,j·, i, j =
1,2 Assuming flu φ 0 (called pivot), it is easy to see that if we multiply
matrix A by the following matrix M(1*, all entries below the diagonal in the first column will become zero.
/ 1
0
0
... 0\
—m2i
1
0
... 0
M(1) =
-m3i
0
1
... 0
0
0
... l j
whe r e = f l j t i /a n k = 2, 3,..., m. The new matrix will have the following form:
The superscript (2) is used to indicate changed entries in A as a result of the matrix multiplication. We now define a matrix Mi2) to make entries below the diagonal in the second column to go to zero (again assuming a nonzero pivot, i.e., a.^2 7^,0)·
6.2 S o l v i n g L i n e a r S y s t e m s o f E q u a t i o n s
M
(2) _
/I 0 0
0 1 0
0 - f f l 3 2 1
o\
0
0
\0 - r n m2 0
V
where m\a — /a ^ k = 3,...,m. The new matrix A(3) will have the follow­
i n g form:
/V3>
11
0
0
a
(3)
12
i (3)
*22
0
0
t(3)
13
“ l m
.0 )
Λ 3 )
2 3 ·' ·
2 m
,( 3)
34
3 m
( 3)
J 3 )
lm3
u m m /
Repeating these steps for all columns in the matrix, we get an upper triangular matrix. Thus,
U = M(m_1)M(m_2)... M(1)A = MA The lower triangular matrix can be written as follows:
LU = A or U - L_1 A Since U = MA, we see that
L-1 = M
Because of the special form of M matrices, it can easily be verified that the product of their inverses, and hence L, is as follows:
/ 1
0
0
0\
"121
1
0
0
L =
"*31
m 2
1
• *
0
Wml
mm2
mm3 ·..
V
When generating M matrices, it was assumed that the diagonal entries are nonzero. Sometimes the diagonal entries do become zero. If the overall system of equations is nonsingular, it is always possible to get a nonzero pivot by reordering the equations, as was done while generating the Gauss-Jordan form.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
The following Mathematica implementation is designed to illustrate these ideas. The function keeps track of row exchanges to avoid zero pivots. Math­
ematica has built-in functions for performing LUDecomposition and LUBack-
Substitution. These functions should be used in actual implementations.
Needs ["OptimizationToolbox'LPSiiBplex'"] ;
?LUDecompositionSteps
JjUDecompositionSteps[A] decomposes matrix A into LU form, printing all
intermediate steps. Equations are reordered, if necessary.
Example 6.7 Determine LU factors for the following matrix:
A =
I 2 1 1
4 1 0
- 2 2 1
Ι-Γ
f 1 o 01 - 2 1 0 1 0 1^
f l 0 0
0 1 0 ^0 3 1
A3'-»
'2 1 1 ’
0 - 1 - 2
.0 3 2,
2 1 1 ’
0 - 1 - 2
{0 0 -4,
1
Product of all M matrices -»
-2
-5
0'
0
1,
1
0
°)
/
(2
1
1
[i’i
L->
2
1
Oj
U-*
0
-1
-2
Equation order -*
2
l - i
-3
l)
lo
0
-4j
U J
Example 6.8 Find the solution of the following system of equations using LU decomposition:
*1 -I- 2x2 + 3*3 + 4*4 = 5 *1 -I- 2*2 + 4*3 — 9*4 — 9 —2*1 — *2 + *3 + 2*4 = 6 —*1 — *2 + *3 + *4 = 6
I n m a t r i x f o r m:
6.2 S o l v i n g L i n e a r S y s t e m s o f E q u a t i o n s
We first generate the LU decomposition of the A matrix as follows. Note that a zero pivot is encountered after the first step. The equations are reordered and the process is continued until complete L and U matrices are obtained.
{L, U, p} = LUDecompositionSteps [A];
' 1
0
0
0
Ί
2
3
4
-1
1
0
0
jk2 v
0
0
1
-13
2
0
1
0
Α -τ
0
3
7
10
, 1
0
ό
1,
.0
1
4
5 .
M1 -»
***Re-ordered equations***
<
1 2 3 4
1
-2-11 2
3
A ->
12 4-9
Equation order -»
2
,-1 - 1 1 ι ,
.4,
M1 -»
M
1 0 0
2 1 0 - 1 0 1
1 0 0
M
1
0
0
0
1
0
0
10
0
1
0
I
0
1
0
0
0
0
1
0
0
0
1
5
■ J
0Ϊ
0
0
1
o\
0
0
l j
0)
0
0
1
A2 -»
(1 2 3 4
0 3 7 10
0 0 1 -13
[0 1 4 5
i l
0
0
0
i l
0
0
0
Product of a l l M matrices -»
2
3
0
0
2
3
0
0
' 1 2
3
7
1
3
7
1
0
4 10 -13
5 Ί
4
10
-13
¥ 1
0
1
0
1
"5
0
0
0
0
1
0
-!
1,
Ii ■*>
’ 1
0
0
0'
Ί
2
3
4
T
-2
1
O'
0
0
3
7
10
3
1
0
1
0
u-»
0
0
1
-13
Equation order -»
2
,-1
1
Ί
5
I
1
,0
0
0
ΨΙ
[4,
The next step is the solution for intermediate variables (forward elimina­
tion). Note the order of the right-hand side is changed to make it consistent with the order of the equations.
1 0 0 0\
(y{\
(5)
-2100
Ϊ2
6
10 10
73
9
-1
3 I v
\yj
w
C h a p t e r 6 L i n e a r P r o g r a m m i n g
yi - 5
y2 = 6 4- 2yi = 16 y3 = 9 - y\ - 4
1 5
JK4 = 6 +yi - - y 3 - -y2 =
- 1
The f o l l owi ng i s t he s ol ut i on f or t he ac t ual vari abl es ( backward s ubs t i t ut i on):
{ 1 2 3 0 3 7 0 0 1 VO 0 0
- _ J L
X4 ---
T
4 \ 1 0 -13
ZQ
3
/
3_
70
6c i ^
( 5 )
*2
= 16
*3
4
w
V - i J
*3—4 4 - 1 3 * 4 =
241
*2
70
16 - 7x3 — IOX4
179
70
Xl = 5 — 2X2 — 3x3 — 4X4 = —
70
The same solution can directly be obtained by using the built-in function, called LinearSolve.
LinearSolve [λ, b]
179 241
"70"' TT
{'To-
- t i
6.3 B a s i c S o l u t i o n s o f a n LP P r o b l e m
As mentioned before, the solution of an LP problem reduces to solving a system of underdetermined (fewer equations than variables) linear equations. From m equations, at most we can solve for m variables in terms of the remaining n — m variables. The variables that we choose to solve for are called basic, and the remaining variables are called nonbasic.
Consider the following example:
Minimize / = —xj -1- x2
Subject to
( x\ — 2x2 > 2 N
Xl + X2 < 4 x i < 3 \xi > o, i = 1, 2)
In the standard LP form
Minimize / = —x\ + *2
x i - 2x2 - *3 = 2 ^ x i + * 2 + * 4 = 4
Subject to
xi + *5 = 3
^Xi > 0, ί = 1,..., 5/
where *3 i s a surpl us vari abl e for t he fi rst const rai nt, and X4 and X5 are sl ack vari abl es for t he t wo l ess - t han t ype const rai nt s. The t otal number o f vari abl es i s n = 5, and t he number o f equat i ons i s m = 3. Thus, we can have t hree basi c vari abl es and t wo nonbasi c vari abl es. If we arbi trari l y choose X3, X4, and X5 as basi c vari abl es, a general sol ut i on o f t he const rai nt equat i ons can readi l y be wri t ten as f ol l ows:
X3 = — 2 + xi — 2x2 X4 — 4 — xi — x2 X5 = 3 — xi
The general solution is valid for any values of the nonbasic variables. Since all variables are positive and we are interested in minimizing the objective function, we assign 0 values to nonbasic variables. A solution from the con­
straint equations obtained by setting nonbasic variables to zero is called a basic solution. Therefore, one possible basic solution for the above example is as follows:
X3 = —2 X4 = 4 X5 = 3
Since all variables must be > 0, this basic solution is infeasible because X 3 is negative.
Let’s find another basic solution by choosing (again arbitrarily) xi, * 4, and X5 as basic variables and x 2 and X3 as nonbasic. By setting nonbasic variables to zero, we need to solve for the basic variables from the following equations:
xi — 2 xi + X4 = 4 xi + X5 = 3
It can easily be verified that the solution is xi = 2, x 4 = 2, and X5 = 1. Since all variables have positive values, this basic solution is feasible as well.
The maximum number of possible basic solutions depends on the number of constraints and the number of variables in the problem, and can be determined
C h a p t e r 6 L i n e a r P r o g r a m m i n g
from the following equation:
Number of possible basic solutions = Binomial[n, m] =
nl
ml
(n—m)!
where stands for factorial. For the example problem where m = 3 and n = 5
therefore, the maximum number of basic solutions is
5!
3! 2!
5 x 4 x 3! 3! x 2
= 10
All these basic solutions are computed from the constraint
and are
summarized in the following table. The set of basic variables for a particular solution is called a basis for that solution.
(1)
(2)
(3)
(4)
(5) 4 6 X
Bas i s
{ X I.X2.X3 }
{X1,X2»X4}
{X1,X2,X5}
{Xl,X3,X4}
{X1,X3,X5}
{Χ1,Χ4,Λ5}
Solution
{3,1, -1,0, 0}
{3, 2’ 0, j, θ}
( 1 2 2 0 0 _ 1 1 I 3 ’ 3* ’ ’ 31
{3, 0,1,1,0} {4, 0, 2, 0,-1} { 2, 0, 0,2,1 }
S t a t u e
I n f e a s i b l e
F e a s i b l e
I n f e a s i b l e
F e a s i b l e
I n f e a s i b l e
F e a s i b l e
5
'2
- 3
- 2
( 7 ) {x2.X3.x4 }
(8) {x 2.x 3.x 5 }
( 9 ) {X2.x4.x5}
( 1 0 ) { x3,x4,x5}
{- }
{0, 4, - 1 0, 0, 3} {0,- l,0, 5, 3} {0, 0, -2, 4, 3}
NoSol ut i oh
Infeasi bl e
Inf easi bl e
Infeasi bl e
The case ( 7) produces an i ncons i s t ent s ys t em of equat i ons and hence, t here i s no sol ut i on. There are onl y t hree cases t hat gi ve a basi c f easi bl e sol ut i on. The obj ect i ve f unct i on val ues are comput ed for t he s e sol ut i ons. Si nce t he ori gi nal probl em was a two- vari abl e probl em, we can obt ai n a graphi cal sol ut i on to gai n f urther i nsi ght i nt o t he basi c f easi bl e sol ut i ons. As s een from t he graph i n Fi gure 6.1, t he t hree basi c f easi bl e sol ut i ons correspond t o t he t hree vert i ces o f t he f easi bl e regi on. The i nf easi bl e basi c sol ut i ons correspond to const rai nt i nt ersect i ons t hat are out si de of t he f easi bl e regi on.
A Brute-force Method for Sol vi ng an LP Probl em
As i l l ust rat ed i n t he previ ous sect i on, t he vert i ces of t he f easi bl e regi on of an LP probl em correspond to basi c f easi bl e sol ut i ons. Furthermore, s i nce the sol ut i on of an LP probl em must l i e on t he boundary o f t he f easi bl e domai n,
6.3 B a s i c S o l u t i o n s o f a n I f P r o b l e m
*2
o n e o f t h e s e b a s i c f e a s i b l e s o l u t i o n s m u s t b e t h e o p t i m u m. T h u s, a brute-force m e t h o d t o f i n d a n o p t i m u m i s to c o m p u t e a l l p o s s i b l e b a s i c s o l u t i o n s. T h e o n e t h a t i s f e a s i b l e a n d h a s t h e l o w e s t v a l u e o f t h e o b j e c t i v e f u n c t i o n i s t h e o p t i m u m s o l u t i o n.
For t h e e x a m p l e p r o b l e m, t h e f o u r t h b a s i c s o l u t i o n i s f e a s i b l e a n d h a s t h e l o w e s t v a l u e o f t h e o b j e c t i v e f u n c t i o n. T h u s, t h i s r e p r e s e n t s t h e o p t i m u m s o l u t i o n for t h e p r o b l e m.
O p t i m u m s o l u t i o n:
x* = 3 x* = 0 x* = l x* = 1 x* = 0 /* = - 3
Mathematica F u n c t i o n t o C o m p u t e A l l B a s i c S o l u t i o n s
C o m p u t a t i o n o f a l l b a s i c s o l u t i o n s i s q u i t e t e d i o u s. T h e f o l l o w i n g Mathematica f u n c t i o n i s d e v e l o p e d to c o m p u t e all p o s s i b l e b a s i c s o l u t i o n s for a n LP p r o b l e m w r i t t e n i n t h e stan d ard form. T h e n u m b e r o f p o s s i b l e c o m b i n a t i o n s i n c r e a s e s v e r y r a p i d l y a s t h e n u m b e r o f v a r i a b l e s a n d t h e c o n s t r a i n t s i n c r e a s e. T he r e f o r e, t h i s f u n c t i o n, b y d e f a u l t, c o m p u t e s a m a x i m u m o f 20 s o l u t i o n s. More s o l u t i o n s c a n b e o b t a i n e d b y u s i n g a l a r g e r n u m b e r w h e n c a l l i n g t h e f u n c t i o n.
r h f l p t H1 β T.fa fta r P m g r a T n m in g
Heeds ["OptimizationToolbox'LPSixnplex'"] ;
TBasicSolutions
B a s i c S o l u t i o n s [ f,e q n s,v a r s,m a x S o l u t i o n s:2 0 ] d e te r m in e s b a s i c s o l u t i o n s
f o r an LP p ro b lem w r i t t e n i n s ta n d a r d form, f i s t h e o b j e c t i v e f u n c t i o n s, eq n s = c o n s t r a i n t e q u a t i o n s, v a r s = l i s t o f v a r i a b l e s. m a x S o lu tio n s = maximum number o f s o l u t i o n s com puted ( d e f a u l t < 2 0 ).
E x a m p l e 6.9 Compute all basic solutions o f the following LP problem:
Minimize / = xi + x 2
(
2xi + *2 £ 8 \
3*i + 2x2 < 10 |
Xi > 0, i = 1,2/
In the standard LP form
Minimize / = xi + x2
/ 2x i + x 2 + X 3 = 8 \
Subject to I 3 x i + 2 x 2 + X 4 = 1 0 I V Xi > 0, i — 1
.............4 /
*2
FIGURE 6.2 A graph showing all basic feasible solutions for the example.
6.4 T h e S i m p l e x M e t h o d
f = *1 + *2 t
e q n s = { 2 x x + x2 + x 3 == 8, 3xx + 2 xz + xi == 1 0 }; B a s i c S o l u t i o n s [ f, e q n s, (xt, x2, x 3 , x 4 } ] ,·
B a s i s
S o l u t i o n
S t a t u s
f 1
( χ ι, X2} { 6, - 4, 0, 0 }
I n f e a s i b l e
—
{ * 1' x 3
O
0
sh
F e a s i b l e
10
* 1, *4
{ 4,0,0,- 2 }
I n f e a s i b l e
—
x 2' x 3,
{ 0,5,3,0 }
F e a s i b l e
5
* 2' x 4,
{ 0,8, 0,- 6}
I n f e a s i b l e
—
.
x 3 / x 4,
{ 0,0,8,10}
F e a s i b l e
0 j
T h e g r a p h o f t h e o r i g i n a l p r o b l e m s h o w n i n Figure 6.2 a g a i n c o n f i r m s th a t t h e t h r e e b a s i c f e a s i b l e s o l u t i o n s c o r r e s p o n d t o t h e t h r e e v e r t i c e s o f t h e f e a s i b l e re gion.
6.4 T h e S i m p l e x M e t h o d
T h e o p t i m u m s o l u t i o n o f a n LP p r o b l e m c o r r e s p o n d s to o n e o f t h e b a s i c f e a s i ­
b l e s o l u t i o n s a n d t h u s c a n b e f o u n d b y e x a m i n i n g a l l b a s i c s o l u t i o n s. H o w ev e r, t h e n u m b e r o f p o s s i b l e b a s i c s o l u t i o n s c a n b e v e r y l a r g e for p r a c t i c a l prob­
l e m s. T h u s, t h e g o a l i s to d e v e l o p a p r o c e d u r e t h a t q u i c k l y f i n d s t h e b a s i c f e a s i b l e s o l u t i o n w i t h t h e l o w e s t o b j e c t i v e f u n c t i o n v a l u e w i t h o u t e x a m i n i n g all p o s s i b i l i t i e s.
6.4.1 B a s i c I d e a
T h e b a s i c i d e a o f t h e s i m p l e x m e t h o d i s t o star t w i t h a b a s i c f e a s i b l e s o l u t i o n an d t h e n t r y t o o b t a i n a n e i g h b o r i n g b a s i c f e a s i b l e s o l u t i o n t h a t h a s t h e ob- j e c t i v e f u n c t i o n w i t h a v a l u e l o w e r t h a n t h e c u r r e n t b a s i c f e a s i b l e s o l u t i o n. With e a c h try, o n e o f t h e c u r r e n t b a s i c v a r i a b l e s i s m a d e n o n b a s i c a n d i s r e ­
p l a c e d w i t h a v a r i a b l e f r o m t h e n o n b a s i c s e t. A n o p t i m u m i s r e a c h e d w h e n n o o t h e r b a s i c f e a s i b l e s o l u t i o n c a n b e f o u n d w i t h a l o w e r o b j e c t i v e f u n c t i o n v a l ue. R u l e s a r e e s t a b l i s h e d s u c h t h a t for m o s t p r o b l e m s, t h e m e t h o d f i n d s a n o p t i m u m i n a l o t f e w e r s t e p s t h a n t h e to t a l n u m b e r o f p o s s i b l e b a s i c s o l u t i o n s.
T h e c o m p l e t e a l g o r i t h m n e e d s p r o c e d u r e s for ( a ) f i n d i n g a s t a r t i n g b a s i c f e a s i b l e s o l u t i o n, (b) b r i n g i n g a c u r r e n t l y n o n b a s i c v a r i a b l e i n t o t h e b a s i c s et, a n d ( c ) m o v i n g a c u r r e n t l y b a s i c v a r i a b l e o u t o f t h e b a s i c s e t to m a k e r o o m for t h e n e w b a s i c variable.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
6.4.2 U s i n g t h e S i m p l e x M e t h o d f o r P r o b l e m s w i t h L E C o n s t r a i n t s
P r o b l e m s t h a t i n i t i a l l y h a v e a l l l e s s t h a n t y p e ( L E, < ) c o n s t r a i n t s a r e e a s i e s t t o d e a l w i t h i n t h e s i m p l e x m e t h o d. T h e p r o c e d u r e w i l l b e e x p l a i n e d w i t h r e f e r e n c e t o t h e f o l l o w i n g e x a m p l e:
M i n i m i z e f = 5x\ — 3x2 — 8x 3
/ 2x\ + 5*2 ~ *3 5 1
S u b j e c t t o
—2 x i — 1 2 x 2 + 3x3 < 9 —3 x j — 8 x 2 + 2 x 3 < 4 Xi > 0, i = 1,..., 3
S t a r t i n g B a s i c F e a s i b l e S o l u t i o n
T h e s t a r t i n g b a s i c f e a s i b l e s o l u t i o n i s e a s y t o o b t a i n f o r p r o b l e m s t h a t i n v o l v e o n l y L E t y p e c o n s t r a i n t s ( a f t e r m a k i n g t h e r i g h t - h a n d s i d e p o s i t i v e, i f n e c e s ­
s a r y ). A d i f f e r e n t s l a c k v a r i a b l e m u s t b e a d d e d t o e a c h L E c o n s t r a i n t t o c o n v e r t i t i n t o e q u a l i t y. W e h a v e a s m a n y s l a c k v a r i a b l e s a s t h e n u m b e r o f c o n s t r a i n t e q u a t i o n s. I f w e t r e a t t h e s e s l a c k v a r i a b l e s a s b a s i c v a r i a b l e s a n d t h e a c t u a l v a r i a b l e s a s n o n b a s i c ( s e t t o 0), t h e n t h e r i g h t - h a n d s i d e o f e a c h c o n s t r a i n t r e p r e s e n t s t h e b a s i c s o l u t i o n. S i n c e t h e r i g h t - h a n d s i d e s m u s t a l l b e p o s i t i v e ( a r e q u i r e m e n t o f t h e s t a n d a r d L P f o r m ), t h i s b a s i c s o l u t i o n i s f e a s i b l e a s w e l l.
F o r t h e e x a m p l e p r o b l e m, i n t r o d u c i n g s l a c k v a r i a b l e s X4, x s, a n d x q, t h e
c o n s t r a i n t s a r e w r i t t e n i n t h e s t a n d a r d L P f o r m a s f o l l o w s.
2 x i + 5 X2 — X3 + * 4 = 1 - 2 x i - 1 2 x2 + 3 x3 + X5 = 9 —3 x i — 8 x 2 + 2 x 3 + Xg — 4
T r e a t i n g t h e s l a c k v a r i a b l e s a s b a s i c a n d t h e o t h e r s a s n o n b a s i c, t h e s t a r t i n g b a s i c f e a s i b l e s o l u t i o n i s
B a s i c: X4 = 1, = 9,Xe = 4
N o n b a s i c: x j = x2 = X3 = 0 /= 0
N o t e t h a t t h e o b j e c t i v e f u n c t i o n i s e x p r e s s e d i n t e r m s o f n o n b a s i c v a r i e t i e s. I n t h i s f i r s t s t e p, w e d i d n o t h a v e t o d o a n y t h i n g s p e c i a l t o a c h i e v e t h i s. H o w ­
6.4 T h e S i m p l e x M e t h o d
e v e r, i n s u b s e q u e n t s t e p s, i t i s n e c e s s a r y t o e x p l i c i t l y e l i m i n a t e b a s i c v a r i a b l e s f r o m t h e o b j e c t i v e f u n c t i o n. T h e f o l l o w i n g r u l e u s e d t o m a k e d e c i s i o n s r e ­
g a r d i n g w h i c h v a r i a b l e t o b r i n g i n t o t h e b a s i c s e t a s s u m e s t h a t t h e o b j e c t i v e f u n c t i o n i s w r i t t e n i n t e r m s o f n o n b a s i c v a r i a b l e s a l o n e.
B r i n g i n g a N e w V a r i a b l e I n t o t h e B a s i c S e t
I n o r d e r t o f i n d a n e w b a s i c f e a s i b l e s o l u t i o n, o n e o f t h e c u r r e n t l y n o n b a s i c v a r i a b l e s m u s t b e m a d e b a s i c. T h e b e s t c a n d i d a t e f o r t h i s p u r p o s e i s t h e v a r i - a b l e t h a t c a u s e s t h e l a r g e s t d e c r e a s e i n t h e o b j e c t i v e f u n c t i o n. T h e n o n b a s i c v a r i a b l e t h a t h a s t h e l a r g e s t n e g a t i v e c o e f f i c i e n t i n t h e o b j e c t i v e f u n c t i o n i s t h e b e s t c h o i c e. T h i s m a k e s s e n s e b e c a u s e t h e n o n b a s i c v a r i a b l e s a r e s e t t o z e r o a n d s o t h e c u r r e n t v a l u e o f t h e o b j e c t i v e f u n c t i o n i s n o t i n f l u e n c e d b y t h e m. B u t i f o n e o f t h e m i s m a d e b a s i c, i t w i l l h a v e a p o s i t i v e v a l u e a n d t h e r e f o r e i f i t s c o e f f i c i e n t i s a l a r g e n e g a t i v e n u m b e r, i t h a s t h e g r e a t e s t p o t e n t i a l o f c a u s i n g a l a r g e r e d u c t i o n i n t h e o b j e c t i v e f u n c t i o n.
C o n t i n u i n g w i t h t h e p r e v i o u s e x a m p l e, w e h a v e t h e f o l l o w i n g s i t u a t i o n:
B a s i c: x 4 = 1, x5 = 9, x e = 4 N o n b a s i c: x i = x2 = X3 = 0 / = 0 = 5 x i - 3 x2 - 8x 3
T h e l a r g e s t n e g a t i v e c o e f f i c i e n t i n t h e / e q u a t i o n i s t h a t o f X3. T h u s, o u r n e x t b a s i c f e a s i b l e s o l u t i o n s h o u l d u s e X3 a s o n e o f i t s b a s i c v a r i a b l e s.
M o v i n g a n E x i s t i n g B a s i c V a r i a b l e O u t o f t h e B a s i c S e t
S i n c e t h e n u m b e r o f b a s i c v a r i a b l e s i s f i x e d b y t h e n u m b e r o f c o n s t r a i n t e q u a ­
t i o n s, o n e o f t h e c u r r e n t l y b a s i c v a r i a b l e s m u s t b e r e m o v e d i n o r d e r t o m a k e r o o m f o r a n e w b a s i c v a r i a b l e. T h e d e c i s i o n t o r e m o v e a v a r i a b l e f r o m t h e b a s i c s e t i s b a s e d o n t h e n e e d t o k e e p t h e s o l u t i o n f e a s i b l e. A l i t t l e a l g e b r a s h o w s t h a t w e s h o u l d r e m o v e t h e b a s i c v a r i a b l e t h a t c o r r e s p o n d s t o t h e s m a l l e s t r a t i o o f t h e r i g h t - h a n d s i d e o f c o n s t r a i n t s a n d t h e c o e f f i c i e n t s o f t h e n e w v a r i a b l e t h a t i s b e i n g b r o u g h t i n t o t h e b a s i c s e t. F u r t h e r m o r e, i f t h e c o e f f i c i e n t i s n e g ­
a t i v e, t h e n t h e r e i s n o d a n g e r t h a t t h e a s s o c i a t e d b a s i c v a r i a b l e w i l l b e c o m e n e g a t i v e; t h u s, d u r i n g t h i s p r o c e s s, w e n e e d t o l o o k a t r a t i o s o f r i g h t - h a n d s i d e s a n d p o s i t i v e c o n s t r a i n t c o e f f i c i e n t s.
T b u n d e r s t a n d t h e r e a s o n i n g b e h i n d t h i s r u l e, l e t's c o n t i n u e w i t h t h e e x ­
a m p l e p r o b l e m. T h e c o n s t r a i n t s a n d t h e c o r r e s p o n d i n g b a s i c v a r i a b l e s a r e a s f o l l o w s:
C h a p t e r 6 L i n e a r P r o g r a m m i n g
C o n s t r a i n t 1: X4 b a s i c: 2x\ + 5x2 — X3 + X4 = 1
C o n s t r a i n t 2: *5 b a s i c: —2x\ — 1 2x2 + 3x 3 + *5 = 9 ■
C o n s t r a i n t 3: xq b a s i c: —3xi — 8 x2 + 2x 3 + xq = 4
T h e v a r i a b l e X3 i s t o b e b r o u g h t i n t o t h e b a s i c s e t w h i c h m e a n s t h a t i t w i l l h a v e a v a l u e g r e a t e r t h a n o r e q u a l t o z e r o i n t h e n e x t b a s i c f e a s i b l e s o l u t i o n. F o r c o n s t r a i n t 1, t h e c o e f f i c i e n t o f X3 i s n e g a t i v e a n d t h e r e f o r e, t h e s o l u t i o n f r o m t h i s c o n s t r a i n t w i l l r e m a i n p o s i t i v e a s a r e s u l t o f m a k i n g X3 b a s i c. I n c o n s t r a i n t s 2 a n d 3, t h e c o e f f i c i e n t s o f X3 a r e p o s i t i v e a n d t h e r e f o r e, t h e s e c o n s t r a i n t s c o u l d r e s u l t i n n e g a t i v e v a l u e s o f v a r i a b l e s. F r o m t h e c o n s t r a i n t 2, w e s e e t h a t t h e n e w b a s i c v a r i a b l e X3 m u s t h a v e a v a l u e l e s s t h a n o r e q u a l t o 9/3 = 3; o t h e r w i s e, t h i s c o n s t r a i n t w i l l g i v e a n e g a t i v e s o l u t i o n. S i m i l a r l y, t h e t h i r d c o n s t r a i n t s h o w s t h a t X3 m u s t h a v e a v a l u e l e s s t h a n o r e q u a l t o 4/2 = 2. T h e c o n s t r a i n t 3 i s m o r e c r i t i c a l a n d t h e r e f o r e, w e s h o u l d m a k e *3 a b a s i c v a r i a b l e f o r t h i s c o n s t r a i n t a n d h e n c e r e m o v e xq f r o m t h e b a s i c s e t.
T h e N e x t B a s i c F e a s i b l e S o l u t i o n
N o w w e a r e i n a p o s i t i o n t o c o m p u t e t h e n e x t b a s i c f e a s i b l e s o l u t i o n. W e n e e d t o s o l v e t h e s y s t e m o f c o n s t r a i n t e q u a t i o n s f o r t h e n e w s e t o f b a s i c v a r i a b l e s. A l s o, t h e o b j e c t i v e f u n c t i o n m u s t b e e x p r e s s e d i n t e r m s o f n e w n o n b a s i c v a r i a b l e s i n o r d e r t o c o n t i n u e w i t h t h e s u b s e q u e n t s t e p s o f t h e s i m p l e x m e t h o d.
F o r t h e e x a m p l e p r o b l e m, t h e c o n s t r a i n t s a r e c u r r e n t l y w r i t t e n a s f o l l o w s:
C o n s t r a i n t 1:
x4 b a s i c:
2 xi + 5x2 — X3 + X4 = 1
C o n s t r a i n t 2:
xs b a s i c:
—2 xi — 1 2 x2 + 3 x5 + * 5 = 9
C o n s t r a i n t 3:
Xq b a s i c:
—3x i - 8 X2 + 2Xs + X6 = 4
T h e n e w b a s i c v a r i a b l e s e t i s ( x 4, X5, x 3 ). W e c a n a c h i e v e t h i s b y e l i m i n a t i n g X3 f r o m t h e f i r s t a n d t h e s e c o n d c o n s t r a i n t s. W e d i v i d e t h e t h i r d c o n s t r a i n t b y 2 f i r s t t o m a k e t h e c o e f f i c i e n t o f X3 e q u a l t o 1.
C o n s t r a i n t 3: X3 b a s i c: - § x i - 4 x2 + X3 + 5X6 = 2
T h i s c o n s t r a i n t i s k n o w n a s t h e pivot row f o r c o m p u t i n g t h e n e w b a s i c f e a s i b l e s o l u t i o n, a n d i s u s e d t o e l i m i n a t e X3 f r o m t h e o t h e r c o n s t r a i n t s a n d t h e o b j e c ­
t i v e f u n c t i o n. V a r i a b l e X3 c a n b e e l i m i n a t e d f r o m c o n s t r a i n t 1 b y a d d i n g t h e p i v o t r o w t o t h e f i r s t c o n s t r a i n t.
C o n s t r a i n t 1: x4 b a s i c: ^ x i + x2 + X4 + ^*6 = 3
6.4 T h e S i m p l e x M e t h o d
F r o m t h e s e c o n d c o n s t r a i n t, v a r i a b l e *3 i s e l i m i n a t e d b y a d d i n g ( - 3 ) t i m e s t h e p i v o t r o w t o i t.
C o n s t r a i n t 2: *5 b a s i c: § * i + *5 - ^xe = 3
T h e o b j e c t i v e f u n c t i o n i s
5 * i —3 x2 - 8x3 = f T h e v a r i a b l e *3 i s e l i m i n a t e d b y a d d i n g e i g h t t i m e s t h e p i v o t r o w t o i t.
- 7 * i - 3 5 * 2 + 4 * 6 = / + 1 6 W e n o w h a v e a n e w b a s i c f e a s i b l e s o l u t i o n, a s f o l l o w s:
B a s i c: *3 = 2, *4 = 3, *5 = 3 N o n b a s i c: *1 = *2 = *6 = 0 / = —16
C o n s i d e r i n g t h e o b j e c t i v e f u n c t i o n v a l u e, t h i s s o l u t i o n i s b e t t e r t h a n o u r s t a r t ­
i n g s o l u t i o n.
T h e O p t i m u m S o l u t i o n
T h e s e r i e s o f s t e p s a r e r e p e a t e d u n t i l a l l c o e f f i c i e n t s i n t h e o b j e c t i v e f u n c t i o n b e c o m e p o s i t i v e. W h e n t h i s h a p p e n s, t h e n b r i n g i n g a n y o f t h e c u r r e n t n o n b a s i c v a r i a b l e s i n t o t h e b a s i c s e t w i l l i n c r e a s e t h e o b j e c t i v e f u n c t i o n v a l u e. T h i s i n d i c a t e s t h a t w e h a v e r e a c h e d t h e l o w e s t v a l u e p o s s i b l e a n d t h e c u r r e n t b a s i c f e a s i b l e s o l u t i o n r e p r e s e n t s t h e o p t i m u m s o l u t i o n.
T h e n e x t s t e p f o r t h e e x a m p l e p r o b l e m i s s u m m a r i z e d a s f o l l o w s:
Co n s t ra i n t 1: *4 b a s ic: \x\ + *2 + *4 + = 3
Co n s t ra i n t 2: *5 b a s ic: | * i + *5 - | * 6 = 3 C o n s t r a i n t 3: *3 b a s ic: — | * i — 4 * 2 + *3 + = 2
Objective: -7* i - 35*2 + 4 * 6 = / + 16
I n t h e o b j e c t i v e r o w, t h e v a r i a b l e *2 h a s t h e l a r g e s t n e g a t i v e c o e f f i c i e n t. T h u s, t h e n e x t v a r i a b l e t o b e m a d e b a s i c i s *2. I n t h e c o n s t r a i n t e x p r e s s i o n s, p o s i t i v e c o e f f i c i e n t o f *2 s h o w s u p o n l y i n t h e f i r s t c o n s t r a i n t, w h i c h h a s *4 a s t h e b a s i c v a r i a b l e. T h u s, *4 s h o u l d b e r e m o v e d f r o m t h e b a s i c s e t. T h e c o e f f i c i e n t o f *2 i n t h e f i r s t r o w i s a l r e a d y 1; t h u s, t h e r e i s n o t h i n g t h a t n e e d s t o b e d o n e t o t h i s
C h a p t e r 6 L i n e a r P r o g r a m m i n g
e q u a t i o n. F o r e l i m i n a t i n g X2 f r o m t h e r e m a i n i n g e q u a t i o n, n o w w e m u s t u s e t h e f i r s t e q u a t i o n a s t h e p i v o t r o w ( P R ) a s f o l l o w s.
C o n s t r a i n t 1: X2 b a s i c: +X2 +X4 + jXe = 3 ( P R )
C o n s t r a i n t 2: *5 b a s i c: jX\ + x5 - §*6 = 3 ( n o c h a n g e )
C o n s t r a i n t 3: *3 b a s i c: \x\ + χ$ + 4x4 + §*6 = 1 4 ( A d d e d 4 x P R )
O b j e c t i v e: ψχι + 35*4 + y x g = / + 1 2 1 ( A d d e d 3 5 x P R )
L o o k i n g a t t h e o b j e c t i v e f u n c t i o n r o w, w e s e e t h a t a l l c o e f f i c i e n t s a r e p o s i t i v e. T h i s m e a n s t h a t b r i n g i n g a n y o f t h e c u r r e n t n o n b a s i c v a r i a b l e s i n t o t h e b a s i c s e t w i l l i n c r e a s e t h e o b j e c t i v e f u n c t i o n. T h u s, w e h a v e r e a c h e d t h e l o w e s t v a l u e p o s s i b l e a n d h e n c e, t h e a b o v e b a s i c f e a s i b l e s o l u t i o n r e p r e s e n t s t h e o p t i m u m s o l u t i o n.
O p t i m u m s o l u t i o n:
B a s i c: x2 = 3, *3 = 1 4, *5 = 3 N o n b a s i c: xj = X4 = xe = 0 f + 121 = 0 g i v i n g /* = - 1 2 1
I t i s i n t e r e s t i n g t o n o t e t h a t t h e t o t a l n u m b e r o f p o s s i b l e b a s i c s o l u t i o n s f o r t h i s e x a m p l e w a s B i n o m i a l [6, 3 ] = 2 0. H o w e v e r, t h e s i m p l e x m e t h o d f o u n d t h e o p t i m u m i n o n l y t h r e e s t e p s.
6.4.3 S i m p l e x T a b l e a u
A t a b u l a r f o r m i s c o n v e n i e n t t o o r g a n i z e t h e c a l c u l a t i o n s i n v o l v e d i n t h e s i m ­
p l e x m e t h o d. T h e t a b l e a u i s e s s e n t i a l l y t h e a u g m e n t e d m a t r i x u s e d i n t h e G a u s s - J o r d a n f o r m f o r c o m p u t i n g t h e s o l u t i o n o f e q u a t i o n s. T h u s, t h e r o w s r e p r e s e n t c o e f f i c i e n t s o f t h e c o n s t r a i n t e q u a t i o n s. T h e o b j e c t i v e f u n c t i o n i s w r i t t e n i n t h e l a s t r o w. T h e f i r s t c o l u m n i n d i c a t e s t h e b a s i c v a r i a b l e a s s o c i ­
a t e d w i t h t h e c o n s t r a i n t i n t h a t r o w. R e c a l l t h a t t h i s v a r i a b l e s h o u l d a p p e a r o n l y i n o n e c o n s t r a i n t w i t h a c o e f f i c i e n t o f 1. T h e l a s t c o l u m n r e p r e s e n t s t h e r i g h t - h a n d s i d e s o f t h e e q u a t i o n s. T h e o t h e r c o l u m n s r e p r e s e n t c o e f f i c i e n t s o f v a r i a b l e s, u s u a l l y a r r a n g e d i n a s c e n d i n g o r d e r w i t h t h e a c t u a l v a r i a b l e s f i r s t, f o l l o w e d b y t h e s l a c k v a r i a b l e s. T h e e x a c t f o r m o f t h e s i m p l e x t a b l e a u i s i l l u s t r a t e d t h r o u g h t h e f o l l o w i n g e x a m p l e s.
E x a m p l e 6.1 0 C o n s i d e r t h e s o l u t i o n o f t h e p r o b l e m f r o m t h e p r e v i o u s s e c ­
t i o n u s i n g t h e t a b l e a u f o r m.
/ 2x\ + 5*2 - *3 < 1 \
—2 x\ — 1 2 * 2 + 3 ^ 3 ^ 9 —3 x j — 8 x 2 + 2 x 3 < 4
< X i > 0,i — 1,...,3 /
I n t r o d u c i n g s l a c k v a r i a b l e s, t h e c o n s t r a i n t s a r e w r i t t e n i n t h e s t a n d a r d L P f o r m a s f o l l o w s:
M i n i m i z e f = 5χι — 3x2 — 8x3
S u b j e c t t o
2xj + 5x2 — X3 + X4 = 1 - 2 x i — 12x2 + 3x3 + xg = 9 —3xi — 8x2 + 2x3 + Xq = 4
T h e s t a r t i n g t a b l e a u i s a s f o l l o w s: /B a s i s x i
I n i t i a l T a b l e a u:
x 4 X5 X6
V Obj.
2
- 2
- 3
5
X2
5
- 1 2
- 8
- 3
X3 —1 3 2 - 8
x 4
1
0
0
0
X5
0
1
0
0
X6
0
0
1
0
R H S\ 1 9 4 f
/
T h e f i r s t t h r e e r o w s a r e s i m p l y t h e c o n s t r a i n t e q u a t i o n s. T h e f o u r t h r o w i s t h e o b j e c t i v e f u n c t i o n, e x p r e s s e d i n t h e f o r m o f a n e q u a t i o n, 5 x i — 3x 2 — 8x 3 = /. T h e r i g h t - h a n d s i d e o f t h e o b j e c t i v e f u n c t i o n e q u a t i o n i s s e t t o /. S i n c e X4 a p p e a r s o n l y i n t h e f i r s t r o w, i t i s t h e b a s i c v a r i a b l e a s s o c i a t e d w i t h t h e f i r s t c o n s t r a i n t. S i m i l a r l y, t h e b a s i c v a r i a b l e s a s s o c i a t e d w i t h t h e o t h e r t w o c o n s t r a i n t s a r e X5 a n d xq. T h e b a s i c v a r i a b l e s f o r e a c h c o n s t r a i n t r o w a r e i d e n t i f i e d i n t h e f i r s t c o l u m n.
F r o m t h e t a b l e a u, w e c a n r e a d t h e b a s i c f e a s i b l e s o l u t i o n s i m p l y b y s e t t i n g t h e b a s i s t o t h e r h s ( s i n c e t h e n o n b a s i c v a r i a b l e s a r e a l l s e t t o 0).
B a s i c: x4 = 1, X5 = 9, xq = 4
N o n b a s i c: χ χ = x2 = X3 = 0 / = 0
W e n o w p r o c e e d t o t h e f i r s t i t e r a t i o n o f t h e s i m p l e x m e t h o d. I b b r i n g a n e w v a r i a b l e i n t o t h e b a s i c s e t, w e l o o k a t t h e l a r g e s t n e g a t i v e n u m b e r i n t h e o b j e c t i v e f u n c t i o n r o w. F r o m t h e s i m p l e x t a b l e a u, w e c a n r e a d i l y i d e n t i f y t h a t t h e c o e f f i c i e n t c o r r e s p o n d i n g t o X3 i s m o s t n e g a t i v e ( —8). T h u s, X3 s h o u l d b e n i a d e b a s i c.
T h e v a r i a b l e t h a t m u s t b e r e m o v e d f r o m t h e b a s i c s e t c o r r e s p o n d s t o t h e s m a l l e s t r a t i o o f t h e e n t r i e s i n t h e c o n s t r a i n t r i g h t - h a n d s i d e s a n d t h e p o s i t i v e
C h a p t e r 6 L i n e a r P r o g r a m m i n g
e n t r i e s i n t h e c o l u m n c o r r e s p o n d i n g t o t h e n e w v a r i a b l e t o b e m a d e b a s i c. F r o m t h e s i m p l e x t a b l e a u, w e s e e t h a t i n t h e c o l u m n c o r r e s p o n d i n g t o x3i t h e r e a r e t w o c o n s t r a i n t r o w s t h a t h a v e p o s i t i v e c o e f f i c i e n t s. R a t i o s o f t h e r i g h t - h a n d s i d e a n d t h e s e e n t r i e s a r e a s f o l l o w s:
R a t i o s: i f = 3, | = 2 }
i n e m i n i m u m r a c c u r r e n t b a s i c v a r i a B a s e d o n t h e s e
T h a t i s, w e n e e d t j e c t i v e f u n c t i o n r o1
i o c o r r e s p o i i u s t o u i c u u i u u u n s i r a m t i b i e. T h u s, w e s h o u l d m a k e x$ n o n b a s i c. d e c i s i o n s, o u r n e x t t a b l e a u m u s t b e o f t h
''B a s i s xi x i *3 x4 *5 xq R H S 1'
X4 — — 0 1 0 — —
*5 — ,,1— 0 0 1 — —
*3 - - 1 0 0 - -
^ O b j. - — 0 0 0 — — j
0 e l i m i n a t e v a r i a b l e *3 f r o m t h e f i r s t, s tv. S i n c e e a c h r o w r e p r e s e n t s a n e q u a t i o:
o r w m c n *6 i s t n e e f o l l o w i n g f o r m.
e c o n d, a n d t h e o b - ti, t h i s c a n b e d o n e
b y a d d i n g o r s u b t r. m u s t b e c a r e f u l i n X4 a n d *5 a s b a s i c v f o r m o f c o l u m n s T h e s y s t e m a t i c f o r m i s t o f i r s t d i v i t s d i a g o n a l e l e m e i T h u s, d i v i d i n g r o w
i c t i n g a p p r o p r i a t e m u l t i p l e s o f r o w s t o g t h o w w e p e r f o r m t h e s e s t e p s b e c a u s e w a r i a b l e s f o r t h e f i r s t a n d t h e s e c o n d c o n s a n d x s m u s t b e m a i n t a i n e d d u r i n g t h e s p r o c e d u r e t o a c t u a l l y b r i n g t h e t a b l e a i d e r o w 3 ( b e c a u s e i t i n v o l v e s t h e n e w i t ( t h e c o e f f i c i e n t c o r r e s p o n d i n g t o t h e n 3 b y 2, w e h a v e t h e f o l l o w i n g s i t u a t i o n:
'B a s i s X2 *3 *4 *5 xq R H S n
Χ4 — — 0 1 0 — —
i t h e r. H o w e v e r, w e e n e e d t o p r e s e r v e t r a i n t s. T h a t i s, t h e e r o w o p e r a t i o n s, u i n t o t h e d e s i r e d b a s i c v a r i a b l e ) b y Lew b a s i c v a r i a b l e ).
W e c a l l t h i s m o d i f i o t h e r r o w s. T h e c o
B a s i s x x4 \
* 5 §
* - 1 -
X5 — — U U i — —
*3 - § - 4 1 0 0 \ 2 ^ O b j. 0 — — — 0 0 — ,
e d r o w a s pivot row ( P R ) a n d u s e i t t o e l i i i m p u t a t i o n s a r e a s f o l l o w s:
:2 * 3 *4 x5 *6 R H S 1 0 1 0 \ 3 « =
3 0 0 1 - | 3 « < = - 3 - 4 1 0 0 \ 2 < =
m i n a t e *3 f r o m t h e
P R + R o w l x P R + R o w 2 P R
O b j. - 7 - 3 5 0 0 0 4 1 6 + f < = 8 x P R + O b j.R o w
6.4 T h e S i m p l e x M e t h o d
T h i s c o m p l e t e s o n e s t e p o f t h e s i m p l e x m e t h o d, a n d w e h a v e a s e c o n d b a s i c f e a s i b l e s o l u t i o n.
/B a s i s
S e c o n d T a b l e a u:
X4
* 5
Xl
1
2
5
2
X2
1
0
X4
1
0
X5
0
*6
1 2
1
2
R H S > 3 3
*3 - - f —4 1 0 0 I 2
\O b j. —7 —3 5 0 0 0 4 1 6 + f/
B a s i c: *3 = 2, X4 = 3, x s = 3 N o n b a s i c: *1 = x2 =» x e = 0 / = - 1 6
T h e s a m e s e r i e s o f s t e p s c a n n o w b e r e p e a t e d f o r a d d i t i o n a l t a b l e a u s. F o r t h e t h i r d t a b l e a u, w e s h o u l d m a k e Χ2 b e b a s i c ( t h e l a r g e s t n e g a t i v e c o e f f i c i e n t i n t h e o b j. r o w = —3 5 ). I n t h e c o l u m n c o r r e s p o n d i n g t o Χ2, o n l y t h e f i r s t r o w h a s a p o s i t i v e c o e f f i c i e n t a n d t h u s, w e h a v e n o c h o i c e b u t t o m a k e x\ ( t h e c u r r e n t b a s i c v a r i a b l e f o r t h e f i r s t r o w ) n o n b a s i c. B a s e d o n t h e s e d e c i s i o n s, o u r n e x t t a b l e a u m u s t b e o f t h e f o l l o w i n g f o r m:
^ B a s i s Χ2 xs X4 X5 xe R H S ^
X2 X5 X3
VObj.
1 0 0 0 0 1 0 0
0
1
0
0
T h e f i r s t r o w a l r e a d y h a s a 1 i n t h e x2 c o l u m n; t h e r e f o r e, w e d o n't n e e d t o d o a n y t h i n g a n d u s e i t a s o u r n e w p i v o t r o w t o e l i m i n a t e *2 f r o m t h e o t h e r r o w s. T h e c o m p u t a t i o n s a r e a s f o l l o w s:
B a s i s * ι Χ2 X3 X4 X5 Χ2 ^ 1 0 1 0
X 6
1
2
R H S
3
P R
* 5
* 3
O b j.
0 0
0
4
3 5
R o w 2 4 x P R + R o w 3 3 5 x P R + O b j.R o w
2 1
2
0
1
0
0
0
1 4
5 2
f 121 + f
T hi s c o m p l e t e s t h e s e c o n d i t e r a t i o n o f t h e s i m p l e x m e t h o d, a n d w e h a v e a third b a s i c f e a s i b l e s o l u t i o n:
/B a s i s x i
X2
T h i r d T a b l e a u:
1
2
x s 2 *3 \
V o b j. £
X2
1
0
0
x s
0
0
1
l i 0
X4
1
0
4
3 5
X 5
0
1
0
0
X 6
1
2
_ 3 2
5
2
^ 4 3
R H S
3
3
1 4
f 1 2 1 +//
C h a p t e r 6 L i n e a r P r o g r a m m i n g
B a s i c: X2 = 3, *3 = 1 4, *5 = 3 N o n b a s i c: * i = *4 = x g = 0 / = —1 2 1
S i n c e a l l c o e f f i c i e n t s i n t h e O b j. r o w a r e p o s i t i v e, w e c a n n o t r e d u c e t h e o b j e c ­
t i v e f u n c t i o n a n y f u r t h e r a n d t h u s h a v e r e a c h e d t h e m i n i m u m. T h e o p t i m u m s o l u t i o n i s
O p t i m u m: x\ = 0, x2 = 3, *3 = 1 4, *4 = 0, *5 = 3, x% — 0 f — — 1 2 1
E x a m p l e 6.1 1 S o l v e t h e f o l l o w i n g L P p r o b l e m u s i n g t h e t a b l e a u f o r m o f t h e s i m p l e x m e t h o d.
M a x i m i z e —7x 1 — 4*2 + 1 5 x 3
20^3
S u b j e c t t o
( * L _ 3 2 * 2
3
9
a _
13x;
6
9
2 x i
16x;
3
9
9
1 8
+ $ > - 3
/
x i > 0, i = 1,..., 3
N o t e t h a t t h e p r o b l e m a s s t a t e d h a s a g r e a t e r t h a n t y p e c o n s t r a i n t. H o w e v e r, s i n c e t h e r i g h t - h a n d s i d e o f a l l c o n s t r a i n t s m u s t b e p o s i t i v e, a s s o o n a s w e m u l t i p l y t h e t h i r d c o n s t r a i n t b y a n e g a t i v e s i g n, a l l c o n s t r a i n t s b e c o m e o f L E t y p e a n d t h e r e f o r e, w e c a n h a n d l e t h i s p r o b l e m w i t h t h e p r o c e d u r e d e v e l o p e d s o f a r.
2xj
1 6 x 2 X3
_ 9 · -
s a m e a s
2 x i 16 * 2 X3 „
*4---------------- < J
3 9 9 ”
I n t h e s t a n d a r d L P f o r m M i n i m i z e / = 7x\ + 4x2
/ * L _ 3 2 x 2 1 20x: '3 9 9
S u b j e c t t o
15*3
+ X4 = 1 ^
*1
6
2x
^ + ^ + X 5 = 2
1 6x2
— § ■ + X6 = 3
/
X i > 0,i = l 6
w h e r e X4, X5, a n d Χβ a r e s l a c k v a r i a b l e s f o r t h e t h r e e c o n s t r a i n t s.
/B a s i s
x i
* 2
X3
X4
* 5
X6
R H S\
X4
1
3
3 2
9
2 0
9
1
0
0
1
I n i t i a l T a b l e a u:
X 5
1
6
1 3
9
5
1 8
0
1
0
2
X 6
2
3
1 6
9
1
9
0
0
1
3
<Obj.
7
4
- 1 5
0
0
0
f /
N e w b a s i c v a r i a b l e = X3( — 1 5 i s t h e l a r g e s t n e g a t i v e n u m b e r i n t h e O b j. r o w ) R a t i o s: {2^79 = 0 - 4 5, = 7.2 } M i n i m u m = 0.4 5 = > x4 o u t o f t h e b a s i c s e t.
B a s i s * i
X2
X3
X4
* 5
Xq
R H S
* 3 J j
8
5
1
9
2 0
0
0
9
2 0
P R ( = R o w l/f )
* 5 5
- 1
0
1
8
1
0
1 5
8
x P R + R o w 2
xe - M
8
5
0
1
2 0
0
1
6 1
2 0
| x P R + R o w 3
O b j. f
- 2 0
0
2 7
4
0
0
γ + f
4 =
1 5 x P R + O b j.R o w
T h i s c o m p l e t e s o n e s t e p o f t h e s i m p l e x m e t h o d, a n d w e h a v e a s e c o n d b a s i c f e a s i b l e s o l u t i o n.
S e c o n d T a b l e a u:
/B a s i s
Xl
X2
X3
X4
* 5
xe
R H S \
* 3
. 3 2 0
8
5
1
9
2 0
0
0
9
20
* 5
1
8
- 1
0
1
8
1
0
1 5
8
xe
1 3
2 0
8
5
0
1
2 0
0
1
6 1
2 0
I, O b j.
3 7
4
- 2 0
0
2 7
4
0
0
? + f )
N e w b a s i c v a r i a b l e = *2( — 2 0 i s t h e l a r g e s t n e g a t i v e n u m b e r i n t h e O b j. r o w ) R a t i o s: {6g ^ ° } M i n i m u m ( o n l y c h o i c e ) = > ■ xe o u t o f t h e b a s i c s e t.
B a s i s
X3
* 5
X2
O b j.
Xl
_ 1
2
_ 9 _ 3 2 _ 1 3 3 2 9 8
X 2 *3
0
0
1
0
1
0
0
0
*4
1
2
JL
3 2
J.
3 2
5 9
8
X5 XQ
0
- 4; 1
0
5
8
5
8
2 5
2
R H S
7
2
1 2 1
3 2
6 1
3 2
3 5 9
8
+ /
I x P R + R o w l P R + R o w 2 P R ( = Ro w3/| ) 2 0 x P R + O b j.R o w
^ B a s i s * 3
x i
_ 1
2
X2 X3 0 1
X4
1
2
X5 X6 0 1
R H S \
7 2
T h i r d T a b l e a u:
Xs X2 V o b j.
_9_ ‘ 3 2 13 '3 2 9 8
0
1
0 -4; 1
0
0 0
3 ^
3 2
X
3 2
5 9
8
5
8
5
8
2 5
2
1 2 1 32 61 32
359 8
ψ +fJ
S i n c e a l l c o e f f i c i e n t s i n t h e O b j. r o w a r e p o s i t i v e, w e c a n n o t r e d u c e t h e o b ­
j e c t i v e f u n c t i o n a n y f u r t h e r, a n d w e h a v e r e a c h e d t h e m i n i m u m. T h e o p t i m u m s o l u t i o n i s
O p t i m u m: x i = 0, X2 = X3 = x a = 0, * 5 = x $ = 0 / =
3 2
6.4.4 U s i n g t h e S i m p l e x M e t h o d f o r P r o b l e m s w i t h G E o r E Q C o n s t r a i n t s
T h e s t a r t i n g b a s i c f e a s i b l e s o l u t i o n i s m o r e d i f f i c u l t t o o b t a i n f o r p r o b l e m s t h a t i n v o l v e g r e a t e r t h a n ( G E ) t y p e ( a f t e r m a k i n g t h e r i g h t - h a n d s i d e p o s i t i v e, i f n e c e s s a r y ) o r e q u a l i t y ( E Q ) c o n s t r a i n t s. T h e r e a s o n i s t h a t t h e r e i s n o u n i q u e p o s i t i v e v a r i a b l e a s s o c i a t e d w i t h e a c h c o n s t r a i n t. A u n i q u e s u r p l u s v a r i a b l e i s p r e s e n t i n e a c h G E c o n s t r a i n t, b u t i t i s m u l t i p l i e d b y a n e g a t i v e s i g n a n d t h u s w i l l g i v e a n i n f e a s i b l e s o l u t i o n i f t r e a t e d a s a b a s i c v a r i a b l e. A n e q u a l i t y c o n s t r a i n t d o e s n o t n e e d a s l a c k/s u r p l u s v a r i a b l e, a n d t h u s, o n e c a n n o t a s s u m e t h a t t h e r e w i l l a l w a y s b e a u n i q u e v a r i a b l e f o r e a c h e q u a l i t y c o n s t r a i n t.
T h e s i t u a t i o n i s h a n d l e d b y w h a t i s k n o w n a s t h e Phase I s i m p l e x m e t h o d. A u n i q u e a r t i f i c i a l v a r i a b l e i s a d d e d t o e a c h G E a n d E Q _ t y p e c o n s t r a i n t. T r e a t i n g t h e s e a r t i f i c i a l v a r i a b l e s a s b a s i c a n d t h e a c t u a l v a r i a b l e s a s n o n b a s i c g i v e s a s t a r t i n g b a s i c f e a s i b l e s o l u t i o n. A n a r t i f i c i a l o b j e c t i v e f u n c t i o n, d e n o t e d b y φ(χ), i s d e f i n e d a s t h e s u m o f a l l a r t i f i c i a l v a r i a b l e s n e e d e d i n t h e p r o b l e m. D u r i n g t h i s s o - c a l l e d P h a s e I, t h i s a r t i f i c i a l o b j e c t i v e f u n c t i o n i s m i n i m i z e d u s i n g t h e u s u a l s i m p l e x p r o c e d u r e. S i n c e t h e r e a r e n o r e a l c o n s t r a i n t s o n φ, t h e o p t i m u m s o l u t i o n o f P h a s e I i s r e a c h e d w h e n φ = 0. T h a t i s w h e n a l l a r t i f i c i a l v a r i a b l e s a r e e q u a l t o z e r o ( o u t o f t h e b a s i s ), w h i c h i s t h e l o w e s t v a l u e p o s s i b l e b e c a u s e a l l v a r i a b l e s a r e p o s i t i v e i n LP. T h i s o p t i m u m s o l u t i o n o f P h a s e I i s a b a s i c f e a s i b l e s o l u t i o n f o r t h e o r i g i n a l p r o b l e m s i n c e w h e n t h e a r t i f i c i a l v a r i a b l e s a r e s e t t o z e r o, t h e o r i g i n a l c o n s t r a i n t s a r e r e c o v e r e d. U s i n g t h i s b a s i c f e a s i b l e s o l u t i o n, w e a r e t h e n i n a p o s i t i o n t o s t a r t s o l v i n g t h e o r i g i n a l p r o b l e m w i t h t h e a c t u a l o b j e c t i v e f u n c t i o n. T h i s i s k n o w n a s Phase II, a n d i s t h e s a m e a s t h a t d e s c r i b e d f o r L E c o n s t r a i n t s i n t h e p r e v i o u s s e c t i o n.
C o n s i d e r t h e f o l l o w i n g e x a m p l e w i t h t w o G E c o n s t r a i n t s:
M i n i m i z e / = 2 x\ + 4 x 2 + 3*3
I n t r o d u c i n g s u r p l u s v a r i a b l e s x4 a n d xs, t h e c o n s t r a i n t s a r e w r i t t e n i n t h e s t a n d a r d L P f o r m a s f o l l o w s:
—*1 +X2 +X3 —X4 = 2 2x\ - f X2 — *5 = 1
N o w i n t r o d u c i n g a r t i f i c i a l v a r i a b l e s a n d *7, t h e P h a s e I o b j e c t i v e f u n c t i o n a n d c o n s t r a i n t s a r e a s f o l l o w s:
6.4 T h e S i m p l e x M e t h o d
p h a s e I P r o b l e m:
M i n i m i z e φ = *6 + *7
ί —Χ\ + X2 + * 3 — X4 + xe = 2\
S u b j e c t t o I 2 * i + X2 — xs + x? = 1 I
\ * j > 0, i = 1,..., 7 /
T h e s t a r t i n g b a s i c f e a s i b l e s o l u t i o n f o r P h a s e I i s a s f o l l o w s:
B a s i c: x§ — 2, *7 = 1
N o n b a s i c: * i = x 2 = *3 = *4 = x s = 0
φ — 3
B e f o r e p r o c e e d i n g w i t h t h e s i m p l e x m e t h o d, t h e a r t i f i c i a l o b j e c t i v e f u n c t i o n m u s t b e e x p r e s s e d i n t e r m s o f n o n b a s i c v a r i a b l e s. I t c a n e a s i l y b e d o n e b y s o l v ­
i n g f o r t h e a r t i f i c i a l v a r i a b l e s f r o m t h e c o n s t r a i n t e q u a t i o n s a n d s u b s t i t u t i n g i n t o t h e a r t i f i c i a l o b j e c t i v e f u n c t i o n.
F r o m t h e c o n s t r a i n t s, w e h a v e
*6 = 2 - f *1 — *2 — X3 + X4 X7 — 1 “ 2*1 — X2 + X5 T h u s, t h e a r t i f i c i a l o b j e c t i v e f u n c t i o n i s w r i t t e n a s
φ = χ β + χ7 = 3 — * i — 2*2 — *3 + *4 + xs
o r
φ — 3 = — *1 — 2*2 — *3 + * 4 + * 5
O b v i o u s l y, t h e a c t u a l o b j e c t i v e f u n c t i o n i s n o t n e e d e d d u r i n g P h a s e I. H o w e v e r, a l l r e d u c t i o n o p e r a t i o n s a r e p e r f o r m e d o n i t a s w e l l s o t h a t a t t h e e n d o f P h a s e I, f i s i n t h e c o r r e c t f o r m ( t h a t i s, i t i s e x p r e s s e d i n t e r m s o n n o n b a s i c v a r i a b l e s o n l y ) f o r t h e s i m p l e x m e t h o d. T h e c o m p l e t e s o l u t i o n i s a s f o l l o w s:
P h a s e I: I n i t i a l T a b l e a u
/ B a s i s
*1
X2
X3
*4
X5
xe
*7
R H S \
xe
- 1
1
1
- 1
0
1
0
2
*7
2
1
0
0
- 1
0
1
1
O b j.
2
4
3
0
0
0
0
/
\A r t O b j.
- 1
- 2
- 1
1
1
0
0
—3 + φ/
N e w b a s i c v a r i a b l e = *2 ( —2 i s t h e l a r g e s t n e g a t i v e n u m b e r i n t h e ArtObj. row )
Ratios: { | = 2, \ = 1} M i n i m u m = 1 = » *7 o u t o f t h e b a s i c set.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
P h a s e I: S e c o n d T a b l e a u
{ B a s i s 1 1 ^ x 3 X4 χς, x q x j R H S \
- 3 0 1 - 1 1 1 - 1 1
Xe XI
O b j. ^ A r t O b j.
—P R + R o w!
2 1 0 0 - 6 0 3 0 3 0 - 1 1
- 1 0 1 4 0 - 4
- 1 0 2
1
-4 + / - 1 + φ )
P R
—4 x P R + O b j.R o w 2 x P R + A r t O b j.R o w
N e w b a s i c v a r i a b l e = *3 ( —1 i s t h e f i r s t l a r g e s t n e g a t i v e n u m b e r i n t h e A r t O b j. r o w )
R a t i o s: { γ } M i n i m u m ( o n l y c h o i c e ) = » xq o u t o f t h e b a s i c s e t.
P h a s e I: T h i r d T a b l e a u
/ B a s i s X3
Χι X2 X3 X4 X5 Xe X7 R H S ^
- 3 0 1 - 1 1 1 - 1 1
PR
R o w 2
—3 x P R + O b j.R o w P R + A r t O b j.R o w
* 2 2 1 0 0 ^ 1 0^ I I
O b j. 3 0 0 3 1 - 3 - 1 - 7 + f
\A r t O b j. 0 0 0 0 0 1 1 φ /
A l l c o e f f i c i e n t i n t h e a r t i f i c i a l o b j e c t i v e f u n c t i o n r o w a r e n o w p o s i t i v e, s i g ­
n a l l i n g t h a t t h e o p t i m u m o f P h a s e I h a s b e e n r e a c h e d. T h e s o l u t i o n i s a s f o l l o w s:
B a s i c: *2 = 1 *3 = 1
N o n b a s i c: *1 = *4 = · · · = *7 = 0 φ = 0
S i n c e t h e a r t i f i c i a l v a r i a b l e s a r e n o w z e r o, t h e c o n s t r a i n t e q u a t i o n s n o w r e p - r e s e n t t h e o r i g i n a l c o n s t r a i n t s, a n d w e h a v e a b a s i c f e a s i b l e s o l u t i o n f o r o u r
o r i g i n a l p r o b l e m.
P h a s e I I w i t h t h e a c t u a l o b j e c t i v e f u n c t i o n c a n n o w b e g i n. I g n o r i n g t h e a r t i f i c i a l o b j e c t i v e f u n c t i o n r o w, w e h a v e t h e f o l l o w i n g i n i t i a l s i m p l e x t a b l e a u f o r P h a s e I I. N o t e t h a t t h e c o l u m n s a s s o c i a t e d w i t h a r t i f i c i a l v a r i a b l e s a r e r e a l l y n o t n e e d e d a n y m o r e e i t h e r. H o w e v e r, a s W i l l b e s e e n l a t e r, t h e e n t r i e s i n t h e s e c o l u m n s a r e u s e f u l i n t h e s e n s i t i v i t y a n a l y s i s. T h u s, w e c a r r y t h e s e c o l u m n s t h r o u g h P h a s e I I a s w e l l. H o w e v e r, w e d o n't u s e t h e s e c o l u m n s f o r a n y d e c i s i o n - m a k i n g.
P h a s e I I: I n i t i a l I k b l e a u
B a s i s X] Χ2 X3 X4
0 1 - 1 1 0 0
0 0 3
* 5 x e X7
1 1 - 1
- 1 0 1
.1 - 3 - 1
R H S ^ 1 1
- 7 + f }
6.4 T h e S i m p l e x M e t h o d
^ 1 c o e f f i c i e n t i n t h e o b j e c t i v e f u n c t i o n r o w ( e x c l u d i n g t h e a r t i f i c i a l v a r i a b l e s ) a r e p o s i t i v e, m e a n i n g t h a t w e c a n n o t f i n d a n o t h e r b a s i c f e a s i b l e s o l u t i o n w i t h ­
o u t i n c r e a s i n g t h e o b j e c t i v e f u n c t i o n v a l u e. T h u s, t h i s b a s i c f e a s i b l e s o l u t i o n i s t h e o p t i m u m s o l u t i o n o f t h e p r o b l e m, a n d w e a r e d o n e.
O p t i m u m s o l u t i o n:
* i = 0, x 2 = 1, *3 = 1. *4 = 0, x 5 = 0 f = 7
6.4.5 T h e B a s i c S i m p l e x F u n c t i o n
T h e a b o v e p r o c e d u r e i s i m p l e m e n t e d i n a Mathematica f u n c t i o n c a l l e d B a s i c - S i m p l e x. T h e f u n c t i o n i s i n t e n d e d t o b e u s e d f o r e d u c a t i o n a l p u r p o s e s. S e v e r a l i n t e r m e d i a t e r e s u l t s c a n b e p r i n t e d t o g a i n u n d e r s t a n d i n g o f t h e p r o c e s s. T h e f u n c t i o n a l s o i m p l e m e n t s t h e p o s t - o p t i m a l i t y ( s e n s i t i v i t y ) a n a l y s i s d i s c u s s e d i n a l a t e r s e c t i o n.
Mathematica
ming, f o r s o l v i n g L P p r o b l e m s. H o w e v e r, t h e s e f u n c t i o n s d o n o t g i v e i n t e r ­
m e d i a t e r e s u l t s. T h e y a l s o d o n o t p e r f o r m a n y s e n s i t i v i t y a n a l y s i s. F o r l a i g e p r o b l e m s, a n d w h e n n o s e n s i t i v i t y a n a l y s i s i s r e q u i r e d, t h e b u i l t - i n f u n c t i o n s s h o u l d b e u s e d.
Nee ds ["O p t i m i z a t i o n T o o l b o x'LPSimplex'n];
?Bas i cSi mpl e x
B a s i c S i m p l e x [ f, g, v a r s, o p t i o n s ]. S o l v e s a n LP p r o b l e m u s i n g P h a s e I a n d I I s i m p l e x a l g o r i t h m, f i s t h e o b j e c t i v e f u n c t i o n, g i s a l i s t o f c o n s t r a i n t s, a n d v a r s i s a l i s t o f v a r i a b l e s. S e e O p t i o n s [ B a s i c S i m p l e x ] t o f i n d o u t a b o u t a l i s t o f v a l i d o p t i o n s f o r t h i s f u n c t i o n.
O p t i o n s U s a g e [ B a s i c S i m p l e x ]
{ U n r e s t r i c t e d V a r i a b l e s -» { } , M a x l t e r a t i o n s - » 1 0, P r o b l e m T y p e - » M i n, S i m p l e x V a r i a b l e s -► { x, s, a } , P r i n t L e v e l -» 1, S e n s i t i v i t y A n a l y s i s -» F a l s e }
U n r e s t r i c t e d V a r i a b l e s i s an o p t i o n f o r LP and s e v e r a l QP p r o b l e m s.
A l i s t o f v a r i a b l e s t h a t a r e n o t r e s t r i c t e d t o b e p o s i t i v e c a n b e s p e c i f i e d w i t h t h i s o p t i o n. D e f a u l t i s { }.
M a x l t e r a t i o n s i s an o p t i o n f o r s e v e r a l o p t i m i z a t i o n methods. I t s p e c i f i e s maximum number o f i t e r a t i o n s a l l o w e d.
P r o b l e m T y p e i s a n o p t i o n f o r m o s t o p t i m i z a t i o n m e t h o d s. I t c a n e i t h e r b e M i n ( d e f a u l t ) o r M a x.
S i m p l e x V a r i a b l e s i s a n o p t i o n o f t h e B a s i c S i m p l e x. t s p e c i f i e s s y m b o l s
t o u s e w h e n c r e a t i n g v a r i a b l e n a m e s. D e f a u l t i s { x,s,a }, w h e r e 'x'
C h a p t e r 6 L i n e a r P r o g r a m m i n g
i s u s e d f o r p r o b l e m v a r i a b l e s, 's' f o r s l a c k/s u r p l u s, a n d 'a' f o r a r t i f i c i a l.
P r i n t L e v e l i s a n o p t i o n f o r m o s t f u n c t i o n s i n t h e O p t i m i z a t i o n T o o l b o x. I t i s s p e c i f i e d a s a n i n t e g e r. T h e v a l u e o f t h e i n t e g e r i n d i c a t e s h o w m u c h i n t e r m e d i a t e i n f o r m a t i o n i s t o b e p r i n t e d. A P r i n t L e v e l - » 0 s u p p r e s s e s a l l p r i n t i n g. D e f a u l t f o r m o s t f u n c t i o n s i s s e t t o 1 i n w h i c h c a s e t h e y p r i n t o n l y t h e i n i t i a l p r o b l e m s e t u p. H i g h e r i n t e g e r s p r i n t m o r e i n t e r m e d i a t e r e s u l t s.
S e n s i t i v i t y A n a l y s i s i s a n o p t i o n o f s i m p l e x m e t h o d. I t c o n t r o l s w h e t h e r a p o s t - o p t i m a l i t y ( s e n s i t i v i t y ) a n a l y s i s i s p e r f o r m e d a f t e r o b t a i n i n g a n o p t i m u m s o l u t i o n. D e f a u l t i s F a l s e.
E x a m p l e 6.1 2 S o l v e t h e f o l l o w i n g L P p r o b l e m u s i n g t h e s i m p l e x m e t h o d:
f = - ^ x l + 2 0 x 2 - j x 3 + 6 x 4;
g - | j x i - 8 x 2 - x 3 +■ 9 x 4 <, 1, j x i - 1 2 x 2 - ^ x 3 + 3 x 4 3 3, x 3 +■ x 4 £ l j - j
B a s i c S i n i p l e x [ f, g, { x l, x 2, x 3, * 4 }, P r i n t L e v e l -» 2 ];
P r o b l e m v a r i a b l e s r e d e f i n e d a s: { x l -» x-l , x 2 -> x 2, x 3 -» x3 , x 4 - » x 4 }
M i n i m i z e — + 2 0 x j — + 6x 4
' ^ - 8 x 2 - x 3 + 9 x 4 < 1 '
S u b j e c t t o 5jL - 1 2 x 2 - ^ + 3 x 4 < 3 x 3 + x 4 £ 1
A l l v a r i a b l e s a 0
* * * * * * * * * * I n i t i a l s i m p l e x t a b l e a u * * * * * * * * * *
N o t e t h a t v a r i a b l e s 5, 6, a n d 7 a r e t h e s l a c k v a r i a b l e s a s s o c i a t e d w i t h t h e t h r e e c o n s t r a i n t s. A l s o, s i n c e t h e p r o b l e m i n v o l v e s o n l y L E c o n s t r a i n t s, n o P h a s e I i s n e e d e d, a n d w e a r e d i r e c t l y i n t o P h a s e I I o f t h e s i m p l e x.
'J e w p r o b l e m v a r i a b l e s:
{xi/
x 2, x 3,
x 4.
S 1 / s2'
8 3}
'B a s i s
1
2 3
4
5
6
7
R H S'
5
1
τ
- 8 - 1
9
1
0
0
1
6
1
2
-12 - J
3
0
1
0
3
7
0
0 1
1
0
0
1
1
, O b j .
3
~ τ
2 0
6
0
0
0
f ,
V a r i a b l e t o b e m a d e b a s i c - » 1 R a t i o s: R H S/C o l u m n 1 - * (4 6 00) V a r i a b l e o u t o f t h e b a s i c s e t -» 5
6.4 T h e S i m p l e x M e t h o d
* * * * * * * * * * p h a s e I I ------ I t e r a t i o n i * * * * * * * * * *
/• B a s i s 1 2 3
1 1 - 3 2 - 4
6 0 4 \
7 Q 0 1
4
5
6
7
RHS
3 6
4
0
0
4
- 1 5
-2
1
0
1
1
0
0
1
1
O b j. 0 - 4 3 3 . 3 0 0 3 + f
V a r i a b l e t o b e m a d e b a s i c -*■ 2 R a t i o s: R H S/C o l u m n 2 -*■ |oo ^ ooj V a r i a b l e o u t o f t h e b a s i c s e t -» 6
* * * * * * * * * * P h a s e I I -------- I t e r a t i o n 2 * * * * * * * * * *
B a s i s 1 2 3 4 5
6
7
RHS 1
8
0
12
1
ΐ
0
1
ΐ
0
1
1
1
0
4 + f j
1 1 0 8 - 8 4 - 1 2 8
2 0 1
3 1 5
S "T 7 0 0 1 1 0
O b j. 0 0 - 2 1 8 1
V a r i a b l e t o b e m a d e b a s i c - » 3 R a t i o s: R H S/C o l u m n 3 - » | j j 1 V a r i a b l e o u t o f t h e b a s i c s e t - > 2
* * * * * * * * * * p h a s e I I I t e r a t i o n 3 **********
B a s i s 1 2 3 4 5 6 7 RHS
1 i ^ o - 4
3 0 § 1 - 1 0
7 0 - | · 0 - 1 1
I O b j .0 ^ 0 - 2
V a r i a b l e t o b e m a d e b a s i c - » 4 R a t i o s: R H S/C o l u m n 4 - » f CO 0 0 ---------
4
“T
4
4 T
5
~ T
8
T
2
T
2
"T
7
T
0 0 * 1 0
¥
2 0 T 2 7 1 τ 6 ^
V a r i a b l e o u t o f t h e b a s i c s e t - » 7
C h a p t e r 6 L i n e a r P r o g r a m m i n g
* * * * * * * * * * P h a s e i x ------ I t e r a t i o n 4 * * * * * * * * * *
B a s i s
1
2
3
4
5
6
7
RHS
1
3
4
1
0
0
73 6 T
8
I T
8
“ TT
0
1
0
0
9
1
TS
4
~ Ί Ί
4
Τ Ϊ
80
TS
TS
~ts
A
10
TT
1
TT
Ψ -
32
TS
1
TS
l Obj .
0
160
3 T
0
0,
47
~TS
73
Ί Ί
2
TT
Γ78 + f
3 3 - + t l
V a r i a b l e t o b e m a d e b a s i c -» 5 R a t i o s: R H S/C o l u m n 5 -► |a> ao i V a r i a b l e o u t o f t h e b a s i c s e t -» 4
* * * * * * * * * * p ^ s e I I I t e r a t i o n 5 * * * * * * * * * *
( B a s i s 1 2 3 4 5 6 7 RHS
1
3
1
0
- 2 4
0
- 2
0
1
7
1
3 3
T
4 7
0
0
2
0
1
1
3
τ
5
7
1
1
Ϊ
2 3
Obj .
0
+ f
O p t i m u m s o l u t i o n - » { { { x l - » 7,x 2 - » 0,x 3 - » l,x 4 - » 0 } } }
2 3
O p t i m u m o b j e c t i v e f u n c t i o n v a l u e - »-------
4
T h e s o l u t i o n c a n b e v e r i f i e d u s i n g t h e b u i l t - i n Mat hemat i ca f u n c t i o n, C o n - s t r a i n e d M i n, a s f o l l o w s:
C o n e t r a i n e d M i n [ £, g, { x l, x 2, x 3, x 4 } ]
' {χ 1 - > 7,χ 2 - > 0,χ 3 - » 1,χ 4 - » 0 } | ·
E x a m p l e 6,1 3 F i n d t h e m a x i m u m o f t h e f o l l o w i n g L P p r o b l e m u s i n g t h e
s i m p l e x m e t h o d. N o t e t h a t v a r i a b l e x2 i s n o t r e s t r i c t e d t o b e p o s i t i v e.
£ = - 3 x 1 + 2 x 2 - 4 x 3 + x 4 - x 5; g - { 2 x 1 + 3 x 2 + x 3 + 4 x 4 + 4 x 5 = = 1 2,
4 x 1 - 5 x 2 + 3 x 3 - x 4 - 4 x 5 = = 1 0, 3 x 1 - x 2 + 2 x 3 + 2 x 4 + x 5 £ 8 };
B a s i c S i m p l e x [ £, g, { x l, x 2, x 3, x 4, x 5 },
Probl emType -»Max, Un r e s t r i c t e d Va r i a b l e s -» {x2 } , Pr i nt Le v e l -» 2] ;
P r o b l e m v a r i a b l e s r e d e f i n e d a s: ( x l -> x - l , x 2 - > x2 - x3 , x 3 -s- x4, x 4 - » X5, x 5 - » X 6 }
M i n i m i z e 3 X! - 2 x2 + 2 x3 + 4 x4 - X5 + x6
'2 χ! + 3 x2 - 3 x 3 + x4 + 4 x s + 4 χ6 = = 1 21
S u b j e c t t o 4 x j - 5 x2 + 5 x3 + 3 x 4 - x5 - 4 x6 = = 1 0
3 x! - x2 + x3 + 2x 4 + 2x 5 + x6 > 8
A l l v a r i a b l e s > 0
* * * * * * * * * * m i t i a l s i m p l e x t a b l e a u ********■<
6.4 T h e S i m p l e x M e t h o d
T h e t h i r d c o n s t r a i n t n e e d s a s u r p l u s v a r i a b l e. T h i s i s p l a c e d i n t h e s e v e n t h c o l u m n o f t h e t a b l e a u. A l l t h r e e c o n s t r a i n t s n e e d a r t i f i c i a l v a r i a b l e s t o s t a r t t h e P h a s e I s o l u t i o n. T h e s e a r e p l a c e d i n t h e l a s t t h r e e c o l u m n s. T h e a r t i f i c i a l o b j e c t i v e f u n c t i o n i s t h e s u m o f t h e t h r e e a r t i f i c i a l v a r i a b l e s. A s e x p l a i n e d e a r l i e r, i t i s t h e n e x p r e s s e d i n t e r m s o f n o n b a s i c v a r i a b l e s t o g i v e t h e f o r m i n c l u d e d i n t h e f o l l o w i n g t a b l e a u.
New p r o b l e m v a r i a b l e s
: { x x
/ x 2 '
X3 , X4
/ X5 t
x 6' s 3 '
a l'
a 2, a 3 }
' B a s i s
1
2
3
4
5
6
7
8
9
1 0
RHS 1
8
2
3
- 3
1
4
4
0
1
0
0
1 2
9
4
- 5
5
3
- 1
- 4
0
0
1
0
1 0
1 0
3
- 1
1
2
2
1
- 1
0
0
1
8
Obj .
3
- 2
2
•4
-1
1
0
0
0
0
f
A r t O b j.
- 9
3
- 3
- 6
- 5
-1
1
0
0
0
- 3 0 + <*>j
V a r i a b l e t o b e m a d e b a s i c - » 1 R a t i o s: R H S/C o l u m n 1 V a r i a b l e o u t o f t h e b a s i c s e t -> 9
(«I !)
**********phas e I -------- I t e r a t i o n i * * * * * * * * * *
B a s i s 1 2 3 4 5 6 7 8 9 1 0 RHS
2
3
4
5
6
7
8
9
1 1
1 1
1
9
£
Λ
1
~ T
~ ~ T
f
Ο
υ
X
5
~ τ
5
τ
3
ΐ
1
"ϊ
- 1
0
0
1
ϊ
1 1
1 1
1
1 1
Λ
1
Λ
3
T
~ T
τ τ
4
υ
~τ
7
s
7
7
?
1
"ϊ
4
0
0
3
“ ϊ
3 3
33
3
2 9
1 Α
-1
Λ
9
τ
- 1 U
J.
υ
τ
8 0 ~ T ~ ~ T f 6 0 1 “ 2 0
i 1 | | - i - 1 0 0 \ 0
1 0 0 J j l · 1 U 4 - 1 0 - I 1
5
Ί
O b j. 0 \ \ 4 0 0 - f 0 - ψ + ΐ
A r t O b j. 0 -23 23 3 _29 _ 1 0 ι 0 § 0 - ψ + φ,
V a r i a b l e t o b e m a d e b a s i c -» 6
R a t i o s: R H S/C o l u m n 6 -» ( — oo i ]
1 6 BJ
V a r i a b l e o u t o f t h e b a s i c s e t -> 10
**********Phase I ------ I t e r a t i o n 2 * * * * * * * * * *
B a s i s 1 2 3 4 5 6 7 8 9 10 RHS
8
1 1 ~t s t s Ι έ ά 0 0 t s ϊ Ϋ
6 0 1 5 'T? '1 5 t t 1 - Ϊ 0 ■ 4 i i
O b j. ο -1 1 2 -3 0 1 0 0 -1 -8 + f
^ O b j.0 ψ f o - J 0 | § - Ϋ +Φ,
C h a p t e r 6 L i n e a r P r o g r a m m i n g
Variable to be made basic 7 Ratios: RHS/Column 7-» od od
Variable out of the basic set - » 8
* * * * * * * * * * Ph a s e I ----- I t e r a t i o n 3 **********
Basis 1 2 3 4 5 6 7 8 9 1 0 RHS
2
3
4
5
6
7
8
9
11
11
1
1
Λ
1
2
5
T2
“ 12
“ T2
ΐ
U
Ί
12
1
1
2
1
1
1
“ T
7
T
2
0
υ
τ
6
11
11
1
3
Λ
1
1
T2
“ T2
“ 12
ΐ
X
5
12
23
23
25
13
2
5
“ 12
T2
12
~ T
0
υ
“ Τ
“ 12
0
0
0
0
0
0
1
1
η Λ XX χ χ X χ A 1 Λ J. 1 25
7 0 -ττ -ττ τ U J- τ T 7 —
Ί Γ 1 1 T
6 0 t 4 _ t 4 - t V t 1 0 τ. ~ t t 0 ^
n v.j η ύ j ^ j ^ j υ η η ί. -> n 7 3 £
Ob ]. 0 j j t z - - ς - O O - 3 · 0 - ~ g - + f
.ArtObj .0 0 0 0 0 0 0 1 1 1 φ
End o f phase I
Va r i a bl e t o be made b a s i c - » 5
50 22 14 X
Ra t i o s: RHS/Column 5-> , ^ ^
Va r i a bl e out o f t he b a s i c s e t - » 6
* * * * * * * * * * Pha s e I I ----- I t e r a t i o n l * * * * * * * * * *
B a s i s 1 2 3 4 5 6 7 8 9 10 RHS
7
n 1 1 1 1 1 n l i i l 4 1 34
0 TF "T F "T F 0 “ T 1 TF 5 -1 T
2
3
4
5
6
11
11
1
A
1
I F
“ TF
“ I F
0
“ Τ
17
17
13
Λ
2
“ TF
TF
TF
ϋ
“ Τ
11
l l!
1
■9
4
Ί Γ
“ Ί Γ
“ 5
1
Τ
37
37
31
Λ
13
TF
"T F
TF
υ
Τ
1 -1 -L / -L / X O r, Δ n * Π
X 1 “ TT TT TT 0 - T U TT T u
1 2 n 26
T F 9 u “ S'
C r t L L L L I L 1 Q n 2 1 n 14
5 0 “5“ “ “5“ “ 5 1 T 0 9 “ 9 0 T
nM n 3 7 3 7 31 n 13 η 1 I n 64 A f
° b 3. 0 ^ 0 T 0 I F “ 9 0 ~~g~ + */
Variable to be made basic -» 3
I 52
Ratios: RHS/Column 3-» oo — oo
I 17
Variable out of the basic set -> 1
Iteration 2**********
4 5 6 7 8 9 10 RHS
Γ 7 0 “ 1 7 1 T 7 1 7 _1 T 7
η o - i i o ^ ^ t ^
T 7 1 Γ 7 0 Γ 7 Γ 7 0 1 7
T 7 0 Ϊ 7 0 A Ά 0 ~ T7 + f J
********** p h a s e
II - -
Basis
1
2
3
7
11
Γ 7
0
0
3
18
Γ 7
-1
1
5
22
1 7
0
0
, Obj .
0
0
Optimum
solution
-{{{=
’}}}
. 14
Optimum objective function value -» -----
17
E x a m p l e 6.1 4 Shortest route problem T h i s e x a m p l e d e m o n s t r a t e s f o r m u l a ­
t i o n a n d s o l u t i o n o f a n i m p o r t a n t c l a s s o f p r o b l e m s k n o w n a s network p r o b l e m s i n t h e L P l i t e r a t u r e. I n t h e s e p r o b l e m s a n e t w o r k o f nodes a n d links i s g i v e n. T h e p r o b l e m i s u s u a l l y t o f i n d t h e m a x i m u m f l o w o r t h e s h o r t e s t r o u t e. A s a n e x a m p l e, c o n s i d e r t h e p r o b l e m o f f i n d i n g t h e s h o r t e s t r o u t e b e t w e e n t w o c i t i e s w h i l e t r a v e l i n g o n a g i v e n n e t w o r k o f a v a i l a b l e r o a d s. A t y p i c a l s i t u a ­
t i o n i s s h o w n i n F i g u r e 6.3. T h e n o d e s r e p r e s e n t c i t i e s a n d t h e l i n k s a r e t h e r o a d s t h a t c o n n e c t t h e s e c i t i e s. T h e d i s t a n c e s i n k i l o m e t e r s a l o n g e a c h r o a d a rft n o t e d i n t h e f i g u r e.
F I G U R E 6.3 A n e t w o r k d i a g r a m s h o w i n g d i s t a n c e s a n d d i r e c t i o n o f t r a v e l b e t w e e n
c i t i e s.
T h e o p t i m i z a t i o n v a r i a b l e s a r e t h e r o a d s t h a t o n e c a n t a k e t o r e a c h t h e d e s t i n a t i o n. I n d i c a t i n g t h e r o a d s b y t h e i n d i c e s o f t h e n o d e s t h a t t h e y c o n n e c t, w i t h t h e o r d e r i n d i c a t i n g t h e d i r e c t i o n o f t r a v e l, w e h a v e t h e f o l l o w i n g s e t o f o p t i m i z a t i o n v a r i a b l e s. N o t e t h a t t w o s e p a r a t e v a r i a b l e s a r e n e e d e d f o r t h e r o a d s w h e r e t r a v e l i n e i t h e r d i r e c t i o n i s p o s s i b l e.
Variables = {*12, xu, *23» *32, *24, *25. *35, *54. *46, *56}
T h e o b j e c t i v e f u n c t i o n i s t o m i n i m i z e t h e d i s t a n c e t r a v e l l e d a n d i s s i m p l y t h e s u m o f m i l e s a l o n g e a c h rou te, as f o l l o w s:
M i n i m i z e f = 1 5 * 1 2 + 1 3 * 1 3 + 9 * 2 3 + 9 * 3 2 + 1 1 * 2 4 + 1 2 * 2 5 + 16x 35 + 4*54 +
17*46 + 14*56
r i h f l p t e r β L i n e a r P r o g r a m m i n g
T h e c o n s t r a i n t s e x p r e s s t h e r e l a t i o n s h i p b e t w e e n t h e l i n k s. T h e inflow a t a n o d e m u s t e q u a l t h e outflow.
N o d e 2:
* 1 2 + * 3 2 = * 2 4 + * 2 5 + * 2 3
N o d e 3:
* 1 3 + * 2 3 = * 3 2 + * 3 5
N o d e 4:
* 2 4 * 5 4 — * 4 6
N o d e 5:
* 3 5 + * 2 5 = * 5 4 + * 5 6
T h e o r i g i n a n d d e s t i n a t i o n n o d e s a r e i n d i c a t e d b y t h e f a c t t h a t t h e o u t f l o w f r o m t h e o r i g i n n o d e a n d i n f l o w i n t o t h e d e s t i n a t i o n n o d e a r e e q u a l t o 1.
N o d e 1: *12 + *13 = 1 N o d e 6: *46 + * 5 6 = 1
I n t e r m s o f Mathematica e x p r e s s i o n s, t h e p r o b l e m i s d e f i n e d a s f o l l o w s:
vars = { Χ χ 2 ' x13 ' x23 ' *32 ' x24 ' x25 · x35 ' x54 ' x46 · x56 } '
ST - {X12 + x32 == x24 + x25 + x23 ' x13 + x23 == x32 + x35 * x24 + x54 == x46 ' x 35 + *25 == x54 + x56 ' x 12 + x13 == x46 + x56 == ^ '
T h e s o l u t i o n i s o b t a i n e d b y u s i n g t h e B a s i c S i m p l e x, a s f o l l o w s:
B a s i c S i m p l e x [ £, g, v a r s, P r i n t L e v e l -» 2 ];
Problem variables redefined a s: {x 12 xl' X13 x 2' x23 x3'
x32 x4 ' x24 x5 ' x25 x 6 ' x35 x7 ' x54 x 8 ' x46 x9 ' x56 x10 Ϊ
M i n i m i z e 1 5 x ^ + 1 3 x 2 + 9 x3 + 9 x 4 + I I X 5 + 1 2 x 6 + 1 6 x 7 + 4 x8 + 1 7 x g + 1 4 x 10
x x - x 3 + x 4 - x 5 - x 6 == 0! x 2 + x 3 - x 4 - χ 7 == 0
x 5 + Xg - Xg = = 0
x 6 + x 7 - x 8 - X10 == 0 x x + x 2 == 1
x 9 + x 10 == 1 i-M/
A l l v a r i a b l e s > 0
* * * * * * * * * * i n i t i a l s i m p l e x t a b l e a u * * * * * * * * * *
N e w p r o b l e m v a r i a b l e s:
( X 1 / x 2 / x 3 t χ 4 r X5 r x g r X7 t X Q / x 9 r x 1 0 7 ^1 f a 2 r a 3 r a 4 r a 5 r a 6 Ϊ
S u b j e c t t o
6.4 T h e S i m p l e x M e t h o d
Basis
8
11 12
1 3
1 4
1 5
1 6 Obj.
ArtObj
1
0
0
0
1
- c u
15
- 2
0
1
0
0
1
0
13
- 2
-1
1
0
0
0
0
9
0
1
-1
0
0
0
0
9
0
-1
0
1
0
0
0
11
0
-1
0
0
1
0
0
12
0
0
- 1
0
1
0
0
16
0
0
0
1
- 1
0
0
4
0
9
10
11
12
13
14
15
16
RHS '
0
0
1
0
0
0
0
0
0
0
0
0
1
0
0
0
0
0
-1
0
0
0
1
0
0
0
0
0
-1
0
0
0
1
0
0
0
0
0
0
0
0
0 .
1
0
1
1
1
0
0
0
0
0
1
1
17
14
0
0
0
0
0
0
f
0
0
0
0
0
0
0
0
- 2 + φι
Var i a b l e t o be made bas i c -» 1
Rat i os: RHS/Column 1 -» (θ « oo oo 1 co)
Var i abl e out o f t he b a s i c s e t - »11
**********Phase I ------ I t e r a t i o n l ******^
Ba s i s 1 2 3 4 5 6
8
1 1 2
13
14
15
16 Obj .
^Art Obj.
0
0
- 1
0
0
1
17
0
1
0
0
0
0
0
0
0
0
1
0
0
1
0
13
- 2
- 1
1
0
0
1
0
24
- 2
1
- 1
0
0
- 1
0
- 6
2
- 1
0
1
0
1
0
26
- 2
- 1
0
0
1
1
0
2 7
- 2
0
- 1
0
1
0
0
16
0
0
0
1
- 1
0
0
4
0
RHS
0
0
0
0
1
1
f
- 2 +φ
1 0
0
0
0
- 1
0
1
14
0
11 12 13 14
15 16
1
0
0
0
- 1
0
-15
2
0
1
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
1
0
0
0
0
0
0
0
0
0
1
0
0
V a r i a b l e t o be made b a s i c -»2 Ratios: RHS/Column 2 -» (« 0 c o c o V a r i a b l e out o f t h e b a s i c s e t - » 1 2
C h a p t e r 6 L i n e a r P r o g r a m m i n g
**********Phase I ----- Iteration 2**********
Basis 1 2 3 4 5 6 7 8
1 1 0 - 1 1 - 1 - 1 0 0
2 0 1 1 - 1 0 0 - 1 0
1 3 0 0 0 0 1 0 0 1
1 4 0 0 0 0 0 1 1 - 1
1 5 0 0 0 0 ί I 1 0
1 6 0 0 0 0 0 0 0 0 Obj. 0 0 11 7 26 27 29 4
ArtObj .0 0 0 0 -2 -2 -2 0
9 10 11 12 13 14 15 16 RHS '
0 0 1 0 0 0 0 0 0
0 0 0 1 0 0 0 0 0
- 1 0 0 0 1 0 0 0 0
0 - 1 0 0 0 1 0 0 0
0 0 - 1 - 1 0 0 1 0 1
1 1 0 0 0 0 ϋ 0 1 1
1 7 1 4 - 1 5 - 1 3 0 0 0 0 f
0 0 2 2 0 0 0 0 - 2 + Φ,
Variable to be made basic -» 5 Ratios: RHS/Column 5 ^ (qo cd 0 oo 1 oo)
Variable out of the basic s e t -»13
********** Phase I Iteration 3 **********
’ Basis 1 2 3 4 5 6 7 8
“ — — — “ “ “ j “ — — — — — —
1 1 0- 1 1 0 - 1 0 1
2 0 1 1 - 1 0 0 — 1 0
5 0 0 0 0 1 0 - 0 1
14 0 0 0 0 0 1 1 -1
15 0 0 0 0 0 1 1 -1
16 0 0 0 0 0 0 0 0
O b j.- - - - - - - - -0 0 I I 7 0 2 7 2 9 - 2 2
ι A r t O b j.0 0 0 0 0 - 2 - 2 2
9 1 0 1 1 1 2 1 3 1 4 1 5 1 6 R H S
- 1 0 1 0 1 0 0 0 0
0 0 0 1 0 0 0 0 0 -1 0 0 0 1 0 0 0 0
0 -1 0 0 0 1 0 0 0
1 0 - 1 - 1 - 1 0 1 0 1
1 1 0 0 0 0 0 1 1
43 14 -15 -1 3 -26 0 0 0 f
-2 0 2 2 2 0 0 0 - 2 + φ,
Variable to be made basic -> 6
R a t i o s: R H S/C o l u m n 6 ( c o c o00 0 1 00)·
6.4 T h e S i m p l e x M e t h o d
v a r i a b l e o u t o f t h e b a s i c s e t - » 1 4 * * * * * * * * * * P h a s e I -------- I t e r a t i o n 4* * * * * * * * * *
B a s i s 1 2 3 4 5 6 7
1
2
5
1
0
0
0
1
0
-1
1
0
1
-1
0
0
0
1
0
0
0
1
-1
0
0
0
1
6
1 5
1 6 Obj .
A r t O b j
0
0
0
0
0
10
0
0
0
0
0
11
0
0
0
11
0
0
0
0
7
0
0
0
0
0
0
1
0
0
0
0
1
0
0
2
0
12
1 3 1 4 1 5 1 6
0 0 5 0
R H S
- 1
0
- 1
0
1
1
- 1
0
0
- 1
1
1
1
0
0
f t
- 1
0
0
1
0
0
- 1
0
1
0
1
0
- 1
0
1
0
0
1
- 1
0
0
0
0
0
1
0
0
0
0
0
0
1
0
0
0
0
1
1
4 3
-2
4 1
-2
- 1 5
2
- 1 3
2
- 2 6
2
- 2 7
2
0
0
0
0
f
-2 +
V a r i a b l e t o b e m a d e b a s i c - » 9
R a t i o s: R H S/C o l u m n 9 (<x> co tx> 00 1 l )
V a r i a b l e o u t o f t h e b a s i c s e t - » 1 6
**********Phase I ------ I t e r a t i o n 5 **********
B a s i s 1 2 3 4 5 6 7
1
2
5
6
1
0
0
0
0
1
0
0
1
0
0
0
0
1
0
8
0
0
1
-1
1 5
0
0
0
0
0
0
0
0
9
0
0
0
- 0
0
0
0
0
Obj.
0
0
11
7
0
0
2
5
ι Art Obj .
0
0
0
0
0
0
0
0
9
10
11
12
13
14
15
16
RHS '
0
0
1
0
1
1
0
1
1
0
0
0
1
0
0
0
0
0
0
1
0
0
1
0
0
1
1
0
-1
0
0
0
1
0
0
0
0
0
-1
-1
-1
-1
1
-1
0
1
1
0
0
0
0
0
1
1
6
-2
- 1 5
- 1 3
- 26
- 2 7
0
- 43
- 4 3 + f
0
0
2
2
2
2
0
2
Φ
N o t e t h a t t h e v a r i a b l e 1 5, w h i c h i s t h e a r t i f i c i a l v a r i a b l e f o r t h e f i f t h c o n s t r a i n t, i s s t i l l i n t h e b a s i s. H o w e v e r, s i n c e i t h a s a z e r o r h s, t h e a r t i f i c i a l o b j e c t i v e f u n c t i o n v a l u e i s r e d u c e d t o 0 a n d w e a r e d o n e w i t h P h a s e I. F u r t h e r m o r e, w e a l s o n o t i c e t h a t i n t h e s a m e c o n s t r a i n t r o w ( f i f t h ), a l l c o e f f i c i e n t s c o r r e s p o n d i n g t o n o n a r t i f i c i a l v a r i a b l e s ( c o l u m n s 1 t h r o u g h 10) a r e 0. T h i s i n d i c a t e s t h a t t h i s c o n s t r a i n t i s r e d u n d a n t a n d c a n b e r e m o v e d f r o m t h e s u b s e q u e n t i t e r a t i o n s. F o r e a s e o f i m p l e m e n t a t i o n, h o w e v e r, t h i s c o n s t r a i n t i s k e p t i n t h e f o l l o w i n g t a b l e a u s.
End of phase I
Variable to be made basic -s·10
Ratios: RHS/Column 10 -» (oo oo 1 oo oo l)
Variable out of the basic set-» 9
**********Phase II ----- Iteration l*******
Basis 1 2 3 4 5 6 7
-1
2
0
1
1
-1
0
0
-1
0
5
0
0
0
0
1
0
0
1
6
0
0
0
0
0
1
1
-1
15
0
0
0
0
0
0
0
0
10
0
0
Ό
0
0
0
0
0
Obj.
0
0
11
7
0
0
2
5
9
10
11
12
13
14
15
16
RHS
0
0
1
0
1
1
0
1
1
0
0
0
1
0
0
0
0
0
-1
0
0
0
1
0
0
0
0
1
0
0
0
0
1
0
1
1
0
0
-1
-1
-1
-1
1
-1
. 0
1
1
0
0
0
0
0
1
1
2
0
- 1 5
- 1 3
- 2 6
- 2 7
0
- 4 1
- 4 1 +
O p t i m u m s o l u t i o n - » { { { x12 1/ * 1 3 -» 0, x 23 -* 0, x 32 -* 0, x 24 -» 0,
x25 "* x35 X 54 -* 0, X46 -* 0, x 56 -» 1} } }
Optimum objective function value->41
T h e s o l u t i o n i n d i c a t e s t h a t t h e s h o r t e s t r o u t e i s 1 2, 2 5, a n d 5 6, w i t h
a t o t a l d i s t a n c e o f 4 1 k i l o m e t e r s.
6.5 U n u s u a l S i t u a t i o n s A r i s i n g D u r i n g t h e S i m p l e x S o l u t i o n
F o r c e r t a i n p r o b l e m s, u n u s u a l s i t u a t i o n s m a y a r i s e d u r i n g t h e s o l u t i o n u s i n g t h e s i m p l e x m e t h o d. T h e s e s i t u a t i o n s s i g n a l s p e c i a l b e h a v i o r o f t h e o p t i m i z a ­
t i o n p r o b l e m b e i n g s o l v e d.
D u r i n g P h a s e I, i f t h e a r t i f i c i a l o b j e c t i v e f u n c t i o n s t i l l h a s a p o s i t i v e v a l u e ( i.e., a n a r t i f i c i a l v a r i a b l e s t i l l i s i n t h e b a s i c s e t ) b u t a l l c o e f f i c i e n t s i n t h e a r t i f i ­
c i a l o b j e c t i v e r o w a r e p o s i t i v e, we o b v i o u s l y c a n n o t p r o c e e d a n y f u r t h e r. T h e o p t i m u m o f P h a s e I h a s b e e n r e a c h e d, b u t w e s t i l l d o n't h a v e a b a s i c f e a s i ­
b l e s o l u t i o n f o r t h e a c t u a l p r o b l e m. T h i s s i t u a t i o n i n d i c a t e s t h a t t h e o r i g i n a l p r o b l e m d o e s n o t h a v e a f e a s i b l e s o l u t i o n. T h e f o l l o w i n g n u m e r i c a l e x a m p l e d e m o n s t r a t e s t h i s b e h a v i o r.
E x a m p l e 6.1 5
f = x 2 ;
g = { x l + x 2 £ -3, x l - 2 x 2 2 -1} ;
B a s i c S i m p l e x [ f, g, { x l, x 2 }, P r o b l e m T y p e - » M a x, P r i n t L e v e l -» 2 ];
Problem variables redefined as: {xl -> x^, x2 -» x 2 }
Minimize - x 2
6.5.1 N o F e a s i b l e S o l u t i o n
Subject to
All variables > 0
**********
****** i n i t i a l simplex tableau
New probl em variabl es: {xx, x2, s1? s2, ax, a2} Basis 1 2 3 4 5 6 RHS
5 - 1 - 1 - 1 0 1 0 3
6 - 1 2 0 - 1 0 1 1
O b j. 0 - 1 0 0 0 0 f
\ArtObj. 2 - 1 1 1 0 0 - 4 + φ,
V a r i a b l e t o b e ma d e b a s i c -> 2
V a r i a b l e o u t o f t h e b a s i c s e t -» 6
**********Phase I ----- Iteration 1 **********
( Basis 1 2 3 4 5 6 RHS
5 2
O b j . t A r t O b j .
E n d o f p h a s e I
3
"2
1
~ 7
1
"2
3
2
0
1
0
0
- 1
0
0
1
1
“ 2
1
“ 2
1
"2
1
2
1
0
0
0
1
2
1
2
1
2
1
Ί
7
1
i
i + f
- j + Φ,
U n b o u n d e d o r i n f e a s i b l e p r o b l e m
T h e g r a p h i c a l s o l u t i o n, s h o w n i n F i g u r e 6.4, c l e a r l y s h o w s t h a t t h e r e i s n o f e a s i b l e s o l u t i o n f o r t h i s p r o b l e m.
X2
F I G U R E 6.4 A g r a p h s h o w i n g t h a t t h e r e i s n o f e a s i b l e s o l u t i o n f o r t h e p r o b l e m.
6.5.2 U n b o u n d e d S o l u t i o n
A s s u m e t h a t t h e o b j e c t i v e f u n c t i o n r o w o f a g i v e n p r o b l e m i n d i c a t e s t h a t o n e o f t h e v a r i a b l e s c a n b e b r o u g h t i n t o t h e b a s i c s e t. T h i s c l e a r l y m e a n s t h a t a l o w e r v a l u e o f t h e o b j e c t i v e f u n c t i o n i s p o s s i b l e. N o w i f c o e f f i c i e n t s o f t h a t v a r i a b l e i n a l l t h e c o n s t r a i n t e q u a t i o n s a r e n e g a t i v e, i t m e a n s t h a t n o n e o f t h e c o n s t r a i n t s i s i n d a n g e r o f b e i n g v i o l a t e d. T h a t i s, w e c a n l o w e r t h e v a l u e o f t h e o b j e c t i v e f u n c t i o n w i t h o u t w o r r y i n g a b p u t c o n s t r a i n t s. C l e a r l y, t h i s m e a n s
t h a t t h e p r o b l e m i s u n b o u n d e d a n d h a s a s o l u t i o n /* = - o o. T h e f o l l o w i n g n u m e r i c a l e x a m p l e i l l u s t r a t e s t h i s s i t u a t i o n:
E x a m p l e 6.1 6
£ = 2 x 1 + 3 x 2;
g = {xl + 3 x 2 £ 3, x l - x 2 5 2 } ;
B a s i c S i m p l e x [ f, g, { x l, x 2 }, P r o b l e m T y p e -> M a x, P r i n t L e v e l
problem variables redefined as : {xl -» x l f x 2 -> x 2 } Minimize - 2 x x - 3 x 2
2 ];
/xi + 3xo a 3\ Subject t o ( ^.X;,s 2 )
A l l v a r i a b l e s > 0 *** ******* I n i t i a l s i mp l e x t a b l e a u * * *** ** ***
New pr o bl e m v a r i a b l e s:{ x i,x2/
sl' s 2
, a i )
f B a s i s
1
2
3 4
5
R H S '
- - - - -
- - - - -
- - - - -
- - - - - - - - - -
- - - - -
- —
5
1
3
- 1 0
1
3
4
1
- 1
0 1
ΰ
2
O b j .
- 2
- 3
0 0
0
f
A r t O b j. V a r i a b l e
- 1 to be
-3
made
1 0 basic -> 2
0 '
-3 + φί
Rat i os: RHS/Column 2 -» ( l oo)
Va r i a bl e out o f t h e b a s i c s e t - »5 **********Phase I ------ I t e r a t i o n 1*
Ba s i s
2
4
Obj .
I Art Obj.
1
T
4
T
-1
0
1
0
0
0
1
~ 7
1
-1
0
4
0
1
0
0
1
T
1
T
1
1
RHS
1
3
3 + f Φ
End o f phas e I
Va r i a bl e t o be made b a s i c - * 1
R a t i o s: R H S/C o l u m n 1 - » | 3 —
V a r i a b l e o u t o f t h e b a s i c s e t - ♦ 4 * * * * * * * * * * p h a s e I I - - - - - - I t e r a t i o n i * * * * * * * * * *
B a s i s
2
1
O b j.
1
0
0
1
τ
1
■ ΐ
5
- τ
4
1
~ τ
3
τ
3
ΐ
1
τ
ι
ϊ
5
ϊ
R H S
i
!
2 1
+ f
U n b o u n d e d o r i n f e a s i b l e p r o b l e m
C h a p t e r 6 L i n e a r P r o g r a m m i n g
T h e g r a p h i c a l s o l u t i o n, s h o w n i n F i g u r e 6.5, c l e a r l y s h o w s t h a t t h e f e a s i b l e r e g i o n i s u n b o u n d e d a n d t h e m i n i m u m i s /* = — o o.
*2
6.5.3 M u l t i p l e S o l u t i o n s
W e k n o w t h a t a t t h e o p t i m u m p o i n t, t h e c o e f f i c i e n t s o f a l l b a s i c v a r i a b l e s i n t h e o b j e c t i v e f u n c t i o n r o w a r e z e r o. N o r m a l l y, t h e c o e f f i c i e n t s o f n o n b a ­
s i c v a r i a b l e s i n t h e r o w a r e p o s i t i v e, i n d i c a t i n g t h a t i f o n e o f t h e s e v a r i a b l e s i s b r o u g h t i n t o t h e b a s i c s e t, t h e o b j e c t i v e f u n c t i o n v a l u e w i l l i n c r e a s e. T h e s a m e r e a s o n i n g w o u l d s u g g e s t t h a t i f t h e c o e f f i c i e n t o f o n e o f t h e s e n o n b a s i c v a r i a b l e s i s z e r o, t h e n a n o t h e r b a s i c f e a s i b l e s o l u t i o n i s p o s s i b l e w i t h o u t i n ­
c r e a s i n g t h e o b j e c t i v e f u n c t i o n v a l u e. T h i s o b v i o u s l y m e a n s t h a t t h e p r o b l e m m a y h a v e m u l t i p l e o p t i m u m p o i n t s ( d i f f e r e n t p o i n t s b u t a l l w i t h t h e s a m e o b j e c t i v e f u n c t i o n v a l u e ). T h i s s i t u a t i o n o c c u r s w h e n o n e o f t h e a c t i v e c o n ­
s t r a i n t s i s p a r a l l e l t o t h e o b j e c t i v e f u n c t i o n. T h e f o l l o w i n g n u m e r i c a l e x a m p l e i l l u s t r a t e s t h i s s i t u a t i o n:
E x a m p l e 6.1 7
f = - x l - x 2;
g = { x l - x 2 £ 1, x l + x 2 Z 2 } ;
B a s i c S i m p l e x [ £ r g, { x l, x 2 },P r i n t L e v e l -* 2];
6.5 Unusual Situations Arising During the Simplex Solution
problem variables redefined as: {xl -»x j, x 2 -♦ x2}
Minimize - x 1 - x 2
A l l v a r i a b l e s > 0
]
* * * * * * * * * * i n i t i a l s i m p l e x t a o i e \j e w p r o b l e m v a r i a b l e s: { x1,x 2, s^, /Basis 1 2 3 4 RHS 1
3 1 - 1 1 0 1
4 1 1 0 1 2
ι O b j. - 1 - 1 0 0 £ ;
/a r i a b l e t o b e m a d e b a s i c - * 1 latios: RHS/Column 1 (l 2)
/ariable out of the basic set -» 3 **********Phase 11 ----- Iteration
ΊΤΊΤΊΤΤΤΊΤΊΤΊΤΊT TT IT
S2 }
I
\
Basis 1 2 3 4 RHS
1 1 - 1 1 0 1 4 0 2 -1 1 1 Obj. 0 -2 1 0 1 + f
/ariable to be made basic-» 2 latios: RHS/Column 2 |oo
V a r i a b l e o u t o f t h e b a s i c s e t - ♦ 4
* * * * * * * * * P h a s e I I - - - - - I t e r a t i o n
' B a s i s 1 2 3 4 R H S
c
c
*
a
1 l 0 1 1 3
1 1 0 Ί 1 Ί
2 η i 1 1 1 * 0 1 “ 2 Ί Ί
Obj. 0 0 0 1 2 + f Optimum s o l u t i o n -» , x 2 -»
Optimum objective function value * The problem may have multiple
A s d e m o n s t r a t e d b y t h e g r a p h s l · i r y o f c o n s t r a i n t g 2 i n t h e f e a s i b l e r<
^ } } }
-» -2
s o l u t i o n s.
i o w n i n F i g u r e 6.6, a n y p o i n t o n t h e b o u n d - s g i o n h a s t h e s a m e o b j e c t i v e f u n c t i o n v a l u e
a s t h e o p t i m u m s o l u t i o n g i v e n b y t h e s i m p l e x m e t h o d.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
X2
\l
N | f e. I N T 1
..
* \. - A 8 Any point \
k i s minimum
&
F e a s i b l e ^ ^\////^ \\ - 0.6 /N i
0:8
1 .....
- 1 0 1 2 3
F I G U R E 6.6 A g r a p h s h o w i n g m u l t i p l e s o l u t i o n s.
6.5.4 D e g e n e r a t e S o l u t i o n
I n a b a s i c f e a s i b l e s o l u t i o n, i f a n y o f t h e b a s i c v a r i a b l e s h a s a z e r o v a l u e, t h e n t h a t s o l u t i o n i s c a l l e d a degenerate b a s i c f e a s i b l e s o l u t i o n. A n L P p r o b l e m i s c a l l e d degenerate i f o n e o r m o r e o f i t s b a s i c f e a s i b l e s o l u t i o n s i s d e g e n e r a t e; o t h e r w i s e, i t i s k n o w n a s nondegenerate.
F o r a n o n d e g e n e r a t e L P p r o b l e m, e a c h i t e r a t i o n o f t h e s i m p l e x m e t h o d r e ­
d u c e s t h e o b j e c t i v e f u n c t i o n v a l u e. T h u s, t h e m e t h o d c a n n o t g o t o a p r e v i o u s b a s i c f e a s i b l e s o l u t i o n b e c a u s e i t w i l l h a v e a h i g h e r o b j e c t i v e f u n c t i o n v a l u e. F o r a d e g e n e r a t e L P p r o b l e m, s i n c e t h e b a s i c v a r i a b l e w i t h a z e r o v a l u e d o e s n o t h a v e a n y i n f l u e n c e o n t h e o b j e c t i v e f u n c t i o n v a l u e, i t i s p o s s i b l e t o h a v e i t e r a t i o n s i n w h i c h t h e o b j e c t i v e f u n c t i o n v a l u e i s n o t d e c r e a s e d. F u r t h e r m o r e, a t l e a s t t h e o r e t i c a l l y, i t i s p o s s i b l e f o r t h e m e t h o d t o k e e p i t e r a t i n g i n f i n i t e l y b e t w e e n t w o a d j a c e n t b a s i c f e a s i b l e s o l u t i o n s w i t h o u t d e c r e a s i n g /. T h i s p h e ­
n o m e n o n i s k n o w n a s cycling. A s t h e f o l l o w i n g e x a m p l e s d e m o n s t r a t e, n o t e v e r y d e g e n e r a t e c a s e w i l l h a v e a c y c l i n g p r o b l e m. H o w e v e r, t o a v o i d a n y p o s s i b i l i t y o f c y c l i n g t h e c o m m e r c i a l i m p l e m e n t a t i o n s o f t h e s i m p l e x m e t h o d, i n c o r p o r a t e a d d i t i o n a l r u l e s t o a v o i d v i s i t i n g a p r e v i o u s b a s i c f e a s i b l e s o l u t i o n.
6.5 U n u s u a l S i t u a t i n g A r i s i n g D u r i n g t h e S i m p l e x S o l u t i o n
O n e o f t h e s i m p l e s t r u l e s t o a v o i d c y c l i n g i s k n o w n a s Bland's rule. A c c o r d i n g t o t h i s r u l e, w h e n d e c i d i n g w h i c h v a r i a b l e t o b r i n g i n t o t h e b a s i c s e t, i n s t e a d o f s e l e c t i n g t h e v a r i a b l e c o r r e s p o n d i n g t o t h e l a r g e s t n e g a t i v e n u m b e r i n t h e o b j e c t i v e f u n c t i o n r o w, w e s i m p l y s e l e c t t h e first negative c o e f f i c i e n t i n t h a t r o w. U s i n g t h i s r u l e, i t i s e a s y t o s e e t h a t a v e r t e x w i l l b e v i s i t e d o n l y o n c e a n d t h u s, t h e r e i s n o p o s s i b i l i t y o f c y c l i n g.
E x a m p l e 6.1 8 Degenerate case C o n s i d e r t h e f o l l o w i n g p r o b l e m w i t h t h r e e v a r i a b l e s a n d t h r e e e q u a l i t y c o n s t r a i n t s:
£ = x l + x 2 + x 3;
g - { - x l + x 2 == 0, - x 2 + x 3 == 0, x l + x 3 == 3 } ;
S i n c e t h e r e a r e t h r e e e q u a l i t y c o n s t r a i n t s, i t i s n o t r e a l l y a n o p t i m i z a t i o n p r o b l e m. T h e s o l u t i o n i s s i m p l y t h e o n e t h a t s a t i s f i e s t h e c o n s t r a i n t e q u a t i o n s. T h u s, s o l v i n g t h e c o n s t r a i n t e q u a t i o n s, w e g e t t h e s o l u t i o n a s f o l l o w s:
S o l v i n g t h e p r o b l e m w i t h t h e s i m p l e x m e t h o d, w e n o t i c e t h a t t h e i n i t i a l b a s i c f e a s i b l e s o l u t i o n f o r P h a s e I i s d e g e n e r a t e. T h e m e t h o d p e r f o r m s s e v e r a l i t e r a ­
t i o n s w i t h o u t a n y i m p r o v e m e n t i n t h e o b j e c t i v e f u n c t i o n. E v e n t u a l l y, f o r t h i s p r o b l e m, t h e m e t h o d d o e s f i n d t h e s o l u t i o n. H o w e v e r, i n g e n e r a l, t h e r e i s n o s u c h g u a r a n t e e f o r p r o b l e m s i n v o l v i n g d e g e n e r a t e b a s i c f e a s i b l e s o l u t i o n s.
B a s i c S i m p l e x [ f, g, { x l, x 2, x 3 >, P r i n t L e v e l -*2];
Problem variables redefined as : {xl -> x^ , x2 x2, x3 -» X 3 }
Minimi ze x^ + ,x 2 + X3
S o l v e [ g ]
( - * 1 +X2 == S u b j e c t t o - x 2 + X3 ==.0
, X 1 + *3 "= 3 -
Al l vari abl es & 0
* * * * * * * * * * I n i t i a l s i m p l e x t a b l e a u * * * * * * * * * * New p r o bl e m v a r i a b l e s: { χ χ, x 2, x 3, a!, a 2 , a 3 }
' B a s i s 1 2 3 4 5 6 RHS
4 - 1 1 0 1 0 0 0
5 0 - 1 1 0 1 0 0
6 1 0 1 0 0 1 3
O b j. 1 1 1 0 0 0 £
\A r t O b j .0 0 -2 0 0 0 -3 + φ.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
Variable to be made basic 3 Ratios; RHS/Column 3 -* (<*> 0 3)
Variable out of the basic set -* 5
**********Phase I ------ Iteration i**********
Basis 1 2 3 4 5 6 RHS
4
3
6
Obj
- 1
0
1
1
1 0
- 1 1
1 0
2 0
1
0
0
0
0 0
1 0
- 1 1
- 1 0
0
0
3
f
- 3 + Φ)
A r t O b j.
- 2
V a r i a b l e t o b e ma d e b a s i c -*► 2 R a t i o s: RHS/Co l u mn 2 - » ( 0 oo 3 ) V a r i a b l e o u t o f t h e b a s i c s e t - » 4
* * * * * * * * * * p h a s e
I -
— I t e r a t i o n
2 *+++*+++*+
" B a s i s
1
2
3
4
5
6
RHS
2
- 1
1
0
1
0
0
0
3
- 1
0
1
1
1
0
0
6
2
0
0
- 1
- 1
1
3
O b j .
3
0 ·
0
- 2
- 1
0
f
A r t O b j.
- 2
0
0
2
2
0
- 3 + Φ,
Vari ahl e to be made basic -» 1
Ratios: RHS/Column 1 -♦ ao —
Variable out of the basic set -» 6 **********Phase I —— Iteration
' Basis
1
2
3
4
5 6
R H S
2
0
1
0
1
2
1 1 "2 2
i
3
0
0
1
1
2
1 1 2 2
3
2
1
1
0
0
1
“ 2
1 1 “ 2 2
2
O b j.
0
0
0
1
~ 2
1 3
2 "7
~2 + f
ArtObj.
0
0
0
1
1 1
Φ ,
End of phase I Optimum solution-
Optimum objective function value-* —
2
E x a m p l e 6.1 9 Nondegenerate case I n o r d e r t o f u r t h e r c l a r i f y t h e d i f f e r e n c e b e t w e e n a d e g e n e r a t e a n d a n o n d e g e n e r a t e c a s e, c o n s i d e r t h e f o l l o w i n g e x ­
a m p l e a g a i n w i t h t h r e e v a r i a b l e s a n d t h r e e e q u a l i t y c o n s t r a i n t s. I n f a c t, t h e p r o b l e m i s j u s t a s l i g h t m o d i f i c a t i o n o f t h e p r e v i o u s e x a m p l e.
6.5 U n u s u a l S i t u a t i o n s A r i s i n g D u r i n g t h e S i m p l e x S o l u t i o n
f = x l + x 2 + x 3 ;
g = { - x l + « 2 == 1, - x 2 + x 3 == I, x l + x 3 == 3 } ;
S o l v i n g t h i s p r o b l e m u s i n g t h e s i m p l e x m e t h o d, w e s e e t h a t e a c h i t e r a t i o n i n v o l v e s a r e d u c t i o n i n t h e o b j e c t i v e f u n c t i o n v a l u e a n d t h u s, t h e r e i s n o c h a n c e o f c y c l i n g.
B a s i c S i m p l e x [ f, g, { x l, x 2, * 3 },P r i n t L e v e l -* 2 ] ;
problem variables redefined a s: {xl -» xx, x2 -» x2, x3 -* X 3 }
Minimize X! + x 2 + X3
- X l + x2 == l 1 Subject to -x 2 +x 3 == 1 . « ι + *3 == 3 J All variables a 0
********** I n i t i a l simplex tableau **********
New problem variables: {x^, x 2 , X3, a^, a2, a 3 }
Basis 1 2 3 4 5 6 RHS
5
6
Obj . tArtObj .
- 1
0
1
0
1
1
1
- 2
0
0
0
0
1
0
0
0
0
1
0
0
1 3 £
-5 + φ )
Va r i a bl e t o be made b a s i c -* 3 Rat i os: RHS/Column 3 -* ( 00 1 3 )
Variable out of the basic set -» 5
j ------ Iteration l**********
f Basis 1 2 3 4 5 6 RHS
4
3
- 1
0
1
- 1
0
1
1
0
0
1
0
0
1
1
6 1 1 Q 0 - 1 1 2
O b j. 1 2 0 0 - 1 0 - 1 + £
I A r t O b j. 0 ~ 2 0 0 2 0 -3 + Φ
V a r i a b l e t o b e ma d e b a s i c - * 2
Ratios: RHS/Column 2 -* (l 00 2)
V a r i a b l e o u t o f t h e b a s i c s e t -* 4
**********Phase I ------- Iteration 2**********
Basis 1 2 3 4 5 6 RHS
2
3
6
Obj . ArtObj.
- 1 1
- 1 0
2 0
3 0
- 2 0
0
1
0
0
0
1
1
- 1
- 2
2
0
1
- 1
- 1
2
0
0
1
0
0
1
2
1
- 3 + £
- 1 + φ,
C h a p t e r 6 L i n e a r P r o g r a m m i n g
Variable to be made basic 1
Ratios: RHS/Column 1 -» |oo oo —
Variable out of the basic set 6 **********Phase I ----- Iteration 3**********
Basis
1
2
3
4
5 6
R H S '
2
0
1
0
1
2
~ i i
3
2
3
0
0
1
1
Ί
1 1 2 Ί
5
2
1
1
0
0
1 1 2
1
2
O b j.
0
0
0
- i
1 3
2 ~ 2
1
tsJ^KD
+
hh
A r t O b j.
0
0
0
1
1 1
Φ
E n d o f p h a s e I
1
x l - » ^, x 2 4 ^ , x 3 - »
D F
O p t i m u m s o l u t i o n - »
O p t i m u m o b j e c t i v e f u n c t i o n v a l u e - * —
E x a m p l e 6.2 0 D e g e n e r a t e c a s e E x a m p l e 6.1 8 s t a r t e d w i t h a d e g e n e r a t e s o l u ­
t i o n. A s t h e f o l l o w i n g e x a m p l e d e m o n s t r a t e s, a d e g e n e r a t e s o l u t i o n c a n a l s o s h o w u p a t a n y s t a g e. F o r t h i s e x a m p l e, t h e i n i t i a l b a s i c f e a s i b l e s o l u t i o n i s n o t d e g e n e r a t e, b u t t h e s o l u t i o n a t t h e n e x t t w o i t e r a t i o n s i s d e g e n e r a t e. H o w e v e r, t h e d e g e n e r a c y d o e s n o t c a u s e a n y d i f f i c u l t y f o r t h i s p r o b l e m a s w e l l.
T h e e x a m p l e a l s o d e m o n s t r a t e s t h a t d e g e n e r a c y i s r e l a t e d t o t h e s o l u t i o n b e i n g overdetermined. T W o e q u a t i o n s a r e e n o u g h t o s o l v e f o r t w o v a r i a b l e s. T h u s, d e g e n e r a c y w i l l o c c u r w h e n e v e r a b a s i c f e a s i b l e s o l u t i o n i s d e t e r m i n e d b y m o r e c o n s t r a i n t s t h a n t h e n u m b e r o f v a r i a b l e s.
f = - 3 x 1 - 2 x 2;
0 = { x l + x 2 £ 4, 2 x 1 + x 2 5 6, x l S 3 } ;
B a s i c S i m p l e x [ £, g, { x l, x 2 }, P r i n t L e v e l -> 2 ];
Problem variables redefined as: {xl -> X!, x2 -* x 2 } Minimize - 3xi - 2x 2
X l + x2 <; 4 '
Subject to 2xi + x 2 < 6
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _,_ _ _x i s 3_ _,_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
A l l v a r i a b l e s f e 0
* * * * * * * * * * I n i t i a l s i m p l e x t a b l e a u * * * * * * * * * * N e w p r o b l e m v a r i a b l e s: { x i, x 2, s i, s 2, s3 }
6.5 U n u s u a l S i t u a t i o n s A r i s i n g D u r i n g t n e s u n p i t a □ U l U u u u
/Basis 1 2 3 4 5 RHS
3 1 1 1 0 0 4
4 2 1 0 1 0 6 5 1 0 0 0 1 3
Obj .- 3 -2 0 0 0 f i
Va r i a bl e t o be made b a s i c -» 1
Ra t i o s: RHS/Column 1-> ( 4 3 3)
Va r i a bl e out o f t he b a s i c s e t -» 5
**********Phase I I ----- I t e r a t i o n i *********<
L 2 3
B a s x s
4
RHS
0
0
1
0
1
1
0
- 2
1
0
0
0
0
1
0
0
- 1
- 2
1
3
1 0 3
9 + f l
3
4 1
O b j.
V a r i a b l e t o b e ma d e b a s i c - » 2 R a t i o s: RHS/Co l u mn 2 - * ( l 0 oo)
V a r i a b l e o u t o f t h e b a s i c s e t - » 4 * * * * * * * * * * P h a s e I I - - - - - - I t e r a t i o n 2 * * * * * * * * * *
B a s i s
1
2
3 4 5
RHS
3
0
0
1 - 1 1
1
2
0
1
0 1 - 2
0
1
1
0
0 0 1
3
. O b j.
0
0
0 2 - 1
9 + f,
V a r i a b l e t o
b e m a d e
b a s i c - » 5
R a t i o s:
R H S/C o l u m n
5 - * ( l 00 3)
Variable out of the basic set -» 3 **********Phase II ----- Iteration 3 **********
Basis
RHS
5 0 0 1 -1 1 1
2 0 1 2 - 1 0 2 1 1 0 - 1 1 0 2 Obj. 0 0 1 1.0 1 0 + f l
Optimum s o l u t i o n -» { { { x l -» 2, x 2 -> 2 } } } Optimum objective function value-»-10
T h e f i r s t a n d t h e s e c o n d i t e r a t i o n s s h o w d e g e n e r a t e s o l u t i o n s. C o n s i d e r t h e s o l u t i o n a t t h e s e c o n d i t e r a t i o n w i t h x i = 3 a n d x2~ 0. A s t h e g r a p h s h o w n i n F i g u r e 6.7 i l l u s t r a t e s, a t t h i s p o i n t t h e r e a r e t h r e e a c t i v e c o n s t r a i n t s, n a m e l y (£2 = 0, £3 = 0, a n d x 2 = 0 ) · S i n c e t h e p r o b l e m h a s o n l y t w o v a r i a b l e s, o n e
o f t h e c o n d i t i o n s m u s t b e r e d u n d a n t, r e s u l t i n g i n a d e g e n e r a t e c a s e. T h e
o p t i m u m w o u l d c l e a r l y n o t c h a n g e i f, s a y, t h e c o n s t r a i n t #3 i s e l i m i n a t e d f r o m t h e p r o b l e m. F u r t h e r m o r e, f r o m t h e K T c o n d i t i o n s p o i n t o f v i e w, i t i s e a s y t o s e e t h a t t h i s p o i n t i s n o t a r e g u l a r p o i n t.
X2
F I G U R E 6.7 A g r a p h i c a l s o l u t i o n s h o w i n g a d e g e n e r a t e b a s i c f e a s i b l e s o l u t i o n e n ­
c o u n t e r e d d u r i n g a s i m p l e x s o l u t i o n.
6.6 P o s t - O p t i m a l i t y A n a l y s i s
T h e f i n a l s i m p l e x t a b l e a u, i n a d d i t i o n t o t h e o p t i m u m s o l u t i o n, c o n t a i n s i n - f o r m a t i o n t h a t a l l o w s u s t o d e t e r m i n e t h e a c t i v e s t a t u s o f c o n s t r a i n t s, r e c o v e r L a g r a n g e m u l t i p l i e r s, a n d d e t e r m i n e a l l o w a b l e r a n g e s o f c o n s t r a i n t r i g h t - h a n d s i d e s a n d o b j e c t i v e f u n c t i o n c o e f f i c i e n t s f o r w h i c h t h e b a s i c v a r i a b l e s r e m a i n t h e s a m e. T h e p r o c e s s i s k n o w n a s s e n s i t i v i t y o r p o s t - o p t i m a l i t y a n a l y s i s. T h i s s e c t i o n s i m p l y p r e s e n t s t h e p r o c e d u r e s f o r s e n s i t i v i t y a n a l y s i s. T h e d e r i v a t i o n o f t h e s e p r o c e d u r e s i s e a s i e r t o s e e u s i n g t h e r e v i s e d s i m p l e x m e t h o d t h a t i s p r e s e n t e d i n t h e n e x t s e c t i o n.
T h e p o s t - o p t i m a l i t y a n a l y s i s p r o c e d u r e s w i l l b e e x p l a i n e d w i t h r e f e r e n c e t o t h e f o l l o w i n g e x a m p l e:
6.6 P o s t - O p t im a lit y A n a ly s is
M in im iz e / = - f x i + 20x 2 - | x 3 + 6 x 4
( \x\ — 8x2 - *3 + 9*4 > 1 \ \x\ - 12X2 - 5*3 + 3x4 = 3 X3 + X4 < 1 Xi > 0, i = /
S u b j e c t t o
T b s t a r t t h e s i m p l e x s o l u t i o n t h e c o n s t r a i n t s a r e w r i t t e n a s f o l l o w s:
C o n s t r a i n t I: i x i —8x 2—Λ5 + 9Χ4—Χ5+ Χ 7 = 1 X5 - > S u r p l u s χη ->■ A r t i f i c i a l C o n s t r a i n t 2: | x i — 12x 2 — 5 x 3 + 3x 4 + x g = 3 x g —> A r t i f i c i a l C o n s t r a i n t 3: X3 + X4 + x g = 1 x& ->■ S l a c k
A f t e r s e v e r a l i t e r a t i o n s o f t h e s i m p l e x m e t h o d, t h e f o l l o w i n g f i n a l s i m p l e x t a b l e a u i s o b t a i n e d:
^ B a s i s x i x i
X2
1 736
- 33
X3 X4 0 0
X5
X6
X7
*8
R H S ^
28
4
28
80
224
33
11
33
33
33
4
10
4
2
32
33
11
33
33
33
4
1
4
2
1
33
11
33
33
33
47
2
47
73
W + f J
33
11
33
33
X3 X4
\O b j.
0 -s-
U 33
0 ——
U 33
0 i s o
U 33
1
0
0
0
1
0
T h e o p t i m u m s o l u t i o n i s:
B a s i c - x i — x n — — x a — —
fia&ic. X i — 33 , X3 — 33, X4 — 33
N o n b a s i c: X2 = X5 = x& — xj = x b = 0 /*
178 ' 33
6.6.1 S t a t u s o f C o n s t r a i n t s
T h e s t a t u s o f a c t i v e a n d i n a c t i v e c o n s t r a i n t s a t o p t i m u m i s d e t e r m i n e d f r o m t h e v a l u e s o f s u r p l u s/s l a c k v a r i a b l e s. I f a s l a c k/s u r p l u s v a r i a b l e a s s o c i a t e d w i t h a g i v e n c o n s t r a i n t i s z e r o, t h e n o b v i o u s l y t h a t c o n s t r a i n t i s a c t i v e.
F o r t h e e x a m p l e t h e o p t i m u m s o l u t i o n s h o w s t h e f o l l o w i n g v a l u e s f o r s l a c k/s u r p l u s v a r i a b l e s.
X5 = 0, xq = 0
T h i s m e a n s t h a t b o t h c o n s t r a i n t s 1 a n d 3 a r e a c t i v e. O f c o u r s e c o n s t r a i n t 2,
b e i n g a n e q u a l i t y t o b e g i n w i t h, i s a l w a y s a c t i v e.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
6.6.2 R e c o v e r y o f L a g r a n g e M u l t i p l i e r s
T h e L a g r a n g e m u l t i p l i e r s f o r t h e c o n s t r a i n t s c a n b e r e a d d i r e c t l y f r o m t h e o b j e c t i v e f u n c t i o n r o w o f t h e f i n a l s i m p l e x t a b l e a u. F o r L E c o n s t r a i n t s, t h e L a g r a n g e m u l t i p l i e r v a l u e s a r e r e a d f r o m t h e a s s o c i a t e d s l a c k v a r i a b l e c o l u m n. F o r G E o r E Q. c o n s t r a i n t s, t h e y a r e r e a d f r o m t h e a s s o c i a t e d a r t i f i c i a l v a r i a b l e c o l u m n. T h e s i g n o f L a g r a n g e m u l t i p l i e r s f o r G E c o n s t r a i n t s s h o u l d a l w a y s b e n e g a t i v e a n d t h a t f o r L E c o n s t r a i n t s s h o u l d a l w a y s b e p o s i t i v e. F o r e q u a l i t y c o n s t r a i n t s, t h e s i g n o f a L a g r a n g e m u l t i p l i e r h a s n o s p e c i a l s i g n i f i c a n c e.
F o r t h e e x a m p l e p r o b l e m, t h e f i r s t c o n s t r a i n t i s a G E c o n s t r a i n t a n d t h e r e ­
f o r e, i t s L a g r a n g e m u l t i p l i e r, u\ = —4 7/3 3, i s r e a d f r o m t h e o b j e c t i v e f u n c t i o n r o w a n d t h e xj c o l u m n ( b e c a u s e t h i s i s i t s a s s o c i a t e d a r t i f i c i a l v a r i a b l e ). T h e s e c o n d c o n s t r a i n t i s a n E Q. c o n s t r a i n t, a n d i t s L a g r a n g e m u l t i p l i e r, w2 — 7 7/3 3, i s r e a d f r o m χ β c o l u m n. T h e t h i r d c o n s t r a i n t i s a n L E c o n s t r a i n t, a n d i t s L a ­
g r a n g e m u l t i p l i e r, «3 = 2/11, i s r e a d f r o m c o l u m n x g a s s o c i a t e d w i t h i t s s l a c k v a r i a b l e. T h u s, t h e L a g r a n g e m u l t i p l i e r s a r e a s f o l l o w s:
4 7 7 3 2
U s i n g K T c o n d i t i o n s, i t c a n b e s h o w n t h a t t h e L a g r a n g e m u l t i p l i e r o f a G E c o n s t r a i n t i s t h e n e g a t i v e o f t h e s a m e c o n s t r a i n t w r i t t e n i n t h e L E f o r m. T h u s, f o r t h e f i r s t c o n s t r a i n t i n t h e e x a m p l e, w e c a n i n t e r p r e t t h e m u l t i p l i e r s a s f o l l o w s:
F o r a c o n s t r a i n t w r i t t e n a s: \x\ — 8x2 — x3 + 9x4 > 1 t h e L a g r a n g e m u l t i p l i e r = — 55 F o r a c o n s t r a i n t w r i t t e n a s: — | x i + 8x2 + X3 — 9x4 < — 1 t h e L a g r a n g e m u l t i p l i e r = | |
6.6.3 A l l o w a b l e C h a n g e s i n t h e R i g h t - H a n d S i d e s o f C o n s t r a i n t s
I f t h e r i g h t - h a n d s i d e c o n s t a n t o f a c o n s t r a i n t i s c h a n g e d, t h e o p t i m u m s o l u t i o n o b v i o u s l y m a y c h a n g e. H o w e v e r, i t i s p o s s i b l e t o d e t e r m i n e t h e a l l o w a b l e c h a n g e s o f t h e r i g h t - h a n d s i d e s o f c o n s t r a i n t s f o r w h i c h t h e b a s i c v a r i a b l e s e t r e m a i n s t h e s a m e. T h i s m e a n s t h a t t h e s t a t u s o f a c t i v e a n d i n a c t i v e c o n s t r a i n t s w i l l n o t c h a n g e a n d t h e r e f o r e, w e c a n u s e t h e L a g r a n g e m u l t i p l i e r s t o o b t a i n a n e w v a l u e o f t h e o b j e c t i v e f u n c t i o n b a s e d o n t h e s e n s i t i v i t y a n a l y s i s d i s c u s s e d
i n C h a p t e r 4 w i t h t h e K T c o n d i t i o n s.
β.β P o s t - O p t i m a l i t y A n a l y s i s
R e c a l l t h a t t h e d e c i s i o n t o r e m o v e a v a r i a b l e f r o m t h e b a s i c s e t i s b a s e d o n t h e r a t i o s o f r i g h t - h a n d s i d e s a n d t h e c o l u m n c o r r e s p o n d i n g t o t h e v a r i a b l e b e i n g b r o u g h t i n t o t h e b a s i c s e t. T h u s, i n t u i t i v e l y, o n e w o u l d e x p e c t t o s e e t h e a l l o w a b l e c h a n g e s b a s e d o n t h e s e r a t i o s. T h e p r o c e d u r e f o r d e t e r m i n i n g t h e a l l o w a b l e c h a n g e s i s p r e s e n t e d b e l o w.
( a ) C h o i c e o f a n a p p r o p r i a t e c o l u m n F o r d e t e r m i n i n g a l l o w a b l e c h a n g e i n t h e r i g h t - h a n d s i d e o f a c o n s t r a i n t, o n e h a s t o c h o o s e a c o l u m n f r o m t h e c o e f f i c i e n t m a t r i x. F o r L E c o n s t r a i n t s, t h e a p p r o p r i a t e c o l u m n c o r r e s p o n d s t o t h e s l a c k v a r i a b l e a s s o c i a t e d w i t h t h e c o n s t r a i n t. F o r G E a n d E Q. c o n s t r a i n t s, t h e a p p r o p r i a t e c o l u m n c o r r e s p o n d s t o t h e a r t i f i c i a l v a r i a b l e a s s o c i a t e d w i t h t h e c o n s t r a i n t.
( b ) C o m p u t a t i o n o f r a t i o s T h e r a t i o s o f t h e n e g a t i v e o f t h e r i g h t - h a n d s i d e s ( r h s ) a n d c o e f f i c i e n t s i n t h e c h o s e n c o l u m n a r e t h e n c o m p u t e d.
( c ) D e t e r m i n a t i o n o f a l l o w a b l e r a n g e T h e l o w e r l i m i t o f t h e a l l o w a b l e
c h a n g e i s g i v e n b y t h e m a x i m u m o f t h e n e g a t i v e r a t i o s ( n e g a t i v e n u m b e r w i t h t h e s m a l l e s t a b s o l u t e v a l u e ), a n d t h e u p p e r l i m i t i s o b t a i n e d b y t h e m i n i m u m o f t h e p o s i t i v e r a t i o s ( s m a l l e s t p o s i t i v e n u m b e r ).
F o r t h e f i r s t c o n s t r a i n t i n t h e e x a m p l e, t h e a p p r o p r i a t e c o l u m n i s t h e χη c o l u m n, a n d t h e n e g a t i v e o f t h e r a t i o s o f t h e e n t r i e s i n t h i s c o l u m n a n d t h e r h s a r e a s f o l l o w s:
ί 2 2 4/3 3 o 3 2/3 3 _ o 1/3 3 l l
1 - 2 8/3 3 “ ’ _ —4/3 3 _ ,_ 4/3 3 ” ~ 4 ]
D e n o t i n g t h e c h a n g e i n t h e r i g h t - h a n d s i d e o f t h e f i r s t c o n s t r a i n t b y ΔΖ» ι, w e g e t t h e f o l l o w i n g r a n g e:
M a x - j < Abi < M i n [8, 8] o r — ~ < Abi < 8
A d d i n g t h e c u r r e n t v a l u e o f t h e r i g h t - h a n d s i d e t o b o t h t h e u p p e r a n d t h e l o w e r l i m i t, w e c a n e x p r e s s t h e a l l o w a b l e r a n g e o f t h e r i g h t - h a n d o f t h i s c o n s t r a i n t a s f o l l o w s:
1 3 ,
h l < 2 » i < 8 + l o r - < 2?i < 9
4 4
T h i s r e s u l t i m p l i e s t h a t t h e b a s i c v a r i a b l e s a t t h e o p t i m u m p o i n t w i l l r e m a i n
t h e s a m e a s l o n g a s t h e r i g h t - h a n d s i d e o f c o n s t r a i n t 1 i s b e t w e e n 3/4 a n d 9.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
F o r t h e s e c o n d c o n s t r a i n t i n t h e e x a m p l e, t h e a p p r o p r i a t e c o l u m n i s t h e x8 c o l u m n, a n d t h e r a t i o s a r e a s f o l l o w s:
[ 2 2 4/3 3 1 4 3 2/3 3 1/3 3
- t t
= - 1 6,
8 0/3 3 5 ’ 2/3 3 ’ - 2/3 3
D e n o t i n g t h e c h a n g e i n t h e r i g h t - h a n d s i d e o f t h e s e c o n d c o n s t r a i n t b y Δ & 2, w e g e t t h e f o l l o w i n g r a n g e:
H H
1 1 4 1
M a x ] — —, — 1 6 | < Δ&2 S ~ o r < Ab2 < -
2 5 2
A d d i n g t h e c u r r e n t v a l u e o f t h e r i g h t - h a n d s i d e t o b o t h t h e u p p e r a n d t h e l o w e r l i m i t, w e c a n e x p r e s s t h e a l l o w a b l e r a n g e o f t h e r i g h t - h a n d o f t h i s c o n s t r a i n t a s f o l l o w s:
1 4 1 ,1 , 7
- T + 3 < h < i + 3 o r -5 < h < -
F o r t h e t h i r d c o n s t r a i n t i n t h e e x a m p l e, t h e a p p r o p r i a t e c o l u m n i s t h e x$ c o l u m n a n d t h e r a t i o s a r e a s f o l l o w s:
224/33 56 32/33 16 1/33
4/11 3 ’ 10/11 1 5 ’ 1/11
- 4 1
D e n o t i n g t h e c h a n g e i n t h e r i g h t - h a n d s i d e o f t h e t h i r d c o n s t r a i n t b y Abz, we g e t t h e f o l l o w i n g r a n g e:
[ 5 6 16 l l
1
M a x
Ρ ~ Ϊ 5 ’ ~ 3
< A h < oo o r
— < Δ 03 < oo 3
A d d i n g t h e c u r r e n t v a l u e o f t h e r i g h t - h a n d s i d e t o b o t h t h e u p p e r a n d t h e l o w e r l i m i t w e c a n e x p r e s s t h e a l l o w a b l e r a n g e o f t h e r i g h t - h a n d o f t h i s c o n s t r a i n t a s f o l l o w s:
1 2
— - + 1 < &3 < o o + 1 o r - < Z»3 < o o 3 3
6.6.4 U s e o f L a g r a n g e M u l t i p l i e r s t o D e t e r m i n e t h e N e w O p t i m u m
T h e s e n s i t i v i t y a n a l y s i s, p r e s e n t e d w i t h t h e K T c o n d i t i o n s, s h o w s t h a t
β.β P o s t - O p t i m a l i t y A n a l y s i s
T h u s, t f t h e r i g h t - h a n d s i d e o f t h e It h c o n s t r a i n t i s c h a n g e d ( a n d t h e c h a n g e i s w i t h i n t h e a l l o w a b l e l i m i t s ), t h e n e w o p t i m u m v a l u e o f t h e o b j e c t i v e f u n c t i o n c a n b e c o m p u t e d u s i n g i t s L a g r a n g e m u l t i p l i e r a s f o l l o w s:
N e w /* = O r i g i n a l /* - «,· ( N e w — O r i g i n a l bi)
I t i s i m p o r t a n t t o n o t e t h a t t h i s e q u a t i o n g i v e s u s a n e w v a l u e o f t h e o b j e c t i v e f u n c t i o n b u t d o e s n o t g i v e u s v a l u e s o f t h e n e w o p t i m i z a t i o n v a r i a b l e s. M o d i f i e d c o n s t r a i n t e q u a t i o n s c a n b e s o l v e d t o g e t n e w o p t i m i z a t i o n v a r i a b l e s v a l u e s, i f d e s i r e d.
A l s o r e c a l l t h a t t h i s e q u a t i o n i s b a s e d o n t h e a s s u m p t i o n t h a t a l l i n e q u a l i t y c o n s t r a i n t s a r e c o n v e r t e d t o L E f o r m. I n L P p r o b l e m s, w e h a v e t o d e a l w i t h G E c o n s t r a i n t s a s w e l l. T h e r e f o r e, f o r p r o p e r a p p l i c a t i o n o f t h i s e q u a t i o n, i t i s b e s t t o f i r s t c o n v e r t t h e m o d i f i e d a n d t h e o r i g i n a l c o n s t r a i n t s a n d t h e i r m u l t i p l i e r s t o L E f o r m a n d t h e n u s e t h e a b o v e e q u a t i o n.
C o n s i d e r t h e s i t u a t i o n w h e n t h e r i g h t - h a n d s i d e o f t h e f i r s t c o n s t r a i n t i n t h e e x a m p l e i s c h a n g e d t o 5. F r o m t h e p r e v i o u s s e c t i o n, w e k n o w t h a t t h i s c h a n g e i s a l l o w a b l e. T h e s i g n o f t h e L a g r a n g e m u l t i p l i e r d e p e n d s o n h o w w e w r i t e t h e c o n s t r a i n t.
F o r a c o n s t r a i n t w r i t t e n a s: \x\ — 8 x2 — X3 + 9x 4 > 1 t h e L a g r a n g e m u l t i p l i e r = —
F o r a c o n s t r a i n t w r i t t e n a s: —\x\ + 8x 2 + x a — 9x 4 < —1 t h e L a g r a n g e m u l t i p l i e r =
T h e m o d i f i e d c o n s t r a i n t i s w r i t t e n a s f o l l o w s:
1 1
- x i — 8 x 2 — X 3 + 9 x 4 > 5 o r — - x i + 8 x 2 + x 3 — 9 x 4 < — 5 4 4
W i t h t h e c o n s t r a i n t s i n t h e L E f o r m, w e c a n g e t t h e n e w o b j e c t i v e f u n c t i o n v a l u e a s f o l l o w s:
N e w f * = O r i g i n a l / * — «,· ( N e w b j — O r i g i n a l b i )
= _ i Z? - f — ) ( - 5 - ( - D ) = —
3 3 \3 3/ 3 3
A s a n o t h e r e x a m p l e, c o n s i d e r t h e e f f e c t o f c h a n g i n g t h e r i g h t - h a n d s i d e
o f t h e s e c o n d c o n s t r a i n t t o i t s m a x i m u m a l l o w e d v a l u e. F r o m t h e p r e v i o u s
s e c t i o n, w e k n o w t h a t t h e m a x i m u m a l l o w a b l e v a l u e f o r t h e r i g h t - h a n d s i d e o f
t h i s c o n s t r a i n t i s 7/2.
C h a p t e r β L i n e a r P r o g r a m m i n g
O r i g i n a l c o n s t r a i n t 2: \x\ — 12x2 — 5X3 + 3x4 — 3
73
L a g r a n g e m u l t i p l i e r = —
33
M o d i f i e d c o n s t r a i n t 2: |xi — 12x2 — 5x3 + 3x4 — \
N e w f * = O r i g i n a l f * - a,( N e w fr, - O r i g i n a l &,·) = - ψ - (g)(f-3) = - ψ
T h i s n e w o p t i m u m v a l u e o f / c a n b e v e r i f i e d b y r e d o i n g t h e p r o b l e m w i t h t h e m o d i f i e d c o n s t r a i n t.
6.6.5 A l l o w a b l e C h a n g e s i n t h e O b j e c t i v e F u n c t i o n C o e f f i c i e n t s
I f a c o e f f i c i e n t i n t h e o b j e c t i v e f u n c t i o n i s c h a n g e d, w i t h o u t m o d i f y i n g t h e c o n s t r a i n t s, t h e n t h e f e a s i b l e d o m a i n w i l l r e m a i n t h e s a m e. T h e o n l y t h i n g t h a t c h a n g e s i s t h e s l o p e o f t h e o b j e c t i v e f u n c t i o n c o n t o u r s. T h u s, a s l o n g a s t h e c h a n g e s a r e w i t h i n c e r t a i n l i m i t s, t h e o p t i m u m p o i n t w i l l r e m a i n t h e s a m e. F r o m t h e e n t r i e s i n t h e f i n a l s i m p l e x t a b l e a u, i t i s p o s s i b l e t o d e f i n e t h e a l l o w a b l e c h a n g e s i n t h e o b j e c t i v e f u n c t i o n c o e f f i c i e n t s f o r w h i c h t h e b a s i c s o l u t i o n w i l l n o t c h a n g e. T h e p r o c e d u r e f o r d e t e r m i n i n g t h e a l l o w a b l e c h a n g e s i s p r e s e n t e d b e l o w.
F o r C h a n g e s i n C o e f f i c i e n t s o f V a r i a b l e s i n t h e N o n b a s i c S e t
F o r c h a n g e s i n c o e f f i c i e n t s o f n o n b a s i c v a r i a b l e s, t h e l o w e r l i m i t f o r t h e a l - l o w a b l e c h a n g e i s n e g a t i v e o f t h e e n t r y i n t h e o b j e c t i v e f u n c t i o n r o w a n d t h e c o l u m n t h a t c o r r e s p o n d s t o t h e v a r i a b l e. T h e u p p e r l i m i t f o r c o e f f i c i e n t s o f a l l n o n b a s i c v a r i a b l e s i s 00.
T h e v a r i a b l e X2 i s a n o n b a s i c v a r i a b l e f o r t h e e x a m p l e p r o b l e m. D e n o t i n g t h e c h a n g e b y Acz, t h e a l l o w a b l e c h a n g e i n t h e c o e f f i c i e n t o f xz i s g i v e n b y
160
< AC2 < 00
33
A d d i n g t h e c u r r e n t v a l u e o f t h e c o e f f i c i e n t t o b o t h t h e u p p e r a n d t h e l o w e r l i m i t, w e c a n e x p r e s s t h e a l l o w a b l e r a n g e o f t h e c o e f f i c i e n t x2 a s f o l l o w s:
β.β P o s t - O p t i m a l i t y A n a l y s i s
T hu s, t h e b a s i c v a r i a b l e s a n d t h e i r o p t i m u m v a l u e s w i l l r e m a i n t h e s a m e r e ­
g a r d l e s s o f h o w m u c h t h e c o e f f i c i e n t o f x2 i n t h e o b j e c t i v e f u n c t i o n i s i n c r e a s e d. H oweve r, i t o n l y g o e s d o w n t o 15.15 w i t h o u t a f f e c t i n g t h e o p t i m u m s o l u t i o n.
i b r C h a n g e s i n C o e f f i c i e n t s o f V a r i a b l e s i n t h e B a s i c S e t
F o r c h a n g e s i n c o e f f i c i e n t s o f b a s i c v a r i a b l e s, t h e a l l o w a b l e r a n g e i s d e t e r m i n e d b y t a k i n g t h e r a t i o s o f e n t r i e s i n t h e o b j e c t i v e f u n c t i o n r o w a n d t h e c o n s t r a i n t r o w f o r w h i c h t h e b a s i c v a r i a b l e i s t h e s a m e a s t h e v a r i a b l e w h o s e c o e f f i c i e n t i s b e i n g c h a n g e d. T h e e n t r i e s i n t h e o b j e c t i v e f u n c t i o n r o w c o r r e s p o n d i n g t o b a s i c a n d a r t i f i c i a l v a r i a b l e s a r e n o t c o n s i d e r e d i n c o m p u t i n g t h e r a t i o s. T h e l o w e r l i m i t o f t h e a l l o w a b l e c h a n g e i s g i v e n b y t h e m a x i m u m o f t h e n e g a t i v e r a t i o s ( t h e n e g a t i v e n u m b e r w i t h t h e s m a l l e s t a b s o l u t e v a l u e ) a n d t h e u p p e r l i m i t i s o b t a i n e d b y t h e m i n i m u m o f t h e p o s i t i v e r a t i o s ( t h e s m a l l e s t p o s i t i v e n u m b e r ).
T h e a l l o w a b l e c h a n g e i n t h e c o e f f i c i e n t o f v a r i a b l e x\ i n t h e e x a m p l e p r o b - l e m i s d e t e r m i n e d f r o m t h e r a t i o s o f t h e o b j e c t i v e f u n c t i o n r o w a n d t h e f i r s t
c o n s t r a i n t r o w ( b e c a u s e x\ i s t h e b a s i c v a r i a b l e a s s o c i a t e d w i t h t h i s c o n s t r a i n t ). T h e r a t i o s a r e a s f o l l o w s:
160/33 5 47/33
- 7 3 6/3 3 2 3 ’ 28/33
_ 47 2/11 _ l l
“ 2 8 ’ 4 7 Ϊ Ϊ ” 2 j
D e n o t i n g t h e c h a n g e b y Ac\, t h e a l l o w a b l e c h a n g e i n t h e c o e f f i c i e n t o f x\ i s g i v e n b y
5 . Γ 4 7 1 Ί 5 1
< Ac\ < M m —, - o r < < -
23 “ |_28 2 J 23 “ 2
A d d i n g t h e c u r r e n t v a l u e o f t h e c o e f f i c i e n t t o b o t h t h e u p p e r a n d t h e l o w e r l i m i t, w e c a n e x p r e s s t h e a l l o w a b l e r a n g e o f t h e c o e f f i c i e n t x\ a s f o l l o w s:
5 3 1 3 89 1
---------- < d < -- o r ---------- < C] < — -
23 4 “ 2 4 92 4
T h e a l l o w a b l e c h a n g e s i n t h e c o e f f i c i e n t o f v a r i a b l e *3 i n t h e e x a m p l e p r o b l e m a r e d e t e r m i n e d f r o m t h e r a t i o s o f t h e o b j e c t i v e f u n c t i o n r o w a n d t h e s e c o n d c o n s t r a i n t r o w ( b e c a u s e X3 i s t h e b a s i c v a r i a b l e a s s o c i a t e d w i t h t h i s c o n s t r a i n t ). T h e r a t i o s a r e a s f o l l o w s:
160/33 _ 47/33 47 2/11
£\} y
8/3 3 4/3 3 4 1 0/1 1
C h a p t e r 6 L i n e a r P r o g r a m m i n g
D e n o t i n g t h e c h a n g e b y AC3, t h e a l l o w a b l e c h a n g e i n t h e X3 c o e f f i c i e n t i n t h e o b j e c t i v e f u n c t i o n i s g i v e n b y
4 7 1 20’ T'5
- o o < Acs < M i n
o r —00 < AC3 < -
5
A d d i n g t h e c u r r e n t v a l u e o f t h e c o e f f i c i e n t t o b o t h t h e u p p e r a n d t h e l o w e r l i m i t, w e c a n e x p r e s s t h e a l l o w a b l e r a n g e o f t h e c o e f f i c i e n t X3 a s f o l l o w s:
1 1 1 3
F i n a l l y, t h e a l l o w a b l e c h a n g e s i n t h e c o e f f i c i e n t o f v a r i a b l e X4 i n t h e e x a m p l e p r o b l e m a r e d e t e r m i n e d f r o m t h e r a t i o s o f t h e o b j e c t i v e f u n c t i o n r o w a n d t h e t h i r d c o n s t r a i n t r o w ( b e c a u s e X4 i s t h e b a s i c v a r i a b l e a s s o c i a t e d w i t h t h i s c o n s t r a i n t ). T h e r a t i o s a r e a s f o l l o w s:
ί 1 6 0/3 3 o 4 7/3 3 4 7 2/1 1 1
{ - 8/3 3 “ ~~ ° ’ - 4/3 3 “ 4"’ 1/1 1 “ \
D e n o t i n g t h e c h a n g e b y Ac4l t h e a l l o w a b l e c h a n g e i n t h e x4 c o e f f i c i e n t i n t h e o b j e c t i v e f u n c t i o n i s g i v e n b y
M a x I — —2 0 I < Ac4 < 2 o r “ I f - ^ C4 - 2
A d d i n g t h e c u r r e n t v a l u e o f t h e c o e f f i c i e n t t o b o t h t h e u p p e r a n d t h e l o w e r l i m i t, w e c a n e x p r e s s t h e a l l o w a b l e r a n g e o f t h e c o e f f i c i e n t x4 a s f o l l o w s:
4 7 2 3
h 6 < C 4 < 2 + 6 o r < c4 < 8
4 4
C o n s i d e r a s p e c i f i c s i t u a t i o n w h e n t h e o b j e c t i v e f u n c t i o n i s m o d i f i e d a s f o l l o w s:
M o d i f i e d / = — | x i + 1 7 x 2 — | x3 — 5 x 4 O r i g i n a l / = - f x i + 2 0 x 2 - 5X3 + 6 x 4
T h e c o e f f i c i e n t s o f xi a n d X4 a r e c h a n g e d i n t h e m o d i f i e d /. B o t h c h a n g e s a r e w i t h i n t h e a l l o w a b l e r a n g e c o m p u t e d a b o v e. T h u s, t h e o p t i m u m v a r i a b l e
v a l u e s r e m a i n t h e s a m e a s t h a t o f t h e o r i g i n a l p r o b l e m.
R a s i r ■ η = 32 _ χ
O p t i m u m v a r i a b l e v a l u e s: 33 ’ 33 ’ 33
N o n b a s i c : X2 = X5 = x& = xj = x 8 = 0
β.β P o s t - O p t i m a l i t y A n a l y s i s
S u b s t i t u t i n g t h e s e v a l u e s i n t o t h e m o d i f i e d o b j e c t i v e f u n c t i o n, w e g e t N e w f * = — 1 |. T h i s s o l u t i o n c a n b e v e r i f i e d b y r e d o i n g t h e p r o b l e m w i t h t h e m o d i f i e d o b j e c t i v e f u n c t i o n.
6.6.6 T h e S e n s i t i v i t y A n a l y s i s O p t i o n o f t h e B a s i c S i m p l e x F u n c t i o n
T h e S e n s i t i v i t y A n a l y s i s o p t i o n o f t h e B a s i c S i m p l e x f u n c t i o n r e t u r n s L a g r a n g e m u l t i p l i e r s a n d a l l o w a b l e r a n g e s f o r r i g h t - h a n d s i d e s a n d c o s t c o e f f i c i e n t s.
O p t i o n s [ B a s i c S i m p l e x ]
{UnrestrictedVariables-» {} , Maxlterations-» 10, ProblemType-»Min, SimplexVariables -» {x, s, a} , PrintLevel -» 1, SensitivityAnalysis -» False}
? S e n s i t i v i t y A n a l y s i s
SensitivityAnalysis i s an option of simplex method. I t controls whether a post-optimality ( sensitivity) analysis i s performed after obtaining an optimum solution. Default i s False.
E x a m p l e 6.2 1
f = 2x1 + x2 - x3;
g - {xl + 2x2 + x3 s B, -xl + x 2 ~ 2x3 2 4};
BasicSimplex[f, g, {xl, x2, x3>, SensitivityAnalysis -» True] ;
Problem variables redefined a s: {xl -» xx, x2 -» x 2 , x3 -» x 3 } Minimize 2 x ± + x 2 - Χ3
X 1 + ^ x 2 + x 3 s ®
^ - X i + x 2 - 2 x3 s 4 1
Al l var i abl e s.z 0
Subject to
********** I n i t i a l simplex tableau ********** New problem variables: {χχ , x 2 , x3 , s x , s 2 }
(Basis 1 2 3 4 5 RHS^
4 1 2 1 1 0 8
5 - 1 1 - 2 0 1 4
O b j. 2 1 - 1 0 0 f
********** Final simplex tableau **********
Basis
1
2
3
4
5
RHS '
3
1
2
1
1
0
8
5
1
5
0
2
1
20
Obj .
3
3
0
1
0
8 + fj
C h a p t e r 6 L i n e a r P r o g r a m m i n g
Optimum solution {{{xl -»0,x2-»0,x3-+8}}}
Opt i mum o b j e c t i v e f u n c t i o n v a l u e - 8 Lagrange multipliers-» {1, 0}
Allowable constraint RHS changes and ranges - 8 s Ah^ ΐ oo 0 < ^ < oo
-20 < Ab 2 ^ 00-----16 s, b 2 £ °°
Allowable cost coeffi cient changes and ranges
-3 ί <oo -1 ί Ci im -3 s Ac 2 ί oo -2 ί c 2 ί ω -oo s Ac 3 s 1 -® ί c 3 £ 0
T h e L a g r a n g e m u l t i p l i e r s t e l l t h a t t h e f i r s t c o n s t r a i n t i s a c t i v e w h i l e t h e s e c o n d i s i n a c t i v e.
A s a n e x a m p l e o f t h e u s e o f L a g r a n g e m u l t i p l i e r s i n s e n s i t i v i t y a n a l y s i s, a s s u m e t h a t t h e r h s o f t h e f i r s t c o n s t r a i n t i s c h a n g e d t o 5. T h e c h a n g e i s ( 5 — 8) = — 3. S i n c e t h e c h a n g e i s a l l o w a b l e, f r o m t h e L a g r a n g e t h e o r e m, t h e n e w m i n i m u m i s
/ = - 8 - 1 x ( - 3 ) = - 5.
R e - s o l v i n g t h e p r o b l e m w i t h m o d i f i e d c o n s t r a i n t s c o n f i r m s t h i s s o l u t i o n.
C o n s t r a i n e < 3 M i n [ 2 x l + x 2 - x 3, { x l + 2 x 2 + x 3 ύ 5, - x l + x 2 - 2 x 3 £ 4 ), { x l, x 2, x 3 } ]
{-5, {xl-»0,x2-»0,x3-»5}}
T h e a l l o w a b l e r a n g e f o r t h e c h a n g e i n t h e c o e f f i c i e n t o f x2 i n t h e o b j e c t i v e f u n c t i o n i s { —3, cx»}. T h u s, t h e s o l u t i o n w i l l r e m a i n t h e s a m e i f w e i n c r e a s e t h e c o e f f i c i e n t o f x2 i n / t o a n y v a l u e b u t w i l l c h a n g e i f t h e c o e f f i c i e n t i s m a d e s m a l l e r t h a n —2. T h i s i s c o n f i r m e d b y r e - s o l v i n g t h e p r o b l e m a s f o l l o w s.
A s d e m o n s t r a t e d b y t h e f o l l o w i n g s o l u t i o n, i f w e m a k e t h e c o e f f i c i e n t ofx2 v e r y l a r g e, s a y 6 0 0, s t i l l t h e o p t i m u m v a l u e s o f v a r i a b l e s r e m a i n t h e s a m e.
C o n s t r a i n e d M i n [ 2 x 1 + 6 0 0 x 2 - x 3, { x l + 2 x 2 + x 3 s 8, - x l + x 2 - 2 x 3 ύ 4 >, { x l, x 2, x 3 } ]
{- 8, {xl -» 0, x2 -» 0, x3 -» 8 }}
H o w e v e r, r e d u c i n g t h e c o e f f i c i e n t o f x2 t o j u s t o u t s i d e o f t h e r a n g e, s a y m a k i n g i t —2.1, c h a n g e s t h e s o l u t i o n d r a s t i c a l l y.
C o n s t r a i n e d M i n [ 2 x 1 - 2.1 x 2 - x 3, { x l + 2 x 2 + x 3 5 8, - x l + x 2 - 2x 3 s 4 }, {xl, x 2, x 3 > ]
{ - 8.4, { x l - » 0,x 2 - > 4.,x 3 - » 0 } }
6.7 T h e R e v i s e d S i m p l e x M e t h o d
T h e t a b l e a u f o r m o f t h e s i m p l e x m e t h o d c o n s i d e r e d s o f a r i s c o n v e n i e n t f o r h a n d c a l c u l a t i o n s b u t i s i n e f f i c i e n t f o r c o m p u t e r i m p l e m e n t a t i o n. T h e m e t h o d c a n a l t e r n a t i v e l y b e p r e s e n t e d i n a m a t r i x f o r m t h a t l e n d s i t s e l f t o e f f i c i e n t n u m e r i c a l i m p l e m e n t a t i o n. I t a l s o g i v e s a c l e a r e r p i c t u r e o f t h e c o m p u t a t i o n s i n v o l v e d a n d m a k e s i t s t r a i g h t f o r w a r d t o d e r i v e s e n s i t i v i t y a n a l y s i s p r o c e d u r e s.
6.7.1 T h e M a t r i x F o r m o f t h e S i m p l e x M e t h o d
\« m l «m2 · · · «mw/
b = [bi, bz, · · ·, bm]T > 0 v e c t o r o f r i g h t - h a n d s i d e s o f c o n s t r a i n t s
I d e n t i f y i n g c o l u m n s o f t h e c o n s t r a i n t c o e f f i c i e n t m a t r i x t h a t m u l t i p l y a g i v e n o p t i m i z a t i o n v a r i a b l e, w e w r i t e t h e c o n s t r a i n t e q u a t i o n s a s f o l l o w s:
w h e r e A,· i s t h e It h c o l u m n o f t h e A m a t r i x. F o r a b a s i c f e a s i b l e s o l u t i o n, d e n o t i n g t h e v e c t o r o f m b a s i c v a r i a b l e s b y x b a n d t h e (n—m) v e c t o r o f n o n b a s i c v a r i a b l e s b y xn, t h e p r o b l e m c a n b e w r i t t e n i n t h e p a r t i t i o n e d f o r m a s f o l l o w s:
C o n s i d e r a n L P p r o b l e m e x p r e s s e d i n t h e s t a n d a r d f o r m a s
F i n d x i n o r d e r t o M i n i m i z e /( x ) = c r x s u b j e c t t o A x = b a n d x > 0 w h e r e
x = [jci , X2,..., Xn]T v e c t o r o f o p t i m i z a t i o n v a r i a b l e s c = [ q, C2,..., c „ ] v e c t o r o f c o s t c o e f f i c i e n t s
/ « 1 1 « 1 2 ^
(*21 « 2 2 · · · «2 h
A =
m x n m a t r i x o f c o n s t r a i n t c o e f f i c i e n t s
A i * i + Α2Χ2 +. ·. + AnXf, = b
( B
w h e r e mx m m a t r i x B c o n s i s t s o f t h o s e c o l u m n s o f m a t r i x A t h a t c o r r e s p o n d t o t h e b a s i c v a r i a b l e s a n d m x (n — m) m a t r i x N c o n s i s t s o f t h o s e t h a t c o r ­
r e s p o n d t o n o n b a s i c v a r i a b l e s. S i m i l a r l y, v e c t o r cb c o n t a i n s c o s t c o e f f i c i e n t s
C h a p t e r β L i n e a r P r o g r a m m i n g
c o r r e s p o n d i n g t o b a s i c v a r i a b l e s, a n d t h o s e t h a t c o r r e s p o n d t o n o n b a s i c v a r i a b l e s.
T h e g e n e r a l s o l u t i o n o f t h e c o n s t r a i n t e q u a t i o n s c a n n o w b e w r i t t e n a s f o l l o w s:
w h e r e wT = c j B"1 a n d t t — cjj — c | b _ 1 N = cJj — wt N.
T h e v e c t o r w i s r e f e r r e d to a s t h e simplex multipliers v e c to r. A s w i l l b e p o in te d o u t la ter, t h e s e m u lt ip lie r s a r e r e la t e d to t h e L a g r a n g e m u lt ip lie r s.
F r o m t h e s e g e n e r a l e x p r e s s io n s, t h e b a s ic f e a s ib le s o l u t i o n fo r t h e p r o b le m i s o b t a in e d b y s e t t i n g x w = 0 a n d t h u s
A ls o f r o m t h e o b j e c tiv e f u n c t i o n e x p r e s s io n, it i s c le a r t h a t i f a n e w v a r ia b le is b r o u g h t in t o t h e b a s ic s e t, t h e c h a n g e i n t h e o b j e c tiv e f u n c t i o n w i l l b e p r o p o r t io n a l to r. T h u s, t h i s t e r m r e p r e s e n t s t h e r e d u c e d o b j e c tiv e f u n c t io n r o w o f t h e s i m p l e x ta b le a u.
U s in g t h i s n e w n o t a t io n, t h e p r o b le m c a n b e e x p r e s s e d i n t h e f o llo w in g fo r m:
B x b + N x n = b
g i v in g x b = B !b — B 1Nxjst
S u b s t it u t in g t h i s in t o t h e o b j e c tiv e f u n c t io n, w e g e t
/ = c j (B xb — Β * Ν Χ ν ) + c J x j v = c j B * b + ( c j - c J b xn ) xn
or
f - w 7 b + rTxN
a n d a b a s ic s o l u t i o n c a n b e w r i t t e n a s fo llo w s:
/B a s i c N o n b a s ic RHS
I Β 1 Ν Β"1 b
\ 0 r T · / - w Tb.
T h i s f o r m g i v e s u s a w a y to w r i t e t h e s i m p l e x t a b l e a u c o r r e s p o n d i n g to a n y b a s i c s o l u t i o n b y s i m p l y p e r f o r m i n g a s e r i e s o f m a t r i x o p e r a t i o n s.
E x a m p l e 6.2 2 C o n s i d e r t h e LP p r o b l e m c o n s i d e r e d i n e x a m p l e 6.12. M i n i m i z e / — — 3/4x i + 20^2 — 1/2 x 3 + 6*4 / 1 /4x i - 8x2 - X3 + 9x4 < 1 \
l/2x i - 12x2 - 1/2 x 3 + 3x 4 < 3 * 3 + * 4 < 1
Sub je ct to
Xi > 0, i = 1,
I n t r o d u c i n g s l a c k v a r i a b l e s, w e h a v e t h e s ta n d a rd LP f o r m as fol l ow s: M i n i m i z e / — —3/4x i + 20x 2 — 1/2 x 3 + 6*4
/ l/4x i — 8x2 — X3 + 9x4 + X5 = 1 \
l/2x i - 12X2 - 1/2 x 3 + 3x4 + * 6 = 3 *3 + X 4 + x? — 1
v x i > 0,i = l..........7
T h e p r o b l e m i n t h e m a t r i x f o r m i s a s f o l l o w s:
S u b j e c t t o
/
c =
3 1 ]
- -,2 0, - ^,6, 0,0,0 ί
- 8 - 1 i - 1 2 - I 0 1
9 1 0 O'
3 0 1 0
1 0 0 r
b =
U s i n g t h e m a t r i x f o r m, w e c a n w r i t e t h e s i m p l e x t a b l e a u c o r r e s p o n d i n g t o a n y s e t o f b a s i c a n d n o n - b a s i c v a r i a b l e s. A s a n e x a m p l e, c o n s i d e r w r i t i n g t h e s i m p l e x t a b l e a u c o r r e s p o n d i n g t o { x 1.x 2.x 7 } a s b a s i c, a n d { X3, X4, X5, Χβ } a s n o n b a s i c v a r i a b l e s.
T h e p a r t i t i o n e d m a t r i c e s c o r r e s p o n d i n g t o s e l e c t e d b a s i c a n d n o n b a s i c v a r i ­
a b l e s a r e a s f o l l o w s:
c b =
B =
c n
= ( 4,6,0,° )
T h e c o r r e s p o n d i n g b a s i c s o l u t i o n t a b l e a u c a n b e w r i t t e n b y s i m p l y p e r f o r m i n g t h e r e q u i r e d m a t r i x o p e r a t i o n s a s f o l l o w s:
C h a p t e r 6 l i n e a r P r o g r a m m i n g
B - 1 =
'- 1 2 8 0 >
- 3 Ϊ 0 0 0 1,
B _ 1N =
B - 1b = I i
w ‘ = c i B 1 = { - 1,—1,0} w r b
„T
r * = c j — w TN
{ - 2,18,1,1}
T h e s i m p l e x t a b l e a u c a n n o w b e w r i t t e n b y s i m p l y p l a c i n g t h e s e v a l u e s i n t h e i r a p p r o p r i a t e p l a c e s.
'B a s i c
N o n b a s i c
RHS
I
Β - 1 N
B ~ * b
0
T
/ - w T b,
/B a s i s x\ x z x? - Χ3
x\ 1 0 0 8
X 2 0 1 0 |
x z 0 0 1 1
V O b j. 0 0 0 - 2
X4
—8 4 _ _ 15 4
1
1 8
X 5 X 6
— 1 2 8 __i
2
0 1
1
4
0
R H S \
1 2 1 4 1
1 4 + /7
E x c e p t f o r a s l i g h t r e a r r a n g e m e n t o f c o l u m n s, t h i s t a b l e a u i s e x a c t l y t h e s a m e a s t h e s e c o n d t a b l e a u g i v e n i n e x a m p l e 6.1 2.
t
U s i n g t h e m a t r i x f o r m, i t i s p o s s i b l e t o p r e s e n t a n a l g o r i t h m f o r s o l v i n g a n L P p r o b l e m w i t h o u t n e e d f o r w r i t i n g c o m p l e t e t a b l e a u s.
I n o r d e T t o d e c i d e w h i c h v a r i a b l e t o b r i n g i n t o t h e b a s i c s e t, w e n e e d t o c o m p u t e t h e r e d u c e d o b j e c t i v e f u n c t i o n c o e f f i c i e n t g i v e n b y v e c t o r r. I n t h e c o m p o n e n t f o r m, w e h a v e
r j ~ c j — w A j j € n o n b a s i c
w h e T e Aj i s t h e ;,t h c o l u m n o f m a t r i x A, a n d w T = c j B'1.
I f a l l o f t h e s e r e d u c e d c o s t c o e f f i c i e n t s a r e p o s i t i v e, w e h a v e r e a c h e d t h e o p t i m u m. O t h e r w i s e, w e c a n b r i n g t h e v a r i a b l e c o r r e s p o n d i n g t o t h e l a r g e s t n e g a t i v e c o e f f i c i e n t i n t o t h e b a s i c s e t. A s p o i n t e d o u t b e f o r e, t h e s e l e c t i o n o f l a r g e s t n e g a t i v e c o e f f i c i e n t c o u l d p o s s i b l y c a u s e c y c l i n g i n c a s e o f d e g e n e r a t e p r o b l e m s. A l t e r n a t i v e l y, f o l l o w i n g B l a n d's r u l e, w e c a n s e l e c t t h e v a r i a b l e
6.7 T h e R e v i s e d S i m p l e x M e t h o d
c o r r e s p o n d i n g t o t h e f i r s t n e g a t i v e rj v a l u e. T h i s r u l e i s a d o p t e d h e r e. F o r t h e f o l l o w i n g d i s c u s s i o n, w e w i l l d e n o t e t h e i n d e x o f t h e n e w b a s i c v a r i a b l e b y p.
T h e n e x t k e y s t e p i n t h e s i m p l e x s o l u t i o n i s t o d e c i d e w h i c h v a r i a b l e t o r e m o v e f r o m t h e b a s i c s e t. T h i s i s d o n e b y t a k i n g t h e r a t i o s o f t h e r h s a n d t h e pth c o l u m n ( u s i n g o n l y t h o s e e n t r i e s t h a t a r e g r e a t e r t h a n 0) o f t h e c o e f f i c i e n t m a t r i x. I n o r d e r t o p e r f o r m t h i s c o m p u t a t i o n, w e d o n't n e e d t h e e n t i r e r e d u c e d s u b m a t r i x B _ 1N. W e s i m p l y n e e d t h e p th c o l u m n o f t h i s m a t r i x, w h i c h c a n b e g e n e r a t e d b y Β~τΑ ρ. T h i s i s w h e r e c o m p u t a t i o n a l l y, t h e r e v i s e d p r o c e d u r e i s m o r e e f f i c i e n t. I n s t e a d o f p e r f o r m i n g a r e d u c t i o n o n t h e e n t i r e m a t r i x N, w e a r e j u s t c o m p u t i n g t h e c o l u m n t h a t i s n e e d e d. U s i n g t h e t e r m i n o l o g y i n t r o d u c e d i n t h e l a s t c h a p t e r f o r n u m e r i c a l o p t i m i z a t i o n m e t h o d s, w e c a n o r g a n i z e t h i s c o m p u t a t i o n a s a d e s c e n t d i r e c t i o n a n d s t e p - l e n g t h c a l c u l a t i o n a s f o l l o w s:
D i r e c t i o n o f d e s c e n t, d = — B- 1 A p
C u r r e n t b a s i c s o l u t i o n, xg = B - 1 b
S t e p l e n g t h, a = M i n f — di < 0, i e b a s i c }
U p d a t e v a l u e s, xp = a— x,· = x,· + adj, i e b a s i c --------------------------------------------------------------------
S i n c e a i s s e t t o t h e m i n i m u m r a t i o, o n e o f t h e c u r r e n t b a s i c v a r i a b l e s w i l l g o
t o z e r o, w h i c h i s t h e n r e m o v e d f r o m t h e b a s i c s e t. A l s o n o t e t h a t i f d > 0, t h e
p r o b l e m i s u n b o u n d e d.
T h e c o m p l e t e r e v i s e d s i m p l e x a l g o r i t h m i s s u m m a r i z e d i n t h e f o l l o w i n g s t e p s:
1. S t a r t w i t h a b a s i c f e a s i b l e s o l u t i o n. I f n e c e s s a r y, t h e s t a r t i n g b a s i c f e a s i b l e s o l u t i o n i s f o u n d u s i n g t h e P h a s e I p r o c e d u r e o f t h e s i m p l e x m e t h o d. R e c a l l t h a t t h e o b j e c t i v e f u n c t i o n f o r t h i s p h a s e i s t h e s u m o f a r t i f i c i a l v a r i a b l e s. T h e f o l l o w i n g v e c t o r s a n d m a t r i c e s a r e k n o w n c o r r e s p o n d i n g t o t h e c u r r e n t b a s i c f e a s i b l e s o l u t i o n.
basic = V e c t o r o f i n d i c e s o f c u r r e n t b a s i c v a r i a b l e s nonBasic = V e c t o r o f i n d i c e s o f c u r r e n t n o n b a s i c v a r i a b l e s x = c u r r e n t s o l u t i o n v e c t o r A = C o m p l e t e c o n s t r a i n t c o e f f i c i e n t m a t r i x c = c o m p l e t e v e c t o r o f c o s t c o e f f i c i e n t s b = v e c t o r o f r h s o f c o n s t r a i n t s
2. F o r m m x m m a t r i x B a n d m x 1 v e c t o r cj?, c o n s i s t i n g o f c o l u m n s o f c o n s t r a i n t c o e f f i c i e n t m a t r i x a n d c o s t c o e f f i c i e n t s c o r r e s p o n d i n g t o c u r r e n t b a s i c
C h a p t e r β L i n e a r P r o g r a m m i n g
v a r i a b l e s. T h a t i s,
■ - [ A j ] a n d c b = [ c, ] u s i n g a l l j e b a s i c
3. C o m p u t e t h e m u l t i p l i e r s w T = c | B - 1.
T h e c o m p u t a t i o n i n v o l v e s t h e i n v e r s e o f m a t r i x B. H o w e v e r, i n s t e a d o f a c t u a l l y i n v e r t i n g m a t r i x B r i t i s m o r e e f f i c i e n t t o o b t a i n w b y s o l v i n g a s y s t e m o f e q u a t i o n s o b t a i n e d a s f o l l o w s:
w T = C g B -1 = Φ · w TB = ο ^ Β ^ Β = = > · B r w — c b
I n o r d e r t o s o l v e e q u a t i o n s B Tw = c b, t h e m a t r i x B i s f i r s t d e c o m p o s e d i n t o l o w e T ( L ) a n d u p p e r t r i a n g u l a r ( U ) m a t r i c e s ( B — L U ). T h e s y s t e m o f e q u a t i o n s i s t h e n U TL Tw = c b. T h e s o l u t i o n i s o b t a i n e d i n t w o s t e p s. F i r s t, a s e t o f i n t e r m e d i a t e v a l u e s ( y ) a r e c o m p u t e d f r o m U Ty = c b b y f o r w a r d s o l u t i o n. F i n a l l y, v e c t o r w i s c o m p u t e d f r o m L Tw = y b y b a c k - s u b s t i t u t i o n.
4. C o m p u t e t h e r e d u c e d c o s t c o e f f i c i e n t s.
rj = cj — w TAj j € n o n b a s i c, w h e r e A j i s t h e 7t h c o l u m n o f m a t r i x A.
5. C h e c k f o r o p t i m a l i t y.
I f rj > 0 f o r a l l j e n o n b a s i c, t h e n s t o p. T h e c u r r e n t b a s i c f e a s i b l e s o l u t i o n i s o p t i m u m.
6. C h o o s e a v a r i a b l e t o b r i n g i n t o t h e b a s i c s e t.
S e l e c t t h e l o w e s t i n d e x p € n o n b a s i c s u c h t h a t rp < 0 ( f o l l o w i n g B l a n d's r u l e t o a v o i d c y c l i n g i n d e g e n e r a t e c a s e s ).
7. C o m p u t e m o v e d i r e c t i o n
T h i s s t e p c o r r e s p o n d s t o e l i m i n a t i n g b a s i c v a r i a b l e s f r o m t h e c o l u m n c o r r e s p o n d i n g t o a n e w b a s i c v a r i a b l e. I n t h e b a s i c s i m p l e x m e t h o d, t h i s s t e p i s c a r r i e d o u t b y d e f i n i n g t h e p i v o t r o w a n d c a r r y i n g o u t r o w o p e r a t i o n s. H e r e i t i s a c c o m p l i s h e d b y s o l v i n g t h e s y s t e m o f e q u a t i o n s B d = — A p, w h e r e A p i s t h e c o l u m n c o r r e s p o n d i n g t o t h e n e w b a s i c v a r i a b l e. S i n c e t h e B m a t r i x i s a l r e a d y f a c t o r e d, w e s i m p l y n e e d t o s o l v e L U d = —A p. T h u s, t h i s s t e p r e q u i r e s a f o r w a r d s o l u t i o n t o g e t t h e i n t e r m e d i a t e s o l u t i o n f r o m L y = —A p a n d t h e n a b a c k - s u b s t i t u t i o n t o g e t d f r o m U d = y.
8. C h e c k f o r u n b o u n d e d n e s s.
I f d > 0, t h e p r o b l e m i s u n b o u n d e d. S t o p.
6.7 T h e R e v i s e d S i m p l e x M e t h o d
9. C o m p u t e s t e p l e n g t h a n d c h o o s e a v a r i a b l e t o b e m a d e n o n b a s i c.
C o m p u t e t h e n e g a t i v e o f ra t io s o f c u r r e n t v a r i a b l e v a l u e s and t h e d i r e c ­
t i o n vector.
a = M i n I — %, di < 0, i € b a s i c ί d j
T h e s t e p l e n g t h a i s t h e m i n i m u m o f t h e s e r a t i o s. T h e i n d e x o f t h e m i n i m u m r a t i o c o r r e s p o n d s t o t h e v a r i a b l e t h a t i s t o g o o u t o f t h e b a s i s. D e n o t e t h i s i n d e x b y q.
1 0. N e w b a s i c a n d n o n b a s i c v a r i a b l e s.
D e f i n e a n e w v e c t o T o f b a s i c v a r i a b l e i n d i c e s b y r e p l a c i n g t h e 4t h e n t r y b y p. S e t t h e n e w v a l u e s o f t h e v a r i a b l e s a s f o l l o w s.
Xp — a Xj = Xi + a d i, I e b a s i c
6.7.3 T h e R e v i s e d S i m p l e x F u n c t i o n
T h e f o l l o w i n g R e v i s e d S i m p l e x f u n c t i o n i m p l e m e n t s t h e r e v i s e d s i m p l e x a l g o ­
r i t h m f o r s o l v i n g L P p r o b l e m s. T h e f u n c t i o n u s a g e a n d i t s o p t i o n s a r e e x p l a i n e d f i r s t. T h e f u n c t i o n i s i n t e n d e d t o b e u s e d f o r e d u c a t i o n a l p u r p o s e s. S e v e r a l i n ­
t e r m e d i a t e r e s u l t s c a n b e p r i n t e d t o g a i n u n d e r s t a n d i n g o f t h e p r o c e s s. T h e p r o c e d u r e f o r p e r f o r m i n g s e n s i t i v i t y a n a l y s i s u s i n g t h e r e v i s e d s i m p l e x p r o ­
c e d u r e i s a l s o i m p l e m e n t e d. T h e d e r i v a t i o n o f t h e s e n s i t i v i t y e q u a t i o n s i s p r e s e n t e d i n a l a t e r s e c t i o n.
T h e b u i l t - i n Mathematica f u n c t i o n s C o n s t r a i n e d M i n a n d L i n e a r P r o g r a m - m i n g d o t h e s a m e t h i n g. F o r l a r g e p r o b l e m s, t h e b u i l t - i n f u n c t i o n s m a y b e m o T e e f f i c i e n t.
Needs["OptimizationToolbox'DPSimplex’"];
?RevisedSimplex
RevisedSimplex[f, g, vars, options]. Solves an LP problem using Phase I and II simplex algorithm, f i s the objective function, g i s a l i s t of constraints, and vars i s a l i s t of variables. See Options[RevisedSimplex] to find out about a l i s t of valid options for this function.
O p t i o n s u s a g e [ R e v i s e d S i m p l e x ]
{UnrestrictedVariables -> f } , Maxlterations -> 10, ProblemType ->Min,
StandardVariableName -> x, PrintLevel -> 1, SensitivityAnalysis -» False)
C h a p t e r β l i n e a r P r o g r a m m i n g
U n r e s t r i c t e d V a r i a b l e s i s a n o p t i o n f o r LP a n d s e v e r a l QP p r o b l e m s.
A l i s t o f v a r i a b l e s t h a t a r e n o t r e s t r i c t e d t o b e p o s i t i v e c a n b e s p e c i f i e d w i t h t h i s o p t i o n. D e f a u l t i s {}.
M a x l t e r a t i o n s i s a n o p t i o n f o r s e v e r a l o p t i m i z a t i o n m e t h o d s. I t s p e c i f i e s ma x i mum n u m b e r o f i t e r a t i o n s a l l o w e d.
P r o b l e m T y p e i s a n o p t i o n f o r m o s t o p t i m i z a t i o n m e t h o d s. I t c a n e i t h e r b e M i n ( d e f a u l t ) o r M a x.
S t a n d a r d V a r i a b l e N a m e i s a n o p t i o n f o r LP a n d QP m e t h o d s. I t s p e c i f i e s t h e s y m b o l t o u s e w h e n c r e a t i n g v a r i a b l e n a m e s d u r i n g c o n v e r s i o n t o t h e s t a n d a r d f o r m. D e f a u l t i s x.
P r i n t L e v e l i s a n o p t i o n f o r m o s t f u n c t i o n s i n t h e O p t i m i z a t i o n T o o l b o x.
I t i s s p e c i f i e d a s a n i n t e g e r. T h e v a l u e o f t h e i n t e g e r i n d i c a t e s h o w m u c h i n t e r m e d i a t e i n f o r m a t i o n i s t o b e p r i n t e d. A P r i n t L e v e l - » 0 s u p p r e s s e s a l l p r i n t i n g. D e f a u l t f o r m o s t f u n c t i o n s i s s e t t o 1 i n w h i c h c a s e t h e y p r i n t o n l y t h e i n i t i a l p r o b l e m s e t u p. H i g h e r i n t e g e r s p r i n t m o r e i n t e r m e d i a t e r e s u l t s.
S e n s i t i v i t y A n a l y s i s i s a n o p t i o n o f s i m p l e x m e t h o d. I t c o n t r o l s w h e t h e r a p o s t - o p t i m a l i t y ( s e n s i t i v i t y ) a n a l y s i s i s p e r f o r m e d a f t e r o b t a i n i n g a n o p t i m u m s o l u t i o n. D e f a u l t i s F a l s e.
E x a m p l e 6.2 3 S o l v e t h e f o l l o w i n g L P p r o b l e m u s i n g t h e T e v i s e d s i m p l e x m e t h o d.
f = + 2 0 x 2 - Tp + 6 x 4;
sr = - 8 x 2 - x 3 + 9 x 4 £ 2, ^ - 1 2 x 2 - ^ + 3 x 4 == 3, x 3 + x 4 * 1 };
v a r s - { x l, x 2, x 3, x 4 } ;
-> { x 2 >, P r i n t L e v e l -» 2 ]
**** P r o b l e m i n S t a r t i n g S i m p l e x F o r m * * * *
N e w v a r i a b l e s a r e d e f i n e d a s -» { x l - » X]^, x 2 -» x 2 - x 3, x 3 -» x 4, x 4 -» x 5 }
A r t i f i c i a l v a r i a b l e s -» { x a, Xg }
x 4 + x 5 + x 7 == 1
A l l v a r i a b l e s > 0
V a r i a b l e s -» ( x x x 2 x 3 x 4 x 5 x 6 x 7 x 8 x g ) c - > f - 5 - 2 0 - 2 0 - ^ 6 0 0 0 θ)
\ 4 2
6,7 T h e R e v i s e d S i m p l e x M e t h o d
12]
11 τ
-8
8
-1
9
-1
0
1
0'
3
A -»
1
-12
12
1
"7
3
0
0
0
1
[ij
lo
0
0
1
1
0
1
0
0,
a r t i f i c i a l Objective Function -» x 8 + x 9 ***** iteration 1 (Phase 1) *****
B a s i c variables -» (x 7—XgXg)
Va l u e s o f b a s i c v a r i a b l e s -» ( l 2 3)
No nba s i c v a r i a b l e s -» ( χ! X2 X3 X4 X5 xg) f -» 0 Art. Obj . -» 5
0
1
o1
Ό1
Ί'
B - »
0
0
1
t
A
0
1
w - »
1
,1
0
Oj
1;
ioj
Reduced cost coef f i cient s, r -> - — 20 -20 — - 12 1
\ 4 2
'V
' 0 '
W
xB -»
2
d-»
1
■ f
Ra t i o s, - x/d - »
8
[3 J
l«J
New basic variable-» (x^)
Variable going out of basic set-» (x9)
Step length, a -» 6
New v a r i a b l e v a l u e s - > | δ 0 0 0 0 0 1 ^ (1
***** I t e r a t i o n 2 ( Pha s e 1) *****
Ba s i c v a r i a b l e s -» ( x! x7 x8)
Nonbasi c variabl es -» (x 2 X3 X4 x 5 x g x 9 )
f 9 1
2 A r t ‘ 0bD ’ 2
f l 0 11
O'
1
B —»
SJfM
0
0
C B
0
w - »
- i
,0 1 Oj
Xi
. 0 ,
Reduced cost coef fi cien ts, r - » [ 2 - 2 — 1 —
'6'
'—24'
£
xB —>
1
d -»
0
Ratios, - x/d -»
00
1
, - 2
i i )
New b a s i c v a r i a b l e - » (X3 )
C h a p t e r β L i n e a r P r o g r a m m i n g
Variable going out of basic set-» (xjJ
Step length, a -» -
New variable values -» |θ 0 — 0 0 0 1 0 0
* * * * * I t e r a t i o n 3 ( P h a s e 1 ) * * * * *
B a s i c v a r i a b l e s - » ( x3 X7 x§)
Values of basic variables-» 1 0
Nonbasic varicibles-» (xx x 2 X4 x 5 Xg Xg)
f -»
-5
A r t.
O b j. -» 0
' 8
0
l 1
f°]
1 ’
B -»
12
0
0
CB
0
w -»
2
- J
, 0
1
0,
.1-
0
f s
Reduced c o s t c o e f f i c i e n t s, r
Ra t i o s, - x/d -»
2 5
0 — - 7 1 -
3 3
1
τ
"Ϊ
1
d -»
- 1
1 - 7 J
New b a s i c v a r i a b l e - » (x5)
V a r i a b l e g o i n g o u t o f b a s i c s e t - » ( x g)
S t e p l e n g t h, a -» 0
New variable values-» ( o O — 0 0 0 1 0 0
* * * * * I t e r a t i o n 4 ( P h a s e 1 ) * * * * *
B a s i c v a r i c i b l e s - » ( X3 X5 X7 )
Values of basic variables -» | — 0 1
Nonbasic variables -» ( x - l x 2 X 4 x6 x 8 x9) f -» -5 Art. Obj . -» 0
' 8
9
ol
Ό1
B -»
12
3
0
0
w -»
. 0
1
1.
Vj
Reduced c o s t c o e f f i c i e n t s, r -» (o 0 0 0 1 l )
***** I t e r a t i o n 5 (Phase 2) *****
Ba s i c v a r i a b l e s - » ( x3 x 5 x7)
Values of basic variables^ ^— 0 1
Nonbasic varicibles-» (χ! x 2 X4 Χβ) f -> -5
6.7 T h e R e v i s e d S i m p l e x M e t h o d
' 8
9
0^
r-20>
( ¥ )
B -*
12
3
0
cB ->
6
w -»
- Ψ
.0
1
1,
l 0 i
0 ,
R e d u c e d c o s t c o e f f i c i e n t s, r - » [ — 0
2
7
1 1 1 7 j
[ *'
/ \ CO
« β -*
4
0
,1 -
d -»
A
23
R a t i o s, - x/d - »
op 21 1 2 7/
New basi c v a r i a b l e s ( x 4 )
V a r i a b l e going out of bas i c s e t -» (χ?)
2 1
Step l ength, a -» —
New v a r i a b l e v a l u e s -» [ 0 0
49 21
184 23 23
***** i t e r a t i o n 6 (Phase 2 ) * * * * *
B a s i c v a r i c i b l e s - » ( x3 x 4 x 5 )
, 49 21
Values of basic variables Nonbasic variables -» f -»
o)
184 23 23
( *l x 2 x 6 X7 )
B -»
121
23
' 8 -1
9'
12 - J
3
0 1
V
37 TS
c B -> *,·* - § §
6
'2 7
Reduced c o s t c o e f f i c i e n t s, r -» [ ^ 0
L tt J
/'- 2 0'\ 1 7 6
3 7 _6_\
23 2 3 )
***** optimum s o l u t i o n a f t e r 6 i t e r a t i o n s * * * * * B a s is -» { { { x 3/ x 4, x g } } }
V a r i a b l e v a l u e s O b j e c t i v e f u n c t i o n
®
·, 49 2 1 2
x l - » 0, x 2 - » - 7^ 7, x3 -» —, x4 -»
184
23
23
}}}
1 2 1 ' 23
E x a m p l e 6.2 4 Stock Cutting T h i s e x a m p l e d e m o n s t r a t e s f o r m u l a t i n g a n d s o l v i n g a t y p i c a l s t o c k - c u t t i n g p r o b l e m t o m i n i m i z e w a s t e. A c a r p e n t e r i s w o r k ­
i n g o n a j o b t h a t n e e d s 2 5 b o a r d s t h a t a r e 3.5 f t l o n g a n d 3 5 b o a r d s t h a t a r e 7.5 f t l o n g. T h e l o c a l l u m b e r y a r d s e l l s b o a r d s o n l y i n t h e l e n g t h s o f 8 f t a n d 12 f t. H o w m a n y b o a r d s o f e a c h l e n g t h s h o u l d h e b u y, a n d h o w s h o u l d h e c u t t h e m, t o m e e t h i s n e e d s w h i l e m i n i m i z i n g w a s t e?
C h a p t e r 6 L i n e a r P r o g r a m m i n g
T h e o p t i m i z a t i o n v a r i a b l e s are t h e d i f f e r e n t w a y s i n w h i c h t h e g i v e n boards c a n b e c u t t o y i e l d b o a r d s o f r e q u i r e d l e n g t h. T h u s, w e h a v e t h e f o l l o w i n g s e t o f variables:
From t h e 8 f t board
*121 One 3.5 ft board with 8.5 ft waste
*122 T^o 3.5 ft boards with 5 ft waste
*123 Three 3.5 ft boards with 1.5 ft waste
*124 On e 7.5 ft board with 4.5 ft waste
*125 One each 7.5 ft and 3.5 ft long boards with 1 ft waste
U s i n g t h e s e c u t t i n g p a t t e r n s, t h e t o t a l w a s t e t o b e m i n i m i z e d i s a s f o l l o w s: / = 4.5 X 8 1 + 0.5 X 8 2 + * 8 3 + 8.5 X 1 2 1 + 5X122 + 1.5 X 1 2 3 + 4.5 X 1 2 4 + * 1 2 5
T h e c o n s t r a i n t s are t h e n u m b e r o f b oard s o f r e q u i r e d l e n g t h.
3.5 f t boards: x g i -I- 2 x 8 3 +*121 4 - 2x 122 + 3 x i 2 3 + X125 ^ 2 5
7.5 f t b o a r d s: X82 + X124 + *125 > 3 5
I n t e r m s o f Mathematica e x p r e s s i o n s, t h e p r o b l e m i s f o r m u l a t e d a s f o l l o w s:
vars = {Xgi , 3*82 ' **83 ' **121' *122 ’ *123 ’ *124 ’ *125 1 ' f — 9 + l/2Xg2 ^ x83 ^7/2x ^21 + 5x^22 ^ 3/2x^23 + 9/23C^24 + ^125 *
Sf = {^81 + 2*83 + *121 + 2x122 + ^*123 + *125 ^ 25' *82 + *124 + *125 ^ 35J '
U s i n g R e v i s e d S i m p l e x, t h e o p t i m u m i s o b t a i n e d a s f o l l o w s:
R e v i s e d S i m p l e x [ f, g, v a r s, P r i n t L e v e l -+ 2 ] ;
**** problein in Starting Simplex Form ****
New varicibles are defined as -s·
{ x 81 -* X1 ' x 82 x2 ' x83 -> x3 ' x 121 x4 ' x 122 ”* x 5 ' x 123 ** x6 ' x124 x7 ' x125 Χβ1 Slack/surplus variables -» {x9, Χχο}
A r t i f i c i a l variables -» [ x n , x12}
X81 O n e 3.5 f t b o a r d w i t h 4.5 f t w a s t e
X82 O n e 7.5 f t b o a r d w i t h 0.5 f t w a s t e
X83 T t o o 3.5 f t b o a r d s w i t h 1 f t w a s t e
From t h e 12 f t b o a r d
6.7 T h e R e v i s e d S i m p l e x M e t h o d
A l l v a r i a b l e s ΐ 0
V a r i a b l e s -4 ( χ χ x 2 x 3 x 4 x 5 x 6 x 7 x 8 x 9 x 10 x x l x 1 2 )
1 7 3 9
2 2 2
i" 1 \ 2 2
c ^ ι — — 1 — 5 — — 1 0 0 0 0
/ 2 5 \ /l 0 2 1 2 3 0 1 - 1 0 1 0
b _ > 1 3 5 A"M o 1 0 0 0 0 1 1 0 - 1 0 1
A r t i f i c i a l O b j e c t i v e F u n c t i o n - » x ^ + Χ χ2 * * * * * i t e r a t i o n 1 ( P h a s e 1 ) * * * * *
B a s i c v a r i a b l e s -» ( χ ^ X12)
V a l u e s o f b a s i c v a r i a b l e s - » ( 2 5 3 5 )
N o n b a s i c v a r i a b l e s - » ( χ! x 2 X3 X4 X 5 Χβ x 7 χ β ^ χ ΐ θ ) f -> 0 A r t. O b j . -» 6 0
1 0\ /l\ f l
B ^ o i j c * ^ { i )
R e d u c e d c o s t c o e f f i c i e n t s, r - i ( - 1 - 1 - 2 - 1 - 2 - 3 - 1 - 2 1 l )
*b - (3I) a - ( i ) -χ/a-* (2„5)
New b a s i c v a r i a b l e -> ( χ χ )
V a r i a b l e g o i n g o u t o f b a s i c s e t - » ( x j,i )
S t e p l e n g t h, a - » 2 5
Ne w v a r i a b l e v a l u e s -» ( 2 5 0 0 0 0 0 0 0 0 0 0 3 5 )
* * * * * i t e r a t i o n 2 ( P h a s e 1 ) * * * * *
B a s i c v a r i a b l e s - » ( x ^ X12)
V a l u e s o f b a s i c v a r i a b l e s -» ( 2 5 3 5 )
N o n b a s i c v a r i a b l e s -» ( x 2 x 3 x 4 x 5 x g x 7 x 8 x 9 x 10 x ^ i }
2 2 5
f -» —— A r t. O b j . -» 3 5 B"( o I ) ° B (1 ) * - > (?
Re duc e d c o s t c o e f f i c i e n t s, r - » ( - 1 0 0 0 0 - 1 - 1 0 1 l )
35 ) d"( - ° i ) Rat i os' “x/d" ( 35
N e w b a s i c v a r i a b l e - » ( x 2 )
V a r i a b l e g o i n g o u t o f b a s i c s e t - » ( x ] _ 2 )
S t e p l e n g t h, a - » 35
N e w v a r i a b l e v a l u e s - » ( 2 5 3 5 0 0 0 0 0 0 0 0 0 θ )
C h a p t e r 6 L i n e a r P r o g r a m m i n g
* * * * * I t e r a t i o n 3 ( P h a s e 1 ) * * * * * B a s i c v a r i a b l e s - » ( χ χ X 2)
V a l u e s o f b a s i c v a r i a b l e s - » ( 2 5 3 5 )
N o n b a s i c v a r i c i b l e s - » ( x 3 x 4 x 5 x g X7 Xe Xg X i o x n x i 2 )
f - » 1 3 0 A r t. O b j . -» 0
° ) ° H ° ) H ° )
R e d u c e d c o s t c o e f f i c i e n t s, r -» ( θ 0 0 0 0 0 0 0 l l )
* * * * * I t e r a t i o n 4 ( P h a s e 2 ) * * * * *
B a s i c v a r i a b l e s - » ( χ χ X2)
V a l u e s o f b a s i c v a r i c i b l e s - » ( 2 5 3 5 )
N o n b a s i c v a r i c i b l e s - » ( x 3 x 4 x 5 x 6 x 7 x 8 x 9 x 1 0 )
f -> 1 3 0
1 0\
0 1 ) °B "*
' \2 v 2
I 9 1
R e d u c e d c o s t c o e f f i c i e n t s, r - » -8 4 - 4 - 1 2 4 - 4 — —
\ 2 2
,2 5
R a t i o s, - x/d - » I ^
( 9 )
9
7
w - »
7
1
1
2 J
2
Xb^ ( 3 5 )
0
N e w b a s i c v a r i a b l e - » ( x 3 )
V a r i a b l e g o i n g o u t o f b a s i c s e t - » ( k ^ ) 2 5
S t e p l e n g t h, a - » —
2
I 2 5
N e w v a r i a b l e v a l u e s - » ( 0 3 5 — 0 0 0 0 0 0 0
* * * * * i t e r a t i o n 5 ( P h a s e 2 ) * * * * *
B a s i c v a r i a b l e s - » ( x2 x 3 )
/ 2 5
V a l u e s o f b a s i c v a r i c i b l e s - » ( 3 5 —
\ 2
N o n b a s i c v a r i a b l e s - » { x 1 x 4 x 5 x 6 x 7 x 8 x 9 x 1( J) f -» 3 0
B*M? 0 (?* w ’
I ’i
7
1
7 )
1 ΐ ι
R e d u c e d c o s t c o e f f i c i e n t s, r - » 4 8 4 0 4 0 — —
\ 2 2
* * * * * O p t i m u m s o l u t i o n a f t e r 5 i t e r a t i o n s * * * * *
B a s i s -> { { { x 2, x 3 } } }
6.7 T h e R e v i s e d S i m p l e x M e t h o d
V a r i a b l e v a l u e s -» ^
!| | x 81 0/ x 82 - >3 5, X83 -» —, X 121 ®' X i 2 2 x 123 x 1 2 4 x 1 2 5 ® } } |
O b j e c t i v e f u n c t i o n - » 3 0
Hi e probl em i ndee d i s a di screte opt i mi zati on problem. Si nce i t was sol ved wi th the assumpti on that t he variables are conti nuous, we got an opt i mum that i s not an integer. Based on t hi s sol uti on, we ne e d to buy and cut boards as follows:
35 boards e ach 8 ft ==» 35 boards each 7.5 ft Waste = 17.5 ft
13 boards each 8 ft =Φ· Each board yi el ds two 3.5 ft boards
Waste = 12 + 4.5 ft ■
Actual wast e = 34 ft.
The probl em al so has mul ti pl e opt i mum sol uti ons. Solvi ng t he probl em usi ng the BasicSimplex, we obtain a di fferent opt i mum sol ut i on wi t h the same val ue o f the objective f unc t i on.
B a s i c S i m p l e x [ f, f f, v a r s ]
P r o b l e m v a r i a b l e s r e d e f i n e d a s:
{ x 8 l - > Χ χ, X82 -»X2, x 83 -» X 3 , X12l x 4 / X l 22 -»x5' x123 -*x6'x 124 x7 ' x125 x 8 }
. . , 9xn x 2 1 7 x 4 „ 3 x g 9 x 7
M i n i m i z e —-*■ +■ — + x, + ---------- + 5 χ ς + — - + —- L + x „
2 2 2 2 2
S u b j e c t t o f o + 2 X 3 + X* + 2 χ 5 + 3χ6 + x 8 * 2 5 )
\ x2 +x7 +x8 >35 j
A l l v a r i a b l e s a 0
* * * * * * * * * * I n i t i a l s i m p l e x t a b l e a u * * * * * * * * * *
N e w p r o b l e m v a r i a b l e s : ( x ^, x2 , X3 , Χ4, X5, X g, x 7, x 8, , s 2 / a ^, 82}
B a s i s
1
2
3
4
5
6
11
1
0
2
1
2
3
12
0
1
0
0
0
0
O b j.
9
Ϊ
1
7
1
1 7
T
5
3
Ί
A r t O b j.
- 1
- 1
- 2
- 1
- 2
- 3
7
8
9
10
11
12
RHS '
0
1
- 1
0
1
0
2 5
1
1
0
- 1
0
1
3 5
9
1
1
0
0
0
0
f
- I
-2
1
1
0
0
- 6 0 + φ,
* * * * * * * * * * F i n a l s i m p l e x t a b l e a u * * * * * * * * * *
C h a p t e r 6 L i n e a r P r o g r a m m i n g
B a s i s
1
2
3
4
5
6
6
2
1
3
0
0
1
2
Ί
0
1
1
0
J
0
1
0
. O b j.
4
0
0
8
4
0
7
8
9
10
11
12
RHS
0
1
4
1
1
1
0
1
0
1
Ϊ
0
-1
1
7
1
J
0
1
J
0
1
1
~ 7
Ψ
35
-30 +
O p t i m u m s o l u t i o n -»
2 5
{ ί i x81 0' x82 35, Xg 3 -» 0, Χχ21 ** 0, x 122 ** x 123 y' x 124 x 125 0} } }
O p t i m u m o b j e c t i v e f u n c t i o n v a l u e - > 3 0 * * T h e p r o b l e m m a y h a v e m u l t i p l e s o l u t i o n s.
6.8 S e n s i t i v i t y A n a l y s i s U s i n g t h e R e v i s e d S i m p l e x M e t h o d
The revi sed si mpl ex formul ati on i s also advantageous for deri vi ng the sensi ­
t i vi t y anal ysi s procedures. For the f ol l owi ng di scussi on, we assume that we kn o w x* to b e an opt i mum sol ut i on o f t he fol l owi ng LP problem:
Mi ni mi ze / = c r x = ( c f c j j ) Subject to Α χ ξ ( β N) = b
where B refers to quanti t i es associ ated wi t h basi c variables and N to those associ at ed wi t h nonbasi c variables.
6.8.1 L a g r a n g e M u l t i p l i e r s
The negati ve o f the mul ti pl i ers w T — c^B- 1 comput ed i n st ep 3 o f the revised si mpl ex al gorithm turn out to be the Lagrange mul ti pl i ers. As ment i oned be­
fore, i n actual computati ons, t h e y are obtai ned b y sol vi ng t he f ol l owi ng syst em o f l i near equati ons.
B rw = Cjj
6.8 S e n s i t i v i t y A n a l y s i s U s i n g t h e R e v i s e d S i m p l e x M e t h o d
6.8.2 C h a n g e s i n O b j e c t i v e F u n c t i o n C o e f f i c i e n t s
In this section, we consider the effect o f changes in the objective function coefficients on the optimum solution. Assume that the modified coefficients are written as follows:
c + a A c = (Cjg + αΔΟβ) + (cn + ccA cm)
where Ac is a vector o f changes in the coefficients and a is a scale factor. Our goal is to find a suitable range for a for which we can determine a new optimum solution without actually solving the modified problem. Since the constraints are not changed, the feasible domain remains the same. For x* to remain optimum, the reduced cost coefficients for the modified objective function should all still be positive. Thus,
(cm + ocA c n ) t — (θΒ + αΔθβ)ΓΒ- 1 Ν > 0
or
( CN _ cbb _ 1n ) + a ( Δ(5ν ~ A c | b - 1N^ = rr + a f T > 0
The vector r is the reduced cost of the original problem. The second term rep- resents the reduced cost due to changes in the objective function coefficients. Thus, the following condition must be satisfied for x* to remain optimum.
a r > —r
All components o f τ should be positive but the components o f r can either be positive or negative. Therefore, we get the following lower and upper limits for a that will satisfy this condition:
«min = Max [ j -zr~, π > 0, i € nonbasic j , — ooj
amax —
Min ri < 0* * € nonbasic j , ooj
The above discussion applies to simultaneous changes in as many coefficients as desired. In practice, however, it is more common to study the effect of changes in one o f the coefficients at a time. The following two cases arise naturally. Note that the same rules were given with the basic simplex procedure but without their derivations.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
Special Case: Changes i n Coefficient o f the I t h Nonbasic Variable
In this case, Ac# = 0 and A c n = e', where e1 is a vector with 1 corresponding to the Ith nonbasic variable whose coefficient is being changed, and 0 everywhere else. Then we have
( c h - c ^ B ^ n ) + « (e’) r > 0
or
rr + a (e')T > 0 =Φ· a > —η
Thus, the change should be larger than the negative o f the reduced cost co­
efficient corresponding to the Ith variable. There is no upper limit. Therefore, the allowable range for the Ith cost coefficient that corresponds to a nonbasic variable is
— < A q < oo
Special Case: Changes i n Coefficient o f the I t h Basic Variable
In this case, A cm = 0 and Acs = e 1, where e1 is a vector with 1 corresponding to the Ith basic variable whose coefficient is being changed, and 0 everywhere else. Then we have
( c j - c J b_1 n ) - a (e*)^B- 1 N > 0 r r — a ( B't y ® > 0
or
Referring to
tableau, we see that (B
represents entries in the Ith row o f the constraint matrix corresponding to the nonbasic variables. Therefore the allowable range for the Ith cost coefficient that corresponds to a basic variable is
(Δ^ίΟπήη = Max ( Ac Omax = M i n
- Π
(0—. («Ν h' > ° J E nonbasi c \ , - o o I
(aN ); J J
- f f
Φ ι
—, (α$ )j < 0
,j € nonbasic
ic j, ooj
where ( a $ ) j refers to the element in the Ith row o f the (B !N) matrix.
6.8 S e n s i t i v i t y A n a l y s i s TTaing t h e R e v i s e d S i m p l e x M e t h o d
6.8.3 C h a n g e s i n t h e R i g h t - h a n d S i d e s o f C o n s t r a i n t s
In this section, we consider the effect of changes in the right-hand sides of constraints on the optimum solution. Assume that the modified constraints are written as follows:
Ax = b + a A b
where Ab is a vector o f changes in the right-hand sides of constraints and a is a scale factor. The reduced cost coefficients do not change and are all positive. Since the feasible domain is now changed, the optimum solution x* may change. If the basic variables remain the same, then for optimality we must have
xb = B 1 (b+aAb) > 0
or
t - i
B b +αΒ Ab > 0 orAb > —b
where b = B- 1 b and Ab = B- 1 Ab. Note that all entries in b must be positive because they represent the solution for basic variables for the original problem. Entries in Ab may be either positive or negative. Thus, we get the following lower limit and upper limit for a in order to meet the feasibility condition.
amin — Max
Max
- b i —
A bi > 0, t = , —oo
Abi
«max — Min
Min
■bi
A b{
, A fy < 0, i = 1
,... m J , oo j
Fo r a v a l u e s w i t h i n t h e a l l o w a b l e r a n g e, t h e n e w o p t i m u m s o l u t i o n c a n b e o b t a i n e d a s f o l l o ws:
x | = a A b + b a n d /* = c ^Xg
N o t e t h a t, i n c o n t r a s t w i t h t h e r u l e s g i v e n w i t h t h e b a s i c s i m p l e x p r o c e d u r e, h e r e i t i s p o s s i b l e t o g e t b o t h t h e o p t i m u m v a l u e o f t h e o b j e c t i v e f u n c t i o n a s w e l l a s t h e o p t i m u m v a l u e s o f t h e o p t i m i z a t i o n v a r i a b l e s.
Special Case: Changes i n t h e Right-hand Side o f the I t h Constraint
The above discussion applies to simultaneous changes in as many constraints as desired. As a special case, we consider the effect o f changes in one of the constraint right-hand sides at a time. In this case, we will get the same rules that were given with the basic simplex method.
The change in the right-hand side o f the Ith constraint can be written as Ab = e', where e* is a vector with 1 in the 1th location, and 0 eveiywhere else. Then we have
B 1b + a B i Ab = b + a B 1e* > 0 =>> aB, > —b
where B, is the j* column o f the inverse o f the B matrix. Thus, we get the following lower limit and upper limit for a in order to meet the feasibility condition.
«min — Max amax = Min
- = ^ -,B ik <
0
,k = l,...m\ Bi k
I t c a n b e s e e n f r o m t h e f o l l o wi n g n u me r i c a l e x a mp l e s t ha t t h e i * c o l u mn o f t h e i n v e r s e o f t h e B ma t r i x i s t h e s a me a s t h e s l a c k/a r t i f i c i a l var i abl e c o l u mn u s e d i n t h e s e n s i t i v i t y a n a l y s i s wi t h t h e b a s i c s i mp l e x pr o c e dur e. No t e a s l i ght c h a n g e i n n o t a t i o n i n t h e n u me r i c a l e x a mp l e s. Sy mbo l bb i s u s e d t o r e f e r t o b f or c o n v e n i e n c e i n i mp l e me n t a t i o n.
6.8.4 T h e S e n s i t i v i t y A n a l y s i s O p t i o n o f t h e
The S e n s i t i v i t y An a l y s i s o p t i o n o f t h e Re v i s e d S i mp l e x f u n c t i o n r e t ur ns La­
g r ange mu l t i p l i e r s a nd a l l o wa bl e r a n g e s f or r i ght - hand s i d e s and c o s t c o e f f i ­
c i e n t s.
O p t i o n s [ R e v i s e d S i m p l e x ]
{ U n r e s t r i c t e d V a r i a b l e s - » { } , M a x l t e r a t i o n s - » 1 0,
P r o b l e m T y p e - » M i n, S t a n d a r d V a r i a b l e N a m e - » x,
P r i n t L e v e l - » 1, S e n s i t i v i t y A n a l y s i s - » F a l s e }
?S e n s i t i v i t y A n a l y s i s
6.8 S e n s i t i v i t y A n a l y s i s U s i n g t h e R e v i s e d S i m p l e x M e t h o d
S e n s i t i v i t y A n a l y s i s i s a n o p t i o n o f s i m p l e x m e t h o d. I t c o n t r o l s w h e t h e r a p o s t - o p t i m a l i t y ( s e n s i t i v i t y ) a n a l y s i s i s p e r f o r m e d a f t e r o b t a i n i n g a n o p t i m u m s o l u t i o n. D e f a u l t i s F a l s e.
Example 6.25
f τ - 3 x 1 - 2 x 2; g = { x l + x 2 ί 4 0, 2 x 1 + x 2 4 6 0 } .· v a r s = ( x l, x 2 > ?
R e v i s e d S i m p l e x [ f, g, v a r s, S e n s i t i v i t y A n a l y s i s -> T r u e, P r i n t L e v e l -> 2 ];
* * * * p r o b l e m i n S t a r t i n g S i m p l e x F o r m * * * *
N e w v a r i a b l e s a r e d e f i n e d a s -> { x l -> x -l , x 2 -» }
S l a c k/s u r p l u s v a r i a b l e s -» { x 3 , x 4 }
M i n i m i z e - 3 χ! - 2 x 2
A l l v a r i a b l e s > 0 V a r i a b l e s -» ( x ^ X2 x 3 x 4 )
c -» (-3 - 2 0 O)
* * * * * I t e r a t i o n 1 ( P h a s e 2 ) * * * * * B a s i c v a r i a b l e s - » ( x 3 x 4 )
V a l u e s o f b a s i c v a r i a b l e s - » ( 4 0 6 θ )
N o n b a s i c v a r i a b l e s -» ( x } x 2 )
f -» 0
Reduced c o s t c o e f f i c i e n t s, r -» ( - 3 - 2)
x B - ^ 6 0 j R a t i o s, - x/d - | 3C
New bas i c v a r i a bl e - » (xjJ
Variable going out of b a s i c s e t - » (x4)
Step l ength, a-» 30
New v a r i a bl e val ues -» (30 0 10 θ)
***** I t e r a t i o n 2 (phase 2) *****
Basic v a r i a bl e s - » (x-l x 3 )
Values of basic variables -» (30 io) Nonbasic v a r i a bl e s - » (x2 X4 )
40
R a t i o s, - x/d - » L p
f -» -90
C h a p t e r 6 L i n e a r P r o g r a m m i n g
1 1\ /- 3
2 0! Cfl 'I 0
( 4 )
.... I 1 3
R e d u c e d c o s t c o e f f i c i e n t s, r - » I - — —
- ( J o ) _ 1 R a t i o S' - X/d ^ (2 0]
N e w b a s i c v a r i a b l e - » ( x 2 )
V a r i a b l e g o i n g o u t o f b a s i c s e t - » ( x 3 ) S t e p l e n g t h, c t - » 2 0 N e w v a r i a b l e v a l u e s - » ( 2 0 2 0 0 θ )
* * * * * i t e r a t i o n 3 ( P h a s e 2 ) * * * * *
B a s i c v a r i a b l e s - » ( x ^ x 2 )
V a l u e s o f b a s i c v a r i a b l e s - » { 2 0 2 0 ) N o n b a s i c v a r i a b l e s - » ( x3 X4)
f -» -100
B (2 I) C® (-2 1
( 5 )
R e d u c e d c o s t c o e f f i c i e n t s, r ^ ( l l )
* * * * * O p t i m u m s o l u t i o n a f t e r 3 i t e r a t i o n s * * * * * B a s i s - » { { { x i, x 2 } } }
V a r i a b l e v a l u e s - » { { { x l - » 2 0, x 2 2 0 } } }
O b j e c t i v e f u n c t i o n - * - 1 0 0 L a g r a n g e m u l t i p l i e r s 1 l\ ( - 3
2 1 J ° B ^ \- 2
L a g r a n g e m u l t i p l i e r s ( l l ) S e n s i t i v i t y t o c o n s t r a i n t c o n s t a n t s
. ( l\ .. ( 20)
4 b'> ) “ ’ ’ ’ ( i O,
A b b - > I M - b b/A b b - > ί ^
^ ( 1 ) bb^ (20) διλ>-* ( Λ ) - bb/Abb^ ( ΐ ο 0)
A l l o w a b l e c o n s t r a i n t RHS c h a n g e s a n d r a n g e s
- 1 0 1 Ab-L < 2 0 3 0 < b i 5 6 0
- 2 0 s A b 2 < 2 0 4 0 < b 2 s 8 0
S e n s i t i v i t y t o o b j e c t i v e f u n c t i o n c o e f f i c i e n t s
m
A c -»
r -»
6.8 S e n s i t i v i t y A n a l y s i s U s i n g t h e R e v i s e d S i m p l e x M e t h o d
/0Ί
\o)
A l l o w a b l e c o s t c o e f f i c i e n t c h a n g e s a n d r a n g e s
-1 < Acj^ <1 - 4 £ c-l < - 2
- 1 ί Ac 2 £ j - 3 s c 2 i - §
Figure 6.8 gi ves a graphical interpretati on o f t he changes i n t he objec­
ti ve f uncti on coef f i ci ent s. The current mi ni mum poi nt i s l abel l ed as b. With changes i n the objecti ve f unct i on coef f i ci ent s al one, t he feasi bl e regi on re­
mai ns t he same. Onl y t he s l ope o f t he objecti ve f unct i on contours change. If the change i n s l ope i s smal l, the s e t o f current basi c variables and thei r val ues wi l l not change. The l i mi t i ng changes i n t he cost c oef f i ci ent s are t hose that wi l l make t he vert ex bef ore or after t he current opt i mum (poi nts a and c) as the ne w opt i mum.
x2 Cost coefficient changes
FIGURE 6.8 Solution with the original and the modified objective functions.
The graphs s hown i n Fi gure 6.9 illustrate t he ef f ect o f changes i n t he right- hand s i des o f t he constraints. I f the changes are wi t hi n t he al l owabl e l imits, the basi s do not change and therefore, the n e w opt i mum can be determi ned s i mpl y b y comput i ng a n e w bas i c sol uti on. I f t he changes are out si de t he l i mi ts, the bas i c variables change and a compl et e n e w s ol ut i on i s required.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
Allowab 50
e change, basis remains the same Excessive change new basis
0 10 20 30 40 50
10 20 30 40 50
FIGURE 6.9 A graph illustrating solutions with two different right-hand sides for g\, Example 6.26
£ = - + 2 0 x 2 - *5? + 6 x 4;
g = - 8 x 2 - x 3 + 9 x 4 i 2, y - 1 2 x 2 - + 3 x 4 == 3, x 3 + x 4 <, 1 >;
v a r s = { x l, x 2, x 3, x 4 };
R e v i s e d S i m p l e x [ f, g, v a r s, U n r e s t r i c t e d V a r i a b l e s -» { x 2 },
S e n s i t i v i t y A n a l y s i s -» T r u e, P r i n t L e v e l -+ 2 ] ?
* * * * P r o b l e m i n S t a r t i n g S i m p l e x F o r m * * * *
N e w v a r i a b l e s a r e d e f i n e d a s { x l -» x ^, x 2 - » x 2 - x 3 , x 3 x 4, x 4 x 5 }
S l a c k/s u r p l u s v a r i a b l e s - * { χ 6, x 7 }
A r t i f i c i a l v a r i a b l e s ( x 8, x 9 }
3 jc ι
M i n i m i z e ---------- + 2 Ox? - 2 0 x ^ ---------- + 6 x c
4 * * 2 5
^ - 8 x 2 + 8 x 3 - x 4 + 9 x s - x 6 + x 8 = = 2 1 S u b j e c t t o ΐ - 1 2 x 2 + 1 2 x 3 - + 3 x 5 + x 9 = = 3
ι X4 + x 5 + x 7 = = 1
A l l v a r i a b l e s > 0
V a r i a b l e s -» ( x ^ x 2
x 3 X4 x 5
x 6
x 7
x8
x 9 )
c
[ -
4 2 0
- 2 0 -
1 6 0 0
2
0
°)
2 1
1 Q
τ _ 8
8 - 1
9
- 1
0
1 o'
b
3
A - »
i -12
12 - J
3
0
0
0 1
l 1)
.0 0
0 1
1
0
1
9 0]
A r t i f i c i a l O b j e c t i v e F u n c t i o n - » Xg + x 9
6.8 S e n s i t i v i t y A n a l y s i s U s i n g t h e R e v i s e d S i m p l e x M e t h o d
* * * * * I t e r a t i o n 1 ( P h a s e 1 ) * * * * *
B a s i c v a r i a b l e s - » ( x 7 x 8 x 9 )
V a l u e s o f b a s i c v a r i a b l e s - » ( l 2 3 )
N o n b a s i c v a r i a b l e s - » x 2 x 3 x 4 x 5 x g )
f -» 0 A r t. O b j . -» 5
0 1 0\
π
B - »
0 0 1 j c B - »
1
w - »
1
,1 0 0/
l j
.0,
Reduced c o s t c o e f f i c i e n t s, r -» 2 0
_ 2 „ |
- 1 2 1
1'
τ
[V
o
R a t i o s, - x/d -»
00
\S}
N e w b a s i c v a r i a b l e - » ( x j J
V a r i a b l e g o i n g o u t o f b a s i c s e t - » ( x 9 )
S t e p l e n g t h, a -» 6 ,
Ne w v a r i a b l e v a l u e s - » | 6 0 0 0 0 0 1 — 0
* * * * * I t e r a t i o n 2 ( P h a s e 1 ) * * * * *
B a s i c v a r i a b l e s - » ( x! x7 x 8)
V a l u e s o f b a s i c v a r i a b l e s - » 1 i - j
N o n b a s i c v a r i a b l e s - » ( x 2 x 3 x 4 x 5 Xg Xg)
9 . 1 f — A r t. O b j . -» —
2 2
B - »
\ 0 V
J 0 0
C B
O'
0
w - »
1
- §
[ 0 1 0 j
UJ
υ
Reduced c o s t c o e f f i c i e n t s, r -» ^2 - 2 ^
1 5
'6>
CM
1
1'
τ
XB
1
a-»
0
R a t i o s, - x/d - »
CD
1
U 2,
1
r r
New basic variable-» (x3)
Variable going out of basic set -» (xjJ
S t e p l e n g t h, a - * — 4
Ne w v a r i a b l e v a l u e s -> | θ 0
0 0 0 1 0 0
C h a p t e r 6 L i n e a r P r o g r a m m i n g
* * * * * I t e r a t i o n 3 { P h a s e 1 ) * * * * * B a s i c v a r i a b l e s -» ( x 3 x 7 x g }
V a l u e s o f b a s i c v a r i a b l e s - » 1 0
N o n b a s i c v a r i a b l e s - » ( χ χ x 2 X 4 X 5 Xg χ θ )
1
w -
f -»
- 5
A r t.
O b j . -» 0
’ 8
0
1'
O'
B -»
12
0
0
CB -»
0
, 0
1
l i j
0
I 1 2 5 V
R e d u c e d c o s t c o e f f i c i e n t s, t - » I — 0 - - 7 1 j j
i l l
Ϊ
l l
“ Ϊ
1
x B -»
1
,0 j
d - »
- 1
1 - 7,
R a t i o s, - x/d - »
1
l o,
N e w b a s i c v a r i a b l e - » ( X5)
V a r i a b l e g o i n g o u t o f b a s i c s e t - » ( x 8 ) S t e p l e n g t h, ct -» 0
N e w v a r i a b l e v a l u e s -» | 0 0 — 0 0 0
* * * * * i t e r a t i o n 4 ( P h a s e 1 ) * * * * * B a s i c v a r i a b l e s -» ( x 3 x 5 H7 )
1 0 0
V a l u e s o f b a s i c v a r i a b l e s - » 0 1
N o n b a s i c v a r i a b l e s - » ( x - l x2 X4 x g x g x 9 )
f - » -
- 5
A r t.
O b j . -» 0
’ 8
9
0 ’
0 ’
0'
B
1 2
3
0
CJ5 -»
0
w -»
0
, 0
1
1,
0,
,0,
R e d u c e d c o s t c o e f f i c i e n t s, r - » ( θ 0 * * * * * i t e r a t i o n 5 ( P h a s e 2 ) * * * * * B a s i c v a r i a b l e s - » ( x 3 x 5 x 7 j
OOll)
V a l u e s o f b a s i c v a r i a b l e s - »
N o n b a s i c v a r i a b l e s - » ( x! x 2 x 4 x 6 ) f - » - 5
' 8
9
O'
- 2 0 1
1 1
T 9
~ ~ T
B -»
12
3
0
CB
6
w -»
. 0
1
1-
, 0 ,
, 0 -
R e d u c e d c o s t c o e f f i c i e n t s, r - »
1 4
0 -
1 1
6.8 S e n s i t i v i t y A n a l y s i s U s i n g t h e R e v i s e d S i m p l e x M e t h o d
A
0
d - >
TE
UJ
2 3
R a t i o s, - x/d - »
oo
21
TS
New ba sic variable-» ( x 4 )
V a r i a b l e g o i n g o u t o f b a s i c set-» (x7) 2 1
S t e p l e n g t h, a- »
2 3
N e w v a r i a b l e v a l u e s - » 0 0
JL
2 3
4 9 2 1
1 8 4 2 3
* * * * * i t e r a t i o n 6 ( P h a s e 2 ) * * * * *
B a s i c v a r i a b l e s - » ( x3 x 4 x 5 )
* ^ / 4 9 2 1 2
Val ues o f b a s i c var i a bl es -:» j j ^
N o n b a s i c v a r i a b l e s - » ( χ! x 2 X 5 x 7 )
1 2 1
°)
f -» ~
2 3
8
12 -i
9\
3
1
C n - »
!- 20 1 1 6
w -»
lo 1
R e d u c e d c o s t c o e f f i c i e n t s, r - »
37
ΤΊ
63
A »
3 7 _6_
2 3 23
* * * * * O p t i m u m s o l u t i o n a f t e r 6 i t e r a t i o n s * * * * * B a s i s -> { { { x 3 , x 4, x5 } } }
{{{
1 21
Variable values -»
49- 2 1
x l -» 0, x 2 -» - , x 3 -» —, x 4 -»
*}}}
Objective f unc t i o n- » - · Lagrange mul t i pl i e r s
2 3
' 8
- 1
9'
- 2 0\
B -»
1 2
1
-?
3
Cb
1
- J
. 0
1
, 6 ,
, . . / 3 7 63 6 \
Lagrange m u l t i p l i e r s - » — - — —
\ 2 3 2 3 2 3/
S e n s i t i v i t y t o c o n s t r a i n t c o n s t a n t s
A b —>
1\
0 1 b b —»
.0/
[ A l ϋ i h.
A b b - »
7
"T M
3
3
T S
\
- b b/A b b -»
i
r 7 <
7
A b ->
1 b b - »
49
1 8 4
S' i h -
A b b -»
5
Ϊ 5
2
2 1
2
- b b/A b b ^
-#)
2 1
~ ~ T
1 ,
C h a p t e r 6 L i n e a r P r o g r a m m i n g
Ab -»
[ 0]
49
TM
3
1 8 4
0
b b - »
&
2 7
A b b -»
2 1
TS
l l;
2
TS
2
TS
- b b/A b b ■
49
- 1 - 1 ,
Allowable co ns trai nt RHS changes and ranges 2 ^ π 4
1
< b n < 9
- j < A b ^ < 7 ^ Ab2 < 1
- 1 < A b j <c o 0 < b 3 <o o S e n s i t i v i t y t o o b j e c t i v e f u n c t i o n c o e f f i c i e n t s
Ac
(1 ^
0
0
0
0
0
10/
&
r b Ί'
0
$
10
i - A l
r/tfb -»
r b ·
- r/r b -»
0
5
2 7
- f a
1 6 0
3 3
1
0
<
1
.
0
A c -»
0
0
r -»
»
r b - »
7
- r/r b - »
ψ
0
Ά ·
3
'- T S £'
, 1 6 j
0
2 3
( ~ 5 ^
f 2 0 ]
0
0
0
-
A c -»
1
0
0
r -»
3 7
6
r b - »
1 1
- r/r b -»
¥
2
7
Ac
( A )
0
1
52
- 2 0
0
0
0
—
0
1
r -»
ϋ
r b -»
TS
- r/r b - »
- ¥
0
0 j
~ T S ‘
3
A l l o w a b l e c o s t } c o e f f i c i e n t c h a n g e s a n d r a n g e s
Example 6.27 Plant operation In this example, we consider the solution of the tire manufacturing plant operations problem presented in Chapter 1. The problem statement is as follows:
A tire manufacturing plant has the ability to produce both radial and bias-ply automobile tires. During the upcoming summer months, they have contracts to deliver tires as follows.
Date
Radial tires
Bias-ply tires
June 30
5,000
3,000
July 31
6,0 0 0
3,000
August 31
4,000
' 5,000
Tbtal
15,000
1 1,0 0 0
The plant has two types of machines, gold machines and black machines, with appropriate molds to produce these tires. The following production hours are available during the summer months.
Month
On gold machines
On black machines
June
700
1,500
July
300
400
August
1,0 0 0
300
The production rates for each machine type and tire combination, in terms of hours per tire, are as follows.
Type
On gold machines
On black machines
Radial
0.15
0.16
Bias-Ply
0.1 2
0.14
The labor cost of producing tires is $10.00 per operating hour, regardless of which machine type is being used or which tire is being produced. The material
C h a p t e r fi T I n e a r P r o g r a m m i n g
cost for radial tires is $5.25 per tire and for bias-ply tires is $4.15 per tire. Finishing, packing and shipping cost is $0.40 per tire. The excess tires are carried over into the next month but are subject to an inventory-carrying charge o f $0.15 per tire. Wholesale prices have been set at $20 per tire for radials and $15 per tire for bias-ply.
How should the production be scheduled in order to meet the delivery requirements while maximizing profit for the company during the three-month period?
The optimization variables are as follows:
* 1
Number of radial tires produced in June on the gold machines
X2
Number of radial tires produced in July on the gold machines
X3
Number of fadial tires produced in August on the gold machines
XA
Number of bias-ply tires produced in June on the gold machines
X5
Number of bias-ply tires produced in July on the gold machines
X
6
Number of bias-ply tires produced in August on the gold machines
X7
JNUJilDCr OX rflQlfll tlT6 $ prOQUCCCl 111 JU116 Oil Txl6 DloCK JllaCillllQS
* 8
Number of radial tires produced in July on the black machines
X9
Number of radial tires produced in August on the black machines
* 1 0
Number of bias-ply tires produced in June on the black machines
* 1 1
Number of bias-ply tires produced in July on the black machines
* 1 2
Number of bias-ply tires produced in August on the black machines
The objective o f the company is to maximize profit. The following expressions used in defining the objective function were presented in Chapter 1.
s a l e s = 2 0 ( x x + x 2 + x 3 + *7 + x 8 + * 9) + (x 4 + *5 * x « * x 10 * X11 + x i a ) f
m a t e r i a l s C o s t = 5 » 2 5 (Xj^ + x 2 + x3 + **7 + x e + x 9) + 4.1 5 ( x 4 + x 5 + x 6 + x 10 + + *12)'
l a b o r C o a t = 1 0 ( 0.1 5 + X j + x 3 ) + 0 .1 6 (x7 + x e + x 9) + 0.1 2 ( x 4 + x 5 + x 6 ) + 0.1 4 ( x 10 +
x l l + x i 2 ) ) t
h a n d l l n g C o s t = 0.4 0 { χ -l + x 2 + x 3 + x 4 + x 5 + x 6 + x 7 + a t 8 + x 9 + x 10 + x ^ + x i 2) ·
i n v e n t o r y C o s t = 0,1 5 { ( Xj ^ + x 7 - 5 0 0 0 ) + ( x 4 + x 1 0 " 5 0 0 0 ) + ( Xj ^ + x 2 + x7 + x 8 " 1 1{>0 0 ) + ( x 4 + x 5 + x 10 + x x l - 6 0 0 0 ) ) j
6.8 S e n s i t i v i t y A n a l y s t U s i n g t h e R e v i s e d S i m p l e x M e t h o d
The production hour limitations are expressed as follows:
p r o d u c t i o n L i m i t a t i o i i S = { O.l S X i + 0.1 2 x 4 s 7 0 0, 0.1 5 x 2 + 0.1 2 x 5 ί 3 0 0, 0.1 5 x 3 +
0 . 12x 6 £ 1000, 0 . 16x 7 + 0 .14x10 £ 1500, 0 .16xe + 0 .1 4 * ^ £ 400, 0 . 16xg + 0 .14x12 £ 300};
Del i very contract constrai nts are wri t t en as follows:
a e l i v e r y C o n e t r a i n t s = { x 1 + x 7 i 5 0 0 0,x 4 + x l o & 3 0 0 0, x 1 -hx2 + x 7-t-xe & 1 1 0 0 0,
Xl O + *!! £ 6 0 0 0, Χχ+Χ2
+lt3 +x7 + χ β +x9 == 1 5 0 0 0, x 4 + x 5 + Xg + x 10 + x i x + x 12 = = 1 1 0 0 0 } ;
Thus, the problem is stated as follows. Note that by multiplying the profit with a negative sign, the problem is defined as a minimization problem.
v a r s = T a b l e [ X i, { i, 1/1 2 } ] ;
f = - ( s a l e s - ( m a t e r i a l s C o s t + l a b o r C o s t + h a n d l i n g C o s t + i n v e n t o r y C o s t ) ) ; g = J o i n [ p r o d u c t i o n L i m i t a t i o n s, d e l i v e r y C o n s t r a i n t s ];
The solution is obtained by using the RevisedSimplex function as follows:
R e v i s e d S i m p l e x f f, g, v a r a, S e n s i t i v l t v A n a l v s l · -» T r u e, P r i n t L e v e l -*
1, M a x l t e r a t i o n s - » 3 0 ];
S l a c k/s u r p l u s v a r i a b l e s - » {x13, x 1 4, x 1 5, x i 6, X 17, xis^ X 1 9' x 20' x 2 l' * 22} A r t i f i c i a l v a r i a b l e s { x 23 , x 24 / X 2 5' x 2 6' x 2 7' χ 2 δ }
M i n i m i z e - 1 3.2 5 x ^ - 13.25x2 - 1 3.25x3 - 9.65x4 - 9.65xs - 9.65xg - 1 3.15x7 - 1 3 . 1 5 x 8 - 1 3 .15xg - 9 .45x ^0 - 9 . 45x1:l - 9 . 45x12
0. ISXj + 0.1 2 x 4 + x 13 == 700 0. 15x2 + 0. 12x 5 + x 14 == 300 0.1 5 x 3 + 0.1 2 x 6 + x ^5 == 1000 0. 16x7 + 0.14x10 + x 16 == 1500 0.16xa + 0.1 4 x 1;L + x 17 == 400 O.I 6X9 + 0.1 4 x 12 + x^g == 300 + x 7 - x ^9 + x 23 == 5000
S u b j e c t t o
x 4 + x 10 “ x 20 + x 24 =" 3 0 0 0 Xl + x 2 + x 7 + Xg - x21 + x25 == 11000 x 4 + x 5 + x 10 + x u - x 22 + x26 == 60° °
Xl + x2 + x 3 + x7 + Xg + Xg + x 27 == 15000 ,x4 + x 5 + x 6 + x^j + Xu + Χχ2 + x2g == 11000,
All v a r i a b l e s > 0
***** Optimum s ol ut i on a f t e r 18 i t e r a t i o n s * * * * *
Basis -> { { ( X l, χ3, χ4, χ5, χ6, χ7, x 8, χ9, χ16, χ18, x 19, x 2o} } }
Vari abl e val ues -»
{ { { x! -» 1 8 6 6.6 7, x2 -» 0., x 3 -» 2666. 67, x 4 -» 3500 ., x 5 -» 2 5 0 0., Xg 5 0 0 0., x7 6633 .3 3, Xg —> 2 5 0 0., x 9 -4 1333 .33, X]_q -» 0, x ^ 0 ·, X]_2 0 * } } }
f -* - 303853.
Lagrange mul t i pl i e r s
C h a p t e r 6 L i n e a r P r o g r a m m i n g
Lagrange m u l t i p l i e r s - »
( 0.6 6 6 6 6 7 0.6 6 6 6 6 7 0.6 6 6 6 6 7 0. 0. 0. 0. 0. 0. 0. 13 .15 9.5 7 )
S e n s i t i v i t y t o o b j e c t i v e f u n c t i o n c o e f f i c i e n t s A l l o wa b l e c o s t c o e f f i c i e n t changes and ranges
- 0.15 < Aci £ 0 . 0 . £ Ac2 £ oo
0. s Ac3 < 0. 01 0 . £ Ac4 s 0.1 2
- o o £ A C g £ 0 .
- a o < A C g < 0 .
0 . £ Ac7 s 0.1 5
- o o £ A C g < 0. - 0.1 s ACg £ 0. -0.12 £ A c u o £ oo - 0.1 2 < A o n 5 00
- 0.1 2 < Ac^2 - 00
- 13.4 s c x £ - 13.25 Ί 3.25 i Cj son - 13.25 < c3 £ -13.15 -9 .65 < c4 <-9.53 -oo £ Cc £ -9 . 65
- ® i c 6 ί - 9 .6 5 - 1 3 .1 5 s c 7 s - 1 3.
-00 £ Cg s - 1 3 .1 5 - 1 3.2 5 i c 9 s - 1 3.1 5 - 9.5 7 < c1(j < oo - 9 .5 7 £ c 1:l s oo - 9.5 7 < c12 < oo
S e n s i t i v i t y to co ns trai nt constants Allowable co ns trai nt RHS changes and ranges
-280. <Ab! < 995.
-280. £ Ab2 £ 60.
-81.25 < Ab3 < 200. ‘ -438.667 < Ab4 < oo -400. £Ab5 £ 960.
-86.6667 s Abg £ oo -oo s Ab7 < 3500.
-co < Abg < 500.
-541.667 £ Abg < 1333:33 -500. £Ab10 £ 1666 . 67 -1333.33 ^ Abu £ 541.666 -1666.67 <Ab12 £ 677.083
420. < ^ £ 1695.
20. £ b 2 < 360.
918.75 < b 3 < 1200.
1061.33 <b4 <oo 0. £ b 5 s 960.
213 .333 ί b 6 < oo -oo s b 7 s 8500.
-oo < bg £ 3500.
10458.3 s b g < 12333.3 5500. £ b 10 £ 7666.67 13666.7 £ b 1;L £ 15541.7
9333.33 £ b 12 £ 11677.1
Using the sensitivity analysis, we can answer the following types of questions, without actually solving the modified problem.
(a) What would be the company's profit if the labor cost goes up from $10 to $10.50 per hour?
With this change, the new labor cost and the objective function are as follows:
n e w L a b o r C o s t = 1 0. 5 (0 .1 5 ( x x + x 2 + * 3) + 0.1 6 ( x 7 + x 8 + x 9 ) + 0.1 2 ( x 4 + x 5 + X g ) +
0 . 1 4 ( X i o + * 1 1 + * 1 2 ) ) »
n e w f = - ( s a l e s - ( m a t e r i a l s C o s t + n e w L a b o r C o s t + h a n d l i n g C o s t + i n v e n t o r y C o s t ) ); E x p a n d [ n e w f ]
-13.175X-L - 13 .175x2 - 1 3.1 7 5 x 3 - 9 . 59x4 - 9.5 9 x 5 - 9.59x6 - 1 3.0 7 x 7 - 1 3.0 7 x 8 - 13 .0 7xg - 9.38x10 - 9. 38x1:l - 9 . 38x^2
6.8 S e n s i t i v i t y A n a l y a i a U s i n g t h e R e v i s e d S i m p l e x M e t h o d
Comparing the coefficients in the modified objective function with their allowable ranges, we see that all are within the limits. Thus, the revised profit can simply be computed by substituting the optimum values of variables into the new objective function.
n e w f/.{ X i -» 1 8 6 6.6 7,x 2 0 *' x 3 2 6 6 6.6 7, x 4 -» 3 5 0 0.,x 5 -» 2 5 0 0. , x 6 -»
5 0 0 0 . r R j 6 6 3 3 * 3 3, > 2 5 0 0., Xq 1 3 3 3.3 3, x^ ^ —> 0 , Χη η
-302016.
(b) Based on t he Lagrange mul ti pl i er val ues, c omme nt on what eff ect t he changes i n t he product i on hours available wi l l have o n t he company's profit.
From t he Lagrange mul t i pl i er val ues, we s e e that t he first three mul ti ­
pl i ers are posi ti ve whi l e t he next t hree are 0. The first t hree mul ti pl i ers correspond to constrai nts based o n producti on hours available o n the gold machi ne. The other t hree correspond to t he product i on hours avail­
able o n t he bl ack machi ne. Thi s means that as l ong as the changes are wi t hi n t he allowabl e limi ts, t he company's profit wi l l not change as a resul t o f changes i n t he production hours o f t he bl ack machi ne. Further­
more, s i nce t he first three Lagrange mul ti pl i ers are t he same, it means that changes i n the producti on hours o f t he gol d machi ne i n any mont h wi l l have t he same ef f ect o n t he profit.
(c) Assume that during June, t he gold machi ne breaks down, and thus the number o f product i on hours available reduces to 600. What ef f ect woul d thi s have on t he company's profit?
The si tuat i on represents reduci ng the right-hand si de o f g\ b y 100. From t he right-hand si de ranges, we s e e that t he al l owabl e range i s 420 < b i < 1695. The change i s allowable. Si nce t he Lagrange mul ti pl i ers for thi s constrai nt i s 2/3, t he n e w objecti ve funct i on val ue as a resul t o f this change woul d be as follows:
N e w/* = - 3 0 3 8 5 3. - 0.6 6 6 6 6 7 ( - 1 0 0 ) = - 3 0 3 7 8 7.
Thus, t he profit wi l l decrease b y about $67.
(d) Based o n t he Lagrange mul t i pl i er val ues, c omme nt o n what ef f ect the changes i n t he del i very schedul e wi l l have o n t he company's profit.
From t he Lagrange mul t i pl i er val ues, we s e e that t he o nl y two del i very constrai nts wi t h posi ti ve Lagrange mul ti pl i ers are t hose that represent t he total demand that mu s t b e met. Thus, reasonabl e changes i n del i very wi t hi n each mont h wi l l have no ef f ect on t he profit. However, i f for some
C h a p t e r 6 L i n e a r P r o g r a m m i n g
reason enough tires cannot be produced, then reduction in radial tire delivery will hurt the company more than that in the bias-ply tires. This conclusion is based on the fact that the former constraint has a larger Lagrange multiplier then the latter.
(e) Assume that because of a strike at the plant, total production o f Radial tires drops by 1000. What effect would this have on the company's profit?
The situation represents reducing the right-hand side of g n by 1000. From the right-hand side ranges, we see that the allowable range is 13666.7 < bn < 15541.7. The change is allowable. Thus, the new objec­
tive function value as a result o f this change would be as follows.
N e w/* = -303853. - 13.15(-1000) = -290703.
where 13.15 is the Lagrange multiplier associated with this constraint.
6.9 C o n c l u d i n g R e m a r k s
Since its introduction in the late 1940s, the simplex method has been one of the most widely used methods for solving large scale linear programming problems. The BasicSimplex and the RevisedSimplex functions discussed in this chapter are very useful for educational purposes because they provide in­
termediate computational details. They also perform sensitivity analysis that the built-in M athe mat i ca functions LinearProgramming, ConstrainedMin, and ConstrainedMax do not. Several public domain and commercial linear pro­
gramming packages are also available. The Optimization Tfechnology Center, a joint enterprise of Argonne National Laboratory and Northwestern Uni­
versity, maintains a World Wide Web page that contains a wealth of infor­
mation on the available software. Currently, the Web page for linear pro­
gramming software is located at http://www-c.mcs.anl.gov/home/otc/Guide/
SoftwareGuide/Categories/linearprog.html.
A lot o f books are available that cover the subject from a variety of different points of view. Readers interested in more details should consult a recent book coauthored by one of the original developers o f the method, Dantzig and Thapa [1997]. Other excellent sources for additional information are Avriel and Golany [1996], Bazarra, Jarvis, and Sherali [1990], Luenberger [1984], and Nash and Sofer [1996]. Those interested in practical applications and guidelines for forming a wide variety of linear programming problems should consult Kolman and Beck [1980], Nazareth [1987], Pannell [1997], and Rardin [1998].
6.1 0 P r o b l e m s
Standard LP Problem
For the following problems, write the problem statement in the standard LP form. Unless stated otherwise, all variables are restricted to be positive.
6.1. Minimize f = x\ +
3
x
2
— 2
x
3
Subject to xi — 2x2 — 2x3 > —2, and 2xi — 3x2 — X3 < —2.
6.2. Maximize z = —x\ + 2x2
Subject to xi — 2x2 + 2 > 0, and 2xi — 3x2 < 3.
6.3. Maximize z = x\ + 3x2 — 2x3 + X4
Subject to 3xj 4 - 2 x2 + X3 -I- X4 < 20, 2χχ 4- X2 + X4 = 10, and 5xi — 2x2 —
x3 4- 2 x4 > 3.
6.4. Minimize / — 3xi + 4x2 + 5xg + 6x4
Subject to 3xi + 4 x2 4- 5 x3 + 6 x4 > 20, 2xi + 3 x2 4- 4 x4 < 1 0, and 5xi — 6 x2 — 7 x3 + 8 x4 > 3. Variables x\ and X4 are unrestricted in sign.
6.5. Maximize z = — 13xj + 3x2 4- 5
Subject to 3xi 4- 5 x2 < 20, 2χι +X2 > 10, 5xi + 2 x2 > 3, and x\ + 2 x2 > 3. xi is unrestricted in sign.
Solution o f l i n e a r System o f Equations
Find general solutions o f the following system of equations using the Gauss- Jordan form.
6.6. Ttoo equations in five unknowns: xi — 2x2 — 2x3 + X4 = —2, and 2xi — 3x2 - X3 + X5 = - 2.
6.7. TWo equations in four unknowns: x i —2x2+2 = X3, and 2xi — 3x2 +X4 = 3.
6.8. Three equations in six unknowns: 3xi 4- 2 x2 4- X3 + X4 + * 5 = 2 0, 2xi 4- X2 4- X4 = 10, and 5xi — 2 x2 — X3 4- 2 x4 — xe = 3.
6.9. Three equations in seven unknowns: 3xi + 4x2 + 5x3 + 6x4 — xs = 20, 2xj 4- 3x2 + 4x4 4- xe = 10, and 5xi — 6x2 — 7x3 4- 8x4 — X7 = 3.
6.10. Four equations in six unknowns: 3xi + 5x2 + X3 = 20, 2χι +Χ2 —Χ4 = 10» 5xi 4- 2x2 — X5 = 3, and xi + 2x2 — xg = 3.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
Basic Solutions o f an LP Problem
For the following problems, find all basic feasible solutions. Draw the feasi­
ble region and mark points on the feasible region that correspond to these solutions. Unless stated otherwise, all variables are restricted to be positive.
6.11. Minimize / = —lOOxi — 80x2
Subject to 5xi 4- 3x2 <15 and xi 4- X2 < 4.
6.12. Maximize z = —xi 4- 2x2
Subject to xi — 2x2 4- 2 > 0 and 2xi — 3x2 < 3.
6.13. Maximize z = xi + 3x2
Subject to xj + 3x2 <10 and xi 4- 2x2 <10.
6.14. Maxi mi z e z = —13xi 4- 3x2 + 5
Subject to 3xi + 5x2 < 20, 2xi 4- X2 > 10, 5xi 4- 2x2 > 3, and xi 4- 2x2 > 3.
xi is unrestricted in sign.
6.15. Minimize / = — 3xi — 4x2
Subject to xi 4- 2x2 < 10, xi 4- X2 < 10, and 3xi + 5x2 < 20.
The Simplex Method
Find the optimum solution of the following problems using the simplex method. Unless stated otherwise, all variables are restricted to be positive. Verify solutions graphically for two variable problems.
6.16. Maximize z = xi + 3x2
Subject to xi 4- 4x2 < 1 0 and xi 4- 2x2 < 10.
6.17. Maximize z = — xj 4- 2x2
Subject to xi — 4x2 + 2 > 0 and 2xj — 3x2 < 3.
6.18. Minimize / = —2xi 4- 2x2 + X3 — 3x4
Subject to xi 4- X2 4- X3 4- X4 < 18, xi — 2x3 4- 4x4 < 12, xi 4- X2 < 18, and X3 4- 2x4 < 16.
6.19. Minimize / = —3xi — 4X2
Subject to xi 4- 2x2 < 10, xi 4- X2 < 10, and 3xi 4- 5x2 < 20.
6.1 0 P r o b l e m s
6.20. Minimize / = —ΙΟΟχι — 80x2
Subject to 5xi 4- 3x2 <15 and xi 4- X2 < 4.
6.21. Maximize z = —x\ 4- 2x2
Subject to xi — 2x2 + 2 > 0 and 2xi — 3x2 < 3.
6.22. Minimize / = xi 4- 3x2 — 2x3
Subject to xi — 2x2 — 2x3 > —2 and 2xi — 3x2 — X3 < —2.
6.23. Maximize z = xi + 3x2 — 2x3 + X4
Subject to 3xi 4- 2x2 4- X3 4- X4 < 20, 2xi 4- X2 + X4 = 10, and 5xi — 2x2 — X3 4- 2x4 > 3.
6.24. Minimize / = 3xi + 4x?> + 5x? + 6x4
Subject to 3xi 4- 4x2 + 5x3 + 6x4 > 20, 2xi + 3x2 4- 4x4 <10, and 5xi —
6x2 - 7x3 + 8x4 > 3. Variables xi and x4 are unrestricted in sign.
6.25. Minimize / = 13xi — 3x2 — 5
Subject to 3xi + 5x2 < 20, 2xi +xg >10, 5xi + 2*2 > 3, and x^ + 2x? > 3. xi is unrestricted in sign.
6.26. Minimize / = 5xi + 2x2 + 3x3 + 5x4
Subject to xi — X2 + 7x3 + 3x4 > 4, xi + 2x2 + 2x3 + X4 = 9, and 2xi +
3x2 4- X3 — 4x4 < 5.
6.27. Minimize / = —3xi + 8x2 — 2x3 + 4x4
Subject to xi — 2x2 + 4x3 4- 6x4 < 0, xi — 4x2 — X3 + 6x4 < 2, X3 < 3, and X4 > 3.
6.28. Minimize / = 3xi 4- 2x2
Subject to 2xi + 2x2 +X3 4-X4 = 10, 2xi — 3x2 + 2x3 = 10. The variables should not be greater than 10.
6.29. Maximize ζ = xi + X2 4- X3
Subject to xi 4- 2x2 < 2, X2—X3 < 3, and xi 4- xi + X3 = 3·
6.30. Minimize / = ~2xi + 5x2 + 3x3
Subject to xi — X2 — *3 < —3 and 2xi + x2 > 1.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
6.31. Hawkey e foods owns two types o f trucks. Ituck type I has a refrigerated capacity of 15 m3 and a nonrefrigerated capacity of 25 m3. Ituck type II has a refrigerated capacity of 15 m3 and nonrefrigerated capacity of 10 m3. One of their stores in Gofer City needs products that require 150 m3 o f refrigerated capacity and 130 m3 of nonrefrigerated capacity. For the round trip from the distribution center to Gofer City, TVuck type I uses 300 liters of fuel, while Truck type II uses 200 liters. Determine the number o f trucks o f each type that the company must use in order to meet the store's needs while minimizing the fuel consumption.
6.32. A manufacturer requires an alloy consisting o f 50% tin, 30% lead, and 20% zinc. This alloy can be made by mixing a number o f available alloys, the properties and costs of which are tabulated. The goal is to find the cheapest blend. Formulate the problem as an optimization problem. Use the basic simplex method to find an optimum.
Available alloys
A
η
η
π
F
i lUpCX llCo
Λ
Lead (%)
1 0
1 0
40
60
30
Zinc (%)
1 0
30
50
30
30
Tin (%)
80
60
1 0
1 0
40
Cost: ($/Ib alloy)
8.2
9.3
1 1.2
13
17
6.33. A company can produce three different types o f concrete blocks, iden­
tified as A, B, and C. The production process is constrained by facilities available for mixing, vibration, and inspection/drying. Using the data given in the following table, formulate the production problem in or- der to maximize the profit. Use the basic simplex method to find an optimum.
Blocks
A
B
C
Available
Mixing (hours/batch)
1
3
9
900
Vibration (hours/batch)
2
3
6
1,2 0 0 .
Inspection/Diying (hours/batch)
0.7
0.8
1
400
Profit: ($/batch)
7
17
30
6.1 0 P r o b l e m s
6.34. A mining company operates two mines, identified as A and B. Each mine can produce high-, medium-, and low-grade iron ores. The weekly demand for different ores and the daily production rates and operating costs are given in the following table. Formulate an optimization prob­
lem to determine the production schedule for the two mines in order to meet the weekly demand at the lowest cost to the company. Use the basic simplex method to find an optimum.
Weekly demand (tons)
Daily production
Ore grade
Mine A (tons)
Mine B (tons)
High .
1 2,0 0 0
2,0 0 0
1,0 0 0
Medium
8,0 0 0
1,0 0 0
1,0 0 0
Low
24,000
5,000
2,0 0 0
Operations cost ($/day)
2 1 0,0 0 0
170,000
6.35. Assignment of parking spaces for its employees has become an issue for an automobile company located in an area with harsh climate. There are enough parking spaces available for all employees; however, some employees must be assigned spaces in lots that are not adjacent to the buildings in which they work. The following table shows the distances in meters between parking lots (identified as 1, 2, and 3) and office buildings (identified as A, B, C, and D). The number of spaces in each lot, and the number of employees who need spaces are also tabulated. Formulate the parking assignment problem to minimize the distances walked by the employees from their parking spaces to their offices. Use the basic simplex method to find an optimum.
Distances from parking lot (m)
Spaces
available
Parking lot
Building A
Building B
Building C
Building D
1
290
410
260
410
80
2
430
350
330
370
1 0 0
3
310
260
290
380
40
* of employees
40
40
60
60
6.36. Hawkeye Pharmaceuticals can manufacture a new drug using any one of the three processes identified as A, B, and C. The costs and quantities o f ingredients used in one batch of these processes are given in the following table. The quantity of new drug produced during each batch o f different processes is also given in the table.
Ingredients used per batch (tons)
Quantity of drug
Process
Cost ($ per batch)
Ingredient I
Ingredient II
produced
A
$1 2,0 0 0
3
2
2
B
$25,000
2
6
5
C
$9,000
7
2
1
The company has a supply o f 80 tons o f ingredient I and 70 tons of ingredient II at hand and would like to produce 60 tons o f new drug at a minimum cost.
Formulate the problem as an optimization problem. Use the basic simplex method to find an optimum.
6.37. A major auto manufacturer in Detroit, Michigan, needs two types of seat assemblies during 1998 on the following quarterly schedule:
Type 1
Type 2
First Quarter
25,000
25,000
Second Quarter
35,000
30,000
Third Quarter
35,000
25,000
Fourth Quarter
25,000
. 30,000
Tbtal
1 2 0,0 0 0
1 1 0,0 0 0
The excess seats from each quarter are carried over to the next quarter but are subject to an inventory-carrying charge of $ 2 0 per thousand seats. However, assume no inventory is carried over to 1999.
The company has contracted with an auto seat manufacturer that has two plants: one in Detroit and the other in Waterloo, Iowa. Each plant can manufacture either type of seat; however, their maximum capacities and production costs are different. The production costs per
6.1 0 P r o b l e m s
seat and the annual capacity at each o f the two plants in terms o f number o f seat assemblies is given as follows:
Quarterly capacity (either type)
Production cost
Type 1
Type 2
a *
Detroit Plant
c5l),UUU
$225
$240
Waterloo Plant
35,000
$165
$180
The packing and shipping costs from the two plants to the auto manu­
facturer are as follows:
Cost/100 seats
Detroit Plant
$ 1 0
Waterloo Plant
$80
Formulate the problem to determine a seat acquisition schedule from the two plants to minimize the overall cost of this operation to the auto manufacturer for the year. Use the basic simplex method to find an optimum.
6.38. A small company needs pipes in the following lengths:
0.5 m 100 pieces
0.6 m 300 pieces
1.2 m 2 0 0 pieces The local supplier sells pipes only in the following three lengths:
4 m 6 m 8 m
After cutt i ng t he ne c es s a ry l engths, the exces s pi pe must be thrown away. The company obvi ousl y wants to mi ni mi ze t hi s waste. Formulate the probl em as a l i near programming probl em. Use the basi c si mpl ex met hod to fi nd an opt i mum.
6.39. Consider t he probl em o f fi ndi ng t he short est route be t we e n two ci ti es whi l e travel i ng o n a gi ven network o f available roads. The network i s shown i n Figure 6.10. The nodes represent ci t i es and t he l i nks are the roads that c o nne c t t hes e ci ti es. The di stances i n ki l omet ers al ong each
C h a p t e r 6 L i n e a r P r o g r a m m i n g
road are noted in the figure. Use the basic simplex method to find the shortest route.
FIGURE 6.10 A network diagram showing distances and direction of travel between cities.
Post-Optimality Analysis
For the following problems, determine Lagrange multipliers and allowable ranges for constraint right-hand sides and objective function coefficients for which the basis do not change. Using this sensitivity analysis, determine the effect on the optimum of the following changes.
(a) A 10% increase (in absolute value) of the coefficient of the first variable in the objective function.
(b) A 10% increase (m absolute value) of the constant term m the first con­
straint.
6.40. Maximize z ~ X\ + 3x
2
Subject to x\ + 4X2 <10 and x\ + 2x
2
<10.
6.41. Maxi mi z e z = — x\ + 2x
2
Subject to x i - 4x2 + 2 > 0 and 2x\ — 3x
2
< 3.
6.42. Minimize / = —2xi ·+- 2*2 + * 3 - 3^4
Subject to Xj + x
2
+ * 3 + * 4 < 18, xi — 2
x$ + 4 x4 < 12, x\ + x
2
< 18, and
X3 + 2x 4 < 1 6.
6.1 0 P r o b l e m s
6.43. Minimize / = —3xi — 4x
2
Subject to x i + 2
x
2
< 10, x\ + x
2
< 10 and 3xi + 5x
2
< 20.
6.44. Minimize / = -lOOxi - 80^2
Subject to 5*i + 3 X2 < 15 and xi 4- X2 < 4.
6.45. Maximize z = —x\ + 2x
2
Subject to χχ — 2 x2 + 2 > 0 and 2xi — 3 X2 < 3.
6.46. Minimize / = X\ + 3 X2 — 2 x3
Subject to xi - 2 x2 - 2 x3 > - 2 and 2xi - 3 x2 - X3 < - 2.
6.47. Maximize z = x 1 + 3 x'2 — 2 x3 + X4
Subject to 3xi + 2x2 + X3 + xa < 20, 2xi 4- *2 + X4 — 10, and 5xi — 2x2 —
x3 + 2x4 > 3.
6.48. Minimize / = 3xi + 4x2 4- 5 x3 + 6 x4
Subject to 3xi 4- 4 x2 4- 5 x3 4 - 6 x4 > 20, 2xi + 3 x2 4- 4x4 < 10, and 5xi -
6 x2 — 7 x3 + 8 x4 > 3. Variables xi and X4 is are unrestricted in sign.
6.49. Minimize / = 13xi — 3 x2 — 5
Subject to 3xi 4- 5 x2 < 20, 2χι +X2 > 10, 5xi + 2 x2 > 3, and Xi + 2 x2 > 3.
xi is unrestricted in sign.
6.50. Minimize / = 5xj + 2x
2
+ 3 x3 + 5 x4
Subject to xi - X2 + 7 X3 4- 3 X4 > 4, xi + 2 x2 + 2 x3 + X4 = 9, and 2xi 4-
3X2 + Xi - 4X4 < 5.
6.51. Minimize f = — 3xj + 8 x2 — 2 x3 + 4 x4
Subject to xi — 2 x2 + 4 x3 4 - 6 x4 < 0, xi — 4 x2 — X3 4- 6 x4 < 2, X3 < 3, and X4 > 3,
6.52. Minimize / = 3xj + 2 x2
Subject to 2xi + 2 x2 + X3 + X4 = 10, 2xi — 3 x2 + 2x3 = 10. The variables should not be greater than 1 0.
6.53. Maximize z = x\ + x
2
+ X3
Subject to xi + 2 x2 < 2, x2 — X3 < 3, and Xi + X
2
+ X
3
= 3.
6.54. Minimize / = —2xi + 5x2 + 3 x3
Subject to xi — X2 - X3 < —3 and 2xi 4 - X2 > 1 ·
C h a p t e r 6 L i n e a r P r o g r a m m i n g
6.55. Using sensitivity analysis, determine the effect o f following changes in the Hawkeye food problem 6.31.
(a) The required capacity for refrigerated goods increases to 160 m3.
(b) The required capacity for nonrefrigerated goods increases to 140 m3.
(c) Gas consumption o f truck type I increases to 320 gallons.
(d) Gas consumption o f truck type II decreases to 180 gallons.
6.56. Using sensitivity analysis, determine the effect of following changes in the alloy manufacturing problem 6.32.
(a) The cost of alloy B increases to $10/Ib.
(b) The tin content in the alloy produced is decreased to 48% and the zinc content is increased to 2 2 %.
6.57. Using sensitivity analysis, determine the effect of following changes in the concrete block manufacturing problem 6.33.
(a) The profit from block type A increases to $ 8 and that from block type C decreases to $28.
(b) The number o f available mixing hours increases to 1,000.
(c) The number of available inspection/drying hours increases to 450,
6.58. Using sensitivity analysis, determine the effect o f following changes in the mining company problem 6.34.
(a) The operations cost at the mine B increase to $180,000 per day.
(b) The weekly demand for medium grade ore increases to 9,000 tons.
(c) The weekly demand for low grade ore decreases to 22,000 tons.
6.59. Using sensitivity analysis, determine the effect of following changes in the parking spaces assignment problem 6.35.
(a) The number o f employees working in building B increases to 45.
(b) The number o f spaces available in parking lot 3 increases to 45.
(c) The number o f spaces available in parking lot 2 decreases to 90.
6.60. Using sensitivity analysis, determine the effect o f following changes in the drug manufacturing problem 6.36.
(a) The cost per batch of process C increases to $10,000.
(b) The supply o f ingredient Π increases to 75 tons.
6.1 0 P r o b l e m s
6.61. Using sensitivity analysis, determine the effect o f following changes in the auto seat problem 6.37.
(a) The production costs at the Waterloo plant increase to $175 and $185, respectively, for the two seat types.
(b) In the third quarter, the demand o f type II seats increases to 27,000.
(c) The shipping cost from the Detroit plant increases to $15.
Revised Simplex Method
Find the optimum solution o f the following problems using the revised simplex method. Unless stated otherwise, all variables are restricted to be positive. Verify solutions graphically for two-variable problems.
6.62. Maximize Z — X i + 3 x2
Subject to Xi + 4 x2 < 10 and Xi 4- 2 x2 <1 0.
6.63. Ma x i mi z e z = — X\ + 2 x2
Subj e c t t o x i — 4 x2 4- 2 > 0 a nd 2xj — 3 x2 < 3.
6.64. Mi n i mi z e / = — 2xj + 2 x2 + X3 — 3 X4
Subj e c t t o Xi + X2 + X3 + X4 < 18, Xi — 2 x3 4- 4 x4 < 12, %\ + X
2
< 18, and X3 + 2 x4 < 16.
6.65. Mi n i mi z e f = —3χχ — 4x2
Subj e c t t o x i + 2 x2 < 10, x i 4- X2 < 10, a n d 3xi + 5 x2 < 20.
6.6 6. Mi n i mi z e / = —ΙΟΟχι — 8 OX2
Subj e c t t o 5xj + 3 X2 < 1 5 a nd Xi 4- X2 < 4.
6.67. Ma x i mi z e z = —x\ + 2 x2
Subj e c t t o x i — 2 x2 + 2 > 0 a nd 2xi — 3 x2 < 3.
6.6 8. Mi n i mi z e / = x i + 3x2 — 2 x3
Subj e c t t o x i — 2 x2 - 2 x3 > — 2 a n d 2xi — 3 x2 — X3 < —2.
6.69. Ma x i mi z e z = x 1 + 3 X2 — 2 x3 4- X4
Subj e c t t o 3xi + 2x2 4- X3 + * 4 < 20, 2xi + X2 + X4 = 10, a nd 5xi - 2 x2 - X
3
+ 2 x4 > 3.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
6.70. Minimize / = 3*i + 4 x2 4- 5 x3 + 6 x4
Subject to 3xi 4- 4x2 4- 6 x3 4- 6 x4 > 20, 2xj + 3 x2 + 4 x4 <1 0, and 5xi —
6 x2 — 7 X3 4- 8 x4 > 3. Variables x\ and X4 are unrestricted in sign.
6.71. Minimize / = 13xi — 3x2 — 5
Subject to 3xi 4 - 5x2 < 20, 2χι +X2 > 10, 5xi 4- 2x2 > 3, and X\ 4 - 2x2 > 3. Xi is unrestricted in sign.
6.72. Minimize / = 5xi + 2x2 4- 3 x3 4- 5 x4
Subject to xi — X2 + 7 x3 4- 3 X4 > 4, xi + 2x2 4- 2 x3 4 - * 4 = 9, and 2xi 4 -
3 x2 + X3 — 4 x4 < 5.
6.73. Minimize / = - 3 x i 4 - 8 x2 - 2 x3 4 - 4 x4
Subject to χχ — 2x2 + 4x3 4- 6X4 < 0, Xi - 4x2 - X3 + 6x4 < 2, X3 < 3, and X4 > 3.
6.74. Minimize / = 3xi + 2x2
Subject to 2xi 4- 2 x2 4 -X3 4 -X4 = 10 and 2xi — 3 x2 4- 2 x3 = 10. The variables should not be greater than 1 0.
6.75. Maximize z = x 1 + X2 4- *3
Subject to xi 4 - 2 x2 < 2, x2 - X3 < 3, and xi 4- X2 + * 3 = 3.
6.76. Minimize / = — 2xi 4- 5 X2 4- 3 x3
Subject to Xi — X2 — X3 < —3 and 2χι + X2 > 1.
6.77. Use the revised simplex method to solve the Hawkeye food prob- lem 6.31.
6.78. Use the revised simplex method to solve the alloy manufacturing prob­
lem 6.32.
6.79. Use the revised simplex method to solve the concrete block manufac­
turing problem 6.33.
6.80. Use the revised simplex method to solve the mining company prob­
lem 6.34.
6.81. Use the revised simplex method to solve the parking spaces assignment problem 6,35.
6.82. Use the revised simplex method to solve the drug manufacturing prob-
l e m 6.36.
6.1 0 P r o b l e m s
6.83. Use the revised simplex method to solve the auto seat problem 6.37.
6.84. Use the revised simplex method to solve the stock-cutting problem 6.38.
6.85. Use the revised simplex method to solve the network problem 6.39.
For the following problems, use the revised simplex method to determine Lagrange multipliers and allowable ranges for constraint right-hand sides and objective function coefficients for which the basis does not change. Using this sensitivity analysis, determine the effect on the optimum o f the following changes.
(a) A 10% increase (in absolute value) of the coefficient of the first variable in the objective function.
(b) A 10% increase (in absolute value) of the constant term in the first con­
straint.
6.8 6. Maximize z — X\ + 3
x
2
Subject to x\ + 4
x
2
< 1 0 and X\ + 2
x
2
<1 0.
6.87. Ma x i mi z e z = —x\ + 2
x
2
Subj e c t t o x\ — 4 * 2 + 2 > 0 a n d 2 x
1
— 3
X
2
< 3.
6.8 8. Mi n i mi z e / = —2 χ Ί + 2
x
2
+ X3 — 3x4
Subj e c t t o X\ + Xz + Xz + X4 < 18, xi — 2 x3 *+- 4 x4 < 12, X\ - f x z < 18, and X3 + 2x4 <16.
6.8 9. Mi n i m i z e / = —3χχ — 4 x2
S u b j e c t t o χχ + 2 x2 < 1 0, x i + x z < 1 0 a n d 3 x i + 5 x2 < 2 0.
6.9 0. Mi n i m i z e / = —ΙΟΟχι — 8 OX2
S u b j e c t t o 5 x i + 3 x2 < 1 5 a n d Xi + x2 < 4.
6.9 1. Ma x i mi z e z — —X\ + 2 x2
S u b j e c t t o Xi — 2 x2 + 2 > 0 a n d 2 x i — 3 X2 < 3.
6.9 2. M i n i m i z e / = Xi + 3 X2 — 2 x3
S u b j e c t t o x i - 2 x2 - 2 x3 > —2 a n d 2 x i - 3 X2 - X3 < —2.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
6.93. Maximize z = x\ + 3x2 — 2 x3 + x
4
Subject to 3xi + 2 x2 + X3 + X4 <2 0, 2xi + x2 + x 4 = 10, and 5xi — 2 x2 -
X3 + 2 x4 > 3.
6.94. Minimize / = 3xi + 4x2 + 5x3 + 6 x4
Subject to 3xi + 4x2 + 5 x3 + 6 x4 > 20, 2xi + 3x2 + 4 x4 <1 0, and 5xi —
6 x2 — 7 x3 + 8 x4 > 3. Variables xi and X4 are unrestricted in sign.
6.95. Minimize / = 13xi — 3x2 — 5
Subject to 3xi + 5x2 < 20, 2xi + x2 > 10, 5*i + 2x2 > 3, and x\ + 2x2 > 3.
Xi is unrestricted in sign.
6.96. Minimize / = 5xi + 2x2 + 3 x3 + 5 x4
Subject to Xi — x2 + 7 x3 + 3 x4 > 4, xi + 2x2 + 2 x3 + X4 = 9, and 2x\ +
3x2 + X3 — 4 x4 < 5.
6.97. Minimize / — —3xi + 8 x2 — 2 x3 + 4 x4
Subject to xj — 2x2 + 4x3 + 6x4 < 0, Xi — 4x2 - X3 + 6X4 < 2, X3 < 3, and x4 > 3.
6.98. Minimize / = 3xi + 2x2
Subject to 2xi + 2x2 +X3 +X4 = 1 0 and 2xi — 3x2 + 2 x3 = 10. The variables should not be greater than 1 0.
6.99. Maximize z — + x2 + X3
Subject to Xi + 2x2 < 2, x2 - X3 < 3, and xi + x2 + = 3.
6.100. Minimize / = —2xi + 5x2 + 3 x3
Subject to xi — x2 — x3 < —3 and 2xi + x2 > 1.
6.101. Using sensitivity analysis based on the revised simplex method, deter­
mine the effect o f following changes in the Hawkeye food problem 6.31.
(a) The required capacity for refrigerated goods increases to 160 m3.
(b) The required capacity for nonrefrigerated goods increases to 140 m3.
(c) Gas consumption of truck type I increases to 320 gallons.
(d) Gas consumption o f truck type II decreases to 180 gallons.
6.1 0 P r o b l e m s
6.102. Using sensitivity analysis based on the revised simplex method, de­
termine the effect of following changes in the alloy manufacturing problem 6.32.
(a) The cost o f alloy B increases to $10/Ib.
(b) The tin content in the alloy produced is decreased to 48% and the zinc content is increased to 22%.
6.103. Using sensitivity analysis based on the revised simplex method, deter­
mine the effect of following changes in the concrete block manufactur­
ing problem 6.33.
(a) The profit from block type A increases to $8 and that from block type C decreases to $28.
(b) The number o f available mixing hours increases to 1,000.
(c) The number of available inspection/drying hours increases to 450.
6.104. Using sensitivity analysis based on the revised simplex method, de- termine the effect of following changes in the mining company prob­
lem 6.34.
(a) The operations cost at the mine B increase to $180,000 per day.
(b) The weekly demand for medium grade ore increases to 9,000 tons.
(c) The weekly demand for low-grade ore decreases to 22,000 tons.
6.105. Using sensitivity analysis based on the revised simplex method, deter­
mine the effect of following changes in the parking spaces assignment problem 6.35.
(a) The number of employees working in building B increases to 45.
(b) The number of spaces available in parking lot 3 increases to 45.
(c) The number of spaces available in parking lot 2 decreases to 90.
6.106. Using sensitivity analysis based on the revised simplex method, deter­
mine the effect of following changes in the drug manufacturing prob­
lem 6.36.
(a) The cost per batch of process C increases to $10,000.
(b) The supply of ingredient II increases to 75 tons.
6.107. Using sensitivity analysis based on the revised simplex method, deter- mine the effect of following changes in the auto seat problem 6.37.
C h a p t e r 6 L i n e a r P r o g r a m m i n g
(a) The production costs at the Waterloo plant increase to $175 and $185, respectively, for the two seat types.
(b) In the third quarter, the demand of type II seats increases to 27,000.
(c) The shipping cost from the Detroit plant increase to $15.
CHAPTER SEVEN
I n t e r i o r P o i n t M e t h o d s
The simplex method starts from a basic feasible solution and moves along the boundary of the feasible region until an optimum is reached. At each step, the algorithm brings only one new variable into the basic set, regardless of the total number of variables. Thus, for problems with a large number of variables, the method may take many steps before terminating. In fact, relatively simple examples exist in which the simplex method visits all vertices of the feasible region before finding the optimum.
This behavior of the simplex method motivated researchers to develop an­
other class of methods known as interior point methods for solving linear programming (LP) problems. As the name implies, in these methods, one starts from an interior feasible point and takes appropriate steps along descent directions until an optimum is found. The following figure contrasts the ap­
proach used in the simplex method with the one used in the interior point methods.
Since most interior point methods make direct use of Karush-Kuhn-TUcker optimality conditions, the first section presents a special form taken by these conditions for LP problems. This section also presents the dual of an LP prob­
lem and examines the relationship between the primal and the dual variables. The remaining sections present two relatively simple but computationally ef­
fective Interior point methods. The first method is known as the primal affine scaling method. The second method is known as the primal-dual method and
C h a p t e r 7 I n t e r i o r P o i n t M e t h o d s
Optimum point
Starting point
FIGURE 7.1 'typical search paths in the simplex and interior point methods.
is emerging as an effective alternative to the simplex method for large-scale problems.
7.1 O p t i m a l i t y C o n d i t i o n s f o r S t a n d a r d L P
Consider a linear programming problem written in the standard LP form as follows:
Minimize c rx
Subject to ( A^ o )
where x is an η χ 1 vector of optimization variables, c is an η χ 1 vector containing coefficients of the objective function, and A is an m χ n matrix of constraint coefficients.
7.1.1 T h e K T C o n d i t i o n f o r a P r i m a l L P P r o b l e m
Following the presentation in Chapter 4, the Lagrangian for the standard LP problem is written as follows:
L(x, u, v, 8) = cTx + u r ( - x + s2) + v T( - A x + b)
7.1 O p t i m a l i t y C o n d i t i o n s f o r S t a n d a r d L P
where u is an η χ 1 vector of Lagrange multipliers associated with positivity constraints, 8 is a vector of slack variables, and v is an m χ 1 vector of Lagrange multipliers associated with equality constraints. The necessary conditions for the minimum of the Lagrangian results in the following system of equations:
( c — u — A Tv = 0 ^
Ax — b = 0
— X + 8 2 = 0 5j«i = 0, i = 1,.,., n \U i > 0, i = l,...,n/
T h e f i r s t t w o s e t s o f e q u a t i o n s a r e l i n e a r, w h i l e t h e o t h e r t w o a r e n o n l i n e a r. H o w e v e r, w e c a n e l i m i n a t e t h e s l a c k v a r i a b l e s 8 b y u s i n g t h e c o m p l e m e n t a r y s l a c k n e s s c o n d i t i o n s u,s,· = 0, a s f o l l o w s.
( a ) I f Ui = 0, then s,· > 0. The third condition then implies that —x\ < 0 or Xi > 0. Thus when u,· = 0, *,· > 0.
(b) If Si = 0, then u, > 0. The third condition implies that —X{ — 0. Thus when Uf > Ό, Xj = 0.
Thus, t he vari abl e s; i n t he compl ement ary sl ackness condi t i ons can be re­
pl aced by Xi. The optimum of the LP problem can therefore be obtained by solving the following system of equations:
f c - u — Ar v = 0 ^
__________________________________Ax - b — 0__________________________________
XiUt = 0, i = 1,..., n \Xi > 0, Ui > 0, i = 1,..., n)
It i s conveni ent to express t he c ompl ement ai y s l ackness condi t i ons i n the matri x form as wel l. For t hi s purpose, we def i ne n x n diagonal matrices:
U = diag[u,·] X = diagfo]
Further defining a n n x l vector e, whose entries are all equal to 1, i.e.,
eT = ( l,l,..■,!)
The c o mp l e me n t a r y s l a c kn e s s c o ndi t i o ns are wr i t t e n as f ol l ows:
X i U j = 0, i = 1,..., n = » X U e = 0
C h a p t e r 7 I n t e r i o r P o i n t M e t h o d s
7.1.2 D u a l L P P r o b l e m
Following the general Lagrangian duality discussion in Chapter 4, it is possible to define an explicit dual function for an LP problem as follows:
M ( n, v) = Min[cTx — u Tx + r T (—A x + b)] «,· > 0, i = 1
,..., n
X
The minimum over x can easily be computed by differentiating with respect to x and setting it equal to zero. That is,
Min[cTx — u rx + v T (—Ax + b)] =>■ c — u — Arv = 0
X
Thus, the dual LP problem can be stated as follows:
Maximize cTx—u Tx 4- vr (—Ax + b)
Subject to ( c - « - A r v = o\
\U j > 0, i = 1,..., n j
the terms as follows:
c rx —u rx + v r (—Ax 4 - b) =$> x T(c—u —A rv) + v rb
Using the constraint, the term in the parentheses vanishes; thus, the dual LP problem can be stated as follows:
Maximize v rb
Subject to Ui > 0, i = 1,..., n
Furthermore, using the constraint equation, the Lagrange multipliers u can be written in terms of v as follows:
c — u — Α Γν = 0 ==Φ u = c —A rv
Thus, the dual LP problem can be written entirely in terms of Lagrange mul­
tipliers for the equality constraints as follows:
Maximize v Tb Subject to c - A rv > 0
Note that the variables in the dual problem are the Lagrange multipliers of the primal problem. Also, as mentioned in Chapter 4, the maximum o f the dual problem is the same as the minimum of the primal problem only i f feasible
7.1 O p t i m a l i t y C o n d i t i o n s f o r S t a n d a r d L P
solutions exist for both the primal and the dual problems. The relationship does not hold i f either the primal or the dual problem is infeasible.
Example 7.1 Construct the dual of the following LP problem. By solving the primal and the dual problems, demonstrate the relationship between the variables in the two formulations.
Minimize f = 6x\ — 9x2
^3xi + 7x2 < 1 5 ^
Subject to
/3xi + 7x2 < 15\ I X j + * 2 > 3 J Vx, > 0, i = 1, 2/
Introducing slack and surplus variables, the LP in the standard form is as follows:
M i n i m i z e 6 x ^ - 9 x 2
/3 x! + 7 x 2 + x 3 ==■ 1 5\
Subj ect t o l X l + X 2 _ X 4 = = 3 A l l v a r i a b l e s £ 0
For the matrix form, we can identify the following matrices and vectors:
6
3 7 1 0 \ /1 5\ - 9
I r ^
o .0 }
A_> i l 1 0 - 1/ ‘T ' \ 3
Subs t i t ut i ng t h e s e i n t o t h e g e n e r a l dual LP f or m, w e g e t t h e f o l l o wi n g dual probl em:
V a r i a b l e s - » ( v! v 2 )
Maxi mi z e -»1 5 + 3v2
' 6 - 3 v! - v 2 > 0 * 9 - l v 1 - v 2 i 0
- v l > 0
v 2 a 0
Slab j e c t t o -
Gr aphi cal s o l u t i o n s o f t h e pr i mal a nd t h e dua l p r o b l e ms are s h o w n i n Fi g­
ure 7.2. As s e e n f r om t h e f i r s t graph, t h e pr i ma l pr o b l e m i s f e a s i bl e, a nd h a s t he mi n i mu m v a l u e o f —4.5 at x\ = 1.5 and x2 = 1.5. The graph o f the dual problem shows that the dual is also feasible and has a maximum value of —4.5 at v\ = —3.75 and i/2 = 17.25. Thus, both the primal and the dual give the same optimum value.
The solutions can be verified by solving the primal problem directly by using the KT conditions as follows.
C h a p t e r 7 I n t e r i o r P o i n t M e t h o d s
x2 v2
FIGURE 7.2 Graphical solutions of the primal and the dual problems.
K T S o l u t i o n [ f, { 1 5 - 3 »! - 7 x 2 " x3 == 0 r 3 - - x 2 + x4 = = 0/
T a b l e [ X i 2: 0, { 1, 1, 4 } ] }, T a b l e [ x ±/ { 1, 1, 4 > ] ] ;
* * * * * L a g r a n g i a n -> ( s i - x ^ + 6 xx + u2 (S2 - x2 ) “ 9x2 + u3 (s 3 - * 3 )
+ v x ( 1 5 - 3 x x - 7 x 2 - x 3 ) + u 4 (34 - x 4 ) + v 2 ( 3 - x 2 - x 2 + x 4 )
* * * * * v a l i d KT P o i n t ( s ) * * * * *
f -> - 4.5 X i -* 1.5 x 2 -* 1.5 x 3 -> 0 x 4 -> 0
Ui -> 0
u 2 - > 0 u 3 - + 3.7 5 u 4 - > 1 7.2 5 s i - > 1.5 s l - > 1.5 s 3 -> 0
S4 —> 0
- > - 3.7 5 V2 - 4 3,7 .2 5
T h e La g r a n g e m u l t i p l i e r s o f t h i s p r o b l e m c o r r e s p o n d t o t h e o p t i m u m s o l u ­
t i o n o f t h e d u a l v a r i a b l e s. N o t e t h a t t h e r e i s n o s i g n i f i c a n c e t o t h e s i g n o f v m u l t i p l i e r s. I n t h i s e x a mp l e, t h e c o n s t r a i n t e x p r e s s i o n s w e r e w r i t t e n i n s u c h a w a y t h a t w e c a m e u p w i t h t h e s a m e s i g n s. I f w e h a d w r i t t e n t h e e q u a l i t y c o n ­
s t r a i n t s b y m u l t i p l y i n g b o t h s i d e s b y a n e g a t i v e s i g n, w e w o u l d h a v e o b t a i n e d s a m e n u m e r i c a l v a l u e s, b u t t h e s i g n s w o u l d h a v e b e e n o p p o s i t e.
7.1 o p t i m a l i t y C o n d i t i o n s f o r S t a n d a r d L P
The BormDualLP Function
A Mathematica function called FormDualLP has been created to automate the process of creating a dual LP.
N e e d s I"O p t i m i z a t i o n T o o l b o x'I n t e r i o r P o i n t'"];
?F o r m D u a l L P
FormDualLP[f, g, vars, options]. Forms dual o f the given LP. f i s the objective function, g i s a l i s t of c o n s tr ai n t s, and var s i s a l i s t of v a r i a b l e s. See Options[FormDualLP] to f i n d out about a l i s t o f v a l i d options fo r t h i s function.
O p t i o n s U e a g e [ F o r m D u a l L P ]
{UnrestrictedVariables -» {} , ProblemType Min,
StandardVariableName -> x, DualLPVariableName v}
UnrestrictedVariables i s an option fo r LP and several QP problems.
A l i s t of v a r i a b l e s that are not r e s t r i c t e d to be p o s i t i v e can be s p e ci f i e d with t h i s option. Default i s {}.
ProblemType i s an option for most optimization methods. I t can ei th er be Min (def a u l t ) or Max.
StandardVariableName i s an option fo r LP and QP methods. I t s p e c i f i e s the symbol to use when creating v a r i a b l e names during conversion to the standard form. Default i s x.
DualLPVariableName i s an option fo r dual i n t e r i o r point methods. I t defines the symbol to use when creating dual v a r i a b l e names (Lagrange m u l t i p l i e r s fo r e q u a l it y c o n s t r a i n t s ). Default i s v.
Example 7.2 Construct the dual of the following LP problem. Demonstrate graphically that the primal is feasible but the solution is unbounded. The corresponding dual problem has no solution.
Minimize f = —Zx\ - X2
£ = -3XJ - X 2 ;
ϊ = { X i - 2 x 2 s 5 }; v a r s = { x ^ x 2 > j
Using the FormDualLP, the dual problem can easily be written as follows:
{ d £, d g, d v } = F o r m D u a l L P [ £, 5, v a r s ];
P r i m a l p r o b l e m
Minimize - - x2
Subject t o (χ! - 2x2 +X3 == 5)
A l l v a r i a b l e s ϊ 0
7.2 T h e P r i m a l A f f i n e S c a l i n g M e t h o d
7.1.3 S u m m a r y o f O p t i m a l i t y C o n d i t i o n s f o r S t a n d a r d L P
The previous discussion can be summarized into the following set of optimality conditions. Usual names associated with these conditions are indicated as well.
7.2 T h e P r i m a l A f f i n e S c a l i n g M e t h o d
The Primal Affine Scaling (PAS) method is one of the simplest interior point methods. The basic idea is to start with a point in the interior of the feasible region. Scale variables so that the point is near the center of the transformed domain. Determine a descent direction based on the maximum possible reduc­
tion in the objective function. Compute an appropriate step length along this direction so that the next point maintains feasibility of variables (i.e., none of the variables becomes negative). Repeat the process until the optimality conditions are satisfied.
The following example is used to illustrate these ideas:
Minimize / = — 5xj — xi
Introduci ng sl ack vari abl es x$ and X4, the problem is written in the standard LP form as follows:
Minimize / = — 5x\ — x%
Ax — b = 0 Pri mal f easi bi l i t y
A Tv + u = c Dual feasibility
XUe — 0 Complementary slackness conditions
Xi > 0, U{ > 0, i = 1,..., n
In matri x not at i on, t he probl em i s wri t t en as fol l ows:
C h a p t e r 7 I n t e r i o r P o i n t M e t h o d s
Minimize cTx subject to Ax = b and x > 0 ---------------- Cr - ( - 5.- 1,0,0 ) A - g \ I ° ) b = Q -------------
Tb start the process, we need an initial point that satisfies all constraint condi­
tions. A procedure to determine the initial interior point will be presented later. For the time being, we can use trial-and-error to determine a point that satis­
fies the constraint equations. For the example problem, we use the following point:
x° = {1/2, 2, 5, 5}r
Since all values are positive and Ax° — b = 0, this is a feasible solution for the problem.
7.2.1 S c a l i n g T r a n s f o r m a t i o n
The first step in the PAS method is to scale the problem so that the current point is near the center of the feasible domain. A simple scaling that moves all points a unit distance away from the coordinates is to divide the variables by their current values. By defining an n x n transformation matrix T* whose diagonal elements are equal to the current point, this scaling transformation is written as follows:
y k — (τ*)-1χ or χ = Tky k
where
( A
0
0
o\
0
0
0 \
rjJf _
0
A
0
0
(T*)_1 =
0
i/A
0
0
u
0
0
A)
I 0
0
0
i f a )
The superscri pt k refers to the iteration number. The scaled problem can then be written as follows:
Minimize c TTky k
Subject to AT*yfc = b and y > 0
Introduci ng t he not at i on ck — T*c and Ak = AT*, the scaled LP problem is as follows:
Minimize (ck)Ty k
Subject to A ky k = b and y > 0
For the example problem, the scaled problem is developed as follows:
r^k __
(\ 0 0 0\ 0 2 0 0 0 0 5 0
( 2 0 0 0\ 0 ^ 0 0 0 0 i 0
v> ο o y
I 0 1)
Vo ο o s y
c t = T‘c = { - 5,- 2,o,o j T and y* = ( t *) 1 x = {a*. | J:
The scaled problem is
Minimize —2, 0,0}
Y2
Ϊ 3
W
Subj ect to 2 o s ) y l = (?)
w
and yi > 0, i = 1,..., 4
Si nce t he pr obl e m h a s f our vari abl es, we o bv i o us l y c a nno t di r e c t l y pl ot a graph o f t hi s s c a l e d pr obl e m. Howe ver, i f we i g no r e t he s l a c k vari abl es, we c a n s e e t hat t he or i gi nal pr o bl e m i s n o w t r ans f or me d as f ol l ows:
Mi ni mi z e — § y i — 2y2
(
y i + 6y 2 < 1 2\ y i + 2y 2 < 8 J yi > 0 ,i = 1, 2/
The graphs of the original and scaled problems are shown in Figure 7.4. The current point in the original space (*i = 1/2 and x2 = 2) is mapped to y\ = 2x\ = 1 and y2 = xi/2 = 1. It is clear from this graph that the scaling changes the shape of the feasible region and the current point is a unit distance away from the coordinates.
C h a p t e r 7 I n t e r i o r P o i n t M e t h o d s
FIGURE 7.4 Graphs of the original and the scaled problems.
7.2.2 D i r e c t i o n o f D e s c e n t
After scaling, the next step is to determine a descent direction. A suitable direction is the one along which the objective function reduces rapidly. At the same time, it must be possible to move in this direction by a reasonable amount (step length) and still remain feasible. The first goal is achieved by moving in the negative gradient direction of the objective function, since this represents the steepest descent direction. The feasibility will be maintained if we move in the null space of matrix A*. (See the appendix of this chapter for a review of concepts of null space and range space from linear algebra.) Thus, a feasible descent direction is the projection of the negative gradient of the objective function on the null space.
d* = P*(—cfc)
where ck is the gradient of the objective function, and P* is an η χ n projection matrix. The following formula for this projection matrix is derived in the appendix.
Pi = I - A ^ ( A fcAfcrr 1Afc
whe r e I i s an η χ n identity matrix. Thus,
d* = 1 [ i — Afcr(AfcAkT) _ 1Afc] = - [c* - A*r (A*(A*A*r) - 1 A*c*]
7.2 T h e P r i m a l A f f i n e S c a l i n g M e t h o d
Defining w k = (A* Afcr) _1 A* ck, we can express the direction as
dk = - [ c k - A kT w*]
Substituting the scaling transformation ck = T*c and A* = A l we can express w* and dfc in terms of the original problem variables as follows:
w* = ( A T *!’* A ^ A T ^ T ^ c
dfc = —[T*c — T* A T w*] — —'T^c — A T w*] = —T* r*
where r* = c — A r w*, A is the original constraint coefficient matrix, and c is the cost coefficient vector. The definition of vector w* indicates matrix inversion. In actual implementations, however, it is more efficient to compute the vector w* as follows.
w* = (AkAkT)~1Akck or (A* AkT)w* = A* ck
Thus, w* can be obtai ned by s ol vi ng the f ol l owi ng l i near s ys t em o f equati ons:
( A T * T * A r ) w* = A T * T * c
For the exampl e probl em, t he di rect i on i s comput ed as fol lows:
* * - (!; 1 1 > ·,- ι - ι - “ ·"Γ
^ - ( S £ ) ^ ‘ - ( ~!) ________________________________
Solving the system of equations (A* A ^ w * = Ak ck, we get
w*=L-Z2L _J29_1T
I 3,382’ 3,382 J
The descent direction is then as follows:
d* = - t c 1 - AlIV ] = -
( S/2\
•Ψ
fl 1\ 6 2 5 0
/ 701 \
I 3,382 1
\ 429 / “ \ 3,382/
/ 2.16587 > 0.5026611 -1.03636
\ U / \U 5/
^—0.6342400/
C h a p t e r 7 I n t e r i o r P o i n t M e t h o d s
7.2.3 S t e p L e n g t h a n d t h e N e x t P o i n t
After determining the feasible descent direction, the next step is to determine the largest possible step in this direction. The equality constraints have already been accounted for during the direction computations. The restriction that the variables remain positive is taken into account in determining the step length. Thus, we need to determine a such that the following condition is satisfied:
y * + 1 = y * + a d k > 0
Since at the current point, the scaled variables yi are all equal to 1, the step length is restricted as follows:
a df > —1, i = 1,..., n
I f al l el e ment s o f the di rect i on vect or are posi ti ve, t hen obvi ousl y, we can take any s t ep i n that di rect i on wi t hout becomi ng i nf easi bl e. Thi s si tuati on means that the probl em i s unbounded and there i s no ne e d to proceed any further. Thus, the actual st ep l engt h i s det ermi ned by the negat i ve ent ri es i n t he d* vector and therefore, st ep l engt h i s gi ven by
a < { - 1/df, df <Q,i = l,
T h i s s t e p l e n g t h w i l l ma k e a t l e a s t o n e o f t h e v a r i a b l e s g o t o z e r o. I n o r d e r t o s t a y i n s i d e t h e f e a s i b l e r e g i o n, t h e a c t u a l s t e p t a k e n s h o u l d b e s l i g h t l y s ma l l e r t h a n t h i s m a x i m u m. T h u s, a is set to fiamax with 0 < β <1. Usually, β = 0.99 is chosen in order to go as far as possible but without actually being on the boundary of a constraint.
For the example problem, using the descent direction computed in the previous step, the step length is computed as follows:
1/d* = {0.461706,1.98941, -0.964907, -1.57669}1 «max = Min[—1 /df, df < 0, i = 1,..., n] = min {0.964907,1.57669}r = 0.964907 With β = 0.99, the step length is
a = 0.99 x 0.964907 = 0.955258 With this step length, in terms of scaled variables, the next point is,as follows:
y * +1 = y * + a d *
Using the scaling transformation, we can express the next point in terms of actual problem variables as follows:
xfc+i — T* (y* + ad k) = x* + aT*dfc For the example problem, the next point therefore is as follows:
x d) =
^ 1/2 ^ 2
Λ
V 5 )
+ 0.955258
(\ 0 0 ON 0 2 0 0 0 0 5 0
\0 0 0 5)
( 2.16588 > 0.502661 -1.03637
\—0.63424^
/1.53449^
2.96034
0.05
\1.97068^
7.2.4 C o n v e r g e n c e C r i t e r i a
Starting with this new point, the previous series of steps is repeated until an optimum is found. Theoretically, the optimum is reached when dfc = 0. Thus, we can define the first convergence criteria as follows:
σι = Normfd*] < €\
where €\ is a small positive number.
In addition to this, because of the presence of round-off errors, the numerical implementations of the algorithm also check the following conditions derived from the KT optimality conditions.
Feasibility
The constraints must be satisfied at the optimum, i.e., Ax* — b = 0. Ib use as convergence criteria, this requirement is expressed in a normalized form, as follows:
HAx*— b |[
(To = :-------------- <
I |b| | + 1 - 2
where is a small positive number. The 1 is added to the denominator to avoid division by small numbers.
Reduced Cost Coefficients
From the revised simplex method, we know that the vector r* ξ c — A Twk represents the reduced cost coefficients appearing in the last row of the simplex
C h a p t e r 7 I n t e r i o r P o i n t M e t h o d s
tableau. Therefore, the nonnegativity of the reduced cost coefficients gives the following convergence criteria:
σ 3 = --------------- < €3
l l c l l + 1 - 3
w h e r e e3 is a small positive number. Again, a 1 is added to the denominator to avoid division by a small number.
Primal-Dual Solutions
The vector w k is related to the Lagrange multipliers that are the variables in the dual LP problem. Thus, another convergence criterion is the difference between the primal solution and the dual solution, as follows.
<74 = Abs[c* x* — h T w*] < €4
where e4 is a small positive number.
7.2.5 F i n d i n g t h e I n i t i a l I n t e r i o r P o i n t
In order to use the Primal Affine Scaling algorithm, we need to start from an interior feasible point. Tb achieve this, similar to the simplex method, we define a Phase I problem during which the goal is to find an initial interior point. Choose an arbitrary starting point x° > 0, say x° = {1,1,..., 1}. Then from the constraint equations, we have
z° = b—Ax°
If z° = 0, we have a starting interior point. If not, we introduce an artificial variable and define a Phase I LP problem as follows:
Minimize a
(
Ax + a z = b
x > 0 « > 0
T h e m i n i m u m o f t h i s p r o b l e m i s r e a c h e d w h e n a = 0, and at that point, Ax* = b, which makes x* an interior point for the original problem. Furthermore, if we set α = 1, then any arbitrary x° becomes a starting interior point for the
phase I problem. Thus, we apply the PAS algorithm to the Phase I problem until a = 0 and then switch over to the actual problem for Phase II.
Note that in the PAS algorithm, Phase I needs only one artificial variable. In contrast, recall from Chapter 6 that Phase I of the simplex method needs as many artificial variables as the number of equality or greater than type constraints.
7.2.6 F i n d i n g t h e E x a c t O p t i m u m S o l u t i o n
The exact optimum of an LP problem lies on the constraint boundary. How­
ever, by design, all points generated by an interior point algorithm are always slightly inside the feasible domain. Therefore, even after convergence, we only have an approximate optimum solution. An exact optimum solution can be ob­
tained if we can identify the basic variables for the vertex that is closest to the interior optimum point. The optimum is then computed by setting the non­
basic variables to zero and solving for the basic variables from the constraint equations. This is known as the purification procedure.
It can be shown that the magnitude of diagonal elements of matrix
Ρ = Τ Α Γ(Α Τ2 Ar)-1 A T
serve as an indicator of the basic variables. Using this matrix, a procedure for identifying basic variables from the approximate interior point optimum is as follows:
1. Define a diagonal scaling matrix T with the diagonal elements set to the converged optimum solution from the PAS method.
2. Compute matrix Ρ = TAT(AT2 AT)_1AT. Set vector p to diagonal ele­
ments of this matrix.
3. Elements of vector p that are close to 1 correspond to basic variables and those that are dose to zero define nonbasic variables. In case the elements of vector p do not give clear indication, repeat calculations of step 2 by defining a new diagonal scaling matrix T with diagonal elements set to vector p. The procedure is stopped when the correct number of basic variables has been identified. A solution of constraint equations in terms of these basic variables then gives the exact vertex optimum.
C h a p t e r 7 I n t e r i o r P o i n t M e t h o d s
7.2.7 T h e C o m p l e t e P A S A l g o r i t h m
The complete Primal Affine Scaling (PAS) algorithm can be summarized in the following steps.
Phase II—Known Interior Starting Point
Given: Constraint coefficient matrix A, constraint right-hand side vector b, objective function coefficient vector c, current interior point xfc, step-length parameter β, and convergence tolerance parameters.
The next point x*+1 is computed as follows:
1. Form scaling matrix.
0 0 0 \
\0 0 0 x */
2. Solve the system of linear equations for w*.
(AT*T*Ar)wk = ATfcT*c
3. Comput e r* = c — A r w*
4. Che c k f or c o nv e r g e nc e.
I f [ Nor m[ dfc] < e l t < e2, flipr - «3, Abs[crx* - bTw*] < e4], we
have the optimum. Go to the purification phase. Otherwise, continue.
5. Compute direction d* = — T*r*.
6. Compute step length a = β Min[—l/iff, df < 0, i = 1,..., n].
7. Comput e the next poi nt x *+1 = x * + aTk d k.
Phas e I —Tb Fi nd I n i t i a l I nt e r i o r Po i n t
Wi th the addi ti on o f an arti fi ci al variable, t he l engt h o f t he sol ut i on vector i n t hi s phase i s n + 1. Initially, all entries in the solution vector are arbitrarily set to 1. At each iteration, the artificial objective function vector is defined as c = {0, 0,..., 0, x£+1}, a column z k = Ax*—b is appended to the matrix A, and
the steps of the Phase II algorithm are repeated until
7.2 T h e P r i m a l A f f i n e S c a l i n g M e t h o d
Purification Phase
After obtaining the interior point optimum from Phase II, use the following
steps to get an exact vertex solution:
1. Set diagonal scaling matrix T to the elements of the converged interior optimum solution.
2. Compute matrix Ρ = Τ Α τ ( A T2 A 7 ) - 1 AT. Set vector p to diagonal elements of this matrix.
3. Elements of vector p that are close to 1 correspond to basic variables, and those that are close to zero define nonbasic variables. In case the elements of vector p do not give clear indication, repeat calculations of step 2 by defining a new scaling matrix T with diagonal elements set to vector p.
4. Set the nonbasic variables to zero and solve the constraint equations Ax = b for the optimum values of basic variables.
The following PrimalAffineLP function implements the Primal Affine algo­
rithm for solving LP problems. The function usage and its options are explained first. The function is intended to be used for educational purposes. Several in­
termediate results can be printed to gain understanding of the process.
H e e d s ["O p t i m i z a t i o n T o o l b o x'I n t e r i o r P o i n t'"];
?P r i m a l A f f i n e L P
P r i m a l A f f i n e L P [ f, g, v a r s, o p t i o n s ]. S o l v e s a n L P p r o b l e m u s i n g
P r i m a l A f f i n e a l g o r i t h m, f i s t h e o b j e c t i v e f u n c t i o n, g i s a l i s t o f c o n s t r a i n t s, a n d v a r s i s a l i s t o f v a r i a b l e s. S e e
O p t i o n s [ P r i m a l A f f i n e L P ] t o f i n d o u t a b o u t a l i s t o f v a l i d o p t i o n s f o r t h i s f u n c t i o n.
O p t i o n s U s a g e [ P r i m a l A f f i n e L P ]
{ U n r e s t r i c t e d V a r i a b l e s -* {} , M a x l t e r a t i o n s - > 2 0, P r o b l e m T y p e -» M i n, S t a n d a r d V a r i a b l e N a m e -» x, P r i n t L e v e l 1, S t e p L e n g t h F a c t o r - > 0.9 9, C o n v e r g e n c e T o l e r a n c e - > { 0.0 0 1,0.2, 2, 0.5 } , S t a r t i n g V e c t o r - * { } }
U n r e s t r i c t e d V a r i a b l e s i s a n o p t i o n f o r LP a n d s e v e r a l QP p r o b l e m s.
A l i s t o f v a r i a b l e s t h a t a r e n o t r e s t r i c t e d t o b e p o s i t i v e c a n b e s p e c i f i e d w i t h t h i s o p t i o n. D e f a u l t i s {}.
M a x l t e r a t i o n s i s a n o p t i o n f o r s e v e r a l o p t i m i z a t i o n m e t h o d s. I t s p e c i f i e s m a x i m u m n u m b e r o f i t e r a t i o n s a l l o w e d.
P r o b l e m T y p e i s a n o p t i o n f o r m o s t o p t i m i z a t i o n m e t h o d s. I t c a n e i t h e r b e
M i n ( d e f a u l t ) o r M a x.
C h a p t e r 7 I n t e r i o r P o i n t M e t h o d s
S t a n d a r d V a r i a b l e N a m e i s a n o p t i o n f o r LP a n d QP m e t h o d s. I t s p e c i f i e s t h e s y m b o l t o u s e w h e n c r e a t i n g v a r i a b l e n a m e s d u r i n g c o n v e r s i o n t o t h e s t a n d a r d f o r m. D e f a u l t i s x.
P r i n t L e v e l i s a n o p t i o n f o r m o s t f u n c t i o n s i n t h e O p t i m i z a t i o n T o o l b o x. I t i s s p e c i f i e d a s a n i n t e g e r. T h e v a l u e o f t h e i n t e g e r i n d i c a t e s h o w m u c h i n t e r m e d i a t e i n f o r m a t i o n i s t o b e p r i n t e d. A P r i n t L e v e l - > o s u p p r e s s e s a l l p r i n t i n g. D e f a u l t f o r mo s t f u n c t i o n s i s s e t t o 1 i n w h i c h c a s e t h e y p r i n t o n l y t h e i n i t i a l p r o b l e m s e t u p. H i g h e r i n t e g e r s p r i n t m o r e i n t e r m e d i a t e r e s u l t s.
S t e p L e n g t h F a c t o r i s a n o p t i o n f o r i n t e r i o r p o i n t m e t h o d s. I t i s t h e r e d u c t i o n f a c t o r a p p l i e d t o t h e c o m p u t e d s t e p l e n g t h t o m a i n t a i n f e a s i b i l i t y. D e f a u l t i s 0.9 9
C o n v e r g e n c e T o l e r a n c e i s a n o p t i o n f o r m o s t o p t i m i z a t i o n m e t h o d s. M o s t m e t h o d s r e q u i r e o n l y a s i n g l e z e r o t o l e r a n c e v a l u e. Some i n t e r i o r p o i n t m e t h o d s r e q u i r e a l i s t o f c o n v e r g e n c e t o l e r a n c e v a l u e s.
S t a r t i n g V e c t o r i s a n o p t i o n f o r s e v e r a l i n t e r i o r p o i n t m e t h o d s. D e f a u l t i s { 1, . . . , 1}.
E x a mp l e 7.3 The complete solution of the example problem used in the previous section is obtained here using the PrimalAffineLP function.
£ = - 5 x 1 - x 2;
g = { 2 x 1 +■ 3 x 2 £ 1 2, 2 x 1 + x 2 £ 8 }; v a r s = { x l, x 2 };
All calculations for th6 first two iterations are as follows:
P r i m a l A f f i n e L P [ f, g, v a r s, P r i n t L e v e l -+ 2,
S t a r t i n g V e c t o r -» { 0.5, 2., 5 -, 5.}, M a x l t e r a t i o n s -> 2 ];
M i n i m i z e - 5X-L - x 2
_ . . /2 x ·. + 3 x 2 + X3 = = 1 2\
S u b j e c t t o ,/ * J
\ 2x 1 · *· χ 2 + X4 = = 8 j
A l l v a r i a b l e s > 0
P r o b l e m v a r i a b l e s r e d e f i n e d a s: { x l - > x ^, x 2 -> x 2 } ,
'- 5'
- 1 1 12
0 [ 8
0
b - >
A ->
2 3 1 0 2 1 0 1
S t a r t i n g p o i n t - > { 0.5,2., 5.,5.}
O b j e c t i v e f u n c t i o n - > - 4.5 S t a t u s - > N o n O p t i m u m
* * * * * I t e r a t i o n 1 ( P h a s e 2 ) * ’
T k [ d i a g o n a l ] - > { 0.5,2.,5.,5.}
A.T k.T k.c -»
r -»
- 14.5
- 6.5
w-
- 0.2 0 7 2 7 4\ - 0.1 2 6 8 4 8/
- 4.3 3 1 7 6 '
’ 2.1 6 5 8 8 1
- 0.2 5 1 3 3 1
d - »
0.5 0 2 6 6 1
0.2 0 7 2 7 4
- 1.0 3 6 3 7
.0.1 2 6 8 4 8 ,
,- 0.6 3 4 2 4,
C o n v e r g e n c e p a r a m e t e r s - » {2 .53378, 0 ., 0.712547, 0 .99793} jS (-1/d) -» (-0.457089,-1.96952, 0.955258, 1.56092} S t e p l e n g t h, q-»0.955258 Ne w p o i n t -» {1.53449, 2.96034, 0.05, 1.97068} O b j e c t i v e f u n c t i o n -» -10.6328 S t a t u s - » N o n O p t i m u m
* * * * * i t e r a t i o n 2 ( P h a s e 2 ) * * * * *
T k [ d i a g o n a l ] - » ( 1.5 3 4 4 9, 2.9 6 0 3 4, 0.0 5, 1.9 7 0 6 8 }
mn .π, /Θ 8.2 9 3 7 3 5.7 0 9 5\
A.T k.T k.A T -» j 3 5 > 7 0 9 5 2 2.0 6 5 8 ]
A.T k.T k.c -»
- 4 9.8 3 7 4\
w ·
- 3 2 .3 1 0 1 j
0.0 8 0 3 3 6 2\ - 1.5 9 4 2 7 J
r - »
" - 1.9 7 2 1 3 ’ 0.3 5 3 2 6 2
d
3.0 2 6 2 1 ^ - 1.0 4 5 7 8
- 0.0 8 0 3 3 6 2 L 1.5 9 4 2 7 ,
0.0 0 4 0 1 6 8 1 , - 3.1 4 1 8 j
C o n v e r g e n c e p a r a m e t e r s - » ( 4 .4 8 5 8 2, 0., 0 .4 2 0 0 1 7, 1.1 5 7 3 5 } β ( - 1/d ) -» ( - 0.3 2 7 1 4 2, 0.9 4 6 6 6 6,- 2 4 6.4 6 4,0.3 1 5 1 0 6 } S t e p l e n g t h, « - > 0.3 1 5 1 0 6 N e w p o i n t - » ( 2.9 9 7 7 4, 1.9 8 4 8 2, 0.0 5 0 0 6 3 3, 0.0 1 9 7 0 6 8 } O b j e c t i v e f u n c t i o n - 1 6.9 7 3 5 S t a t u s - » N o n O p t i m u m
* * * * * N o n O p t i i m i m s o l u t i o n a f t e r 2 i t e r a t i o n s * * * * *
I n t e r i o r s o l u t i o n - * ( x l 2 .9 9 7 7 4, x 2 -» 1.9 8 4 8 2 } O b j e c t i v e f u n c t i o n - » - 1 6.9 7 3 5
C o n v e r g e n c e p a r a m e t e r s - » ( 4.4 8 5 8 2, 0., 0.4 2 0 0 1 7, 1.1 5 7 3 5 }
The function is allowed to run until convergence.
{ s o l, h i s t o r y } = P r i m a l A f f i n e L P [ f, g, v a r s,
S t a r t i n g V e c t o r -+ ( 0.5, 2., 5., 5.},M a x l t e r a t i o n s -* 2 0 ];
* * * * * O p t i m u m s o l u t i o n a f t e r 6 i t e r a t i o n s * * * * *
I n t e r i o r s o l u t i o n -» ( x l ^ 3 .9 9 9 8 6, x 2 -> 0.0 0 0 1 9 6 0 5 4 } O b j e c t i v e f u n c t i o n - * - 1 9.9 9 9 5
C o n v e r g e n c e p a r a m e t e r s -»
{ 0.0 0 0 3 7 3 2 4 5, 9.0 6 3 2 2 χ 1 0"13 , 0.4 7 8 0 2 4, 0.0 0 0 5 2 3 8 9 2 }
N e a r e s t v e r t e x s o l u t i o n -» ( x l -» 4, x 2 -» 0 } O b j e c t i v e f u n c t i o n -» - 2 0 S t a t u s - » F e a s i b l e
C h a p t e r 7 I n t e r i o r P o i n t M e t h o d s
The history of points computed by the PAS method is extracted. Using Graph- icalSolution, this search path is shown on the graph.
x h i s t = T r a n s p o s e [ T r a n s p o s e [ h i s t o r y ] [ [ { 1, 2 } ] ] ]; T a b l e F o r m [ x h i s t ]
0.5 2.
1.5 3 4 4 9 2.9 6 0 3 4
2.9 9 7 7 4 1.9 8 4 8 2
3.0 1 9 6 3 1.9 6 0 5 4
3.9 9 0 1 5 0.0 1 9 6 0 5 4
3.9 9 9 8 6 0.0 0 0 1 9 6 0 5 4
3.9 9 9 8 6 0.0 0 0 1 9 6 0 5 4
x2
FIGURE 7.5 A graphical solution showing a solution path.
Example 7.4 Solve the following LP problem using the Primal Affine Scaling method.
f = x l + x 2 +■ x 3 + x 4; g = { x l + 2 x 2 - x 3 + 3 x 4 £ 1 2, x l + 3 x 2 + x 3 + 2 x 4 £ 8,
2 x 1 - 3 x 2 - x 3 + 2 x 4 S 7 > ; v a r s = { x l, x 2, x 3, x 4 } ;
7.2 T h e P r i m a l A f f i n e S c a l i n g M e t h o d
Intermediate calculations are shown for the first two iterations. The Phase I procedure is started with an arbitrary starting point.
P r i m a l A f f i n e L P [ f, g, v a r s, P r o b l e m T y p e -+ M a x, P r i n t L e v e l -* 2,
M a x l t e r a t i o n s - > 2 ];
M i n i m i z e - ~ x 2 “ x 3 “ x 4
ί Χχ + 2x 2 - X 3 + 3x 4 + x 5 = = 12'
S u b j e c t t o Χχ + 3 x 2 + x 3 + 2 x 4 + x 6 == 8
2 χ ι - 3 x 2 - Χ 3 + 2x 4 + x 7 = = 7 ,
A l l v a r i a b l e s a- 0
P r o b l e m v a r i a b l e s r e d e f i n e d a s: { x l -» x ^, x 2 -+ x 2, x 3 -+ x 3, x 4 -> x 4 ]
i - 1 ^
b
-1
-1
-1
0
0
0
12\
8
7
1 2 - 1
3
1
0
o 1
A- >
1 3 1
2
0
1
0
,2 - 3 - 1
2
0
0
l j
S t a r t i n g p o i n t
—f
{ l w
1.
, 1.
{ 1., 1., 1., 1., 1., 1., 1.} O b j e c t i v e f u n c t i o n - » S t a t u s N o n O p t i m u m
* * * * * I t e r a t i o n 1 ( P h a s e 1 ) * * * * *
Tk [diagonal] - » { 1.,1.,1.,1.,1.,1.,1,
( 5 2. 12. 3 9.'
12. 16. - 4.
39. - 4. 5 5.j
A. T k. T k. AT -»
A.T k.T k.c -*
6 Λ 0. 1 6 -
’ 0.129032 ' - 0.0 9 4 0 8 6 ,0.0107527,
- 0.0 5 6 4 5 1 6
0.0564516
0.233871
- 0.2 2 0 4 3
- 0.1 2 9 0 3 2
0.094086
- 0.0 1 0 7 5 2 7
0.16129
0.0564516 - 0.0 5 6 4 5 1 6 - 0.2 3 3 8 7 1 0.22043 0.129032 - 0.0 9 4 0 8 6
0.0107527 - 0.1 6 1 2 9
Convergence pa r a me t e r s - * { 0.4 0 1 6 1, 0., 0.2 0 0 8 0 5, 0 .129032} β ( - 1/d) -> { - 1 7.5 3 7 1, 1 7.5 3 7 1, 4.2 3 3 1,- 4.4 9 1 2 2,
- 7.6 7 2 5, 1 0.5 2 2 3, - 92 .07, 6.1 3 8 }
Step lengt h, a-» 4.2331
New p o i n t - » { 1.2 3 8 9 7, 0.7 6 1 0 3 4, 0.0 1,
1.9 3 3 1, 1.5 4 6 2 1, 0.6 0 1 7 2 4, 1.0 4 5 5 2 } A r t i f i c i a l o b j e c t i v e funct ion -»0.317241
C h a p t e r 7 I n t e r i o r P o i n t M e t h o d s
O b j e c t i v e f u n c t i o n - » - 3 .9 4 3 1 S t a t u s N o n O p t i m u m
* * * * * i t e r a t i o n 2 ( P h a s e 1 ) * * * * *
Tk [ d i a g o n a l ] { 1.2 3 8 9 7, 0.7 6 1 0 3 4, 0.0 1, 1.9 3 3 1,
A.Tk.Tk.AT■
1.5 4 6 2 1, 0.6 0 1 7 2 4, 1.0 4 5 5 2, 0.3 1 7 2 4 1 }
4 0.2 3 9 2 2 7.4 3 1 3 2 2.3 8 1 1'
2 7.4 3 1 3 2 2.0 5 7 3 1 2.8 0 5
\2 2.3 8 1 1 1 2.8 0 5 2 7.7 5 8 1,
A. T k. T k. c -»
0.0 6 0 7 7 3 2.8 3 5 7 6 X l 0 ~ 17
0.0 6 0 7 7 3
w
0 .0 1 0 1 7 9 4 - 0.0 1 2 5 1 8 1 - 0.0 0 0 2 4 3 5 0 7
r -»
0.0 0 2 8 2 5 7 4 ' 0.0 1 6 4 6 5 1 0.0 2 2 4 5 4 1 - 0.0 0 5 0 1 4 9 6 - 0.0 1 0 1 7 9 4
d -»
'- 0.0 0 3 5 0 0 9 9 ■ - 0.0 1 2 5 3 0 5 - 0.0 0 0 2 2 4 5 4 1 0.0 0 9 6 9 4 4 4 0.0 1 5 7 3 9 5
0.0 1 2 5 1 8 1 0.0 0 0 2 4 3 5 0 7 , 0.2 9 8 3 2 9 ,
- 0.0 0 7 5 3 2 4 7 - 0.0 0 0 2 5 4 5 9 1 , - 0.0 9 4 6 4 2 3 ,
C o n v e r g e n c e p a r a m e t e r s - » { 0.0 9 7 5 9 6 1, 0.1 0 7 9 1 4, 0.2 2 7 8 3 6, 0.0 8 0 3 3 8 7 } jS ( - 1/d ) -» { 2 8 2.7 7 7, 7 9.0 0 7 3, 4 4 0 9., - 1 0 2.1 2,
- 6 2.8 9 9 1, 1 3 1.4 3 1, 3 8 8 8.5 9, 1 0.4 6 0 4 }
S t e p l e n g t h, a - » 1 0.4 6 0 4
New p o i n t - » { 1.1 9 3 5 9, 0.6 6 1 2 8 2, 0.0 0 9 9 7 6 5 1,
2.1 2 9 1 4, 1.8 0 0 7 8, 0.5 5 4 3 1 2, 1.0 4 2 7 3 }
A r t i f i c i a l o b j e c t i v e f u n c t i o n - » 0.0 0 3 1 7 2 4 1 O b j e c t i v e f u n c t i o n - » - 3 .9 9 3 9 9 S t a t u s - » N o n O p t i m u m
I n t e r i o r s o l u t i o n { x l -» 1.1 9 3 5 9, x 2 -» 0 . 6 6 1 2 8 2,
x 3 -» 0.0 0 9 9 7 6 5 1, x 4 -» 2.1 2 9 1 4 }
O b j e c t i v e f u n c t i o n -» 3 .9 9 3 9 9
C o n v e r g e n c e p a r a m e t e r s - » { 0.0 9 7 5 9 6 0 8, 0.1 0 7 9 1 4, 0.2 2 7 8 3 6, 0.0 8 0 3 3 8 7 }
Calculations are allowed to run until the optimum is found.
{ s o l, h i s t o r y } = P r i m a l A f f i n e L P [ f, g, v a r s, P r o b l e m T y p e -+ M a x ];
* * * * * o p t i m u m s o l u t i o n a f t e r 1 1 i t e r a t i o n s * * * * *
I n t e r i o r s o l u t i o n - » { x l -» 4 .5 6 4 1, x 2 h > 0 . 0 0 0 1 0 4 0 2 9,
x 3 -» 3 .4 3 5 3 6, x 4 -» 0.0 0 0 1 0 2 1 2 4 }
O b j e c t i v e f u n c t i o n -» 7.9 9 9 6 6
C o n v e r g e n c e p a r a m e t e r s { 0.0 0 0 2 3 3 3 3 4, 0.1 0 8 4 1 5, 0.8 1 6 4 9 7, 0.0 0 0 3 3 7 1 1 8 }
* * * * * F i n d i n g - n e a r e s t v e r t e x s o l u t i o n * * * * *
N e a r e s t v e r t e x s o l u t i o n -> { x l -» 5, x 2 -» 0, x 3 -» 3, x 4 -» 0} O b j e c t i v e f u n c t i o n - » 8 S t a t u s -» F e a s i b l e
T h e h i s t o r y o f p o i n t s c o m p u t e d i s a s f o l l o w s:
schist = Transpose [Transpose [history] [ [{1, 2,3,4}]]]; TableForm[xhist]
1.
1.
1.
1.
1.2389
0.761034
0.01
1.9331
1.19359
0.661282
0.00997651
2.12914
1.19359
0.661282
0.00997651
2.12914
2.85016
0.926912
0.0102189
1.17667
4.43891
1.1739
0.0103702
0.0117667
4.54767
1.1381
0.0113538
0.0107107
4.56279
1.13621
0.025652
0.000107107
4.56409
1.04029
0.314812
0.000102906
4.5641
0.0104029
3 .40446
0.000102625
4.5641
0.000104029
3.43536
0.000102124
4.5641
0.000104029
3-.43536
0.000102124
E x a m p l e 7.5 U n b o u n d e d s o l u t i o n
f = xl - 2x2;
g = {2x1 - x2 £; 0, -2x1 + 3x2 £ 6} ; vars = {xl, x2} ;
{sol, history} = PrimalAffineLP[f, g, vars, Maxlterations 10];
Minimize x-l - 2x2
, . / 2x-t - x2 - x3 == 0\
Subject t o „ -
-,
--------------------------------------------------------
\-2x! + 3x2 + X4 == 6
)
A l l v a r i a b l e s > 0
Pr o bl e m v a r i a b l e s r e d e f i n e d a s: { x 1 - » x;l, x2 - > x2 }
* * * * * Unbo unde d s o l u t i o n a f t e r 9 i t e r a t i o n s * * * * *
I n t e r i o r s o l u t i o n - » { x l - » 1.8 9 4 1 1 χ 1 0 6 8, x 2 - » 1.2 6 2 7 4 χ 1 0 6 8 }
Ob j e c t i v e f u n c t i o n - » - 6.3 1 3 7 χ 1 0 67
Co n v e r g e n c e p a r a me t e r s - > { 3.6 4 5 2 2 χ 1 0 6 7, 1.4 1 7 4 5 χ 1 05 3 ,
0.2 0 8 5, 6.3 1 3 7 X1 0 6 7 }
T h e h i s t o r y o f p o i n t s c o m p u t e d b y t h e P A S m e t h o d i s e x t r a c t e d, a n d t h e s e a r c h p a t h i s s h o w n o n t h e g r a p h.
x h i s t = Tr a n s p o s e [ Tr a n s p o s e [ h i s t o r y ] [ [ { 1, 2 } ] ] ]; Ta bl e Fo r m [ x h i s t ]
1. 1.
1.2 2 6 2 9 2.2 4 4 5 7
1.2 2 6 2 9 2.2 4 4 5 7
1.6 4 0 4 4 3.0 7 4 7
2.2 6 5 9 3.4 9 7 2 1
1 4 1 2.6 4 9 4 3.7 4 9
2.0 4 7 0 9 x l O1 1 1.3 6 4 7 2 x l O1 1
4.2 9 7 6 x 1 02 9 2 . 8 6 5 0 7 χ 1 02 9
1 - 8 9 4 1 1 x l O6 8 1.2 6 2 7 4 χ 1 06 8
1.8 9 4 1 1 χ 1 06 8 1.2 6 2 7 4 x 1 0^ 8
C hap ter 7 I n t e r io r Point Methods
-1 0 1 2 3 4 5
F I G U R E 7.6 A g r a p h i c a l s o l u t i o n s h o w i n g a s o l u t i o n p a t h.
E x a m p l e 7.6 Nb f e a s i b l e s o l u t i o n C o n s i d e r t h e f o l l o w i n g L P p r o b l e m:
f = 4x1 + x2 + x3 + 3x4; g = {2x1 + x2 + 3x3 + x4 z.
12,
3x1 + 2x2 4- 4x3 = = 5/
2x1 - x2 +
2x3 + 3x4 = - 8,
3x1 + 4x2 + 3x3 + x4 £ 16} ? vars = {xl, x2 , x3 , x4 } ;
PrimalAffineLP
[£,
g, vars, ProblemType -+ Max] ;
Minimize - 4X! - x 2 - X3 - 3x4
' 2xj + x 2 + 3x 3 + x 4 - x 5 == 12 '
, , 3x-i + 2xo + 4x? = = 5
Subject to 0 ^ e
2X! - x2 + 2x3 + 3x4 == 8
,3χ! + 4x 2 + 3x 3 + Χ4 - x 6 == 16,
All variables > 0
Problem variables redefined as; {xl -> χ1, χ2 -» x2, x3 -» X 3 , x4 -»x 4 }
Phase I ended with no feasible solution New point -»
{0.00459891, 3.98552, 0.00436602, 3.48881, 0.0000959518, 7.96127} Convergence parameters -4 {0.0004121, 0.0844611, 0.0875652, 0.197076} Constraint values norm-» 0 .311477 $Aborted
7.2 T he P r i m a l A f f i n e Scaling Method
T h e p r o b l e m h a s n o f e a s i b l e s o l u t i o n. T h e b u i l t - i n C o n s t r a i n e d M i n f u n c t i o n r e t u r n s t h e s a m e c o n c l u s i o n.
ConstrainedMin
[-£, g,
{xl, x2, x3, x4}]
ConstrainedMin::nsat: The specified constraints cannot be satisfied. ConstrainedMin[
-4x1 - x2 - x3 - 3x4,
{2x1 + x2 + 3x3 + x4 > 12, 3x1 + 2x2 + 4x3 = = 5,
2x1 - x2 + 2x3 + 3x4 == 8, 3x1 + 4x2 + 3x3 + x4 a 16}, {xl, x2, x3, x4} ]
E x a m p l e 7.7 T h e m e t h o d g e t s t r a p p e d a t a n o n o p t im u m p o i n t T h i s e x a m p l e d e m o n s t r a t e s t h a t, i n s o m e s i t u a t i o n s, t h e P A S m e t h o d g e t s t r a p p e d a t a n o n o p ­
t i m u m p o i n t
f = -xl - x2 + 2x4;
q
_ {3x1 + x2 + 3x3 + 2x4 == 10, xl - 3x2 +
2x3 £ 7, xl + 2x2 + 3x3 +
x4 £ 4} ; vars = {xl, x2, x3, x4} ;
PrimalAffineLP [f, g, vars, Maxlterations -» 20];
Minimize - x^ - x2 + 2x4
' 3χ^ + x2 + 3x3 + 2x4 == 10 i Subject to Xi - 3x2 + 2x3 + x5 == 7
,xi + 2x2 + 3x3 + x4 - x6 == A,
All variables > 0
Problem varicibles redefined as: {xl -» χι, x2 -» x2, x3 -»x3, x4 -* x4 }
***** Optimum solution after 10 iterations *****
Interior solution -» {xl -» 3 .01693, x2 0.720208,
x3 -> 0 .0000470621, x4 -> 0.0000687285}
Objective function -» -3 .737
Convergence parameters-»{0.000212405, 0.105786, 0.971915, 0.136996}
***** Finding nearest vertex solution *****
Γ 16 2 }
Nearest vertex solution -» jxi -» —, x2 -» —, x3 -» 0, x4 -» 0 j·
Objective function-*- —
Status -» Feasible
U s i n g t h e d e f a u l t s t e p - l e n g t h p a r a m e t e r β = 0.9 9, t h e o p t i m u m r e t u r n e d b y t h e P A S m e t h o d h a s a n / v a l u e o f - 1 8/5. A s i m p l e x s o l u t i o n r e t u r n s / = — 1 0, w h i c h i s t h e c o r r e c t s o l u t i o n. S e v e r a l t e c h n i q u e s h a v e b e e n p r o p o s e d t o o v e r c o m e t h i s d i f f i c u l t y. R e f e r t o r e c e n t b o o k s a n d a r t i c l e s o n a d v a n c e d l i n e a r p r o g r a m m i n g t o l e a m a b o u t t h e s e r e f i n e m e n t s. F o r t h i s e x a m p l e, t h o u g h, w e c a n g e t t h e c o r r e c t s o l u t i o n s i m p l y b y r e d u c i n g t h e s t e p l e n g t h p a r a m e t e r β t o
0.9 5.
PrimalAffineLP[f, g, vars, Maxlterations -*
20, StepLengthFactor -+ .95] ;
*****
optimum solution after 14 iterations *****
Interior solution{xl -> 0 . 000338359, x2 -* 9 .75706,
x3 -» 0.0000194353, x4 -> 0.0000445125}
Objective function-9.75731
Convergence parameters-»{0.000702174, 0.111818, 1.56115, 0.242692}
***** rinding nearest vertex solution *****
Nearest vertex solution -» {xl 0, x2 10, x3 -> 0, x4 -> 0}
Objective function -» -10 Status -» Feasible
7.3 The Primal-Dual I n terio r P o in t Method
A s s e e n f r o m s e c t i o n 7.1, t h e o p t i m u m o f t h e L P p r o b l e m c a n b e o b t a i n e d b y - s o l v i n g t h e f o l l o w i n g s y s t e m o f e q u a t i o n s:
A x — b = 0 P r i m a l f e a s i b i l i t y
A r v + u = c D u a l f e a s i b i l i t y
X U e ~ 0 C o m p l e m e n t a r y s l a c k n e s s c o n d i t i o n s
X i > 0 a n d «; > 0, i = 1,..., n
T h e p r i m a l - d u a l i n t e r i o r p o i n t m e t h o d i s b a s e d o n t r y i n g t o s o l v e t h e a b o v e s y s t e m o f e q u a t i o n s d i r e c t l y. B e c a u s e o f t h e c o m p l e m e n t a r y s l a c k n e s s c o n d i ­
t i o n s, t h e c o m p l e t e s y s t e m o f e q u a t i o n s i s n o n l i n e a r. W e c a n u s e a n i t e r a t i v e m e t h o d, s u c h a s t h e N e w t o n - R a p h s o n m e t h o d, t o s o l v e t h i s s y s t e m. T h e l i n e a r e q u a t i o n s w i l l b e s a t i s f i e d e x a c t l y a t e a c h i t e r a t i o n. T h e e r r o r i n s a t i s f y i n g t h e c o m p l e m e n t a r y s l a c k n e s s c o n d i t i o n s h o u l d g e t s m a l l e r a s t h e i t e r a t i o n s p r o g r e s s. U s i n g μ > 0 a s a n i n d i c a t o r o f t h i s e r r o r, a t a g i v e n i t e r a t i o n, t h e c o m p l e m e n t a r y s l a c k n e s s c o n d i t i o n s a r e o f t h e f o l l o w i n g f o r m:
X U e = μ β
7.3.1 D i r e c t i o n U s i n g t h e N e w t o n - R a p h s o n M e t h o d
R e c a l l f r o m C h a p t e r 3 t h a t e a c h i t e r a t i o n o f t h e N e w t o n - R a p h s o n m e t h o d f o r s o l v i n g a s y s t e m o f n o n l i n e a r e q u a t i o n s F ( x ) = 0 i s w r i t t e n a s f o l l o w s:
7.3 The Primal-Dual I n t e r io r P oi nt Method
χ *+ι _ χ λ + Δ χ? k _ 1^
where t he change Δ χ i s determi ned from sol vi ng the f ol l owi ng equation:
J ( x * ) A x fc = — F ( x * )
T h e J a c o b i a n m a t r i x J c o n s i s t s o f p a r t i a l d e r i v a t i v e s o f t h e f u n c t i o n s w i t h r e s p e c t t o t h e s o l u t i o n v a r i a b l e s. F o r t h e s o l u t i o n o f e q u a t i o n s r e s u l t i n g f r o m K T c o n d i t i o n s f o r a n L P p r o b l e m, a t e a c h i t e r a t i o n w e n e e d t o c o m p u t e n e w v a l u e s o f 2 n + m v a r i a b l e s ( x, u, a n d v ). T h e r e f o r e, t h e J a c o b i a n m a t r i x i s o b t a i n e d b y d i f f e r e n t i a t i n g e q u a t i o n s w i t h r e s p e c t t o x, u, a n d v a s f o l l o w s:
(
A O 0 \ < = F r o m p r i m a l f e a s i b i l i t y 0 I A T j < = F r o m d u a l f e a s i b i l i t y U X 0 / < = F r o m c o m p l e m e n t a r y s l a c k n e s s
D e n o t i n g t h e c h a n g e s i n v a r i a b l e s a s d x, d u, a n d d y, t h e N e w t o n - R a p h s o n m e t h o d g i v e s t h e f o l l o w i n g s y s t e m o f e q u a t i o n s:
(
A O 0 \ /d x\ / A x * — b \
0 I A T J j d H I = - I A r v * 4 - u k — c I
U X 0 / \d v J \ X U e - ^ e /
N o t e t h a t t h e m a t r i c e s U a n d X a r e d e f i n e d b y u s i n g t h e k n o w n v a l u e s a t t h e c u r r e n t i t e r a t i o n. H o w e v e r, t o s i m p l i f y n o t a t i o n, s u p e r s c r i p t k i s n o t u s e d o n t h e s e t e r m s. W e c a n p e r f o r m t h e c o m p u t a t i o n s m o r e e f f i c i e n t l y b y w r i t i n g t h e t h r e e s e t s o f e q u a t i o n s e x p l i c i t l y a s f o l l o w s:
( a ) A d x = — A x k - f b = r p
( b ) d M 4 - A r d t, = — Α τ\* — u k + c ==
( c ) U d * + X d „ = — X U e +
F r o m t h e t h i r d e q u a t i o n, w e g e t
d „ = —X _ 1 U d x - X _ 1 X U e + μ * Χ - 1 β
o r
d „ = —X - 1 U d * - U e + /A * X - 1 e
Chapter 7 I n t e r i o r P o i n t Methods
Introducing the notation r c = —U e + μ * Χ - 1 β
du = —X-1Ud* + tc Multiplying both sides of equation (b) by AXU'1, we get
A X U ~ 1 d M + A X U ~ 1 A r d i/ = A X t T 1 ^
Substituting for d w, we get
—A X U _ 1 X _ 1 U d x + A X U - 1 r c + A X U'1 A r d*, = A X U'1^
From the form of X and U matrices, it is easy to see that
X U _ 1 X _ 1 U = I Therefore, using equation (a), we have
— T p + A X U - 1 r c + A X U - 1 Α τ ά ν = A X U - 1 ^
or
A X U - 1 A r d t, = T p - A X U - 1 r c + A X U - 1 ^
Introducing the notation D = X U - 1, we have
ADATdu = r p + A D ( - r c - f r j )
This system of equations can be solved for dv. Using this solution, the other two increments can be calculated as follows:
From (b) du - -ATdv + rd
From (c) d x = U ~ 1 X ( - d u + rc) or d x = D ( - d u + rc)
7.3.2 S t e p - L e n g t h C a l c u l a t i o n s
The previous derivation did not take into account the requirement that x; > 0 and Ui > 0. Assuming that we start from positive initial values, we take care of these requirements by introducing step-length parameters ap and ad-
xfc+1 = + oipdx
u f c + 1 = u k + a d d u
y k + i = v * + orrfdy
7.3 T he Primal -Dual I n t e r io r P o i n t Method
The maximum value of the step length is the one that will make one of the x, or ui values go to zero.
Xi + otpdxi> 0 and u,· + oid^ui > 0 i =l,...,n
Var i abl e s wi t h po s i t i v e i nc r e me nt s wi l l o bv i o us l y r e ma i n po s i t i v e r e g a r dl e s s o f t he s t e p l e ng t h. Al s o, i n or de r t o s t r i c t l y ma i nt a i n f e a s i bi l i t y, t he a c t ual s t e p l e ng t h s ho ul d be s l i g ht l y s ma l l e r t ha n t he above ma x i mum. The a c t ual s t e p l e ng t h i s c h o s e n as f ol l ows:
a = jftamax where β = 0.999...
Thus, the maximum step lengths are determined as follows:
ctp = Min [1, -βχι/άχί, dxi < 0] and ad — Min[l, ~fiui/dui, dui < 0]
The variables are therefore updated as follows:
Xk+1
= χ * + Οίράχ
u fc+1 = u k + a d d u yk+1=yk + addv
7.3.3 C o n v e r g e n c e C r i t e r i a
P r i m a l F e a s i b i l i t y
The constraints must be satisfied at the optimum, i.e., Ax* — b = 0. Tb use as convergence criteria, this requirement is expressed as follows:
l | A x fc — b | j ^ σ ρ" l l b i i + 1 £ f l
where €\ is a small positive number. The 1 is added to the denominator to avoid division by small numbers.
D u a l F e a s i b i l i t y
We also have the requirement that
A r v * + u fc - c = 0
Chapter 7 I n t e r io r P o i n t Methods
This gives the following convergence criteria:
i t a i i ^
= i - € 2
| | c | | + 1
where €2 is a small positive number.
C o m p l e m e n t a r y S l a c k n e s s
The value of parameter μ determines how well complementary slackness conditions are satisfied. Numerical experiments suggest defining an average value of μ as follows:
k ( x fc) r u
μ = -------
n
where n — number of optimization variables. This parameter should be zero at the optimum. Thus, for convergence
μ < € 3
where €3 is a small positive number.
7.3.4 C o m p l e t e P r i m a l - D u a l A l g o r i t h m
The complete primal-dual algorithm can be summarized in the following steps: Algorithm
Given: Constraint coefficient matrix A, constraint right-hand side vector b, objective function coefficient vector c, step-length parameter β, and conver­
gence tolerance parameters.
Initialization: k — 0, arbitrary initial values (> 0), say xk = uk = e (vector with all entries 1) and v* = 0.
The next point xk+1 is computed as follows:
1. Set μ k = /( k + 1). If < €lt < c2, μk < €3], we have
the optimum. Otherwise, continue.
2. Form:
D = XU-1 = diag [xi/ui] rp = -Ax11 + b rj = — Arv* - Uk + c Tc = - u fc + μ k'XΓl e
3. Solve the system of linear equations for dj,:
ADATdj, = Tp + AD (—tc + rj)
4. Compute increments:
d u — — ATdt, + r4 dx = D (~-du + rc)
5. Check for unboundedness:
Primal is unbounded if rp = 0, d* > 0, and c Tdx < 0 Dual is unbounded if rj = 0, du > 0, and b Tdv > 0 If either of these conditions is true, stop. Otherwise, continue.
6. Compute step lengths:
ap - Min [1, - β Xi/dxi, dxi < 0] and ad = Min [1, -fiui/dui, dui < 0]
7. Compute the next point:
xk+1 = x k+ otpdx U fc+1 = u fc + a ddu
y k + l = y k +
P u r i f i c a t i o n P h a s e
After obtaining the interior point optimum, we use the same steps as those for the PAS method to get an exact vertex solution.
1. Set the diagonal of scaling matrix T to the interior point optimum.
2. Compute matrix Ρ = ΤΑτ(ΑΤ2Ατ)-1 AT. Set vector p to diagonal elements of this matrix.
Chapter 7 In t e r io r P o i n t Methods
3. Elements of vector p that are close to 1 correspond to basic variables, and those that are close to zero define nonbasic variables. In case the elements of vector p do not give clear indication, repeat calculations of step 2 by defining a new scaling matrix T with diagonal elements set to vector p.
4. Set the nonbasic variables to zero and solve the constraint equations Ax = b for the optimum values of basic variables.
The P r i m a l D u a l L P F u n c t i o n
The following PrimalDualLP function implements the above algorithm for solving LP problems. The function usage and its options are explained first. Several intermediate results can be printed to gain understanding of the pro­
cess.
Heeds["OptimizationToolbox'InteriorPoint'"];
PPrimalDualLP
PrimalDualLP[f, g, vars, options]. Solves an LP problem using
Interior Point algorithm based on solving KT conditions using------------
the Newton-Raphson method, f is the objective function, g is a list of constraints, and vars is a list of variables. See Options[PrimalDualLP] to find out about a list of valid options for this function.
OptionsUsage[PrimalDualLP]
{UnrestrictedVariables -> {}, Maxlterations -» 20,
ProblemType -»Min, StandardVariableName -+ x,
PrintLevel -»1, StepLengthFactor ->0.99,
ConvergenceTolerance -> 0.0001, StartingVector { } }
UnrestrictedVariables is an option for LP and several QP problems.
A list of variables that are not restricted to be positive can be specified with this option. Default is {}.
Maxlterations is an option for several optimization methods. It specifies maximum number of iterations allowed.
ProblemType is an option for most optimization methods. It can either be Min (default) or Max.
StandardVariableName is an option for LP and QP methods. It specifies the symbol to use when creating variable names during conversion to the standard form. Default is x.
PrintLevel is an option for most functions in the OptimizationToolbox.
It is specified as an integer. The value of the integer indicates how much intermediate information is to be printed. A PrintLevel-*0 suppresses all printing. Default for most functions is set to 1 in which case they print only the initial problem setup. Higher integers print more intermediate results.
7.3 The Primal-Dual I n t e r io r P o i n t Method
StepLengthFacfcor is an option for interior point methods. It is the reduction factor applied to the computed step length to maintain feasibility. Default is 0.99
ConvergenceTolerance is an option for most optimization methods. Most methods require only a single zero tolerance value. Some interior point methods require a list of convergence tolerance values.
StartingVector is an option for several interior point methods. Default
Example 7.8 Solve the following LP problem using the primal-dual interior point method:
f = - 5x1 - x2;
g = {2x1 + 3x2 £ 12, 2x1 + x2 ύ
8}; vars = {xl, x2} ;
All calculations for the first two iterations are as follows:
PrimalDualLP[f, g, vars, PrintLevel 2, Maxlterations -> 2] ;
Minimize - 5x^ - x2
**** Starting vectors ****
Primal vars (χ) - ^ { Ι.,Ι.,Ι.,Ι.}
Dual vars (u) -> {1., 1., 1., 1. }
Multipliers (v)->{0.,0.}
Objective function-»-6. Status -» NonOptimum
***** iteration 1 *****
D [diagonal ] - » { l.,l.,l.,l.}
is {1,...,1}.
Subject to
All variables > 0
Problem variables redefined as: {xl x^, x2 -> x2 }
'-5'
- 1
0
Chapter 7 I n t e r io r P o i n t Methods
Parameters: op -> 0 .467579 CTd->l.06259 μ-»1.
rp+A.D(-rc+rd) -» {-13., -11. } dv-* {-0.0285714, -1.8}
du {-2.34286,-0.114286,-0.971429, 0.8} dx-» {2.34286, 0.114286, 0.971429, -0.8}
-β
χ/dx (co, oo, oo, 1.2375}
-β
u/du-*· {0.422561, 8.6625, 1.01912, oo} ctp -»1 ad -* 0 .422561
New primal vars (x) -* {3.3428
6,
1.11429, 1.97143, 0.2} New dual vars (u) -»{0.01, 0.951707, 0.589512, 1.33805} New multipliers (v) -»{-0.0120732, -0.76061}
Objective function -» -17.8286 Status-► NonOptimum
***** iteration 2 *****
D[diagonal] ->{334.286, 1.17083, 3.34417, 0.149471}
(-3 .46463
[0.0843689]
/o.\
-1.15488
-0.668601
rp^ (o.J
-0.577439
rc -*
-0.429495
-0.577439,
, 0.23926 j
Parameters: σ p-»0. σ d-» 0.613579 μ
-+ 0.315462
, ~ /1351.02 1340.66\
‘ ' [l340.66 1338.46]
rp+A.D(-rc+rd) -*{-2374.96, -2373.45} dv
-¥
{0.290696, -2.06444}
du-* {0.0828534, 0.0374745, -0.868135, 1.487} dx-»{0.506597, -0.826693, 1.46688,-0.186502}
-β
x/dx-+ {oo, 1.3344,oo, 1.06165}
-β
u/du -» {oo, oo, 0,672266, oo} otp —> 1 ad -» 0. 672266
New primal vars (x> -* {3.84945, 0.287593, 3.43831, 0.0134985}
New dual vars (u) -* {0.0656995, 0.9769, 0.00589512, 2.33771}
New multipliers (v) {-0.183351,-2.14846}
Objective function -» -19 . 5349 Status -»NonOptimuiti
***** NonOptimum solution after 2 iterations *****
Interior solution -* {xl -> 3 . 84945, x2 -» 0 .287593 }
Objective function->-19 .5349
The complete solution is obtained using the PrimalDualLP function.
(sol, history} = PrimalDualLP[f, g,
-vars];
7.3 The Primal-Dual I n t e r i o r P oi n t Method
***** Optimum solution after 7 iterations *****
Interior solution -> {xl -> 3 . 99985 , x2 -» 0 . 000192359} Objective function -19. 9994
***** Finding nearest vertex solution *****
Nearest vertex solution-» {xl 0}
Obj ective function->- 20 Status -* Feasible
The history of points computed by the method are extracted and are shown on the graph in Figure 7.7. Notice that the method takes a little more direct path to the optimum as compared to the PAS algorithm.
xhist = Transpose [Transpose [history] [ [{1, 2}] ] ] ; TableForm [xhist]
3.34286 3.84945 3.98951 3.99545 3.99907
3.99985
3.99985
1.
1.11429
0.287593
0.00287593
Q.QQ568681
0.00116336
0.000192359
0.000192359
-1 0 1 2 3 4 5
FIGURE 7.7 A graphical solution showing a solution path.
Chapter 7 I n t e r i o r P o i n t Methods
Example 7.9 Solve the following LP problem using the primal-dual method:
£ = xl + x2 + x3 + x4
; g =
{xl + 2x2 - x3 + 3x4 £ 12, xl + 3x2 + x3 + 2x4 5 8,
2x1 - 3x2 - x3 + 2x4 S 7}; vars = {xl, x2, x3, x4} ;
Intermediate calculations are shown for the first two iterations.
PrimalDualLP[£, g,
vars, ProblemType -»■ Max, PrintLevel -» 2,
Maxlterations -» 2];
Minimize - x^ - x2 - *3 - X4
Subject to
xj^ + 2x2 - X3 + 3x4 + x5 == 12 X! + 3x2 + X3 + 2x4 + x6 == 8 ,2xi
- 3x2 - X3 + 2x4 + x7 == 7
All variables a 0
Problem variables redefined as: {xl -*
Χχ, x2 -» x2, x3 -> X3, x4 -*
X4 }
'-1'
- 1 -1 - 1
Γ12ϊ 8 ι 7
0
0
0
1
1
\2
2
3
-3
- 1
1
- 1
3
2
2
1
0
0
0
1
0
0
0
1/
**** Starting vectors ****
Primal vars (x) -» {1., 1., 1., 1., 1., 1., 1.} Dual vars (u) - » { 1.,1.,1.,1.,1.,1.,1.) Multipliers {v) -* { 0 ., 0., 0.}
Objective f u n c t i o n -4. Status -»NonOptimum
***** Iteration 1 *****
D[diagonal] -»{1.,1.,1.,1.,1.,1.,1.}
-2.'
Ό.'
-2.
0.
f6, ’
-2.
0.
rp->
0.
~¥
-2.
rc
0.
16 > ι
-1.
0.
- 1.
0.
-1 ·,
· 1
Parameters: σ p-» 0
.498219
a
16. 1 2.
3.
A.D.AT-*
1 2. 16.
-4.
1 3 -
-4.
19.,
1.
7.3 The Primal -Dual In t e r io r P o i n t Method
rp+A.D(-rc+rd) -* (-5., -15., 5.} dv-* {1.16667, -1.89236, -0.319444} du-* {-0.635417, 0.385417, 0.739583,
-1.07639, -2.16667, 0.892361,-0.680556} dx-* {0.635417, -0.385417, -0.739583,
1.07639, 2.16667,-0.892361, 0.680556}
-g x/dx -> {qq, 2.56865, 1.33859, co, co, 1.10942, co}
-β
u/du -> {1.55803, o o, o o, 0.919742, 0 .456923, o o, 1.45469} dp -* 1 ct,j -* 0.456923
New primal vars (x) -*{1.63542, 0.614583, 0.260417, 2.07639,
3.16667,0.107639,1.68056}
New dual vars (u) {0.709663, 1.1761, 1.33793, 0.508173,
0.01, 1.40774, 0.689038}
New multipliers (v) -*{0.533077,-0.864663,-0.145962} Objective function -* -4.58681 Status -* NonOptimum
***** Iteration 2 *****
D[diagonal] -»{2.3045, 0.522558, 0.194641,
4.08599, 316.667, 0.0764622, 2.43899}
' -1.08615
' -0.507524
-1.08615
-0.638209
77636 χ 10"151
-1.08615
-0.0684953
0.
rd -»
-1.08615
rc
-0.348963
0.
-0.543077
0.0943945
-0.543077
1.66348
,-0.543077j
,-0.492328 )
rp "*
Parameters: a
p-* 1.043 x10 16
358.03 29.7611
A.D.AT-* 29.7611 23.6226
,26.1842 16.0553 32.8986,
rp+A.D(-rc+rd) -* {-212.506, -8.42679,-7.91467} dv-* {-0.63118, 0.389863, 0.0715204}
a
d-* 0.789072 26 .1842'
16.0553
μ -*0.330583
du-» {-0.987878, -0.778821, -2.03568,-0.11538,
0.0881033,-0.93294,-0.614597} dx-*{1.10697,0.0734783,0.382894,-0.9544153,
1.99221, 0.198528, 0.298213}
~β
X/dx-* { oo, oo, oo, 2-153801, CD, oo, oo}
-β
u/du-* {0.711188, 1.49501, 0.65067, 4.36029,oo, 1.49384, 1.10991}
Op-*l ad -* 0.65067
New primal vars (x) -*{2.74239, 0.688062, 0.643311, 1.12197, 5.15888,
0.306167,1.97877}
New dual vars (u) -*{0.0668814, 0.66935, 0.0133793, 0.433099, 0.0673262,
0.800704,0.289139}
New multipliers (v) ->{0.122387, -0.610991,-0.0994254}
Objective function -» -5.19574 Status -* NonOptimum
Chapter 7 I n t e r io r P o i n t Methods
***** NonOptimum solution after 2 iterations *****
Interior solution -» {xl -► 2 . 74239, x2 -^ 0.688062,
x3 -♦ 0.643311, x4 ->1.12197}
Objective function -» 5.19574
Calculations are allowed to run until the optimum is found.
{sol/ hist} = PrimalDualLP [f, g,
vars, ProblemType -* Max];
***** optimum solution after 7 iterations *****
Interior solution -» {xl -► 3 .25621, x2 -» 0 .000351426,
x3 -» 4.7406, x4 0.000713895}
Objective function-» 7 .99788
***** Finding nearest vertex solution *****
Nearest vertex solution -» {xl -» 5, x2 -» 0, x3 -» 3, x4 -» 0}
Objective function-» 8 Status -» Feasible
xhist = Transpose[Transpose[hist][[{1,
2,
3, 4}] ] ]; TableForm[xhist]
1.
1.
1.
—n
------------
1.63542
0.614583
0.260417
2.07639
2.74239
0.688067
0.643311
1.12197
3.82294
0.00688062
3.27992
0.342731
4.26952
0.0200827
3.57756
0.00342731
3.53977
0.00305149
4.43303
0.00596975
3.25621
0.000351426
4.7406
0.000713895
3.25621
0.000351426
4.7406
0.000713895
Example 7.10 Unbounded solution
f = xl - 2x2;
g = {2x1 - x2 £ 0, -2x1 + 3x2 5 6}; vars = (xl, x2} ;
{sol, history} = PrimalDualLP [f, g, vars];
Minimize x^ - 2x2
Subject to f 2xi-x2 -x3 ==0 + 3x2 + X4 =* 6
All variables a 0
Problem variables redefined as: {xl -*X!, x2 -» x2 }
***** unbounded solution after 3 iterations *****
Interior solution-» {xl -» 14.8277, x2 -»11.85752}
Objective function -» -8.88663
The history of points computed by the method are extracted and are shown on the graph in Figure 7.8.
xhist = Transpose [Transpose [history] [ [{1, 2}] ] ]; TableForm [xhist]
1. 1.
2.25714 2.91429
3.58721 4.38119
14.8277 11.8572
x2
-1 o 1 2 3 4 5
F I G U R E 7.8 A g r a p h i c a l s o l u t i o n s h o w i n g a s o l u t i o n p a t h.
E x a m p l e 7.1 1 Plant operation In this example, we consider the solution of the tire manufacturing plant operations problem presented in Chapter 1. The problem statement is as follows:
A tire manufacturing plant has the ability to produce both radial and bias-ply automobile tires. During the upcoming summer months, they have contracts to deliver tires as follows.
Chapter 7 I n t e r io r P oint Methods
Date
Radial tires
Bias-ply tires
June 30
5,000
3,000
July 31
6,000
3,000
August 31
4,000
5,000
Total
15,000
11,000
The plant has two types of machines, gold machines and black machines, with appropriate molds to produce these tires. The following production hours are available during the summer months:
Month
On gold machines
On black machines
June
700
1,500
July
300
400
August
1,000
300
The production rates for each machine type and tire combination, in terms of hours per tire, are as follows:
Type
On gold machines
On black machines
Radial
0.15
0.16
Bias-Ply
0.12
0.14
The labor cost of producing tires is $10.00 per operating hour, regardless of which machine type is being used or which tire is being produced. The material cost for radial tires is $5.25 per tire and for bias-ply tires is $4.15 per tire. Finishing, packing, and shipping cost is $0.40 per tire. The excess tires are carried over into the next month but are subjected to an inventoiy-carrying charge of $0.15 per tire. Wholesale prices have been set at $20 per tire for radials and $15 per tire for bias-ply.
How should the production be scheduled in order to meet the delivery requirements while maximizing profit for the company during the three-month
period?
7.3 The Pr-tmal-Dual I n t e r i o r P oint Method
The optimization variables are as follows:
Xl
N u m b e r o f r a d i a l t i r e s p r o d u c e d i n J u n e o n t h e g o l d m a c h i n e s
* 2
N u m b e r o f r a d i a l t i r e s p r o d u c e d i n J u l y o n t h e g o l d m a c h i n e s
x3
N u m b e r o f r a d i a l t i r e s p r o d u c e d i n A u g u s t o n t h e g o l d m a c h i n e s
X
4
N u m b e r o f b i a s - p l y t i r e s p r o d u c e d i n J u n e o n t h e g o l d m a c h i n e s
* 5
N u m b e r o f b i a s - p l y t i r e s p r o d u c e d i n J u l y o n t h e g o l d m a c h i n e s
* 6
N u m b e r o f b i a s - p l y t i r e s p r o d u c e d i n A u g u s t o n t h e g o l d m a c h i n e s
X7
N u m b e r o f r a d i a l t i r e s p r o d u c e d i n J u n e o n t h e b l a c k m a c h i n e s
XS
N u m b e r o f r a d i a l t i r e s p r o d u c e d i n J u l y o n t h e b l a c k m a c h i n e s
* 9
N u m b e r o f r a d i a l t i r e s p r o d u c e d i n A u g u s t o n t h e b l a c k m a c h i n e s
* 1 0
N u m b e r o f b i a s - p l y t i r e s p r o d u c e d i n J u n e o n t h e b l a c k m a c h i n e s
*11
N u m b e r o f b i a s - p l y t i r e s p r o d u c e d i n J u l y o n t h e b l a c k m a c h i n e s
* 1 2
N u m b e r o f b i a s - p l y t i r e s p r o d u c e d i n A u g u s t o n t h e b l a c k m a c h i n e s
The objective of the company is to maximize profit. The following expressions used in defining the objective function were presented in Chapter 1.
sales = 20 (xj + x2 + x3 + x·; ■*· x8 + x9) +15 (x4 + x5 + x6 + x10 + + x12 ) ;
materialeCost = 5.25 (x^ + x2 + X3 + x7 + Xg + x9) +4.15 (x4 + x5 + xg + x10 + x^i + X12) ·
laborCost = 10 (0 .15 (x^^ + x2 + x3) + 0.16 (x7 + x8 + x9) + 0 .12 (x4 + x5 + x6)
+ 0.14 (x^o + *11 + *12 ))'
handlingCost = 0.40 (Xj + x2 + x3 + x4 + x5 ■*· x6 + x7 +■ Xg + x9 + χχρ + x^ +· Χχ2) >
inventoryCost = 0 .15 ((Xi + X7 - 5000) + (x4 + x10 - 5000) + (xj. + x2 + x7 + Xg - 11000) + (x4 + x5 + x10 + xtl - 6000)) t
The production hour limitations are expressed as follows:
productionljimi tat ions =
{0 .15*! + 0 . 12x4 5 700, 0. 15x2 + 0.12xs 5 300, 0 .15x3 + 0.12x6 £ 1000,
0.1 6x7 + 0.14x10 5 1500, 0.16x8 + 0.14χιχ S 400, 0.16x9 + 0.14x12 S 300};
Delivery contract constraints are written as follows:
deliveryConstrainte =
{*! + x7 i
5000, x4 + X10 & 3000, xx + X2 + X7 + xa a 11000, x4 + xs + x10 + x^ & 6000,
Xi + x2 + *3 + x7 + x8 + x9 == iSOOO, x4 + x5 + Xg + Xjq + Xxl + x12 == 11000} ;
Chapter 7 I n t e r i o r P oint Methods
Thus, the problem is stated as follows. Note by multiplying the profit with a negative sign, the problem is defined as a minimization problem.
vars = Table[xly {i, 1,
12 }];
f = - (sales - (materialsCost + laborCost + handlingCost + inventoryCost)); g =
Join[productionLimitations, deliveryConstraints];
The solution is obtained using the PrimalDualLP, as follows:
Minimize - 13 .25Χχ - 13 .25x2 “ 13 . 25x3 - 9 . 65x4 - 9 . 65xs - 9 .65xg - 13 . 15x7 13.15x8 - 13 .15xg - 9.45χχ0 - 9. 45χχχ “ ® ·45χχ2
0 . 15χχ + 0 . 2x4 + x^3 == 700 0. lSxg + 0 .2x5 + x14 == 300 0 - lSx^ + 0 . 2xg + x^5 — — 1 , 0 0 0 0 * lGx^ + 0.14x^0 xi 6 == 1/500
Siobj ect to
0.16xg + 0.14χχχ + x17 == 400 0 .16X9 + 0 · 1 4 x ^ 2 + == 3 0 0
Χχ + x7 - x19 == 5, 000 x i + x 10 “ x 20 == 3/0 0 0 X1 + x 2 + x 7 + x 8 “ X21 == 1 1,0 0 0 x4 + x5 + x10 + Χχχ - x22 == 6,000
Χχ + x2 + X3 + X7 + Xe + X9 == 15/ 000 tx4 + x5 + χ6 + x10 + Χχχ + x12 == 11, 000,
All variables > 0
Problem variables redefined as:
{Χχ -* Χχ , x2 -* x2 , x3 -* x3 , x4 -* x4, x5 -*
x5, x6 -* Xg ,
X7 -* X7 , Xg -> Xg , Xg -* Xg , Χχο ~* X1Q ' X11 "* X11 ' *12 X12 1
***** optimum solution after 13 iterations *****
Interior solution-»
{Χχ -*1,759.51, Xj ^ 99.7596, x3 -» 2,674.05, x4 -* 3, 633 . 93, x5 -»2,375-29,Xg ->4,990.75, x7 -*6,664.86,Xg -*2,484.5, x9 -> 1,317.3, x10 -* 0.00491385, Χχχ -* 0.0049176, x12 -* 0.00491642}
Objective function -* -303853.
***** Finding nearest vertex solution *****
Nearest vertex solution
{xi-»l,866.67,x2 -* 0, x3 -*2,666.67,x4 ~*3,500., X5-»2,500.,Xg-*5,000.,X7 -*6,633.33,Xg-*2,500., Xg -* 1,333 .33, Χχφ -* 0, Χχχ 0, Χχ 2 0}
Objective function -* -303 ,853 .
S t a t u s - » F e a s i b l e
7.5 A p p e n d i x —Null and Range Spaces
The same solution was obtained in Chapter 6 using the revised simplex method.
7.4 Concluding Remarks
Since the publication of L.G. Khachiyan’s original paper in 1979 [Fang and Puthenpura, 1993], a wide variety of interior point methods have been pro­
posed. The main motivation for this development has been to devise a method that has superior convergence properties over the simplex method. Examples exist for which the simplex method visits every vertex before finding the op­
timum. However, for most practical problems, the convergence is quite rapid. On the other hand, most interior point methods have been proven to have good theoretical convergence rates, but realizing this performance on actual large- scale problems is still an open area. Commercial implementations of interior point methods are still relatively rare.
The goal of this chapter was to provide a practical introduction to the inte­
rior point methods for solving linear programming problems. TWo relatively simple-to-understand methods were presented. The Mathematica implemen­
tations, PrimalAffineLP and PrimalDualLP, should be useful for solving small- scale problems and for developing an understanding of how these methods work. For a comprehensive treatment of this area, refer to the books by Fang and Puthenpura [1993], Hertog [1994], Megiddo [1989], and Padberg [1995],
7.5 A ppen d ix—Null and Range Spaces
The concepts of null and range spaces from linear algebra are used in the derivation of Primal Affine scaling algorithm. These concepts are reviewed briefly in this appendix.
N u l l a n d R a n g e s p a c e s
Consider an m χ n matrix A with m < n. Assume that all rows are linearly independent and thus, rank of matrix A is m. The null space of matrix A
Chapter 7 I n t e r io r Point Methods
(denoted by Null(A)) is defined by the set of n - tn linearly independent vectors pf that satisfy the following relationship:
Ap; = 0
It is clear from this definition that each vector p; has n elements and forms the basis vectors for the null space of A. The basis vectors defining null space are not unique and can be determined in a number of different ways. The following example illustrates a simple procedure based on solving systems of equations.
The range space of matrix A (denoted by Range(A)) is the set of linearly independent vectors obtained from the columns of matrix A. The range space of matrix A T, rather than of A, is of more interest because of its important relationship with the null space of matrix A. Since rank of matrix A is m, all tn columns of matrix AT are linearly independent and form the range space. Any vector q in the range space of Ar can be written as a linear combination of columns of matrix Ar. That is,
q = A Ta for some m χ 1 vector a
Since the dimension of range space of AT is m and that of null space of A is n — m, together the two spaces span the entire n dimensional space. Furthermore, by considering the dot product of vector q in the range space of AT and a vector p in the null space of A, it is easy to see that the two subspaces are orthogonal to each other.
qrp = αΓΑρ = 0
Example 7.12 Consider the following 2 x 4 matrix:
A= {{1, 2, 1, 3}, {2, 1, 1, - 4 } };
MatrixForm [A]
The range space of AT is simply the columns of the transpose of A (or rows of matrix A). Thus, the basis for the range space of A T is
{ { 1, 2, 1, 3} , {2, 1, 1, - 4 }}
7.5 A p p e n d i x —Null and Range Spaces
The basis vectors for the null space of matrix A are determined by considering solutions of the system of equations Ap = 0. Since we have only two equations, we can solve for two unknowns in terms of the other two and thus, there are two basis vectors for the null space.
P= a2' a3' * * }'
eqns = A.p ■ ■ {0, 0}-----------------------------------------------------------------------
{a-L + + 33 + 3a4, 2a^ + a2 + a3 — 4a^} =» { 0, 0}
If we choose a\ = 1 and «2 = 0 (arbitrarily), we get the following solution:
Solve [βςριβ/.ί*! 4 l,a2-»0}]
{{a3^ - T'a4^?}}
If we choose a\ = 0 and a2 — I, we get the following solution:
Solve[e<xue/. {ax -* 0,
a2 -» 1>]
{{a3 a4
Thus, we have following two basis vectors for the null space of A:
Pi = {1, 0,-10/7, 1/7}f
P2 =
{0/ lr -11/7, -1/7);
There are obviously many other possibilities for the basis vectors. However, we can only have two linearly independent basis vectors. Any other vector in the null space can be written in terms of the basis vector. As an illustration consider another vector in the null space as follows:
Solve [eons/· {a1-»-3,a2-»2}]
P= {-3, 3, 8/7, -5/7};
Clearly, this vector can be obtained by the linear combination — 3pi -I- 2p2-
“3Pi + 2p2
Κ
Next, we numerically demonstrate that the two subspaces (range space of A T and null space of A) are orthogonal to each other.
Chapter 7 I n t e r io r P oi nt Methods
β ι.Ρ 2' <l2.Pl' <*2.^ 2}
{ 0, 0, 0, 0 }
There i s a bui l t-i n Mathematica function NuUSpace to produce null space basis vectors. The following basis vectors are obtained using this function:
MullSpace[A]
{{11, -10, 0, 3} , {-1, -1, 3, 0}}
Null Space Projection Matrix
Any vector x can be represented by two orthogonal components, one in the range space of Ar, and the other in the null space of A:
x = p + q
The null space component p can be determined as follows:
P = x - q or ρ = x - ATor Multiplying both sides by A, we get
Ap = Ax — AATa
Since Ap = 0, we get
Ax = AATa giving a = (AAT)-1 Ax
Therefore,
ρ = X - AT(AAT)- 1 Ax = [I - Ar(AAT)-1 A] x = Px The nxn matrix P is called the null space projection matrix.
Ρ = I — AT(AAT)-1 A The range space component q is obtained as follows:
q — ATor = Ar(AAT)-1Ax = Rx where the range space projection matrix is as follows:
R = A1(AAr)_1A
7.5 A p p e n d i x —Null and Range Spaces
E x a m p l e 7.13 Consider the following 1 x 2 matrix:
A = {{1,2}};
The basis vector for the range space of A T is q = { l r 2} ;
The nul l space basi s vector i s computed usi ng t he Nul l Space functi on:
p = First[NullSpace[A]]
{ - 2,1 }
It i s eas y to s ee that t he two spaces are orthogonal to each other.
q.p
0
The orthogonal projecti on matri ces are
R = Transpose[A].Inverse[A.Transpose[A]].A;
Mat r ixForm[R]
1 1 2
5 3
2 4
\ Έ 5
P = IdentityMatrix[2] - R; MatrixForm[P]
f 4 2 1
5 "5
Given a two-di mensi onal vector, we can resol ve i t i nto two orthogonal compo­
nent s usi ng t hes e projecti on matrices:
vec = {3,-4};
pv a p .vec; qv = R.vec;
{pv, qv}
{{4, -2} , {-1, -2}}
These vectors are plotted in Figure 7.9. From the figure, it is clear that the components p and q of the given vector are in the two subspaces.
Chapter 7 I n t e r io r Point Methods
FIGURE 7.9 Components of a vector in the null space and range space.
7.6 Problems
D u a l L P
Construct the duals for the following LP problems. If possible, use KT condi­
tions or graphical methods to demonstrate the relationship between the primal and the dual problems.
^ 1 7 7 Μ 3 Β η ΐ ι ζ β Ύ ^ χ Γ Ψ 3 χ 2
Subject to xi + 4x2 <10 and x\ + 2x% <10.
7.2. Maximize z = —x\ + 2x2
Subject to x\ — 4x2 + 2 > 0 and 2x\ — 3x2 < 3.
7.3. Minimize / = —2x\ + 2x2 + *3 — 3x4
Subject to xi + X2 -I- xj + x* < 18, xi — 2x3 + 4x4 < 12, xi + X2 < 18, and X3 + 2x4 < 16.
7.6 Problems
7.4. Minimize f = — 3xi — 4^2
S u b j e c t t o xi
+ 2x2
< 10, xi +X
2 < 1 0, a n d 3 x i + 5^2 < 2 0.
7.5. Minimize / = —lOOxj - 80x2
Subject to 5xi + 3x2 <15 and xi + X2 < 4.
Primal A f f i n e S c a l i n g M e t h o d
Find the optimum solution of the following problems using the primal affine scaling method. Unless stated otherwise, all variables are restricted to be posi­
tive. Verify solutions graphically for two variable problems.
7.6. Maximize z = xi + 3x2
Subject to xi + 4x2 <10 and xi + 2x2 < 10·
7.7. Maximize z = —Xj + 2x2
Subject to xi — 4x2 + 2 > 0 and 2xi — 3X2 < 3.
7.8. Minimize f = —2x\ + 2x2 + x$ — 3x4
Subject to xi + X2 + X3 + X4 < 18, xi — 2x3 + 4x4 < 12, xi + X2 < 18, and X3 + 2x4 < 16.
7.9. Minimize / = —3xi — 4x2
Subject to xi + 2x2 < 10, xi + X2 £ 10, and 3xi + 5x2 < 20.
7.10. Minimize / = —ΙΟΟχι — 8OX2
Subject to 5xi + 3x2 < 15 and xi + x2 < 4.
7.11. Maximize z = — xi + 2x2
Subject to xi — 2x2 + 2 > 0 and 2xi — 3x2 < 3.
7.12. Minimize / = x 1 + 3x2 — 2x3
Subject to xi — 2x2 — 2x3 > —2 and 2xi — 3x2 — *3 < —2.
7.13. Maximize z = x 1 + 3x2 — 2x3 + x4
Subject to 3xi + 2x2 + X3 + xa < 20, 2xi + X2 + x4 = 10, and 5xi — 2x2 —
X3 + 2x4 > 3.
Chapter 7 I n t e r i o r Point Methods
7.14. Minimize / = 3xi + 4x2 + 5x3 + 6x4
Subject to 3xi + 4x2 + 5x3 + 6x4 > 20, 2xi + 3x2 + 4x4 < 10, and 5xi — 6x2 -
7x3 + 8x4 > 3. Variables xi and X4 are unrestricted in sign.
7.15. Minimize / = 13xi — 3x2 — 5
Subject to 3xi 4- 5x2 < 20, 2x\ + X2 > 10, 5xi + 2x2 > 3, and x\ + 2x2 > 3. xi is unrestricted in sign.
7.16. Minimize / = 5xi + 2x2 + 3x3 + 5x4
Subject to xi - X2 + 7x3 + 3x4 > 4, xj + 2x2 + 2x3 + X4 = 9, and 2x\ + 3x2 + X3 — 4x4 < 5.
7.17. Minimize f = —3xi 4- 8x2 — 2x3 + 4x4
Subject to xi — 2x2 + 4x3 + 6x4 < 0, xi — 4x2 — *3 + 6x4 < 2, X3 < 3, and X4
> 3.
7.18. Minimize / = 3xj + 2x2
Subject to 2xi + 2x2+ X3 + X4 = 10, and 2xi — 3x2 + 2x3 = 10. The variables should not be greater than 10.
7.19. Minimize f = —2xj 4- 5x2 + 3x3
Subject to xi — X2 — X3 < —3 and 2xi + X2 > 1.
7.20. Hawkeye foods owns two types of trucks. Ttuck type I has a refrigerated capacity of 15 m3 and a nonrefrigerated capacity of 25 m3. Ituck lype II has a refrigerated capacity of 15 m3 and non-refrigerated capacity of 10 m3. One of their stores in Gofer City needs products that require 150 m3 of refrigerated capacity and 130 m3 of nonrefrigerated capacity. For the round trip from the distribution center to Gofer City, truck type I uses 300 gallons of gasoline while truck type II uses 200 gallons. For­
mulate the problem of determining the number of trucks of each type that the company must use in order to meet the store's needs while minimizing gas consumption. Use the PAS method to find an optimum.
7.21. A manufacturer requires an alloy consisting of 50% tin, 30% lead, and 20% zinc. This alloy can be made by mixing a number of available alloys, the properties and costs of which are tabulated. The goal is to find the cheapest blend. Formulate the problem as an optimization problem. Use
the PAS method to find an optimum.
7.6 Problems
A v a i l a b l e a l l o y s
P r o p e r t i e s
A
B
C
D
E
L63.Q
1 0
10
4 0
OU
3 0
Z i n c ( % )
1 0
3 0
5 0
3 0
3 0
T i n ( % )
80
60
1 0
1 0
4 0
C o s t: ( $/l b a l l o y )
8.2
9.3
1 1.2
13
1 7
7.22. A company can produce three different types of concrete blocks, iden­
tified as A, B, and C. The production process is constrained by facilities available for mixing, vibration, and inspection/drying. Using the data given in the following table, formulate the production problem in order to maximize the profit. Use the PAS method to find an optimum.
B l o c k s
A
B
C
A v a i l a b l e
M i x i n g ( h o u r s/b a t c h )
1
3
9
9 0 0
V i b r a t i o n ( h o u r s/b a t c h )
2
3
6
1 2 0 0
I n s p e c t i o n/d i y i n g ( h o u r s/b a t c h )
0.7
0.8
1
4 0 0
P r o f i t: ( $/b a t c h )
7
1 7
3 0
7.23. A mining company operates two mines, identified as A and B. Each mine can produce high-, medium-, and low-grade iron ores. The weekly demand for different ores and the daily production rates and operating costs are given in the following table. Formulate an optimization problem to determine the production schedule for the two mines in order to meet the weekly demand at lowest cost to the company. Use the PAS method to find an optimum.
W e e k l y d e m a n d ( t o n s )
D a i l y p r o d u c t i o n
O r e g r a d e
M i n e A ( t o n s )
M i n e B ( t o n s )
H i g h
1 2,0 0 0
2,0 0 0
1,0 0 0
M e d i u m
8,0 0 0
1,0 0 0
1,0 0 0
L o w
2 4,0 0 0
5,0 0 0
2,0 0 0
O p e r a t i o n s c o s t ( $/d a y )
2 1 0,0 0 0
1 7 0,0 0 0
Chapter 7 I n t e r i o r Point Methods
7.24. Assignment of parking spaces for its employees has become an issue for an automobile company located in an area with harsh climate. There are enough parking spaces available for all employees; however, some employees must be assigned spaces in lots that are not adjacent to the buildings in which they work. The following table shows the distances in meters between parking lots (identified as 1, 2, and 3) and office buildings (identified as A, B, C, and D). The number of spaces in the lots and the number of employees who need spaces are also tabulated. Formulate the parking assignment problem to minimize the distances walked by the employees from their parking spaces to their offices. Use the PAS method to find an optimum.
D i s t a n c e s f r o m p a r k i n g l o t ( m )
S p a c e s
a v a i l a b l e
P a r k i n g L o t
B u i l d i n g A
B u i l d i n g J B
B u i l d i n g C
B u i l d i n g D
1
2 9 0
4 1 0
2 6 0
4 1 0
8 0
2
4 3 0
3 5 0
3 3 0
3 7 0
1 0 0
.3
3 1 0
2 6 0
2 9 0 ,
3 8 0
,4 0
#
o f e m p lo y e e s
40
40
60
60
7.25. Hawkeye Pharmaceuticals can manufacture a new drug using any one of the three processes identified as A, B, and C. The costs and quantities of ingredients used in one batch of these processes are given in the following table. The quantity of new drug produced during each batch of different processes is also given in the table.
I n g r e d ie n t s u s e d p e r b a t c h ( t o n s )
P r o c e s s
C o s t ($ p e r b a t c h )
I n g r e d ie n t I
In g r e d ie n t I I
Q u a n t i t y o f d r u g p r o d u c e d
A
$12,000
3
2
2
B
$25,000
2
6
5
C
$9,000
7
2
1
The company has a supply of 80 tons of ingredient I and 70 tons of ingredient II at hand and would like to produce 60 tons of new drug at a minimum cost.
Formulate the problem as an optimization problem. Use the PAS method to find an optimum.
7.6 Problems
7.26. A major auto manufacturer in Detroit, Michigan needs two types of seat assemblies during 1998 on the following quarterly schedule:
T y p e 1
T y p e 2
F i r s t Q u a r t e r
2 5,0 0 0
2 5,0 0 0
S e c o n d Q u a r t e r
3 5,0 0 0
3 0,0 0 0
T h i r d Q u a r t e r
3 5,0 0 0
2 5,0 0 0
F o u r t h Q u a r t e r
2 5,0 0 0
3 0,0 0 0
T b t a l
1 2 0,0 0 0
1 1 0,0 0 0
The excess seats from each quarter are carried over to the next quarter but are subjected to an inventoiy-cariying charge of $20 per thousand seats. However, assume no inventoiy is carried over to 1999.
The company has contracted with an auto seat manufacturer that has two plants: one in Detroit and the other in Waterloo, Iowa. Each plant can manufacture both types of seats; however, their maximum capacities and production costs are different. The production costs per seat and the annual capacity at each of the two plants in terms of number of seat assemblies is given as follows:
Q u a r t e r l y c a p a c i t y ( e i t h e r t y p e )
P r o d u c t i o n c o s t
T y p e 1
T y p e 2
D e t r o i t p l a n t
3 0,0 0 0
$ 2 2 5
$ 2 4 0
W a t e r l o o p l a n t
3 5,0 0 0
$ 1 6 5
$ 1 8 0
The packing and shipping costs from the two plants to the auto manu­
facturer are as follows:
C o s t/1 0 0 s e a t s
D e t r o i t p l a n t
$ 1 0
W a t e r l o o p l a n t
$ 8 0
Formulate the problem to determine a seat acquisition schedule from the two plants to minimize the overall cost of this operation to the auto manufacturer for the year. Use the PAS method to find an optimum.
Chapter 7 I n t e r io r P oi nt Methods
7.27. A small company needs pipes in the following lengths:
0.5 m 100 pieces
0.6 m 300 pieces
1.2 m 200 pieces
The local supplier sells pipes only in the following three lengths:
4 m 6 m 8 m
After cutti ng t he nec e s s ai y l engt hs, t he excess pi pe must be thrown away. The company obvi ousl y wants to mi ni mi ze t hi s waste. Formulate t he probl em as a l i near programming problem. Use t he PAS method to fi nd an opti mum.
7.28. Consider the probl em of fi ndi ng t he short est route bet ween two ci t i es whi l e t ravel i ng on a gi ven network o f available roads. The network i s shown i n Figure 7.10. The nodes represent ci ti es, and t he l i nks are the roads that connect t hese ci ti es. The di st ances i n ki l omet ers al ong each road are noted i n t he figure. Use the PAS met hod to fi nd the shortest route.
7.6 Problems
P r i m a l - D u a l M e th o d
F i n d t h e o p t i m u m s o l u t i o n o f t h e f o l l o w i n g p r o b l e m s u s i n g t h e p r i m a l - d u a l m e t h o d. U n l e s s s t a t e d o t h e r w i s e, a l l v a r i a b l e s a r e r e s t r i c t e d t o b e p o s i t i v e. V e r i f y s o l u t i o n s g r a p h i c a l l y f o r t w o v a r i a b l e p r o b l e m s.
7.2 9. M a x i m i z e z — x\ + 3x 2
S u b j e c t t o x i +4X2 < 1 0 a n d x i + 2x 2 < 1.
7.3 0. M a x i m i z e z = —x i + 2x 2
S u b j e c t t o x i — 4x 2 + 2 > 0 a n d 2 x i — 3x 2 < 3.
7.3 1. M i n i m i z e / = — 2 x i +.2x 2 + X 3 — 3x 4
S u b j e c t t o x i + X2 + X3 + X4 < 1 8, x i — 2x 3 + 4x 4 < 1 2, x i + X2 < 1 8, a n d X3 + 2x 4 < 1.
_________________________________________________________________________________________
7.3 2. M i n i m i z e / = — 3 x i - 4x 2
S u b j e c t t o x i + 2x 2 < 1 0, x i + X2 < 1 0, and3x\ + 5x 2 < 2 0.
7.3 3. M i n i m i z e / = — Ι Ο Ο χ ι — 8 OX2
S u b j e c t t o 5 x i + 3x 2 < 1 5 a n d x i + X2 < 4.
7.3 4. M a x i m i z e z = — xj + 2x 2
S u b j e c t t o x i — 2x 2 + 2 > 0 a n d 2 x i — 3x 2 < 3.
7.3 5. M i n i m i z e / = x i + 3x 2 — 2x 3
S u b j e c t t o x i — 2x 2 — 2x 3 > — 2 a n d 2 x i — 3x 2 — X3 < — 2.
7.3 6. M a x i m i z e z = x 1 + 3x 2 — 2x 3 + X4
S u b j e c t t o 3 x i + 2x 2 + X3 + X4 < 20, 2x i + X2 + x ± = 1 0, a n d 5 x i — 2x 2 —
X3 + 2x 4 ^ 3.
7.3 7. M i n i m i z e / = 3 x i + 4x 2 + 5x 3 + 6x 4
S u b j e c t t o 3 x i + 4x 2 + 5x 3 + 6x 4 > 20, 2 x i + 3x 2 + 4x 4 < 1 0 a n d 5 x i — 6x 2 — 7x 3 + 8x 4 > 3. V a r i a b l e s x i a n d x 4 a r e u n r e s t r i c t e d i n s i g n.
7.3 8. M i n i m i z e / = 1 3 x i — 3x 2 — 5
S u b j e c t t o 3 x i + 5x 2 < 2 0, 2 χ ι + Χ 2 > 1 0, 5 x i + 2x 2 > 3, a n d x i + 2x 2 > 3.
x i i s u n r e s t r i c t e d i n s i g n.
Chapter 7 I n t e r io r Point Methods
7.39. Minimize f = 5xi + 2x2 + 3x3 + 5x4
Subject to xi — X2 + 7x3 + 3x4 > 4, xi + 2x2 + 2x3 + X4 = -9, and 2xi + 3x2 + x3 — 4x4 < 5.
7.40. Minimize f = —3xi + 8x2 - 2x3 + 4x4
Subject to xi - 2x2 + 4x3 + 6x4 < 0, xi — 4x2 — X3 + 6x4 < 2, X3 < 3, and x4 > 3.
7.41. Minimize f = 3xi + 2x2
Subject to 2xi + 2x2 + X3 + X 4 = 10 and 2xi — 3x2 + 2x3 = 10. The variables should not be greater than 10.
7.42. Minimize / = -2xi + 5x2 + 3xj
Subject to xi — X2 — x3 < —3 and 2xi + X2 > 1.
7.43. Use the primal-dual method to solve the Hawkeye food problem 7.20.
7.44. Use the primal-dual method to solve the alloy manufacturing prob- lem 7.21.
7.45. Use the primal-dual method to solve the concrete block manufacturing problem 7.22.
7.46. Use the primal-dual method to solve the mining company problem 7.23.
7.47. Use the primal-dual method to solve the parking spaces assignment prob­
lem 7.24.
7.48. Use the primal-dual method to solve the drug manufacturing problem
7.25.
7.49. Use the primal-dual method to solve the auto seat problem 7.26.
7.50. Use the primal-dual method to solve the stock-cutting problem 7.27.
7.51. Use the primal-dual method to solve the network problem 7.28.
CHAPTER EIGHT
Q u a d r a t i c P r o g r a m m i n g
When the objective function is a quadratic function of the optimization vari- ables and all constraints are linear, the problem is called a quadratic pro­
gramming (QP) problem. Several important practical problems may directly be formulated as QP. The portfolio management problem, presented in Chap­
ter 1, is such an example. QP problems also arise as a result of approximating more general nonlinear problems by linear and quadratic functions.
Since QP problems have linear constraints, their solution methods are rela­
tively simple extensions of LP problems. The first section introduces a standard form for QP problems and presents KT conditions for its optimality. Sections 2 and 3 present extensions of two interior point methods (Primal Affine Scaling and primal-dual) to solving QP problems. Active set methods for primal and dual QP, using the conjugate gradient method, are presented in the last two sections. These methods exploit the special structure of the QP problems typ­
ically encountered as subproblems during the solution of general nonlinear programming problems discussed in the following chapter.
8.1 KT Conditions for Standard QP
Quadratic programming (QP) problems have quadratic objective functions and linear constraints. Tb simplify discussion, unless stated otherwise, it will be assumed that a QP is written in the following standard form:
Chapter 8 Quadratic Programming
Minimize cTx + | x TQx Subject to ( ^ “ o)
whe r e x i s an η χ 1 v e c t or o f opt i mi z at i on vari abl e s, A i s an m χ n c ons t r ai nt c o e f f i c i e n t mat ri x, b i s an m x 1 ve c t or o f c ons t r ai nt ri ght - hand s i de s, c i s an n x l ve c t or c ont ai ni ng c o e f f i c i e n t s o f l i n e a r t e r ms, and n x n mat r i x Q, c ont ai ns c o e f f i c i e n t s o f s quare and mi xe d t e r ms i n t he obj e c t i ve f unc t i on. Conve rs i on o f c ons t r ai nt s t o t he st andard f orm i s t he s ame as t hat di s c us s e d f or LP i n Chapt er 6. For t he me t hods pr e s e nt e d i n t hi s chapt er, i t i s no t n e c e s s a r y t o have po s i t i v e ri ght - hand s i d e s o f c ons t r ai nt s as was t he c as e f or t he s i mpl e x me t hod. Wri t i ng a quadrat i c f unc t i o n i n t he above mat ri x f orm was di s c us s e d i n Chapt er 3. Thus, al l QP probl ems c a n b e wr i t t e n i n t h e above s t andard f orm wi t h l i t t l e ef f ort. Furt hermore, i t i s e a s y t o s e e t hat a QP pr obl e m i s c onve x i f t he mat r i x Q. i s at l e a s t po s i t i v e s e mi de f i ni t e.
Fol l owi ng t he pr e s e nt at i on i n Chapt er 4, t he Lagrangi an f or t he st andard QP probl em i s as f ol l ows:
L ( x. i i. v. s ) = c r x + ^ x r Q x + ι ι Γ ( - x + s 2 ) + v T i - A x + b )
whe re u>Oi s annxl vector of Lagrange multipliers associated with positivity constraints, s is a vector of slack variables, and v is an m χ 1 vector of Lagrange multipliers associated with equality constraints. Differentiating the Lagrangian with respect to all variables results in the following system of equations:
d L
~ = 0 =>■ c + Qx — u — Arv = 0 or — Qx + Arv + u = c
dx
8 L
— = 0 =Φ· Ax — b = 0
3v
BL 2
—— = 0 = φ —X +8 = 0
du
— = 0 = φ · t i i S j = 0,i = 1,..., r t
as
m > 0, i = 1,..., n
The f i rs t t wo s e t s o f e quat i ons are l i near, whi l e t he o t he r t wo are nonl i ne ar. We c an e l i mi na t e s b y no t i ng t hat t he c ondi t i ons UjSi — 0, say that either u\ = 0 (in which case, —x\ < 0) or the corresponding i; =0 (in which case, Xi = 0). Thus,
8.1 K T Conditions f o r Standard QP
by maintaining the positivity of x; explicitly, the optimum of the QP problem can be obtained by solving the following system of equations:
A x — b = 0
— Q x + A r v + u = c
UjXi = 0, i = 1,..., n Xi>0,Ui>0,i=l,...,n
It is convenient to express the complementary slackness conditions (u,Xj = 0) in the matrix form as well. For this purpose, we define nxn diagonal matrices:
U = diag [Ui] X = diag [*,·]
Further, by defining an tt χ 1 vector e, all of whose entries are 1
e r = ( 1,1,..., 1 )
t h e c o mp l e me n t a r y s l a c k n e s s c o n d i t i o n s ar e wr i t t e n a s f o l l o ws:
X U e = 0
8.1.1 D u a l Q P P r o b l e m
Using the concept of Lagrangian duality presented in the last section of Chap­
ter 4, it is possible to define an explicit dual QP problem. The dual function is as follows:
M (u, v) = Min ^crx + ^xTQx — urx + v T (—Ax + b ) j u\ > 0, i = 1,..., n
The minimum can easily be computed by differentiating with respect to x and solving the resulting system of equations:
c + Q x — u — A r v = 0
Taking the transpose and multiplying by x
c T x + x r Q x — u T x — v r A x = 0
or
c r x — u r x — v r A x = —x T Q x Substituting into the dual function, we get
Chapter 8 Quadratic Programming
M (u, v ) — — ^xrQx + v r b Ui > 0, i = 1,..., n
Zx
The complete dual QP problem can be stated as follows:
Maximize — ^xTQx + vrb
For the special case when the matrix Q. is positive definite, and therefore the inverse of Q exists, the primal variables x can be eliminated from the dual by using the equality constraint. x = Q.- 1 £ — c + u + ArvJ The dual QP is then written as follows: Maximize —\[ —c + u + A T v ] r Q._ 1 [—c + u + A r v ] + v T b Subject to Uj > 0, i = 1,.,., n
Computationally, it may be advantageous to solve the dual problem since it has simple bound constraints.
T h e F o r m D u a l Q P F u n c t i o n
A Mathematica function called FormDualQP has been created to automate the process of creating a dual QP.
Needs["OptimizationToolbox'QuadraticProgramming'"];
?FormDualQP
FormDualQP[f, g, vars, options]. Forms dual of the given QP. f is the objective function, g is a list of constraints, and vars is a list of variables. See Options[FormDualQP] to find out about a list of valid options for this function.
OptionsUsage[FormDualQP]
{UnrestrictedVariables {} ,
ProblemType -»Min,
StandardVariableName -> x, DualQPVariableNames -» {u, v}}
UnrestrictedVariables is an option for LP and several QP problems.
A list of variables that are not restricted to be positive can be specified with this option. Default is {}.
ProblemType is an option Min (default) or Max.
8.1 K T Con ditions f o r Standard QP
StandardVariableName is an option for LP and QP methods. It specifies the symbol to use when creating variable names during conversion to the standard form. Default is x.
DualQPVariableNames-> symbols to use when creating dual variable names, Default is {u, v}.
Example 8.1 Construct the dual of the following QP problem. Solve the primal and the dual problems using KT conditions to demonstrate that they both give the same solution. Minimize f = —6x1^2 + 2xJ + 9x2 — 18xi + 9x|
Α-» (l 2) b-» (l5)
It is easy to see that Q is a positive definite matrix and therefore, using its inverse, we can write the dual in terms of Lagrange multipliers alone.
2 2 f = -6x^x2 + 2*1 + 9*2 ” 18x3_ + 9x2;
The dual is written as follows:
(df , dg, dvars, dv} = FormDualQP [f, g, vars]; Primal problem
2 2 Minimize -> -18X! + 2x^ + 9x2 - 6xjx2 + 9x2
Subject to -» (x-l + 2x2 == 15)
and -» (x! >0 x2 t
θ)
O 3 I O
Dual QP problem
Variables -» (u-l u2 v^)
Chapter 8 Quadratic Programming
The primal and the dual problems are solved using the KT conditions as follows:
KTSolution[f/ {g, Thread[vars
i
0] }, vars];
2 2 Minimize f -» -18x! +■
2xj + 9x2 - 6xix2 + 9x2
Vf ->
-18 + 4xi - 6x2
9 - 6xi + 18x2
***** LE constraints & their gradients gi -* -Xj sO g2 -» -X2 s 0
vgi
-1
0
vg2
***** EQ constraints & their gradients hj -> -15 + xi + 2 x2 == 0 V -» Jj)
***** Lagrangian -» Ui |si - - Ιδχχ + 2xf + u2 |s2 - x2) + 9x2 - 6x1x2 + 9x2
Vi (-15 + Xi + 2 x2)
■ 18 - Uj + V]_ + 4x! - 6 x2 == 0\
VL = 0 -*
9 - u2 +
2νι τ 6x! +
18x2 = s f - x j == 0 s2 - x2 == 0 - 15 + Xi + 2x2 = = 0
2 s i —— 0
2s2u2 == 0 ***** valid KT Point(s) *****
f -» -54.6983 Xl ->9.31034 x2 -> 2.84483 U1
= 0
u,
0
0
si ->9.31034 s2 -> 2
. 84483 v-l -> -2.17241
KTSolution [-df, dg, dvars, KTVarNames -» {U, S, V}];
117 15ui
2
1 Ul
— — + — - — + — - t - 2 u 5 2 2 4 ^
Minimize f
i
15 Ui Uo 5vi
„ ui u2 7v!
2 2 ul u2 u2 7vi Suivi 7u2Vi 29vi
18
18
36
7 bUi 7u, 29vi
W + 6“ + “ΓΪΓ + "ΤΐΠ'
*****
LE constraints & their gradients gi -*
-Ui <0 g2 -» -u2 s 0
8.1 KT Conditions f o r Standard Q p
r-l\ 0 '
Vg! -*
0 Vg2 -+ -1
0 J 0 j
|-Ui + Si j % -t- |-u2 + S 2) U2
-U! + Si] Ui -t- i-u2 + S 2) U2
VL = 0 -* 2 6 18 18
- Ui + Si == 0
- u2 + S2 == 0
2S 1U1 = = 0 2 S 2 U2 == 0
***** Valid KT Point(s) *****
f -> 5 4.6 9 8 3
u1 -> Q---------------------------------------------------------------------------------------
u 2 ®
V i —> 2.1 7 2 4 1 U i ^ 9.3 1 0 3 4 U2 -» 2.8 4 4 8 3
51 ^ 0
5 2 ^ 0
Both sol ut i ons are i dent i cal except for t he si gn o f v\. As pointed out in Chap­
ter 6, the sign of multipliers for the equality constraints is arbitrary. The multiplier will have a negative sign if both sides of the equality constraint are multiplied by a negative sign.
E x a m p l e 8.2 Construct the dual of the following Q P problem. Solve the primal and the dual problems using KT conditions to demonstrate that they both give the same solution.
^2
Minimize / = - 2xi + - 6x2 — *1*2 + *2
f = -2*! + - 6x2 - X 1X2 + x2
·
9 = { 3 x! + x 2 £ 2 5, - X i + 2 x 2 5 1 0 } ; v a r s = { x ^ x 2 } ;
Ch ap te r 8 Quadratic Programming
The dual is written as follows:
{df, dg, dvars, dv) = FormDualQP[f, g, vars]; Primal problem
Xl , 2
Minimize -» -2xi + — - 6x2 - ΧιΧ2 + *2
Subject to
3xi + x2 + x3 == 25 + 2Xo
+ Xa == 10
and -*
(xi a 0 x2 a 0 x3 > 0 b -»
x4
0)
3
1 1
0
1-1
2 0
1
-2'
r 1
-6
Q —►
-1
0
0
, 0 ,
, 0
25
10
- 1
2
0
0
0
0
0
0
The matrix Q is clearly not a positive definite matrix and therefore, the dual cannot be written in terms of Lagrange multipliers alone.
Dual QP problem
Variables -» (u! u2
u3 u4 Vi v2
Xl x 2 x 3 x 4 )
2
X- 2
Maximize -> 25vi + 10v2-----+ XiX2 - X2
2
1 - 2 - ui - 3vi + v2 + Xi - x2 == 0
6 - u2 - vi - 2v2 - Xi + 2x2 == 0
- u3 - Vi == 0
' - u4 - v2 -= 0
Ui a 0
u2 a 0 U3 a 0 u4 i 0
Subj ect t o -»
Us i n g KT c o n d i t i o n s, i t c a n e a s i l y b e v e r i f i e d t h a t b o t h t h e p r i ma l a n d t h e d ua l p r o b l e ms g i v e t h e s a me s o l u t i o n:
x i = 6.3 6 x2 = 5.9 2 / = - 3 0.6 2
8.2 T h e P r i m a l A f f i n e S c a l i n g M e t h o d f o r C o n v e x QP
The pri mal affi ne scal i ng (PAS) al gori thm for convex QP probl ems i s based on exactl y t he same i deas as t hose used for the PAS met hod for LP probl ems. Starting from an interi or f easi bl e poi nt, t he key st eps to compute the next poi nt
8.2 T he Primal A f f i n e Scaling Method f o r Convex QP
are scaling, descent direction, and step length. TWo scaling transformations are used to derive the descent direction. The step-length computations are a little more complicated because the optimum of a QP problem can either be at the boundary or in the interior of the feasible region.
The following example illustrates the ideas:
X2 7
Minimize f = —2x\ + -j- — 6x2 — *1*2 +
3*i X 2 < 2 5 — x i + 2 x 2 < 1 0
Subject to
xi + 2x2 < 15 X i > 0,i = 1,2
Introduci ng sl ack vari abl esx3, X4, andxs, the problem is written in the standard form as follows:
X" ry
Minimize / = —2xi + — 6x2 - *1X2 + x^
/ 3xi + X 2 + X3 = 2 5 — Xl + 2X2 + X4 = 1 0 x i + 2x 2 + X5 = 1 5 V X i > 0, i = 1,..., 5
Subject to
The probl em i s now i n t he standard QP form wi t h the fol l owi ng vectors and matrices:
c =
A =
( - A
/I
- 1
0
0
0>
- 6
- 1
2
0
0
0
0
Q.
=
0
0
0
0
0
0
0
0
0
0
0
\o J
\0
0
0
0
<v
( 3
1
1
0
°\
<25\
- 1
2
0
1
o l
b
= 1
10
2 0 0 1,
8.2.1 F i n d i n g a n I n i t i a l I n t e r i o r P o i n t
An initial interior point must have all values greater than zero and satisfy the constraint equations. Since the actual objective function does not enter into these considerations, the Phase I procedure of the PAS algorithm for linear problems is applicable. Thus, choose an arbitrary starting point x° > O, say
Chapter 8 Quadratic Programming
x° = [1,1,..., 1], and then from the constraint equations, we have
z° = b—Ax°
If z° = O, we have a starting interior point. If not, we introduce an artificial variable and define a Phase I LP problem as follows.
Minimize a
(
Ax -I- az = b\ χ > 0 I * > 0 )
The minimum of this problem is reached when a = 0 and at that point, Axfc = b, which makes x* an interior point for the original problem. Furthermore, if we set a = 1, then any arbitrary x° becomes a starting interior point for the Phase I problem. We apply the PAS algorithm for LP to this problem until a = 0 and then switch over to the QP algorithm for the actual problem (Phase II).
The following starting point for the example problem, however, was not determined by using the above procedure. It was written directly, after few trials, to satisfy the constraints. The goal was to have a starting solution that did not involve many decimal places. This avoided having to carry many significant figures in the illustrative example. Later examples use the above Phase I procedure to get an initial interior point.
x° — {5/2,17/4, 53/4, 4, 4}T
It is easy to see Ax° — b = 0 and therefore, this is a feasible solution for the problem:
A = { {3, 1, 1., 0/ 0}# {—1, 2, 0/ 1, 0} / {1/ 2, 0, 0, 1}}# b = {25, 10, 15}; c = (-2, - 6, 0,0,0};
Q = {{1, -1, 0, 0, 0}, {- 1, 2, 0, 0, 0}, {0, 0, 0, 0, 0},
{0, 0, 0, 0, 0}, {0, 0, 0, 0, 0}},·
xk = tf / 4, 4};
A,xk - b
{0, 0, 0}
8.2.2 D e t e r m i n i n g a D e s c e n t D i r e c t i o n
Similar to the LP case, a feasible descent direction is the projection of the negative gradient of the objective function on the nullspace of the constraint
8.2 T he Primal A f f i n e Scaling Method f o r Convex QP
coefficient matrix. After introducing two transformations and a lengthy series of manipulations (see the appendix to this chapter for detailed derivation), the direction vector is expressed as follows:
d* = - [ I - Hfc A r ( A H * A T ) - 1 A ] H fc( Q x * + c)
where Q and c define the objective function, A is the constraint coefficient matrix, is current feasible interior point, and
Hfc = [Q Ί- (T*)-2]-1
/l/( x j ) 2 0 0 0
0 1/θ φ 2 0 0
• · * *
« · * ·
\0 δ 0 l/( 4 ) 2/
By introducing the following additional notation, the direction vector can be written in a very simple form:
Define w* = (AH^A7)-1 AH^Qx* + c)
Then d* = —Hk(Qxk + c) + H*Arw* = —H^Qx11 + c—A Twk)
Further defining sk — Qx* + c — ATwk
we have
dk = - h V
Note that instead of inverting the matrix as indicated in its definition, w* is computed more efficiently by solving a linear system of equations as follows:
( A r f A V -= AHfc(Qx* + c)
It is interesting to observe that a QP problem reduces to an LP problem if matrix Q is a zero matrix. The descent direction formula given here also reduces to the one given for an LP problem if matrix Q is a zero matrix.
For the example problem, the descent direction is computed as follows:
Hk = Inverse [Q + DiagonalMatrix [1 „ / (xk)2 ] ]; MatrixForm[Hk]
Chapter 8 Quadratic Programming
1.48485
0.722428
0.
0.
0.
0.722428
0.838016
0.
0.
0.
0.
0.
175.562
0.
0.
0.
0.
0.
16.
0.
0.
0.
0.
0.
16. ,
AHkA = A.H k.T r a n s p o s e [A] ; M a t r i x F o r m [ A H k A ]
1Q4.0Q9
0.833617
11.1876
0.833617
17.9472
1.86721
11.1876
1.86721
23.7266
Qxo = Q .xk + c
{-H, 0,0, 0.0}
w = LinearSolv·[AHkA, A.Hk.Qxc]
{-0.0753651,0.0567999,-0.431975)
ek = Qxc - T r a n s p o s e [A] .w
{-3.03 513,0.825715, 0.0753651,-0.0567999,0.431975}
dk =-Hk.sk
{3.9102, 1.5007, -13.2313, 0.908799,-6.9116}
8.2.3 S t e p L e n g t h a n d t h e N e x t P o i n t
After knowing the feasible descent direction, the next step is to determine the largest possible step in this direction. Only the equality constraints are used during direction computations. The requirement that variables be positive is taken into consideration during the step-length calculations. Thus, we need to find the largest a such that the following conditions are satisfied in terms of scaled and original variables:
xk+1 = x fc + ordfc > O
Since the optimum of a QP can lie either inside the feasible domain or on the constraint boundary, we must examine two possibilities. If the optimum is on the constraint boundary, we have a situation similar to the LP case. Thus, the step length is determined by the negative entries in the dfc vector.
αι = βΜνη[-χί}ά\, dk < 0, i — 1,..., ti]
As for the LP case, the parameter β (with 0 < β < 1) is introduced to. ensure that the next point is inside the feasible region. Usually β = 0.99 is chosen in order to go as far as possible without a variable becoming negative.
If the optimum of QP is inside the feasible region, then the problem is essentially unconstrained. The step length is now determined by minimizing
8.2 T h e Primal A f f i n e Scaling Method f o r Convex QP
the objective function along the given direction. Thus, we solve the following one-dimensional problem:
Find a to minimize cT(xk + adfc) + %(xk + adk)TQ(xk + adfc)
The necessary condition for the minimum is that the first derivative with respect to or be zero. Thus, we get the following equation:
cTdk + i(d*)rQ(x* + ad*) + ^(x* + ad^QXd*) = 0
/i ii
or
( x fc + a d * ) r Q.( d fc) = —c r d *
or
x fcrQ.d * + a d fcTQ.d fc = — t ^ d *
Giving
_ crdfc + x * r Q d fc _ d f e r ( Q x * + c ) d f c r Q d fc d * r Q d fc
In practice, since we don't know whether the optimum is on the boundary or inside the feasible domain, we determine both ai and 012 and select the smaller of the two. The next interior feasible point is then given by
x*+1 = xfc + adfc
where a = Min[ai, «2]
For the example problem, the step length and the next point are computed as follows:
N[xk]
{2.5, 4.25, 13.25, 4., 4.} dk
{3.9102, 1.5007, -13.2313, 0.908799,-6.9116}
Kk/dk
{0.639354, 2.83201, -1.00141, 4.40142, -0.578737}
al = 0.99Hin [select [-Jtk/dk, Positive] ]
0.57295
aa = -dk. {Q.xk + c) /{dk.Q.dk)
1.81976
α = Hinfal, a21
0.57295
The new point is therefore as follows: xki = xk + adk
{4.74035, 5.10983, 5.66913, 4.5207, 0,04}
8.2.4 C o n v e r g e n c e C r i t e r i a
Starting with this new point, the previous series of steps is repeated until an optimum is found. Theoretically, the optimum is reached when dfc = 0. Thus, we can define the first convergence criteria as follows:
σι = Norm[dfc] < €\
where €\ is a small positive number.
In addition to this, because of the presence of round-off errors, the numerical implementations of the algorithm also check the following conditions derived from the KT optimality conditions.
Feasibility
The constraints must be satisfied at the optimum, i.e., Ax1 — b = 0. To use as convergence criteria, this requirement is expressed as follows:
_ ||Axfc — b[|
2 l|b|| + l - 2
where is a small positive number. The 1 is added to the denominator to improve numerical performance.
Dual Feasibility
It can be seen from section 1 that the following vector is related to the Lagrange multipliers u of the dual QP problem.
s fc = Q x * + c —
At the optimum, these dual variables should be positive; therefore, we have the following convergence criteria:
0"3 — ----------:--------------------- <
| | Q x * + c | | + l “ where €3 is a small positive number.
8.2 T h e Primal Affine Sca ling Method f o r Convex QP
Complementary Slackness
The primal and dual variables must satisfy the following complementary slack­
ness condition:
xkTsk = 0
Thus, we can define another convergence criteria as follows:
= Abs[xfcrsfc] < 64 where 64 is a small positive number.
8.2.5 C o m p l e t e P A S A l g o r i t h m f o r Q P P r o b l e m s
Phase I of the PAS algorithm for a QP problem is identical to that of an LP problem. The computations in Phase II are as follows.
Phase Π
Given: Constraint coefficient matrix A, constraint right-hand side vector b, ob­
jective function coefficient vector c, quadratic form matrix Q, current interior point x*, step-length parameter β, and convergence tolerance parameters. The next point χ*+1 is computed as follows:
1. Form scaling matrix:
A/( 4 ) 2
0
0
0 \
(Τ*Γ2 =
0
1 n 4 ) 2 0 0
\ 0 0 0 l/(x*)V
Hk = [Q. + (Τ*Γ2] _1
2. Solve system of linear equations for w k:
(AH.kAT)wk = AHfc(Qxfc + c)
3. Compute sk = Q x k + c — A T w k.
4. Compute direction dfc = —Hksfc.
C h ap te r 8 Quadratic Programming
5. Check for convergence. If
[ NOnn[d‘] £ "m + i" - 621 IIQ^'+cN+l 5 i3’AbstxlV] £ ««
we have the optimum. Otherwise, continue.
6. Compute step length a = Min[ai, a2l·
αχ = /iMin xi/dj, df < 0, i = 1,..., «j dkT(Qx* + c)
«2 =
dkTQdfc
7. Compute the next point x*+1 = xk + ctdk.
T h e F r i m f l l A f f i n e Q P F u n c t i o n
The following PrimalAffineQP function implements the Primal Affine algo­
rithm for solving QP problems. The function usage and its options are ex­
plained first. Several intermediate results can be printed to gain understanding of the process.
Needs["OptimizationToolbox'QuadraticProgramming'"];
?PrimalAffineQP
PrimalAffineQP[f, g, vars, options]. Solves a convex QP problem using Primal Affine algorithm, f is the objective function, g is a l i s t of constraints, and vars is a l i s t of variables. See
Options[PrimalAffineQP] to find out about a l i s t of valid options for this function.
OptionsUsage[PrimalAffineQP]
{UnrestrictedVariables -> {} , Maxlterations -» 20,
ProblemType ->Min, StandardVariableName -» x, PrintLevel -» 1, OptimizationToolbox'QuadraticProgramming'StepLengthFactor -*
0.99, ConvergenceTolerance-»{0.001,0.2, 2, 0.5} ,
OptimizationToolbox'QuadraticProgramming'StartingVector->{}}
UnrestrictedVariables is an option for LP and several QP problems.
A l i s t of variables that are not restricted to be positive can be specified with this option. Default is {}.
xlterations is an option for several optimizati specifies maximum number of iterations allowed.
8.2 The Primal A f f i n e Sca ling Method f o r Convex QP
problemType is an option for most optimization methods. It can either be Min (default) or Max.
StandardVariableName is an option for LP and QP methods. It specifies the symbol to use when creating variable names during conversion to the standard form. Default is x.
PrintLevel is an option for most functions in the OptimizationToolbox.
It is specified as an integer. The value of the integer indicates how much intermediate information is to be printed. A PrintLevel-» 0 suppresses all printing. Default for most functions is set to 1 in which case they print only the initial problem setup. Higher integers print more intermediate results.
StepLengthFactor is an option for interior point methods. It is the reduction factor applied to the computed step length to maintain feasibility. Default is 0,99
ConvergenceTolerance is an option for most optimization methods. Most methods require only a single zero tolerance value. Some interior point methods require a list of convergence tolerance values.
StartingVector is an option for several interior point methods. Default is {1, . . ., 1}.
E x a m p l e 8.3 The complete solution of the example problem used in the pre­
vious section is obtained in this example using the PrimalAffineQP function.
Minimize / = —2xi + — 6x2 — *1*2 + *2
(
3 x i + *2 < 2 5 \
- xi + 2x2 < 10 1 *,+2*2<15 Xi > 0, i
= 1, 2 /
Clear [xl, x2] ;
2
f = -2x1 + ^ - 6x2 - xlx2 + x22 ;
g = {3x1 + x2 S 25, -xl + 2x2 S 10, xl + 2x2 3 15} ; vars = {xl, x2} ;
All intermediate calculations are shown for the first two iterations.
PrimalAffineQP [ f, g, vars, PrintLevel -* 2,
Maxlterations -» 2,
StartingVector -»N[{§, 4, 4}] ];
Xi o
Minimize -» -2x3^ +-----6x2 “ xix2 + x2
2
Subject to -»
3xi + x2 + x3 == 25 i
- Xt + 2x2 + x4 == 10
, xx + 2x2 + xs == 15 j
Chapter 8 Quadratic P rogram mi ng
and -»
(X1 2= 0
x2 * 0
x3 > 0
X4
Problem variables
redefined e
3
1 1
0
O'
’25'
A ->
-1
2 0
1
0
b -»
10
, i
2 0
0
ly
15;
’-2'
r 1
-a
0 C
0
-6
-1
2
0 C
0
C -»
0
Q -*
0
0
0 0 0
0
0
0
0 C
0
0
I 0
0
0 c
0
Convex problem. Principal minors of Q-»{1,1,0,0,0} Starting point-»{2.5, 4.25, 13.25, 4., 4.}
Objective function
-*
-19 .9375 Status -»NonOptimum
***** Iteration 1 (Phase 2} *****
Tk2[diagonal] {0.16, 0.0553633, 0.00569598, 0.0625, 0.0625}
Π.16 -1 0
-1 2.05536 0
0 0 0.00569598
0 0 0
Q+Tk2
0
0
0
0.0625
0
0 0 0 0
0.0625;
.48485
0 .722428
0.
0. 0.
722428
0.838016
0.
0. 0.
0.
0.
175.562
0. 0.
0.
0.
0.
16. 0.
0.
0.
0.
0. 16.
H -»
(-3.75 0.
-19.4137
’ 5
.17698
3.0053
175.562
0.
0. 1
A. H ->
-0
039996
0,953605
0. 16.
0.
2
.92971
2.39846
0.
0.
16. j
’ 194.099
0.833617
11.1876’
A.H. AT -►
0.833617
17.9472
1.86721
t 11.1876
1.86721
23.7266,
Qx+c
0.
0.
0.
A.H.(Qx+c)
0.149985
-10.9864
w ■
' -3.03513 '
' 3.9102 '
-0.0753651'
0.825715
1.5007
0.0567999
s -»
0.0753651
d -»
-13.2313
-0.431975,
-0.0567999
0.908799
. 0.431975 ,
,-6.9116 ,
Convergence parameters-»{15.5308, 0., 0.668709, 1.57925} d. (Qx+c) -» -14.6632 d.Q.d-> 8.05778 «2-> 1.81976
8.2 T h e Primal A f f i n e Scaling Method f o r Convex QP
-β x/d -» (-0.63296 - 2.80369 0.9914 - 4.3574 0.57295)
0.57295
New point-» {4.74035, 5.10983, 5.66913, 4.5207, 0.04} Objective function -» -27.0162 Status -» NonOptimum
***** Iteration 2 (Phase 2) *****
T k 2 [ d i a g o n a l ] -> { 0.0 4 4 5 0 2. 0.0 3 8 2 9 9 r 0.0 3 1 1 1 4 8, 0.0 4 8 9 3 1 6,6 2 5.}
Q+Tk2 -»
1.0445
- 1
0
0
0
- 1
2.0383
0
0
0
0
0
0.0311148
0
0
0
0
0
0.0489316
0
0
0
0
0
625,
H->
'1.80539
0.885734
.0.
0.
0. ’
0.885734
0.925151
0.
0.
0.
0.
0.
32.139
0.
0. :■'! ,
0.
0.
0.
20.4367
0.
0.
0.
0.
0.
0.0016,
(
6.3019
3.58235
32.139
0.
0.
Α.Η-»
-0
0339227
0.964568
0. 20.4367
0.
3
.57686
2.73604
0.
0.
0.0016,
'54.6271
0.862799
13.4666'
A.H. AT -*
0.862799
22.3998
1.89521
.13.4666
1.89521
9.05053,
Qx+c -»
-2.36948
-0.520696
0.
0.
0.
A.H.(Qx+c)
-16.7975
-0.421868
-9.89993
w -»
-0.0557946'
0.0700823
s ->
' -1.1065 ' 1.44595 0.0557946
d -»
s 0.716946 -0.357653 -1.79319
, -1.02551 ,
-0.0700823 , 1.02551 ,
1.43225
,-0.00164081,
Convergence paramet ers-»{2.4308, 0., 0.610507, 2.18384}
<i. (Qx+c) -> -1.51256 d.Q.d -»1.28268 α2->1.17922
~jS x/d-> (-6.54574 14.1443 3.12987 - 3.12479 24.1344)
a -> 1.17922
New p o i n t - » {5.58579, 4.68807, 3.55457, 6.20964, 0.0380651} Objective funct i on -» -27 .9081 St at us -»NonOptimum
***** NonOptimum solution after 2 iterations
I n t e r i o r s ol ut i on -» {xl -» 5 .58579, x2 -> 4.68807} Objecti ve funct i on -»-27.9081
Convergence par amet er s-»{2.4308, 0., 0 . 610507, 2.18384}
Chapter 8 Quadratic Programming
The procedure is allowed to run until the optimum point is obtained.
{sol, history} =
PrimalAffineQP [f, g,
vars, StartingVector-» N ^'·
***** Optimum solution after 5 iterations *****
Objective function -» -27 .9496 Convergence parameters -»
{0.0000108694, 6 . 81389χ10'16, 0.317952, 0.000414031}
The graph in Figure 8.1 confirms the solution obtained by the PAS method. The plot also shows the solution history.
xhist = Transpose [Transpose [history] [ [{1, 2>] ] ]; TableForm[xhist]
2.5 4.25
4.74035 5.10983
5.58579 4.68807
5.57839 4.71062
5.59985 4.69988.
5.59985 4.69988
0 2 4 6 8 10
FIGURE 8.1 A graphical solution showing the search path.
8.2 T he Primal Affine Scaling Method f o r Convex QP
Example 8.4 Solve the following QP problem using the PrimalAffineQP method.
f = -6x1x2 + 2xl2 + 9x22 - 18x1 + 9x2; g = {xl + 2x2 3 15} ; vars = {xl, x2} ;
PrimalAffineQP[f, ff, vars, PrintLevel -> 2, Maxlterations -» 2];
2 2 M i n i m i z e -» - 18χ;ι + 2 x i + 9x 2 - 6x ^ x 3 + ^ x 2
Subject to (xj + 2x2 + X3 == is) and-» (xj > 0 x2 > 0 χ3 ϊ θ)
P r o b l e m v a r i a b l e s r e d e f i n e d a s: { x l - » X }, x 2 -» x 2 }
A-> (l 2 l) b -» (15)
-18’
' 4
-6
0’
c -»
9
Q -»
-6
18
0
, 0 ,
, 0 ■
0
0,
Convex problem. Principal minors of Q-> {4,36, 0}
Starting poi nt-» {1, 1, 1} Objective function -» -4 Status -» NonOptimum
***** Iteration 1 (Phase 1) *****
Tk [diagonal] -» {1, 1,
1, 1}
A.Tk.Tk. AT -» (l27)
A.T k.T k.c ( l l ) ( ^ )
11
11
T T 7
127
22
22
r -»
T T 7
d -»
TT7
11
11
“ TT7
TT7
6
6
127
Convergence pa r a met e r s -»| θ.217357, 0, 0.108679,
β
(-1/d) -» {-11.43, -5.715, -11.43, 20.955} Step length, a-» 20.955
New point -» {2.815,4.63,2.815} Artificial objective function -» 0.01 Objective function-» 121.58 Status-»NonOptimum
***** iteration 2 (Phase 1) *****
Tk [diagonal] -» {2.815, 4.63,2.815,0.01}
C h ap te r 8 Quadratic Programming
A.Tk.Tk. AT -» (101.596)
A.Tk.Tk.c -* (i.l χ 10-7) w —> (1.08272χ 10"9)
-1.08272 x 10“9'
’3.04785 χ IO"9'
-2.16544 xIO-9
d
1.0026 x 10”8
-1.08272 x 10~9
3.04785 xIO-9
0.01
i - 0.0 0 0 1 j
Convergence parameters-»
{0.0001, 0.00680625, 0.00990099, 0.0000999838} β
(-1/d) -» {-3.24819x10s, -9.87435xl07, -3.24819 x 10s, 9900.}
Step length, a
-» 0
New point-»{2.815, 4.63,2 .815} Artificial objective function-*0.01 Objective function-»121. 58 Status -» Optimum
*****
NonOptimum solution after 2
iterations *****
Interior solution {xl 2 . 815, x2 -» 4.63} Objective function -» 121.58 Convergence parameters
-*
{0.0001, 0.00680625, 0 .00990099, 0.0000999838}
The procedure is allowed to run until the optimum point is obtained.
{sol, history} = PrimalAffineQP [f, g,
vars];
*****
optimum solution after 5 iterations *****
Interior solution -» {xl -» 7.5006, x2 -» 1. 99962} Objective function -» -58.5 Convergence parameters -»
(0.000732458, 0.006875, 0.0113432, 0.0144618}
The graph in Figure 8.2 confirms the solution obtained by the PAS method. The optimum solution is inside the feasible domain; hence, the problem is essentially unconstrained.
xhist = Transpose [Transpose [history] [ [ {1, 2} ] ] ]; TableForm [xhist]
1
1
2.815
4.63
2.815
4.63
7.36173
1.93731
7.5006
1.99962
7.5006
1.99962
8.2 T he P r imal Affine Scaling Method f o r Convex QP
x 2
F I G U R E 8.2 A g r a p h ic a l s o lu t io n s h o w in g t h e s e a r c h p a t h.
Example 8.5 Solve the following QP problem using the PrimalAffineQP method. The variable x2 is unrestricted in sign.
£ = xl 2 + xlx2 + 2x2 2 + 2x32 + 2x2x3 + 4x1 + 6x2 + 12x3; g = {xl + x2 + x3 £ 6, -xl - x2 + 2x3 i.
2 } ; vars = {xl, x2 , x3 } ;
Intermediate results are printed for the first two iterations.________________
PrimalAffineQP [f, g, vars, UnrestrictedVariables -> {x 2 },PrintLevel -*■
2,
Maxlterations -» 2];
2 2 2 2 Minimize -» 4x1 + Χχ + 6x2 +X1X2 +2x2 - 6x3 -X1X3 - 4x2x3 + 2x3 + 12x4 + 2x2x4 - 2x3x4 + 2x4
Subject to -»
Xj + x2 - x3 + x4 - x5 == 6
-Xl - x2 + X3 + 2x4 - x6 = = 2 J and-» (xj_ >0 x2 i0 x3 a 0 x4 s 0 x5 a 0 Xg > θ)
Problem variables redefined as: {xl -* x
^, x2 -» x2 - X3, x3 -> x4 }
> I 1 1 - 1 1 ",1 ° ) b 16
- 1
1 2
- 1
Chapter 8 Quadratic Programming
r 4 ■
2
1
-1
0
0
0'
6
1
4
-4
2
0
0
-6
-1
-4
4
-2
0
0
12
Q -»
0
2
-2
4
0
0
0
0
0
0
0
0
0
, 0 ,
k o
0
0
0
0
0,
Convex problem. Pri nci pal minors of Q-> {2, 7, 0, 0, 0, 0}
Starti ng poi nt-» {1, 1, 1,
1, 1, 1}
Objective function-» 19 Status -»NonOptimum
***** i t erat i on 1 (Phase 1) *****
Tk[diagonal] -» {1, 1, 1, 1, 1, 1, 1}
f
30 9
9 12 J
A.Tk.Tk.AT
A.Tk.Tk.c -»
i - A
Ά
A
8
- j i
14
5 7
13
5 S
a->
w -»
3
Ϊ Τ
{14
5 1
A
A
14
- 5 1
5
“ 5T
13
"5 1
{0.373878, 0, 0.186939, —}
Convergence parameters
β
(-1/d) -» {-10.23,-10.23, 10.23,-3.83625, 6.57643, 18.414, 7 .08231} Step length, a-» 6.57643 New point-» {1.63643, 1.63643, 0.363571, 2.69714, 0.01, 0.646429} Arti fi ci al objective function-»0.0807143 Objective function -» 75.9651 Status -»NonOptimum
***** iteration 2 (Phase 1) *****
Tk[diagonal] -»
{1.63643, 1.63643, 0.363571, 2.69714, 0.01, 0.646429, 0.0807143}
f12.7637 9.0616 \
9.0616 35.0043 j
0.000212213 \
^
0.0000848851
j
W‘
-0.0000205628 0.0000336496
-0.0000205628 0.0000336496
0.0000205628 -7 .47606x10~6
0.0000368331 -1.82607 χ 10"7 1.48819 χ IO-6
A. Tk, Tk. AT -»
A.Tk.Tk.c -»
0.0000182607 -2.30217 x 10_f
-0.0000136563 0.0000182607 -2.30217 χ 10-6
0.0807073
a
-0.00651423
B.2 The P rimal A f f i n e Scaling Method f o r Convex QP
Convergence parameters -»{0.00651451, 0.054553/ 0-0746796, 0.00640984} y3 (-1/d) -> {-29420.8, -29420.8, 132423-, -26878. ,
5.42149 χ 106, -665239., 151.975}
Step length, a
->151.975
New point-»{1.6448, 1.6448, 0.363158, 2.71224, 0.00999972, 0.646575}
Artificial objective function -» 0.000807143 Objective function-»76.5792 Status -»NonOptimum
*****
NonOptimum solution after 2 iterations *****
Interior solution -» {xl -» 1.6448, x2 -» 1.28164, x3 -> 2 .71224}
Objective function-» 76.5792
Convergence parameters-» {0.00651451, 0.054553, 0.0746796, 0.00640984}
The optimum solution for the problem is computed as follows:
PrimalAffineQP [f, g, vars, UnrestrictedVariables -» {x2} ];
*****
optimum solution after 9
iterations *****
Interior solution -» {xl -> 4 . 0984, x2 -> - 0.964673, x3 -» 2 .49501)
Objective function -» 62 .8866
Convergence parameters
-*
{0.000669959, 0.0546009, 0.50019, 0.0166818}
The solution is fairly close to the one obtained by using KT conditions. Note that we must add the positivity constraints before solving the problem using KT conditions. In the PAS method, these are always implicit but not in the case of the KT solution.
KTSolution[f , Join[g, {xl i 0, x3 & 0} ] , vars] ;
Lagrangian -» 4x1+xl2 +6x2 +xlx2 +2x22 +12x3 + 2x2x3 +2x32 + ^6 - xl - x2 - x3 + sfj u^ +
***** Valid KT Point(s) *****
f -» 68.6667 xl -» 4.33333 x2 -» -1, x3 -» 2 . 66667 ux -» 14 .6667 u 2 3 . u3 -» 0 u4 -» 0
51 -» 0
52 -» 0 ->4.33333
S4 -» 2.66667
Chap ter 8 Quadratic Programming
8.3 The Primal-Dual Method for Convex QP
The necessary conditions for the optimum of QP problems are represented by the following system of equations:
Ax — b = 0 Primal feasibility
—Qx + Arv + u = c Dual feasibility
X U e = 0 Complementary slackness conditions
Xi > 0, «,· >0, i = 1,..., κ
The primal-dual interior point method is based on trying to solve the above system of equations directly. Because of the complementary slackness condi­
tions, the complete system of equations is nonlinear. We can use an iterative method, such as the Newton-Raphson method, to solve this system. The linear equations will be satisfied exactly at each iteration. The error in satisfying the complementary slackness condition will be reduced at each iteration. Using μ > 0 as an indicator of this error, at a given iteration, the complementary slackness conditions are of the following form:
X U e = μ β
8.3.1 D i r e c t i o n U s i n g t h e N e w t o n - R a p h s o n M e t h o d
Similar to the LP case, at each iteration we need to compute new values of 2n+m variables (x, u, and v). Using the Newton-Raphson method, the changes in these variables are obtained from solving the following system of equations:
(
A 0 0\ /d *\ / Axfc — b \
—Q I AT 1 I dM J = — I -Qx + ATv k + uk - c I U X 0 / \d J \ X U e -V e /
Note that the matrices U and X are defined by using the known values at the current iteration. However, to simplify notation, superscript k is not used on these terms. We can perform the computations more efficiently by first writing the three sets of equations explicitly, as follows:
8.3 T h e p n m a l - D u a l Method f o r Convex QP
(a) Ad* = —Axfc + b s r p
( b) —Qd* + d u + AT dv = Qx — ATv* — uk + c =
( c ) U d j c + X d » = - X U e + μ * β = r c
Multiplying equation (b) by X and substituting for X d M from equation (c), we get
- X Q d * + rc ~ V d x + XAT dv = Xrd
or
-[XQ_ + U]dx + XAr dv = Xrd - T c
Now multiplying by A and using equation (a), we get the following system of equations for solving for d^:
A f x Q, + u r 1 x Ar dv = rp + A fX Q. + Up1 (Xrd - rc)
Once dt, is known, we can compute dx from (d) and du from (c) as follows:
From (d) dx = [XQ. + U]"1 XAr dv - [XQ. + U]_1 (Xrd - rc)
or dx = [XQ. + U]"1 (XAr dv - Xrd + rc)
From (c) Xdu ~-rc - \Jdx
or d„ = X_I (re - Ud,)
8.3.2
The previous derivation did not take into account the positivity requirements Xi > 0 and iq > 0. Assuming that we start from positive initial values, we take care of these requirements by introducing a step-length parameter a.
X f c +1 = x * + a p d x
u f c + 1 = n k + a d d u v*+1 = v k + cxddv
Chapter 8 Quadratic Programming
The maximum value of the step length is the one that will make one of the jc,· or ui values go to zero.
Xi I (Κρίίχί ^ 0 U i H P l A t i u i ■*> 0 I — 1, 5 ft
Variables wi t h posi t i ve i ncrements obvi ousl y wi l l remai n posi ti ve regardl ess of t he step l engt h. Simi lar to t he LP case, i n order to st ri ctl y mai ntai n feasi bi l i ty, the actual st ep l engt h shoul d be sl i ght l y smal l er t han t he maxi mum. The actual step l engt h i s chos en as follows:
ctp = β Min [1, - X i/d x i, d x i < 0] otj = β Min [1, - m/d u i, d ui < 0]
The parameter β is between 0 and 1 with a usual value of β = 0.999. The variables are then updated as follows:
x*+1 = xfc + ofp dx u k + 1 = u k + a d d u vfc+1 = v k + ad dv
8.3.3 C o n v e r g e n c e C r i t e r i a
F e a s i b i l i t y
The constraints must be satisfied at the optimum, i.e., Axfc — b = 0. Tb use as convergence criteria, this requirement is expressed as follows:
_ HAx* — b|| ap l|b|| +1 - fl
where €\ is a small positive number. The 1 is added to the denominator to avoid division by small numbers.
Dual Feasibility
We also have the requirement that
—Qx + Arv* + uk = c
8.3 T h e P r i m a l - D u a l M e t h o d f o r C o n v e x Q P
T h i s g i v e s t h e f o l l o w i n g c o n v e r g e n c e c r i t e r i a:
I t a l l
Od = --------:------------ —— < e 2
| | Q x f c + c | | + l
w h e r e 6 2 i s a s m a l l p o s i t i v e n u m b e r.
C o m p l e m e n t a r y S l a c k n e s s
T h e v a l u e o f p a r a m e t e r μ d e t e r m i n e s h o w w e l l c o m p l e m e n t a r y s l a c k n e s s c o n d i t i o n s a r e s a t i s f i e d. N u m e r i c a l e x p e r i m e n t s s u g g e s t s d e f i n i n g a n a v e r a g e v a l u e o f μ a s f o l l o w s:
tt
w h e r e t t = n u m b e r o f o p t i m i z a t i o n v a r i a b l e s. T h i s p a r a m e t e r s h o u l d b e z e r o a t t h e o p t i m u m. T h u s, f o r c o n v e r g e n c e,
μ < €3
w h e r e 6 3 i s a s m a l l p o s i t i v e n u m b e r.
8.3.4 C o m p l e t e P r i m a l - D u a l Q P A l g o r i t h m
G i v e n: C o n s t r a i n t c o e f f i c i e n t m a t r i x A, c o n s t r a i n t r i g h t - h a n d s i d e v e c t o r b, o b ­
j e c t i v e f u n c t i o n c o e f f i c i e n t v e c t o r c, s t e p - l e n g t h p a r a m e t e r β, a n d c o n v e r g e n c e t o l e r a n c e p a r a m e t e r s.
I n i t i a l i z a t i o n: k = 0, a r b i t r a r y i n i t i a l v a l u e s ( > 0 ), s a y x * = u * = e ( v e c t o r w i t h a l l e n t r i e s 1) a n d v * = 0.
T h e n e x t p o i n t x fc+1 i s c o m p u t e d a s f o l l o w s.
1. S e t μ k = [ ^ r n ]/( f c + l ). C h e c k f o r c o n v e r g e n c e. I f
l | A x fc - b | | l | b | | + l
< e i,
I t a l l
| | Q x fc + c | | + 1
w e h a v e t h e o p t i m u m. O t h e r w i s e, c o n t i n u e.
2. F o r m:
r p = — A x k + b
Td = Q x fc — A T v * - u k + c
r c = —X U e + μ * β
3. S o l v e t h e s y s t e m o f l i n e a r e q u a t i o n s f o r d „:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -A [ X Q.+ U p1 X A r ^ = r p + A [ X Q. + U p1 ( X r j - r c )
4. C o m p u t e i n c r e m e n t s:
d x = [ X Q. + U ]"1 ( X A r d *, - X r d + r c ) d u = X - 1 ( r c — U d j c )
5. C o m p u t e s t e p l e n g t h s:
o t p - β M i n [ 1, - X i/d x i, d x i < 0 ] a d = β M i n [ 1, - U i/d u i, d ui < 0 ]
6. C o m p u t e t h e n e x t p o i n t:________________________________________________________________________________
X*+1 = x fc + 0f pdx
u fc+i _ u fc + a d v Jc+1 = v* + ad dv
T h e P r i m a l D u a l Q P F u n c t i o n
T h e f o l l o w i n g P r i m a l D u a l Q P f u n c t i o n i m p l e m e n t s t h e P r i m a l D u a l a l g o r i t h m f o r s o l v i n g Q P p r o b l e m s. T h e f u n c t i o n u s a g e a n d i t s o p t i o n s a r e e x p l a i n e d f i r s t. S e v e r a l i n t e r m e d i a t e r e s u l t s c a n b e p r i n t e d t o g a i n u n d e r s t a n d i n g o f t h e p r o c e s s.
N e e d s ["O p t i m i z a t i o n T o o l b o x'Q u a d r a t i c P r o g r e m m i n g'"] ;
?P r i m a l D u a l Q P
P r i m a l D u a l Q P [ f, g, v a r s, o p t i o n s ]. S o l v e s a n QP p r o b l e m u s i n g I n t e r i o r P o i n t a l g o r i t h m b a s e d o n s o l v i n g KT c o n d i t i o n s u s i n g t h e N e w t o n - R a p h s o n m e t h o d, f i s t h e o b j e c t i v e f u n c t i o n, g i s a l i s t o f c o n s t r a i n t s, a n d v a r s i s a l i s t o f v a r i a b l e s. S e e O p t i o n s [ P r i m a l D u a l Q P ] t o f i n d o u t a b o u t a l i s t o t v a l i d o p t i o n s f o r t h i s f u n c t i o n.
8.3 T h e p r i t n f l l - D n a l M e t h o d f o r C o n v e x Q P
O p t i o n s U s a g e [ P r i m a l D u a l Q P ]
{ U n r e s t r i c t e d V a r i a b l e s -» { }, M a x l t e r a t i o n s -» 2 0, p r o b l e m T y p e -»Mi n, S t a n d a r d V a r i a b l e N a m e -» x, P r i n t L e v e l -» 1, S t e p L e n g t h F a c t o r - > 0.9 9, C o n v e r g e n c e T o l e r a n c e -> 0 . 0 0 0 1, S t a r t i n g V e c t o r -» {}}
U n r e s t r i c t e d V a r i a b l e s i s a n o p t i o n f o r L P a n d s e v e r a l QP p r o b l e m s.
A l i s t o f v a r i a b l e s t h a t a r e n o t r e s t r i c t e d t o b e p o s i t i v e c a n b e s p e c i f i e d w i t h t h i s o p t i o n. D e f a u l t i s {}.
M a x l t e r a t i o n s i s a n o p t i o n f o r s e v e r a l o p t i m i z a t i o n m e t h o d s. I t s p e c i f i e s m a x i m u m n u m b e r o f i t e r a t i o n s a l l o w e d.
p r o b l e m T y p e i s a n o p t i o n f o r m o s t o p t i m i z a t i o n m e t h o d s. I t c a n e i t h e r b e Mi n ( d e f a u l t ) o r Max.
S t a n d a r d V a r i a b l e N a m e i s a n o p t i o n f o r LP a n d QP m e t h o d s. Xt s p e c i f i e s t h e s y m b o l t o u s e wh e n c r e a t i n g v a r i a b l e n a me s d u r i n g c o n v e r s i o n t o t h e s t a n d a r d f o r m. D e f a u l t i s x.
P r i n t L e v e l i s a n o p t i o n f o r m o s t f u n c t i o n s i n t h e O p t i m i z a t i o n T o o l b o x.
I t i s s p e c i f i e d a s a n i n t e g e r. T h e v a l u e o f t h e i n t e g e r i n d i c a t e s how mu c h i n t e r m e d i a t e i n f o r m a t i o n i s t o b e p r i n t e d. A P r i n t L e v e l - » 0 s u p p r e s s e s a l l p r i n t i n g.D e f a u l t f o r m o s t f u n c t i o n s i s s e t t o 1 i n w h i c h c a s e t h e y p r i n t o n l y t h e i n i t i a l p r o b l e m s e t u p. H i g h e r i n t e g e r s p r i n t m o r e i n t e r m e d i a t e r e s u l t s.
S t e p L e n g t h F a c t o r i s a n o p t i o n f o r i n t e r i o r p o i n t m e t h o d s. I t i s t h e r e d u c t i o n f a c t o r a p p l i e d t o t h e c o m p u t e d s t e p l e n g t h t o m a i n t a i n f e a s i b i l i t y. D e f a u l t i s 0.9 9
C o n v e r g e n c e T o l e r a n c e i s a n o p t i o n f o r m o s t o p t i m i z a t i o n m e t h o d s. Mo s t m e t h o d s r e q u i r e o n l y a s i n g l e z e r o t o l e r a n c e v a l u e. Some i n t e r i o r p o i n t m e t h o d s r e q u i r e a l i s t o f c o n v e r g e n c e t o l e r a n c e v a l u e s.
S t a r t i n g V e c t o r i s a n o p t i o n f o r s e v e r a l i n t e r i o r p o i n t m e t h o d s. D e f a u l t i s {1, . . . , 1}.
E x a m p l e 8.6 S o l v e t h e f o l l o w i n g Q P p r o b l e m u s i n g t h e P r i m a l D u a l Q P m e t h o d.
a 9
Mi ni mi z e / — - 2 x\ + - 6 x2 - X
1
X
2
+ x i
(
3xi +X2 < 2 5 \
— xi + 2 x 2 £ 1 0 I xi + 2 x2 < 15 I
Xi > 0, i = 1, 2 /
f = - 2 x 1 + ^ 1 ------ 6x2 - x l x 2 + x 2 2 ;
g = {3x1 + x 2 £ 2 5, - x l + 2x2 £ 1 0, x l + 2x2 ύ 1 5 }; v a r s = { x l, x 2 };
A l l i n t e r m e d i a t e c a l c u l a t i o n s a r e s h o w n f o r t h e f i r s t t w o i t e r a t i o n s.
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
P r i m a l D u a l Q P [ f, g, v a r s, P r i n t L e v e l -» 2, M a x l t e r a t i o n s -» 2 ];
. ■ ■ „ X1 2 M i n i m i z e - 2 x! + —— 6X2 - Χ χ Χ 2 + X2
2
S u b j e c t t o
r 3Xi + x 2 + X3 = = 2 5 - Xi + 2 x 2 + X4 == 1 0 , X i + 2 x 2 + X5 = = 1 5 ,
and -* (χ! ί 0 X2 ^ 0 x 3 ϊ 0 X4 i 0 x5 a θ) problem va r i a bl e s r edefi ned a s: { x l -»Xj^, x 2 -» x2 }
3
1
1
0
0]
( 2 5\
A ->
- 1
2
0
1
0
b -»
1 0
, 1
2
0
0
l j
l i s j
'- 2'
- 6
0
O -*■
£> 0 c 00
c
00c
'l ΙΊ C H *"j* C
0
l0 ,
0 0 0 0 0
l 0 0 0 0 0,
C o n v e x p r o b l e m. P r i n c i p a l m i n o r s o f Q - > { 1, 1, 0, 0, 0 } * * * * S t a r t i n g v e c t o r s * * * *
P r i m a l v a r s ( x ) - » { l.,l.,l · ·,!,,!.}
D u a l v a r s ( u )
M u l t i p l i e r s ( v ) -» { 0., 0., 0.}
O b j e c t i v e f u n c t i o n -» -,7 .5 S t a t u s N o n O p t i m u m
* * * * * I t e r a t i o n 1
f - 3.
f0·]
to
0
- 6.
0.
00
r d^
rt
1
0.
l n - J
- 1.
- 1.,
Ο O
P a r a m e t e r s: o p -* 0.7 6 0 0 6 3 a d - » l.0 8 5 0 5 μ - » 1.
rl _'
Ί '
X[ di agonal ] -*
1. 1 - 1 .
U[diagonal] -*
1.
X.
1
.1 ·,
^ ------- 0^ 0^ ^
-1. 3. 0. 0. 0.
X.Q +
0. 0. 1. 0. 0.
0
H
0
0
0
H
0
0
0
0
8.3 T h e P r i r a a l - D a a l M e t h o d f o r C o n v e x Q P
0.6 0.2 0. 0. 0 - 1
0.2 0.4 0. 0. 0.
I n v e r s e [X.Q + O] -»
o
o
O
o
o. 0. 0. 1. o.
0^ 0 ^ 0 ^
A.I n v e r s e ( X.Q + 0 ).X.A T -
8.
2.2 2 0 4 5 x l O - 1 6 4.
2.2 2 0 4 5 x l O - 1 6 4.
2.4
1.
1.
4.
7.'
’ 2.1 9 0 7 9'
r p + A. I n v e r s e [ X.Q + U ] ( X.r d - r c ) - »
4.
d v - »
2.7 6 3 1 6
1.
,- 2.6 3 1 5 8,
d x ■
4.1 9 7 3 7
f - 4.1 9 7 3 7\
4.2 1 7 1 1
- 4.2 1 7 1 1
3.1 9 0 7 9
d u - »
- 3.1 9 0 7 9
3.7 6 3 1 6
- 3.7 6 3 1 6
l - l. 6 3 1 5 8 J
I 1.6 3 1 5 8
- x/d x - » (
O B O D O B -
Q.6 1 2 9 0 3 ) c t p - » 0.6 0 6 7 7 4
- u/d u - » ( 0.2 3 8 2 4 5 0.2 3 7 1 2 9 0.3 1 3 4 0 2 0.2 6 5 7 3 4 < x > ) a d - » 0.2 3 4 7 5 8 N e w p r i m a l v a r s - » ( 3.5 4 6 8 5 3.5 5 8 8 3 2.9 3 6 0 9 3.2 8 3 3 9 O.O l )
N e w d u a l v a r s - » ( 0.0 1 4 6 3 3 4 0.0 1 0.2 5 0 9 3 6 0,1 1 6 5 6 8 1.3 8 3 0 3 )
N e w m u l t i p l i e r s - » ( θ.5 1 4 3 0 6 0.6 4 8 6 7 4 - 0.6 1 7 7 8 5 )
O b j e c t i v e f u n c t i o n - 2 2 .1 1 4 S t a t u s N o n O p t i m u m
* * * * * I t e r a t i o n 2 * * * * *
- 2.3 0 3 0 7
[ Ί.8 6 4 5 2\
3.1 4 5 8 1
1 4.3 2 5 4 8 J
r d
- 3.0 1 5 2 8
- 0.7 6 5 2 4 2
- 0.7 6 5 2 4 2
- 0.7 6 5 2 4 2
r „ - *
f O.0 7 0 1 8 0 4 0.0 8 6 4 9 4 6 - 0.6 1 4 6 8 8 - 0.2 6 0 6 5 5 0.1 0 8 2 5 3
P a r a m e t e r s: c r p 0.2 9 8 8 7 6 a d - » 0.9 6 7 4 6 6 μ - * 0.1 2 2 0 8 2 3
X [ d i a g o n a l ]
( ■ 3 .5 4 6 8 5 ’ i
( 0.0 1 4 6 3 3 4 ^
3.5 5 8 8 3
0.0 1
2.9 3 8 0 9
T J [ d i a g o n a l ] - »
0.2 5 0 9 3 6
3.2 8 3 3 9
0.1 1 6 5 6 8
. 0.01 ,
„ 1.3 8 3 0 3 ;
X.Q + O - »
3.5 6 1 4 9 - 3.5 4 6 8 5
- 3.5 5 8 8 3
0.
0.
0.
7.1 2 7 6 6
0.
0.
0.
0.
0.
0.2 5 0 9 3 6
0.
0.
0.
0.
0.
0.1 1 6 5 6 8
0.
0.
0.
0.
0.
1.3 8 3 0 3/
0.5 S 8 4 8 8 ------0.2 7 7 9 1 4
I n v e r s e [ X.Q + O]
0.2 7 8 8 5 2
0.
0.
0.
0.2 7 9 0 6
0.
0.
0.
0.
.9 8 5 0 8
0 -
0.
0.
0.
8.5 7 8 6 9
0.
0.
0.
0.
0.7 2 3 0 5 2/
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
A.I n v e r s e (X.Q + U) .Χ.ΑΤ-»
3 6.4 5 5 8
0.9 8 8 8 7 2
1 4.8 5 2 2
0.9 8 8 8 7 2
3 0.1 6 4 4
1.9 9 1 6 4
1 4.8 5 2 2 1.9 9 1 6 4 9.9 1 6 8 1/
- 2 6.7 7 8 5'
- 0.4 9 0 9 3 1'
r p + A.I n v e r s e [ X.Q + U ] ( X.r d - r c ) -»
- 1 9.1 9 7 5
d v - »
- 0.5 8 3 4 1 7
,- 1 3.9 9 8 1
,- 0.5 5 9 1 2 6,
1.9 9 2 6 6
f 0.0 1 1 5 6 5 5\
1.1 2 6 5 3
0.0 2 1 1 3 8 8
0.7 6 0 0 0 3
d u -»
- 0.2 7 4 3 1
2 .8 8 5 4 1
- 0.1 8 1 8 2 5
U.0 7 9 7 6 2 6 J
1 - 0.2 0 6 1 1 6/
d x -»
- x/d x - » (a > oo oo o d oo) CCp - » 1
- u/d u - » (00 oo 0.9 1 4 7 8 9 0.6 4 1 1 0 1 6.7 0 9 9 4 ) a d -» 0.6 3 4 6 9
N e w p r i m a l v a r s -» ( 5.5 3 9 5 2 4.6 8 5 3 6 3.6 9 6 0 9 6.1 6 8 7 9 0.0 8 9 7 6 2 6 )
N e w d u a l v a r s -» ( 0.0 2 1 9 7 3 9 0.0 2 3 4 1 6 6 0.0 7 6 8 3 4 0.0 0 1 1 6 5 6 8 1.2 5 2 2 1 )
N e w m u l t i p l i e r s - » { 0.2 0 2 7 1 7 0.2 7 8 3 8 5 - 0.9 7 2 6 5 6 )
O b j e c t i v e f u n c t i o n - * - 2 7.8 5 0 1 S t a t u s - » N o n O p t i m u m
* * * * * N o n O p t i m u m s o l u t i o n a f t e r 2 i t e r a t i o n s
I n t e r i o r s o l u t i o n -» { x l 5 .53952, x2 -» 4.68536} O b j e c t i v e f u n c t i o n - » - 2 7.8 5 0 1
The procedure i s al l owed to run unt i l t he opt i mum poi nt i s obtai ned.
{ s o l/ h i s t o r y } = P r i m a l D u a l Q P [ £, g, v a r s ];
* * * * * O p t i m u m s o l u t i o n a f t e r 7 i t e r a t i o n s * * *
I n t e r i o r s o l u t i o n -» { x l -* 5 . 59981, x 2 -» 4.69992} O b j e c t i v e f u n c t i o n -> - 27.9496
The graph i n Figure 8.3 conf i rms t he sol ut i on obtai ned and shows the path taken by t he met hod.
x h i s t = T r a n s p o s e [ T r a n s p o s e [ h i s t o r y ] [ [ { 1/ 2 } ] ] ]; T a b l e F o r m [ x h i s t ]
1.
3.5 4 6 8 5
5.5 3 9 5 2
5.5 7 4 7 2
5.5 9 4 3 8
5.5 9 8 8 8
5.5 9 9 8 1
1.
3.5 5 8 8 3 4.6 8 5 3 6 4.6 8 9 8 6 4.6 9 7 8 2 4.6 9 9 5 4 4.6 9 9 9 2
5.5 9 9 8 1 4.6 9 9 9 2
8.3 T h e P r i m a l - D u a l M e t h o d f o r C o n v e x Q P
x 2
10
0
4
6
2
8
x l
0 2 4 6 8 1 0
F I G U R E 8.3 A g r a p h i c a l s o l u t i o n s h o w i n g t h e s e a r c h p a t h.
E x a m p l e 8.7 S o l v e t h e f o l l o w i n g Q P p r o b l e m u s i n g t h e P r i m a l D u a l Q P m e t h o d. T h e v a r i a b l e x 2 i s u n r e s t r i c t e d i n s i g n.
£ = x l 2 + x l x 2 + 2x 22 + 2 x 3 2 +■ 2 x 2 x 3 + 4 x 1 + 6 x 2 + 1 2 x 3; g = { X l + x 2 + x 3 ί 6, - X l - x 2 + 2 x 3 & 2 } ; v a r s = { x l, x 2 , x 3 } ;
I n t e r m e d i a t e r e s u l t s a r e p r i n t e d f o r t h e f i r s t t w o i t e r a t i o n s.______________________________
P r i m a l D u a l Q P [ £, q, v a r s, t f t i r e s t r i c t e d V a r i a b l e s -» { x 2 },
P r i n t L e v e l -* 2, M a x l t e r a t i o n s -» 2 ] ;
• 2 2 2 2 M i n i m i z e -» 4 x - l +Χ χ + 6x 2 + χ ι χ 2 + 2*2 ~ 6x 3 - 4 x 2x 3 + 2*3 + 12x 4 + 2x 2X 4 - 2X3X4 + 2x 4
a n d - * ( χ 1 >0 x 2 £ 0 x 3 > 0 x 4 £ 0 x 5 i 0 x 6 & θ )
P r o b l e m v a r i a b l e s r e d e f i n e d a s: { x l -* x l r x 2 - » x 2 - x 3 , x 3 - » X 4}
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
’ 4
r 2
1
-1
0
0
°\
6
1
4
-4
2
0
0
-6
Q
-1
-4
4
-2
0
0
12
0
2
-2
4
0
0
0
0
0
0
0
0
0
. o'.
, 0
0
0
0
0
Oj
C o n v e x p r o b l e m. P r i n c i p a l m i n o r s o f Q - » { 2,7,0,0, 0,0 }
**** s t a r t i n g v e c t o r s ****
P r i m a l v a r s (x)
D u a l v a r s (u) - » { 1./ 1./ 1., 1., 1., 1.} M u l t i p l i e r s ( v ) - » { 0.,0.)
O b j e c t i v e f u n c t i o n -» 1 9. S t a t u s -»NonOpt i mum
* * * * * I t e r a t i o n 1 *****
5.
8. -10. 1 5. -1. U i. J
Ό.'
0.
0.
0.
0.
^0 J
P a r a m e t e r s: o p - » 0.7 3 5 2 2 1 o d - » 0.9 1 4 3 2 4 μ - * 1.
X [ d i a g o n a l ]
ϋ [ d i a g o n a l ]
X.Q + Π
3.
1.
- 1.
0.
1.
5.
- 4,
2.
- 1.
- 4.
5.
- 2.
0,
2.
0.
0.
0.
0.
0.
0.
0.
0.
0.
0.
0.
0.
1.
0.
0.
1 J
I n v e r s e [ X.Q +■ U ]
0.3 6 6 3 3 7
- 0.0 4 9 5 0 5
0.0 4 9 5 0 5
0.0 3 9 6 0 4
0.
0.
- 0.0 4 9 5 0 5
0.5 7 4 2 5 7
0.4 2 5 7 4 3
- 0.0 5 9 4 0 5 9
0.
0.
0 - 0 4 9 5 0 5
0.4 2 5 7 4 3
0.5 7 4 2 5 7
0.0 5 9 4 0 5 9
0.
0.
0.0 3 9 6 0 4
- 0.0 5 9 4 0 5 9
0.0 5 9 4 0 5 9
0.2 4 7 5 2 5
0.
0.
0,
0.
0 -
0.
0.
1 -,
A. I n v e r s e ( X.Q + U ).X.A T - +
1.5 5 4 4 6 - 0.0 4 9 5 0 5
- 0.0 4 9 5 0 5 2.7 7 2 2 8
(
1 0 7 7 2 3\ 1 7
6 7 5 2 4 7 J' d v ~* U'
7.0 1 1 4 9
5 6 0 9 2
8.3 T h e P r i m a l - D u a l M e t h o d f o r C o n v e x Q P
d x ■
0.1 3 5 6 3 2
- 0.1 3 5 6 3 2\
0.5 2 1 8 3 9
- 0.5 2 1 8 3 9
1.4 7 8 1 6
d u -*
- 1.4 7 8 1 6
- 0.1 9 0 8 0 5
0.1 9 0 8 0 5
- 6.0 1 1 4 9
6.0 1 1 4 9
t - 1.5 6 0 9 2 ,
1.5 6 0 9 2 ,
- x/d x - » (g> oo oo 5.2 4 0 9 6 0.1 6 6 3 4 8 0.6 4 0 6 4 8 ) ct p-» 0.1 6 4 6 8 5
- u/d u -* ( 7.3 7 2 8 8 1.9 1 6 3 0.6 7 6 5 1 6 00 00 00) a d -» 0.6 6 9 7 5 1
Ne w p r i m a l v a r s ->
( 1.0 2 2 3 4 1.0 8 5 9 4 1.2 4 3 4 3 0.9 6 8 5 7 7 0.0 1 0.7 4 2 9 4 1 )
Ne w d u a l v a r s -»
( 0.9 0 9 1 6 0.6 5 0 4 9 8 0.0 1 1.1 2 7 7 9 5.0 2 6 2 1 2.0 4 5 4 3 )
N e w m u l t i p l i e r s - » ( 4.6 9 5 9 6 ,1.7 1 5 1 8 )
O b j e c t i v e f u n c t i o n -» 1 7 .2 7 2 3 S t a t u s -> N o n O p t i m u m
* * * * * I t e r a t i o n 2 * *
* * *
f 1.9 9 7 2 4 4.6 9 8 2 5
'- 0.5 7 0 2 5 5' - 0.3 4 7 1 8 8
/4.1 7 6 5 8\ r P"( l. 6 7 0 6 3 ] r * -
- 5.3 5 8 7 5 6.3 0 5 2 2 - 0.3 3 0 2 4 9 ,- 0 .3 3 0 2 4 9,
r c -»
0.3 4 6 7 7 8 - 0.7 3 3 1 4 1 0.3 0 8 9 5 ,- 1.1 6 0 4 2 >
P a r a m e t e r s: o p -* 0.6 1 4 1 4 1 a d - * 0.4 5 5 2 0 2 μ - » 0.3 5 9 2 1 2
X [ d i a g o n a l ]
'1.0 2 2 3 4 '
r 0.9 0 9 1 6'
1.0 8 5 9 4
0.6 5 0 4 9 8
1.2 4 3 4 3
O [ d i a g o n a l ] -»
0.0 1
0.9 6 8 5 7 7
1.1 2 7 7 9
0.0 1
5.0 2 6 2 1
,0.7 4 2 9 4 1,
, 2.0 4 5 4 3 ,
X.Q + U -
( 2.9 5 3 8 3 1.0 8 5 9 4 - 1.2 4 3 4 3 0.
0.
0.
1.0 2 2 3 4 4 .9 9 4 2 5 - 4.9 7 3 7 2 1.9 3 7 1 5 0.
0.
I n v e r s e [ X.Q + U ] -»
- 1.0 2 2 3 4
- 4.3 4 3 7 6
4.9 8 3 7 2
- 1.9 3 7 1 5
0.
0.
0.
0.3 7 9 1 2 5
- 0.0 0 1 5 5 3 3 5
0.1 1 5 6 9 9
0.0 4 5 4 0 8 3
0.
0.
- 0.0 0 1 4 6 2 3 7 0.0 9 5 1 2 6 9
1.5 1 6 9 7
1.5 1 2 7 5
- 0.0 0 1 6 3 6 3
0.
0.
A.i n v e r s e ( X.Q + U ).X.A T
1.3 2 1 1 5
1.5 9 6
0.1 0 6 4 4 1
0.
0.
0.5 6 6 3 5
2.1 7 1 8 8
- 2.4 8 6 8 6
5.0 0 2 1
0.
0.
0.
0.
0.
5.0 2 6 2 1
0.
0.
0.
0.
0.
2.0 4 5 4 3
0.0 4 7 9 2 8 6
- 0.0 0 1 8 3 4 5 6
0.1 3 6 6 4 5
0.2 5 3 5 4 5
0.
0.
- 0.0 9 0 7 4 5 1
0.
0.
0.
0.
0.1 9 8 9 5 7
0.
0.
0.
0.
0.
0.
0.4 8 8 8 9 5/
- 0.0 9 0 7 4 5 1 2.1 9 0 5 5
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
r p + A. I n v e r s e [X.Q + U] ( X.r d - r c ) -»
' 2 - 8 7 9 5 3 ' 1.0 1 6 7 7 1.3 1 8 9 8
d u -* '
- 2.6 6 3 5 6
1 1.6 8 9 6
- 3.1 1 8 5 5 ’ - 0.9 2 8 7 7 8 0.2 6 8 2 8 1
1.6 3 7 4 7
0.0 3 8 2 1 0 6
1 - 0.9 7 3 0 1 9;
1.1 1 6 9 4
- x/dx-* (od od αι σ> od 0.763542] otp -* 0.755906
- u/d u -» (O .2 9 1 5 3 3 0.7 0 0 3 8 oo 0.4 2 3 4 1 5 oo w) a d -» 0.2 8 8 6 1 7
New p r i m a l v a r s -»
( 3.1 9 8 9 9 1.8 5 4 5 2 2.2 4 0 4 6 2.2 0 6 3 5 0.0 3 8 8 8 3 7 0.0 0 7 4 2 9 4 1 )
New d u a l v a r s -»
( 0.0 0 9 0 9 1 6 0.3 8 2 4 3 6 0.0 8 7 4 3 0 5 0.3 5 9 0 4 3 8.4 0 0 0 2 2.3 6 7 8 )
New m u l t i p l i e r s -» ( 8.1 6 5 0 9 2.1 3 2 8 6 )
O b j e c t i v e f u n c t i o n -» 54 .2 8 6 3 S t a t u s -» NonOpt i mum
* * * * * N o n O p t i m i m s o l u t i o n a f t e r 2 I t e r a t i o n s * * ***
I n t e r i o r s o l u t i o n -» { x l -> 3.1 9 8 9 9, x2 -» - 0.3 8 5 9 3 4, x3 - > 2.2 0 6 3 5 } O b j e c t i v e f u n c t i o n - » 54 .2 8 6 3
T h e o p t i m u m s o l u t i o n f o r t h e p r o b l e m i s c o m p u t e d a s f o l l o w s:
{ s o l, h i s t } = P r i m a l D u a l Q P [ £, g, v a r s, U n r e s t r i c t e d V a r i a b l e s -» { x 2 } ];
* * * * * Opt i mum s o l u t i o n a f t e r 7 I t e r a t i o n s *****
I n t e r i o r s o l u t i o n -* { x l -» 4.3 3 3 3 4, x2 -» - 1.0 0 0 0 4, x3 - » 2.66672}
O b j e c t i v e f u n c t i o n -» 6 8.6 6 7 4
T h e c o m p l e t e s e a r c h h i s t o r y, i n t e r m s o f o r i g i n a l v a r i a b l e s, i s a s f o l l o w s:
T a b l e F o r m [Map [ { * [ [!] ],* [ [ 2 ] ] - #[ [3] ] , * [ [ 4] ] ) *, h i s t ] ]
1.0 2 2 3 4 - 0.1 5 7 4 9 1 0.9 6 8 5 7 7
3.1 9 8 9 9 - 0.3 8 5 9 3 4 2.2 0 6 3 5
4.0 0 5 1 - 0.8 3 9 6 7 4 2.5 4 1 6 9
4.3 3 3 6 4 - 1.0 0 1 1 3 2.6 6 8 4 5
4.3 3 3 5 8 - 1.0 0 0 3 7 2.6 6 6 9 2
4.3 3 3 3 4 - 1.0 0 0 0 4 2.6 6 6 7 2
4.3 3 3 3 4 - 1.0 0 0 0 4 2.6 6 6 7 2
1.
0.
1.
E x a m p l e 8.8 P o r t f o l i o m a n a g e m e n t C o n s i d e r t h e o p t i m u m s o l u t i o n o f t h e f o l l o w i n g p o r t f o l i o m a n a g e m e n t p r o b l e m, f i r s t d i s c u s s e d i n C h a p t e r 1. T h e p r o b l e m s t a t e m e n t i s a s f o l l o w s.
8.3 T h e P r i m a l - D u a l M e t h o d f o r C o n v e x Q P
A p o r t f o l i o m a n a g e r f o r a n i n v e s t m e n t c o m p a n y i s l o o k i n g t o m a k e i n v e s t ­
m e n t d e c i s i o n s s u c h t h a t t h e i n v e s t o r s w i l l g e t a t l e a s t a 10% r a t e o f r e t u r n w h i l e m i n i m i z i n g t h e r i s k o f m a j o r l o s s e s. F o r t h e p a s t s i x y e a r s, t h e r a t e s o f r e t u r n i n f o u r m a j o r i n v e s t m e n t t y p e s a r e a s f o l l o w s:
Type
Annual rates of return
Average
Blue-chip stocks
18.24
12.12
15.23
5.26
2.62
10.42
10.6483
Tfechnology stocks
12.24
19.16
35.07
23.46
-10.62
-7.43
11.98
Real estate
8.23
8.96
8.35
9.16
8.05
7.29
8.34
Bonds
8.12
8.26
8.34
9.01
9.11
8.95
8.6317
T h e o p t i m i z a t i o n v a r i a b l e s a r e a s f o l l o w s:
xi
Portion of capital invested in blue chip stocks
*2
Portion of capital invested in technology stocks
*3
Portion of capital invested in real estate
X4
Portion of capital invested in bonds
T h e o b j e c t i v e i s t o m i n i m i z e r i s k o f l o s s e s. C o n s i d e r i n g c o v a r i a n c e b e t w e e n i n v e s t m e n t s a s a m e a s u r e o f r i s k, t h e f o l l o w i n g o b j e c t i v e f u n c t i o n w a s d e r i v e d i n C h a p t e r 1:
Minimize / = 29.0552xf + 80.7818x2xi - 0.575767x3xi - 3.90639x4xi + 267.344x2 + 0.375933x1 + 0.159714xJ + 13.6673x2x3 - 7.39403x2x4- 0.1 1 3 2 6 7 X3 X4
T h e c o n s t r a i n t s a r e a s f o l l o w s:
T b t a l i n v e s t m e n t x i + x2 + X3 + X4 = 1
D e s i r e d r a t e o f r e t u r n 1 0.6 4 8 3 x i - f - 1 1.9 8 x 2 + 8.3 4x 3 + 8.6 3 1 7 x 4 > 1 0
A l l o p t i m i z a t i o n v a r i a b l e s m u s t b e p o s i t i v e.
T h e f o l l o w i n g M a t h e m a t i c a e x p r e s s i o n s g e n e r a t e t h e a b o v e o b j e c t i v e f u n c ­
t i o n a n d c o n s t r a i n t s d i r e c t l y f r o m t h e g i v e n d a t a.
b l u e C h i p s t o c k s = { 1 8.2 4, 1 2.1 2, 1 5.2 3, 5.2 6, 2.6 2, 1 0.4 2 };
t e c h S t o c k s = { 1 2.2 4, 1 9.1 6, 3 5.0 7, 2 3.4 6,- 1 0.6 2,- 7.4 3 };
r e a l E s t a t e = {8 .2 3, 8.9 6, 8.3 5, 9 .1 6, 8. 0 5, 7 .2 9 } ;
b o n d s = { 8.1 2, 8.2 6, 8.3 4, 9.0 1, 9.1 1, 8.9 5 };
r e t u r n s = { b l u e C h i p s t o c k s, t e c h S t o c k s, r e a l E s t a t e, b o n d s } ;
a v e r a g e R e t u r a s = Map [ A p p l y [ P l u s, #] /L e n g t h [#] r e t u r n s ]
{ 1 0.6 4 8 3,1 1.9 8,8.3 4,8.6 3 1 6 7 }
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
c o v a r i a n c e [ x., y.] : = Mo d u l e [ { x b, y b, n = L e n g t h [ x ] }, x b = A p p l y [ P l u s, x] /n; y b = A p p l y [ P l u s, y ] /n;
A p p l y [ P l u s, (x - xb ) ( y - yb ) ] /n ] ;
Q = O u t e r [ c o v a r i a n c e, r e t u r n s, r e t u r n s, 1 ]; M a t r i x F o r m [ Q ]
2 9.0 5 5 2 4 0.3 9 0 9 - 0.2 8 7 8 8 3 3 - 1.9 5 3 2
4 0.3 9 0 9 2 6 7.3 4 4 6.8 3 3 6 7 - 3.6 9 7 0 2
- 0.2 8 7 8 8 3 3 6.8 3 3 6 7 0.3 7 5 9 3 3 - 0.0 5 6 6 3 3 3 - 1.9 5 3 2 - 3.6 9 7 0 2 - 0.0 5 6 6 3 3 3 0.1 5 9 7 1 4 ,
C l e a r [ x ];
v a r s = T a b l e [ X i, { i, 1, L e n g t h [ a v e r a g e R e t u m s ] } ];
£ = E x p a n d [ v a r s.Q.v a r s ]
2 9.0 5 5 2 x 1 + 8 0.7 8 1 8 x 1x 2 + 2 6 7.3 4 4 x | - 0.575767XJX3 + 1 3.6 6 7 3 x 2x 3 + 0.3 7 5 9 3 3 x § - 3.90639X 3X4 - 7.3 9 4 0 3 x 2 x 4 - 0.1 1 3 2 6 7 x 3 x 4 + 0.1 5 9 7 1 4 x 4
g = { A p p l y [ P l u s, v a r s ] = = 1, a v e r a g e R e t u m s,v a r s & 1 0 }
{ x 2 + x 2 + x 3 + X4 == 1, 1 0 .6 4 8 3 x - l + 1 1. 9 8 x 2 + 8 .3 4 x 3 + 8.6 3 1 6 7 x 4 & 1 0 }
{ s o l, h i s t } = P r i m a l r m a l Q P [ f, g, v a r s ];
M i n i m i z e -* 2 9 . 0 5 5 2 χ ^ + 8 0 . 7 8 1 8 x ^ 2 + 2 6 7 . 3 4 4 x 2 ~ 0 · 5 7 5 7 6 7 x 1 x 3 + 1 3 .6 6 7 3 x 2 X3 +
0.3 7 5 9 3 3 x § - 3.9 0 6 3 9 χ α χ 4 - 7.3 9 4 0 3 x 2 x 4 - 0 .1 1 3 2 6 7 x 3 x 4 + 0.1 5 9 7 1 4 X 4
S u b j e c t t o x a + x 2 + x 3 + x 4 « l \
J \1 0.6 4 8 3 X i + 1 1.9 8 x 2 + 8.3 4 x 3 + 8.6 3 1 6 7 X 4 - x 5 = = 1 0/
a n d - » ( x- l > 0 x 2 > 0 x 3 > 0 x 4 > 0 x 5 > θ )
P r o b l e m v a r i a b l e s r e d e f i n e d a s: { χ! -» , x 2 -» x 2, x 3 -» x 3, x 4 -» X4 }
A_ > 1 1 0.6 4 8 3 1 1.9 8 8.3 4 8.6 3 1 6 7 - l j 1 1 0
0 0
' 5 8.1 1 0 4 8 0.7 8 1 8
8 0.7 8 1 8
5 3 4.6 8 9
- 0.5 7 5 7 6 7
1 3.6 6 7 3
- 3.9 0 6 3 9
- 7.3 9 4 0 3
0
6
>
c -»
0
Q τ»
- 0.5 7 5 7 6 7
1 3.6 6 7 3
0.7 5 1 8 6 7
- 0.1 1 3 2 6 7
0.
•lf
0
- 3.9 0 6 3 9
- 7.3 9 4 0 3
- 0.1 1 3 2 6 7
0.3 1 9 4 2 8
0
,0,
0
0
0
■ 0
0,
C o n v e x p r o b l e m.
P r i n c i p a l m i n o r s o f Q- » { 5 8.1 1 0 4, 2 4 5 4 5.3, 6 1 5 1.3 7, 1 1 7.2 5 3, 0.}
* * * * * O p t i mu m s o l u t i o n a f t e r 8 i t e r a t i o n s * * * * *
I n t e r i o r s o l u t i o n -»
{ x j -» 0.6 2 9 2 3 4, X2 -» 0.0 2 9 6 8 3 7, x 3 -» 0 .0 0 0 0 1 8 5 6 4, x 4 -» 0.3 4 1 0 6 4 }
O b j e c t i v e f u n c t i o n -» 1 2 .3 5 3 8
T h e c o m p l e t e h i s t o r y o f t h e a c t u a l f o u r v a r i a b l e s d u r i n g i t e r a t i o n f o l l o w s:
T a b l e P o r m [ T r a n s p o s e [ D r o p [ T r a n s p o s e [ h i s t ], - 1 ] ] ]
8.4 A c t i v e S e t M e t h o d
1.
1.
1.
1.
0.5 7 5 3 4 6
0.5 0 3 3 8 3
0.4 2 4 3 9 1
0.9 8 7 0 7 7
0.5 5 7 1 2 2
0.3 3 9 8 7
0.0 0 4 2 4 3 9 1
1.0 4 4 0 4
0.6 2 1 3 4 8
0.0 4 1 7 3 2 3
0.0 2 0 7 3 0 9
0.3 3 6 1 6 3
0.6 2 5 8 3 3
0.0 3 2 4 3 8 8
0.0 0 3 7 2 6 2 7
0.3 3 8 0 0 2
0.6 2 8 5 1 2
0.0 3 0 2 6 9
0.0 0 0 7 4 2 7 3 7
0.3 4 0 4 7 7
0.6 2 9 1 4 6
0.0 2 9 7 6 0 3
0.0 0 0 1 2 8 4 4 4
0.3 4 0 9 6 6
0 02 9 6 8 3 7
0.0 0 0 0 1 8 5 6 4
0 3 4 1 0 6 4
y( JeJV
0.6 2 9 2 3 4
0.0 2 9 6 8 3 7
0.0 0 0 0 1 8 5 6 4
0.3 4 1 0 6 4
T h e o p t i m u m s o l u t i o n i n d i c a t e s t h a t, u n d e r t h e g i v e n c o n d i t i o n s, t h e p o r t f o l i o m a n a g e r s h o u l d i n v e s t 6 3 % o f c a p i t a l i n b l u e c h i p s t o c k s, 3 % i n t e c h n o l o g y s t o c k s, 0 % i n r e a l e s t a t e, a n d 3 4 % i n b o n d s.
8.4 A c t i v e S e t M e t h o d
I n t h i s s e c t i o n, w e c o n s i d e r s o l v i n g Q P p r o b l e m s o f t h e f o l l o w i n g f o r m:
M i n i m i z e c r x + ^ x T Q x
N o t e t h a t t h e o p t i m i z a t i o n v a r i a b l e s a r e n o t r e s t r i c t e d t o b e p o s i t i v e. T h e f i r s t p c o n s t r a i n t s a r e l i n e a r e q u a l i t i e s, a n d t h e r e m a i n i n g a r e l e s s t h a n ( L E ) t y p e i n ­
e q u a l i t i e s. F u r t h e r m o r e, i t w i l l b e a s s u m e d t h a t Q, i s a p o s i t i v e d e f i n i t e m a t r i x. P r o b l e m s o f t h i s f o r m a r i s e d u r i n g t h e d i r e c t i o n - f i n d i n g p h a s e w h e n s o l v i n g g e n e r a l n o n l i n e a r o p t i m i z a t i o n p r o b l e m s u s i n g l i n e a r i z a t i o n t e c h n i q u e s, d i s ­
c u s s e d i n t h e n e x t c h a p t e r.
O b v i o u s l y, a Q P o f t h i s t y p e c a n b e s o l v e d b y f i r s t c o n v e r t i n g t o s t a n d a r d Q P f o r m a n d t h e n u s i n g t h e m e t h o d s d i s c u s s e d i n t h e p r e v i o u s s e c t i o n s. H o w e v e r, s i n c e t h e a b o v e f o r m i s s i m i l a r t o a g e n e r a l n o n l i n e a r p r o b l e m, i t i s m o r e c o n v e n i e n t t o u s e i t d i r e c t l y f o r d i r e c t i o n f i n d i n g, a s w i l l b e s e e n i n C h a p t e r 9.
A s b e f o r e, s t a r t i n g f r o m a n a r b i t r a r y p o i n t x °, o u r b a s i c i t e r a t i o n i s o f t h e f o l l o w i n g f o r m:
x fe+1 = x k + a d k
w h e r e a i s a s t e p l e n g t h a n d d i s a s u i t a b l e d e s c e n t d i r e c t i o n.
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
8.4.1 D i r e c t i o n F i n d i n g
A t t h e c u r r e n t p o i n t x fc, s o m e o f t h e i n e q u a l i t y c o n s t r a i n t s m a y b e s a t i s f i e d a s e q u a l i t i e s · T h e s e c o n s t r a i n t s t o g e t h e r w i t h t h e e q u a l i t y c o n s t r a i n t s c o n s t i t u t e w h a t w e c a l l a n a c t i v e s e t. T h e s e t o f a c t i v e c o n s t r a i n t i n d i c e s c a n t h e r e f o r e b e d e f i n e d a s f o l l o w s:
w * = { l,..., p ] U { i'l a/x * = b i, t = p + 1,..., m j
w h e r e t h e s y m b o l U s t a n d s f o r t h e u n i o n o f t w o s e t s. T h e d i r e c t i o n - f i n d i n g p r o b l e m c a n t h e r e f o r e h e s t a t e d a s f o l l o w s:
M i n i m i z e c r ( x f c + d f c ) + ^ ( x f c + d f c ) r Q _ ( x f c + d k )
S u b j e c t t o a j ( x fc + d fc) = fo,· i e w k
E x p a n d i n g a n d i g n o r i n g t h e c o n s t a n t t e r m s i n t h e o b j e c t i v e f u n c t i o n, w e h a v e
M i n i m i z e [ Q x fc + c ] r d fc + ^ d f c r Q d fc
S u b j e c t t o a f d fc — f o, — a f x fc = 0 i e w k
I n t r o d u c i n g t h e n o t a t i o n
= Q x * + c a n d A = i € w k
W e h a v e t h e f o l l o w i n g e q u a l i t y - c o n s t r a i n e d Q P p r o b l e m f o r d i r e c t i o n d f c: M i n i m i z e g * r d fc + ^ d k T Q ^ d fc
S u b j e c t t o A d k = 0
T h e K T c o n d i t i o n s f o r t h e m i n i m u m o f t h i s p r o b l e m g i v e t h e f o l l o w i n g s y s t e m o f e q u a t i o n s:
Q, d k - I- g * + A r v = 0 A d fc = 0
w h e r e v i s a v e c t o r o f L a g r a n g e m u l t i p l i e r s a s s o c i a t e d w i t h a c t i v e c o n s t r a i n t s. B o t h e q u a t i o n s c a n b e w r i t t e n i n a m a t r i x f o r m, a s f o l l o w s:
( » ΐ ) (?Η ΐ )
T h e d i r e c t i o n, t o g e t h e r w i t h t h e L a g r a n g e m u l t i p l i e r s, c a n b e c a l c u l a t e d b y s o l v i n g t h e a b o v e l i n e a r s y s t e m o f e q u a t i o n s.
8.4 A c t i v e S e t M e t h o d
8.4.2 S t e p - L e n g t h C a l c u l a t i o n s
A f t e r c o m p u t i n g t h e d i r e c t i o n, t h e s t e p l e n g t h a i s s e l e c t e d t o b e a s l a r g e a s p o s s i b l e t o m a i n t a i n f e a s i b i l i t y w i t h r e s p e c t t o i n a c t i v e i n e q u a l i t y c o n s t r a i n t s.
a f ( x.k + a d k ) < b i i £ w *
C l e a r l y, a f d k < 0 c a n n o t c a u s e a n y c o n s t r a i n t v i o l a t i o n; t h e r e f o r e, t h e s t e p l e n g t h i s s e l e c t e d a s f o l l o w s:
a a f d fc < b j — a f x * i £ w k
b i - a f x fc
a = M i n
1,
a.f d k
f o r a f d k > 0 a n d i £ w *
8.4.3 A d j u s t m e n t s t o t h e A c t i v e S e t
F r o m t h e s t e p - l e n g t h c a l c u l a t i o n s, i t i s c l e a r t h a t w h e n e v e r a < 1, a n e w i n e q u a l i t y c o n s t r a i n t b e c o m e s a c t i v e a n d m u s t b e i n c l u d e d i n t o t h e a c t i v e s e t f o r t h e n e x t i t e r a t i o n. F r o m C h a p t e r 4, w e k n o w t h a t L a g r a n g e m u l t i p l i e r s f o r a l l L E c o n s t r a i n t s m u s t b e p o s i t i v e. T h u s, i f a n e l e m e n t i n t h e v v e c t o r c o r r e s p o n d i n g t o a n i n e q u a l i t y c o n s t r a i n t i n t h e a c t i v e s e t i s n e g a t i v e, t h i s m e a n s t h a t t h e c o n s t r a i n t c a n n o t b e a c t i v e a n d m u s t b e d r o p p e d f r o m t h e a c t i v e s e t. I f t h e r e a r e s e v e r a l a c t i v e i n e q u a l i t i e s w i t h n e g a t i v e m u l t i p l i e r, u s u a l l y t h e c o n s t r a i n t w i t h t h e m o s t n e g a t i v e m u l t i p l i e r i s d r o p p e d f r o m t h e a c t i v e s e t.
B e c a u s e o f t h e a b o v e r u l e, a l l a c t i v e c o n s t r a i n t s a t a g i v e n p o i n t m a y n o t
s e t i s n o t a p p r o p r i a t e. F o r t h i s r e a s o n, s o m e a u t h o r s p r e f e r t o c a l l t h e s e t o f c o n s t r a i n t s u s e d f o r d i r e c t i o n f i n d i n g a w o r k i n g s e t. H o w e v e r, a c t i v e s e t t e r m i n o l o g y i s m o r e c o m m o n a n d i s u s e d h e r e.
8.4.4 F i n d i n g a S t a r t i n g F e a s i b l e S o l u t i o n
I n t h e p r e s e n t a t i o n s o f a r, i t h a s b e e n a s s u m e d t h a t w e h a v e a s t a r t i n g f e a s i b l e p o i n t x °. U s u a l l y, w e w i l l b e s t a r t i n g f r o m a n a r b i t r a r y p o i n t; t h e r e f o r e, w e n e e d t o f i n d a s t a r t i n g f e a s i b l e s o l u t i o n. W e c a n u s e t h e s t a n d a r d P h a s e I s i m p l e x
p r o c e d u r e t o f i n d a s t a r t i n g f e a s i b l e s o l u t i o n. B y i n t r o d u c i n g o n e a r t i f i c i a l
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
v a r i a b l e f o r e a c h c o n s t r a i n t, x n +\, Xn + i, · - ■, Xn+m, t h e P h a s e I p r o b l e m i s s t a t e d a s f o l l o w s:
M i n i m i z e φ = χ π + ι + X n +2 H---------
T h e s o l u t i o n o f t h i s p r o b l e m i s w h e n φ = 0, g i v i n g u s a p o i n t t h a t i s f e a s i b l e f o r t h e o r i g i n a l p r o b l e m.
T h i s P h a s e I L P c a n b e s o l v e d u s i n g a n y o f t h e m e t h o d s d i s c u s s e d i n t h e p r e v i o u s c h a p t e r s.
8.4.5 C o m p l e t e A c t i v e S e t Q P A l g o r i t h m
I n i t i a l i z a t i o n: S e t k = 0. I f t h e r e a r e e q u a l i t y c o n s t r a i n t s, s t a r t b y s e t t i n g x ° t o t h e s o l u t i o n o f t h e s e c o n s t r a i n t s; o t h e r w i s e, s e t x ° = 0. I f t h i s x ° s a t i s f i e s i n e q u a l i t y c o n s t r a i n t s a s w e l l, w e h a v e a s t a r t i n g f e a s i b l e s o l u t i o n. I f n o t, d e t e r m i n e t h e s t a r t i n g f e a s i b l e s o l u t i o n x ° b y u s i n g t h e P h a s e I p r o c e d u r e d i s c u s s e d a b o v e. S e t w ° t o t h e i n d i c e s o f t h e c o n s t r a i n t s a c t i v e a t x °.
1. F o r m:
3. I f d * = 0, g o t o s t e p 6 t o c h e c k f o r o p t i m a l i t y. O t h e r w i s e, c o n t i n u e.
4. C o m p u t e s t e p l e n g t h:
I f a < 1, a d d t h e c o n s t r a i n t c o n t r o l l i n g t h e s t e p l e n g t h t o t h e a c t i v e s e t.
5. U p d a t e:
g * - Q x * + c
2. S o l v e t h e f o l l o w i n g s y s t e m o f e q u a t i o n s:
x f c f l = x fc + a d k k — k + 1 a n d g o t o s t e p 1.
8.4 A c t i v e S e t M e t h o d
6. C h e c k t h e s i g n o f L a g r a n g e m u l t i p l i e r s c o r r e s p o n d i n g t o i n e q u a l i t y c o n ­
s t r a i n t s. I f t h e y a r e a l l p o s i t i v e, s t o p. W e h a v e t h e o p t i m u m. O t h e r w i s e, r e m o v e t h e c o n s t r a i n t t h a t c o r r e s p o n d s t o t h e m o s t n e g a t i v e m u l t i p l i e r a n d g o t o s t e p 2.
T h e A c t i v e S e t Q P F u n c t i o n
T h e f o l l o w i n g A c t i v e S e t Q P f u n c t i o n i m p l e m e n t s t h e A c t i v e S e t a l g o r i t h m f o r s o l v i n g Q P p r o b l e m s. T h e f i i n c t i o n u s a g e a n d i t s o p t i o n s a r e e x p l a i n e d f i r s t. S e v e r a l i n t e r m e d i a t e r e s u l t s c a n b e p r i n t e d t o g a i n u n d e r s t a n d i n g o f t h e p r o ­
c e s s.
N e e d s ["O p t i m i z a t i o n T o o l b o x'Q u a d r a t i c P r o g r a i n m i n g'"];
?A c t i v e S e t Q P
A c t i v e Se t Q P [ f,g, v a r s,o p t i o n s ]. s o l v e s a n QP p r o b l e m u s i n g Ac t i v e s e t a l g o r i t h m, f i s t h e o b j e c t i v e f u n c t i o n, g i s a l i s t o f c o n s t r a i n t s, a n d v a r s i s a l i s t o f v a r i a b l e s. S e e O p t i o n s [ A c t i v e S e t Q P ] t o f i n d o u t a b o u t a l i s t o f v a l i d o p t i o n s f o r t h i s f u n c t i o n.
o p c i o n s u s a g e [ A c t i v e S e t Q P ]
{ M a x l t e r a t i o n s -» 2 0, P r o b l e m T y p e -»Mi n,
M a x l t e r a t i o n s i s a n o p t i o n f o r s e v e r a l o p t i m i z a t i o n m e t h o d s. I t s p e c i f i e s maximum n u m b e r o f i t e r a t i o n s a l l o w e d.
P r o b l e m T y p e i s a n o p t i o n f o r m o s t o p t i m i z a t i o n m e t h o d s. I t c a n e i t h e r b e Mi n ( d e f a u l t ) o r Max.
P r i n t L e v e ] i s a n o p t i o n f o r m o s t f u n c t i o n s i n t h e O p t i m i z a t i o n T o o l b o x.
I t i s s p e c i f i e d a s a n i n t e g e r. T h e v a l u e o f t h e i n t e g e r i n d i c a t e s how muc h i n t e r m e d i a t e i n f o r m a t i o n i s t o b e p r i n t e d. A P r i n t L e v e l - » 0 s u p p r e s s e s a l l p r i n t i n g. D e f a u l t f o r m o s t f u n c t i o n s i s s e t t o 1 i n w h i c h c a s e t h e y p r i n t o n l y t h e i n i t i a l p r o b l e m s e t u p. H i g h e r i n t e g e r s p r i n t m o r e i n t e r m e d i a t e r e s u l t s.
C o n v e r g e n c e T o l e r a n c e i s a n o p t i o n f o r m o s t o p t i m i z a t i o n m e t h o d s. Mo s t m e t h o d s r e q u i r e o n l y a s i n g l e z e r o t o l e r a n c e v a l u e. Some i n t e r i o r p o i n t m e t h o d s r e q u i r e a l i s t o f c o n v e r g e n c e t o l e r a n c e v a l u e s.
E x a m p l e 8.9 S o l v e t h e f o l l o w i n g Q P p r o b l e m u s i n g t h e A c t i v e S e t Q P m e t h o d: M i n i m i z e / = — 2x\ +■ § ■ — 6x 2 — * 1*2 + *2
/ 3 * i + X 2 < 2 5 \
S u b j e c t * - V ^ s » 1 0 )
\ Xi > 0, i = 1, 2 /
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
f = - 2 x x + 1/2 * 1 - 6 x 2 “ x l x 2 + x 2 ;
g = { 3 x ± + x 2 5 2 5' _X1 + 2x2 s 1 0'* 1 + 2x2 £ 1 5, x i fc 0, x 2 ;» v a r s 3 t x l' x2 > <
{ s o l Λ x h i s t } = A c t i v e S e t Q P [ f, g, v a r s, P r i n t L e v e l -» 2 ] ;
2
Xl 2
Minimize - 2x^ + —— 6x3 - XiX2 + x2 «
' 3 x t + x 2 £ 2 5
- Χ χ + 2 x 2 i 1 0
L E C o n s t r a i n t s -»
X j + 2 x 2 < 1 5
- X i < 0
- x 2 £ 0
- 2
- 6
1 - 1
- 1 2
LE c o n s t r a i n t s: Α - »
'3 1'
'2 5
- 1 2
1 0
1 2
b - »
1 5
- 1 0
0
„ 0 - 1,
> 0,
EQ c o n s t r a i n t s: A - > { } b -* {}
C o n v e x p r o b l e m. P r i n c i p a l m i n o r s o f Q - » { 1, 1 }
* * * * I t e r a t i o n 1 * * * *
C u r r e n t p o i n t - » ( l l )
A c t i v e s e t - » { { } }
A -» {} g k - » | _ | J
L E C o n s t r a i n t v a l u e s - » ( - 2 1 - 9 - 1 2 - 1 - 1}
E q u a t i o n s: l h s - » I ^ j r h s - »
Sol ut i on: dk- »|^J uk-» { } vk-» { }
I n a c t i v e s e t - » ( l 2 3 4 5 )
b i - A i.x ( i e I n a c t i v e ) -» ( 2 1 9 1 2 1 l )
A i.d ( i e I n a c t i v e ) -» ( 3 4 5 2 3 - 9 - 7)
„ /2 1 9 1 2
b x - Α ι. x/A i. d -» — — — od 00
1 34 5 23
12 23
131 107
S t e p l e n g t h, a -» — New a c t i v e s e t - » (3) New p o i n t ■
23 23
8.4 A c t i v e S e t M e t h o d
**** I t e r a t i o n 2 ****
C u r r e n t p o i n t -» j A c t i v e s e t -» ( 3)
1 3 1 1 0 7 \
2 3 2 3 /
A -» ( l 2 ) g k -
22
TS 5 5
\~TS>
/ 7 5
L E C o n s t r a i n t v a l u e s - »
1 4 7
2 3
0
1 3 1
2 3
1 0 7 \ 2 3 ]
1 - 1 X\
2 2
TS
E q u a t i o n s: l h s -»
- 1 2 2 ι 1 2 oj
r h s -»
55
TS
, 0
11
'TT5
S o l u t i o n: d k
u k -
f f i -
v k - * O
I n a c t i v e s e t - » ( l 2 4 5)
b i - A i .x ( i e I n a c t i v e ) -»
75 147 131 1 0 7\ 23 /
23 23 23
11 22 11
46 115 115 735 655
22 11
2 3 0 1
A i.c t ( i e I n a c t i v e ) b i - A i .x/A i. d -» 00
S t e p l e n g t h, a - » l New a c t i v e s e t - » ( 3)
128 4 7'
N e w p o i n t -» f — —
---------------------------- \ 5------- 1 0 1
* * * * I t e r a t i o n 3 * * * *
( 2 8 4 7
C u r r e n t p o i n t -» — —
I 5 1 0
A c t i v e s e t -» ( 3}
A -» ( l 2) g k ■
1 1\
’ Ι ϋ
11
L E C o n s t r a i n t v a l u e s
7
12"
3 1 „ 2 8
V T 0 - T
47
10
' 1 - 1 1 ] E q u a t i o n s: l h s - » - 1 2 2
r h s
t t
11
1
2
S o l u t i o n: d k
(0) (H)
v k - » { }
* * * * * Optimum s o l u t i o n a f t e r 3 i t e r a t i o n s * * * * *
I 2 8 4 7
S o l u t i o n ··» x -l —> - — X2 ~>
10
L a g r a n g e m u l t i p l i e r s f o r EQ c o n s t r a i n t s (v) - » { { } }
, . . * · 559 O b j e c t i v e f u n c t i o n
L a g r a n g e m u l t i p l i e r s f o r LE c o n s t r a i n t s ( u) -» | θ 0 ^ 0 oj
T a b l e F o r m [ x h i s t ]
1 1
1 3 1
Ί Γ Γ
1 0 7
T 3 ~
2 8
"5"
4 7
TO
T h e g r a p h i n F i g u r e 8.4 s h o w s t h e s e a r c h p a t h t a k e n b y t h e a l g o r i t h m:
0
6
8
10
F I G U R E 8.4 A g r a p h i c a l s o l u t i o n s h o w i n g t h e s e a r c h p a t h.
E x a m p l e 8.1 0 S o l v e t h e f o l l o w i n g Q P p r o b l e m u s i n g t h e A c t i v e S e t Q P m e t h o d.
2 2
£ = - 6x ^ 2 + 2 x i + 9 x 2 - 1 8 x i + 9 x 2; g = { X! + 2 x 2 ί 1 5, x i i 0, x 2 ί 0 }; v a r s = { x i, x 2 }/
{ s o l, x h i s t } = A c t i v e S e t Q P [ f, g, v a r s, P r i n t L e v e l -» 2 ];
8.4 A c t i v e S e t M e t h o d
M i n i m i z e -* -Ιβχ^^ + 2 χ χ + 9x 2 - 6x ^ 2 + 9 x § Xl +■ 2x2 £
15
-Xl < 0
- x 2 s 0
LE C o n s t r a i n t s
-18
9
4 - 6
- 6 18
r 1 2 1
15'
LE c o n s t r a i n t s: Α-»
-1 0
b -»
0
0
1
, 0 ,
EQ c o n s t r a i n t s: A - » { } b - » { }
Co nve x p r o b l e m, p r i n c i p a l m i n o r s o f Q - » { 4,3 6 }
**** i t e r a t i o n 1 ****
C u r r e n t p o i n t -* ( l l )
A c t i v e s e t - » { { } }
* - < > 9 l t - ( i f l )
LE C o n s t r a i n t v a l u e s - » (-12 -1 - 1 )
E q u a t i o n s: lhs
S o l u t i o n: d k
4 - 6
- 6 18
r h s
20
-21
Y I uk - {} vk -» {}
I n a c t i v e s e t -» ( l 2 3)
b i - A i.x ( i e I n a c t i v e ) -» ( l 2 1 l )
17 13
2 2
A i.d ( i e I n a c t i v e )
- 1
I 24
b i - A i.x/A i.d -» — oo oo
117
^St e p l e n g t h, gt -> 1 New a c t i v e s e t - » {{}}
New point
15
**** I t e r a t i o n 2 ****
Current point -* 2
Act i ve s e t - » { { } } a -» { } gk-> |°J
LE Const r ai nt val ues -
„ / 4 - 6
E quat i o n s: l hs -> _g lg
7
2
1 5
- T - 2
r h s
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
S o l u t i o n: d k -» | ^ | u k - » { } v k - » { }
* **** o p t i m u m s o l u t i o n a f t e r 2 i t e r a t i o n s *****
I 15 J
S o l u t i o n -» I x j -» — x 2 -» 2 j
L a g r a n g e m u l t i p l i e r s f o r L E c o n s t r a i n t s ( u ) -» ( Ο 0 θ ) L a g r a n g e m u l t i p l i e r s f o r EQ c o n s t r a i n t s ( v ) - » { { } }
1 1 7
O b j e c t i v e f u n c t i o n - » — —
T h e g r a p h i n F i g u r e 8.5 c o n f i r m s t h e s o l u t i o n:
*2
F I G U R E 8.5 A g r a p h i c a l s o l u t i o n s h o w i n g t h e s e a r c h p a t h.
E x a m p l e 8.1 1 S o l v e t h e f o l l o w i n g Q P p r o b l e m u s i n g t h e A c t i v e S e t Q P m e t h o d:
8.4 A c t i v e S e t M e t h o d
I n t e r m e d i a t e r e s u l t s a r e p r i n t e d f o r t h e f i r s t i t e r a t i o n:
{ s o l, x h i s t ) = A c t i v e S e t Q P [ £, g, v a r s, P r i n t L e v e l -* 2] ;
Minimize -» 4x 3^ + χχ + 6x2 + ΧχΧ2 + 2x2 + 12x3 + 2x 2x 3 + 2x§
LE C o n s t r a i n t s
-xj. - x2 - x 3 s - 6
X1 + x 2 - 2 x 3 5 “ 2
------------- Xi < 0
1
- x 3 < 0
' 4
2
1
O'
c ->
6
Q -»
1
4
2
,12,
0
2
- 1
- 1
- 1'
- 6'
1
1.
- 2
b -»
- 2
- 1
0
0
0
> 0
0
- l j
, 0
LE c o n s t r a i n t s: A
EQ c o n s t r a i n t s: Α-» {} b - » { }
Co nve x p r o b l e m, p r i n c i p a l m i n o r s o f Q -» {2, 7, 20}
P h a s e I p r o b l e m: A r t. O b j. c o e f f i c i e n t s -» {0, 0, 0, 0, 0, 0, 1, 1, 1, 1}
C o n s t r a i n t s e x p r e s s e d a s GE t y p e
C o n s t r a i n t c o e f f i c i e n t s
C o n s t r a i n t r h s -» (6 2 0 o)
Γ10 8 \
P h a s e I - L P s o l u t i o n | — , 0, ^ ·, 0, 0, 0, 0, 0 ^ 0, 0 ^
1 1
1
1
- 1
- 1
- 1
1
0
0
0)
- 1
- 1
2
1
1
- 2
0
1
0
0
1
0
0
- 1
0
0
0
0
1
0
, 0
0
1
0
0
- 1
0
0
0
1,
* * * * i t e r a t i o n 1 * * * *
110 „
C u r r e n t p o i n t - * I — 0 A c t i v e s e t -» ( l' 2)
-1 -1 -1 1 1 - 2
g k
3 2
T
4 4
T
68
T
L E C o n s t r a i n t v a l u e s - » 0 0
E q u a t i o n s: l h s
10
3
2
1
0
- 1
1
3 2
"T
1
4
2
- 1
1
4 4
"T
0
2
4
- 1
- 2
r h s -*
68
-1
-1
-1
0
0
"T
1
1
-2
0
0
0
0
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
S o l u t i o n: d k
' 1 '
-1 .0)
I n a c t i v e s e t - » (3 4)
u k
44 <
"I” I v k - » {}
. , /1 ° 8\
b i - A i.x ( i e I n a c t i v e ) - » j — — I
A i .d ( i e I n a c t i v e ) - » ( - 1 θ )
b i - A i .x/A i .d -» (oo co)
S t e p l e n g t h, a - » l New a c t i v e s e t - » ( l 2}
1 13 “ 8\
New p o i n t -» I — - 1 — J
**** I t e r a t i o n 2 ****
I !3 ,8
C u r r e n t p o i n t - » I — - 1 —
A c t i v e s e t - » ( l 2 )
- 1
1
g k -
3 5 ^ "3" 3 5 T 6 2 ~T
L E C o n s t r a i n t v a l u e s - » 0 0
1 3 8\
3 " 3/
E q u a t i o n s: l h s ■
< 2 1 0 - 1 1
1 4 2 - 1 1
0 2 4 - 1 - 2
- 1 - 1 - 1 0 0
1 1 - 2 0 0
r h s
3 5
"3"
3 5
"T
- ¥
0
0
S o l u t i o n: d k ·
u k
v k - » { }
* * * * * O p t i m u m s o l u t i o n a f t e r 2 i t e r a t i o n s
1 3 . 8 1
S o l u t i o n ■
- * Xi—*
χ·,—*—-1 x.
L a g r a n g e m u l t i p l i e r s f o r L E c o n s t r a i n t s ( u ) -» 3 0 θ |
L a g r a n g e m u l t i p l i e r s f o r EQ c o n s t r a i n t s ( v ) - » { { } }
. . 2 0 6 O b j e c t i v e f u n c t i o n - »
-------
E x a m p l e 8.1 2 P o r t f o l i o m a n a g e m e n t F i n d t h e o p t i m u m s o l u t i o n o f t h e p o r t ­
f o l i o m a n a g e m e n t p r o b l e m, c o n s i d e r e d i n E x a m p l e 8.8 a n d f i r s t p r e s e n t e d i n C h a p t e r 1, u s i n g t h e A c t i v e S e t Q P m e t h o d.
8.4 A c t i v e S e t M e t h o d
T h e o b j e c t i v e a n d t h e c o n s t r a i n t f u n c t i o n s f o r t h e p r o b l e m a r e g e n e r a t e d u s i n g M a t h e m a t i c a s t a t e m e n t s ( s e e C h a p t e r 1 f o r d e t a i l s ).
b l u e C h i p s t o c k s = { 1 8.2 4, 1 2.1 2. 1 5.2 3 r 5.2 6, 2.6 2, 1 0.4 2!;
t e c h S t o c k s = { 1 2.2 4, 1 9.1 6, 3 5.0 7, 2 3.4 6, - 1 0.6 2, - 7.4 3 };
r e a l E s t a t e = { 8.2 3, 8.9 6, 8.3 5, 9.1 6, 8.0 5, 7.2 9 };
b o n d s = { 8.1 2, 8.2 6, 8.3 4, 9.0 1, 9.1 1, 8.9 5 };
r e t u r n s = { b l u e C h i p s t o c k s, t e c h S t o c k s,r e a l E s t a t e,b o n d s };
a v e r a g e R e t u m s = M a p [ A p p l y [ P l u s, #] /L e n g t h [#] r e t u r n s ] ;
c o v a r i a n c e [ x _, y,] : = M o d u l e [ { x b, y b, n = L e n g t h [ x ] },
x b = A p p l y [ P l u s, x ] /n; y b = A p p l y [ P l u s, y ] /n;
A p p l y [ P l u s, ( x - x b ) ( y - y b ) ] /n ] ; q = O u t e r [ c o v a r i a n c e, r e t u r n s, r e t u r n s, 1 ] ;
C l e a r [ x ];
v a r s = T a b l e [ x i r { i, 1, L e n g t h [ a v e r a g e R e t u m s ] } ] ; f =■ E x p a n d [ v a r s. Q. v a r s ] ; g = F l a t t e n [ { A p p l y [ P l u s, v a r s ] = » 1,
a v e r a g e R e t u m s .v a r s a 1 0, T h r e a d [ v a r s £ 0 ] } ];
{ s o l, h i s t } = A c t i v e S e t Q P [ £, g, v a r s, P r i n t L e v e l -» 2 ];
M i n i m i z e -> 2 9 . 0 5 5 2 x ± + 8 0.7818X3X 2 + 2 6 7.3 4 4 x | - 0.5 7 5 7 6 7 x 1 x 3 + 1 3.6 6 7 3 x 2 x 3 + 0.3 7 5 9 3 3 X 3 - 3.9 0 6 3 9 Χ ] Χ 4 - 7.3 9 4 0 3 x 2 x 4 - 0.1 1 3 2 6 7 x 3 x 4 + 0.1 5 9 7 1 4 X 4 ■ 1 0.6 4 8 3 X! - 1 1.9 8 x 2 - 8.3 4 x 3 - 8.6 3 1 6 7 x 4 s - 1 0
- l.X i < 0
- l.x 2 < 0
- 1. x 3 <, 0
- l.x 4 < 0
L E C o n s t r a i n t s
EQ C o n s t r a i n t s -» ( x ^ + x 2 + x 3 + x 4 = = l )
r o i
5 8.1 1 0 4
B O.7 8 1 8
- 0.5 7 5 7 6 7
- 3 .9 0 6 3 9
c ->
0
Q -»
8 0.7 8 1 8
5 3 4.6 8 9
1 3.6 6 7 3
- 7.3 9 4 0 3
0
- 0.5 7 5 7 6 7
1 3.6 6 7 3
0.7 5 1 8 6 7
- 0.1 1 3 2 6 7
\.0.
,- 3.9 0 6 3 9
- 7.3 9 4 0 3
- 0.1 1 3 2 6 7
0.3 1 9 4 2 8 j
L E c o n s t r a i n t s : A -»
- 1 0.6 4 8 3 - 1 1.9 8 - 8.3 4 - 8.6 3 1 6 7 ^ - 1 0 0 0 0 - 1 0 0
b - ►
- 1 0 ’
0
0
0 0 - 1 0 0 0 0 - 1
o o
E Q c o n s t r a i n t s: A ■ C o n v e x p r o b l e m.
( l l l l ) b - > ( l )
P r i n c i p a l m i n o r s o f Q - » { 5 8.1 1 0 4, 2 4 5 4 5.3, 6 1 5 1.3 7, 1 1 7.2 5 3 }
* * * * I t e r a t i o n 1 * * * *
C u r r e n t p o i n t - > ( l 0 0 θ )
A c t i v e s e t - » ( 3 4 5 )
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
1
1
1
1 '
' 5 8.1 1 0 4 '
0
- 1
0
0
g k -*
8 0.7 8 1 8
0
0
- 1
0
- 0.5 7 5 7 6 7
,0
0
0
- 1,
,- 3.9 0 6 3 9,
LE C o n s t r a i n t v a l u e s EQ C o n s t r a i n t v a l u e s
( - 0.6 4 8 3 3 3
(o)
1. 0. 0.
0.)
E q u a t i o n s:
l h s
.1 1 0 4
8 0.7 8 1 8
- 0.5 7 5 7 6 7
- 3.9 0 6 3 9
1
0
0
0
.7 8 1 8
5 3 4.6 8 9
1 3.6 6 7 3
- 7.3 9 4 0 3
1
- 1
0
0
5 7 5 7 6 7
1 3.6 6 7 3
0.7 5 1 8 6 7
- 0.1 1 3 2 6 7
1
0
- 1
0
.9 0 6 3 9
- 7.3 9 4 0 3
- 0.1 1 3 2 6 7
0.3 1 9 4 2 8
1
0
0
- 1
1
1
1
1
0
0
0
0
0
- 1
0
0
0
0
0
0
0
0
r l.
0
0
0
0
0
0
0
0
- 1
0
0
0
o
r h s
- 5 8.1 1 0 4
- 8 0.7 8 1 8
0.5 7 5 7 6 7
3.9 0 6 3 9
0
0
0
0
S o l u t i o n: d k -
- 3.0 5 5 8 3 x l 0 “1€ - 5.7 5 3 4 x 1 Q “17
2 2.6 7 1 3 ’
u k -»
- 5 8.6 8 6 2
v k -»
i
- 6 2.0 1 6 8,
**** i t e r a t i o n 2 ****
C u r r e n t p o i n t - * ( l 0 0 θ)
A c t i v e s e t - * (3 4)
5 8.1 1 0 4
8 0.7 8 1 8
- 0.5 7 5 7 6 7
- 3.9 0 6 3 9
1 1
0 -1 β 0
1
0
- 1
1)
0
0
g k
L E C o n s t r a i n t v a l u e s -» ( - 0
.6 4 8 3 3 3 - 1.
O
o
o
)
EQ C o n s t r a i n t v a l u e s - » ( θ )
E q u a t i o n s:
' 5 8.1 1 0 4
8 0.7 8 1 8
- 0.5 7 5 7 6 7
- 3.9 0 6 3 9
1
0
0
8 0.7 8 1 8
5 3 4.6 8 9
1 3.6 6 7 3
- 7.3 9 4 0 3
1
- 1
0
- 0.5 7 5 7 6 7
1 3.6 6 7 3
0.7 5 1 8 6 7
- 0.1 1 3 2 6 7
1
0
- 1
l h s -»
- 3.9 0 6 3 9
- 7.3 9 4 0 3
- 0.1 1 3 2 6 7
0.3 1 9 4 2 8
1
0
0
1
1
1
1
0
0
0
o
0
o
o
o
o
o.
0
- 1
0
0
0
0,
- 5 8.1 1 0 4 - 8 0.7 8 1 8 0.5 7 5 7 6 7 r h s -» 3.9 0 6 3 9
0 0 Ο
S o l u t i o n: d k ·
- 0.9 3 6 2 0 7 4.8 1 6 9 7 x l O -18 - 7.9 4 9 9 2 χ I O-17 0.9 3 6 2 0 7
u k -
( - 1.8 1 8 8 8 \ ι ,
( - 0.1 9 2 6 2 1/ v k. ( - 0.0 4 9 8 4 9 6 )
I n a c t i v e s e t - » ( l 2 5)
b i - A i.x ( i e I n a c t i v e ) - » ( 0.6 4 8 3 3 3 1. 0.)
A i.d ( i € I n a c t i v e ) -» ( l. 8 8802 0.9 3 6 2 0 7 - 0.9 3 6 2 0 7 )
b i - A i.x/A i.d - » ( θ.3 4 3 3 9 4 1.0 6 8 1 4 oo)
S t e p l e n g t h, a - » 0.3 4 3 3 9 4 New a c t i v e s e t - » ( l 3 4)
New p o i n t -» ( θ.6 7 8 5 1 2 1.6 5 4 1 2 χ 1 0"18 - 2 .7 2 9 9 5 x I O -17 0.3 2 1 4 8 8 )
**** I t e r a t i o n 3 ****
C u r r e n t p o i n t - » ( θ. 6 7 8 5 1 2 1. 6 5 412 x 1 0 “18 - 2.7 2 9 9 5 χ 1 0 -17 0.3 2 1 4 8 8 )
A c t i v e s e t -» ( l 3 4)
1
1
1
1
3 8.1 7 2 8
- 1 0.6 4 8 3
- 1 1.9 8
- 8.3 4
- 8.6 3 1 6 7
g k - »
5 2.4 3 4 3
0
- 1
0
0
- 0.4 2 7 0 7 9
o
0
- 1
0
- 2.5 4 7 8 4
LE C o n s t r a i n t v a l u e s -»
( 0. - 0.6 7 8 5 1 2 - 1.6 5 4 1 2 χ 1 0'18
EQ C o n s t r a i n t v a l u e s - » ( θ.) E q u a t i o n s:
2.7 2 9 9 5 x l O -17 - 0.3 2 1 4 8 8 )
l h s
< 5 8.1 1 0 4 8 0.7 8 1 8 - 0.5 7 5 7 6 7 - 3 .9 0 6 3 9 1
8 0.7 8 1 8 5 3 4.6 8 9 1 3.6 6 7 3 - 7.3 9 4 0 3
1_____
- 0.5 7 5 7 6 7 1 3.6 6 7 3 0.7 5 1 8 6 7 - 0.1 1 3 2 6 7 1
- 3.9 0 6 3 9 - 7.3 9 4 0 3 - 0.1 1 3 2 6 7 0.3 1 9 4 2 8 1
- 1 0.6 4 8 3
- 1 1.9 8
- 8.3 4
- 8.6 3 1 6 7
0
0
-1
0
0
0
0
0
-1
0
0
r h s
- 1 0.6 4 8 3
0
0
- 3 8.1 7 2 8\ - 5 2.4 3 4 3 0.4 2 7 0 7 2.5 4 7 8 4 0 0 0 0
- 1 1.9 8
- 1
0
- 8.3 4
0
- 1
- 8.6 3 1 6 7
0
0
0
0
0
0
0
0
0
0
0
0
0
0
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
S o l u t i o n: d k ·
3.5 5 4 3 6 χ I O-15 - 1.0 9 9 6 7 x l 0 “17 - 1.8 5 4 2 5 x 10
-16
u k ·
,- 4.5 8 8 1 7 x I O"15,
**** I t e r a t i o n 4 ****
C u r r e n t p o i n t - » ( 0.6 7 8 5 1 2 1.6 5 4 1 2 x 1 0"* ®
A c t i v e s e t - » ( l 4)
1 1 1 1
2 0.1 9 2 ' - 1 2.6 2 7 5 8.01011 ,
v k - » ( 1 7 6.8 3 9 )
2.7 2 9 9 5 x l O -17 0.3 2 1 4 8 8 )
1
- 1 0.6 4 83
1
- 1 1.9 8
1
■8.34
- 8.6 3167
g k -
3 8.1 7 2 8
5 2.4 3 4 3
0 0 - 1
LE C o n s t r a i n t v a l u e s -»
( O. - 0.6 7 8 5 1 2
EQ C o n s t r a i n t v a l u e s - » ( θ.) E q u a t i o n s:
l h s
- 0.4 2 7 0 7 9
- 2.5 4 7 8 4
- 1.6 5 4 1 2 x 1 0
■18
2.7 2 9 9 5 x l 0 ~ 17
- 0.3 2 1 4 8 8
f 5 8.1 1 0 4
8 0.7 8 1 8
- 0.5 7 5 7 6 7
- 3.9 0 6 3 9
1
- 1 0.6 4 8 3
0
8 0.7 8 1 8
5 3 4.6 8 9
1 3.6 6 7 3
- 7.3 9 4 0 3
1
- 1 1.9 8
0
- 0.5 7 5 7 6 7
1 3.6 6 7 3
0.7 5 1 8 6 7
- 0.1 1 3 2 6 7
1
- 8.3 4
- 1
- 3.9 0 6 3 9
- 7.3 9 4 0 3
- 0.1 1 3 2 6 7
0.3 X 9 4 2 8
1
- 8.6 3 1 6 7
0
1
1
1
1
0
0
0
- 1 0.6 4 8 3
- 1 1.9 8
- 8.3 4
- 8.6 3 1 6 7
0
0
0
0
0
- 1
0
0
0
0 ,
r h s
- 3 8.1 7 2 8
- 5 2.4 3 4 3
0.4 2 7 0 7
2.5 4 7 8 4
0
0
0
S o l u t i o n: d k -
- 0.0 4 9 2 6 5 1 0.0 2 9 6 7 1 8 . 0 1 4 2 9 x l O - 1 7 0.0 1 9 5 9 3 2
u k -»
1 9.9 3 3 3 ]
8.3 8 7 0 3/
v k - » ( 1 7 4.6 2 7 )
I n a c t i v e s e t - » (2 3 5 )
b i - A i.x ( i e I n a c t i v e ) -» ( o.6 7 8 5 1 2 1.6 5 4 1 2 χ 1 0 - 1 8 0.3 2 1 4 8 8 )
A i.d ( i e I n a c t i v e ) -» ( o.0 4 9 2 6 5 1 - 0.0 2 9 6 7 1 8 - 0.0 1 9 5 9 3 2 )
b i - A i.x/A i.d. ( 1 3.7 7 2 7 00 00)
S t e p l e n g t h, a - » l N e w a c t i v e s e t - » ( l 4 )
N e w p o i n t - » ( 0.6 2 9 2 4 7 0.0 2 9 6 7 1 8 1.2 8 4 3 4 x I O'17 0.3 4 1 0 8 1 )
· * * * i t e r a t i o n 5 * * * *
C u r r e n t p o i n t. ( 0.6 2 9 2 4 7 0.0 2 9 6 7 1 8 1.2 8 4 3 4 χ 1 0"17 0.3 4 1 0 8 1 )
A c t i v e s e t - » ( l 4 )
8.4 A c t i v e S e t M e t h o d
A.
1
-10.64B3
1 1 -11.98 -8.34
1
-8.63167
gk .
' 37.6304 1 64.1749 0.00460216
o
0 - 1
o
, - 2.56853 ,
L E C o n s t r a i n t v a l u e s .
( 1.77636 χ 10-15 - 0.629247 - 0.0 2 9 6 7 1 B - 1. 2B434 x IO-17 - 0.3 4 1 0 Bl )
EQ C o n s t r a i n t v a l u e s - » ( - 9.9 9 2 0 1 χ 10- 16)
E q u a t i o n s:
58.1104
80.7818
80.7818
534.689
- 0.575767
13.6673
- 3.9 0 6 3 9
- 7.39403
1
1
- 10.64B3 - 1 1.9B
0 ' 0
- 0.5 7 5 7 6 7
13.6673
0.751867
- 0.113267
1
- 8.3 4
- 1
l hs .
- 3.9 0 6 3 9
- 7.39403
- 0.113267
0.31942B
1
- 8.63167
0
1
1
1
1
0
0
0
- 10.6483
- 1 1.9 8 ·
- B.34
- B.63167
0
0
0
o
0
- 1
0
0
0
0 ,
f - 3 7.6 3 0 4 ' 64 1749
r h s .
- 0.00460216 2.56853 0 0
0 j
-
2.5 4 5 0 8 χ IO"15
Sol ut i on: d k.
* * * * * O p t i m u m £
Soluti on ( x x .0.6 2 9 2 4 7
1. 0 7 1 1 4 xl O"16 - 3.5 4 9 1 4 X 1 0"16 ,- 3.5 4 3 2 6 χ IO-15;
i dut i o n a f t e r 5
x 2 . 0.0 2 9 6 7 1 B
’* - * ( ” « 7 03) I 1 7 4'6 2 7 )
i t e r a t i o n s * * * ** x 3 .1.2 8 4 3 4 xl O-17 x 4 . 0.3 4 1 0 8 1 )
Lagrange mul t i pl i e r s f or LE c ons t r a i nt s (u) . ( 19.9333 0 0 B.3B703 θ) Lagrange mul t i pl i e r s f or EQ c o ns t r a i nt s (v) -» ( 174.627)
Obj ect i ve f unct i on . 12.3535
T h e c o m p l e t e h i s t o r y o f t h e a c t u a l f o u r v a r i a b l e s d u r i n g i t e r a t i o n i s a s f o l l o w s:
T a b l e F o r m [ h i s t ]
1 0 0 0
1 0 0 0
0.6 7 B 5 1 2 1.6 5 4 1 2 χ 1 0 - 1 8 - 2.7 2 9 9 5 χ 1 0 - 1 7 0.3 2 1 4 8 8
0.6 7 B 5 1 2 1.6 5 4 1 2 x 1 0 ~ 1 8 - 2.7 2 9 9 5 x I O'1 70.3 2 1 4 8 8 0.6 2 9 2 4 7 0.0 2 9 6 7 1 8 1.2 B 4 3 4 χ 1 0 ~ 17 0.3 4 1 0 8 1
T h e o p t i m u m s o l u t i o n i s t h e s a m e a s t h a t o b t a i n e d i n E x a m p l e 8.8.
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
8.5 A c t i v e S e t M e t h o d f o r t h e D u a l Q P P r o b l e m
I n t h i s s e c t i o n, w e c o n s i d e r a s p e c i a l f o r m o f Q P p r o b l e m t h a t a r i s e s b y t a k ­
i n g t h e d u a l o f a p r o b l e m i n w h i c h Q. i s i n v e r t i b l e. T h e p r o b l e m i n v o l v e s a q u a d r a t i c o b j e c t i v e f u n c t i o n a n d s i m p l e n o n n e g a t i v i t y c o n s t r a i n t s o n p r o b ­
l e m v a r i a b l e s. U s i n g t h e s t a n d a r d n o t a t i o n f o r v a r i a b l e s, t h e p r o b l e m c a n b e e x p r e s s e d i n t h e f o l l o w i n g f o r m:
M i n i m i z e / ( x ) = c T x + | x r Q x
S u b j e c t t o Xi > 0, i = 1,..., m < n
w h e r e x i s a n η x 1 v e c t o r o f o p t i m i z a t i o n v a r i a b l e s, c i s a n η χ 1 v e c t o r c o n t a i n i n g c o e f f i c i e n t s o f l i n e a r t e r m s, a n d n x n s y m m e t r i c m a t r i x Q. c o n ­
t a i n s c o e f f i c i e n t s o f s q u a r e a n d m i x e d t e r m s. T h e f i r s t m < « v a r i a b l e s a r e c o n s t r a i n e d t o b e n o n n e g a t i v e.
T h i s p r o b l e m c a n o b v i o u s l y b e s o l v e d b y u s i n g a n y o f t h e Q P a l g o r i t h m s d i s c u s s e d s o f a r. H o w e v e r, b e c a u s e o f t h e s i m p l e n a t u r e o f c o n s t r a i n t s, a m o r e e f f i c i e n t m e t h o d c a n b e d e v e l o p e d b y a s i m p l e e x t e n s i o n o f a n y o f t h e m e t h o d s f o r u n c o n s t r a i n e d m i n i m i z a t i o n. I n t h i s s e c t i o n, t h e c o n j u g a t e g r a d i e n t m e t h o d i s e x t e n d e d t o s o l v e t h i s s p e c i a l Q.P p r o b l e m.
8.5.1 O p t i m a l i t y C o n d i t i o n s
B y i n t r o d u c i n g a v e c t o r o f s l a c k v a r i a b l e s, s, a n d t h e L a g r a n g e m u l t i p l i e r s,
ui > 0, i = 1..., m a n d u % — 0, i = m + 1..., n, t h e L a g r a n g i a n f o r t h e p r o b l e m
i s a s f o l l o w s:
I ( x, u, s ) - / ( x ) + u T ( - x.+ s 2 )
T h e n e c e s s a i y o p t i m a l i t y c o n d i t i o n s a r e:
V f ( x ) — u = 0 Ui > 0, i = 1..., m, a n d «,· = 0, i = m + 1..., n Uj Sj = 0, i = 1,..., m ( s w i t c h i n g c o n d i t i o n s )
F r o m t h e s w i t c h i n g c o n d i t i o n s, e i t h e r «,· = 0 o r s* = 0. I f «,· = 0, t h e n t h e g r a d i e n t c o n d i t i o n s a y s t h a t t h e c o r r e s p o n d i n g p a r t i a l d e r i v a t i v e d f/d x i m u s t b e z e r o. I f s, ~ 0, t h e n t h e r e i s n o s l a c k a n d t h e r e f o r e, jc,· = 0. F u r t h e r m o r e,
s i n c e n o w «,· φ 0, t h e g r a d i e n t c o n d i t i o n s a y s t h a t t h e p a r t i a l d e r i v a t i v e d f/d x i m u s t b e g r e a t e r t h a n o r e q u a l t o z e r o. T h u s, t h e o p t i m a l i t y c o n d i t i o n s f o r t h e p r o b l e m c a n b e w r i t t e n e n t i r e l y i n t e r m s o f o p t i m i z a t i o n v a r i a b l e s Xj, w i t h o u t e x p l i c i t l y i n v o l v i n g t h e L a g r a n g e m u l t i p l i e r s, a s f o l l o w s:
/d f/d x i = 0 i f X{ > 0\ f .
[ d f/d x i > O i f x i = ° J ° Γ 1 —
d f/d x i = 0 f o r i = m + 1,..., n
8.5.2 C o n j u g a t e G r a d i e n t A l g o r i t h m f o r Q u a d r a t i c F u n c t i o n w i t h N o n n e g a t i v i t y C o n s t r a i n t s
A c o n j u g a t e g r a d i e n t a l g o r i t h m f o r m i n i m i z i n g a n u n c o n s t r a i n e d f u n c t i o n w a s p r e s e n t e d i n C h a p t e r 5. T W o s i m p l e e x t e n s i o n s a r e i n t r o d u c e d i n t o t h i s a l g o ­
r i t h m t o m a k e i t s u i t a b l e f o r t h e p r e s e n t s i t u a t i o n. F i r s t, a n a c t i v e s e t i d e a i s i n ­
t r o d u c e d. A c t i v e v a r i a b l e s a r e d e f i n e d a s t h o s e v a r i a b l e s t h a t a r e e i t h e r g r e a t e r t h a n z e r o o r t h a t v i o l a t e t h e o p t i m a l i t y c o n d i t i o n. T h e r e m a i n i n g v a r i a b l e s a r e c a l l e d p a s s i v e v a r i a b l e s. B y s e t t i n g p a s s i v e v a r i a b l e s t o z e r o, t h e o b j e c t i v e f u n c t i o n i s e x p r e s s e d i n t e r m s o f a c t i v e v a r i a b l e s o n l y. T h e s t a n d a r d c o n j u g a t e g r a d i e n t a l g o r i t h m f o r u n c o n s t r a i n e d p r o b l e m s i s u s e d t o f i n d d i r e c t i o n w i t h r e s p e c t t o a c t i v e v a r i a b l e s. T h e s e c o n d m o d i f i c a t i o n i s i n t h e s t e p - l e n g t h c a l c u ­
l a t i o n s. T h e s t e p l e n g t h g i v e n b y t h e s t a n d a r d c o n j u g a t e g r a d i e n t a l g o r i t h m f o r a q u a d r a t i c f u n c t i o n i s f i r s t c o m p u t e d. I f t h i s s t e p l e n g t h d o e s n o t m a k e a n y o f t h e r e s t r i c t e d v a r i a b l e s t a k e o n a n e g a t i v e v a l u e, t h e n i t i s a c c e p t e d a n d w e p r o c e e d t o t h e n e x t i t e r a t i o n. I f t h e s t a n d a r d s t e p l e n g t h i s t o o l a r g e, a s m a l l e r v a l u e i s c o m p u t e d t h a t m a k e s o n e o f t h e a c t i v e v a r i a b l e s t a k e a z e r o v a l u e. T h i s v a r i a b l e i s r e m o v e d f r o m t h e a c t i v e s e t f o r t h e s u b s e q u e n t i t e r a t i o n, a n d t h e p r o c e s s i s r e p e a t e d u n t i l t h e o p t i m a l i t y c o n d i t i o n s a r e s a t i s f i e d. T h e c o m p l e t e a l g o r i t h m i s a s f o l l o w s:
I n i t i a l i z a t i o n: S e t k = 0. C h o o s e a n a r b i t r a r y s t a r t i n g p o i n t x °.
A. T f e s t f o r o p t i m a l i t y: I f d f/d x i = 0 f o r x% > 0 a n d 3f/d x i > 0 f o r x f = 0 f o r i = 1,..., m a n d d f/d x i = 0 f o r i = m + 1,..., n, t h e n s t o p. W e h a v e a n o p t i m u m. O t h e r w i s e, c o n t i n u e.
B. D e f i n e t h e s e t o f a c t i v e v a r i a b l e s c o n s i s t i n g o f t h o s e i n d i c e s f o r w h i c h e i t h e r X i > 0 o r X i — O b u t c o r r e s p o n d i n g 3//3 x,· < 0. T h e r e m a i n i n g v a r i a b l e s a r e p a s s i v e v a r i a b l e s. A l l u n r e s t r i c t e d v a r i a b l e s a r e c o n s i d e r e d a c t i v e. T h u s,
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
τ α = !*!( *? > 0) a r ( x f = 0 a n d d f/3 x f < 0) o r ( i > m ) } l kp = {*f i £ l j }
C. S e t p a s s i v e v a r i a b l e s t o z e r o a n d u s e t h e c o n j u g a t e g r a d i e n t a l g o r i t h m t o m i n i m i z e / ( x ) w i t h r e s p e c t t o a c t i v e v a r i a b l e s,
1. S e t t h e i t e r a t i o n c o u n t e r i = 0.
2. C o m p u t e V/( x *) w i t h r e s p e c t t o a c t i v e v a r i a b l e s.
5. I f | | d,+ 1 | | < t o l, w e h a v e f o u n d t h e m i n i m u m o f /( x ) w i t h r e s p e c t t o a c t i v e v a r i a b l e s. G o t o s t e p ( A ) t o c h e c k f o r o p t i m a l i t y. O t h e r w i s e, c o n ­
t i n u e.
6. C o m p u t e s t e p l e n g t h, k e e p i n g i n m i n d t h a t n o n e o f t h e v a r i a b l e s s h o u l d b e c o m e n e g a t i v e.
a = M i n [ — d t + l T V f ( x 1)/d,+ 1'r Q d'+ 1, M i n [ —x?j/d *, d * > 0,j = 1,, m ] ]
7. U p d a t e x,+1 = x ’ + a d i + 1.
8. I f a n e w v a r i a b l e w i t h a n i n d e x l e s s t h a n o r e q u a l t o m h a s r e a c h e d a z e r o v a l u e, m o v e t h a t t o t h e p a s s i v e v a r i a b l e. S e t i = i + 1 a n d g o t o s t e p ( 2 ). T h e a l g o r i t h m i s i m p l e m e n t e d i n a M a t h e m a t i c a f u n c t i o n c a l l e d A c t i v e S e t D u a l Q P. T h e f u n c t i o n s h o u l d b e l o a d e d u s i n g t h e f o l l o w i n g n e e d s c o m m a n d p r i o r t o i t s u s e.
N e e d s ["O p t i m i z a t i o x i T o o l b o x'Q u a d r a t i c P r o g r a j n m i n g'"] ;
E x a m p l e 8.1 3 F i n d t h e m i n i m u m o f t h e f o l l o w i n g Q P p r o b l e m:
/ = + 12x2 + 2x f + 4 x% ,— X\X 2 — X 2%3 — 2x 4 + 3 * i s u b j e c t t o *3 a n d X 4 > 0
Li /
X? 2 2 2
f = -y- + 12x j + 2x 3 + 4x 4 - χ χ χ 2 - X2x3 - 2x 4 + 3 χ χ; v a r s = { x:, x 2 , x 3 , x 4 } ; f r e e V a r s = { x x, x 2 };
U s i n g A c t i v e S e t D u a l Q P, t h e s o l u t i o n i s a s f o l l o w s. A l l i n t e r m e d i a t e r e s u l t s a r e p r i n t e d u s i n g n o t a t i o n u s e d i n d e s c r i b i n g t h e a l g o r i t h m.
4. C o m p u t e d i r e c t i o n d « = ( y f J $?f i d, ] > ° )
8.5 A c t i v e S e t M e t h o d f o r t h e D u a l Q P P r o b l e m
A c t i v e S e t D u a l Q P [ f y v a r s, P r i n t L e v e l . 2, U n r e s t r i c t e d V a r i a b l e s . f r e e v a r s ];
2
X! ^ 2 2
M i n i m i z e -> 3 χ χ + — - Χχ Χ2 + 12*2 - X2X3 + 2x1 - 2 x 4 + 4 x 4 S u b j e c t t o. ( x 3 2 0 x 4 > 0 )
U n r e s t r i c t e d v a r i a b l e s - » ( x! x 2 )
I 1 2 3 4
O r d e r o f v a r i a b l e s .
\x 3 x 4 X]_ x 2 g p S o l u t i o n O p t i m a l i t y C h e c k 1
x 3
1.'
' 3. '
F a l s e ’
x 4
.
1.
V f .
6.
O p t i m a l i t y s t a t u s .
F a l s e
* 1
1.
3.
F a l s e
x 2-
1.,
2 2.,
.F a l s e,
* * * * q p s o l u t i o n P a s s 1 * * * *
A c t i v e v a r i a b l e s. ( x 3 x 4 x^ x 2 )
1. 1. 1.)
C u r r e n t p o i n t . ( 1.
- ί -
0
0
- 1
- 2
Q -*
0
8
0
0
3
0
0
1
- 1
o
i - i
0
- 1
2 4/
V f .
» C o n j u g a t e G r a d i e n t I t e r a t i o n 1
6.
3.
( 2 2.)
' - 3.
' 3. ’
- 6.
V f .
6.
- 3.
3.
- 2 2.,
.22. j
I I Vf | | .2 3,1 9 4 8 0.0. a l .0.0 4 6 0 4 1 9
- p t/d. ( 0.3 3 3 3 3 3 0.1 6 6 6 6 7 0.3 3 3 3 3 3 0.0 4 5 4 5 4 5 )
a 2 .0,1 6 6667
a . 0.0 4 6 0 4 1 9 C G - S t a t u s . NonOpt i mum
New p o i n t. ( 0.8 6 1 8 7 4 0.7 2 3 7 4 8 0.8 6 1 8 7 4 - 0.0 1 2 9 2 2 6 )
O b j e c t i v e f u n c t i o n . 5.1 1 4 7 2
» C o n j u g a t e G r a d i e n t I t e r a t i o n 2
- 3 .7 1 4 0 8
f 3.4 6 0 4 2
- 4.2 9 7 3
Vf .
3.7 8 9 9 9
- 4.1 2 8 4 5
3.8 7 4 8
,0.1 7 3 7 3 4,
- 2.0 3 3 8 9,
I| Vf | | . 6.7 4 4 5 7 β . 0.0 8 4 5 5 2 5 3 a l . 0.2 0 3 6 1 7
- p t/d. ( 0.2 3 2 0 5 6 0.1 6 8 4 1 9 0.2 0 8 7 6 4 0.0 7 4 3 8 1 3 )
a 2 .0.1 6 8 4 1 9
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
a - > 0.1 6 8 4 1 9 CG-S t a t u s .NonOp t i mum New p o i n t - » ( θ.2 3 6 3 5 2 0. 0,1 6 6 5 6 3 0.0 1 6 3 3 7 6 )
O b j e c t i v e f u n c t i o n . 0.6 2 1 9 0 6 New v a r t o b e ma d e p a s s i v e. ( x4 }
**** q p s o l u t i o n P a s s 2 ****
A c t i v e v a r i a b l e s. (X 3 Χχ x 2 )
C u r r e n t p o i n t. ( 0.2 3 6 3 5 2 0.1 6 6 5 6 3 0.0 1 6 3 3 7 6 )
Ό’
' 4
0
-1'
r 0.929071 '
c .
3
Q - >
0
1
-1
Vt .
3.15023
0,
-1
-1
24j
-0.0108132,
> > C o n j u g a t e G r a d i e n t I t e r a t i o n 1
- 0.9 2 9 0 7 1'
’ 0.9 2 9 0 7 1 ’
d.
- 3 .1 5 0 2 3
V f .
3.1 5 0 2 3
0.0 1 0 8 1 3 2,
- 0.0 1 0 8 1 3 2,
| | V f | | . 3.2 8 4 3 9 0.0.
Q l .0.8 0 0 9 7 3
- p t/d. ( 0.2 5 4 3 9 6 0.0 5 2 8 7 3 4 - 1.5 1 0 9 )
a 2 .0.2 5 4 3 9 6
a . 0.2 5 4 3 9 6 C G - S t a t u s . N o n O p t i m u m
N e w p o i n t . ( - 2.7 7 5 5 6 x 1 0"17 - 0.6 3 4 8 4 2 0.0 1 9 0 8 8 4 )
O b j e c t i v e f u n c t i o n .- 1.6 8 6 5 2
N e w v a r t o b e m a d e p a s s i v e . ( x 3 )
* * * * q p S o l u t i o n P a s s 3 * * * *
A c t i v e v a r i a b l e s, ( x! x 2 )
C u r r e n t p o i n t. ( - 0,6 3 4 8 4 2 0.0 1 9 0 8 8 4 )
f 3\ _ I 1 - l\ _ ( 2.3 4 6 0 7 )
c .
V f .
l Oj l - l 2 4 1 ~ ' \1.0 9 2 9 6
» C o n j u g a t e G r a d i e n t I t e r a t i o n 1
- 2.3 4 6 0 7\ /2.3 4 6 0 7\
- 1.0 9 2 9 6 ] [ l.0 9 2 9 6 )
I | V f I I .2.5 8 8 1 7 0.0. a l.0.2 3 0 6 2 6
a. 0.2 3 0 6 2 6 C G - S t a t u s .N o n O p t i m u p i N e w p o i n t. ( - 1.1 7 5 9 1 - 0.2 3 2 9 7 7 )
O b j e c t i v e f u n c t i o n .- 2.4 5 8 9 6 > > C o n j u g a t e G r a d i e n t I t e r a t i o n 2] f - 1 0.3 6 7 6\ / 2.0 5 7 0 7
Vf
0.5 4 3 9 2 4
- 4.4 1 5 5 4
8.5 A c t i v e S e t M e t h o d f o r t h e D u a l Q P P r o b l e m
I I Vf I I . 4.8 7 1 2 0. 3 .5 4 2 3 1 a l . 0 .1 8 8 5 2 3
a . 0,1 8 8 5 2 3 C G - S t a t u s . N o n O p t i m u m Ne w p o i n t - » ( - 3.1 3 0 4 3 - 0.1 3 0 4 3 5 )
O b j e c t i v e f u n c t i o n - » - 4.6 9 5 6 5 >> C o n j u g a t e G r a d i e n t I t e r a t i o n 3
/- 4.7 1 2 1 9 x 1 0 ~ 1 5\ / 0.
d _ > [ - 2 - 1 9 5 2 7 x l 0'1 5 j \2 .4 4 2 4 9 χ 1 0 ~ 15
| | V f f | .2.4 4 2 4 9 x l O -15 β -» 4. 5 4 5 1 2 χ I O -16
a -» 0 C G - S t a t u s -» O p t i m u m
N e w p o i n t - » ( - 3.1 3 0 4 3 - 0.1 3 0 4 3 5 )
O b j e c t i v e f u n c t i o n -» - 4 . 6 9 5 6 5
QP S o l u t i o n O p t i m a l i t y c h e c k 2
M
0
0.1 3 0 4 3 5 1
'T r u e 1
x 4
X1
.
0
- 3.1 3 0 4 3
,- 0.1 3 0 4 3 5;
Vf .
- 2 . 0 0
O p t i m a l i t y s t a t u s .
F a l s e T r u e .T r u e,
**** q p s o l u t i o n P a s s 1 ****
A c t i v e v a r i a b l e s . ( x 4 x 2 )
C u r r e n t p o i n t . ( θ - 3 .1 3 0 4 3 - 0.1 3 0 4 3 5 )
[ - 21
( 8
0
0
- 2.
c .
3
Q.
0
1
- 1
V f .
0.
l o J
10
- 1
24.!
i 2.4 4 2 4 9 x l O - 1 5,
> > C o n j u g a t e G r a d i e n t I t e r a t i o n 1
2.
ί ~ 2'
d .
0.
V f .
0.
,- 2.4 4 2 4 9 x l O * 1 5,
,2.4 4 2 4 9 x l O - 1 5,
! I vf | | -» 2 . 0.0. a l. 0.1 2 5
- p t/d. ( θ C o m p l e x l n f i n i t y - 5.34024 χ 1013) a 2 —> oo
_________________________________________________ j
a . 0 .1 2 5 C G - S t a t u s . N o n O p t i m u m N e w p o i n t. ( 0.2 5 - 3.1 3 0 4 3 - 0.1 3 0 4 3 5 )
O b j e c t i v e f u n c t i o n . - 4.9 4 5 6 5 > > C o n j u g a t e G r a d i e n t I t e r a t i o n 2
1.7 9 9 5 8 x l O"29
■4 ■ 4 4 0 8 9 x 1 0 ~16
4.8 8 4 9 8 χ 1 0'15
0.
Vf . 4,4 4 0 8 9 x l 0 ~16
- 4.8 8 4 9 8 x I O- 1 5,
I |V f | | . 4.9 0 5 1 3 x l O -15 0. 8 .9 9 7 9 4 x I O -30
a . 0 C G - S t a t u s . Opt i mum
C h a p t e r 8 Q u a d r a t i c P r o g r a m m i n g
New p o i n t, ( θ.25 - 3.1 3 0 4 3 - 0.1 3 0 4 3 5 ) O b j e c t i v e f u n c t i o n . - 4 .9 4 5 6 5
QP S o l u t i o n O p t i m a l i t y C h e c k 3
M
x 4
.
0
0.2 5
Vf .
0.1 3 0 4 3 5'
0
O p t i m a l i t y s t a t u s .
'T r u e'
T r u e
X1
l x2 i
- 3.1 3 0 4 3 ,- 0.1 3 0 4 3 5,
u
0 ; j
T r u e
.True,
T h e s o l u t i o n c a n e a s i l y b e c o n f i r m e d b y s o l v i n g t h e p r o b l e m d i r e c t l y b y u s i n g K T c o n d i t i o n s.
K T S o l u t i o n [ £, {x3 * 0,x 4 £ 0 }, v a r s ] ;
* **** L a g r a n g i a n.
2
X1 2 / ο \ 2 / 2 \ 2
3 x i + — - χ ι χ 2 + 1 2 x 2 ( β! - x 3 ] - x 2x 3 + 2X 3 + u 2 ( s 2 - x 4 J - 2 x 4 + 4 x 4
* **** V a l i d KT P o i n t ( s ) * * * * *
f . - 4.9 4 5 6 5 X l .- 3.1 3 0 4 3 x 2 . - 0 .1 3 0 4 3 5 x 3 . 0 x 4 . 0.2 5 u x . 0.1 3 0 4 3 5 u 2 . 0 s i . 0
s 2 . 0.2 5
E x a m p l e 8.1 4 C o n s t r u c t t h e d u a l o f t h e f o l l o w i n g Q P p r o b l e m a n d t h e n s o l v e t h e d u a l u s i n g t h e A c t i v e S e t D u a l Q P m e t h o d. f = x f + x\x i - f 2 x q + 2 x f -I- 2x 2x 3 4 - 4 x ^ 4 - 6x 2 - I - 12*3
(
Xl + X2 + XS = 6 \
- x i - X 2 + 2 x 3 = 2 I χ,· > 0,ϊ = 1,...,3/
2 2 2 £ = Xl + X i X2 + 2 x 2 + 2 x 3 + 2 x 2 x 3 + 4 x x + 6 x 2 + 1 2 x 3 ;
g = {X i + χ 2 + X3 6, — X i — x 2 + 2x 3 = = 2} /
v a r s = { χ 1# χ 2, x 3 } ;
T h e d u a l Q P i s c o n s t r u c t e d a s f o l l o w s:
{ d f, d g, d v a r s, d v } = F o r m D u a l Q P [ £, g, v a r s ] ;
P r i m a l p r o b l e m
8.5 A c t i v e S e t M e t h o d f o r t h e D u a l Q P P r o b l e m
M i n i m i z e -» 4 x x + χ χ + 6x 2 + x · ^ + 2 x 2 + 1 2 x 3 + 2 x 2x 3 + 2x 3
S u b j e c t t o -» [ X1 + x 2 + x 3 - \~ χ χ - x 2 + 2 x 3
a n d -» (χχ a 0 x 2 i O χ3 >ϋ)
Χ 21 ) b -
' 4 '
2
1
O'
c -*
6
Q -*
1
4
2
Λ2;
,0
2
4;
3
1
1
5
“ 5
Τϋ
—4 + — V 2
1
2
5
1
“ Ε
-C+U+AT. v ->
— 6 + U 2 + — V 2
1
ν τ ϋ
1
“ 5
A j
-12 + U 3 + v i + 2v 2 ;
S o l u t i o n ■
Xl _ > + Ι ϋ ι _ H i + H i + Z l _ Ώ. V 1 5 5 5 10 2 ,5
4 Ui 2 u 2 u 3 3 vt
x, _» i + — -=■ ------ -
5 _ 5 5 5 5
1 7 ^ u 2 7 u ^
4 Vn
20
D u a l QP p r o b l e m
V a r i a b l e s ^ (u^ u 2 U3 v 2 )
M a x i m i z e -> -
1 1 4 1 2 u x 3 u i
ί ϋ + u l u 2
5 5
u 2
17u-i
5 5 1 0 5 5 5 5
2 2
u l v l U3V1 3 v i 3 6 v 2 U1V2 3 u 2 v 2 4 u 3 v 2 6v 2
~ 2 4 a ” + 5 + 5 + 5 5 5 ~
u l u 3 U2 U3 1 0 5
7u f
40
+ llvx
S u b j e c t t o
/Ui > 0
u 2 2: 0 U3 i. 0,
T h e d u a l Q P i s s o l v e d u s i n g A c t i v e S e t D u a l Q P, s h o w i n g a l l i n t e r m e d i a t e c a l c u ­
l a t i o n s.
A c t i v e S e t D u a l Q P [ - d f, d v a r s, P r i n t L e v e l -» 2, U n r e s t r i c t e d V a r i a b l e s -> d v ];
. . 114 12U-, 3ui 4u, u-,u2 u2 17u-> U1 U3 u2u3 7u3
M i n i m i z e -> --------- + — - + — ----------------+ — ------------ + 1 J --------4 - * - + — - - l l v i +
5 5 10 5 5 5 5 10 5 40
2 2 u ^ v ^
Автор
dima202
dima202579   документов Отправить письмо
Документ
Категория
Наука
Просмотров
526
Размер файла
24 269 Кб
Теги
mathematica, 2000
1/--страниц
Пожаловаться на содержимое документа