Heuristic Optimization

Introduction to Heuristic Optimization

Athens 2005 Heuristic Optimization (http://laurel.datsi.fi.upm.es/docencia/cursos/heuristic_optimization)

Transcript

1. Athens: Heuristic Optimization Oscar Cubo Medina (ocubo@fi.upm.es)
2. Outline 1.  What are optimization problems? 2.  Exhaustive vs Heuristic approaches 3.  Typical problems
3. Optimization l  Search for the “best” configuration of a set of variables to achieve some goals l  Two categories: l  Discrete variables l  Real-valued variables l  Each optimization problem is specified defining l  Possible solutions (states) l  Objective
4. Optimization l  A optimization problem is a pair (X, f ) being X the search space (all the possible solutions), and f a function f : X → R l  The solution xopt ∈ X is optime if f (xopt )≤f (x ), ∀x ∈ X
5. Search Space x y fitness x y fitness x y fitness x y fitness
6. Exhaustive algorithms l  Generate all possible solutions l  Evaluates all generated solutions l  Select the best solution A B D C E F
7. Exhaustive algorithms l  Generate all possible solutions l  Evaluates all generated solutions l  Select the best solution A-7 B-10 D-15 C-12 E-17 F-14
8. Exhaustive algorithms l  Generate all possible solutions l  Evaluates all generated solutions l  Select the best solution A-7 B-10 D-15 C-12 E-17 F-14
9. Heuristic algorithms l  Generate a subset of solutions (random) l  Evaluates the generated solutions l  Select the best solutions l  Generate new subset of solution from the selected A B D C E F
10. Heuristic algorithms l  Generate a subset of solutions (random) l  Evaluates the generated solutions l  Select the best solutions l  Generate new subset of solution from the selected A-7 B-10 D-15 C-12 E-17 F-14
11. Heuristic algorithms l  Generate a subset of solutions (random) l  Evaluates the generated solutions l  Select the best solutions l  Generate new subset of solution from the selected A-7 B-10 D-15 C-12 E-17 F-14
12. Heuristic algorithms l  Generate a subset of solutions (random) l  Evaluates the generated solutions l  Select the best solutions l  Generate new subset of solution from the selected D-15 E-17 F-14 Y Z X
13. Exhaustive vs Heuristic algorithms l  Exhaustive algorithms l  Generate all possible solutions l  Evaluates all generated solutions l  Select the best solution l  Heuristic algorithms l  Generate a subset of solutions (random) l  Evaluates the generated solutions l  Select the best solutions l  Generate new subset of solution from the selected
14. Exhaustive vs Heuristic algorithms l  Exhaustive algorithms l  Small search space l  Known search space l  Need the best solution l  Deterministic l  Heuristic algorithms l  Huge search space l  Unknown search space l  A suboptimal solution is valid l  Not deterministic
15. Typical problems l  Some problems: l  Travel Salesman Problem (TSP) l  The 0-1 knapsack problem l  Scheduling problem l  Steps to solve: l  Find a representation of the possible solutions of the problem l  Create a fitness function that evaluates all individuals l  Compendium of NP optimization problems: http://www.nada.kth.se/~viggo/problemlist/compendium.html
16. TSP l  A salesman has to find a route which visits each of n cities, and which minimizes the total distance travelled l  Input: Given an integer N ≥ 3 and a n n matrix C=(cij ) where each cij is a nonnegative integer with the distance between the cities i and j l  Solution: which cyclic permutation Π of integers from 1 to N minimizes the distance?
17. The 0-1 knapsack problem l  Input: A set of n items to be packed into a knapsack with capacity C units. Each item i has value vi and uses up ci units of capacity l  Solution: the subset of items which should be packed to maximize the total value without exceeding the capacity
18. Scheduling problem l  Input: Set T of tasks, number m of processors, length l(t,i)∈ Ζ+ for each task t ∈T and processor i∈[1..m] l  Solution: An m-processor schedule for T that minimize the completion time for the schedule
19. Planning Day 1 Day 2 Day 3 Day 4 Day 5 9:00 10:45 · Presentation · Introduction to heuristic optimization · Genetic Algorithms · EDAs · Genetic programming · High- performance optimization · Multiobjective optimization 10:45 11:00 Rest 11:00 12:00 · Simple methods · Genetic Algorithms · EDAs · Hybrid Algorithms · Difficulty on optimization problems · Other optimization techniques 12:00 12:45 Lunch 12:45 15:00 Practical work: Simulated Annealing Practical Work: Genetic Algorithms Practical Work: EDAs Practical Work: High- Performance Optimization Exam