Informed Search

Russell and Norvig: Ch. 4.1 - 4.3 Rich and Knight

Outline
Informed = use problem-specific knowledge Which search strategies?

Best-first search and its variants How to invent them Hill climbing, simulated annealing, local beam search,

Heuristic functions?

Local search and optimization

Tree Search Algorithm

function TREE-SEARCH(problem,fringe) return a solution or failure fringe INSERT(MAKE-NODE(INITIAL-STATE[problem]), fringe) loop do if EMPTY?(fringe) then return failure node REMOVE-FIRST(fringe) if GOAL-TEST[problem] applied to STATE[node] succeeds then return SOLUTION(node) fringe INSERT-ALL(EXPAND(node, problem), fringe)

Tree Search Algorithm (2)

function EXPAND(node,problem) return a set of nodes successors the empty set for each < action, result> in SUCCESSOR-FN[ problem] (STATE[node]) do s a new NODE STATE[s] result PARENT-NODE[s] node ACTION[s] action PATH-COST[s] PATH-COST[node] + STEP-COST(node, action,s) DEPTH[s] DEPTH[node]+1 add s to successors return successors

Previously: Graph search algorithm

function GRAPH-SEARCH(problem,fringe) return a solution or failure closed an empty set fringe INSERT(MAKE-NODE(INITIAL-STATE[problem]), fringe) loop do if EMPTY?(fringe) then return failure node REMOVE-FIRST(fringe) if GOAL-TEST[problem] applied to STATE[node] succeeds then return SOLUTION(node) if STATE[node] is not in closed then add STATE[node] to closed fringe INSERT-ALL(EXPAND(node, problem), fringe)

A strategy is defined by picking the order of node expansion

Closed list:-> it stores every expanded node Open List:-> fringe of unexpanded nodes

Search Algorithms
Blind search BFS, DFS, uniform cost

no notion concept of the right direction can only recognize goal once its achieved

Heuristic search we have rough idea of how good various states are, and use this knowledge to guide our search

Best-first search
General approach of informed search:

Best-first search: node is selected for expansion based on an evaluation function f(n)

Idea: evaluation function measures distance to the goal.

Choose node which appears best

Implementation: fringe is queue sorted in decreasing order of

desirability. Special cases: Greedy search, A* search

Informed Search
Add domain-specific information to select the best path along which to continue searching Define a heuristic function, h(n), that estimates the goodness of a node n. Specifically, h(n) = estimated cost (or distance) of minimal cost path from n to a goal state. If n= goal node then h(n)=0. The heuristic function is an estimate, based on domain-specific information that is computable from the current state description, of how close we are to a goal

Greedy Best-First Search

f(N) = h(N) greedy best-first Example of route finding problem in Romania Heuristic:- Straight line distance ( h SLD ) Greedy best-first search expands the node that appears to be closest to goal

Algorithm Greedy Best first search

1. Start with OPEN containing just the initial state. 2. Until a goal is found or there are no nodes left on
open do: (a) Pick the best node on OPEN (b) Generate its successors (c) For each successor do:

(i) if it has not been generated before, evaluate it, add it to OPEN and (ii) if it has been generated before, change the parent if this new path is better than the previous one . In that case update the cost of getting to this node and to any successors that this node may already have.

Example Greedy BFS Map of Romania

Value of hSLD to Bucharest

City Arad Bucharest Craiova Dobreta Eforie Fagaras Giurgiu Hirsova Iasi Lugoj Heuristic 366 0 160 242 161 176 77 151 226 244 City Mehandia Neamt Oradea Pitesti Rimnicu Vilcea Sibiu Timisoara Urziceni Vaslui Zerind Heuristic 241 234 380 100 193 253 329 80 199 374

Example

Attention: path[Arad, Sibiu, Fagaras] is not optimal!

Properties of greedy best-first search

Complete? No can get stuck in loops; Yes, if we can avoid repeated states Time? O(bm), but a good heuristic can give dramatic improvement Space? O(bm) -- keeps all nodes in memory Optimal? No
17

A* search
A* search combines Uniform-cost and Greedy Best-first Search Evaluation function: f(N) = g(N) + h(N) where:

0 < c(N,N) (no negative cost steps). Order the nodes in the fringe in increasing values of f(N)

g(N) is the cost of the best path found so far to N h(N) is an admissible heuristic f(N) is the estimated cost of cheapest solution through N

A* search example

19

A* search example

20

A* search example

21

A* search example

22

A* search example

23

A* search example

24

Admissible heuristic
Let h*(N) be the true cost of the optimal path from N to a goal node Heuristic h(N) is admissible if: 0 h(N) h*(N) An admissible heuristic is always optimistic If h(n) is admissible, A*using TREE-SEARCH is optimal

Optimality of A*(standard proof)

Suppose suboptimal goal G2 in the queue. Let n be an unexpanded node on a shortest to optimal goal G.
f(G2 ) = g(G2 ) since h(G2 )=0 =g(G2)>C* C* is cost of optimal solution For node n that is optimal solution we know f(n)=g(n) +h(n) <=C* f(n) <=C*<f(g2) So G2 will never be expanded and A* must return an optimal solution

BUT graph search

Discards new paths to repeated state.

Previous proof breaks down Add extra bookkeeping i.e. remove more expensive of two paths. Ensure that optimal path to any repeated state is always first followed.
Extra requirement on h(n): consistency

Solution:

(monotonicity)

Consistency
h ,, ' +n ( c a) h) n ( n ( ) n '
if h(n) is consistent then the values of f(n) along any path are nondecreasing. Proof:
If n is successor of n then g(n)=g(n) + c(n,a,n) for some a f(n) = g(n)+h(n) =g(n)+c(n,a,n)+h(n) >=g(n) + h(n) >=f(n)

Optimality of A*(more useful)

A* expands nodes in order of increasing f value Contours can be drawn in state space

Uniform-cost search adds circles.

F-contours are gradually Added: 1) nodes with f(n)<C* 2) Some nodes on the goal Contour (f(n)=C*).

Contour I has all Nodes with f=fi, where fi < fi+1.

A* search, evaluation
Completeness: YES

Since bands of increasing f are added Unless there are infinitely many nodes with f<f(G)

A* search, evaluation
Completeness: YES Time complexity:

Number of nodes expanded is still exponential in the length of the solution.

A* search, evaluation
Completeness: YES Time complexity: (exponential with path length) Space complexity:

It keeps all generated nodes in memory Hence space is the major problem not time

A* search, evaluation
Completeness: YES Time complexity: (exponential with path length) Space complexity:(all nodes are stored) Optimality: YES

Cannot expand fi+1 until fi is finished. A* expands all nodes with f(n)< C* A* expands some nodes with f(n)=C* A* expands no nodes with f(n)>C*

Memory-bounded heuristic search

Some solutions to A* space problems (maintain completeness and optimality)

Iterative-deepening A* (IDA*)
Here cutoff information is the f-cost (g+h) instead of

depth

Recursive best-first search(RBFS)

Recursive algorithm that attempts to mimic standard

best-first search with linear space.

(simple) Memory-bounded A* ((S)MA*)

Drop the worst-leaf node when memory is full

Recursive best-first search

function RECURSIVE-BEST-FIRST-SEARCH(problem) return a solution or failure return RFBS(problem,MAKE-NODE(INITIAL-STATE[problem]),) function RFBS( problem, node, f_limit) return a solution or failure and a new fcost limit if GOAL-TEST[problem](STATE[node]) then return node successors EXPAND(node, problem) if successors is empty then return failure, for each s in successors do f [s] max(g(s) + h(s), f [node]) repeat best the lowest f-value node in successors if f [best] > f_limit then return failure, f [best] alternative the second lowest f-value among successors result, f [best] RBFS(problem, best, min(f_limit, alternative)) if result failure then return result

Recursive best-first search

Keeps track of the f-value of the bestalternative path available.

If current f-values exceeds this alternative f-value than backtrack to alternative path. Upon backtracking change f-value to best f-value of its children. Re-expansion of this result is thus still possible.

Recursive best-first search, ex.

Path until Rumnicu Vilcea is already expanded Above node; f-limit for every recursive call is shown on top. Below node: f(n) The path is followed until Pitesti which has a f-value worse than the f-limit.

Recursive best-first search, ex.

Unwind recursion and store best f-value for current best leaf Pitesti
result, f [best] RBFS(problem, best, min(f_limit, alternative))

best is now Fagaras. Call RBFS for new best

best value is now 450

Recursive best-first search, ex.

Unwind recursion and store best f-value for current best leaf Fagaras
result, f [best] RBFS(problem, best, min(f_limit, alternative))

best is now Rimnicu Viclea (again). Call RBFS for new best

Subtree is again expanded. Best alternative subtree is now through Timisoara.

Solution is found since because 447 > 417.

RBFS evaluation
RBFS is a bit more efficient than IDA*

Still excessive node generation (mind changes)

Like A*, optimal if h(n) is admissible Space complexity is O(bd).

IDA* retains only one single number (the current f-cost limit) Depends on accuracy if h(n) and how often best path changes.

Time complexity difficult to characterize

IDA* and RBFS suffer from too little memory.

(simplified) memory-bounded A*
Use all available memory.

I.e. expand best leafs until available memory is full When full, SMA* drops worst leaf node (highest f-value) Like RFBS backup forgotten node to its parent

What if all leafs have the same f-value?

Same node could be selected for expansion and deletion. SMA* solves this by expanding newest best leaf and deleting oldest worst leaf.

SMA* is complete if solution is reachable, optimal if optimal solution is reachable.

Heuristic functions

E.g for the 8-puzzle

Avg. solution cost is about 22 steps (branching factor +/- 3) Exhaustive search to depth 22: 3.1 x 1010 states. The corresponding states for 15 puzzle problem is 1013 A good heuristic function can reduce the search process.

Heuristic Function
Function h(N) that estimates the cost of the cheapest path from node N to goal node. Example: 8-puzzle
5 8 4 2 1 7 3 6 N 1 2 3 4 5 6 7 8 goal h1(N) = number of misplaced tiles =6

Heuristic Function
Function h(N) that estimate the cost of the cheapest path from node N to goal node. Example: 8-puzzle
5 8 4 2 1 7 3 6 N 1 2 3 4 5 6 7 8 goal h2(N) = sum of the distances of every tile to its goal position =2+3+0+1+3+0+3+1 = 13

City block distance or Manhattan distance

8-Puzzle
5

f(N) = h1(N) = number of misplaced tiles

3 3 4

2 4 3 3 4 5 4 2 1 0

8-Puzzle

f(N) = g(N) + h(N) with h1(N) = number of misplaced tiles

3+3 1+5 2+3 3+4 5+2

0+4 1+3 2+3

3+2 3+4 1+5 2+4

4+1 5+0

8-Puzzle
6

f(N) = h2(N) = distances of tiles to goal

2 5 4 3 4 6 5 2 1 0

8-Puzzle

EXERCISE: f(N) = g(N) + h2(N) with h2(N) = distances of tiles to goal

0+4

Heuristic quality
Effective branching factor b*

Is the branching factor that a uniform tree of depth d would have in order to contain N+1 nodes.

N = b () ++) + 1 *b . ( d 1++ 2 . b * *

Measure is fairly constant for sufficiently hard problems.

Can thus provide a good guide to the heuristics overall

usefulness. A good value of b* is 1.

Heuristic quality and dominance

1200 random problems with solution lengths from 2 to 24. If h2(n) >= h1(n) for all n (both admissible) then h2 dominates h1 and is better for search

Inventing admissible heuristics

Admissible heuristics can be derived from the exact solution cost of a relaxed version of the problem:

Relaxed 8-puzzle for h1 : a tile can move anywhere

As a result, h1(n) gives the shortest solution

Relaxed 8-puzzle for h2 : a tile can move to any adjacent square.

As a result, h2(n) gives the shortest solution.

The optimal solution cost of a relaxed problem is no greater than the optimal solution cost of the real problem.

Another approach Local Search

Previously: systematic exploration of search space.

Path to goal is solution to problem E.g 8-queens

Hill-climbing or Gradient descent Simulated Annealing Genetic Algorithms, others

YET, for some problems path is irrelevant.

Different algorithms can be used: Local Search

Also applicable to optimization problems

systematic search doesnt work however, can start with a suboptimal solution and improve it

Local Search algorithms operate using a single current state and generally move only to the neighbor of that state.

Local search and optimization

Local search= use single current state and move to neighboring states. Advantages:

Use very little memory Find often reasonable solutions in large or infinite state spaces. The goal is to find best state according to some objective function.

Are also useful for pure optimization problems.

Local search and optimization

State Space Landscape

Hill-climbing search
is a loop that continuously moves in the direction of increasing value

It terminates when a peak is reached.

Hill climbing does not look ahead of the immediate neighbors of the current state. Hill-climbing chooses randomly among the set of best successors, if there is more than one. Hill-climbing a.k.a. greedy local search Some problem spaces are great for hill climbing and others are terrible.

Hill-climbing search
function HILL-CLIMBING(problem) return a state that is a local maximum input: problem, a problem local variables: current, a node. neighbor, a node.

current MAKE-NODE(INITIAL-STATE[problem]) loop do neighbor a highest valued successor of current if VALUE [neighbor] VALUE[current] then return STATE[current] current neighbor

8-queens state with heuristic cost estimate h=17

18 14 14 15 A 17 18 14 12 16 12 14 14 B 14 14 14 13 18 14 17 16 C 13 13 15 13 D 15 18 15 17 13 12 15 13 E 15 15 12 12 14 12 16 14 F 14 14 14 12 14 13 16 15 G 12 14 16 14 16 16 H 16 18

h=17 A B AC BC DE DF DG EF EG FG FH

GH AE BF CG BD CE BH

With h=1
A B C D E F G H

Hill climbing example

start -5 2 8 3 1 6 4 7 5 h = -4 -5 2 8 3 1 4 h = -3 7 6 5 -3 2 3 1 8 4 7 6 5 -4 2 3 1 8 4 h = -2 7 6 5 goal -2 1 2 3 8 4 h = -1 7 6 5 1 2 3 8 4 h=0 7 6 5

h = -3

-4 f(n) = -(number of tiles out of place)

Example of a local maximum

1 2 5 7 4 8 6 3 -4 goal 1 2 5 7 4 0 8 6 3

start 1 2 5 7 4 8 6 3 -3

1 2 5 7 4 -4 8 6 3 1 2 5 7 4 -4 8 6 3

An 8-Puzzle Problem Solved by the Hill Climbing Method

61

Drawbacks of hill climbing

Problems: Local maxima: is a peak that is higher than each of its neighboring states but lower than the global maximum. Plateaus: the space has a broad flat region that gives the search algorithm no direction. It can be a flat local maxima or a shoulder (random walk) Ridges: flat like a plateau, but with drop offs to the sides; steps to the North, East, South and West may go down, but a combination of two steps (e.g. N, W) may go up. Remedy:

Introduce randomness

Hill-climbing variations
Stochastic hill-climbing

Random selection among the uphill moves. The selection probability can vary with the steepness of the uphill move. Stochastic hill climbing by generating successors randomly until a better one is found. Tries to avoid getting stuck in local maxima. If at first you dont succeed, try, try again

First-choice hill-climbing

Random-restart hill-climbing

Simulated annealing
Escape local maxima by allowing bad moves.

Idea: but gradually decrease their size and frequency.

Origin; metallurgical annealing Bouncing ball analogy:

Shaking hard (= high temperature). Shaking less (= lower the temperature).

If T decreases slowly enough, best state is reached. Applied for VLSI layout, airline scheduling, etc.

Simulated annealing
function SIMULATED-ANNEALING( problem, schedule) return a solution state input: problem, a problem schedule, a mapping from time to temperature local variables: current, a node. next, a node. T, a temperature controlling the probability of downward steps

current MAKE-NODE(INITIAL-STATE[problem]) for t 1 to do T schedule[t] if T = 0 then return current next a randomly selected successor of current E VALUE[next] - VALUE[current] if E > 0 then current next else current next only with probability eE /T

Simulated Annealing
applet Successful application: circuit routing, traveling sales person (TSP)

Local beam search

Keep track of k states instead of one Initially: k random states Next: determine all successors of k states

If any of successors is goal finished Else select k best from successors and repeat.

Major difference with random-restart search Information is shared among k search threads. Can suffer from lack of diversity.

Stochastic variant: choose k successors at proportionally to state success.

Genetic algorithm
Successor states are generated by combining two parent states. GAs begin with a set of k randomly generated states population Each state is rated by the evaluation function or the fitness function. Various techniques may be used for the production of next generation of states:

Selection Crossover Mutation

When to Use Search Techniques?

The search space is small, and

There is no other available techniques, or It is not worth the effort to develop a more efficient technique There is no other available techniques, and There exist good heuristics

The search space is large, and

Summary: Informed Search

Heuristics Best-first Search Algorithms

Greedy Search A*
Admissible heuristics

Constructing Heuristic functions Local Search Algorithms