Heuristic (Informed) Search (Chapter 4) ======================================= -- Take advantage of information about the problem -- Used in our search algorithms: ``heuristic evaluation function'' A metric on objects in the search space; an estimate of the distance from a node to a goal Use it to decide which node to explore next; try to explore promising parts of the state space, and hope to find the goal in a reasonable amount of time. We will use ``h'' to refer to such a function. (text, p 93: `Eval-Fn') h: state --> numerical estimate of goodness Where specifically is it used in what the text calls general search? (in our tree-search?) --> The order of the nodes on ``nodes'' (open) determines the order in which the nodes are searched The queuing function determines the order of the nodes on ``nodes'' (open) So, h is used in the queuing function -- A heuristic need not *always* improve decision making, but it should improve it more often than not. In a supermarket, how choose which line to stand in? Choose the shortest one -- Evaluation criteria for search (from Chapter 3) -- optimality: when several solutions, chooses the best; by "best", we mean that the cost of the path to the goal is less than (or equal to) the costs of the paths to all other goals Generally speaking, finding the very best solution is not the kind of improvement we hope to gain with heuristics. Can prove optimality for some heuristic searches It can be worth the savings in time and space to not try for the best solution, but be satisfied with any solution. Simon: people are ``satisficers'': people often don't seek optimal solutions. As soon as they find a solution that meets certain requirements, they stop. (Choosing a line at the grocery store; finding a parking space). -- completeness (guaranteed to find a solution if there is one) Can prove completeness for some heuristic searches -- time complexity (how long does it take to find a solution) == how much of the search space is searched to find a solution Reducing the amount of state space searched, on average, is the main thing we hope from using a heuristic -- space complexity (how much space needed to do search) Heuristic search strategies differ from one another along a number of dimensions. Examples: Basic search strategy depth-first, breadth-first, best-first, or a mixture? Does it take account of cycles in the state space? Does it save paths to goals? Does it keep information so as to allow complete backtracking if needed? Or, does it irrevocably prune parts of the tree, so that it can never return to the pruned parts? Does it only ``look ahead'' toward the goal, or does it also consider how far it has come so far? Is the algorithm iterative, starting by looking at a small part of the state space, and then successively looking at larger parts of it? It is possible that a heuristic search does not, for a particular problem instance, search less of the state space than an exhaustive search would. In this case, is it more expensive to use the heuristic search algorithm? Yup -- *at least* evaluating the h-function, and choosing the node with the best h-function, take extra time. In a particular system, the search strategy may be a mixture of many different strategies. Experiment. When is it possible that you *not* find a solution? 1. obvious -- when there isn't one in the state space 2. when the search strategy irrevocably prunes the state space 3. if moves cannot be undone 4. if the state space is infinite Best-First Search (greedy search) ================================== A simple kind: assume the state space is a tree (not checking for duplicate nodes) don't save paths (just return a goal state) Recall depth-first search: (defun depth-first-search (start) (tree-search #'append (list start))) (defun tree-search (myfun open) (cond ((null open) 'fail) ((goalp (first open)) (first open)) (t (tree-search myfun (funcall myfun (successors (first open)) (rest open)))))) Let * be (first open). Suppose * has three kids: 8, 9, 10. Upon call: (* ...) After depth-first-search: (8, 9, 10, ...) After breadth-first search: (..., 8, 9, 10) Best first search? sort {...} U {8, 9, 10} according to h-function. That way, (first open) on the next recursive call will be the best node on open. (What's wrong with the name BEST-first-search?) Pseudo-code for the function argument: args: list1, list2 ~~Sort (append list1 list2) in non-decreasing order by h-function. See Picture: (A) (D B C) (B E C F) (E C H G F) (J I C H F G) May go depth-first for awhile, but if something more shallow turns out to be relatively better... For the lisp code, we will use the Lisp function ``sort'': (sort <2-place function>) Suppose that the 2-place function is ``f''. Then, sort returns a list, say ``l'', such that forall i,j such that 1 <= i < j <= length of l, f (ith element of l, jth element of l) is true (non-nil). I.e., if you take any 2 elements from ``l'' and feed them to ``f'' in the order in which they appear in ``l'', the result is non-nil. If f is <, the sort is in increasing order. If f is >, the sort is in decreasing order. What do we want? We have a list of states that are to be ordered by the h-function: (defun ordering-function (state1 state2) (< (h state1) (h state2))) SO, here's the code, without using tree-search: (defun best-fs (open) (cond ((null open) 'fail) ((goalp (first open)) (first open)) (t (best-fs (sort (append (rest open) (successors (first open))) #'ordering-function))))) Here's the code using tree-search and lambda functions: (defun best-first-search (start) (tree-search #'(lambda (list1 list2) (sort (append list1 list2) #'(lambda (state1 state2) (< (h state1) (h state2))))) (list start))) Improve efficiency of best-first-search? Assuming that open is ordered in increasing order by h-function: insert each successor into (rest open) in its proper place. ===================================================== A* search: ========== Order paths on open by function f: f(n) = h(n) + g(n) g(n): actual cost from start to n h(n): estimated distance from n to a goal Paths must be saved (see treeSearchPathsSaved.lisp) Using the g value gives the search a breadth-first flavor. Even if h continuously returns good values for states along a path, if no goal is reached, g will eventually dominate h and force backtracking back to a more shallow state. Let us assume that h is "admissible": it never over-estimates the cost to the nearest goal. The algorithm in the text, p. 97: saves paths, but searches the space as if it were a tree A* with an admissible heuristic is optimal ------------------------------------------- Let f* be the cost of the optimal solution path (optimal means least-cost path) A* expands all nodes with f values < f* (plus perhaps some with f values = f*) The first solution found must be optimal. ABWOC: First goal found, say x, is not optimal. Let: os be an optimal solution, so that f(os) = f* Note: since os and x are goals, and h(any goal) = 0, f* = f(os) = g(os) and f(x) = g(x). g(x) > g(os) (or else x would be optimal too) For x to be expanded before os, x must be before os on open. This cannot be, because f(x) > f(os) (since g(x) > g(os), and h(x) = h(os) = 0). (Another possibility: os has not yet been generated yet; but this cannot be either: the nodes along the path to os have lower f-values than the path to x, since h is admissible and the g-vals are lower, so the nodes on that path, and os, would be expanded.) A* with an admissible heuristic is complete -------------------------------------------- Suppose that there exists a goal, and there are cycles in the state space. Could A* get caught in an infinite loop? Nope: eventually, the g-vals of the nodes in the cycle will be larger values than the g-val of the goal node, so they will be placed after the goal node on open. Complexity of A* ----------------- The big problem is space. Just as breadth-first search adds levels, A* adds "f-contours" of nodes. Will expand all nodes with f-value i before expanding any nodes with f-value i+1. Space complexity is exponential. Example heuristics for the 8-puzzle =================================== A. The number of tiles out of place B. The sum of all the distances by which the tiles are out of place. C. N * the number of direct tile reversals (for some constant N) D. B + C Let h*(n) be the actual minimal cost from n to a goal (an oracle -- defines what we want h to be) A* is admissible if h(n) <= h*(n), for all n (i.e., if h never overestimates the distance to the goal). Which of A-D are admissible? (A, B) Consider h''(n) = 0. Just breadth-first search, which of course is guaranteed to find an optimal solution. Clearly, A and B are better than h''. For 2 admissible heuristic function h1 and h2, if h1(n) <= h2(n) for all n, h2 is more informed than h1 and...fewer nodes will be expanded, on average, with h2 than with h1. The larger the values the better (as long as still admissible). h(n) = 0; # tiles out of place; sum of distances of tiles out of place Extensions of A* ================ IDA* - Iterative Deepening A* search. Iterative deepening, where f values are used rather than depth values. Each iteration expands all nodes within a particular f-value range. In the worst case, IDA requires O(bd) storage, where b is the branching factor, and d is the length of the optimal solution path (assuming unit-cost operators). The number of iterations grows as the number of possible f values grows. Algorithm: p 107 The next f-limit will be the minimum one found that is greater than the current one. IDA* (start) f-limit <-- f(start) loop solution, f-limit <-- DFS-Contour (start, f-limit) if solution is non-nil, return solution if f-limit = infinity, return failure end loop DFS-Countour(node, f-limit) if f(node) > f-limit then return nil, f(node) if goalp(node) then return node, f-limit for each s in successors(node): solution, new-f <-- DFS-Countour(s,f-limit) if solution is non-nil, then return solution, f-limit next-f <-- min(next-f, new-f) return nil, next-f SMA* (Simplified Memory-Bounded A*): uses whatever memory is available to it, so avoids repeated states as far as its memory allows. Often better than IDA*. Hill-Climbing Search ==================== Depth-first search without backtracking, except that at each point you generate all successors and move to the best one. No backtracking means that you... throw away the children that were generated but not chosen. The name? Consider the states laid out on a landscape; the height of any point is the h value of the state at that point. Idea: move around the landscape trying to find the highest peaks (the optimal solutions). [See picture]: If you could see, you would go behind A in front of B climb D traverse the plateau and keep going. But it's foggy...you step in all four directions and take the step that increases altitude the most. Very *local*, but very cheap; sometimes, the best kind of solution to use. Algorithm: p. 112 In all of A,B,C, you get stuck: A. You will go up A and get stuck (no state is an improvement). foothill; local maximum. B. If you get to D you are stuck (all neighbors have the same h-values). Plateau. C. Not in picture: Ridge. There is a direction in which we would like to move, because it would get us closer to the goal, but none of the operators takes in that direction. You could oscillate from side to side, making little progress. ``Random-restart hill-climbing:'' a series of hill-climbing searches from randomly generated initial states. Could use a fixed number of iterations, or continue until the results from the searches have not improved for a certain number of iterations. ``Simulated Annealing:'' when you get stuck on a local max, allow some downhill steps to escape the local max. Beam Search =========== For problems with many solutions, it may be worthwhile to discard unpromising paths. Beam search: a best-first-search that keeps only a fixed number of states on open. Search as shining a light on the search space. A* -- light spreads as we go deeper. Beam search: a fixed-width beam. (defun best-fs (open) (cond ((null open) 'fail) ((goalp (first open)) (first open)) (t (best-fs (sort (append (rest open) (successors (first open))) #'(lambda (state1 state2) (< (h state1) (h state2)))))))) add a parameter -- the beam-width. truncate the sorted list to the beam-width. (defun best-fs (beam-width open) (cond ((null open) 'fail) ((goalp (first open)) (first open)) (t (best-fs beam-width (truncate beam-width (sort (append (rest open) (successors (first open))) #'(lambda (state1 state2) (< (h state1) (h state2))))))))) (defun truncate (width list) ;; Returns list truncated to width elements ;; e.g., (truncate 3 '(1 2 3 4 5 6 7)) returns '(1 2 3)) With a beam-width of infinity: best-first-search With a beam-width of 1: hill-climbing Yet another search technique: iterative widening (keep running the search with wider and wider beams)