TCD-CS-93-10 Smith B., Keane M.T.
The Proper Treatment of Case-Based PlanningJuly 1993.
One of the major obstacles to progress in Artificial Intelligence (AI)
is the absence of an adequate methodology; in particular, a methodology
for evaluation the fragmentary systems which make up much of the corpus
of AI research. This paper proposes a novel evaluation methodology
which is designed to assess planning systems relative to an ideal
system. The focus is on case-based planners because their diversity
presents a considerable challenge to this form of ideal
evauation. Two main priciples about the costs incurred in
case-based planning systems are proposed; one which deals with the cost
of knowledge acquisition and one with the cost of computation. We then
use these principles to define the ideal case-based planning system and
develop precise metrics that can be applied to existing systems in order
to determine how far they are from this ideal. These metrics are
applied to two case-based planners in the literature in order to
illustrate their ease of application. Finally, the importance and
implications of this type of evaluation for case-based planning, and AI
in general are considered.
Technical Reports Home Page