Search |  Contact |  SRI Home Do not follow this link, or your host will be blocked from this site. This is a spider trap. Do not follow this link, or your host will be blocked from this site. This is a spider trap. Do not follow this link, or your host will be blocked from this site. This is a spider trap.A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A A ASRI International.  333 Ravenswood Avenue.  Menlo Park, CA 94025-3493. SRI International is a nonprofit corporation.

Publication Details

Halo Pilot Project

by Barker, K. and Chaudhri V. and Clark, P. and Israel, D. and Porter, B. and Romero, P.

Technical Report
Institution: SRI International
May 2003.


The Halo Pilot Project is the first phase of a projected multi-phase effort by Vulcan Inc. whose ultimate goal is the creation of a “digital Aristotle”, an expert tutor in a wide variety of subjects. The Halo Pilot was a six-month effort intended to assess the state-of-the-art in question answering, with an emphasis on deep reasoning. The program was structured around the challenge of responding to variants of AP Chemistry questions that focused on a portion of the AP Chemistry syllabus. Over a four-month period, we developed a question-answering system for Halo based upon SHAKEN, a Knowledge Representation and Reasoning system that the SRI team has developed in a series of extremely productive collaborations on DARPA-funded projects. This involved encoding the corpus in the formalism of SHAKEN (KM), developing problem solving and reasoning methods to answer the questions and creating a new answer explanation facility for producing concise and understandable explanations of the use of those methods to answer the questions. The system we developed was sequestered while we spent an additional two weeks encoding 150 mostly novel, that is, previously unseen chemistry questions. These questions were then processed on the sequestered system. . Vulcan’s methodology emphasized the following elements of such a challenge: (i) coverage, the system’s ability to answer novel questions from the specified syllabus; (ii) question encoding, the ability to robustly create high-fidelity translations of the questions into SHAKEN, and (iii) explanation generation, the ability to provide concise and coherent explanations for the answers produced. In addition, in order to help direct future research in this area, the Halo participants developed ”brittleness" taxonomy of failure modes of KRR systems and each system’s failures were classified into cells of that taxonomy. Two features, in particular, of the Halo Pilot were especially challenging: competence in chemistry, as in many of the physical sciences, depends crucially on the ability to reason with a large number of laws, at somewhat different levels of generality, combined in the right way and in the right order. Domains treated in previous projects have not had this character. Second, the ability of a system to produce coherent and concise explanations of its reasoning was central to the Halo Pilot, as it would be to the trustworthiness and hence utility of any complex software system.

Electronic Copies




Associated Projects

Halo Pilot

A System for Representing Textbook Knowledge

The goal of the project is to demonstrate the current state-of-the-art in knowledge representation by attempting to answer the questions in an advance placement test in chemistry.

AIC Personnel

Name Title E-mail
Barker, Ken Research Associate, Univ. of Texas-Austin
Chaudhri, Vinay K Program Director
Clark, Peter E. Allen Institute of Artificial Intelligence
Israel, David J Principal Scientist
Porter, Bruce Professor, University of Texas at Austin
Romero, Pedro R Alumnus

SRI International
©2019 SRI International 333 Ravenswood Avenue, Menlo Park, CA 94025-3493
SRI International is an independent, nonprofit corporation. Privacy policy