InfoWiz: An Animated Voice Interactive Information System
The InfoWiz project is centered around the idea of putting a
interactive kiosk into the lobby of SRI.
People who have a few minutes to spend should be able to
learn something about SRI, enjoy themselves, and walk away
with a good feeling of having seen something interesting and unusual.
One of the design decisions of the project has been to use speech recognition
as the main form of user input to the system. Using a mouse to
navigate through scrollbars and web links might be a concept unfamiliar
with many visitors to SRI. Using a touchscreen is possible, but, hey,
let's aim high. If we are successful in building a natural
spoken language interface, this should contribute to the user's
overall impression of ease of use, advanced technology, and general
intelligence of the system. If we are not successful, we can always
put back a touch screen.
In order to encourage spoken interaction with the system, we
added an animated character, a cartoon wizard, who attempts
to engage the user in conversations about SRI. As Don Nielson (to
whom this system is dedicated) used to say, talking to a computer
without an animated persona who can speak back is like talking to a
wall. Initially, we did our own animations based on a popular
clip-art figure. In later incarnations of the
interface, we adopted Microsoft Agent's Merlin character to provide
the graphics for the system.
The InfoWiz Kiosk
Interactions with the Wizard
After an audio and visual welcome from the InfoWiz Wizard,
the user is presented with
a screen containing a web browser,
and the Wizard himself. The Wizard's job is to instruct, advise,
suggest and demonstrate; he will take you on tours of the InfoWiz
information space or the complete set of SRI's web pages, and he
can answer questions you might have along the way.
The Wizard is able to cross window boundaries, and interact with
any part of the display, flying and pointing with his magic wand
to various places on the screen.
All audio interactions with the system occur through a telephone next
to the kiosk display. Users can make natural spoken requests about
information presented on the current web page or about general topics
related to SRI.
The Wizard is capable of answering questions and
providing supplementary information (videos, etc.) about the
InfoSpace. When the user is looking at web pages outside of
it's own dowmain (the InfoSpace), the InfoWiz can scroll pages for you
or help you navigate links, but of course, does not have any
special supplementary information about these pages.
InfoWiz Kiosk Screen
Interactions with the Wizard
The biggest challenge of this project is to create a system that
appears intelligent, easy to use, and not overly constraining,
to an untrained, not necessarily technically aware person who
walks in off the street. A potential user only may have a few
minutes to spend with the system, so it must be intuitively and
immediately obvious what to do.
Our approach is as follows:
The system is implemented on top of the
Open Agent ArchitectureTM (OAATM)
, which provides the means for bringing
together the component technologies in a plug-and-play manner. The OAA
is an open and distributed system; components can be written in different
programming languages, and be distributed over multiple computers.
In addition, technologies can be added incrementally -- we can
start out with a simple natural language processing component, and then
as the system is used and user input analyzed in more detail, we can
replace the prototype natural language agent with our most capable
- Provide interaction styles that are familiar even to people who have
never seen Microsoft Windows or a web browser:
a telephone as an input device, and
a human character with whom to converse.
- Combine numerous state-of-the-art artificial intelligence
technologies in a flexible and incremental fashion. Allow the system
to be workable soon, but easily expandable in its capabilities.
- Find a way to allow easy updating of the content, vocabulary and
knowledge required by the system to appear intelligent.
A number of different technologies are integrated using the OAA to
create the InfoWiz Kiosk. The plug and play nature of OAA has allowed
us to experiment with various versions and vendors for the component
technologies. But the basic elements are, as pictured below,
animation graphics, speech recognition, natural language
interpretation, dialog management, text-to-speech, and then knowledge
about the InfoSpace.
Since these technologies are brought together by the
Open Agent Architecture,
it is probable that other OAA-enabled agents will be brought to bear on
this project. Existing agents which may be considered for this project
- Speech Recognition is provided by technology created in SRI's
STAR Laboratory and commercialized
by Nuance Communications, a spinoff company. We have also tried IBM's
ViaVoice product in it's stead.
- Natural Language processing is handled by a mixture of Nuance's NL api,
and SRI's DCG-NL parser. Other more powerful systems may be brought in
later as needed.
- Wizard animation is handled by
Under UNIX, we used DFKI's
PPP Animated Presentation Agent
, which provided both graphics and a
a powerful multimedia generation system based on planning technology.
We also used for a time an Java-based animation package called the
Gamelet toolkit written by Mark Tachi.
- We have written agents enabling us to incorporate either Netscape
or Internet Explorer as the content viewer. A previous incarnation of
the InfoWiz project made use of a modified variation of
NCSA's Mosaic version 2.6.
- Currently, for text output, we use text-to-speech provided as
part of Microsoft Agent. In previous versions,
the Wizard's voice was composed either of sequenced audio files played
by the PlayWav agent, OR text to speech generated by Entropic's TrueTalk
system. Combining recorded utterances limits what the Wizard can say, but
sounds much better. The system adapts dynamically to the current set of
agents connected to the network, using whichever output modalities are
- To author, test and manage the knowledge that the InfoWiz has
over the InfoSpace, we created our own knowledge management tools and
a dialog processor to help guide the animations and verbal responses
of the character.
- Small Vision Module (SVM) stereo camera and algorithms for
detecting whether people are near the kiosk.
Gemini Natural Language Understanding System,
our most robust NL parser for building
spoken language systems.
is an example of a project that first
started using the DCG-NL agent and later upgraded to Gemini as the domain
became better understood.
- Telephone control: ex: User: "I'm here to see Adam Cheyer", Wizard: "Let me call him for you..."
FASTUS: a system for extracting information from free text might be
applied to extracting knowledge from ever-changing webpages.
- Robots: SRI's family
of mobile robots have been integrated into the OAA (an OAA-coordinated
multiple-robot approach recently won the Office Navigation task of the
AAAI Robot Contest). How about: user: "I'm here to see Adam Cheyer",
Wizard: "Let me send someone to show you the way there..."
If you are interested in the evolution of this project, see the
InfoWiz Project Timeline.