Multimodal Maps: An Agent-based Approach
The SRI Multimedia Interfaces Group is currently building a series of
prototype map-based applications that accept handwritten, verbal and gestural
requests issued in a synergistic fashion.
These applications are distinguished by comparatively rich natural language
capabilities, access to existing data sources including the World Wide
Web, and a mobile handheld interface. To implement the described application,
a hierarchical distributed network of heterogeneous software agents was
augmented by appropriate functionality for developing synergistic multimodal
- Multimodal Input: A user interacts with the system through
handwritten, gestural, vocal, and direct manipulation commands.
- Multimedial Output: Responses are provided by the system through
textual, graphical, audio and video media.
- Agent Interaction: User requests are processed by a distributed
community of software agents, which automate tasks such as natural
language processing, interaction with planners or simulators, and
accessing data from remote datasources (Prolog DBs, X.500 DBs, WWW, etc.)
- To see a demonstration (including video) of the multimodal map interface,
- To see a picture representing the agents used in the multimodal map
demonstration, click here.
- Multimodal Maps: An Agent-based Approach. With Luc Julia.
International Conference on Cooperative Multimodal Communication (CMC/95),
24-26 May 1995, Eindhoven, The Netherlands.
- A Multimodal Computer-augmented Interface for Distributed Applications.
With Luc Julia. HCII'95, July, 1995, Tokyo.
Created by Adam Cheyer,
19 February 1995