CHI 2008Workshop: Usable Artificial Intelligence

Attending

The workshop will be held on Saturday 5 April in room 210 of the Lorenese Pavilion within the Fortezza da Basso Center. Registration opens at 7:30 am, and there will be signs and student volunteers available to help you find you way. Please arrive early to be sure you have plenty of time to register and find the room.

Accepted Submissions

Please read all accepted postition statements prior to the workshop.


Schedule

9:00 - 9:15 Brief introduction and overview by the organizers
9:15 - 10:30 Discussion of themes presented by three participants
10:30 - 11:00 Morning break
11:00 - 11:30 Group discussion: map out the topic "Usable Artificial Intelligence" and focus setting
11:30 - 12:30 Breakout sessions in which smaller groups work out answers to questions (see descriptions below)
12:30 - 1:30 Lunch break
1:30 - 3:00 Continuation of breakout sessions
3:00 - 3:30 Afternoon break
3:30 - 5:00 Group discussion synthesizing the results of the breakout sections and working out concrete action items
   
Breakout Sessions
General descriptions, subject to revision.
Breakout 1 Identifying gaps in HCI for AI design theory

Goal: Develop a practical, bottom-up, list of HCI for AI design problems that can be used to drive future research

Description: What are the gaps in design theory (computational theory, evaluation theory, interaction design approaches) that would benefit from systematic research? Do AI systems need new design methods? Prototyping tools? How do we identify and fill these gaps?

Post Workshop: Generate list to help inspire research. Contribute to a published journal article on HCI for AI.

Breakout 2 Documenting HCI design patterns for AI systems

Goal: Document the current user interface / interaction design patterns that have been applied to AI applications.

Description: Given that designers are currently tackling problems in the HCI for AI design space, what are the systematic challenges of designing and evaluating AI systems that differ from non-AI systems? What design patterns have emerged (both from research and from practice) to resolve these challenges?

Post Workshop: This should lead to conference or journal publication, a website, and a possibly a CHI tutorial for 2009

Breakout 3 Evaluating the usability of AI systems

Goal: Document the current set of usability evaluation methods that have been applied to AI systems and identify gaps in these methods

Description: Given that designers are currently tackling problems in the HCI for AI design space, how do the existing usability evaluation methods hold up? Which ones work best for which kinds of applications / algorithms? Are there systematic gaps in evaluation methods? For example, what evaluation methods are appropriate for applications with user intent recognition and/or machine learning algorithms?

Breakout 4 Working toward a relevant journal

Goal: Provide a solid foundation for a proposed journal on interaction with intelligent systems

Description: On the basis of an initiative of senior members of the intelligent user interfaces community, a proposal for a journal on the topic area of this workshop is being prepared (by John Riedl and Anthony Jameson). The plan is to submit it to the Publications Board of ACM. An ambitious journal proposal requires input from representatives of all perspectives within the field. In this breakout session, we will collect ideas on questions like the following - starting for efficiency from an overview of the answers that have been worked out so far: How should the topic area to be covered by the journal be defined, and how can it be expressed in a journal title? What specific topics can be listed that fall within this topic area? In what communities, conference series, and journals can relevant contributions be found? How can the added value of the new journal relative to existing publication forums be explained? What specific examples of outstanding work in this area can be presented as examples of what we want to see published?

Post-Workshop: The results will flow directly into the journal proposal that will be submitted to ACM. Interested participants will be welcome to continue to contribute to the effort to establish the journal.

 

Maps
Fortezza da Basso Center
Lorenese Pavilion

Workshop Description

“The AI and HCI communities have often been characterized as having opposing views of how humans and computers should interact” observes Winograd in Shifting Viewpoints. Reconciling these views requires a thoughtful balancing of assistance and control, of mental and system representations, and of abstract process and contextualized workflow. This workshop examines the gap between HCI and artificial intelligence (AI), with the goal of improving usability of AI systems.

Developments in artificial intelligence hold the potential to assist humans at work, at home, and across society, but realizing this potential depends on designing human-usable AI systems. Most importantly, the AI system must directly fit a real need and workflow. More concretely, the user must have an understanding of the abilities of the system, of how to direct its operation, and how those operations support desired tasks. The system must have information that is formal and complete enough for its reasoning and learning and must allow this information to be applied and altered at times appropriate for the user. Finally, the user must be able to assimilate and respond to the output of the system and, possibly, to its internal state or processing.

We characterize the salient issues within the two themes of Knowledge Capture and Representation and End User Control. Specific topics of interest include, but are not limited to:

  • Semantic technologies
  • Knowledge acquisition and validation techniques
  • Learning systems
  • Intelligent assistants
  • Explanation and trust
  • Emotive and affective systems
  • Mixed initiative interaction
  • Interaction principles for AI
  • End-user knowledge representations and control methods
  • Case studies of deployed systems
  • Evaluation methodologies, metrics, and measures
  • Increasing collaboration between AI and HCI communities

Audience

The intended audience of this workshop consists of practitioners and researchers of human-computer interaction, artificial intelligence, knowledge capture, user-oriented design, and assistive and agent technologies. A key objective is to foster interdisciplinary interactions among the participants, in order to develop insight into the usability challenges in developing AI systems, and effective means of meeting these challenges.

Workshop Goals

The immediate and longer-term goals of the workshop, rather than to showcase specific technologies, are:

  • To understand processes and methods for the UI design of AI systems.
  • To develop insight into the usability challenges in developing AI systems, and effective means of meeting these challenges.
  • To collect a set of example approaches and solutions, working towards understanding underlying patterns and principles for designing and evaluating intelligent systems.
  • To consider how to increase awareness and appreciation of usability issues in the AI community.
  • To gain an understanding of the most promising areas for collaboration between the HCI and AI communities.

This workshop follows the success of the AAAI 2007 Spring Symposium, Interaction Challenges for Intelligent Assistants, a forum that brought together practitioners from HCI and AI fields to explore HCI issues from the perspective of artificial assistive agents.

Call For Participation

Potential participants are invited to submit a position paper, 2-4 pages in length. Position papers should either (i) situate the participants’ interests and background among the themes of the workshop, or (ii) report on case studies of AI systems from an HCI perspective.

Papers should conform to the CHI 2008 Extended Abstracts format (www.chi2008.org/formatting.html). To participate, at least one author is required to register for the workshop and one day of CHI 2008.

Submissions, in PDF format, should be sent to spaulding@ai.sri.com by 19 October, 2007.

Important Dates:
Submission: 19 October 2007 (5:00pm PDT)
Notification: 28 November 2007

Organizers:

Aaron Spaulding is a Computer Scientist and Interaction Designer at SRI’s Artificial Intelligence Center. His interests revolve around the intersection between design, human-computer interaction and artificial intelligence, with a focus on developing useable interfaces for AI systems that meet real user needs.

Jonathan Grudin is a Principal Researcher in Eric Horvitz's Adaptive Systems and Interaction Group at Microsoft Research. He was previously Professor of Information and Computer Science at University of California, Irvine, where he focused on studies of the adoption and use of collaboration support technologies. He worked in and with AI groups at Wang Laboratories and MCC in the 1980s. He writes and edits a column on HCI history for ACM Interactions magazine; the September 2006 column explored the historical relationship of AI and HCI.

Anthony Jameson is a principal researcher at DFKI, the German Research Center for Artificial Intelligence, and adjunct professor for human-computer interaction at the International University in Germany. He has been conducting research in the intersection of AI and HCI since the 1980s, playing a leading role in the communities of user modeling and intelligent user interfaces. His research most specifically relevant to this workshop, conducted since 2004 within Project Halo, concerns the design and evaluation of systems for knowledge capture by nontechnical users.

Neil Yorke-Smith is a Computer Scientist at SRI's Artificial Intelligence Center. His research interests center on applying artificial intelligence to help human decision makers in practical problems. Under the auspicious of the Association for the Advancement of Artificial Intelligence, Dr. Yorke-Smith chaired the 2007 Spring Symposium on Interaction Challenges for Intelligent Assistants.

Jack Zaientz is a Research Scientist and HCI Sector Lead for Soar Technology. His works focuses on the use of agent-based systems for information visualization of, and human interaction with, complex knowledge-rich environments including C4ISR, data fusion, and military modeling and simulation. His research draws on cognitive psychology interests in human internal, external and shared knowledge representations and computer science interests in intelligent user interfaces. Mr. Zaientz has twice given tutorials on HCI and Modeling and Simulation at I/ITSEC.


CHI 2008 home