Slide 66 of 78
The subject’s screen (on the left) is setup to encourage pure pen and voice input - all scrollbars, dialog boxes, tool and button bars, and menus have been hidden, leaving maximal screen space for the interactive display.
The Wizard arranges the application with all direct manipulation components visible and easily accessible. When the subject issues a request using pen and voice, the Wizard will try to achieve the request as efficiently and quickly as possible, using any combination of GUI and/or multimodal input.
Input from both participants in the experiment are of considerable interest, and are recorded. From the subject, we learn a great deal about what functionality a user would want from the system and how users would interact with the system in the most natural way possible. From the Wizard, we learn how well the system’s current functionality matches the desires of the subject, how well the automated algorithms perform on the Wizard’s input, and how the Wizard (under duress) chooses to make use of multimodal vs. direct manipulation input.