D20:
Interaction with Multifaceted Display Devices





D20 is an early exploration of the use of flexible displays for the design of new interactive devices. Flexible display technologies potentially allow for the making of displays with arbitrary pixel order and 2D shapes, e.g., triangles or circles. Such displays can be assembled into many shapes, forming continuous curved or convex 3D display surfaces. The current interaction design and user experiences are limited by the WIMP interface paradigm, designed specifically for flat rectilinear displays. We are interested in the potential of displays that have complex shapes. In the future any shape around us or any object might have an inexpensive and high-quality display wrapped around it. The main goal of this project was to investigate handheld devices with multifaceted displays. D20 is a device that has a regular icosahedron shape. The entire surface of the device forms one continuous multisided display. The entire interaction is conducted by manipulating, touching and shaking the display device.


D20 Interaction principles


Selection with Bubble Metaphor

The interaction with D20 is based on an air bubble metaphor. Imagine a glass ball filled with water with a small bubble of air inside. As we rotate the ball the air bubble moves to the top. The user can select an icon mapped around the device by aligning the bubble with the icon. Selection is then confirmed by touching the icon or shaking the device.


Gstalt Principles

We can effortlessly recognize forms and images from individual parts, such as lines,curves and dots. Gestalt psychologists proposed a number of principles to explain how we perceive forms. For example, when presented with ambiguous visual stimuli we are able to see two or more interpretations by switching back and forth between them. In Figure C, one can perceive the image of two faces or a vase. This principle is called multistability. By slightly altering the image we can make one of the interpretations dominant, i.e. if we decrease the distance between faces, then the vase is more likely to be seen. We took advantage of such in the design of the multifaceted displays shape. The D20 is an icosahedron display, on which different face groupings can be used as containers for user interface elements.


Interaction Containers

On the D20, a user can see several shapes at the same time and switch between them as we do in the case of the previous image of the vase illustrating multistability. Each of these shapes, or sets of facets, is used as an interface containers, and mapped to gestural interaction.


Menu

We implemented a number of basic interaction techniques. This figure demonstrates a hierarchical menu navigation: the strip shape is used to contain menu items. We scroll through the menu by rotating the device. Sicne the device is curved we can have an infinite number of the menu items. When the desired icon is selected the submenu appears on an orthogonal strip intersecting the selected icon. To indicate the transition, the entire strip flashes once before displaying the new submenu (Middle Figure). To return back to the top-level menu we simply shake the device.


Descrete Menu

This figure shows how a pentagon shape is used to implement a pie menu. The illuminated face represents the current selection, the user rotates the device around the vertical axis to switches between items. We illuminate the corners and the center of the pentagon to help the user view the pie menu clearly.


Continuous Selection

This figure shows continuous control using a pentagon. The size of the illuminated area corresponds to the value we are controlling. We change the value by rotating the device around the vertical axis, which decreases or increases the controlled value depending on the direction of rotation.


Continuous Selection

Finally, this figure shows continuous control using a rhomboid. The zero value is presented as a line in the middle, by tilting the device to the left or to the right we can increase or decrease the controlled value.


Publications


D20: interaction with multifaceted display devices.
I. Poupyrev, H. Newton-Dunn and O. Bau. 2006. In CHI '06 EA. ACM.[ACM]

Information input/output device, information processing method and computer program.
H. Newton-Dunn, I. Poupyrev, and O. Bau. 2006. (pending)

D20 : interaction avec un écran multifacette. [pdf]
O. Bau. Rapports de stages de Magistère 2ième année, Université Joseph Fourier, Grenoble 1. September 2005.


Team and Credits


This work was conducted in 2006 while at Sony CSL in Tokyo, Japan with Ivan Poupyrev, Sony CSL, and Henry Newton-Dunn, Sony SET Design Studio.