También disponible en Español


The digital magazine of InfoVis.net

by Juan C. Dürsteler [message nº 189]

Reactable is an electronic musical instrument that combines the features of a modular synthesizer with a multi touch tangible user interface. This permits creating a real time cognitive experience where the information visualised depicts the sound pattern intended to be created allowing the user to blend visual, tactile and aural perception with cognition. 
Reactable2.gif (361064 bytes)
Reactable: two persons interacting with the synthesizer interface, producing and transforming sounds.
Source: Courtesy of S. Jordá Music Technology Group Universitat Pompeu Fabra.
Click on the image to enlarge it.

Reactable is a remarkable  electronic musical instrument designed to create music in a collaborative way by means of a tactile interface that is able to detect multiple touches of several users at a time. In a simplified way we could say that it's a classic modular synthesizer provided with a powerful and creative multi-touch tangible interface. 

More specifically it is a round table of translucent surface, that acts as the screen of a beamer that resides in its interior and projects onto it different shapes and wave patterns as a response to user interaction. The users control the table by placing different kinds of objects of diverse shapes and sound effects on its surface . Each object represents a component of a modular synthesizer.

The table is illuminated from beneath by infrared light whose reflection on the inner surface of the table is captured by a camera detecting the position of the objects, their distinctive identification and the situation and movement of the fingers of the user when touching the surface. This way the projected visual information and the information detected about objects and fingers go into separate spectral channels avoiding their interference.

According to Sergi Jordà head of this project elaborated by a team of digital lluthiers, in Reactable all that is relevant is shown and all that is shown is relevant for playing the instrument. This way, when we place on the table a "square wave" object a squared wave is depicted between the object and the center of the table with the exact properties of the wave the synthesizer is generating in real time. 

By changing the orientation and placement of the object we can modify its behaviour. For example the frequency of the sound it produces or the properties of its range of features if it does not produce a sound wave but modifies or filters the sound being produced by other objects. A luminous arc surrounding the object as soon as it touches the table indicates at which point of its frequency or performance range the output of the synthesizer is situated. A shiny dot we can capture with our finger enables us to increase or decrease the volume of the sound produced. All this is drawn dynamically in real time so that, for example, the waves vibrate and pulses are transmitted from one place to another in a realistic way. 

Onda_cuadrada.gif (178179 bytes) Reactable1.gif (244290 bytes)
Reactable: two snapshots of different configurations of objects. To the left we see a square wave generator showing the wave it is producing. To the right we appreciate flat and cubic objects along with the machine readable codes that allow the system to identify the functionalities of the same.
Source: Courtesy of S. Jordá Music Technology Group Universitat Pompeu Fabra.
Click on the image to enlarge it.

Reactable detects the position of our fingers when touching the surface of the table, allowing us to modify the "dial" of each objects, as we have seen already, but we can also draw the type of wave we want to obtain by just moving our finger close to the wave generator of our interest. We can temporarily stop the flux of sound by placing our finger on top of the drawing of the wave or permanently cutting a connexion by crossing it with our finger on top of it. 

Objetos.gif (385195 bytes)
Different types of Objects in Reactable. We can appreciate multi-functional cubes and different shapes of flat slices . The codification is apparent in the cubic objects.
Source: Courtesy of S. Jordá Music Technology Group Universitat Pompeu Fabra.
Click on the image to enlarge it.

The objects can be thick slices with different shapes in their periphery or just cubes. The shape of the slices can be squares, rounded squares, pentagons, etc. Each object wears a human readable icon on top of it and a machine readable code in its bottom, in contact with the table. Cubes have 5 different machine readable labels (the sixth has the human readable icon) and act as a storage of multiple functionalities since you can obtain different sound effects depending on the face in contact with the table.  

Some objects can be programmed just by placing a paper containing the corresponding code against the table near them, so that the object is recognised from this moment on with the functionality recently transferred,like for example a piano or any other instrument sample.

Explaining its many functions in detail would be quite tedious. The interested reader can read thearticle by Jordà, Geiger, Alonso and Kaltenbrunner. But in fact this is not necessary since Reactable's declared objective, among others, is being intuitive so that no instruction manual is needed. Let's see what it intends to be 

  • collaborative: several users playing simultaneously, locally or remotely

  • intuitive: zero manual, zero instructions

  • challenging and interesting from the sound generation standpoint

  • learnable y and masterable even by kids

  • suitable for novices (installations) and advanced electronic musicians (concerts)

The truth is that the experience of playing with this instrument is equally interesting and pleasing. It's quite easy to get interesting sounds and small melodies in the first session just by playing with the objects and the sounds they produce and transform. According to Jordà with a lot of practice you can obtain remarkable results.

Maybe for this reason the Icelandic singer Björk is using this instrument in the context of her "Volta" tour. It has already been shown at the Coachella 2007 festival from which you can see some videos in Youtube, a page that has contributed to a noticeable visibility to Reactable since some demonstrative videos were uploaded.

Björk using Reactable in two songs of her "Volta" tour. 
Source: As can be seen in the videos of the Coachella 2007 festival downloadable form Youtube.
Click on the image to enlarge it

Reactable is a notable example among a long list of projects of similar contents and objectives that use multitouch tangible interfaces for music generation.

It's very interesting the fact that here the information being visualised allows you to know the waves you are generating in real time and how the application of filters modifies it in a way where interaction and visualization take part in a real time loop while we hear the output of the visual patterns we create. All this forms a cognitive fabric where vision, audition and tact combine together into a unified experience of musical composition

The instrument was developed under the direction of Sergi Jordà. The "Interactive Sonic Systems" team is working in the Music Technology Group within the Audiovisual Institute at the Universitat Pompeu Fabra in Barcelona Spain.

Links of this issue:

http://mtg.upf.edu/   Musical Technology Group (UPF)
http://www.iua.upf.edu/~sergi   Sergi Jordà personal page
http://mtg.upf.edu/reactable?team   Reactable digital luthiers team
http://mtg.upf.edu/reactable/pdfs/reactable_tei2007.pdf   Article about Reactable
http://www.youtube.com/watch?v=aM8SXfB5NHg   Björk with Reactable. Youtube video
http://www.youtube.com/results?search_query=reactable   All Youtube videos related with Reactable
http://mtg.upf.edu/reactable/?related    Reactable related projects
http://www.iua.upf.edu   website of the Instituto Audiovisual of the UPF
http://www.upf.edu   Pompeu Fabra University in Barcelona
© Copyright InfoVis.net 2000-2014