Workshops - UbiComp

vise. Beginning improvisers typically get stuck on ‘noodling’around individual chords from moment to moment andare unable to interact meaningfully with the strategic, longerterm harmonic elements, for example, chord progressionsand modulations, which are typically essential to higherlevelstructure in western tonal music, including jazz andmuch popular music.Harmony Space draws on cognitive theories of harmonicperception, providing consistent uniform spatial metaphorsfor virtually all harmonic phenomena, which can be translatedinto spatial phenomena such as trajectories, whoselength, direction and target all encode important information.Thus, Harmony Space enables numerous harmonic relationshipsto be re-represented in a way that may be morecognitively tractable.We will use the Harmony Space representation to providemusicians with a tactile representation of the harmonic relationshipsof music they are currently playing. This will beachieved by having the musicians wear a vest with a 6x8 arrayof tactile actuators where each actuator will represent anote that the musician is playing. The notes will be identi-d directly in the case of electronic instruments, or sensedusing microphones and pitch trackers in the case of acoustic(monophonic) instruments. We predict that representingpitch movement in this way will facilitate the developmentof a spatial understanding of musical relationships, whichwill transfer to improved performance in a wide variety ofmusical tasks, including improvisation. We will investigatewhether performance is linked to the interface becomingtransparent.CONCLUSIONThe E-Sense project is taking an interdisciplinary approachto investigating the extended mind, in particular the natureof sensory augmentation. We will use a rapid prototypingapproach to build 3 novel tactile interfaces that mediate differentsensory modalities (ultrasound, pitch and ‘virtual’ location).As well as testing the practical utility of these systems,we hope to gain more insight into the conditions underwhich technologies become transparent as well as gathermore evidence for the theoretical viability of the sensorimotorcontingency model.ACKNOWLEDGEMENTSThis research is supported by the Arts and Humanities ResearchCouncil grant number: AH/F011881/1.REFERENCES1. Bach-y-Rita, P. Brain Mechanisms in SensorySubstitution. Academic Press, NY, 1972.2. Arduino electronics prototyping platformhttp://www.arduino.cc/Retrieved June 20083. Bird, J., d’Inverno, M. and Prophet, J. Net Work: AnInteractive Artwork Designed Using anInterdisciplinary Performative Approach. DigitalCreativity, 18 (1), (2007), 11–23.4. Bird, J., Stokes, D., Husbands, P., Brown, P. and Bigge,B. Towards Autonomous Artworks. LeonardoElectronic Almanac, (forthcoming).5. Clark, A. Natural-Born Cyborgs: Minds, Technologiesand the Future of Human Intelligence. OxfordUniversity Press, NY, 2003.6. Heidegger, M. Being and Time. Harper and Row, NY,1962.7. Ho, C., Tan, H. Z. and Spence, C. Using spatialvibrotactile cues to direct visual attention in drivingscenes. Transportation Research Part F: TrafPsychology and Behaviour, 8, (2005), 397–412.8. Holland, S. l Intelligence, Education andMusic. PhD thesis, IET, The Open University, MiltonKeynes. Published as CITE report No. 88, 1989.9. Holland, S. Learning about harmony with HarmonySpace: An overview. In M. Smith and G. Wiggins,(Eds.) Music Education: An l IntelligenceApproach. Springer Verlag, London, 1994.10. Kohler, I. The formation and transformation of theperceptual world. Psychological Issues, 3, (1964),1-173.11. LilyPad sewable electronic componentshttp://www.cs.colorado.edu/ buechley/LilyPad/index.htmlRetrieved June 200812. Maravita, A. and Iriki, A. Tools for the Body (Schema)Trends in Cognitive Sciences, 8, (2004), 79–86.13. O’Regan, J. K. and Noë, A. A Sensorimotor Account ofVision and Visual Consciousness Behavioral and BrainSciences, 24(5), (2001), 939-73.14. Processing programming language and environmenthttp://www.processing.org/Retrieved June 200815. Ramachandran, V. S. and Blakeslee, S. Phantoms in theBrain: Probing the Mysteries of the Human Mind.Fourth Estate, London, 1998.16. Robinson, D. and Campbell, R. Contributory Factors toRoad Accidents Transport Statistics: Road Safety,Department for Transport, 2005. http://www.dft.gov.uk/Retrieved June 200817. Rogers, Y. and Muller, H. A Framework for DesigningSensor-Based Interactions to Promote Exploration and International Journal of Human-ComputerStudies, 64 (1) (2005), 1–15.18. Rogers, Y., Scaife, M. and Rizzo, A. Interdisciplinarity:an Emergent or Engineered Process? In S. Derry, C. D.Schunn and M. A. Gernsbacher (Eds.) InterdisciplinaryCollaboration: An Emerging Cognitive Science. LEA,(2005), 265–286.19. Stratton, G. M. Some preliminary experiments ofvision without inversion of the retinal image.Psychological Review, 3, (1896), 611-617.5