Introduc on Previous Studies Experiment - AHRC Research Centre ...

Music and shape: preliminary findings 

from musicians’ and nonmusicians’ 

visualiza9ons of sound 

Mats Küssner 

Department of Music, King’s College London 

AHRC Research Centre for Musical Performance as Crea9ve Prac9ce

<strong>Introduc</strong>)on Previous Studies Experiment -‐ 

Methods 

Experiment -‐ 

Results 

Discussion Literature 

Why is visualised sound important? 

• It can tell us which sound characteris9cs are 

most salient to people and what strategies 

they use to represent them 

• Visualiza9on of sound can give an insight into 

how musicians and nonmusicians perceive 

and process sounds 

• The very act of visualising sound involves fine-‐ 

tuned motor movements that can tell us 

something about the specificity of musicians’ 

highly trained motor skills


Methods 

Visualiza9on of Sound 

• Walker studies (1978, 1981, 1985, 1987) 

• Frequency – 

ver9cal space 

• Waveform – 

paYern 

• Amplitude – 

size 

• Dura9on – 

horizontal space 


Results 

Discussion Literature


Methods 


Results 

Visualiza9on of Music 

• Tan & Kelly (2004): short musical 

composi9ons 



Methods 

What is missing? 

• A study that inves9gates 


Results 

– performers’ visual representa9ons of 

– basic characteris9cs of sound and music with 

– systema9c manipula9on of the s9muli, 

– free real-‐9me representa9ons of the sound, and 

– exact visualiza9on measurements. 

Here we go… 



Methods 

• S9muli 

– 18 sequences of sine tones varied in pitch, loudness and 

tempo (length between 5 and 15 seconds) 

– 2 musical examples: beginning of Chopin’s 6 th prelude 

(played by Cortot, 1926 and Argerich, 1975) 

• Drawing procedure 


Results 


– Wacom intuos4 L graphics tablet plus pen 

– Par9cipants could see their drawings on a screen in front 

of them at all 9mes 

– More pressure on tablet resulted in thicker line/dot 

– Data collec9on: x, y, and pressure values every ~45 ms


Methods 


Results 



Methods 

• Drawing condi9ons 


Results 


– Instruc9on ALWAYS: represent sound visually 

– “Performance”: par9cipants drew along with the 

sounds AS THEY WERE PLAYED 

– “Contempla9on”: par9cipants drew AFTER THE 

SOUNDS WERE PLAYED 

– Par9cipants were presented with the same full set 

of 20 s9muli in both the “performance” part and 

“contempla9on” part 

NB: Only results from the “performance” condi9on will be reported during this 

research report.


Methods 

• Procedure 


Results 

– Familiarisa9on with tablet and pen 

– Prac9ce trial: represen9ng sound visually 

– “Performance” part first (in 50% of the cases) 

– Ques9onnaire (demographic data, music 

educa9on, listening/prac9cing habits etc.) 

– “Contempla9on” part second (in 50% of the cases) 

– Feedback in form of short interview 



Methods 


Results 

Par9cipants 

• 73 (42 female, mean age: 28.51 years [SD=7.71]) 


• 32 nonmusicians (18 female, mean age: 28.34 [SD=7.11]) 

< Grade 2, stopped playing more than 6 years ago and never played 

longer than 2 years 

• 41 musicians (24 female, mean age: 28.63 [SD=8.23])


Methods 


Results 


Representa9on of Pitch (verbal report)


Methods 

Representa9on of Pitch (verbal report) 

• Musicians 

– Other: pressed down harder when pitch was going 

down and vice versa (ppt_11) 

• Nonmusicians 


Results 

– Mixed: height // the lower the note the bigger the 

weight on the pen (ppt_54) 

– Other: represented my feelings (ppt_19) 



Methods 


Results 


Representa9on of Loudness (verbal report)


Methods 

Representa9on of Loudness (verbal report) 

• Musicians 

– Mixed: 

• force // how wide the wiggles were (louder sound à� wider 

wiggles) (ppt_18) 

• thicker line // big circle (when sound was thick) for louder tones 

(ppt_30) 

– Other: 


Results 

• soC sounds: pressed down harder; when louder: went up and built 

up with the loudness, and then went down quickly when it got 

really, really soC (ppt_11) 

• up and down on screen; when conflic9ng with pitch, 

representa9on of loudness would win (ppt_34) 



Methods 

Representa9on of Loudness (verbal report) 

• Nonmusicians 

– Mixed: 

• more pressure when louder // bigger circles when louder (ppt_45) 

• intensity/thickness of drawing // height (ppt_57) 

• first rule: up and down, if clash with pitch, second rule: soCer/thicker lines for 

soCer/louder tones (ppt_60) 

• some9mes I used more/less pressure on pen (or even leC blank for very soC 

moments), other 9mes up and down (ppt_66) 

• bigger, larger circles and heavier dots // higher on screen (ppt_71) 

– Other: 

• spots for quieter parts (ppt_10) 

• increase height of waves (ppt_17) 

• size of shape (bigger shapes -‐-‐> louder) (ppt_24) 

• drew bigger and smaller shapes for louder and quieter tones, respec9vely (ppt_29) 

• thickness of line, achieved by (fast) movements up and down; larger movements when sound 

was louder (ppt_41) 

• bigger circles when louder (ppt_47) 

• high frequency oscilla9on (ppt_58) 


Results 



Methods 

“Mainstream” analysis 

• Focus on most common representa9on strategies 

à� height -‐ pitch 

à� thickness/pressure -‐ loudness 

• Equally sized subsamples of musicians (30) and 

nonmusicians (31) achieved by 

– Visual inspec9on of drawings 

– Musicians: at least grade 8 


Results 

– Musicians (pitch): at least 2 hours per week 

– Musicians (loudness): at least 4 hours per week 



Methods 


Results 

Consistency and Difficulty 


But how well did they really perform??


Methods 


Results 


Correla9on pitch – height 

Example drawing “performances” of sound_14 

(pitch: up-‐down, amplitude: constant) 

participant 42 participant 48 

(nonmusician) 

rho = 0.4278 (p < 0.001) 

(musician) 

rho = 0.9106 (p < 0.001)


Methods 


Results 



36 (out of 61) drawings of sound_14 (ordered with increasing correla9ons from top leu to 

boYom right corner)


Methods 


Results 



36 (out of 61) drawings of sound_14 – blue frame: musicians


Methods 


Results 


Mean correla9ons between pitch and height for all sound s9muli 

– musicians in blue, nonmusicians in green


Methods 


Results 


Correla9on loudness – thickness/pressure 

Example drawing “performances” of sound_31 

(pitch: up-‐down, amplitude: decreasing-‐increasing) 

participant 57 participant 27 

(nonmusician) 

rho = 0.0230 (p > 0.2 ) 

(musician) 

rho = 0.7896 (p < 0.001 )


Methods 


Results 


Mean correla9ons between loudness and thickness/pressure for 

all sound s9muli – musicians in blue, nonmusicians in green


Methods 


Results 

Summary of findings 


• Pitch representa9on: majority of musicians (98%) 

and nonmusicians (84%) used height 

• Loudness representa9on: majority of musicians 

(93%) and nonmusicians (72%) used thickness/ 

pressure 

• Overall, par9cipants were rela)vely consistent in 

applying these strategies, and they found the tasks 

fairly easy 

• However, musicians outperform nonmusicians for 

both pitch and loudness representa9on


Methods 


Results 

Conclusion 

At this stage, it is not clear yet what the cause for 

musicians’ superiority of shaping sounds is 

à� Are they beYer at perceiving the sounds? 


à� Or do musicians and nonmusicians perceive the 

sounds equally well, but musicians’ motor responses 

are more fine-‐tuned? 

à� If the laYer was true this would point to a non-‐ 

specificity of highly trained motor movements in 

musicians


Methods 


Results 


More analyses and experiments to come…


Methods 


Results 


Tan, S., & Kelly, M. (2004). Graphic representa9ons of short musical 

composi9ons. Psychology of Music, 32(2), 191-‐212. 

Walker, A. R. (1985). Mental Imagery and Musical Concepts: Some Evidence 

from the Congenitally Blind. Bulle9n of the Council for Research in Music 

Educa9on(85), 229-‐237. 

Walker, R. (1978). Percep9on and music nota9on. Psychology of Music, 6(1), 

21-‐46. 

Walker, R. (1981). The Presence of Internalized Images of Musical Sounds and 

Their Relevance to Music Educa9on. Bulle9n of the Council for Research in 

Music Educa9on, 107-‐111. 

Walker, R. (1987). The effects of culture, environment, age, and musical 

training on choices of visual metaphors for sound. Percep9on & 

Psychophysics, 42(5), 491-‐502.

Introduc on Previous Studies Experiment - AHRC Research Centre ...

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?