music 2

Psychology of Music http://pom.sagepub.com/

How music touches: Musical parameters and listeners' audio-tactile metaphorical mappings Zohar Eitan and Inbar Rothschild Psychology of Music 2011 39: 449 originally published online 8 November 2010 DOI: 10.1177/0305735610377592 The online version of this article can be found at: http://pom.sagepub.com/content/39/4/449

Published by: http://www.sagepublications.com

On behalf of:

Society for Education, Music and Psychology Research

Additional services and information for Psychology of Music can be found at: Email Alerts: http://pom.sagepub.com/cgi/alerts Subscriptions: http://pom.sagepub.com/subscriptions Reprints: http://www.sagepub.com/journalsReprints.nav Permissions: http://www.sagepub.com/journalsPermissions.nav Citations: http://pom.sagepub.com/content/39/4/449.refs.html

>> Version of Record - Oct 24, 2011 OnlineFirst Version of Record - Nov 8, 2010 What is This?

Downloaded from pom.sagepub.com at UNIV AUTONOMA DE NUEVO LEON on August 20, 2012

Article

How music touches: Musical parameters and listeners’ audio-tactile metaphorical mappings

Psychology of Music 39(4) 449–467 © The Author(s) 2010 Reprints and permission: http://www. sagepub.co.uk/journalsPermission.nav DOI: 10.1177/0305735610377592 pom.sagepub.com

Zohar Eitan and Inbar Rothschild School of Music, Tel Aviv University, Israel

Abstract Though the relationship of touch and sound is central to music performance, and audio-tactile metaphors are pertinent to musical discourse, few empirical studies have investigated systematically how musical parameters such as pitch height, loudness, timbre and their interactions affect auditory–tactile metaphorical mappings. In this study, 40 participants (20 musically trained) rated the appropriateness of six dichotomous tactile metaphors (sharp–blunt, smooth–rough, soft–hard, light–heavy, warm–cold and wet–dry) to 20 sounds varying in pitch height, loudness, instrumental timbre (violin vs. flute) and vibrato. Results (repeated measures MANOVA) suggest that tactile metaphors are strongly associated with all musical variables examined. For instance, higher pitches were rated as significantly sharper, rougher, harder, colder, drier and lighter than lower pitches. We consider several complementary accounts of the findings: psychophysical analogies between tactile and auditory sensory processing; experiential analogies, based on correlations between tactile and auditory qualities of sound sources in daily experience; and analogies based on abstract semantic dimensions, particularly potency and activity.

Keywords cross-modal interaction, haptic, loudness, metaphor, music performance, tactile

Background For most performing musicians, there is an immediate, embodied connection between tactile and auditory qualities: touch produces sound, while tactile and haptic information serve, together with audition and vision, as feedback gauging the performed outcome (Rovan & Hayward, 2000). These intimate inter-modal relationships are often expressed in the terminology used to describe sound: musical sounds are commonly referred to as ‘warm’, ‘soft’, ‘sharp’

Corresponding author: Zohar Eitan, School of Music, Tel Aviv University, Tel Aviv 69978, Israel. [email: [email protected]]


450

Psychology of Music 39(4)

or ‘rough’, mappings that seem to be applied with notable consistency that ‘strongly suggests a connection more than associative and external between tone and tactile values’ (Gunther & O’Modhrain, 2003, after Mursell, 1937). Recent psychophysical and neurophysiological research indeed suggests that the relationships of ‘tone and tactile values’ may be deeply rooted in behavior and its related cortical processes. Psychophysical studies indicate that concurrent vibro-tactile stimuli facilitate hearing (e.g., Schürmann, Caetano, Hlushchuk, Jousmäki, & Hari, 2006), while auditory stimuli may change concurrent tactile perception (Guest, Catmur, Lloyd, & Spence, 2002). Correspondingly, Schroeder et al. (2001), Schürmann et al. (2006), and Hlushchuk (2007) all demonstrate that tactile input is processed in the auditory cortex (posterior auditory belt area). Foxe at al. (2002) show auditory–tactile interaction in the left superior temporal gyrus, such that the responses to auditory–tactile stimulus pairs were stronger than the sum of responses to the unimodal stimuli presented alone, suggesting combined processing of the two modalities; and Hötting, Rösler, and Röder (2003), in an event-related potential (ERP) study, demonstrate that attending to auditory stimuli affects early (50–170 ms) brain processing of tactile stimuli located in the same position and vice versa. Little is confidently known, however, about how basic auditory qualities such as pitch height, loudness or timbre affect listeners’ audio-tactile mappings. In contrast with the wealth of psychophysical and cognitive studies examining interactions of auditory parameters with visual or spatial dimensions such as brightness, size or elevation (for reviews of recent research, see Eitan & Granot, 2006; Marks, 2004), few empirical studies have investigated audio-tactile mappings directly, and results are often inconclusive. Walker and Smith (1984, 1986) applied both adjective ratings and the Stroop paradigm to examine the interaction of low- (50 Hz) and highpitched (5500 Hz) sinusoids with antonymous cross-modal metaphors. While many visual or kinesthetic adjectives were strongly associated with pitch height, tactile antonyms, including rough–smooth, cool–warm, and hard–soft, were weakly associated with high and low pitch. Eitan and Timmers (2010) examined similar cross-modal metaphors in a musical context, asking participants to rate how appropriate they are to musical segments differing in pitch register. Unlike Walker and Smith’s, their results demonstrate highly significant correlations between touch-related adjective and pitch: high register music was rated as lighter, smoother and softer than low register music; heat, though, was not significantly related to pitch height. Recently, Eitan, Katz, and Shen (2010) systematically manipulated (using factorial design) pitch height, loudness and tempo in two musical phrases from Varese’s Density 21.5 for flute solo, and asked children (aged 8 and 11) and adults to rate how appropriate 15 metaphor antonyms are, including smooth–rough, sharp–round and light–heavy, to each manipulated phrase. Results indicate that higher pitch is significantly associated with roughness, sharpness and lightness, and increased loudness with roughness, sharpness, and heaviness. Several psychophysical studies, with conflicting results, have examined audio-tactile roughness perception. Peeva, Baird, Izmirli and Blevins (2004) asked subjects to match loudness and pitch levels to a given roughness and vice versa. Subjects associated louder sounds with rougher textures; they also showed strong correlation between pitch and roughness, though the direction of the correlation (higher–smoother or higher–rougher) varied among subjects. Guest et al. (2002) and Zampini, Guest and Spence (2003) show that reducing loudness and attenuating high frequencies increases perceived tactile smoothness. In contrast, subjects in Jousmäki and Hari (1998) judged tactile smoothness to increase as loudness and frequency increased. Finally, though not using actual musical or auditory stimuli, results of Osgood’s well-known Semantic Differential experiments may be particularly relevant to the issue of tactile-auditory


451

Eitan and Rothschild

metaphorical mapping. In the Semantic Differential (SD) technique (Osgood, Suci, & Tannenbaum, 1957) diverse objects (e.g., my mother, snow, death, United Nations) are rated on a large number of bipolar adjective scales (e.g., good–bad, fast–slow, hard–soft). Typically, factor analysis on the different adjective ratings reduces into three main independent factors, interpreted as evaluation (e.g., good–bad, pleasant–unpleasant) activity (e.g., fast–slow, active– passive) and potency or power (e.g., strong–weak, big–small). These three factors have emerged in different experimental paradigms, performed cross-linguistically and cross-culturally. The 50 adjective scales used by Osgood in many early SD experiments include the antonyms loud–soft and bass–treble, denoting the poles of pitch height and loudness. In addition, several tactile antonyms are used, including hard–soft, heavy–light, rough–smooth, sharp–blunt, hot– cold, and wet–dry. Notably, for American English speakers (Osgood et al., 1957, Table 1) both auditory antonyms loud–soft (.44) and bass–treble (.47) feature relatively high loadings into the potency factor. So do several of the tactile antonyms: the tactile dimension heavy–light presents the highest loading of all adjective pairs constituting the potency factor (.62), while hard–soft (.55) and rough–smooth (.36) are also loaded highly into this factor. Later analysis (Osgood, 1964) suggests that sharp–dull (.45) and hot–cold (.47) are also strongly associated with potency (though they also relate to activity). Thus, the tactile qualities of heavy, hard, rough, sharp and hot, and the auditory qualities low (for pitch) and loud are all associated with qualities denoting high potency and power. This semantic association of tactile and auditory qualities should be noted – and we will look at it again when interpreting our own results.

Predictions? What do the existing empirical data suggest regarding audio-tactile metaphorical mappings? Table 1 summarizes the tactile mappings of pitch height and loudness, as suggested by the studies surveyed above. As the table indicates, only few of the predictions are unequivocal: louder sound is sharper, heavier, harder and warmer, while higher pitch is lighter. Furthermore, of these correlations, two (loud→warm and loud→hard) are not based on studies involving actual auditory or tactile stimuli, but are implied indirectly from semantic differential data (see above). Predictions concerning other audio-tactile relationships are contradictory or simply missing. Louder and higher sounds were associated, in different studies, with both poles of the smooth– rough dimension, and higher pitch was likewise associated with both ‘sharp’ and ‘dull’. Mappings of pitch into the soft–hard and hot–cold dimensions are indicated by some studies,

Table 1. Summary of audio-tactile mappings, as suggested by Eitan & Timmers, 2010 (E & T); Guest et al., 2002 (G); Jousmäki & Hari, 1998 (J & H); Eitan et al., 2010 (E, K, & S); Osgood, 1964 (O), Osgood et al., 1957; Peeva et al., 2004 (P); Walker & Smith, 1984 (W & S), Zampini et al., 2003 (Z).

Soft–Hard Smooth–Rough Sharp–Dull Heavy–Light Warm–Cold Wet–Dry

Louder volume

Higher pitch

Hard (O) Smooth (J & H) Rough (E, K, & S; P; G; Z, O) Sharp (E, K, & S; O) Heavy (E, K, & S; O) Warm (O) ?

Soft (E & T, O), None (W & S) Smooth (E & T; Z; G; O), Rough (E, K, & S; J & H), None (W & S) Sharp (E, K, & S) Dull (O) Light (E & T, O) Cold (O), None (W & S) ?


452


but others do not suggest any significant correlations between these dimensions; and no study known to us has examined how pitch and loudness map into the wet–dry dichotomy. As to other musical variables, such as vibrato and specific instrumental timbre – both investigated in this study – as far as we know, only anecdotal evidence (e.g., expressions such as ‘warm vibrato’, or ‘smooth flute sound’, sometimes uttered by musicians) can serve as a basis for predictions.

Aims and general design This study systematically investigates how loudness, pitch height, instrumental timbre (flute vs. violin), vibrato and their interactions affect listeners’ application of tactile metaphors for musical sound. As shown above, the small body of empirical research addressing comparable issues is inconclusive with regard to some audio-tactile interactions, while others were never examined empirically. Several critical gaps in this literature thus need to be addressed. First, the effects of musical or auditory variables, except pitch, on audio-tactile associations have hardly been examined. Second, no study has systematically examined how interactions of different auditory parameters affect the tactile associations of sound. Lastly, previous studies did not examine listeners’ tactile mapping of musical sound per se: independent variables were either verbal only (Eitan & Timmers, 2010, Experiment 1; Osgood et al., 1957), sinusoids (Walker & Smith, 1984), noise (Peeva et al., 2004) or, on the other hand, entire musical complexes (Eitan & Timmers, 2010, Experiment 3; Eitan et al., 2010). The present study thus provides a necessary intermediate level in the range of converging experiments examining audio-tactile mappings. It bridges the gap between experiments using controlled, artificial auditory stimuli with no similarity to actual musical sound and experiments applying actual music (which necessarily involves many uncontrolled musical variables) by using ‘natural’ sound of musical instruments, played by professional performers, while controlling musical variables. Comparing our results with those of experiments using other stimuli – artificial or ecological – may provide a fuller picture of audio-tactile mappings in musical contexts. As independent variables we have chosen four musical features: the basic auditory parameters of pitch height and loudness; instrumental timbre, as represented by two instruments (violin and flute) similar in their pitch range but contrasting in timbre (the violin’s harmonics-rich sound contrasts with the flute’s, which, particularly at its higher register, approaches pure tone); and vibrato, an important tool in musicians’ (particularly string players’) expressive manipulation of sound. As dependent variables we use ratings for six tactile antonyms: sharp–blunt, smooth–rough, soft–hard, light–heavy, warm–cold1 and wet–dry. These terms are commonly used to describe sound, in music and elsewhere. ‘Sharp shrill’, ‘rough, hoarse voice’, ‘a blunt thud’, or ‘a still and soft voice’ (Kings I, 19, King James translation) are but a tiny sample of the expressions applying such audio-tactile mappings. Google, for instance (accessed 29 September 2009), lists about 253,000 instances of the term ‘soft sound’ or ‘soft sounds’, 205,800 of ‘warm sound/sounds’, 183,200 ‘heavy sound/ sounds’, 159,300 ‘rough sound/sounds’, 92,300 ‘sharp sound/sounds’ and 91,500 ‘dry sound/sounds’. Furthermore, as the above survey indicates, these tactile terms have been used in most relevant research; their employment here may thus enable comparison with that research. The present study examines, then, how several musical variables and their interactions affect the application of common tactile metaphors for sound, while using actual musical sounds, systematically manipulated. The study employs 18 sound stimuli, comprising all combinations of three pitch registers, three loudness levels and two instrumental timbres (violin


453


and flute). Two additional sounds introduced vibrato, using medium pitch and loudness levels. Participants (musicians and non-musicians) rated each sound on six bipolar adjective scales: sharp–blunt, smooth–rough, soft–hard, light–heavy, warm–cold and wet–dry. Using repeated measures MANOVAs, we examined how the musical variables related to each of the six adjective ratings.

Method Participants Forty participants, 21 women, 19 men; mean age: 28.85, range 18–60, SD = 10.77. 20 participants were musically trained (undergraduate or graduate music students and music professionals, with >4 years of formal musical studies; 15.2 years of musical training or professional musical activity, on average), while the remaining 20 had little or no formal musical training (1.1 years of musical training, on average). The musicians’ group was, on average, younger than the non-musicians’ (mean age 25 vs. 32.7). Participants were paid for their services.

Sound stimuli Twenty 4-second sounds, each consisting of a single tone with constant loudness, were played by a professional violinist and a professional flute player (10 sounds each), and recorded through a single Schoephs CMC5-U condenser microphone to a CD. Eighteen sounds, played without vibrato, created a 3 × 3 × 2 matrix, containing all combinations of two instrumental timbres (flute and violin), three pitches (A4, A5, A6) and three distinct loudness levels; two additional sounds (A5, medium loudness) were played with vibrato by the two instruments. Sound editing (using Wavelab 5.1) included equalizing the duration of all sounds to 4 minutes, introducing onset and decay gradations (<200 ms), and equalization of each of the three loudness levels across pitch and instrument, based on evaluations of four expert musicians. The peak amplitudes of all sounds used (in dB-SPL) are presented in Table 2.2

Procedure Participants were tested individually in a quiet room, using David Clark 10S/DC sound-isolating earphones. They listened to each sound two to three times, as needed, with intervals of approximately 10 seconds between reiterations, and 30 seconds between different sounds. A break of 2 minutes was introduced in mid-session. Twenty different quasi-randomized orderings of the stimuli were used, each starting with a different tone. Participants rated each sound on six bi-polar 5-degree scales: sharp–blunt, smooth–rough, soft–hard, light–heavy, warm–cold and wet–dry. Scales were presented as horizontal lines between each two dichotomous adjectives, divided by five short vertical lines (e.g., Wet |____|____|____|____| Dry). Participants circled one of the vertical lines to indicate which antonym was more appropriate as a metaphor for the sound, and to what degree; ratings were converted into numerical scales (1–5). The order of antonyms within each pair (right/left of the scale) was counterbalanced among participants, and the order of the six pairs was quasirandomized, such that for each six or seven participants, a different adjective pair appeared first on the form. Orderings were kept constant for each participant.


454


Table 2. Peak amplitudes of all stimuli (dB-SPL). Instrument

Dynamics

Pitch

Vibrato

dB-SPL

Violin Violin Violin Violin Violin Violin Violin Violin Violin Violin Flute Flute Flute Flute Flute Flute Flute Flute Flute Flute

pp pp pp mf mf mf ff ff ff mf pp pp pp mf mf mf ff ff ff mf

A4 A5 A6 A4 A5 A6 A4 A5 A6 A5 A4 A5 A6 A4 A5 A6 A4 A5 A6 A5

N N N N N N N N N Y N N N N N N N N N Y

63.1 59.1 62.2 70.2 72.5 69.8 79.5 79.8 81 77.8 66.5 72.5 69.6 76.2 78 82 83.5 83.8 90 84.5

At a session’s end, participants were requested to freely comment on their rating criteria, the relative difficulty of the categories, and any other issue concerning the experiment. They were also asked to select of the 12 adjectives the three most appropriate metaphors for musical sound. Participants provided demographic information, including age, gender, musical instruments played, years of musical training, and current musical occupation (if any).

Results The effects of musical variables on ratings of tactile metaphors Statistical analysis. Since the dependent variables have shown considerable co-variation (see ‘Correlations between adjective pairs’ below), Multivariate analysis of variance (MANOVA) was conducted (Wilks’ Lambda). Multivariate tests of significance were first conducted on the entire data set (excluding vibrato stimuli), showing highly significant main effects (p < .00001) for Instrument (F = 9.27), Pitch (F = 25.26), and Loudness (F = 13.39), and a significant Loudness × Instrument interaction (F = 2.28, p < .05); a similar analysis was conducted for the four stimuli used in examining vibrato effects, showing a significant effect of Vibrato (F = 6.11, p < .005) and a significant Vibrato–Instrument interaction (F = 3.45, p < .01). These analyses were followed by separate multivariate tests for repeated measure for each of the six dependent variables (adjective pairs’ ratings). For each adjective pair, a repeated measures MANOVA was conducted for ratings of the 18-sound non-vibrato matrix, with Loudness (pp, mf or ff), Pitch (A4, A5 or A6) and Instrument (violin or piano) as within-subject independent variables, Musical Training as a between-subject independent variable, and adjective ratings (1–5) as the dependent variable. Results for the two vibrato sounds were compared only to non-vibrato sounds with the same Loudness (mf) and Pitch (A5) levels, in repeated-measures MANOVAs with Vibrato and Instrument as within-subject independent variables. Downloaded from pom.sagepub.com at UNIV AUTONOMA DE NUEVO LEON on August 20, 2012

455


Results are presented in Table 3, which summarizes MANOVA results, and in the box-plots of Figure 1, depicting the distributions of adjective ratings as related to the musical variables of Loudness, Pitch Height, Instrument and Vibrato. Table 3. Significant effects of musical and population variables on ratings for tactile metaphors (repeated measures MANOVA–Wilks test). Tactile Metaphor

Musical variable

F value

p

M * (SE)

M ** (SE)

Sharp–Blunt

Instrument Pitch Loudness Instrument Pitch Loudness Instrument– Loudness Pitch–Loudness Instrument– Vibrato Instrument– Training Instrument Pitch Loudness Instrument– Loudness Instrument– Training Loudness– Training Pitch Loudness Vibrato Instrument– Pitch–Loudness Instrument Pitch Loudness Vibrato Instrument– Vibrato Instrument Pitch Vibrato Instrument– Pitch Instrument– Vibrato

13.13 81.72 20.69 47.24 5.78 35.67 6.4

0.0008 0.0000 0.0000 0.0000 0.0065 0.0000 0.004

2.84 (0.13) 3.53 (0.14) 3.05 (0.17) 2.85 (0.13) 2.97 (0.15) 2.77 (0.15)

2.47 (0.14) 2.47 (0.15) 2.67 (0.16) 3.50 (0.12) 3.15 (0.15) 3.11 (0.14)

3.4 (0.16) 3.65 (0.15)

3.89 5.69

0.0102 0.0221

6.89

0.0124

14.07 38.51 75.03 4.1

0.0005 0.0000 0.0000 0.0246

2.82 (0.14) 2.52 (0.16) 2.26 (0.14)

3.26 (0.13) 3.12 (0/16) 3.01 (0.15)

3.49 (0.16) 3.85 (0.14)

5.06

0.0302

3.31

0.0474

18.8 26.6 5.78 6.04

0.0000 0.0000 0.0211 0.0008

3.27 (0.15) 2.30 (0.14) 2.7 (0.24)

2.75 (0.14) 2.83 (0.13) 3.06 (0.25)

9.82 35.54 18.74 12.59 6.83

0.0033 0.0000 0.0000 0.0012 0.0128

3.08 (0.13) 2.60 (0.15) 2.92 (0.15) 2.65 (0.23)

3.39 (0.13) 3.32 (0.14) 3.24 (0.15) 3.34 (0.27)

3.79 (0.15) 3.55 (0.16)

23.8 4.8 25.75 4.7

0.0000 0.014 0.0000 0.015

3.09 (0.12) 3.05 (0.14) 2.46 (0.24)

3.50 (0.12) 3.26 (.14) 3.3 (0.23)

3.58 (0.15)

11.32

0.0017

Smooth–Rough

Soft–Hard

Light–Heavy

Warm–Cold

Wet–Dry

Ratings were translated to numerical values of 1–5: Left antonym: 1; Right antonym: 5. N: M * (SE) M ** (SE) M *** (SE)

Pitch 240 Pitch: A4 Pitch: A5 Pitch: A6

Loudness: 240 Loudness: pp Loudness: mf Loudness: ff

Instrument 360 Instrument: Flute Instrument: Violin

Vibrato 80 Vibrato: + Vibrato: -


M *** (SE)

1.97 (0.15) 2.26 (0.16)

2.52 (0.14) 3.41 (0.15)

456


Sharp–Blunt (1–5) box plots for instrument, pitch and loudness

Smooth–Rough (1–5) box plots for instrument, pitch and loudness

Soft–Hard (1–5) box plots for instrument, pitch and loudness

Light–Heavy (1–5) box plots for pitch, loudness and vibrato

Warm–Cold (1–5) box plots for instrument, pitch, loudness and vibrato

Wet–Dry (1–5) box plots for instrument, pitch and vibrato

· Mean

95% Confidence Interval

Median

Quartile

Figure 1. Distributions of adjective ratings as related to the musical variables of Loudness, Pitch, Instrument and Vibrato. Downloaded from pom.sagepub.com at UNIV AUTONOMA DE NUEVO LEON on August 20, 2012

457


Results indicate significant main effects for all four musical variables (Pitch, Loudness, Instrument, and Vibrato), and some significant interactions among variables. • Highly significant main effects of pitch height for all six adjective pairs. Higher pitches were rated as sharper, rougher, harder, lighter, colder and drier than lower pitches. • Highly significant main effects of loudness for five adjective pairs. Louder sounds were rated as sharper, rougher, harder, heavier and colder than quieter sounds. Note that results for louder dynamics and higher pitch are parallel in all but one measure: higher pitch is more lightweight, but louder sound is heavier. • Significant main effects of instrument: violin sound was rated as blunter (less sharp), rougher, harder, colder and drier, as compared to flute. Instrument did not affect light– heavy ratings. • Significant main effects of vibrato: vibrato sounds were rated as lighter, warmer and wetter than non-vibrato sounds. Vibrato did not affect sharp–blunt and soft–hard ratings. Several significant interactions between musical dimensions were found. Most of these interactions involve the Instrument variable (flute or violin), suggesting that the effects of the other musical variables examined (Pitch, Loudness, and Vibrato) on the application of tactile metaphor may vary for different musical instruments. Note, however, that significant interactions are limited to few metaphors, particularly smooth–rough and soft–hard; moreover, only one interaction (Instrument–Vibrato for smooth–rough) presents opposite effects of the two interacting variables. In all other interactions, only the magnitude of the effect, rather than its direction, is involved. • Significant interactions between Instrument and Loudness were found for the smooth– rough antonym (F = 6.4; p = .004), where rating differences between instruments were larger at lower dynamic levels, and for soft–hard, where rating differences were larger at higher dynamic levels (F = 4.1; p = .025). • A significant interaction between Instrument and Pitch was found for the wet–dry antonym only. For the flute, rating differences occurred between the two higher pitch registers, while for the violin, they occurred mainly between the two lower registers (F = 4.7; p = .015): • Significant interactions between Instrument and Vibrato were found for the warm–cold (F = 6.83; p = .012) and wet–dry (F = 11.3; p = .0017) antonyms, in which rating differences between vibrato and non-vibrato were considerably larger for the violin, and for the smooth–rough antonym, in which vibrato was rated as smoother than non-vibrato for the violin, while for the flute it was rated as rougher (F = 5.27; p = .022). • A significant interaction between Pitch Height and Loudness was found for the smooth– rough antonym only (F = 3.89; p = 0.01). While for low and medium dynamic levels (pp, mf), middle register pitch was rougher than low-register pitch, this was not the case for the loudest dynamics (ff). The effect of musical training. No main effect was found for Musical Training. However, Musical Training interacted with Instrument for the smooth–rough (F = 6.89; p = .012) and soft–hard (F = 5.06; p = .03) dichotomies, and with loudness for soft–hard (F = 3.31; p = .047), such that differences between ratings for the two instruments, as well as for loudness values, were larger for the musically trained participants. In addition, a significant three-way interaction between Instrument, Vibrato and Musical Training emerged for sharp–blunt ratings (F = 4.82; p = .034), such that musicians associated reduced sharpness with violin non-vibrato, while non-musicians associated reduced sharpness with flute non-vibrato. Downloaded from pom.sagepub.com at UNIV AUTONOMA DE NUEVO LEON on August 20, 2012

458


Table 4. Preferred adjectives and easy/difficult categories. Category Sharp–Blunt

Smooth–Rough

Soft–Hard

Light–Heavy

Warm–Cold

Wet–Dry

Easy 6 Difficult 3 Preferred 15

8 8 8

15 2 25

3 9 8

8 7 30

3 18 3

3

7

11

17

4

1

Table 5. Pearson correlation coefficients between adjectives pairs, based on the entire data set.

Soft–Hard Sharp–Blunt Smooth–Rough Light–Heavy Warm–Cold

Sharp–Blunt

Smooth–Rough

Light–Heavy

Hot–Cold

Wet–Dry

-0.34*

0.21 0.20

0.42** 0.08 0.27(*)

0.51*** -0.30(*) 0.23 0.03

0.05 -0.17 0.39* 0.21 0.002

(*) p < .1 * p < .05, ** p < .01, *** p < .001

Preferred adjectives and easy/difficult categories. In their free verbal responses participants noted, among other things, which of the antonym pairs were the easiest, and which were the most difficult to rate. In addition, they were asked to choose, out of the 12 adjectives presented, up to three that they believe are most suitable as metaphors for sound. Table 4 quantifies these responses. As these responses were adapted from freely composed comments and are limited in number, we do no analyze them statistically. Nevertheless, data indicate that the easiest pair to apply as a metaphor to sound was by far soft–hard, while the most difficult pair was wet–dry. Most participants considered ‘warm’ and ‘soft’ highly suitable metaphors for sound, while ‘wet’, ‘dry’, ‘blunt’ and ‘cold’ were considered suitable by very few participants. Note the strong preference toward one of the two terms in some pairs: warm rather than cold, heavy rather than light, sharp rather than blunt. Correlations between adjective pairs. To examine whether different tactile metaphors for sound tend to associate with each other, we computed Pearson correlation coefficients, based on the entire data set, between all adjective pairs (Table 5). Several significant correlations (p < .05) can be observed, most involving the pair soft–hard (which, as mentioned above, was also considered the tactile dimension easiest to apply to sound). Soft–hard correlates positively with warm–cold and light–heavy, and negatively with sharp–blunt. Soft, then, tends to associate with warm, lightweight and blunt, while hard associates with cold, heavy and sharp. In addition, smooth–rough correlates positively with wet–dry, and (marginally) with light–heavy, and sharp–blunt presents a marginal negative correlation with warm–cold.

Discussion This study indicates that tactile metaphors applied to sound are systematically affected by musical factors, particularly pitch height and loudness, and suggests specific relationships between


459


the two modes. Only few of the cross-modal interactions investigated here have been empirically studied before, and fewer were examined using actual musical sound. Thus, for instance, highly significant associations, rarely studied before, were established between loudness and the dimensions of softness, smoothness, weight, heat and wetness. In the following, we shall briefly discuss several central issues stemming from the results. These include the correspondence between pitch height and loudness, as revealed by their shared tactile metaphors, as well as several complementary accounts of tactile mappings for sound. Finally, the issue of ecological validity will be addressed. We shall discuss supporting evidence suggesting that results of this study – its rarified experimental settings notwithstanding – are musically relevant, and briefly suggest how further studies may examine the present results in the context of musical performance.

Pitch and loudness: similarities and an important contrast Importantly, the effects of pitch and loudness on tactile metaphors are similar for all dimensions but one, as higher pitch and louder sound were both rated as sharper, rougher, harder and colder. This congruence is consistent with studies indicating perceptual correlation of rising and falling pitch with increasing and decreasing loudness, respectively (Nakamura, 1987; Neuhoff & McBeath, 1996; Neuhoff, McBeath, & Wanzie, 1999). Comparably, studies of musicinduced imagery (Eitan & Granot, 2006; Eitan & Tubul, in press), and of perceptual congruence effects (Eitan, Schupak, & Marks, 2008), suggest that rise and fall in loudness and pitch convey similar spatio-temporal associations (e.g., diminuendo strongly suggests spatial fall, like ‘fall’ in pitch). Together with these earlier studies, our results thus point at intriguing similarities concerning the web of cross- and a-modal associations these two basic dimensions of sound convey to listeners, similarities that may deeply affect musical experience. Note, however, that amid these similarities stands one important contrast: increased loudness is ‘heavy’ while high pitch is ‘light’. Similar pitch–loudness contrasts were observed for the related dimension of size: high pitch is associated with small physical size (Marks, Hammeal, Bornstein, & Smith, 1988; Walker & Smith, 1984), but high loudness – ‘volume’ – is indeed voluminous (Lipscomb & Kim, 2004; Stevens, 1934; Walker, 1987). In decoding what sound may convey to listeners, these contrasts in ‘potency’ (Osgood et al., 1957), standing amid striking similarities in other domains, should be an important consideration.

Possible sources of audio-tactile mappings Given our rudimentary understanding of auditory–tactile interaction, any explanation of the specific sound-touch associations presented here would be speculative. Nevertheless, several complementary accounts may be tentatively pointed out. Tactile sensations of the ear: Do cross-modal metaphors reflect analogous sensory processing? The senses of touch and hearing correspond in some fundamental ways (Soto-Faraco & Deco, 2009). Both are based on receptors that respond to pressure stimuli, transferring them (converted into electrochemical stimuli) through the nerves to the brain for processing; and both process vibrations, analyzing (albeit with very different subtlety) amplitude, frequency and waveform, within perceptual ranges and just noticeable differences (JNDs) that are often roughly compatible. For instance, the vibrotactile frequency response range is approximately 20–1000 Hz, and the vibrotactile intensity response ranges about 55 dB from the lower


460


threshold (Gunther & O’Modhrain, 2003). Sometimes, as when exploring surface texture with a probe, the very same vibrations reach both skin and ear (see Lederman, Klatzki, Morgan, & Hamilton, 2002). Such analogies in sensory processing may give rise, at higher processing levels, to perceptual and verbal correlations. Some audio-tactile mappings exhibited in the present study may demonstrate such correlations, illustrated by the sound images in Figure 2. Light–heavy; soft–hard. Louder sounds generate greater pressure on our hearing receptors, just as heavier and harder objects activate greater pressure on pressure receptors in the skin. Hence, sound waves with higher amplitudes are perceived as heavy and hard. (Compare images 9 and 10 (pp) with images 11 and 12 (ff) in Figure 2.) Smooth–rough. In vibrotactile perception, the contrast between purer and richer waveforms is represented as smoothness vs. roughness (Rovan & Hayward, 2000). Comparably, a flute’s simpler sound wave (Figure 2, image 1) was rated as smoother than the violin’s (Figure 2, image 2); louder sounds (Figure 2, images 11 and 12), possessing higher amplitudes and richer in audible partials, were rated as rougher than quieter sounds (Figure 2, images 9 and 10). Higher pitch, however (Figure 2, images 7 and 8) was rated as rougher than lower pitch (Figure 2, images 5 and 6), in apparent variance with the above hypothesis. Possibly, another factor, the shorter, ‘spikier’ wave lengths of higher pitch, may have created the audio-tactile analogy here. Sharp–blunt. Violin sound (Figure 2, image 2), higher pitch (Figure 2, images 7 and 8) and louder tone (Figure 2, images 11 and 12) were rated as ‘sharper’. These correlations are consistent with an accepted psycho-acoustic definition of sharpness (Bismarck, 1974), as they all increase spectral energy in higher frequency regions. However, the relationships also apply to a more general definition of a wave’s ‘sharpness’: sharpness rises as steepness rises – as the amplitude grows and the wave length shortens. Obviously, such account of sensory correspondence should be qualified. To support the hypothesis that cross-modal metaphorical mapping, performed verbally at a high cognitive level, is associated with correlations in low-level sensory processing, the path from such basic sensory processes, mostly subconscious, to high-level cognitive operations should be accounted for. In particular, one should specify whether such path begins with low-level interaction, based on neural encoding of sensory correspondence (see Foxe et al., 2000, Hötting at al., 2003, for relevant studies of early brain potentials). Alternatively, stimuli in different modalities may be separately processed at lower levels, and then mapped, through higher-level, language-related processes, into cross-domain concepts such as lightness or softness (see Martino & Marx 1999, 2001, for a relevant model). To examine these alternatives, converging or combined studies using implicit perceptual measures, such as response time in cross-modal tasks, physiological measures and brain imaging techniques (fMRI and ERP) may be conducted. Audio-tactile mappings and experiential congruence. Regardless of low-level analogies of sensory processing, we may learn to relate certain tactile and auditory properties because these are often encountered together, associated with the same objects, in daily experience. Thus, for instance, subjects associate louder impact sound with larger and heavier objects (Burro & Grassi, 2001). Overall, higher loudness, as well as higher loudness of the spectral centroid, are also associated with harder objects or surfaces (Freed, 1990, Giordano, 2005). Higher loudness (Guest et al., 2002; Lederman, 1979) and higher frequency (Zampini et al., 2003) are both associated with roughness.


461


Figure 2. Sound images of selected stimuli. Downloaded from pom.sagepub.com at UNIV AUTONOMA DE NUEVO LEON on August 20, 2012

462


Such repeated cross-modal associations may impart in structural invariants that are perceived as indicators of the source object’s attributes, such as softness, dryness or roughness. Ecological psychology has suggested that an organism directly perceives complex visual (Gibson, 1966) or auditory (Gaver, 1993a, 1993b) invariant structures that specify source objects, their properties and actions. The relationships observed by participants in the present study may take part in constituting auditory invariants (variants also involving other acoustic features, such as amplitude envelope and spectral structure) that specify to the listener basic tactile (indeed, cross-modal) features of the source object. Note that these invariants specify objects and their features through events or actions performed upon them, rather than by passive perception alone: sound specifying ‘soft’ vs. ‘hard’ is the sound related to acting upon (e.g., hitting) a soft or hard object (Gaver, 1993a, 1993b). Structural cross-modal invariants may induce the habitual use of verbal cross-modal mappings, and the metaphors resulting from such mappings may themselves become the principal means of describing the auditory target dimension (e.g., ‘soft’, as used for reduced loudness). Such prevalent verbal use, in turn, reinforces cross-modal association.2 One way to examine the effect of experiential correspondences on auditory–tactile mappings would be developmental studies, involving infants and young children, as compared to older children and adults. Developmental investigations of other cross-modal interactions have suggested different developmental tracks for different cross-modal mappings. For instance, associations of pitch and loudness with brightness are traced in infancy or early childhood, while the associations of pitch and loudness with size mature only in late childhood (age 9–11), suggesting that they are largely determined by children’s exposure to cross-modal correlations in daily experience (Lewkowicz & Turkewitz, 1980; Marks et al., 1988; Smith & Sera, 1992). Similar studies involving auditory–tactile interactions may distinguish between interactions based upon repeated exposure to audio-tactile correlations and those whose sources lie elsewhere. Underlying semantic dimensions. As suggested by Osgood’s findings (Osgood, 1964; Osgood et al., 1957; see above), associations of auditory and tactile features, as well as correlations among the tactile features themselves, may be related to abstract semantic dimensions such as activity or potency. Thus, it might have been claimed that some of our findings may not primarily stem from the presentation of actual auditory stimuli, but from an a-modal semantic space, relating auditory and tactile adjectives through their shared connotations to abstract dimensions such as high (or low) power, activity or evaluation. Specifically, as mentioned, adjectives denoting the poles of pitch and loudness (bass–treble, loud–soft), as well as five of the six antonyms used in the present study, loaded highly into the potency factor (Osgood, 1964; Osgood et al., 1957), such that low and loud sound, as well as the tactile features hard, rough, heavy, sharp and hot, were all high in potency. One may then expect that these features would correlate, all being ‘potent’ or ‘powerful’. Our results, however, indicate that abstract semantic dimensions, such as potency, explain audio-tactile mappings only partially, if at all; nor do they account very well for correlations among tactile features. Indeed, some correlations between tactile dimensions predicted by Osgood’s potency scores are presented in our results (Table 5): hard is positively correlated with heavy, and rough and heavy are also marginally correlated. However, other correlations implied by Osgood’s data are reversed or absent. Warm is not correlated with hard, sharp, rough and heavy, as would be predicted by shared high potency. Rather, it correlates with soft and blunt,


463


and shows no significant correlations with the heavy–light and smooth–rough dichotomies. Likewise, sharp is correlated with cold, rather than warm, as implied by Osgood’s data, and is not associated with rough or heavy. Rather than based upon abstract semantic dimensions, some of the correlations between tactile features may be modality specific, related to concrete tactile experience. Chen, Shao, Barnes, Childs and Henson (2009) asked participants to rate various surfaces on bi-polar scales for warm–cold, slippery–sticky, smooth–rough, hard–soft, bumpy–flat, and wet–dry. They found high correlations between ratings for warm–cold and soft–hard (.74), and between smooth–rough and wet–dry (.65), correlations also found in the present study. Neither correlation is predicted by Osgood’s data. Comparing audio-tactile mappings (Table 3) with those implied by Osgood’s potency loadings indicates that high loudness indeed associates significantly with most tactile features high in potency – hard, rough, heavy and sharp – though louder sound is cold rather than warm. The tactile mappings of loudness may thus stem in part from an abstract potency (power) dimension, underlying both loudness and its tactile correlates. Indeed, such mappings make ecological sense, since louder sound is associated with potency (i.e., with larger, more massive sounding bodies, capable of powerful actions or effects). In contrast, low pitch (also associated with high potency) is rated as soft, smooth and dull, rather than as hard, rough and sharp, attributes loaded highly into potency. Thus, while the audio-tactile mappings of loudness may largely stem from its potency connotations, those of pitch height lie elsewhere. Still, audio-tactile mappings may indeed relate to dimensions such as evaluation, activity or potency, but these may derive from the participants’ experience of the actual auditory stimuli presented to them, rather than from abstract, pre-conceived semantic relationships. Thus, for instance, in their free verbal responses participants often noted that some sounds were more pleasant than others, and related this evaluation to the tactile qualities those sounds metaphorically possessed. Generally, pleasant sounds were described as warm, soft, lightweight, blunt and smooth, while unpleasant sounds were cold, hard, heavy, sharp and rough (notably, these evaluations only partially concur with Osgood’s valence loadings – they do for the dimensions soft–hard, smooth–rough, and light–heavy, but not for warm–cold and sharp–blunt). As noted, these terms also correlated with each other (Table 5). In addition, the terms ‘soft’ and ‘warm’ were favored by most participants as metaphors for music and sounds (Table 4). These converging results suggest that an evaluation dimension, contrasting ‘pleasant’ sounds (described as soft, smooth, warm, lightweight and blunt, with ‘unpleasant’ ones – hard, rough, cold, heavy and sharp) may have underscored specific tactile metaphors for sound. Yet the source of this dimension is not in an abstract semantic space, but actual affective experience, involving concrete sound.

Ecological validity: Relevance to music listening and performance This is an exploratory study. By necessity, it examined a limited set of pitch and loudness levels, and only two instrumental timbres. Furthermore, to manage and control our auditory variables, we restricted our stimuli to single sounds. Even a short, simple melodic phrase would feature a host of additional variables – pitch contour, interval size, articulation, and a variety of tonal and rhythmic features – that may affect the use of tactile metaphors and interact in complex, non-additive ways with the variables of pitch register, loudness, instrumental timbre and vibrato examined here.


464


Would the audio-tactile mappings observed here, then, be featured in a musical context? Two recent studies by our group may help examining this crucial issue. In Eitan and Timmers (2010: Experiment 3), participants heard two segments from the variation movement of Beethoven’s piano sonata, op. 111, differing in pitch register but otherwise similar, and rated each segment on 35 bi-polar scales, including five of the six tactile antonyms used in the present study. Four of these antonyms were rated similarly in both studies: hard, light and sharp were all associated significantly with high pitch in both studies; cold was significantly associated with high pitch in the present study, and marginally so (p < .1) in Eitan and Timmers’ (2010) experiment. The smooth–rough antonym, however, was rated in opposite ways: in the present study, high pitch is rougher, while in Eitan and Timmers, low pitch is rougher. Notably, Eitan and Timmers used a very different instrumental timbre (piano) and a wider pitch range than that used in our study, which might have affected roughness ratings more than other ratings. This notwithstanding, the similarity of most audio-tactile mappings in ratings of single violin and flute sounds and in those of complex piano music suggests that results of this study, its impoverished stimuli notwithstanding, may largely apply to actual musical contexts. Further corroboration of the present study’s ecological validity comes from Eitan et al. (2010). In that study, pitch, loudness and tempo were manipulated factorially in two phrases from a 20th-century flute solo piece (Varese, Density 21.5). For each musical stimulus, participants (children aged 8 and 11 and adults) rated, among other adjectives, the tactile antonyms soft–hard, smooth–rough, and sharp-round, also examined here. As in the present study, music higher in pitch or louder was rated as harder, sharper and rougher than music lower in pitch or of reduced loudness. Again, then, our results correspond well with those obtained from ‘real’ music, suggesting that their ecological validity is high. How can one further examine the musical relevance of tactile metaphors? The present article concentrated on the listening end of the musical communication chain. Yet tactile metaphors, as noted in the beginning of this article, seem to be as important for performers, serving to communicate sound quality and performance expression. Further studies of the use of touch metaphors in music should, then, address more closely aspects of musical sound that are primarily within the realm of expressive performance, such as articulation (e.g., staccato vs. legato), or within-instrument timbre (e.g., sul ponticello vs. sul tasto in string instruments). Furthermore, to investigate how musicians interpret tactile metaphors in performance, a paradigm complementary to that used here may be applied: performers would be instructed to play sounds possessing a specified ‘tactile’ quality (e.g., ‘warm’ ‘soft’ or ‘rough’ sound), or perform the same melodic line in several different expressions, described by tactile metaphors. The results of these different performances would then be analyzed acoustically, examining how qualities such as sound envelope and spectral composition are related to the sound’s tactile descriptions. A qualitative investigation of the role of tactile metaphors in performers’ training and communication may importantly supplement such quantitative approaches in enhancing our understanding of the actual musical functions of tactile metaphors.

Coda Differing accounts of their sources notwithstanding, our results indicate that tactile metaphors for musical sound are neither coincidental nor subjective, but relate systematically to basic qualities of sound. Our findings add to the accumulated evidence suggesting that a rich, yet consistently applied, array of cross-modal connotations underlies the perception


465


of auditory stimuli, and musical sound specifically. The consistency of such cross-modal associations suggests that an account of music cognition in purely auditory terms may be insufficient: aspects of the experienced ‘meaning’ of sound may inhere in the web of crossmodal connotations it imparts. To the practising musician (performer and composer), such insights may both validate and enrich the palette of expressive nuances. To the researcher, they call for further studies concerning the manifold ways musical and auditory attributes touch us. Acknowledgments We thank Itzkhak Mizrakhi for his assistance in stimuli preparation, and Alex Gotler and Tal Galili for assistance in statistical analysis.

Notes 1. Note that the Hebrew word used, Kham, may denote both ‘warm’ and ‘hot’. 2. Since equal loudness curves in use (e.g., Fletcher–Munson, ISO226: 2203) were determined using pure tones or noise, they are hardly appropriate to harmonic tones, particularly across different musical instruments. Determining equal loudness levels individually for each participant (the optimal option) was unfortunately not a practical alternative. Hence, loudness adjustment was based on independent judgments of four expert musicians, including the two players, a concert pianist (the second author), and a composer/recording engineer. Notably, after equal loudness adjustment violin sounds were, on average, 7 dB-SPL softer than comparable flute sounds. 3. Indeed, linguistic stimuli may create cross-modal congruence effects as strongly as the actual sensory stimuli they represent (Martino & Marks, 1999; Walker & Smith, 1984, 1986).

References Bismarck, G. von (1974). Timbre of steady sounds: A factorial investigation of its verbal attributes. Acustica, 30, 146–159. Burro, R., & Grassi, M. (2001). Experiments on size and height of falling objects. Phenomenology of Sounds Events, Sounding Object Group Report, 4, 31–39. Chen, X., Shao, F., Barnes, C., Childs, T., & Henson, B. (2009). Exploring relationships between touch perception and surface physical properties. International Journal of Design, 3(2), 67–77. Eitan, Z., & Granot, R. Y. (2006). How music moves: Musical parameters and images of motion. Music Perception, 23(3), 221–247. Eitan, Z., Katz, A., & Shen, Y. (2010, August). Effects of pitch register, loudness and tempo on children’s metaphors for music. Poster presented at the 11th International Conference on Music Perception and Cognition (ICMPC11), Seattle, WA. Eitan, Z., Schupak, A., & Marks, L. E. (2008). Louder is higher: Cross-modal interaction of loudness change and vertical motion in speeded classification. In K. Miyazaki, Y. Hiraga, M. Adachi, Y. Nakajima, & M. Tsuzaki (Eds.), Proceedings of the 10th International Conference on Music Perception & Cognition (ICMPC10) (pp. 67–76). Sapporo, Japan: ICMPC10. Eitan, Z., & Timmers, R. (2010). Beethoven’s last piano sonata and those who follow crocodiles: Crossdomain mappings of auditory pitch in a musical context. Cognition, 114 (3), 405–422. Eitan, Z., & Tubul, N. (in press). Musical parameters and children’s images of motion. Musicae Scientiae. Foxe, J. J., Morocz, I. A., Murray, M. M., Higgins, B. A., Javitt, D. C., & Schroeder, C. E. (2000). Multisensory auditory-somatosensory interactions in early cortical processing revealed by high-density electrical mapping. Cognitive Brain Research, 10(1), 77–83.


466


Foxe, J. J., Wylie, G. R., Martinez, A., Schroeder, C. E., Javitt D. C., Guilfoyle, D., et al. (2002). Auditorysomatosensory multisensory processing in auditory association cortex: An fMRI study. Journal of Neurophysiology, 88(1), 540–543. Freed, D. J. (1990). Auditory correlates of perceived mallet hardness for a set of recorded percussive events. Journal of the Acoustic Society of America, 87(1), 311–322. Gaver, W. W. (1993a). What in the world do we hear? An ecological approach to auditory event perception. Ecological Psychology, 5(1), 1–29 Gaver, W. W. (1993b). How do we hear in the world? Explorations of ecological acoustics. Ecological Psychology, 5(4), 285–313 Gibson, J. J. (1966). The senses considered as perceptual systems. Boston, MA: Houghton Mifflin. Giordano, B. L. (2005). Sound source perception in impact sounds. Unpublished doctoral dissertation, University of Padova, Italy. Guest, S., Catmur, C. Lloyd, D., & Spence, C. (2002). Audiotactile interactions in roughness perception. Experimental Brain Research, 146(2), 161–171. Gunther, E., & O’Modhrain, S. (2003). Cutaneous grooves: Composing for the sense of touch. Journal of New Music Research 32 (4), 369–381. Hlushchuk, Y. (2007). Tactile processing in human somatosensory and auditory cortices. Unpublished doctoral dissertation, University of Helsinki, Finland. Hötting, K., Rösler, F., & Röder, B. (2003). Crossmodal and intermodal attention modulate event-related brain potentials to tactile and auditory stimuli. Experimental Brain Research, 148(1), 26–37. Jousmäki, V., & Hari, R. (1998). Parchment-skin illusion: Sound-biased touch. Current Biology, 8(6), 190. Lederman, S. J. (1979). Auditory texture perception. Perception, 8(1), 93–103. Lederman, S. J., Klatzki, R. L., Morgan, T., & Hamilton, C. (2002). Integrating multimodal information about surface texture via a probe: Relative contribution of haptic and touch produced sound sources. In Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, 2002 (pp. 97–104). London: IEEE Computer Society’s Conference Publishing Services. Lewkowicz, D. J. (2000). The development of intersensory temporal perception: An epigenetic systems/ limitations view. Psychological Bulletin, 126(2), 281–308. Lewkowicz D. J., & Turkewitz G. (1980). Cross-modal equivalence in early infancy: Auditory–visual intensity matching. Developmental Psychology, 16, 597–607. Lipscomb, S. D., & Kim, E. M. (2004). Perceived match between visual parameters and auditory correlates: An experimental multimedia investigation. In S. Lipscomb, R. Ashley, R. Gjerdingen, & P. Webster (Eds.), Proceedings of the 8th International Conference on Music Perception and Cognition (ICMPC8), Evanston, IL, 3–8 August, 2004 (pp. 72–75). Adelaide: Causal Productions. Marks, L. E. (2004). Cross-modal interactions in speeded classification. In G. Calvert, C. Spence, & B. E. Stein (Eds.), Handbook of multisensory processes (pp. 85–106). Cambridge, MA.: MIT Press. Marks, L. E., Hammeal, R. J., Bornstein, M. H., & Smith, L. B. (1988). Perceiving similarity and comprehending metaphor. Monographs of the Society for Research in Child Development, 52 (1, Serial No. 215). Martino, G., & Marks, L. E. (1999). Perceptual and linguistic interactions in speeded classification: Tests of the semantic coding hypothesis. Perception, 28(7), 903–923. Martino, G., & Marks, L. E. (2001). Synesthesia: Strong and weak. Current Directions in Psychological Science, 10(2), 61–65. Mursell, J. L. (1937). The psychology of music. New York: Norton. Nakamura, A. (1987). The communication of dynamics between musicians and listeners through musical performance. Perception and Psychophysics, 41(6), 525–533. Neuhoff, J. G., & McBeath, M. K. (1996). The Doppler illusion: The influence of dynamic intensity change on perceived pitch. Journal of Experimental Psychology: Human Perception and Performance, 22(4), 970–985. Downloaded from pom.sagepub.com at UNIV AUTONOMA DE NUEVO LEON on August 20, 2012

467


Neuhoff, J. G., McBeath, M. K., & Wanzie, W. C. (1999). Dynamic frequency change influences loudness perception: A central, analytic process. Journal of Experimental Psychology: Human Perception and Performance, 25(4), 1050–1059. Osgood, C. E. (1964). Semantic differential technique in the comparative study of cultures. American Anthropologist, 66, 171–200. Osgood, C. E., Suci, G. J., & Tannenbaum, P. H. (1957). The measurement of meaning. Chicago: University of Illinois Press. Peeva, D., Baird, B., Izmirli, O., & Blevins, D. (2004). Haptic and sound correlations: Pitch, loudness and texture. In Proceedings of the Eighth International Conference on Information Visualization, (IV’04) (pp. 659–664). London: IEEE Computer Society. Rovan, J., & Hayward, V. (2000). Typology of tactile sounds and their synthesis in gesture-driven computer music performance. In M. Wanderley & M. Battier. (Eds.), Trends in gestural control of music (pp. 389–405). Paris: IRCAM. Schroeder, C. E., Lindsley, R. W., Specht, C., Marcovici, A., Smiley, J. F., & Javitt, D. C. (2001). Somatosensory input to auditory association cortex in the macaque monkey. Journal of Neurophysiology, 85(3), 1322–1327. Schürmann, M., Caetano, G., Hlushchuk, Y., Jousmäki, V., & Hari, R. (2006). Touch activates human auditory cortex. NeuroImage, 30(4), 1325–1331. Smith, L. B., & Sera, M. D.(1992). A developmental analysis of the polar structure of dimensions. Cognitive Psychology, 24, 99–142. Soto-Faraco, S., & Deco, G. (2009). Multisensory contributions to the perception of vibrotactile events. Behavioral Brain Research, 196, 145–154. Stevens, S. S. (1934). The volume and intensity of tones. American Journal of Psychology, 46(3), 397–408. Walker, P., & Smith, S. (1984). Stroop interference based on the synaesthetic qualities of auditory pitch. Perception, 13(1), 75–81. Walker, P., & Smith, S. (1986). The basis of Stroop interference involving the multimodal correlates of auditory pitch. Perception, 15(4), 491–496. Walker, R. (1987) The effects of culture, environment, age, and musical training on choices of visual metaphors for sound. Perception and Psychophysics, 42(5), 491–502. Zampini, M., Guest, S., & Spence, C. (2003). The role of auditory cues in modulating the perception of electric toothbrushes. Journal of Dental Research, 82(11), 929–932.

Biographies Zohar Eitan is an associate professor at the School of Music, Tel Aviv University, Israel, where he teaches and researches music cognition and music theory. His research topics include crossdomain mappings in music, the perception of motivic-thematic structure, the perception of large-scale musical form, and absolute pitch. His recent work has been published in Cognition, Music Perception, Musicae Scientiae and Empirical Musicology Review. Inbar Rothschild is an Israeli pianist. She has performed widely in Europe and the USA, and recorded for Israeli, Czech and Swiss radio stations. She graduated with an M.Mus in Piano Performance from the Buchman-Mehta School of Music, Tel Aviv University, Israel, where she studied with the late Pnina Salzman.


music 2

Recommend Documents