Say it like you mean it

Transcript

1. Say it like you mean it: Lexical meaning influences prosody Molly Lewis and Duane Watson University of Illinois at Urbana- Champaign AMLaP 2011 September 1
2. Can lexical meaning affect how a word is articulated? Lexical /word/ /word/ Meaning shout
3. (Garrett, 1980; Bock & Levelt, 1994) Articulatory Representations Language Production System (syntax, morphology, phonology) Message
4. Outline I. Prior evidence suggesting link between meaning and acoustic-phonetic features • In comprehension • In production II. Two experiments exploring link within motor systems III. Implications for language production models
5. Evidence in comprehension (Shintel & Nusbaum, 2007) The horse is brown (slow) (fast) (slow) (fast) / The horse is brown
6. (Shintel & Nusbaum, 2007) • Task: Determine whether object in picture mentioned in previously heard sentence • Measure: Reaction time to response • Result: Matched faster than mismatched • Suggests: facilitated processing when meaning congruent with prosodic form Evidence in comprehension
7. Evidence in production (Shintel et al., 2006) (down) (up) It is going down / It is going up
8. (Shintel & Nusbaum, 2007) • Measure: mean fundamental frequency (F0) for tokens of “up” and “down” • Result: “down” lower F0 than “up” • Suggests: lexical meaning influences acoustic-phonetic realization Evidence in production
9. Lexical /word/ Meaning maps directly onto acoustic-phonetic properties Meaning /word/ Do these effects extend to motor systems?
10. Semantic Domain: Motivated by situated cognition –Basic computational mechanism for predictive processing –Simulate previous perceptual experiences by activating related motor representations (Barsalou, 2009) Motor representations Conceptual representations
11. Articulatory representations Message ↑activation yell ↑ activation chat
12. What acoustic properties index articulatory representations? • Properties correlated with high levels of prominence (Wagner & Watson, 2010) – Fundamental frequency (F0) – Duration – Intensity (amplitude)
13. Prediction yell (F0, duration, intensity) < chat (F0, duration, intensity)
14. Experiment #1
15. Task and Design • Task: Reading out loud • 2 Conditions (within-subject) –High effort vocal words (10) • yelling, screaming, lecturing, etc. –Low effort vocal words (10) • chatting, whispering, talking, etc. • 60 fillers
16. 16 yelling High-Effort Condition
17. 17 chatting Low-Effort Condition
18. • Acoustic measures: – F0 (min, max, mean) – Duration – Intensity • Mixed effect model: – Fixed: condition, frequency, length, gender – Random: subject, item Experiment #1: Analysis
19. Experiment #1: Results
20. Low−effort Vocal High−effort Vocal Low−effort Vocal High−effort Vocal Semantics Minimum F0 132 134 136 138 140 142 144 146 148 N = 36 * Minimum F0
21. Lexical /word/ Message …but what if this is about general arousal? Semantic features directly influence articulatory representations
22. Articulatory representations Message yell general arousal run kick jump jog
23. Predictions yell (F0, duration, intensity) < run (F0, duration, intensity) Direct Link: yell (F0, duration, intensity) = run (F0, duration, intensity) General Arousal:
24. Experiment #2
25. Task and Design • Same task as Exp. 1 • 2 Conditions (within-subject) –Vocal words (12) • yelling, screaming, chatting, etc. –Non-vocal foot words (12) • running, jogging, kicking, etc. • 48 fillers
26. 26 yelling Vocal Condition
27. 27 running Non-Vocal Condition
28. Experiment #2: Results
29. Foot Vocal Foot Vocal Semantics Intensity (dB) 49 50 51 52 53 54 55 N = 63 * Intensity
30. Intensity (dB) Foot Low −effort Vocal H igh−effort Vocal 49 50 51 52 53 54 55 N = 63 * Intensity ( ) *
31. • Exp. 1: High-effort Vocal > Low-effort Vocal • Exp. 2: Vocal > Foot • Suggests: lexical semantics of a word can influences its acoustic-phonetic realization Summary /word/ Message
32. Why effects on different measures? • Experiment 1: Effects on Min. F0 • Experiment 2: Effects on Intensity • Both correlates of prominence in English – both reflect a semantic-articulatory link • However, future work will need to explore whether reflects different underlying processes in each case
33. What do these data tell us about the language production system? /word/ Message
34. (Garrett, 1980; Bock & Levelt, 1994) Articulatory representations Language Production System (syntax, morphology, phonology) Message
35. How should the model be modified? Language Production System (syntax, morphology, phonology) Articulatory representations Message Two Possibilities ①Direct Link ②Tagging
36. ①Direct Link Language Production System (syntax, morphology, phonology) Articulatory representations Message yell
37. ②Semantic Feature Tagging Language Production System (syntax, morphology, phonology) Articulatory representations Message yell
38. How should the model be modified? Language Production System (syntax, morphology, phonology) Articulatory representations Message Two Possibilities ①Direct Link ②Tagging
39. Conclusion • Evidence for a link between meaning and articulation in the motor system • Possible modifications to models of language production – Direct Link Modification – Semantic Feature Tagging Modification /word/ Message
40. Thanks! • NIH Grant: R01DC008774 • Communication and Language Lab Members • Conversation Lab Members • Production Lab Meeting Members