Inferring the meaning of
chord sequences via lyrics
Tom O'Hara
CS Adjunct at Texas State 2010-11 ([email protected])
[email protected]
Texas State University - Computer Science Talk
based on talk at ACM Workshop on Music Recommendation and
Discovery (WOMRAD)
5 Dec 2011
Talk Overview
Introduction: Lyric chord annotations for
unsupervised learning
Background: Supervised music categorization;
parallel corpora in NLP
Process: Co-occurrence statistics via
contingency tables
Analysis: Major vs. minor chord associations
Conclusion: Summary and future plans
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
2 / 26
Introduction Overview
Typical music recommendation approach
Parallel text corpora usage in NLP
New resource for music recommendation
Online sites for tabs and chords
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
3 / 26
Introduction
Typical music recommendation approach
 Suggest songs based on common categories
(e.g., mood)
 Human annotations of song category
In the beginning was the Word,
En el principio era el Verbo,
the Word
with level
God,
y el Verbo era con Dios,
 Typically and
done
at was
song
and the Word was God.
y el Verbo era Dios.
 Tedious/subjective to do at segment level
“Bridge over Troubled Water”
Parallel text corpora in NLP
[Uplifting]
When you're weary.
Feeling
small.
[Sad]
 Same documents
in
two
or
more
languages
When tears are in your eyes
[Sad]
I willfor
dry human
them all. readers (e.g., UN
[Reassuring]
 Developed
delegates)
I'm on your side
[Reassuring]
 Invaluable
forwhen
automatic
machine translation
ohhhh
times get rough.
[Sad/Reassuring?]
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
4 / 26
Introduction (continued)
New resource for music recommendation
 Guitar tablature (tabs) and chord annotations
 Kept up to date by musicians
 Augments human annotations
 Finer granularity (chord sequence vs. song)
e---------------------------------------------------------b---------------------------------------------------------g---------------------------------------------------------d------------0------------------0------------------0--4-2-0
a---------2------------------2------------------2---------e--0-0-4-------------0-0-4---------------0-0-4-------------
e----------------------------------------b----------------------------------------g----------------------------------------d-----------0-4-2-0--------------0-4-2-0-a--------2-------------------2-----------e-0-0-4---------------0-0-4---------------
Online sites for tabs and chords
 Usenet (e.g., alt.guitar.tab w/ 10K+ songs)
 Web sites (e.g., www.chordie.com w/ 200K+)
A
Pretty Woman
A
Pretty Woman
D
Pretty Woman
F#m
Walking Down The Street
F#m
The Kind I Like To Meet
E
I don't believe you Your're not the truth
E7
No One can Look as good as you
(mercy)
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
5 / 26
Background Overview
Learning meaning of music
Translation lexicon induction
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
6 / 26
Pitchfork album review:
Background
Simon & Garfunkel's 1970 swan song, Bridge Over Troubled Water, was both their most
effortless record and their most ambitious. The duo spent most of the 1960s as a highly
regarded folk act distinguished by their intuitive harmonies and Paul Simon's articulate
songwriting, yet compared to the Greenwich Village revivalists, whom they tried to
emulate on songs like "A Simple Desultory Philippic" and "Bleecker Street", they were
pretty square. By Bookends in 1968, they were settling into themselves, losing their folk
revival pretensions and emphasizing quirky production techniques to match their
soaring vocals. Two years later, Bridge did that album one better by revealing a
voracious musical vocabulary that spanned gospel, rock, R&B, and even classical. As this
thoughtful reissue attests, the album sounds unique even 40 years later, driven and
defined entirely by their own personal musical and political obsessions.
…
Learning meaning of music (MIR)
 Generally combines audio and textual features
 Supervised classification
o Mood/meaning categories like Happy, Sad, Bizarre
o User annotations (e.g., CAL500): Turnbull et al. (2008)
 Unsupervised classification
o Online reviews: Whitman and Ellis (2004)
 Lyric analysis and social tags
o Affect filtering: Hu at al (2009)
o Usage, readability, etc.: McKay el al. (2010)
Translation lexicon induction (NLP)
o Co-occurrence analysis: Fung and Church (1994)
o Linkage refinements: Melamed (2000)
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
7 / 26
Process Steps
1. Obtain song data with chords annotated
2. Extract lyrics proper with chord annotations
3. Optional: Map lyrics into meaning categories
Chord
Word
CW
C~W
~CW
~C~W
B7
G
C
C
D7
A
D7
Bm
B
C
underground
1
71
0
17522
Mime-Version: 1.0
two
2
2212
23
17521
Content-Type:
TEXT/PLAIN;
charset=US-ASCII
morning
1
1929
31
17522
Feed
1
1929
8
17522
water [sic]
;;soChord Lyrics
1Bridge
265Over Trouble
142 17522
Simon
And
Garfunkel
? things
When 4
1552
51
17519
Capo 1
D are
you're
weary.
1
265
162 17522
G is are :G1(355433)
Feeling
3 D7(xx0212)
197
290
17520
Chords
D9(xx0210)A7(x02020)
B7(x21202)
D sittingE9(020102)
small.
When
1 E7(020100)
478
1Gm(355333)
17522Gmaj7(320002)
Chord
Dice
MI
X2
B7sus4(xx2202)
Fdim(xx0101)
E9maj(020101)
C mean Word
tears1
1929
3 Jaccard
17522
B7
underground
0.027
0.014
7.933
243.375
G
are
GD
twoin your 0.002
0.001
-0.486 0.259
Intro:
G A7 Fdim D G A7
D G D G0.001
CG D D9 morning
0.001
-1.664 1.651
eyes <endl> I will
CD
Feed
0.001 C
0.164
0.014
dry Gthem 0.001
all.
D
D
G
D
G
D7
so
0.005
0.002
-1.085
0.607
When
you're
weary.
Feeling
small.
When
tears
are
in
your
eyes
G
???
D things
G
D
G
D
AD
0.005
0.002
-0.162
0.055
???
I will dry
them all.
D7
are
0.005
0.002
-1.272 0.850
G A Bm ???
A
D
D/C#
D7
Bm
is
0.012
0.006
-0.117
0.020
D
<endl>
I'm on your ???
side ohhhh
when I'm
times get rough.
BA
sitting
0.004
0.002
4.232
17.307
on
...
CBm
mean
0.001
0.001
1.333
1.018
your
A
side ohhhh when times get
D
rough.
D/C#
???
D7
??? <endl>
a. Get tagged data on meaning categories for lyrics
b. Preprocess lyrics and untagged chord annotations
c. Train to categorize over words and hypernyms
d. Classify each lyric line from chord annotations
4. Fill contingency table
5. Determine chord(s)/token associations
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
8 / 26
Obtain song data with chords annotated
Taken from Usenet alt.guitar.tab forum
 CRD in subject line
Sample
[C] They're gonna put me in the [F] movies
[C] They're gonna make a big star out of [G] me
We'll [C] make a film about a man that's sad and [F] lonely
And [G7] all I have to do is act [C] naturally
Lyrics are from “Act Naturally” by Johnny Russell and Voni Morrison,
with chord annotations for song as recorded by Buck Owens.
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
9 / 26
Extract lyrics proper with annotations
Notes
 Removes e-mail headers and other extraneous text
 Two column table (one row per chord change)
o Chord, and words for that chords
o Includes end of line and verse indicators
Sample
C
F
C
G
C
F
G7
C
Teas State U.
They're gonna put me in the
movies <endl>
They're gonna make a big star out of
me <endl> We'll
make a film about a man that's sad and
lonely <endl> And
all I have to do is act
naturally <endl> <endp>
5 Dec 2011
O’Hara: chord meaning inference
10 / 26
Fill contingency table
General Format
X\Y
+
-
+
XY
¬X Y
X ¬Y
¬X ¬Y
Sample: G versus 'film'
+
Teas State U.
5 Dec 2011
+
1
0
2,213
17,522
O’Hara: chord meaning inference
11 / 26
Determine chord(s)/token associations
Compute co-occurrence statistics
 Ex: Average mutual information
  P (X
x
= x, Y = y)  log
y
P (X = x, Y = y)
2
P (X = x)  P (Y = y)
MI(G, film) = 3.156102
AvgMI(G, film) = 0.000160
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
12 / 26
Optional Process Step
Oh Baby , don ' t you want to go Oh Baby , don ' t you want to go Back to
the land of California To my sweet home Chicago
Oh Baby , don ' t you want to go Oh Baby , don ' t you want to go Back to
the land of California To my sweet home Chicago
Now one and one is two Two and two is four I ' m heavy loaded baby I ' m
booked , I gotta go Cryin ' , baby Honey , don ' t you want to go Back
to the land of California To my sweet home Chicago
...
1. Obtain song data with chords annotated
2. Extract lyrics proper with chord annotations
3. Optional: Map lyrics into meaning categories
4: American_state#1 2: state#2 province#1 3: administrative_district#1
1: child#2 kid#4 2: offspring#1 progeny#1 issue#6 3: relative#1 relation#3 4: person#1
1: girl#1 miss#1 missy#1 young_lady#1 young_woman#1 fille#1 2: woman#1 adult_female#1 3: female#2
Text Categorization Settings
...
- Tokens
for wordsThey're
and WordNet
semantic classes
C
gonna put me in the
C
Light-Playful
- Default
Rainbow
settings
(e.g.,
no
stemming)
F
movies <endl>
F
Light-Playful
- TF/IDF feature selection
...
...
G7
all I have to do is act
G7
Light-Playful
C
naturally <endl> <endp>
C
Light-Playful
a. Get tagged data on meaning categories for lyrics
b. Preprocess lyrics and untagged chord annotations
c. Train to categorize over words and hypernyms
d. Classify each lyric line from chord annotations
4. Fill contingency table
5. Determine chord(s)/token associations
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
13 / 26
Get tagged data on lyric meaning categories
CAL500 for training data
Turnbull et al. (2008)
 500 songs (but only 300 lyrics obtained)
 Annotated by at least 3 users
 135 categories in broad groups
Emotion category frequency (with one category chosen)
Label
Angry-Aggressive
Arousing-Awakening
Bizarre-Weird
Calming-Soothing
Carefree-Lighthearted
Cheerful-Festive
Emotional-Passionate
Exciting-Thrilling
Happy
Teas State U.
5 Dec 2011
Freq
31
77
7
91
28
9
23
2
6
Label
Laid-back-Mellow
Light-Playful
Loving-Romantic
Pleasant-Comfortable
Positive-Optimistic
Powerful-Strong
Sad
Tender-Soft
O’Hara: chord meaning inference
Freq
7
1
1
3
0
3
3
2
14 / 26
Preprocess lyrics and chord annotations
Isolate punctuation
Add semantic classes for each word
 WordNet hypernyms
Miller (1990)
WordNet Sample
movie#1, film#1, picture#6, moving picture#1, ...
=> show#3
=> social event#1
=> event#1
=> ...
=> product#2, production#3
=> creation#2
=> artifact#1, artefact#1
=> whole#2, unit#6
=> ...
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
15 / 26
Train to categorize w/ words & hypernyms
Rainbow text categorization
McCallum (1996)
 Song documents with meaning category labels
 Tokens for words and WordNet semantic classes
 Default Rainbow settings (e.g., no stemming)
 TF/IDF feature selection
FYI:
 Survey of WordNet text categorization work
Mansuy and Hilderman (2006)
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
16 / 26
Classify each lyric line from annotations
Each line classified as mini-document
 Verse included for more context
Original annotations
C
F
...
G7
C
They're gonna put me in the
movies <endl>
all I have to do is act
naturally <endl> <endp>
Result
C
F
...
G7
C
Teas State U.
Light-Playful
Light-Playful
Light-Playful
Light-Playful
5 Dec 2011
O’Hara: chord meaning inference
17 / 26
Process Summary
1. Obtain song data with chords annotated
2. Extract lyrics proper with chord annotations
3. Optional: Map lyrics into meaning categories
a. Get tagged data on meaning categories for lyrics
b. Preprocess lyrics and untagged chord annotations
c. Train to categorize over words and hypernyms
d. Classify each lyric line from chord annotations
4. Fill contingency table
5. Determine chord(s)/token associations
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
18 / 26
Analysis Overview
Individual chords with word tokens
Chord sequences with meaning category tokens
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
19 / 26
Individual chords with word tokens
Major vs. minor key differences
Avg. MI
.00034
.00005
.00030
.00008
.00176
.00018
.00071
.00032
.00039
.00542
.00068
Teas State U.
Chord
C
G
Dm
Em
F
Am
Bm
Bb
Em
Dm
C
5 Dec 2011
Word
happy
happy
happy
happy
bright
bright
sad
sad
sad
sorrow
sorrow
XY
7
4
3
2
10
3
3
2
3
2
2
O’Hara: chord meaning inference
X¬Y
1,923
2,210
341
548
971
962
197
325
1,097
342
1,928
¬XY
13
16
17
18
3
10
4
5
6
5
5
20 / 26
Chord sequences with
meaning category tokens
Most frequent chord sequence associations
Avg. MI
.0027
.0037
.0032
.0032
.0032
.0032
.0012
.0018
.0022
.0014
Teas State U.
Chord Sequence Category
D7, D7, D7, D7 Bizarre
Em, G, G6, Em Carefree
D, A, A, C#min Carefree
C#min, D, A, A Carefree
A, C#min, D, A Carefree
A, A, C#min, D Carefree
D7, G, C, G
Bizarre
C, D7, G, C
Bizarre
D, A, A, D
Powerful
C, D, C, D
Happy
5 Dec 2011
XY
30
18
14
14
14
14
14
14
13
13
O’Hara: chord meaning inference
X¬Y
36
6
2
2
2
2
17
19
8
39
¬XY
1,358
594
598
598
598
598
1,374
1,374
667
502
21 / 26
Conclusion Overview
Summary
Conclusion
Future work
References
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
22 / 26
Summary
Introduction: Lyric chord annotations for
unsupervised learning
Background: Supervised music categorization;
parallel corpora
Process: Co-occurrence statistics via contingency
tables
Analysis: Major vs. minor associations; sequence
samples
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
23 / 26
Conclusion
Can indeed learn meaning of chord sequences
from annotated lyrics
Large untapped resource now exploitable for
music recommendation
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
24 / 26
Much future work
Objective measures for evaluation
 Complication: subjectivity of chord sequence meaning
Additional aspects of music for modeling meaning
 Tempo and note sequences
 Better informed by music theory (Schmidt-Jones and Jones 2007)
Association over phrases, etc.
 Relational tuples (e.g., <guy, loses, girl>)
Songwriter aids
 Suggest chord sequences for lyrics
 And vice versa!
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
25 / 26
References
P. Fung and K. W. Church. K-vec: A new approach for aligning parallel texts. In Proc.
COLING, 1994.
X. Hu, J. S. Downie, and A. F. Ehman. Lyric text mining in music mood classification. In
Proc. ISMIR, pages 411-6, 2009.
T. Mansuy and R. Hilderman. Evaluating WordNet features in text classification models.
In Proc. FLAIRS, 2006.
A. K. McCallum. Bow: A toolkit for statistical language modeling, text retrieval,
classification and clustering. www.cs.cmu.edu/∼mccallum/bow, 1996.
C. McKay et al. Evaluating the genre classification performance of lyrical features
relative to audio, symbolic and cultural features. In Proc. ISMIR, 2010.
I. D. Melamed. Models of translational equivalence among words. Computational
Linguistics, 26(2):221-49, 2000.
G. Miller. Special issue on WordNet. International Journal of Lexicography, 3(4), 1990.
C. Schmidt-Jones and R. Jones, editors. Understanding Basic Music Theory. Connexions,
2007. http://cnx.org/content/col10363/latest.
D. Turnbull et al. Semantic annotation and retrieval of music and sound effects. IEEE
TASLP, 16 (2), 2008.
B. Whitman and D. Ellis. Automatic record reviews. In Proc. ISMIR, 2004.
Teas State U.
5 Dec 2011
O’Hara: chord meaning inference
26 / 26
Descargar

Inferring the meaning of chord sequences via lyrics