モーションキャプチャと音楽情報を用いた舞踊動作解析手法 - 池内研究室

† †† †
The Structure Analysis of Dance Motions using Motion Capture
and Musical Information
Takaaki SHIRATORI † , Atsushi NAKAZAWA †† , and Katsushi IKEUCHI †
()
1.
[1]
(metrics)
metrics
†
Institute of Industrial Science, The University of Tokyo
††
Cybermedia Center, Osaka University
[2]
()
()
[3]
X Vol. Jxx–X No. xx pp. 1–10 xxxx xx 1

xxxx/xx Vol. Jxx–X No. xx
Self Motion Element
HMM [4]
[5]
Dynamic
Time Warping
[6]
Labanotation
[7] Labanotation
Labanotation
1
()
2
1 Flash
[3]
2
2
1
2.
[8]
Kim
[9]
1
Fig. 1 Algorithm overview
3
4
5
6
3.
2

[10]
3. 1
()
3. 2
t f
d(t, f) 1
⎧
⎪⎨ max(p(t, f), p(t + 1, f)) − p ′
d(t, f) = (min(p(t, f), p(t + 1, f)) > = p ′ ) (1)
⎪⎩
0 (otherwise)
p ′ ≡ max(p(t − 1, f), p(t − 1, f ± 1)) (2)
p(t, f) t f
2 d(t, f)
1 t t + 1 2
2
Fig. 2 Onset component extraction
2
1
D(t) = ∑ d(t, f)
f
t
3. 3
3. 2 D(t)
D(t)
R DD ( 3)
R DD (τ) = 1 T∑
D(t)D(t + τ) (3)
T
t=1
R DD(τ) D(t) τ D(t + τ)
2
D(t) D(t)
τ (t rhythm )
P (t) D(t) P (t) R DP
( 4)
R DP (τ) = 1 T∑
D(t)P (t + τ) (4)
T
t=1
D(t) P (t)
t max
D(t) P (t + t max )
t max (t st )
3. 4
3. 3
3. 3 D(t)
3

xxxx/xx Vol. Jxx–X No. xx
( 3)
Fig. 4
4
Body center coordinate system
3
Fig. 3 Beat tracking considering slgiht changes of
rhythm
4.
(
)
4. 1
x
y x y
z ( 4)
()
0
0
4. 2
4. 1
0 0
( 5)
4. 1
0
( 6)
4. 3
3
4

5
Fig. 5 Segmentation candidates extraction - Hands
& CM
6
Fig. 6 Segmentation candidates extraction - Feet
(t rhythm /10)
7
1 3
2
4
7
Fig. 7 Candidates refinement with musical rhythm
Broken line : segmentation candidate
Dotted line : estimated music rhythm
4. 4
4. 3
4. 3 2
1 2
2
1 2
( 2)
5.
5. 1
3
USB
PC wav (32000Hz
16bit/sample)
Vicon Motion Systems(
)
2 2
5

xxxx/xx Vol. Jxx–X No. xx
20 ( 120fps
2400
200fps 4000
)
20
9 12
( 9)
5. 2
FFT
1024
256
0.704 85M.M. 0.576
104M.M. (M.M. Mälzel’s Metronome
1 )
8(a) 8(b)
5. 3
[8]
5. 3. 1
10 12
(a) Aizu-bandaisan dance
(b) Jongara-bushi dance
8
3. 2
D(t)
Fig. 8 Results of beat tracking
Top : amplitude, Middle : spectrum,
Bottom : D(t) as described in 3. 2
Straight line in spectrum : music rhythm
4. 4
6

1
Table 1 Result of Aizu-bandaisan dance - a Female
Dancer
(/)
13 89% (8/9) 5
9 100% (9/9) 0
2
Table 2 Result of Aizu-bandaisan dance - a Male
Dancer
(/)
3 33% (3/9) 2
9 100% (9/9) 0
4. 4 1
5. 3. 2
11
3
3
Table 3 Result of Jongara-bushi dance
(/)
6 50% (6/12) 0
9 75% (9/12) 0
4. 4 1
3
6.
(CREST)
()
[1] D. M. Gavlira: “The visual analysis of human movement”,
Journal of Computer Vision and Image Understanding,
73, 1, pp. 82–98.
[2] D. Miyazaki, T. Oishi, T. Nishikawa, R. Sagawa,
K. Nishino, T. Tomomatsu, Y. Takase and K. Ikeuchi:
7

xxxx/xx Vol. Jxx–X No. xx
(a)Aizu-bandaisan dance
9
Fig. 9
(b)Jongara-bushi dance
Keyposes extracted from manuals
10
–
Fig. 10 Results of key poses extraction - Aizu-bandaisan of a female dancer
Top : speed sequence, Middle : key poses extracted by our method,
Bottom : key poses extracted by dancers
Broken line : segmentation candidate, Straight line : music rhythm
8

11
–
Fig. 11 Results of key poses extraction - Jongara-bushi dance
Top : speed sequence, Middle : key poses extracted by our method,
Bottom : key poses extracted by dancers
Broken line : segmentation candidate, Straight line : music rhythm
“Great buddha project: Modeling cultural heritage
through observation”, In Proc. of the Sixth International
Conference on Virtual Systems and MultiMedia
(VSMM 2000), pp. 138–145 (2000).
[3] T. Flash and H. Hogan: “The coordination of arm
movements”, J. Neuroscience, pp. 1688–1703 (1985).
[4] T. Inamura, Y. Nakamura, H. Ezaki and I. Toshima:
“Imitation and primitive symbol acquisition of humanoids
by the integrated mimesis loop”, In Proc. of
International Conference on Robotics and Automation
(ICRA2001), pp. 4208–4213 (2001).
[5] O. C. Jenkins, M. J. Metaric and S. Weber: “Primitive
based movement classification for humanoid
imitation”, In Proc. of First IEEE-RAS International
Conference on Humanoid Robotics (Humanoids
2000) (2000).
[6] , , “
”, , 15 8 ,
pp. 878–885 (2000).
[7] , , “ labanotation
– labanreader
labaneditor–”,
, 38-6, pp. 61–68 (1998).
[8] A. Nakazawa, S. Nakaoka, K. Ikeuchi and K. Yokoi:
“Imitating human dance motions through motion
structure analysis”, In Proc. of International Conference
on Intelligent Robots and Systems(IROS 2002),
pp. 2539–2544 (2002).
[9] T. Kim, S. I. Park and S. Y. Shin: “Rhythmic-motion
synthesis based on motion-beat analysis”, In Proc. of
ACM SIGGRAPH 2003 (2003).
[10] M. Goto: “An audio-based real-time beat tracking
system for music with or without drum-sounds”,
Journal of New Music Research, 30, 2, pp. 159–171
(2001).
xx xx xx
2002
2004
9

xxxx/xx Vol. Jxx–X No. xx
1997
2001
()
(
)2003
1973
1978
MIT
CMU
1996
2000
D.Marr (ICCV:1990 )IEEE (CVPR:1991
) (AI Journal:1992 )Fu
(IEEE Trans. R&A1998 ) IEEE Distinguished
Lecturer(SPS 2000 - 2001CS 2004 - 2006)IEEE
Fellow
10

Abstract This paper describes a dance motion analysis method considering music rhythm. In these
days, many important intangible cultural properties are being lost because of the lack of successive performers,
and we have started our digital archiving project of cultural intangible heritages. Our goal for
archiving dance motions is to detect “primitive motions”, which consists of dance motions. In addition, we
believe these primitives must be synchronized to the musical rhythm, so our proposed method automatically
segments the original motion considering with estimated musical rhythm to detect the primitive motions.
Key words
Motion Capture, Primitive Motions, Music, Beat Tracking