RMCP: Remote Music Control Protocol --- Design and Applications ---
RMCP: Remote Music Control Protocol --- Design and Applications ---
RMCP: Remote Music Control Protocol --- Design and Applications ---
Create successful ePaper yourself
Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.
No.1<br />
No.2<br />
<strong>RMCP</strong>:<br />
<strong>Remote</strong> <strong>Music</strong> <strong>Control</strong> <strong>Protocol</strong><br />
<strong>Design</strong> <strong>and</strong> <strong>Applications</strong><br />
School of Science <strong>and</strong> Engineering, Waseda University, Japan<br />
Masataka Goto Ryo Neyama Yoichi Muraoka<br />
1. Introduction<br />
No.3<br />
No.4<br />
Our Goal<br />
Integrate MIDI <strong>and</strong> computer networks<br />
Network musical applications<br />
live MIDI broadcasts<br />
remote sessions via network<br />
Computer supported sessions<br />
human-computer improvisation<br />
interactive-graphics displays<br />
A<br />
1997/09/27 ICMC97<br />
Network <strong>Protocol</strong> for <strong>Music</strong>al Information<br />
Network musical applications<br />
Internet<br />
Transmit symbolized musical information via network<br />
Distributed implementation of music-related software<br />
achieve good load-balancing<br />
exploit various facilities<br />
connected with different computers<br />
Efficient information sharing<br />
over network is important<br />
Related Work<br />
MIDI-based network protocols<br />
MIDI<br />
connection-oriented<br />
Network<br />
not emphasize low-latency information sharing<br />
among multiple distributed processes<br />
weak in efficient information sharing among devices<br />
low b<strong>and</strong>width / just for local communication<br />
Non-MIDI-based music protocols<br />
generally presuppose the use of special devices<br />
B<br />
2. <strong>RMCP</strong><br />
No.5<br />
No.6<br />
No.7<br />
No.8<br />
(<strong>Remote</strong> <strong>Music</strong> <strong>Control</strong> <strong>Protocol</strong>)<br />
Network <strong>Protocol</strong> for <strong>Music</strong> <strong>Applications</strong><br />
Share symbolized musical information via networks<br />
integrate MIDI <strong>and</strong> LAN (Ethernet)<br />
Efficient information sharing among processes<br />
connection-less protocol on UDP/IP<br />
support broadcast-based information sharing<br />
without the overhead of multiple transmission<br />
<strong>RMCP</strong> packet<br />
include various musical information<br />
Multi-server Multi-client Model<br />
<strong>RMCP</strong> clients<br />
generate <strong>RMCP</strong> packets<br />
<strong>RMCP</strong> servers<br />
<strong>RMCP</strong> server<br />
<strong>RMCP</strong> client<br />
broadcast them to all <strong>RMCP</strong> servers<br />
MIDI<br />
<strong>RMCP</strong> server<br />
connection-less one-directional transmission<br />
receive all the broadcast <strong>RMCP</strong> packets<br />
utilize them in various ways<br />
Ethernet<br />
MIDI messages<br />
note on<br />
note off<br />
LAN<br />
<strong>RMCP</strong> server<br />
<strong>RMCP</strong> client<br />
sound output / music visualization / music-driven CG<br />
add various servers without any extra packets<br />
<strong>Design</strong> Policy<br />
Implement necessary functions<br />
as small different processes<br />
so that they can be reusable<br />
Easy system implementation <strong>and</strong> expansion<br />
each process can be devoted to its own small function<br />
new function is achieved just by adding a new server<br />
Good load-balancing<br />
easy to allocate <strong>RMCP</strong> servers <strong>and</strong> clients<br />
on distributed computers<br />
Basic <strong>RMCP</strong> Servers <strong>and</strong> Clients<br />
MIDI<br />
<strong>RMCP</strong> MIDI Receiver<br />
<strong>RMCP</strong> Sound Server<br />
MIDI<br />
<strong>RMCP</strong> clients<br />
<strong>RMCP</strong> MIDI Station<br />
Ethernet<br />
<strong>RMCP</strong> Display Server<br />
<strong>RMCP</strong> servers<br />
<strong>RMCP</strong> SMF Player<br />
<strong>RMCP</strong> Animation Server
3. Implementation of <strong>RMCP</strong><br />
No.9<br />
No.10<br />
No.11<br />
No.12<br />
<strong>RMCP</strong> Packet<br />
Basic message<br />
types<br />
MIDI information:<br />
beat information:<br />
chord information:<br />
animation information:<br />
2 bytes (16 bits)<br />
<strong>RMCP</strong> magic number flag server number<br />
header length type of message<br />
message length<br />
client/user identification (R)<br />
client/user identification (G)<br />
client/user identification (B)<br />
type of time stamp<br />
time stamp (sec)<br />
reserved<br />
time stamp (msec)<br />
<strong>RMCP</strong> Message Body<br />
optional<br />
<strong>RMCP</strong> Header<br />
transmitting MIDI messages<br />
beat synchronization<br />
chord name <strong>and</strong> voicing<br />
controlling computer graphics<br />
Time Scheduling Using Time Stamps<br />
Two kinds of <strong>RMCP</strong> packets<br />
packet with time stamp<br />
received before time stamp processed on time<br />
received after time stamp processed immediately<br />
packet without time stamp processed immediately<br />
<strong>RMCP</strong> Time Synchronization Server<br />
enable <strong>RMCP</strong> servers to h<strong>and</strong>le time stamps<br />
as if all the internal clocks were synchronized<br />
Offset Table<br />
133.9.68.56 +50<br />
133.9.68.57 0<br />
133.9.68.58 +70<br />
Offset Table<br />
133.9.68.56 0<br />
133.9.68.57 -50<br />
133.9.68.58 +20<br />
IP Address:<br />
133.9.68.57<br />
Host B 1250<br />
+50<br />
1050<br />
<strong>RMCP</strong> over the Internet<br />
<strong>RMCP</strong> Gateway<br />
IP Address:<br />
133.9.68.56<br />
1200 Host A<br />
-20<br />
1000<br />
LAN<br />
IP Address:<br />
133.9.68.58<br />
1180 Host C<br />
<strong>RMCP</strong> client<br />
<strong>RMCP</strong> Gateway<br />
Host B 1350<br />
1050<br />
Host A 1300<br />
-50<br />
-20<br />
Host E 3780 Host D 3800<br />
<strong>RMCP</strong> server<br />
<strong>RMCP</strong> Gateway<br />
1480<br />
LAN<br />
Offset Table<br />
133.9.68.56 -20<br />
133.9.68.57 -70<br />
133.9.68.58 0<br />
980<br />
bidirectional relay of <strong>RMCP</strong> packets over the Internet<br />
connection-oriented protocol on TCP/IP<br />
<strong>RMCP</strong> servers <strong>and</strong> clients can communicate<br />
as if different LANs were the same network<br />
relay with the<br />
specified latency<br />
live MIDI broadcasts<br />
<strong>Remote</strong>GIG<br />
Experimental Results<br />
+2000<br />
+500<br />
<strong>RMCP</strong> programming libraries in both C <strong>and</strong> Java<br />
tested on various computers <strong>and</strong> OSs<br />
IRIX-5.3<br />
HP-UX<br />
Communication delay between server <strong>and</strong> client<br />
Ave:<br />
SD:<br />
0.30 ms<br />
0.06 ms<br />
IRIX-6.2<br />
Linux-2.0<br />
Min:<br />
Max:<br />
0.28 ms<br />
1.24 ms<br />
Fast enough compared with MIDI (31.25 Kbps)<br />
1500<br />
1000<br />
connection<br />
specified latency for relay<br />
internal clock difference compensation<br />
WAN<br />
+2000<br />
+/- 500<br />
Solaris-2.5 SunOS-4.1.3<br />
Windows-95 Windows-NT<br />
10Mbps Ethernet<br />
note on LAN<br />
client server<br />
SGI Indigo2 (IRIX-5.3)<br />
4. <strong>Applications</strong><br />
A<br />
Networked Session<br />
Several players perform music together via Ethernet<br />
<strong>RMCP</strong> MIDI Station<br />
or<br />
<strong>RMCP</strong> MIDI Receiver<br />
No.13<br />
No.14<br />
No.15<br />
No.16<br />
listen to other players’ performances<br />
see visualized performances<br />
<strong>RMCP</strong> Display Server<br />
SMF B<br />
<strong>RMCP</strong> SMF Player<br />
Networked Session<br />
Improvision<br />
A<br />
LAN<br />
<strong>RMCP</strong> Sound Server<br />
<strong>RMCP</strong> Display Server<br />
<strong>RMCP</strong> MIDI Station<br />
or<br />
<strong>RMCP</strong> MIDI Receiver<br />
<strong>Music</strong>al-instrument interface for untrained novices<br />
improvise unconventional music easily<br />
by clicking <strong>and</strong> dragging a computer mouse<br />
Improvision<br />
<strong>RMCP</strong> Improvision<br />
Improvision<br />
B<br />
<strong>RMCP</strong> Improvision<br />
<strong>RMCP</strong> Improvision Display Server<br />
Improvision<br />
A B C<br />
LAN<br />
<strong>RMCP</strong> Sound Server<br />
direction:<br />
speed:<br />
Improvision<br />
pitch<br />
volume<br />
C<br />
<strong>RMCP</strong> Improvision
No.17<br />
No.18<br />
No.19<br />
No.20<br />
VirJa Session<br />
Virtual jazz session system<br />
Virtual Jazz Trio<br />
Player 2: Bassist<br />
Human<br />
Computer<br />
multimodal interaction among all players<br />
Real Camera<br />
Real Camera<br />
Virtual Camera<br />
Player 1: Pianist<br />
Human<br />
Computer<br />
Player 3: Drummer<br />
Virtual Camera<br />
Real-time CG Real-time CG<br />
VirJa Session<br />
CG Dancer<br />
sounds / 3D computer graphics / gestures<br />
Human<br />
Computer<br />
Virtual Dancer ‘‘Cindy’’<br />
Sound<br />
Interactive Performance<br />
CG<br />
Player A<br />
Drummer<br />
Sound<br />
Sound<br />
CG<br />
Sound<br />
Player B<br />
Guitarist<br />
Pianist<br />
Player 2: Bassist<br />
Human<br />
Computer<br />
<strong>RMCP</strong> Player Server<br />
Bassist<br />
Pianist<br />
Player 1: Pianist<br />
Human<br />
Computer<br />
<strong>RMCP</strong> MIDI Receiver <strong>RMCP</strong> Sound Server<br />
<strong>RMCP</strong> Camera Analyzer<br />
<strong>RMCP</strong> Beat Provider<br />
Ethernet<br />
Player 3: Drummer<br />
Human<br />
Computer<br />
<strong>RMCP</strong> Player Server<br />
<strong>RMCP</strong> Beat Provider<br />
<strong>RMCP</strong> Animation Server <strong>RMCP</strong> Animation Server<br />
Drummer<br />
Interactive performance of a music-controlled CG dancer<br />
two players choreograph Cindy by their improvisation<br />
interact through music <strong>and</strong> 3D computer animation<br />
Virtual Dancer ‘‘Cindy’’<br />
MIDI Instrument<br />
MIDI<br />
CG Dancer<br />
<strong>RMCP</strong> Sound Server <strong>RMCP</strong> Animation Server<br />
<strong>RMCP</strong> MIDI Receiver<br />
<strong>RMCP</strong> <strong>Music</strong> Analyzer<br />
<strong>RMCP</strong> MIDI Receiver<br />
MIDI MIDI<br />
Player A<br />
Player B<br />
Drums<br />
Piano<br />
Guitar<br />
No.21<br />
No.22<br />
No.23<br />
<strong>Remote</strong>GIG<br />
<strong>Remote</strong> session with delay<br />
impossible to avoid network latency over the Internet<br />
models of traditional sessions are not useful<br />
<strong>RMCP</strong> gateways provide a certain constant latency<br />
with very small deviation<br />
Innovative remote session over the Internet<br />
overcome long network latency<br />
offer a new possibility for future remote sessions<br />
<strong>Remote</strong>GIG<br />
Assumption<br />
Site B<br />
Player B<br />
tempo: constant<br />
chord progression: repetitive (ex. 12-bar blues)<br />
Turn the network latency to its advantage<br />
other players’ performances are delayed<br />
for the constant period of chord progression<br />
A<br />
1 period 1 period 1 period<br />
time<br />
Site A<br />
Player A A-1 A-2 A-3<br />
B-1 B-2<br />
A-1 A-2<br />
B-1 B-2 B-3<br />
playing background drums to keep the tempo<br />
delayed performances can fit the chords<br />
5. Conclusion<br />
No.24<br />
Summary<br />
<strong>RMCP</strong>: Network protocol for music applications<br />
Multiple distributed processes can share<br />
symbolized musical information such as MIDI<br />
Features:<br />
efficient broadcast-based information sharing over LAN<br />
time-scheduling using time stamps<br />
live MIDI transmission over WAN such as Internet<br />
Utilized for various applications<br />
Future Work<br />
Provide an API ‘‘Jam’’ to make<br />
<strong>RMCP</strong> <strong>and</strong> MIDI usable in Java applets<br />
Implement various <strong>RMCP</strong>-based applications<br />
<strong>RMCP</strong> Software Package Distribution<br />
http://www.info.waseda.ac.jp/muraoka/members/goto/<strong>RMCP</strong>/<br />
basic <strong>RMCP</strong> server <strong>and</strong> client programs<br />
<strong>RMCP</strong> programming libraries<br />
B
Change language