04.03.2013 Views

RMCP: Remote Music Control Protocol --- Design and Applications ---

RMCP: Remote Music Control Protocol --- Design and Applications ---

RMCP: Remote Music Control Protocol --- Design and Applications ---

SHOW MORE
SHOW LESS

Create successful ePaper yourself

Turn your PDF publications into a flip-book with our unique Google optimized e-Paper software.

No.1<br />

No.2<br />

<strong>RMCP</strong>:<br />

<strong>Remote</strong> <strong>Music</strong> <strong>Control</strong> <strong>Protocol</strong><br />

<strong>Design</strong> <strong>and</strong> <strong>Applications</strong><br />

School of Science <strong>and</strong> Engineering, Waseda University, Japan<br />

Masataka Goto Ryo Neyama Yoichi Muraoka<br />

1. Introduction<br />

No.3<br />

No.4<br />

Our Goal<br />

Integrate MIDI <strong>and</strong> computer networks<br />

Network musical applications<br />

live MIDI broadcasts<br />

remote sessions via network<br />

Computer supported sessions<br />

human-computer improvisation<br />

interactive-graphics displays<br />

A<br />

1997/09/27 ICMC97<br />

Network <strong>Protocol</strong> for <strong>Music</strong>al Information<br />

Network musical applications<br />

Internet<br />

Transmit symbolized musical information via network<br />

Distributed implementation of music-related software<br />

achieve good load-balancing<br />

exploit various facilities<br />

connected with different computers<br />

Efficient information sharing<br />

over network is important<br />

Related Work<br />

MIDI-based network protocols<br />

MIDI<br />

connection-oriented<br />

Network<br />

not emphasize low-latency information sharing<br />

among multiple distributed processes<br />

weak in efficient information sharing among devices<br />

low b<strong>and</strong>width / just for local communication<br />

Non-MIDI-based music protocols<br />

generally presuppose the use of special devices<br />

B<br />

2. <strong>RMCP</strong><br />

No.5<br />

No.6<br />

No.7<br />

No.8<br />

(<strong>Remote</strong> <strong>Music</strong> <strong>Control</strong> <strong>Protocol</strong>)<br />

Network <strong>Protocol</strong> for <strong>Music</strong> <strong>Applications</strong><br />

Share symbolized musical information via networks<br />

integrate MIDI <strong>and</strong> LAN (Ethernet)<br />

Efficient information sharing among processes<br />

connection-less protocol on UDP/IP<br />

support broadcast-based information sharing<br />

without the overhead of multiple transmission<br />

<strong>RMCP</strong> packet<br />

include various musical information<br />

Multi-server Multi-client Model<br />

<strong>RMCP</strong> clients<br />

generate <strong>RMCP</strong> packets<br />

<strong>RMCP</strong> servers<br />

<strong>RMCP</strong> server<br />

<strong>RMCP</strong> client<br />

broadcast them to all <strong>RMCP</strong> servers<br />

MIDI<br />

<strong>RMCP</strong> server<br />

connection-less one-directional transmission<br />

receive all the broadcast <strong>RMCP</strong> packets<br />

utilize them in various ways<br />

Ethernet<br />

MIDI messages<br />

note on<br />

note off<br />

LAN<br />

<strong>RMCP</strong> server<br />

<strong>RMCP</strong> client<br />

sound output / music visualization / music-driven CG<br />

add various servers without any extra packets<br />

<strong>Design</strong> Policy<br />

Implement necessary functions<br />

as small different processes<br />

so that they can be reusable<br />

Easy system implementation <strong>and</strong> expansion<br />

each process can be devoted to its own small function<br />

new function is achieved just by adding a new server<br />

Good load-balancing<br />

easy to allocate <strong>RMCP</strong> servers <strong>and</strong> clients<br />

on distributed computers<br />

Basic <strong>RMCP</strong> Servers <strong>and</strong> Clients<br />

MIDI<br />

<strong>RMCP</strong> MIDI Receiver<br />

<strong>RMCP</strong> Sound Server<br />

MIDI<br />

<strong>RMCP</strong> clients<br />

<strong>RMCP</strong> MIDI Station<br />

Ethernet<br />

<strong>RMCP</strong> Display Server<br />

<strong>RMCP</strong> servers<br />

<strong>RMCP</strong> SMF Player<br />

<strong>RMCP</strong> Animation Server


3. Implementation of <strong>RMCP</strong><br />

No.9<br />

No.10<br />

No.11<br />

No.12<br />

<strong>RMCP</strong> Packet<br />

Basic message<br />

types<br />

MIDI information:<br />

beat information:<br />

chord information:<br />

animation information:<br />

2 bytes (16 bits)<br />

<strong>RMCP</strong> magic number flag server number<br />

header length type of message<br />

message length<br />

client/user identification (R)<br />

client/user identification (G)<br />

client/user identification (B)<br />

type of time stamp<br />

time stamp (sec)<br />

reserved<br />

time stamp (msec)<br />

<strong>RMCP</strong> Message Body<br />

optional<br />

<strong>RMCP</strong> Header<br />

transmitting MIDI messages<br />

beat synchronization<br />

chord name <strong>and</strong> voicing<br />

controlling computer graphics<br />

Time Scheduling Using Time Stamps<br />

Two kinds of <strong>RMCP</strong> packets<br />

packet with time stamp<br />

received before time stamp processed on time<br />

received after time stamp processed immediately<br />

packet without time stamp processed immediately<br />

<strong>RMCP</strong> Time Synchronization Server<br />

enable <strong>RMCP</strong> servers to h<strong>and</strong>le time stamps<br />

as if all the internal clocks were synchronized<br />

Offset Table<br />

133.9.68.56 +50<br />

133.9.68.57 0<br />

133.9.68.58 +70<br />

Offset Table<br />

133.9.68.56 0<br />

133.9.68.57 -50<br />

133.9.68.58 +20<br />

IP Address:<br />

133.9.68.57<br />

Host B 1250<br />

+50<br />

1050<br />

<strong>RMCP</strong> over the Internet<br />

<strong>RMCP</strong> Gateway<br />

IP Address:<br />

133.9.68.56<br />

1200 Host A<br />

-20<br />

1000<br />

LAN<br />

IP Address:<br />

133.9.68.58<br />

1180 Host C<br />

<strong>RMCP</strong> client<br />

<strong>RMCP</strong> Gateway<br />

Host B 1350<br />

1050<br />

Host A 1300<br />

-50<br />

-20<br />

Host E 3780 Host D 3800<br />

<strong>RMCP</strong> server<br />

<strong>RMCP</strong> Gateway<br />

1480<br />

LAN<br />

Offset Table<br />

133.9.68.56 -20<br />

133.9.68.57 -70<br />

133.9.68.58 0<br />

980<br />

bidirectional relay of <strong>RMCP</strong> packets over the Internet<br />

connection-oriented protocol on TCP/IP<br />

<strong>RMCP</strong> servers <strong>and</strong> clients can communicate<br />

as if different LANs were the same network<br />

relay with the<br />

specified latency<br />

live MIDI broadcasts<br />

<strong>Remote</strong>GIG<br />

Experimental Results<br />

+2000<br />

+500<br />

<strong>RMCP</strong> programming libraries in both C <strong>and</strong> Java<br />

tested on various computers <strong>and</strong> OSs<br />

IRIX-5.3<br />

HP-UX<br />

Communication delay between server <strong>and</strong> client<br />

Ave:<br />

SD:<br />

0.30 ms<br />

0.06 ms<br />

IRIX-6.2<br />

Linux-2.0<br />

Min:<br />

Max:<br />

0.28 ms<br />

1.24 ms<br />

Fast enough compared with MIDI (31.25 Kbps)<br />

1500<br />

1000<br />

connection<br />

specified latency for relay<br />

internal clock difference compensation<br />

WAN<br />

+2000<br />

+/- 500<br />

Solaris-2.5 SunOS-4.1.3<br />

Windows-95 Windows-NT<br />

10Mbps Ethernet<br />

note on LAN<br />

client server<br />

SGI Indigo2 (IRIX-5.3)<br />

4. <strong>Applications</strong><br />

A<br />

Networked Session<br />

Several players perform music together via Ethernet<br />

<strong>RMCP</strong> MIDI Station<br />

or<br />

<strong>RMCP</strong> MIDI Receiver<br />

No.13<br />

No.14<br />

No.15<br />

No.16<br />

listen to other players’ performances<br />

see visualized performances<br />

<strong>RMCP</strong> Display Server<br />

SMF B<br />

<strong>RMCP</strong> SMF Player<br />

Networked Session<br />

Improvision<br />

A<br />

LAN<br />

<strong>RMCP</strong> Sound Server<br />

<strong>RMCP</strong> Display Server<br />

<strong>RMCP</strong> MIDI Station<br />

or<br />

<strong>RMCP</strong> MIDI Receiver<br />

<strong>Music</strong>al-instrument interface for untrained novices<br />

improvise unconventional music easily<br />

by clicking <strong>and</strong> dragging a computer mouse<br />

Improvision<br />

<strong>RMCP</strong> Improvision<br />

Improvision<br />

B<br />

<strong>RMCP</strong> Improvision<br />

<strong>RMCP</strong> Improvision Display Server<br />

Improvision<br />

A B C<br />

LAN<br />

<strong>RMCP</strong> Sound Server<br />

direction:<br />

speed:<br />

Improvision<br />

pitch<br />

volume<br />

C<br />

<strong>RMCP</strong> Improvision


No.17<br />

No.18<br />

No.19<br />

No.20<br />

VirJa Session<br />

Virtual jazz session system<br />

Virtual Jazz Trio<br />

Player 2: Bassist<br />

Human<br />

Computer<br />

multimodal interaction among all players<br />

Real Camera<br />

Real Camera<br />

Virtual Camera<br />

Player 1: Pianist<br />

Human<br />

Computer<br />

Player 3: Drummer<br />

Virtual Camera<br />

Real-time CG Real-time CG<br />

VirJa Session<br />

CG Dancer<br />

sounds / 3D computer graphics / gestures<br />

Human<br />

Computer<br />

Virtual Dancer ‘‘Cindy’’<br />

Sound<br />

Interactive Performance<br />

CG<br />

Player A<br />

Drummer<br />

Sound<br />

Sound<br />

CG<br />

Sound<br />

Player B<br />

Guitarist<br />

Pianist<br />

Player 2: Bassist<br />

Human<br />

Computer<br />

<strong>RMCP</strong> Player Server<br />

Bassist<br />

Pianist<br />

Player 1: Pianist<br />

Human<br />

Computer<br />

<strong>RMCP</strong> MIDI Receiver <strong>RMCP</strong> Sound Server<br />

<strong>RMCP</strong> Camera Analyzer<br />

<strong>RMCP</strong> Beat Provider<br />

Ethernet<br />

Player 3: Drummer<br />

Human<br />

Computer<br />

<strong>RMCP</strong> Player Server<br />

<strong>RMCP</strong> Beat Provider<br />

<strong>RMCP</strong> Animation Server <strong>RMCP</strong> Animation Server<br />

Drummer<br />

Interactive performance of a music-controlled CG dancer<br />

two players choreograph Cindy by their improvisation<br />

interact through music <strong>and</strong> 3D computer animation<br />

Virtual Dancer ‘‘Cindy’’<br />

MIDI Instrument<br />

MIDI<br />

CG Dancer<br />

<strong>RMCP</strong> Sound Server <strong>RMCP</strong> Animation Server<br />

<strong>RMCP</strong> MIDI Receiver<br />

<strong>RMCP</strong> <strong>Music</strong> Analyzer<br />

<strong>RMCP</strong> MIDI Receiver<br />

MIDI MIDI<br />

Player A<br />

Player B<br />

Drums<br />

Piano<br />

Guitar<br />

No.21<br />

No.22<br />

No.23<br />

<strong>Remote</strong>GIG<br />

<strong>Remote</strong> session with delay<br />

impossible to avoid network latency over the Internet<br />

models of traditional sessions are not useful<br />

<strong>RMCP</strong> gateways provide a certain constant latency<br />

with very small deviation<br />

Innovative remote session over the Internet<br />

overcome long network latency<br />

offer a new possibility for future remote sessions<br />

<strong>Remote</strong>GIG<br />

Assumption<br />

Site B<br />

Player B<br />

tempo: constant<br />

chord progression: repetitive (ex. 12-bar blues)<br />

Turn the network latency to its advantage<br />

other players’ performances are delayed<br />

for the constant period of chord progression<br />

A<br />

1 period 1 period 1 period<br />

time<br />

Site A<br />

Player A A-1 A-2 A-3<br />

B-1 B-2<br />

A-1 A-2<br />

B-1 B-2 B-3<br />

playing background drums to keep the tempo<br />

delayed performances can fit the chords<br />

5. Conclusion<br />

No.24<br />

Summary<br />

<strong>RMCP</strong>: Network protocol for music applications<br />

Multiple distributed processes can share<br />

symbolized musical information such as MIDI<br />

Features:<br />

efficient broadcast-based information sharing over LAN<br />

time-scheduling using time stamps<br />

live MIDI transmission over WAN such as Internet<br />

Utilized for various applications<br />

Future Work<br />

Provide an API ‘‘Jam’’ to make<br />

<strong>RMCP</strong> <strong>and</strong> MIDI usable in Java applets<br />

Implement various <strong>RMCP</strong>-based applications<br />

<strong>RMCP</strong> Software Package Distribution<br />

http://www.info.waseda.ac.jp/muraoka/members/goto/<strong>RMCP</strong>/<br />

basic <strong>RMCP</strong> server <strong>and</strong> client programs<br />

<strong>RMCP</strong> programming libraries<br />

B

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!