Download pdf - Andrea Molino

an introduction to
MeRit 2.3
Interactive Video System
Concept and Idea:
Realisation and Programming:
With support of:
Andrea Molino / Holger Stenschke
Holger Stenschke
Cornelius Bohn & Giulio Tami
26.06.2008

MeRit 2.3 — Interactive Video System
MeRit is an interactive real time video instrument
which was developed especially with
multimedia performances in mind.
The goal was to create a system that has to:
• play back video files accurately, in highest
possible quality and with negligible starting
delay
• treat high resolution live video signals with
least possible input-to-output delay
• be extremely flexible in terms of functionality
and expandability
• be cost-effective
• be stable and reliable for show-use
• be easy to operate
• allow creative and playful interaction with
video signals (playback or live)
MeRit BLOCK DIAGRAM AV
CAM
1
4x Video Input Signals
6x Video Output
MIDI Keyboard & Trigger
CAM
2
CAM
3
Y Y Y
audio & video mixer
a / v
interface
CAM
4
av av av
a / v
interface
a / v
interface
FW FW FW
Y
av
ETH
av
ETH
master keyboard
MIDI
MIDI
Interface
USB
REMOTE
CONTROL
ethernet hub
av
ETH
ETH
ETH
a / v
interface
FW
ETH
MIDI
ETH
a / v
interface
FW
an introduction
trigger
drum trigger
module
a / v
interface
FW
Overview
AUDIO
mac
1
AUDIO
mac
2
AUDIO
mac
3
AUDIO
mac
4
AUDIO
mac
5
AUDIO
mac
6
The whole system set-up consists of:
• any number of trigger input devices (MIDIinstruments,
MIDI-fied percussion instruments
or sensors of any kind)
• any number of video input devices (cameras,
DVD Player, Computer etc.)
• a control/remote-computer to administer
and control the system
• a virtually unlimited number of video
content/playback/processing computer, depending
on the demands. Each of those
computers keeps the entire set of video files
used for a project and outputs up to nine
independent video streams through their
video output
• video / audio mixing consoles
The hardware
Standard Apple desktop computers running
OSX are required (recommended are at least
double processor computers with min. 2GHz
for video playback only or 3GHz dual quad
core for live video treatments).
Video files are either streamed directly from
disk, fast hard drives are recommended (preferably
RAID 0, with two or more channels) or
can be loaded into RAM for even faster access.
Video files can have any Apple Quicktime®
supported compression and format. Support
for multichannel audio (up to 16 channels) is
provided.
VGA
1
VGA
2
VGA
3
Live audio and video signals need to be digitised
and converted into uncompressed
streams for fastest data acquisition and best
quality. Recommendable are all-in-one broadcast
quality converters like e.g. AJA IO HD.
All computers are connected to an ethernet
hub and communicate via TCP/IP.
The software
The Software is entirely written in Max/MSP
and Jitter, using Apple Quicktime® and
OpenGL technologies.
Incoming trigger signals are either stepping
through or jumping to specific programs, organised
in scenes, or directly affect specific
video playback / realtime processing algorithms.
Each program refers to a data set that
holds information which sets playback parameters
for each video file.
All video and data content is redundant and
stored on all video servers. In case of computer
problems during a perfor-mance, the
remote computer allows to quickly re-allocate
each connected video server to the data sets of
any of the screens.
VGA
4
VGA
5
VGA
fig. 1: exemplary system scheme for a six-screen set-up
6
1 Holger Stenschke, 2008

MeRit 2.3 — Interactive Video System
an introduction
The software is divided into three modules:
A) The Composer module
The Composer module can be run on any video
content server and allows the user to configure
and program the pre-existing video content,
the allocation of the live video signals,
layout and screen distribution for a whole
show. All video signals can be distributed to
nine independent layers per screen (computer),
allocable via a separate browser window.
Per scene 9999 programmable steps are
programmable. Each step stores information
about:
• video playback file name and path or live
input signal for each layer
• x-/y-position, size and opacity for each layer
• MIDI input selection, MIDI transfer tables,
MIDI-gate times etc.
• setting the Control Matrix for routing of
trigger signals
• setting parameters for audio / video processing
(start, stop, pause, variable delay
buffer controls (slow-down, speed-up, timelapse
effect), set playback points, recall
playback points) for each layer
• basic audio controls for each layer
The Composer module allows the user to
automatically update the data sets of all connected
video content and processing servers
and the remote machine. The user interface of
the Composer is designed to visualise the resulting
appearance of a performance already
during pre-production:
B) The Player module
The Player module is a background application
that is running on each video content
server, allowing to independently play back
and process up to nine independent layers at
the same time. The number of video content
servers is basically unlimited hence the number
of simultaneous video streams is unlimited,
too.
The Player module provides a minimal playback
latency (< ~5 ms) and frame-precise synchronicity
of all running video files of all connected
video content servers.
C) The Remote module
The Remote module allows the user to administer
and control all the connected video content
servers. It gives visual feedback of any
currently played video file (file names, playback
position, next video file in list) and shows
the file loading and network connectivity status
of all servers.
fig. 3: screen shot of a part of a four layers Remote module
The Remote software does not contain or play
back video files, therefore a less powerful
computer is sufficient. All input devices are
connected (e.g. via a MIDI-interface) to the remote
computer. Incoming trigger signals are
translated and sent via TCP/IP to all the connected
video content servers.
fig. 2: screen shot of the Composer module
Furthermore, the software gives the following
controls:
• scene selection
• step number selection / manual override
• advanced MIDI-settings
• network configuration
• screen settings
• system performance settings
2 Holger Stenschke, 2008

MeRit 2.3 — Interactive Video System
an introduction
Outlook
Future releases of the software will include
administration and processing of multiple live
video inputs, soft edging for seamless blending
of screens and sub-screens. MeRit 3 will be
ported to Max5 and will have a newly designed
graphical user interfaces. (Expected for summer
2009)
3 Holger Stenschke, 2008