Applications of Digital Audio Effects in Broadcast - Visual Soundscape
Applications of Digital Audio Effects in Broadcast:
Yesterday, Today and Tomorrow!
!
Frank Melchior!
Lead Technologist – Audio!
BBC Research & Development!
Research & Development
1
© BBC MMXII
Re-launch of audio research at the BBC
www.bbc.co.uk/rd/audio!
Research & Development
2
© BBC MMXII
BBC Audio Research Partnership
www.bbc.co.uk/rd/audio!
Research & Development
3
© BBC MMXII
Digital audio effects (DAFX)!
Input
signal
DAFX
Output
signal
Acoustical
and visual
representation
Control
parameters
Acoustical
and visual
representation
[Arfib, 1999]
Research & Development
4
© BBC MMXII
Original image from: http://en.wikipedia.org/wiki/File:2LO.jpg
Sound file from: http://www.radiorewind.co.uk/sounds/2lo_calling.mp3
Research & Development
5
© BBC MMXII
Historical broadcast chain!
Research & Development
Broadcasting House 1932
6
© BBC MMXII
Research & Development
7
© BBC MMXII
Historical broadcast chain (1935)!
Studio (entirely passive)!
Central Technical Area
(valves)!
Microphone!
Mic Amp!
EQ!
Channel Fader!
Mixing Bus
Amp!
Main Fader!
Mixer Output
Amp!
Line Send
Amp!
Transmitter!
Picture provided by Rupert Brun
Speaker!
Speaker Amp!
Research & Development
8
© BBC MMXII
Research & Development
9
http://www.bbc.co.uk/programmes/galleries/p00nn0vh
© BBC MMXII
The BBC Radiophonic workshop!
In 1957, a small group of BBC producers and studio managers began using
‘radiophonic’ techniques to create pioneering music and drama programmes.
The process consisted of recording real sounds, such as those created by the
human voice, bottles, bells, musical instruments, percussion devices or even
boxes of gravel or pebbles.
These were then manipulated to produce entirely new material. Tape
machines provided reverse playback, speed and pitch changes, or were
used to create sound loops, whilst reverberation and equalisation could
modify the sound quality.
Various elements of the work were edited together using tape-splicing
techniques, often note-by-note. This was a time-consuming business,
requiring endless skill and patience, but the results were often very impressive.
These processes, similar to musique concrète, created or enhanced the
atmosphere in a programme, but weren’t considered an ‘art in itself’.
!
Research & Development
10
© BBC MMXII
The BBC Radiophonic workshop!
An early view looking north through Rooms 14 and 13. Daphne Oram is cueing up a disc on one of the turntables on a BBC TD/7 gramophone; note the
parallel-tracking arms. To the right, Desmond Briscoe is editing a tape on a Ferrograph recorder, alongside 'Dickie' Bird. The equipment bays on the left
remained in use until the late seventies. The turntables in the foreground don't appear to be a standard BBC variety. (http://whitefiles.org/rwg/large-11.html)
Research & Development
11
© BBC MMXII
The BBC Radiophonic workshop!
An early picture, taken onthe 13th of May, 1958, looking through the window of Room 15 towards Rooms 13 and 14. The room on this side is the recording
area, complete with microphone. In the distance there's Desmond Briscoe with a tape loop, with a Motosacoche tape machine beyond him, 'Dickie' Bird the
engineer and Donald McWhinnie, in charge, standing in front of a Ferrograph recorder. Daphne Oram is at the controls of the 'Albert Hall' mixer, on top of
which there are a pair of PEUs. Note the flowers on the loudspeaker.
Research & Development
12
© BBC MMXII
The BBC Radiophonic workshop – Tape Loops!
Dick Mills, who joined the Workshop in 1958, sits in front of an EMI
BTR/2 tape machine, as he keeps his eye on a tape loop running
through a machine operated by John Harrison. This picture was
taken in 1960. (http://whitefiles.org/rwg/large-17.html)
EMI TR/90 tape machine was equipped
with the Tempophon, whose spinning
head allowed the pitch of a recording to
be changed without changing the tempo.
Research & Development
13
© BBC MMXII
The BBC Radiophonic workshop – Oramics!
Research & Development
14
© BBC MMXII
The BBC Radiophonic workshop – Oramics!
Video from: http://news.bbc.co.uk/1/hi/programmes/click_online/9671950.stm
Research & Development
15
© BBC MMXII
The virtual Radiophonic workshop!
Research & Development
16
© BBC MMXII
The virtual Radiophonic workshop!
Research & Development
17
Chris Lowis and colleagues
© BBC MMXII
The virtual Radiophonic workshop – Demo!
Chris Lowis and colleagues
http://www.bbc.co.uk/blogs/researchanddevelopment/2012/05/web-audio-radiophonics-1.shtml!
Research & Development
18
© BBC MMXII
Current broadcast chain (2012)!
Analogue
Digital
Microphone!
Mic Amp!
A/D!
Mixer DSP! Contribution
(EQ, compression, limiting etc)! Lossy Codec
(e.g. ISDN, 3G
etc)!
Router /
SRC!
Speaker!
Speaker Amp!
D/A!
Transmitter!
Storage!
(May include lossy
codec)!
DAW (may
include SRC,
bit depth
conversion, eq,
compression,
limiting etc)!
Picture provided by Rupert Brun
Research & Development
Mixer DSP!
(EQ, compression,
limiting etc)!
Tx Processor
(multi-band
compression)!
19
Router / SRC!
Distribution
Lossy Codec!
Emission
Lossy Codec!
Emmission
Lossy Codec!
IP Platforms!
© BBC MMXII
Current broadcast chain (2012)!
Analogue
Digital
Microphone!
Mic Amp!
A/D!
Mixer DSP! Contribution
(EQ, compression, limiting etc)! Lossy Codec
(e.g. ISDN, 3G
etc)!
Router /
SRC!
Speaker!
Speaker Amp!
D/A!
Transmitter!
Storage!
(May include lossy
codec)!
DAW (may
include SRC,
bit depth
conversion, eq,
compression,
limiting etc)!
Picture provided by Rupert Brun
Research & Development
Mixer DSP!
(EQ, compression,
limiting etc)!
Tx Processor
(multi-band
compression)!
20
Router / SRC!
Distribution
Lossy Codec!
Emission
Lossy Codec!
Emmission
Lossy Codec!
IP Platforms!
© BBC MMXII
!
How can we successfully turn a creative idea into
an experience which will be delivered and reproduced
using different reproduction systems in
uncontrolled environments including interaction
and environment awareness? !
!
!
!
!
!
!
Research ! & Development
21
© BBC MMXII
virtual sound scene
Production - Storage - Perception!
virtual
sound event
audio signals
recording room
virtual
sound event
DSP
diffuse
sound event
recording/
playback
scene
representation
DSP!
sound
editor
sound event
sound scene
scene
representation
Research & Development
22
© BBC MMXII
virtual
sound event
Production - Storage - Perception!
virtual
sound event
DSP
DSP!
recording/
playback
scene
representation
sound
editor
scene
representation
virtual sound scene
Research & Development
23
© BBC MMXII
Production - Storage - Perception!
DSP!
scene
representation
virtual sound scene
Research & Development
24
© BBC MMXII
listening room
Production - Storage - Perception!
virtual sound scene
recording room
virtual
sound event
audio signals
auditory scene
virtual
sound event
DSP
DSP!
recording/
playback
scene
representation
sound
editor
scene
representation
Research & Development
25
© BBC MMXII
Digital Audio Effects (DAFX)!
Input
signal
DAFX
Output
signal
Acoustical
and visual
representation
Control
parameters
Acoustical
and visual
representation
Diagram after [Arfib, 1999]
Research & Development
26
© BBC MMXII
Digital Audio Effects (DAFX)!
DAFX
Acoustical
and visual
representation
Control
parameters
Acoustical
and visual
representation
Research & Development
27
© BBC MMXII
Digital Audio Scene Effects (DASFX)!
DASFX
Acoustical
and visual
representation
Control
parameters
Acoustical
and visual
representation
Research & Development
28
© BBC MMXII
Digital Audio Scene Effects (DASFX)!
Isn’t this
mastering?!
Isn’t this just
DAFX for
mixed signal?!
It is different because a combination of scene description
and audio processing is used for visualisation and both
elements are modified.!
Research & Development
29
© BBC MMXII
Object-based audio formats!
Research & Development
30
© BBC MMXII
Object-based application in broadcast: NetMix!
Commentary!
Player!
Court atmosphere!
Pictures provided by Rupert Brun
Research & Development
31
© BBC MMXII
Object-based application in broadcast: Perceptive Media!
!
Research & Development
32
© BBC MMXII
Object-based application in broadcast: Perceptive Media!
!
• Experimental radio drama in a new editorial
format!
• Integration of listener information into the
story line!
• Client side audio procession and
personalization!
• Object-based audio storage and delivery!
http://www.futurebroadcasts.com/
• HTML 5 API for audio processing!
!
!
!
Research & Development
33
© BBC MMXII
Digital Audio Scene Effects (DASFX)!
DASFX
Acoustical
and visual
representation
Control
parameters
Acoustical
and visual
representation
Research & Development
34
© BBC MMXII
Digital Audio Scene Effects – Spatial Mastering!
Recording! Mixing! Mastering!
Mastering!
• Modification of the overall impression of an auditory scene!
• Compilation of difference scenes to one distribution package!
• Channel-based: Adjusting the final speaker signals!
• Object-based: Adjusting the audio signals and/or scene description!
• Spatial: Direction dependent processing!
!
[Melchior, 2011]
Research & Development
35
© BBC MMXII
Digital Audio Scene Effects – Spatial Mastering!
!
• “I want to have "
more brightness in the front.”!
• “I want to have "
a darker sound in the left corner.”!
• “I want to have "
more reverb in the back.”!
!
Research & Development
!
36
[Melchior, 2011]
© BBC MMXII
Digital Audio Scene Effects – Spatial Mastering!
Research & Development
37
[Melchior, 2011]
© BBC MMXII
Digital Audio Scene Effects – Spatial Mastering!
Research & Development
38
[Melchior, 2011]
© BBC MMXII
Digital Audio Scene Effects – Spatial Mastering!
Directional Function!
Rotation!
Spreading!
Directional!
emphasis!
Research & Development
39
[Melchior, 2011]
© BBC MMXII
Digital Audio Scene Effects – Spatial Mastering!
Research & Development
40
[Melchior, 2011]
© BBC MMXII
Digital Audio Scene Effects – Reactive Source!
!
• Use low level audio features to modify control parameter!
• Represent the source cloud as a virtual scene to make it format
agnostic!
• Create a wide source using a number of uncorrelated sources!
!
!
!
Research & Development
41
© BBC MMXII
Digital Audio Scene Effects – Reactive Source!
!
• Use low level audio features to modify control parameter!
• Represent the source cloud as a virtual scene to make it format
agnostic!
• Create a wide source using a number of uncorrelated sources!
!
!
!
Research & Development
42
© BBC MMXII
Digital Audio Scene Effects – Reactive Source!
g 2 (n)
Adaptive
audio processing
Format agnostic
scene
Reproduction system
adaptation
Feature
extraction
Feature
mapping
Auditory
scene
x 1 (n)
Digital audio
effect
:
y' 1 (n)
y' n (n)
Realtime
rendering
y 1 (n)
y m (n)
Research & Development
43
[Melchior, 2012]
© BBC MMXII
Digital Audio Scene Effects – Reactive Source!
!
• Use low level audio features to modify control parameter!
• Represent the source cloud as a virtual scene to make it format
agnostic!
• Create a wide source using a number of uncorrelated sources!
!
!
!
Research & Development
44
© BBC MMXII
Digital Audio Scene Effects – Reactive Source!
g 1 g 2
g 3
Pitch
extraction
Pitch
mapping
Loudness
extraction
Loudness
mapping
Modulation
matrix
Crossover
calculation
Source
cloud
x 1 (n)
Filterbank
:
y' 1 (n)
y' 5 (n)
Amplitude
panning
y l (n)
y r (n)
[Melchior, 2012]
Research & Development
45
© BBC MMXII
Digital Audio Scene Effects – Reactive Source!
Research & Development
46
[Melchior, 2012]
© BBC MMXII
Digital Audio Effects (DAFX)!
Input
signal
DAFX
Output
signal
Acoustical
and visual
representation
Control
parameters
Acoustical
and visual
representation
[Arfib, 1999]
Research & Development
47
© BBC MMXII
Digital Audio Effects (DAFX)!
Input
signal
DAFX
Output
signal
Acoustical
and visual
representation
Control
parameters
Acoustical
and visual
representation
Research & Development
48
© BBC MMXII
Digital Audio Effects (DAFX)!
Input
signal
DAFX
Output
signal
Perceptual model
Acoustical
and visual
representation
Control
parameters
Acoustical
and visual
representation
Research & Development
49
© BBC MMXII
Thank you for your kind attention.
Special thanks to Rupert Brun (BBC Audio and Music) for providing the block
diagrams of the broadcasting chain. Thanks to Christoph Lowis (BBC Research and
Development) for providing the Radiophonic HTML 5 prototypes.!
Research & Development
50
© BBC MMXII
References and Acknowledgement!
!
[Arfib, 1999] ! !D. Arfib. Visual representations for digital audio effects and their control. In Proc. DAFX-99 Digital Audio Effects
! !Workshop, pp. 63-68, Trondheim, December 1999!
!
[Niebur, 2010] ! !L. Niebur, Special Sound The Creation and Legacy of the BBC Radiophonic Workshop, Oxford University !
! !Press, 2010!
!
[Melchior, 2011] !F. Melchior, U. Michaelis, R. Steffens , “Spatial Mastering – a new concept for spatial sound design in !
! !object-based audio scene”,, Paper presented at the ICMC 2011!
!
[Melchior, 2012] !F. Melchior, “The Reactive Source: A Reproduction Format Agnostic and Adaptive Spatial Audio Effect”, e-brief !
! !accepted for publication at the 133 rd AES Convention, 2012!
!
!
!
!
!
!
!
!
!
!
Research & Development
51
© BBC MMXII
Change language