Applications of Digital Audio Effects in Broadcast - Visual Soundscape

visual.soundscape.de

Applications of Digital Audio Effects in Broadcast - Visual Soundscape

Applications of Digital Audio Effects in Broadcast:

Yesterday, Today and Tomorrow!

!

Frank Melchior!

Lead Technologist – Audio!

BBC Research & Development!

Research & Development

1

© BBC MMXII


Re-launch of audio research at the BBC

www.bbc.co.uk/rd/audio!

Research & Development

2

© BBC MMXII


BBC Audio Research Partnership

www.bbc.co.uk/rd/audio!

Research & Development

3

© BBC MMXII


Digital audio effects (DAFX)!

Input

signal

DAFX

Output

signal

Acoustical

and visual

representation

Control

parameters

Acoustical

and visual

representation

[Arfib, 1999]

Research & Development

4

© BBC MMXII


Original image from: http://en.wikipedia.org/wiki/File:2LO.jpg

Sound file from: http://www.radiorewind.co.uk/sounds/2lo_calling.mp3

Research & Development

5

© BBC MMXII


Historical broadcast chain!

Research & Development

Broadcasting House 1932

6

© BBC MMXII


Research & Development

7

© BBC MMXII


Historical broadcast chain (1935)!

Studio (entirely passive)!

Central Technical Area

(valves)!

Microphone!

Mic Amp!

EQ!

Channel Fader!

Mixing Bus

Amp!

Main Fader!

Mixer Output

Amp!

Line Send

Amp!

Transmitter!

Picture provided by Rupert Brun

Speaker!

Speaker Amp!

Research & Development

8

© BBC MMXII


Research & Development

9

http://www.bbc.co.uk/programmes/galleries/p00nn0vh

© BBC MMXII


The BBC Radiophonic workshop!

In 1957, a small group of BBC producers and studio managers began using

‘radiophonic’ techniques to create pioneering music and drama programmes.

The process consisted of recording real sounds, such as those created by the

human voice, bottles, bells, musical instruments, percussion devices or even

boxes of gravel or pebbles.

These were then manipulated to produce entirely new material. Tape

machines provided reverse playback, speed and pitch changes, or were

used to create sound loops, whilst reverberation and equalisation could

modify the sound quality.

Various elements of the work were edited together using tape-splicing

techniques, often note-by-note. This was a time-consuming business,

requiring endless skill and patience, but the results were often very impressive.

These processes, similar to musique concrète, created or enhanced the

atmosphere in a programme, but weren’t considered an ‘art in itself’.

!

Research & Development

10

© BBC MMXII


The BBC Radiophonic workshop!

An early view looking north through Rooms 14 and 13. Daphne Oram is cueing up a disc on one of the turntables on a BBC TD/7 gramophone; note the

parallel-tracking arms. To the right, Desmond Briscoe is editing a tape on a Ferrograph recorder, alongside 'Dickie' Bird. The equipment bays on the left

remained in use until the late seventies. The turntables in the foreground don't appear to be a standard BBC variety. (http://whitefiles.org/rwg/large-11.html)

Research & Development

11

© BBC MMXII


The BBC Radiophonic workshop!

An early picture, taken onthe 13th of May, 1958, looking through the window of Room 15 towards Rooms 13 and 14. The room on this side is the recording

area, complete with microphone. In the distance there's Desmond Briscoe with a tape loop, with a Motosacoche tape machine beyond him, 'Dickie' Bird the

engineer and Donald McWhinnie, in charge, standing in front of a Ferrograph recorder. Daphne Oram is at the controls of the 'Albert Hall' mixer, on top of

which there are a pair of PEUs. Note the flowers on the loudspeaker.

Research & Development

12

© BBC MMXII


The BBC Radiophonic workshop – Tape Loops!

Dick Mills, who joined the Workshop in 1958, sits in front of an EMI

BTR/2 tape machine, as he keeps his eye on a tape loop running

through a machine operated by John Harrison. This picture was

taken in 1960. (http://whitefiles.org/rwg/large-17.html)

EMI TR/90 tape machine was equipped

with the Tempophon, whose spinning

head allowed the pitch of a recording to

be changed without changing the tempo.

Research & Development

13

© BBC MMXII


The BBC Radiophonic workshop – Oramics!

Research & Development

14

© BBC MMXII


The BBC Radiophonic workshop – Oramics!

Video from: http://news.bbc.co.uk/1/hi/programmes/click_online/9671950.stm

Research & Development

15

© BBC MMXII


The virtual Radiophonic workshop!

Research & Development

16

© BBC MMXII


The virtual Radiophonic workshop!

Research & Development

17

Chris Lowis and colleagues

© BBC MMXII


The virtual Radiophonic workshop – Demo!

Chris Lowis and colleagues

http://www.bbc.co.uk/blogs/researchanddevelopment/2012/05/web-audio-radiophonics-1.shtml!

Research & Development

18

© BBC MMXII


Current broadcast chain (2012)!

Analogue

Digital

Microphone!

Mic Amp!

A/D!

Mixer DSP! Contribution

(EQ, compression, limiting etc)! Lossy Codec

(e.g. ISDN, 3G

etc)!

Router /

SRC!

Speaker!

Speaker Amp!

D/A!

Transmitter!

Storage!

(May include lossy

codec)!

DAW (may

include SRC,

bit depth

conversion, eq,

compression,

limiting etc)!

Picture provided by Rupert Brun

Research & Development

Mixer DSP!

(EQ, compression,

limiting etc)!

Tx Processor

(multi-band

compression)!

19

Router / SRC!

Distribution

Lossy Codec!

Emission

Lossy Codec!

Emmission

Lossy Codec!

IP Platforms!

© BBC MMXII


Current broadcast chain (2012)!

Analogue

Digital

Microphone!

Mic Amp!

A/D!

Mixer DSP! Contribution

(EQ, compression, limiting etc)! Lossy Codec

(e.g. ISDN, 3G

etc)!

Router /

SRC!

Speaker!

Speaker Amp!

D/A!

Transmitter!

Storage!

(May include lossy

codec)!

DAW (may

include SRC,

bit depth

conversion, eq,

compression,

limiting etc)!

Picture provided by Rupert Brun

Research & Development

Mixer DSP!

(EQ, compression,

limiting etc)!

Tx Processor

(multi-band

compression)!

20

Router / SRC!

Distribution

Lossy Codec!

Emission

Lossy Codec!

Emmission

Lossy Codec!

IP Platforms!

© BBC MMXII


!

How can we successfully turn a creative idea into

an experience which will be delivered and reproduced

using different reproduction systems in

uncontrolled environments including interaction

and environment awareness? !

!

!

!

!

!

!

Research ! & Development

21

© BBC MMXII


virtual sound scene

Production - Storage - Perception!

virtual

sound event

audio signals

recording room

virtual

sound event

DSP

diffuse

sound event

recording/

playback

scene

representation

DSP!

sound

editor

sound event

sound scene

scene

representation

Research & Development

22

© BBC MMXII


virtual

sound event

Production - Storage - Perception!

virtual

sound event

DSP

DSP!

recording/

playback

scene

representation

sound

editor

scene

representation

virtual sound scene

Research & Development

23

© BBC MMXII


Production - Storage - Perception!

DSP!

scene

representation

virtual sound scene

Research & Development

24

© BBC MMXII


listening room

Production - Storage - Perception!

virtual sound scene

recording room

virtual

sound event

audio signals

auditory scene

virtual

sound event

DSP

DSP!

recording/

playback

scene

representation

sound

editor

scene

representation

Research & Development

25

© BBC MMXII


Digital Audio Effects (DAFX)!

Input

signal

DAFX

Output

signal

Acoustical

and visual

representation

Control

parameters

Acoustical

and visual

representation

Diagram after [Arfib, 1999]

Research & Development

26

© BBC MMXII


Digital Audio Effects (DAFX)!

DAFX

Acoustical

and visual

representation

Control

parameters

Acoustical

and visual

representation

Research & Development

27

© BBC MMXII


Digital Audio Scene Effects (DASFX)!

DASFX

Acoustical

and visual

representation

Control

parameters

Acoustical

and visual

representation

Research & Development

28

© BBC MMXII


Digital Audio Scene Effects (DASFX)!

Isn’t this

mastering?!

Isn’t this just

DAFX for

mixed signal?!

It is different because a combination of scene description

and audio processing is used for visualisation and both

elements are modified.!

Research & Development

29

© BBC MMXII


Object-based audio formats!

Research & Development

30

© BBC MMXII


Object-based application in broadcast: NetMix!

Commentary!

Player!

Court atmosphere!

Pictures provided by Rupert Brun

Research & Development

31

© BBC MMXII


Object-based application in broadcast: Perceptive Media!

!

Research & Development

32

© BBC MMXII


Object-based application in broadcast: Perceptive Media!

!

• Experimental radio drama in a new editorial

format!

• Integration of listener information into the

story line!

• Client side audio procession and

personalization!

• Object-based audio storage and delivery!

http://www.futurebroadcasts.com/

• HTML 5 API for audio processing!

!

!

!

Research & Development

33

© BBC MMXII


Digital Audio Scene Effects (DASFX)!

DASFX

Acoustical

and visual

representation

Control

parameters

Acoustical

and visual

representation

Research & Development

34

© BBC MMXII


Digital Audio Scene Effects – Spatial Mastering!

Recording! Mixing! Mastering!

Mastering!

• Modification of the overall impression of an auditory scene!

• Compilation of difference scenes to one distribution package!

• Channel-based: Adjusting the final speaker signals!

• Object-based: Adjusting the audio signals and/or scene description!

• Spatial: Direction dependent processing!

!

[Melchior, 2011]

Research & Development

35

© BBC MMXII


Digital Audio Scene Effects – Spatial Mastering!

!

• “I want to have "

more brightness in the front.”!

• “I want to have "

a darker sound in the left corner.”!

• “I want to have "

more reverb in the back.”!

!

Research & Development

!

36

[Melchior, 2011]

© BBC MMXII


Digital Audio Scene Effects – Spatial Mastering!

Research & Development

37

[Melchior, 2011]

© BBC MMXII


Digital Audio Scene Effects – Spatial Mastering!

Research & Development

38

[Melchior, 2011]

© BBC MMXII


Digital Audio Scene Effects – Spatial Mastering!

Directional Function!

Rotation!

Spreading!

Directional!

emphasis!

Research & Development

39

[Melchior, 2011]

© BBC MMXII


Digital Audio Scene Effects – Spatial Mastering!

Research & Development

40

[Melchior, 2011]

© BBC MMXII


Digital Audio Scene Effects – Reactive Source!

!

• Use low level audio features to modify control parameter!

• Represent the source cloud as a virtual scene to make it format

agnostic!

• Create a wide source using a number of uncorrelated sources!

!

!

!

Research & Development

41

© BBC MMXII


Digital Audio Scene Effects – Reactive Source!

!

• Use low level audio features to modify control parameter!

• Represent the source cloud as a virtual scene to make it format

agnostic!

• Create a wide source using a number of uncorrelated sources!

!

!

!

Research & Development

42

© BBC MMXII


Digital Audio Scene Effects – Reactive Source!

g 2 (n)

Adaptive

audio processing

Format agnostic

scene

Reproduction system

adaptation

Feature

extraction

Feature

mapping

Auditory

scene

x 1 (n)

Digital audio

effect

:

y' 1 (n)

y' n (n)

Realtime

rendering

y 1 (n)

y m (n)

Research & Development

43

[Melchior, 2012]

© BBC MMXII


Digital Audio Scene Effects – Reactive Source!

!

• Use low level audio features to modify control parameter!

• Represent the source cloud as a virtual scene to make it format

agnostic!

• Create a wide source using a number of uncorrelated sources!

!

!

!

Research & Development

44

© BBC MMXII


Digital Audio Scene Effects – Reactive Source!

g 1 g 2

g 3

Pitch

extraction

Pitch

mapping

Loudness

extraction

Loudness

mapping

Modulation

matrix

Crossover

calculation

Source

cloud

x 1 (n)

Filterbank

:

y' 1 (n)

y' 5 (n)

Amplitude

panning

y l (n)

y r (n)

[Melchior, 2012]

Research & Development

45

© BBC MMXII


Digital Audio Scene Effects – Reactive Source!

Research & Development

46

[Melchior, 2012]

© BBC MMXII


Digital Audio Effects (DAFX)!

Input

signal

DAFX

Output

signal

Acoustical

and visual

representation

Control

parameters

Acoustical

and visual

representation

[Arfib, 1999]

Research & Development

47

© BBC MMXII


Digital Audio Effects (DAFX)!

Input

signal

DAFX

Output

signal

Acoustical

and visual

representation

Control

parameters

Acoustical

and visual

representation

Research & Development

48

© BBC MMXII


Digital Audio Effects (DAFX)!

Input

signal

DAFX

Output

signal

Perceptual model

Acoustical

and visual

representation

Control

parameters

Acoustical

and visual

representation

Research & Development

49

© BBC MMXII


Thank you for your kind attention.

Special thanks to Rupert Brun (BBC Audio and Music) for providing the block

diagrams of the broadcasting chain. Thanks to Christoph Lowis (BBC Research and

Development) for providing the Radiophonic HTML 5 prototypes.!

Research & Development

50

© BBC MMXII


References and Acknowledgement!

!

[Arfib, 1999] ! !D. Arfib. Visual representations for digital audio effects and their control. In Proc. DAFX-99 Digital Audio Effects

! !Workshop, pp. 63-68, Trondheim, December 1999!

!

[Niebur, 2010] ! !L. Niebur, Special Sound The Creation and Legacy of the BBC Radiophonic Workshop, Oxford University !

! !Press, 2010!

!

[Melchior, 2011] !F. Melchior, U. Michaelis, R. Steffens , “Spatial Mastering – a new concept for spatial sound design in !

! !object-based audio scene”,, Paper presented at the ICMC 2011!

!

[Melchior, 2012] !F. Melchior, “The Reactive Source: A Reproduction Format Agnostic and Adaptive Spatial Audio Effect”, e-brief !

! !accepted for publication at the 133 rd AES Convention, 2012!

!

!

!

!

!

!

!

!

!

!

Research & Development

51

© BBC MMXII

More magazines by this user
Similar magazines