Distributed Multimedia Systems - BSCW Shared Workspace Server

Distributed Multimedia Systems 

9. Multimedia Adaptation 

László Böszörményi Distributed Multimedia Systems Multimedia Adaptation - 1

• Best effort 

• Reservation 

• Adaptation 

Resource Management 


• Enough food: do nothing 

• Not enough food 

Adaptation in general 

a) Learn to live with few 

(defensive adaptation) 

b) Learn to find better place 

(offensive adaptation) 


The Adaptation Principle 

• In the case of shortage of resources 

– Be creative, instead of giving up 

• Stream-level (defensive) adaptivity 

– We adapt – typically reduce – the quality of the 

streams to the given constraints 

– E.g. usage of layered encoding or priority information 

• System-level (offensive) adaptivity 

– We try to acquire more or better resources 

– Components migrate/replicate in the network 

• Application-level adaptivity 

– E.g. use a smaller window, a different representation 


Media Server 

LAN 

Meta-Database 

LAN 

DMMS architecture 

Streaming video 

LAN 

Router 

WAN 

Business Users 

“Last 

mile“ 

Proxy Server 

Business Users 

Home Users 

László Böszörményi Distributed Multimedia Systems Multimedia Adaptation - 5 

LAN

What is wrong with video streaming? 

• video ≠ video 

1. We may get the wrong video 

– Bad search facilities 

2. We may get the wrong quality-variation 

– E.g too high quality level for a PDA 

3. We may get the proper variation in bad quality 

– E.g jitter is too high 

4. We may get the video at the wrong time 

– E.g. too high start-up delay 


What can be done better? 

• WYNWYG: What You Need is What You Get 

– A kind of context-awareness 

1. Provide good semantic search 

– CODAC project, MPEG-7 

2. Acquire the client’s QoS capabilities and 

requirements and adapt the videos to them 

– Covered by our QBIX project, MPEG-21 

3. Bring the video as near to the client as necessary 

– Covered by our ADMS video server and X2X projects 

4. Monitor and predict client behavior and try to act 

proactively 

– Covered by our ADMS video server and X2X projects 


Stream Scaling 

• Temporal Scaling 

– Only a subset of all video data is used 

• Spatial Scaling 

– Reduction of the resolution 

• Frequency Scaling 

– Reduction of the DCT coefficients (per block) 

• Quantization Scaling 

• Color Scaling 

– Chrominance information is reduced (by FS or QS) 

• Transcoding 

– Transformation between compression formats 


Adaptive Video Server 

� # of nodes changes dynamically 

� Replicate/Migrate functions 

Data Manager 

New DC 

Data Collector 

� Host recommendation 

� Offensive adaptation 

Cluster Server 

New DMs 


AMS Component Architecture 


Advantages compared to static distribution 

• Increased scalability due to proactive server-level 

adaptations by regarding 

– client locality, client request parameters and variations in 

object popularities 

• Reduced startup delay and network traffic 

– due to near-to-client service 

• Increased load-balancing by 

– dynamically allocating nodes for DMs and DCs 

• Increased resource usage by sharing idle nodes 

with other types of adaptive applications 

– e.g. adaptive web-server 


Video Proxy Caching - Motivation 

• Original motivation for Caching 

1. Reduce network traffic 

2. Reduce server load and 

3. Reduce response time 

• Special motivation for the future 

– Multimedia content increases 

– Video data will dominate 

• Problem 

– Actual caching mechanisms are designed for typical 

web pages (web objects) 

– Videos are big 

+ But videos follow the WORM-principle 


• Caches popular data 

– Server for clients 

– Client for servers 

What is a Proxy Cache? 

• Adaptive MPEG-4 Proxy 

– Caches MPEG-4 videos 

– Change properties of the cached videos: 

• bit rate, frame rate, 

• size, color, … 


Video Caching Proposals 

• Full Caching (Original Caching) 

– Not efficient (store small number of videos) 

– But: Companies do it (Real Networks, Inktomi etc.) 

• Partial Caching (cache specific parts) 

– Prefix Caching (reduce start-up latency) 

– Cache whole videos, remove parts from the tail 

– Cache variable parts (bursts) 

– Popularity based partial caching etc. 

• Problems 

– Overhead, Interactivity, loose original advantages 


Video Caching Proposals 

• Quality-based caching 

– Use different quality levels at replacement 

1. Quality reduction (replacement) 

2. Quality adaptation (reduce and restore quality) 

– Number of videos (hits) increases 

– Can be time and 

resource (CPU) 

consuming 

– Proper range of quality 

adaptation 

– Reload (modeling, 

realization) 


User 

constraints 

QBIX - Design 

Video 

variations 


Read from IO 

Adaptation Engine 

Adapt 

Write to IO 


Session Management 


Proxy Gateway Scenario 


Quality Based Intelligent ProXy (QBIX) 

• QBIX tailors videos to the requirements of the 

client and caches the result 

– Must be supported on client side by providing metadata, 

such as 

• Terminal Capabilities, User Preferences (MPEG-21) 

– Caching (LRU) 

• Finding the best version for a client 

– Maximize Q for reasonable costs 

• Gain 

– Gateway functionality 

– Request filtering 


• Feature Set 

Calculating Quality 

– Visual: F={dimX, avgBitRate, color, fps} 

• dimY can be calculated from the aspect ratio 

– Audio: F={samplingRate, numOfChannels, avgBitRate} 

– Other: F={avgBitRate} 

• Request 

– A request R specifies for each feature x ⊂ F an 

acceptance range with an associated weight for 

importance: 

[min, best, max] / weight 


Calculating Quality 

• A request R specifies for each feature x ε F an acceptance range with 

associated weight (importance) 

range X =[min, best, max] / weight 

– val x is the feature value of a stream for feature x 

• Q(val x,range x,imp. x)= 

– Max. if, val x = best 

– Q x = weight, if best 

– Q x = 0, if val x outside 

• Quality (F)= 

Q( 

F ) = 

0, 

∃x 

∈ 

| F | 

∑ 

x= 

1 

Q( 

val 

x 

F 

| 

, range 

Q 

x 

x 

= 

0 

∧ 

, weight 

x 

weight 

) , 

else 

x 

> 

0 


Cost Function - Requirements 

• QoS control doesn’t work without a notion of cost 

• The higher the actual load of a resource, the 

higher is the price to use that resource 

• Cache hits should be preferred over cache misses 

• It should be possible to assign weights to 

resources, according to their importance 

• Maximize quality if enough resources are 

available 


Cost Function 

• For a given timepoint t, the load of the resources network 

(NET), disk (HD) and CPU (CPU) is known. 

• Each resource has a price 

– p net , p hd , p cpu 

• Each request r consumes on average a certain percentage of 

each resource per second 

– net r , hd r , cpu r 

• AdmissionControl 

NET+net r < 1, CPU + cpu r

Example 

• Source: dimX=352, bitrate=750000 bps, 

color=true, framerate=30.00 fps 

• Request: 

– dimX: (256, 320, 352)/0.20 

– bitrate: (300000,512000, 768000)/0.20 

– color: (1,1,1)/0.20 

– framerate: (15.00, 25.00, 30.00)/0.20 

– maximum delay: (0,0,1000)/0.20 

• 10 Mbit/sec network, 10MB/sec disk, 75*10^6 

pixels/sec CPU 

• Price for all resources=10, Cache Miss 


• First check source 

Example (2) 

• Then check all versions that can be created 

according to stream creation rules 

– (264,456184,{15,20,25,30}) 

– (308,{310460,620920},{15,20,25,30}) 

– (352, 405504,{15,20,25,30}) 

– 16 versions in total (+ source) 

• Color always true 

• Let’s assume two cases 

a) CPU-load is 0, load on net and HDD is 0 

b) CPU-load is 80%, load on net and HDD is 0 


Example (check possible variants) 

• Load on CPU is 0 (top) resp. 0.8 (bottom) 

Features Res.Cost Quality FinalCost 

(264,456184,25.0) 

(308,620920,25.0) 

(352,405504,25.0) 

(352,750000,30.0) 

Features 

(264,456184,25.0) 

(308,620920,25.0) 

(352,405504,25.0) 

(352,750000,30.0) 

3.260879 

3.847359 

4.009895 

2.259521 

Res.Cost 

15.820722 

27.648633 

74.245014 

2.259521 

0.695109 

0.789666 

0.629579 

0.372656 

Quality 

0.695109 

0.789666 

0.629579 

0.372656 

0.994212 

0.809232 

1.485350 

1.417497 

FinalCost 

4.823595 

5.815458 

27.501922 

1.417497 


Advantages of QoS aware proxy-ing 

• Automatically adjusts itself to request pattern 

– can act as dynamic gateway 

• Considers costs, quality and resource usage 

• Adaptation in the decompressed domain is 

feasible when combined with admission control 

and cost/quality function 

– Layered encoding will help 

– few devices will request adaptation 


Multimedia Presentations (Scenes) 

• Scenes are composed of media objects 

– Video, audio, text, images,… 

• Scene Description specifies the content of the scene to be 

presented 

• Spatial and temporal model 

is interpreted by the player 

• Interaction with the user is 

possible 

• Standards like SMIL, 

MPEG-4 (BIFS, XM-T) 

• LASeR 


IOD 

MPEG-4 content access example 

Scene Description Stream 

Object Descriptor Stream 

Visual Stream 

Visual Stream 

Visual Stream 

Audio Stream 

... 

Interactive Scene Description 


Scene-based Adaptation 

• Goal: A single presentation should 

be dynamically optimized according 

to 

1. The available resources of dev. + 

nw. (Adaptation) 

2. The preferences of the user 

(Personalization) 

� Possible Adaptations 

� Low Bandwidth 

- Replace with still images 

� Small Screen faces 

- Zoom active person 

- Switch between persons 

- Enlarge text 

� Muted Device 

- Provide subtitles 

• Quality metrics for multimedia 

presentations and performance of 

adaptations 

• Generic approach for adaptation 

and personalization of scenes 

and presentations 


State-of-the art Streaming 

• Client/Server 

– Scalability is poor, single point of failure 

– The distance of source and destination is fix 

• P2P 

– Too fragile: streaming may take longer than usual 

connectivity time of client nodes 

– Not transparent to the clients 

• Content Distribution Networks (CDN) 

– Static configuration 

– Expensive and proprietary solutions 

– Not transparent to the content providers 


CDN-Example (Akami) 

• Client requests are redirected to “best” Akamai 

server 


ProXy-to-ProXy (X2X) Streaming 

• Content is replicated/migrated to proxies 

– Full resp. partial replication, the latter in different domains: 

time, quality, semantics… 

• Proxies are usually always on 

• Proxies build layered networks covering small 

“geographic” regions 

• Clients are redirected to the nearest group, resp. 

new group is created if necessary 

• Combination of P2P and CDN technology 

– Low cost, high availability 

– Transparent both to provider and client 


X2X Architecture 


X2X Architecture and Affinity 

• Groups are built based on group 

affinity 

– Weighted ∑ of Network Closeness, 

Semantic Closeness and load balance 

• Clients find their group of client affinity 

– Weighted ∑ of NW, SC and load lightness 

• Videos are replicated to groups to 

they have a content affinity 

• Affinity function descr. in XML 

– The behavior of the system can be 

dynamically “reprogrammed” 


Multiple Source Streaming 

� Increase number of serviceable requests 

� May solve some last mile problems 

E.g. ADSL: download bandwidth 

>> upload bandwidth 


Advantages of X2X + QBIX together 

• QBIX realizes a defensive adaptation strategy 

and adapts the multimedia data to the clients’ 

need 

– For QBIX a reactive mode is sufficient, 

– especially if layered coding available 

• X2X realizes offensive adaptation strategy and 

brings the resources near to their clients 

– X2X adaptation itself can be time consuming (we copy 

entire videos) 

– Proactive mode is advisable: 

– Client behavior prediction 


Client behavior prediction (1) 

• Client behavior is periodically monitored 


Client behavior prediction (2) 

• Video is proactively replicated 


General Conclusions 

• State-of-the art video delivery is insufficient 

1. Videos must be adapted to the available 

resources and the user requirements 

2. Videos must be brought to the proper place to 

provide smooth streaming 

3. Client behavior prediction enables proactive 

offensive and defensive adaptation

Distributed Multimedia Systems - BSCW Shared Workspace Server

Create successful ePaper yourself

Delete template?

Save as template?