嵌入式计算机系统 - Computer Science and Engineering

嵌入式计算机系统 

Lecture #8 

MeeGo Multimedia 

内容来自于 meego.com 以及 MeeGo 相关公开教程

Outline 

• MeeGo Multimedia Module Overview 

• Qt Mobility Multimedia 

• GStreamer Framework 

• APIs in Qt Mobility Multimedia

Multimedia in MeeGo Architecture

Multimedia in MeeGo Architecture

Media Stack Architecture 

Media Application 

Core OS 

GStreamer 

Qt-Mobility 

MeeGo Touch 

framework 

Qt 

libva 

Other core modules 

HW/Driver 

Video 

Decoder 

Video 

Encoder 

Graphics 

SST audio 

engine 

5

Multimedia in MeeGo Platform API 

• GStreamer 

• PulseAudio

Multimedia in MeeGo API

Qt Mobility Overview 

• Qt Mobility Project provides desirable functions for 

mobile platform 

• Qt Mobility covers a collection of APIs or frameworks 

Bearer 

Management 

Contacts Location Message Multimedia 

Publish and 

subscribe 

Qt Service 

Framework 

Sensors 

API 

System 

Info 

…

Qt Mobility Multimedia Module 

• QtMobility’s media library will 

• Easily create applications with system media capabilities 

• Cover the items of most direct interest to developers 

• playback 

• recording 

• playlist 

• management 

• metadata 

• radio 

• media editing 

• TV 

• transcoding 

• streaming


• Advantage: 

• Implement fundamental Multimedia functions with 

minimal code 

• A great deal of flexibility with the media source 

• Many different codecs are supported 'out of the box'


• High Level APIS 

• Encapsulate the ability to manipulate media stream at varying level 

without detailed knowledge of media subsystem 

QT Mobility Multimedia High Level APIs 

Playback List 

Recording Playback Radio Camera .. 

Audio 

Video 

QT Mobility Multimedia Service

QtMobility Multimedia Module 

• Service - access to low-level of Multimedia function 

• QAudio* 

• QAudioDeviceInfo, QAudioInput, QAudioOutput 

• Low level cross platform audio API 

• Low latency notifications, simple game, audio data processing/visualisation. 

• QVideo* 

• QAbstractVideoBuffer, QVideoFrame, QabstractVideoSurface 

• Low level tools for passing video frames. 

• Abstraction for video buffers and surfaces

QtMobility Multimedia Module 

• QT MMF supports rich backend: 

• GStreamer 

• Quicktime 

• Directshow 

• … 

QT MMF 

QuickTimeService 

Plugin 

QuickTimeService 

GStreamer Service 

Plugin 

QGstreamerService 

MyService

What is GStreamer 

• A framework for creating media applications 

• Pipeline based 

• Not a player 

• One of the most obvious uses is to build a media player 

• Main advantage 

• Pluggable elements could be mixed to form arbitrary 

pipelines to support various kinds of video/audio codecs 

• Unified API for user to create pipelines and media players

GStreamer Framework

Concepts in GStreamer 

• Element 

• Pad 

• Bin 

• Pipeline 

• Communication Mechanism

Concepts in GStreamer-element 

• Element 

• Element is the most important object in GStreamer, it is derived 

from GstObject 

• Create a chain of elements linked together and let data flow through 

this chain of elements


• The way to create an element 

GstElement *element; 

element = gst_element_factory_make(“fakesrc”, “source”); 

if(!element) { 

g_print(“Failed to create element\n”); 

return -1; 

}


• Using an element as a GObject 

GstElement *element; 

gchar *name; 

element = gst_element_factory_make(“fakesrc”, “source”); 

if(!element) { 

g_print(“Failed to create element\n”); 

return -1; 

} 

//get the name of element 

g_object_get(G_OBJECT(element), “name”, &name, NULL); 

g_print(“The name of element is %s.\n”,name); 

g_free(name);


• Linking elements 

GstElement *pipeline; 

GstElement *source, *filter, *sink; 

pipeline = gst_pipeline_new(“my-pipeline”); 

source = gst_element_factory_make(“fakesrc”, “source”); 

filter = gst_element_factory_make(“identity”, “filter”); 

sink = gst_element_factory_make(“fakesink”, “sink”); 

gst_bin_add_many(GST_BIN(pipeline), source, filter, sink, NULL); 

if(!gst_element_link_many(source, filter, sink, NULL)) { 

g_warning(“Failed to link element!”); 

}


• Element States 

• GST_STATE_NULL 

• GST_STATE_READY 

• GST_STATE_PAUSED 

• GST_STATE_PLAYING 

• Setting the element state 

• gst_element_set_state() 

• gst_element_sync_state_with_parent()

Concepts in GStreamer-pad 

• Pad 

• Pad is used to negotiate links and data flow between 

elements in GStreamer 

• Pad could be sink pad or source pad 

• Pad can restrict the type of data that flows through it


• Pad 

GObject 

+----GstObject 

+----GstElement 

+----ffdemux_ogg 

Pad Templates: 

SRC template: 'video_%02d' 

Availability: Sometimes 

Capabilities: 

ANY 

SRC template: 'audio_%02d' 

Availability: Sometimes 

Capabilities: 

ANY 

SINK template: 'sink' 

Availability: Always 

Capabilities: 

application/ogg 

GObject 

+----GstObject 

+----GstElement 

+----GstMpeg2dec 

Pad Templates: 

SRC template: 'src' 


Capabilities: 

video/x-raw-yuv 

format: { YV12, I420, Y42B } 

width: [ 16, 4096 ] 

height: [ 16, 4096 ] 

framerate: [ 0/1, 2147483647/1 ] 

SINK template: 'sink' 


Capabilities: 

video/mpeg 

mpegversion: [ 1, 2 ] 

systemstream: false


• An example of using Dynamic pads 

static void cb_new_pad(GstElement *element, GstPad *pad, gpointer data) { 

gchar *name; 

name = gst_pad_get_name(pad); 

//setup a new pad link for the newly create pad 

… 

} 

GstElement *demux; 

demux = gst_element_factory_make(“oggdemux”, “demuxer”); 

//listen for newly create pads 

g_signal_connect(demux, “pad-added”, G_CALLBACK(cb_new_pad), NULL);

Concepts in GStreamer-Bin&Pipeline 

• Bin & Pipeline 

• A Bin is a container for a collection of elements. E.g. playbin 

• A pipeline is a special subtype of a bin that allows execution of all of 

its contained child elements. 

• A pipeline is a top-level bin. As you set it to PAUSED or PLAYING 

state, data flow will start and media processing will take place. 

• GstPipeline is derived from GstBin


• Pipeline example - a simple ogg player


• Creating a bin and add elements 

GstElement *bin, pipeline, *source, sink; 

//create pipeline 

pipeline = gst_pipeline_new(“my_pipeline”); 

//create bin 

bin = gst_bin_new(“my_bin”); 

source = gst_element_factory_make(“fakesrc”, “ source”); 

sink = gst_element_factory_make(“fakesink”, “sink”); 

//add elements to the bin 

gst_bin_add_many(GST_BIN(bin), source, sink, NULL); 

//add the bin to the pipeline 

gst_bin_add(GST_BIN(pipeline), bin);

Concepts in GStreamer-Communications 

• Mechanism for communication and data exchange 

between the application and the pipeline 

• buffers: sent between elements in the pipeline 

• events: sent between elements or from the application to 

element 

• messages: posted by element and collected by 

application 

• queries: application or element queries information from 

other element


• Different communications flows in GStreamer 

pipeline


• Example of using bus 

static GMainLoop *loop; 

static gboolean my_bus_callback(Gbus *bus, GstMessage *message, 

gpointer data) { 

switch(GST_MESSAGE_TYPE(message)) { 

case GST_MESSAGE_ERROR: { 

… 

break; 

} 

case GST_MESSAGE_EOS: { 

… 

break; 

} 

default: 

break; 

} 

}


• Example of using bus 

GstElement *pipeline; 

GstBus *bus; 

pipeline = gst_pipeline_new(“my_pipeline”); 

bus = gst_pipeline_get_bus(GST_PIPELINE(pipeline)); 

gst_bus_add_watch(bus, my_bus_callback, NULL); 

loop = g_main_loop_new(NULL, FALSE); 

g_main_loop_run(loop);

Tools in GStreamer 

• gst-inspect 

• Print info about a GStreamer plugin or element 

• Example: 

gst-inspect oggdemux 

• gst-launch 

• Build and run a GStreamer pipeline 

• Example: 

gst-launch filesrc location=/home/child.ogg ! mad ! 

alsasink

Example of video player 

QT MMF API QT MMF Backend GStreamer Base 3 rd party plugins 

Player->play() 

QServicePlugin 

QGstremerService 

Playbin2 

Typefind/autoplug 

Demuxer 

Decoder 

Developing 

Developed 

Filter 

…

Multimedia Module API Hierarchy 

QObject 

QMediaObject 

QMediaService 

QMediaControl 

QMediaPlaylist QMediaRecorder QVideoWidget 

QAudioCaptureSource 

QCameraControl 

QCamera 

QMediaPlayerControl 

QMediaImageViewer 

QMediaRecorderControl 

QMediaPlayer

Multimedia Module API 

• Multimedia module consists of APIs related to 

• audio 

• video 

• image 

• camera



• audio 

• video 

• image 

• camera

Audio 

• Classes related to Audio 

• QAudioCaptureSource 

• QAudioDeviceInfo 

• QAudioEncoderControl 

• QAudioEncoderSettings 

• QAudioEndpointSelector 

• QAudioFormat 

• QAudioInput 

• QAudioOutput


• Provides an interface to query and select an audio input 

endpoint 

• Get and Set audio input to use


• Functions in QAudioCaptureSource 

• Qlist QAudioCaptureSource::audioInputs() const 

Returns a list of available audio inputs 

• void QAudioCaptureSource::setAudioInput(const Qstring &name) 

Set the active input to name


• Example to use QAudioCaptureSource 

QAudioCaptureSource *audiosource = new QAudioCaptureSource; 

//get audio input device names 

Foreach(const QString &device,audiosource->audioInputs()) { 

ui->deviceBox->addItem(device,QVariant(device)) 

} 

//set audio input device name 

audiosource.setAudioInput(boxValue(ui->codecBox).toString()); 

//set the audiosource to the recorder 

QMediaRecorder *capture = new QMediaRecorder(audiosource); 

capture->record();

QAudioEncoderSettings 

• Provides a set of audio encoder settings 

int bitRate() 

Return the bit rate of the compressed audio stream in bits per second 

void setBitRate(int rate) 

Set the audio bit rate in bits per second 

QString codec() 

Return the audio codec 

void setCodec(const QString &codec) 

Set the audio codec 

QtMultimediaKit::EncodingQuality quality() 

Returns the audio encoding quality 

void setQuality(QtMultimediaKit::EncodingQuality quality) 

Set the audio encoding quality

QAudioEncoderSettings 

• Example to use QAudioEncoderSettings 

QAudioEncoderSettings audioSettings; 

audioSettings.setCodec(“audio/mpeg”); 

audioSettings.setQuality(QtMultimediaKit::HighQuality); 

audioSettings.setChannelCount(2); 

aduioSettings.setEncodingMode(QtMultimediaKit::ConstantQualityEncoding); 

//set the audio encoder setting when record 

QMediaRecorder *capture = new QMediaRecorder(audiosource); 

capture.setEncodingSettings(audioSettings);

QMediaRecorder 

• Be used for the recording of media content 

• Intended to be used with other media object 

such as QAudioCaptureSource, QRadioTuner 

• State of QMediaRecorder 

• QMediaRecorder::StoppedState 

• QMediaRecorder:: RecordingState 

• QMediaRecorder:: PausedState


• Signals in class QMediaRecorder 

• void durationChanged(qint64 duration) 

emitted when the duration of the recorded media has 

changed 

• void stateChanged(QMediaRecorder::State state) 

emitted when a media recorder’s state has changed 

• void mutedChanged(bool muted) 

emitted when the muted state has changed 

• void error(QMediaRecorder::Error error) 

emitted when an error has occurred


• Functions in class QMediaRecorder 

• void record() 

start recording 

• void stop() 

stop recording 

• void pause() 

pause recording 

• bool setOutputLocation(const Qurl &location) 

set the destination location of media content 

• void setEncodingSettings(const QAudioEncoderSettings &audio) 

set the audio and video encoder settings


• Example of using QMediaRecorder

Examples 

• Audioinput 

• audiooutput 

11-8-18 47 

Embedded Computer Systems



• audio 

• video 

• image 

• camera

Video 

• Classes related to video 

• QMediaPlayer 

• QMediaPlayerList 

• QVideoWidget

QMediaPlayer 

• Properties in QMediaPlayer 

• duration: the duration of the current media 

• muted: the muted state of the current media 

• playbackRate: the playback rate of the current media 

• playlist: the media playlist being used by the player 

object 

• position: the playback position of the current media 

• volume: the current playback volume

QMediaPlayer 

• Properties in QMediaPlayer 

• media: the active media source being used by the 

player object 

• state: the media player’s playback state 

QMediaPlayer::StoppedState 

QMediaPlayer::PlayingState 

QMediaPlayer::PausedState

QMediaPlayer 

• Functions in QMediaPlayer 

• void play() 

start or resuming playing the current source 

• void pause() 

pause playing the current source 

• void stop() 

stop playing and reset the play position to 

beginning 

the

QMediaPlayer 

• Example to use QMediaPlayer 

QMediaPlayer *player = new QMediaPlayer; 

player->setMedia(QUrl::fromLocalFile(“/home/song.mp3”)); 

player->setVolume(50); 

player->play();

QMediaPlaylist 

• Provide a list of media content to play 

• Used with other media object like QMediaPlayer or 

QMediaImageViewer

QMediaPlaylist 

• Functions in QMediaPlaylist 

• bool addMedia(const QMediaContent &content) 

append the media content to the playlist 

• bool insertMedia(int pos, const QMediaContent &content) 

insert the media content to the playlist at position pos 

• int mediaCount() const 

return the number of items in the playlist 

• void next() 

advanced to the next media content in playlist 

• void previous() 

return to the previous media content in playlist

QVideoWidget 

• Provides a widget which presents video produced by a 

media object

QVideoWidget 

• Properties in QVideoWidget 

• brightness: int 

the brightness of displayed video ranging between -100 and 100 

• contrast: int 

the contrast of displayed video ranging between -100 and 100 

• hue: int 

the hue of displayed video ranging between -100 and 100 

• saturation: int 

the saturation of displayed video ranging between -100 and 100 

• fullScreen:bool 

whether video display is confined to a window or is fullScreen

Example 

• Example to use QMediaPlayer with QMediaPlaylist and QVideoWidget 

playlist = new QMediaPlaylist; 

playlist->addMedia(Qurl(“http://example.com/movie1.mp4”)); 

playlist->addMedia(Qurl(“http://example.com/movie2.mp4”)); 

player = new QMediaPlayer; 

player->setPlayList(playlist); 

videoWidget = new QVideoWidget; 

player->setVideoOutput(videoWidget); 

videoWidget->show(); 

player->play();

Example 

• Videowidget 

11-8-18 59 

Embedded Computer Systems



• audio 

• video 

• Image 

• camera

Image 

• Classes related to Image: 

• QMediaImageViewer


• Properties in QMediaImageViewer 

• elapsedTime: int 

time elapsed since the current image was loaded 

• timeout: int 

time that an image is displayed for before moving to 

the next image 

• state: QMediaImageViewer::State 

playlist control state of a slide show


• Example of using QMediaImageViewer with 

QVideoWidget 

viewer = new QMediaImageViewer; 

videoWidget = new QVideoWidget; 

viewer->bind(videoWidget); 

videoWidget->show();


• Example of using QMediaImageViewer with QMediaPlaylist 

playlist = new QMediaPlaylist; 

playlist->setPlaybackMode(QMediaPlaylist::Loop); 

playlist->addMedia(image1); 



view->setPlaylist(playlist); 

view->setTimeout(5000); 

view->show();



• audio 

• video 

• Image 

• camera

Camera 

• Provides a framework to use Camera device 

features when supported by the platform 

• Take still photo 

• Take video 

• Basic settings including flash mode, zoom level etc 

• Classes related to Camera 

• QCamera 

• QCameraViewfinder 

• QCameraImageCapture

Camera 

• using camera to capture an image 

QCamera *camera = new QCamera; 

QCameraViewfinder *viewFinder = new QCameraViewfinder(); 

viewFinder->show(); 

camera->setViewfinder(viewFinder); 

QCameraImageCapture *imageCapture = new 

QCameraImageCapture(camera); 

camera->setCaptureMode(QCamera::CaptureStillImage); 

camera->searchAndLock; 

imageCapture->capture(); 

camera->unlock();

Camera 

• using camera to record video 

QCamera *camera = new QCamera; 

QMediaRecorder *mediaRecorder = new 

QMediaRecorder(camera); 

camera->setCaptureMode(QCamera::CaptureVideo); 

camera->start(); 

mediaRecorder->record();

嵌入式计算机系统 - Computer Science and Engineering

Create successful ePaper yourself

Delete template?

Save as template?