Accessing Hardware-Accelerated

Video Codecs on Android

Steve Lukas

Staff Engineer, QuIC, Inc.


• Overview

• Requirements

• OpenMAX & iOMX

• Building

• Integration

• Q&A



Android Video

– Limitations

– Opportunities

Snapdragon Chipset

Hardware-Accelerated Video Codecs

– Increased Performance


• Device Hardware

– Device with Qualcomm Snapdragon Chipset

• MSM 7x30

• MSM 8x55

• MSM 8x60

– Optional: SD Card

• Device Software

Android Froyo 2.2

– Optional: rwx shell access


• PC Hardware

– Machine capable of building Android from source

• PC Software

Android SDK Platform 2.2 (API 8+)

Android Source Tree

– Cygwin (if using Windows)


• Technical background

Android application development

– OpenMAX IL


• Khronos Group


• Standard APIs

– Components

• Roles

– Buffers

• Buckets to hold data

• Provide Input: Empty Buffer

• Request Output: Fill Buffer


• Wrapper to OpenMAX IL

• Availability in Android

– Froyo 2.2 Stagefright

– Gingerbread 2.3

• Used by Android player

Sample Code Overview

Sample Code Overview

• Under the Hood

– Instantiate an OpenMAX IL Component Handler

– Derive from bnOmxObserver

• Report when Component has completed commands

• Callbacks occur in separate unmanaged thread

– Configure input and output parameters

– File fed from a thread until empty

– File written to on output callback


• Instantiate encoder / decoder & configure

• Determine callbacks

– Output handler

– Input callback

• Initialize component

• Feed component

• Signal completion

• Deinitialize Component


• New Sample Code

– Released Today

– Minimual latency

– Improved stability

– Callbacks for custom integration

– Can be used “out of the box” for basic needs

• Can be expanded upon for additional functionality

Building and Running

How to use

• Apk

– Runs direct on capable handsets

– Commercial deployment

– Integrate via shared library

• Command-line

– Development

– Requires rwx access

– Instant testing

Development Platform

Snapdragon MDP

– Full root access

– Pre-release Qualcomm Hardware

• Leveraging the Snapdragon 8660 MDP for App

Development, Wednesday 2:30pm in Elizabeth F


• Download Qualcomm-specific files

– Patch into Android source tree

• Build Android source

– Create Executable

– Create Shared Library

NDK Integration

• Export shared library

• Create Java->C bindings

• Build ndk shared library

• Write to Java bindings

• Example included in Sample Code

Sample Code

• Basic File I/O

• H.264 streams

• Output Callback

• Input Threads

Integration – Interface Code

• Start and Stop methods

– int omx_interface_init(void *obj); // initialize, prepare for input

– int omx_interface_deinit(void *obj); // de-initialize, stop threads

– int omx_interface_error(void *obj); // get last error

• Additional Decoder Configuration (optional)

– int omx_decoder_set_input_packing_format(void *obj, int


– int omx_decoder_set_input_compression_format(void *obj, int


• Feeding the component

– int omx_interface_send_input_data(void *obj, void *buffer, size_t size,

int timeStampLfile);

– int omx_interface_send_end_of_input_flag(void *obj, int


Integration – Sample Callbacks

• Input Integration Sample

– omx_interface_send_input_data fails if no available buffer

– Waits on semaphore for next available buffer

– Input callback received when buffer is available

• Frees up semaphore

– Decide how input data is handled

• Read from file, read from stream, etc.

• Output Integration Sample

– int handleOutputData(void *obj, void *buffer, size_t bufferSize, void


– Decide how output data is handled

• Write to file, write to stream, etc.



• Qualcomm Developer Network



• OpenMAX IL


QUALCOMM Innovation Center, Incorporated

5775 Morehouse . Drive

San Diego, CA. 92121-1714


Copyright © 2011 Qualcomm Innovation Center, Incorporated. All rights reserved.

Not to be used, copied, reproduced in whole or in part, nor its contents revealed in any manner to others without the express written permission of Qualcomm.

Qualcomm and Uplinq are trademarks of Qualcomm Incorporated, registered in the United States. Other product and brand names may be trademarks or registered

trademarks of their respective owners.

Nothing in these materials is an offer to sell any of the components or devices referenced herein. Certain components for use in the U.S. are available only through

licensed suppliers. Some components are not available for use in the U.S

Similar magazines