sysfx.pdf

Custom Audio Effects inWindows VistaMarch 8, 2012AbstractWindows Vista allows third-party audio hardware manufacturers to include customhost-based digital signal processing effects as part of their audio driver's value-addedfeatures. These effects are packaged as user-mode System Effect Audio ProcessingObjects (sAPOs).There are three insertion points for sAPOs: pre-mix render, post-mix render, andcapture. Each logical device’s audio engine supports one instance of a pre-mix rendersAPO per stream (render LFX) and one post-mix render sAPO (GFX). The audio enginealso supports one instance of a capture sAPO (capture LFX) that is inserted in eachcapture stream.This white paper provides guidelines for device driver developers who want to createand install custom sAPOs.This information applies to the Windows Vista operating system.Future versions of this preview information will be provided in the MicrosoftWindows Driver Kit (WDK).The current version of this paper is maintained on the Web at:Custom Audio Effects in Windows VistaReferences and resources discussed here are listed at the end of this paper.Disclaimer: This document is provided “as-is”. Information and views expressed in this document, includingURL and other Internet Web site references, may change without notice. You bear the risk of using it.This document does not provide you with any legal rights to any intellectual property in any Microsoftproduct. You may copy and use this document for your internal, reference purposes.© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 2Document HistoryDateChangeMarch 8, 2012 Reworded material in sAPO Initialization paragraphMay 10, 2006 First publicationContentsIntroduction.................................................................................................................... 3How Custom sAPOs are Implemented........................................................................3How Custom sAPOs are Installed................................................................................4Windows Vista Audio Architecture................................................................................. 5Support for Multiple Devices......................................................................................6Audio Sessions............................................................................................................ 6How to Install sAPOs....................................................................................................... 8Registry Settings......................................................................................................... 8INF Files for sAPOs.................................................................................................... 10Enhancements Property Page Replacement............................................................ 11Run-Time Behavior of sAPOs.........................................................................................11sAPO Bypass..............................................................................................................12sAPO Initialization.....................................................................................................12Endpoint Property Store Settings............................................................................. 13sAPO Format Negotiation.........................................................................................13Data Formats........................................................................................................ 14The sAPO LockForProcess Method.......................................................................14sAPO Failure Monitoring and Automatic Disabling.................................................. 15Application Control over sAPOs.................................................................................... 15How to Implement a Custom sAPO...............................................................................16Required Interfaces.................................................................................................. 16The CBaseAudioProcessingObject Class................................................................... 16Signal Processing Requirements...............................................................................17How to Implement a UI for Configuring the Effects......................................................18Resources...................................................................................................................... 18March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 3IntroductionThe Microsoft Universal Audio Architecture (UAA) family of class drivers and theassociated INF files are included with Windows Vista. These drivers support a set ofuser-mode System Effect Audio Processing Objects (sAPOs) that use the WindowsVista system effect infrastructure. Windows Vista uses sAPOs to implement severaldigital signal processing (DSP) audio effects algorithms including:• Speaker fill• Headphone virtualization• Room correction• Output encoding for external decoders• Automatic gain control• Microphone array filtersWhen users install an audio device driver by using the standard INF file, theyautomatically have access to the system's sAPOs. Independent hardware vendors(IHVs) and original equipment manufacturers (OEMs) can provide additional customsystem effects while still using the Microsoft class drivers. They do so by packagingtheir DSP algorithms as sAPOs and modifying the standard INF file to insert theirsAPOs into the audio engine’s signal processing graph.This white paper provides guidelines for vendors who want to create and installcustom sAPOs.How Custom sAPOs are ImplementedCustom sAPOs are amplemented as in-process COM objects, so they run in user modeand are packaged in a dynamic-link library (DLL). There are three types of sAPO,based on where they are inserted in the signal processing graph. Each logical devicecan be associated with one sAPO of each type.• A local render effect (render LFX) sAPO processes an audio stream from aparticular application just before mixing. If multiple applications are involved,each application has one instance of the LFX sAPO per stream.• A local capture effect (capture LFX) sAPO processes an audio stream from anapplication just before mixing. If multiple applications are involved, eachapplication has one instance of the capture LFX sAPO per stream.• A global effect (GFX) sAPO processes the audio stream after mixing. A logicaldevice, such as line out, can have only one instance of the GFX sAPO. If otherlogical devices such as headphone output or speaker output exist, they get theirown instance of the GFX sAPO.Note: For convenience, the terminology in this white paper refers mostly to outputdevices. However, the technology is symmetric and works essentially in reverse forinput devices. The primary difference is that there is no way to insert a GFX sAPO intothe capture graph.March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 4LFX and GFX sAPOs operate on a single input or output stream. They cannot be usedin “full-duplex” mode to process both input and output data. The infrastructure is notdesigned to accommodate effects such as acoustic echo cancellation (AEC) andmicrophone array processing. With Windows Vista, these types of processingalgorithms are located in the application graph, above the audio subsystem. WindowsVista ships with high-quality implementations of both algorithms in that location.More details on how to implement custom sAPOs are given later in this paper.How Custom sAPOs are InstalledCustom sAPOs are installed with the audio device driver and linked to a specific Plugand Play hardware ID. This ensures that a hardware manufacturer can specify the LFXand GFX sAPOs for an audio device and be confident that they will work well together.• Each Plug and Play hardware ID can be associated with only one GFX and one LFXsAPO. However, if developers require variable behavior that is based on differentstream or device characteristics, an sAPO can contain multiple DSP algorithmsinternally that can be used exclusively or together.• Modern PCs normally have separate logical devices for headphone, line out,Sony/Philips Digital Interface (SPDIF) out, speaker out, and so on. When there aremultiple logical devices, each one can have its own LFX and GFX sAPO.• As with drivers, custom sAPOs must go through the Windows Logo Programsigning process. There are two ways to submit sAPOs to WHQL for signing:• An sAPO that is associated with a custom driver should be submitted with thedriver package.• An sAPO that is linked by an INF to one of the Windows Vista UAA classdrivers can be submitted separately.An sAPO commonly provides a user interface (UI) that allows a user to configure theeffects. This UI can, for example, allow the user to select from several different signalprocessing algorithms. Microsoft provides a configuration UI for the standardWindows Vista sAPOs. If a custom sAPO has user-accessible settings, themanufacturer must provide an appropriate configuration UI. The configuration UI isinstalled with the device driver and is associated with the sAPO by registration.Note: The Microsoft-supplied Enhancements property page is associated with theMicrosoft home theater sAPOs and cannot be modified. However, manufacturers canreplace this property page with a custom property page that is designed to supporttheir sAPOs. However, if manufacturers replace the native Windows Vista sAPOs,they must either mirror the functionality of the native features or wrap thosefeatures into their own offering to prevent any loss of features for Windows Vistausers.More details on how to install custom sAPOs are given later in this paper.March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 5Windows Vista Audio ArchitectureFigure 1 shows how the sAPOs for a logical device are incorporated into the WindowsVista audio architecture. The audio engine runs in user mode and supports a signalprocessing graph that processes the audio stream. An sAPOs is essentially a plug-in tothe signal processing graph. Some notes on the figure include:• There is one LFX sAPO for each logical device, but each application that isstreaming to the audio engine has its own instance of that sAPO.• The GFX sAPO is inserted in the engine’s device graph. Each logical device hasonly a single instance of the audio engine and GFX sAPO. The GFX sAPO passesthe audio stream to the PortCls audio adapter and ultimately to the output device.• An sAPO runs as LocalService and cannot access any resources that do not havethe appropriate access control list (ACL) setting.• An sAPO runs with limited privileges. The exact set depends on which services arerunning in the audio service's svchost instance. However, sAPOs should not beperforming any operations that require privileges.• An sAPO should not access the network.Application1 Application 2 Application 3L F X L F X L F XAudio EngineG F XU ser ModePortCls Audio AdapterKernel ModeC u sto mComponentsMicrosoft WaveRTHD Audio FunctionClass DriverMicrosoftComponentsMicrosoftHD Audio Bus DriverLogical DeviceFigure 1. Basic Windows Vista audio architectureMarch 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 6Support for Multiple DevicesAlthough each logical device can have only one audio engine—LFX sAPO, and GFXsAPO—the system can support multiple logical devices. Each has its own audio engineand LFX and GFX sAPOs. Figure 2 shows a system with three logical devices.ApplicationApplicationApplicationApplicationApplicationApplicationApplicationApplicationApplicationLFX1LFX1LFX1LFX2LFX2LFX2LFX3LFX3LFX3Audio Engine 1Audio Engine 2Audio Engine 3GFX 1GFX 2GFX 3U ser ModeKernel ModeLogical Device 1Logical Device 2Logical Device 3Figure 2. Windows Vista audio architecture with multiple devicesAudio SessionsAn audio session is a group of related audio streams that a client can managecollectively. Each session represents a subset of the streams that form the global mixthat plays through a particular audio device such as headphones. The global mixcombines all of the sessions from all of the applications. Clients control the volumelevel and mute state of each individual session, and the system applies these settingsuniformly to all of the streams in the session.Typically, a session consists of one or more streams from a single process. However,applications can define cross-process sessions that combine streams from two ormore processes. Figure 3 shows how sAPOs are integrated into audio sessions. Notethat:• Each audio stream has its own instance of the LFX sAPO.• Streams from different processes can belong to the same session.• Multiple streams from the same process can belong to the same or differentsessions.• The streams for all the device’s sessions pass through a single GFX.March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 7Audio ClientInitializedwith“Cross Processsession” flagPr ocess DPr ocess CPr ocess BPr ocess AIAudioClientIAudioClientIAudioClientIAudioClientIAudioClientIAudioClientApplicationO nC rossProcessTransportC rossProcessTransportC rossProcessTransportC rossProcessTransportC rossProcessTransportC rossProcessTransportW A V EA P O sLim iterLim iterLim iterLim iterLim iterLim iterLocalG r aphL F XL F XL F XL F XL F XL F XVolum e Volum e Volum e Volum e Volum e Volum eM ixerM ixerM ixerM ixerM ixerM ixerSession 4 Session 3 Session 2 Session 1G F XGFX Inserted atend of graph ifencoding GF XLim iterM eterVolum eMeter APO onlyinserted if no HWvolume supportVolume APOonly inserted if noHW volumesupportGlobalG r aphF M TFMT onlyinserted if KS pindoesn’t supportfloat renderingK STransportFigure 3. Audio session architectureFor further information on audio sessions and the core audio API, see the white papertitled Device Finish-Install Actions in Windows Vista.March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 8How to Install sAPOsCustom sAPOs are installed with the associated device driver by using an INF file. Thedriver package includes the LFX and GFX sAPOs and any associated configuration UI.The device installation program or a setup program copies the sAPOs andconfiguration UI to the system and registers them.A GFX or LFX sAPO is identified by its COM class ID (CLSID). The INF can associate eachphysical output device’s PnP ID with only one GFX and one LFX. The association isspecified with registry directives in the INF file. The audio engine accesses thisregistry information through the IPropertyStore interfaces on Multimedia Device API(MMDevAPI) objects.Because an audio adapter can support multiple audio inputs and outputs, an sAPOmight not be compatible with all input types. If so, the sAPO must be explicitlyassociated with the compatible kernel-streaming (KS) node types.Specifying the compatible KS node types may still not be sufficient for multiplexedcapture devices such as microphone arrays, which can have multiple inputs with thesame KS node type. In that case, the sAPO must determine whether it can operate onthe current endpoint by querying the audio driver or endpoint property store foradditional information. If the system effect cannot operate on the endpoint, the sAPOshould behave like a copy or pass-through APO. This means that the sAPO shouldbypass its internal DSP algorithms and not fail initialization.Note: The audio engine does not normally load an unsigned sAPO. However, signingis not done until the development process is complete and the driver has beensubmitted to WHQL. For development and test purposes, the signing requirementcan be bypassed by setting the DisableProtectedAudioDG registry value to 1, asshown in this example:HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\AudioRegistry Settings"DisableProtectedAudioDG"=dword:00000001The association between LFX or GFX sAPOs and the related device is stored under theregistry key for the device interface. The DLL that contains the sAPOs must selfregisterby including a ‘RegisterDlls’ statement in the INF file.The HKEY_LOCAL_MACHINE/SYSTEM/CurrentControlSet/Control/DeviceClasses keyhas a subkey for each device interface that is named with the interface’s CLSID string.Under each CLSID key is a subkey for each device that exports the interface. Forexample,##?#PCI#VEN_1106&DEV_3059&SUBSYS_810A1043&REV_60#3&61AAA01&1&8D#{6994ad04-93ef-11d0-a3cc-00a0c9223196} is the subkey of {6994AD04-93EF-11D0-A3CC-00A0C9223196} that represents the device for the VIA onboard AC '97 audiocontroller with a RealTek AC '97 codec.March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 9Each device key has subkeys for all the interfaces that it exposes. For example, thedevice subkey under discussion has subkeys for wave, topology, and UniversalAsynchronous Receiver Transmitter (UART) interfaces that are named #Wave,#Topology, and #UART, respectively. The following example shows these settingsschematically. Note that the device subkey string has been truncated for readability.HKEY_LOCAL_MACHINESYSTEMCurrentControlSetControlDeviceClasses…{6994AD04-93EF-11D0-A3CC-00A0C9223196}##?#PCI#VEN_1106&DEV_3059&SUBSYS_810A1043&REV_60…#Topology#UART#Wave…To register a GFX or LFX sAPO, add a DeviceParameters subkey to the appropriatetopology subkey, followed by an FX subkey. The FX key can have one or more systemeffect subkeys, one for each effect's sAPO. The system effect subkey names must beintegers, starting with zero.The data for the property store is contained in a series of values that are associatedwith the system effect key. The value names are globally unique identifier (GUID)strings, followed by an ID, much like property store key names. The property namesand the associated data are listed in the following table.Property nameIDTypeDataSysFxAssociation 0 Multistring KSNODE_TYPE GUIDs that associate the sAPOwith the connection types that the sAPOsupports. Use a NULL GUID to associate thesAPO with any connection type. If theconnection type does not match the GUID, thesystem effects are not added.PreMixEffect 1 String The CLSID that is associated with the data andcode that are used to create the LFX sAPO.PostMixEffect 2 String The CLSID that is associated with the data andcode that are used to create the GFX sAPO.UserInterface 3 String The CLSID that is associated with the data andcode that are used to create the configurationUI for the system effects package.FriendlyName 4 String A friendly name for the system effects package.Additionalentries asappropriate>4 UserdefinedInitialization data for the sAPO.The value name for each property has the form “{GUID},ID”, where GUID is thestandard property key GUID string, {D04E05A6-594B-4fb6-A80D-01AF5EED7D1D}. It isMarch 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 10defined in AudioEngineBaseAPO.idl and is the same for all properties. For example,the value for the SysFxAssociation property is named “{D04E05A6-594B-4fb6-A80D-01AF5EED7D1D},0”.Figure 4 is a screenshot of RegEdit that shows a typical example of how system effectkeys and values appear in the registry. The device has one sAPO that is namedMicrosoft Audio Home Theater Effects. It works with any connection type, and hasLFX, GFX, and configuration UI sAPOs.Figure 4. Registry settings for Microsoft Audio Home Theater EffectsINF Files for sAPOsThis section shows examples from an INF that is used to install an sAPO. Forreadability, it is useful to define friendly names for the property store value names.For example:; PropertyKeysPKEY_FX_Association = "{D04E05A6-594B-4fb6-A80D-01AF5EED7D1D},0"PKEY_FX_PreMixClsid = "{D04E05A6-594B-4fb6-A80D-01AF5EED7D1D},1"PKEY_FX_PostMixClsid = "{D04E05A6-594B-4fb6-A80D-01AF5EED7D1D},2"PKEY_FX_UiClsid= "{D04E05A6-594B-4fb6-A80D-01AF5EED7D1D},3"PKEY_FriendlyName = "{ D04E05A6-594B-4fb6-A80D-01AF5EED7D1D },4"The following example shows the INF text to add the keys and property values to theregistry:HKR,"FX\\0",%PKEY_FriendlyName%,,%FX_FriendlyName%HKR,"FX\\0",%PKEY_FX_PreMixClsid%,,%FX_PREMIX_CLSID%HKR,"FX\\0",%PKEY_FX_PostMixClsid%,,%FX_POSTMIX_CLSID%HKR,"FX\\0",%PKEY_FX_UiClsid%,,%FX_UI_CLSID%HKR,"FX\\0",%PKEY_FX_Association%,,%KSNODETYPE_ANY%The final element of each line is a friendly name for the associated property storedata. The exact values are defined elsewhere in the INF file, but not shown here.March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 11It is possible to register multiple sAPOs by creating additional subkeys under FX andadding the sAPO's property store data to these keys. However, the names of thesubkeys that contain the property store data must be sequential integers, startingwith zero. The above example registers a single sAPO, so the property store data goesunder FX\0. The code to register a second sAPO would look similar, but the datawould go under FX\1, the third under FX\2, and so on.Enhancements Property Page ReplacementMicrosoft supplies an Enhancements property page for the Audio Control Panel item.The page is associated with the Microsoft Audio Home Theater Effects sAPO. Vendorscan replace this property page with a custom page by implementing and registering acustom property page provider. To implement the provider, use the standardtechniques that are documented in the Platform SDK. To replace the default propertypage, install the DLL that contains the custom property page implementation andmodify the appropriate registry value.The Enhancements property page is registered under theHKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\MMDevices\Audio\Render\Endpoint\FXkey. Endpoint is the identifier of the endpoint to which the propertysheet is to be associated. The property page is specified by assigning its CLSID to theregistry value that is associated with the UserInterface property store value. Forexample:HKLMSOFTWAREMicrosoftWindowsCurrentVersionMMDevicesAudioRenderEndpointFXProperties..."{d04e05a6-594b-4fb6-a80d-01af5eed7d1d},3"="{YOUR CLSID GOES HERE}"To replace the default property page with a custom implementation, replace its CLSIDwith the CLSID of the custom property page. The custom property page provider isinstalled with the rest of the driver package, by using an INF. The following example isan excerpt from an INF that shows how to register an Enhancements property pageDLL:[SysFx.AddReg]…HKR,"FX\\0",%PKEY_SYSFX_UiClsid%,,%SYSFX_UI_CLSID%…[Strings]March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 12PKEY_SYSFX_UiClsid = "{D04E05A6-594B-4FB6-A80D-01AF5EED7D1D},3"SYSFX_UI_CLSID…= "{YOUR GUID GOES HERE}"Note: The INF does not modify the FXProperties registry key directly. Instead, itmodifies the FX key as shown in the preceding example, and the value is thenmigrated to the FXProperties key in the MMDevice section.Run-Time Behavior of sAPOsAfter the device driver has been installed and started up, it creates a device interfacethat the system can use to play or record audio. The audio server component listensfor the arrival of device interfaces. When one arrives, the audio server opens thedevice interface’s registry entry and looks for the FX subkey. If the subkey is present,the audio server reads the information under the key.At the same time, the audio engine generates the endpoints, based on policy settings.It reads the device interface registry keys and propagates the data from theassociated values to the appropriate property store.sAPO BypassAfter the endpoints and endpoint property stores have been created, the audioservice builds the audio device graph that handles the signal processing. Theendpoint property store has a global effects bypass setting,PKEY_Endpoint_Disable_SysFx, which enables or disables system effects. A check boxon the Audio Control Panel item allows the user to set this property. By default,system effects are disabled. If system effects are enabled and LFX or GFX sAPOs areassociated with the device, the audio service adds them to the graph along with thestandard audio sAPOs.The audio service registers for property store notifications so it can monitor changesin the GFX and LFX settings as well as changes in the effects bypass setting. If theaudio service receives a notification, it signals a configuration change to all associatedaudio servers. From that point, the operation proceeds as it does for device formatchanges.sAPO InitializationAfter the sAPOs are inserted in the graph, the graph builder initializes them. An sAPOmust export IAudioSystemEffects, an interface that identifies the sAPO as an audioeffect. If the query for IAudioSystemEffects is successful, the graph builder fills outthe initialization structure, APOInitSystemEffects, and passes it to the sAPO’sInitialize method. The structure includes the following data:• An IPropertyStore interface pointer for the device endpoint’s property store.• An IPropertyStore interface pointer for the audio effect’s property store.• A string that contains the device’s PnP hardware ID.• An IMMDeviceCollection object that contains IMMDevice interface pointersto several objects.March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 13Note: ImmDeviceCollection and IMMDevice are part of the MMDeviceAPI, which isnew for Windows Vista. These interfaces are declared in Mmdeviceapi.h. Also notethat the collection that is referenced by the IMMDeviceCollection object containsitems that support the IMMDevice interface. This collection contains only one audioendpoint item that supports IMMEndpoint.To find this item, the APO can traverse the list of items and call QueryInterface oneach item until it finds the one that supports IMMEndpoint. Once the APO hasidentified the endpoint item, the APO can use IMMEndpoint::GetDataFlow todetermine whether that endpoint is an input or an output. The APO can also accessthe property store to get more information about thePKEY_AudioEndpoint_FormFactor.The number and types of other items in the collection depend on the underlyingimplementation of the audio endpoint. For typical audio devices that are supportedby PortCls, AVStream, and other Kernel Streaming drivers, the collection itemsrepresent the KS filters that are implemented by the driver. If the APO needs tointeract directly with the audio hardware driver, the APO can do so via theIDeviceTopolgy interface of one or more of the IMMDevice items (not the endpointitem) referenced by the IMMDeviceCollection object. To activate and use theIDeviceTopolgy interface for an IMMDevice item, the APO must callIMMDevice::Activate, and provide the REFIID for the IDeviceTopology interface forthe item. However for most APOs, direct interaction with the hardware driver shouldnot be necessary.Endpoint Property Store SettingsThe endpoint property store serves as the communication mechanism between theUI and the LFX and GFX sAPOs. Because the user can change settings at any time, GFXand LFX sAPOs should register for change notifications on any audio endpointproperty that holds one of their settings.During initialization, the sAPO’s Initialize method receives interface pointers thatprovide access to the endpoint property store. Several categories of settings in theendpoint property store could be of interest:• Common properties: well-known Microsoft-defined metadata such asheadphones, microphone arrays, or geometry.• Custom properties: IHV-defined metadata that further characterize the endpoint.• Custom system effect properties: IHV-defined metadata that characterize thesAPOs.sAPO Format NegotiationAfter the sAPO has been initialized, the audio service negotiates the sample formatand activates the APO graph for processing. During format negotiation, the LFX isprovided the mix format as the input’s opposite (output). The LFX can return S_FALSEto allow system channel matrixing. Otherwise, if the LFX can successfully process thisinput/output pair, it returns S_OK. After format negotiation succeeds for the entireMarch 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 14graph, the sAPOs are given the final format when their LockForProcess methods arecalled.From the perspective of the audio subsystem, an sAPO is a black box with unknowncapabilities. However, the graph builder and other APOs must negotiate a compatiblesample format. To do so, they are queried for supported input or output formats byinstantiating and initializing the sAPO and then calling itsIAudioProcessingObject::IsInputFormatSupported orIAudioProcessingObject::IsOutputFormatSupported methods. That means that aGFX must be able to respond without the presence of an LFX and vice versa. An sAPOshould respond to the query as follows:• If the sAPO supports the suggested format, it returns S_OK and adds a referenceto the successful format.• If the sAPO does not support the suggested format, it suggests an alternative byselecting the “closest” format that it does support and returning S_FALSE.However, an sAPO should suggest an alternate format only if it has some basis fordetermining “closest.”• If selecting a “closest" format would be nothing more than a guess, the sAPOindicates that it cannot support the suggested format by returningAPOERR_FORMAT_NOT_SUPPORTED.• For a detailed description of IsInputFormatSupported andIsOutputFormatSupported, see the comments in AudioEngineBaseAPO.idl .Data FormatsAll the system sAPOs are designed to process float32 pulse code modulation (PCM)data. To prevent unnecessary format conversions, LFX and GFX sAPOs should processdata in float32 PCM format. A typical GFX for an int16 PCM endpoint uses the float32PCM format for both input and output. The graph builder attaches a system-providedfloat32-int16 converter to the endpoint-facing side of the sAPO to convert the audiostream to or from int16 PCM format. GFX sAPOs can omit the system-supplied formatconverter and process data in the native device format. However, the audio enginefacingside of the GFX must still support the float32 PCM format.GFX sAPOs can support an encoded output format such as AC3 that can handlemultiple speaker configurations. In that case, the sAPO should get the speakerconfiguration information from the endpoint property store. It can then returnS_FALSE to suggest an input (mix) format when IsInputFormatSupported is calledwith the encoded format as opposite. The GFX sAPO should return a float format thathas the sample rate and speaker configuration of the “decoded” format.Note: An sAPO can suggest an audio format only during calls toIsInputFormatSupported or IsOutputFormatSupported.March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 15The sAPO LockForProcess MethodAfter the sample format has been finalized, the graph builder calls the sAPO’sIAudioProcessingObjectConfiguration::LockForProcess method to finalize the inputand output formats and give the sAPO a final opportunity to finish initialization. If thisis successful, the object is locked in its current state. No format configuration changesare allowed after the object is locked. TheIAudioProcessingObjectConfiguration::UnlockForProcess method is called to unlockthe sAPO.sAPO Failure Monitoring and Automatic DisablingThe audio system monitors sAPO return codes to determine whether sAPOs are beingsuccessfully incorporated into the graph. It does so by tracking the HRESULT valuesthat are returned by any of several designated methods. The system maintains afailure count value for each sAPO that is being incorporated into the graph, typicallyan LFX and a GFX sAPO. A single registry value, PKEY_Endpoint_Disable_SysFx,disables both sAPOs when it is set.• The system increments an sAPO's failure count each time one of the designatedmethods fails. Certain failure codes might be excluded from this count.• If an sAPO indicates that it has been successfully incorporated into the graph, itsfailure count is reset to zero.• If the failure count for either sAPO reaches a system-specified limit, both sAPOsare disabled by setting PKEY_Endpoint_Disable_SysFx. Currently, this limit is setto 10, but that might change in the future.The system monitors four primary API calls:• CoCreateInstance: The attempt to create an sAPO can fail for a number ofreasons, including:• The graph is running protected content, and the sAPO is not properly signed.• The sAPO is not registered.• The sAPO has been renamed or tampered with.• IsInputFormatSupported/IsOutputFormatSupported:IsOutputFormatSupported: The system increments thesAPO's failure count if either method returns a failure code.• LockForProcess. If this call is successful, the sAPO failure count is reset to zero.Otherwise, the failure count is incremented.Application Control over sAPOsIf an application doesn’t want any processing on either the render or capture pipeline,it should open the endpoint in exclusive mode. However, this approach has somelimitations that often make it undesirable:• No other application can use the endpoint.• The signal processing graph does not process the audio stream. This means thatfeatures such as volume control and automatic gain control (AGC) are disabled.March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 16Note: Currently, capture effects are limited to post-split, application-specificprocessing. This means that they can be implemented only by LFX sAPOs.Applications cannot query for the type of effects that an sAPO applies to the audiostream. That means that applications cannot make any meaningful decisions on whateffects to offer to the user or what effects to enable in the streaming pipeline. As apractical matter, this is typically not a major problem. Most problems can often beavoided by using some intelligence. For example, try the native stream format beforeinserting an HP visualization.How to Implement a Custom sAPOCustom sAPOs are implemented as in-process COM objects and packaged as DLLs.This section outlines the basics of how to implement an sAPO. For details, see thesAPO software development kit (SDK), which is part of the Platform SDK. Thefollowing list gives some basic design considerations.• An sAPO can have one input and one output connection. This connection is anaudio buffer and can be multichannel.• An sAPO must be real-time compatible. This means:• All buffers that are processed by the sAPO must be nonpageable.• All code and data in the process path must be nonpageable.• All methods that are members of real-time interfaces must be implementedas nonblocking. They should not block, use paged memory, or call anyblocking system routines. Note that most system routines are blocking. Realtimeinterfaces are identified by an “RT” at the end of their name, such asIAudioProcessingObjectRT.• Real-time compatible sAPOs can be used in contexts that do not require realtimeprocessing.• An sAPO can modify only the audio data that is passed to it through itsIAudioProcessingObjectRT::APOProcess routine. It cannot change the settings ofthe underlying logical device, including its KS topology.• GFX sAPOs must process data by using a format with a fixed frame size. Theframe size is specified by the advertised supported format. The format is staticfor both input and output.• GFX sAPOs should avoid introducing more than 10 ms of latency into the audioprocessing chain. An sAPO reports this latency through theIAudioProcessingObject::GetLatency API.Required InterfacesAn sAPO must export three required interfaces in addition to IUnknown:• IAudioProcessingObject. An interface that handles setup tasks such asinitialization and format negotiation.• IAudioProcessingObjectRT. A real-time interface that handles audio processing. Itcan be called from a real-time processing thread.• IAudioProcessingObjectConfiguration. The configuration interface.March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 17For detailed documentation of the required interfaces, see AudioEngineBaseAPO.idl.Note: Custom sAPOs should not export the IAudioProcessingObjectVBR interface.The CBaseAudioProcessingObject ClassCBaseAudioProcessingObject is a base class that implements much of thefunctionality that an sAPO requires. It provides default implementations for most ofthe methods in the three required interfaces. The primary exception isIAudioProcessingObjectRT::APOProcess, which is where the custom signal processingalgorithms are implemented. The class declaration is contained inBaseAudioProcessingObject.h, which is included with the Windows Driver Kit (WDK).Using CBaseAudioProcessingObject can significantly reduce the effort that isrequired to implement an sAPO. If an sAPO has no special format requirements andoperates on the default float32 format, the default implementations of the interfacemethods that are included in CBaseAudioProcessingObject should be sufficient andonly two methods must be implemented: IAudioProcessingObjectRT::APOProcessand ValidateAndCacheConnectionInfo. However, CBaseAudioProcessingObject isnot suitable for all sAPOs.• If an sAPO processes only uncompressed PCM data, usingCBaseAudioProcessingObject is strongly recommended.• If an sAPO must support non-PCM formats such as AC3 or MP3, it cannot useCBaseAudioProcessingObject. Vendors must provide their own implementationsof the required interfaces.To use CBaseAudioProcessingObject, create a custom class that inherits fromCBaseAudioProcessingObject and implement two of its methods.• IAudioProcessingObjectRT::APOProcess. This method handles the actual signalprocessing and contains the custom signal processing algorithms.• ValidateAndCacheConnectionInfo. The graph builder callsIAudioProcessingObjectConfiguration::LockForProcess to finalize formatnegotiation and lock the sAPO for processing. The default implementation of thismethod that is included in CBaseAudioProcessingObject has no specificknowledge of the sAPO. It does some standard connection checking andessentially forwards the query to ValidateAndCacheConnectionInfo. The sAPO’simplementation of ValidateAndCacheConnectionInfo should allocate memoryand store any details about format that might be required later. Format detailstypically include items such as channel count, sampling rate, sample depth,channel mask, and so on.AudioEngineBaseApo.idl is included in the WDK and contains detailed comments onhow to implement all of the required interfaces. In particular, examine the commentsfor IAudioProcessingObjectConfiguration::LockForProcess.ValidateAndCacheConnectionInfo has an identical signature and must performessentially the same tasks, so the comments are also relevant for that method.March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 18Signal Processing RequirementsAn sAPOs should observe the following limitations on its processing algorithms.• Do not use nonlinear processing. It can interfere with other processing algorithmsthat the system’s application audio graph provides, such as AEC.• Processing should be nonblocking. In general, the time that is required forprocessing to run from beginning to end should stay within a well-defined upperbound.• Support IEEE 32-bit float input and output. The audio engine’s native sAPOs alloperate on a float32 data path.• An sAPO can have one input and one output audio connection.How to Implement a UI for Configuring the EffectsResourcesThe CLSID of the system effect's UI sAPO is available from the audio endpoint’sproperty store. The Audio Control Panel item gets this CLSID from the audio endpointthat is currently in context. When the Audio Control Panel item launches theappropriate custom system effect UI, it passes it the audio endpoint. The UI can thenaccess the endpoint property store to read and adjust property settings. The UIshould also register for property store notifications in case some other programmodifies the settings.White Papers and Related Content:A Wave Port Driver for Real-Time Audio Streaminghttp://www.microsoft.com/whdc/device/audio/wavertport.mspxAudio Device Technologies for Windowshttp://www.microsoft.com/whdc/device/audio/default.mspxDevice Finish-Install Actions in Windows Vistahttp://msdn.microsoft.com/en-us/library/ff544940.aspxPin Configuration Guidelines for High Definition Audio Deviceshttp://www.microsoft.com/whdc/device/audio/PinConfig.mspxPlug and Play Guidelines for High Definition Audio Deviceshttp://www.microsoft.com/whdc/device/audio/HD-aud_PnP.mspxUniversal Audio Architecturehttp://www.microsoft.com/whdc/device/audio/uaa.mspxUAA Hardware Design Guidelineshttp://www.microsoft.com/whdc/device/audio/UAA_HWdesign.mspxWindows Vista Driver Developmenthttp://www.microsoft.com/whdc/driver/WDK/March 8, 2012© 2012 Microsoft Corporation. All rights reserved.

Custom Audio Effects in Windows Vista - 19Development Kits :Windows Driver Kit (WDK)http://www.microsoft.com/whdc/DevTools/WDK/WDKpkg.mspxWindows SDKhttp://msdn.microsoft.com/en-us/windows/bb980924.aspxMarch 8, 2012© 2012 Microsoft Corporation. All rights reserved.