Data Management for New Trading Opportunities - Sybase
Data Management for New Trading Opportunities - Sybase
Data Management for New Trading Opportunities - Sybase
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
Researched and written by:<br />
<strong>Data</strong> <strong>Management</strong> <strong>for</strong> <strong>New</strong> <strong>Trading</strong><br />
<strong>Opportunities</strong><br />
September 2010<br />
An industry briefing prepared<br />
by A-Team Group <strong>for</strong>
<strong>Data</strong> <strong>Management</strong> <strong>for</strong> <strong>New</strong> <strong>Trading</strong> <strong>Opportunities</strong><br />
Introduction<br />
As high-frequency and quantitative trading techniques mature, trading firms are finding it<br />
harder to make money. Market practitioners are recognizing that speed of market access<br />
alone is no longer sufficient to stay ahead of the pack, as low-latency connectivity enters<br />
the mainstream.<br />
The emphasis is returning to the quality of the trading model. In order to gain an edge,<br />
innovators are using more sophisticated techniques to improve the effectiveness of their<br />
trading models. These techniques range from analysis of unstructured data to generate<br />
an assessment of market sentiment, to the use of emerging machine-readable news<br />
services.<br />
In addition to a focus on the quality of the trading model, firms are using new capabilities<br />
to identify cross-asset opportunities: across linked instruments, linked execution<br />
venues and linked geographies. And they’re applying many of the techniques used in<br />
listed markets like equities, futures and options to over-the-counter markets. The market<br />
has already witnessed firms deploying event-driven and algorithmic models <strong>for</strong> <strong>for</strong>eign<br />
exchange trading, and fixed income and derivatives capabilities are emerging.<br />
Key to success in all of these new trading approaches is a sophisticated data management<br />
strategy, as firms are <strong>for</strong>ced to draw upon new data sources to drive their models<br />
and trading applications. These new sources range widely in type: from traditional but<br />
unstructured data, such as text-based company reports, to machine-readable news<br />
(MRN) and unfamiliar data sets used in OTC markets, like synthetic pricing and valuations<br />
models.<br />
This briefing looks at how trading firms are using innovative techniques to take trading to<br />
the next level, and the data management challenges that they need to overcome to be<br />
successful.<br />
An industry briefing prepared by A-TEAM GROUP <strong>for</strong> <strong>Sybase</strong>, Inc., an SAP company 2
<strong>Data</strong> <strong>Management</strong> <strong>for</strong> <strong>New</strong> <strong>Trading</strong> <strong>Opportunities</strong><br />
Current Landscape and Trends<br />
Over the past 18 months, high-frequency trading (HTF) has emerged as a major contributor<br />
to trading volumes both in U.S. equity and listed derivatives markets and, increasingly,<br />
in European markets. The ubiquity of low-latency connectivity and high-per<strong>for</strong>mance<br />
trading plat<strong>for</strong>ms— often offered by prime brokers to hedge funds and a new generation<br />
of independent proprietary trading outfits—has meant that players small and large have<br />
been able to move swiftly in and out of increasingly fast markets. This, and the emergence<br />
of new types of liquidity providers, like Citadel and Getco, has boosted the proportion<br />
of trading volumes attributed to HFT.<br />
High-frequency trading<br />
essentially involves<br />
the use of high-speed<br />
market access to quickly<br />
take advantage of market<br />
opportunities as they<br />
arise. This often means<br />
scheduling quantitative<br />
models to generate<br />
many small trades, each yielding relatively small incremental returns. Many high-frequency<br />
traders employ statistical arbitrage models, which allow them to identify and execute<br />
on pricing discrepancies between individual securities, sectors or markets.<br />
“As the high-frequency trading segment<br />
enters maturity, emphasis has returned to<br />
the quality of the trading strategy. . ”<br />
Such has been the success of the high-frequency trading community that new entrants<br />
continue to adopt its techniques, aided by the broad availability of the kinds of trading<br />
plat<strong>for</strong>ms needed to participate. As a result, some estimate that high-frequency trading<br />
now accounts <strong>for</strong> as much as 70% to 80% of total equities volumes in the U.S., with Europe<br />
at more than 30% and growing fast as emerging execution venues offer new arbitrage<br />
opportunities. Many expect the developing Asian markets to follow much the same<br />
path.<br />
As the high-frequency trading segment enters maturity, though, emphasis has returned to<br />
the quality of the trading strategy as alpha seekers attempt to wring profit out of a more<br />
competitive and complex trading environment. Hedge funds, prop traders and other users<br />
of quantitative and algorithmic trading models are now looking elsewhere to find an edge.<br />
Increasingly, that involves drawing upon a wider range of data inputs to drive their trading<br />
strategies. Traders are looking beyond traditional data sources— the price data feeds<br />
from the major exchanges and aggregators/redistributors — to find new factors potentially<br />
affecting the value of financial instruments.<br />
Many of these new sources of data are unstructured. That is to say, data is available in<br />
a raw <strong>for</strong>mat only, and is not subject to the usual data cleansing, <strong>for</strong>matting and other<br />
processes typically offered according to a commercial supplier’s data model.<br />
These new data sources are wide-ranging in character. They include raw, text-based<br />
company financial data, web-based news services, social media sites and even feeds<br />
of closed-caption TV coverage. Even where the data is more structured—such as is the<br />
case among the growing number of machine-readable news services—trading firms are<br />
using the new sources in unique ways to gain an edge. By feeding a machine-readable<br />
news feed into a trading algorithm, <strong>for</strong> example, many firms are coming up with their own<br />
assessments of market sentiment, based on their own reading of news content.<br />
An industry briefing prepared by A-TEAM GROUP <strong>for</strong> <strong>Sybase</strong>, Inc., an SAP company 3
<strong>Data</strong> <strong>Management</strong> <strong>for</strong> <strong>New</strong> <strong>Trading</strong> <strong>Opportunities</strong><br />
Whatever its potential, this explosion of useful data sources is posing major headaches<br />
<strong>for</strong> data managers charged with helping analysts and traders identify and exploit market<br />
opportunities. The challenges of combining traditional structured data sources with new<br />
and disparate unstructured ones are manifold.<br />
Many would agree the challenge of managing today’s high-speed direct exchange feeds<br />
alone is a significant one. Adding demands <strong>for</strong> incorporating unstructured data from a<br />
wide variety of sources is adding to the data management challenge facing CIOs and<br />
their staffs.<br />
<strong>New</strong> data management technologies, though, are presenting financial institutions with opportunities<br />
to harness this data deluge. The ability to embrace unstructured data is helping<br />
firms build sentiment analysis, trade signaling and stock-screening applications that<br />
can take into account text-based in<strong>for</strong>mation such as company reports and, increasingly,<br />
machine-readable news services.<br />
At the same time, firms are looking to expand the use of techniques they’ve learned from<br />
listed securities—equities, and related futures and options—to other, over-the-counter asset<br />
classes. Particular beneficiaries of this trend are expected to be players in the <strong>for</strong>eign<br />
exchange and fixed income markets.<br />
<strong>New</strong> Client Applications<br />
As they strive to differentiate their execution strategies from those of their rivals, trading<br />
firms are drawing upon new techniques and new sources of in<strong>for</strong>mation. Armed with<br />
access to a broader range of data, they are able to devise new ways of looking at the<br />
marketplace, with a view to giving themselves a unique perspective on the trade decision-making<br />
process.<br />
The adoption of technologies from other industries—most notably from military surveillance,<br />
which processes vast amounts of data—is allowing firms to process in<strong>for</strong>mation<br />
from some unlikely sources. While many are using familiar unstructured text-based data<br />
like company reports, regulatory filings and the like, others are looking further afield.<br />
Practitioners report the use of sources like websites, social media and even closed-caption<br />
TV feeds.<br />
The latter may represent the obscure end of the data spectrum, to say the least. But<br />
recent months have witnessed the emergence of several asset-based social media sites,<br />
which have grabbed the attention of traders in their respective markets and are being<br />
factored into the broader trade decision-making process. Another area that has seen<br />
perhaps the most significant advancement is in the use of machine-readable news, which<br />
is increasingly behind an array of new decision-support application types.<br />
The analysis of these types of data can create high yields in applications like sentiment<br />
measurement. Here, investment analysts and traders seek to gauge the market’s feeling<br />
about a stock, pair of stocks, sector or index, based on a view of relevant parameters,<br />
rather than on simple price or index values or straight news coverage.<br />
A number of decision-support applications providers, like RavenPack and Alphacet, have<br />
developed models <strong>for</strong> assessing the impact of market events on a given investment target.<br />
Their applications can be used to measure sentiment by, <strong>for</strong> example, analyzing the<br />
volume of ‘positive’ vs. ‘negative’ news items on a given stock. The resulting sentiment<br />
An industry briefing prepared by A-TEAM GROUP <strong>for</strong> <strong>Sybase</strong>, Inc., an SAP company 4
<strong>Data</strong> <strong>Management</strong> <strong>for</strong> <strong>New</strong> <strong>Trading</strong> <strong>Opportunities</strong><br />
measure can be fed into trading algorithms often running within complex event processing<br />
(CEP) frameworks, and executed as per the sponsoring broker’s assessment of the<br />
situation.<br />
There are many variants on the use of machine-readable news. Practitioners are using<br />
MRN to drive traditional models, like directional trading signals and other trading algorithms,<br />
and <strong>for</strong> back-testing their trading strategies. But they’re also using such services,<br />
available from suppliers like Dow Jones, Reuters and Market <strong>New</strong>s International, to drive<br />
stock-screening applications. And more innovative developers are developing custom<br />
applications that, <strong>for</strong> example, can predict the impact on specific stocks or sectors of<br />
potentially volatile trading days, based on trend analysis.<br />
Indeed, volatility is a recurring theme among trading model developers, particularly as<br />
the days of exploiting volatility per se may be drawing to a close. The widely watched VIX<br />
Volatility Index calculated by the Chicago Board Options Exchange has entered what appears<br />
to be a period of lower values.<br />
This could prove significant, as VIX is regarded as a measure of the market’s expectations<br />
of future volatility. Lower VIX index values typically mean that volatility—the lifeblood<br />
of the high-frequency trader—is heading lower. Should that be the case, it will become<br />
increasingly important to wring maximum leverage out of any volatility-based applications.<br />
As a result, traders are watching, and are programming their algorithms to watch<br />
<strong>for</strong> any indications of volatility so that they can take full advantage of the opportunities it<br />
presents.<br />
Here, the proliferation of content through social media like Twitter is becoming increasingly<br />
important. By monitoring ‘tweets’, traders’ algorithms are able to pick up on high<br />
volumes of activity of in<strong>for</strong>mation around a given target stock much earlier than through<br />
traditional news outlets. In a world where milliseconds count, social media can offer traders<br />
minutes’ and even tens of minutes’ worth of advantage over competitors relying on<br />
traditional in<strong>for</strong>mation sources.<br />
Other emerging decision-support applications include chart pattern recognition and<br />
actionable investment research. Chart pattern recognition tools typically draw upon a<br />
wide variety of unstructured ‘primary’ in<strong>for</strong>mation sources, including other analytical<br />
applications, broker and other research, and internal back-testing systems to evaluate<br />
how a given security behaves over time versus investment targets. Actionable investment<br />
research, meanwhile, attempts to automate the interpretation of technical, fundamental<br />
and other analytics to generate trading ideas.<br />
To harness the potential of these techniques, players are turning to emerging technologies<br />
such as complex event processing (CEP), which offer the ability to manage and process<br />
vast quantities off data in real time. Being able to respond quickly to market events and<br />
opportunities is a major step <strong>for</strong>ward <strong>for</strong> financial institutions, which only a decade ago<br />
may have had to rely on overnight (or worse!) batch processing to run the kind of complex<br />
analytics CEP can handle in real time today.<br />
CEP engines are increasingly used by financial institutions to gather intelligence from<br />
events in real time that may be related to specific market conditions or emerging trading<br />
patterns. CEP can allow firms to develop sophisticated tools <strong>for</strong> defining the processing<br />
and analysis of events, notifications of which may arrive in any variety of message <strong>for</strong>mats.<br />
An industry briefing prepared by A-TEAM GROUP <strong>for</strong> <strong>Sybase</strong>, Inc., an SAP company 5
<strong>Data</strong> <strong>Management</strong> <strong>for</strong> <strong>New</strong> <strong>Trading</strong> <strong>Opportunities</strong><br />
An event-driven architecture based on CEP can deliver the resources to assimilate, assemble,<br />
interrelate and analyse incoming messages, creating new high-level messages<br />
that can be applied to analytical applications. As such, these CEP engines correlate data<br />
from multiple sources, and from them derive richer and more complete data sets that can<br />
be analysed more effectively, often resulting in faster decision-making.<br />
The CEP-based infrastructure can be used to identify trading patterns – or individual<br />
events that are exceptions to these patterns – again offering users the opportunity to<br />
respond quickly. As part of the overall picture, firms gain a more comprehensive view on<br />
impacts of market events, backed up by summaries, statistics and trend analysis. Finally,<br />
as a byproduct, the financial institutions is able to create a historical database of standardised<br />
data that can be used to underpin future analytical applications.<br />
One financial institution that recently took advantage of modern CEP-based technologies<br />
to extend sophisticated analysis to its trading operation is London-based Mitsubishi UFJ<br />
Securities International (MUSI). Starting with a blank slate, the firm – part of the Mitsubishi<br />
UFJ Financial Group – had a rare opportunity to develop a trading infrastructure <strong>for</strong> its<br />
Rates <strong>Trading</strong> Group that could capture and process tick data and combine it with historically<br />
rich market data.<br />
Taking into account internal IT expertise, MUSI’s system requirements included: fast capture<br />
of all tick data; predicting price movements and identifying arbitrage opportunities;<br />
processing real-time market data; storing massive amounts of data and retrieving this<br />
in<strong>for</strong>mation rapidly; and quickly analyzing historical trends and relationships.<br />
MUSI embarked on an evaluation of a variety of existing technology products, benchmarking<br />
them to determine how well they met key criteria, including:<br />
• Fast data retrieval<br />
• Rapid implementation<br />
• Capture of real-time tick data<br />
• Efficient storage<br />
• Leveraging existing IT skill sets<br />
• Maintenance and availability of technical support personnel.<br />
Following the evaluation, the firm chose <strong>Sybase</strong>’s CEP solution Aleri Streaming Plat<strong>for</strong>m<br />
combined with the <strong>Sybase</strong> RAP – The <strong>Trading</strong> Edition to support decision processes and<br />
allow its 50 rates traders to trade more profitably.<br />
<strong>Sybase</strong> RAP – <strong>Trading</strong> Edition is a unique consolidated analytics plat<strong>for</strong>m designed to<br />
manage high-volume real-time data feeds as well as massive historical and market data<br />
sets. It offers access to huge repositories of leaf-level market data, and can store additional<br />
data vital to capital markets analytics. The result is greater speed and more efficient<br />
data compression, which is combined with an in-memory database <strong>for</strong> the capture of<br />
real-time streaming data.<br />
Designed <strong>for</strong> real-time analysis of streaming data, <strong>Sybase</strong> Aleri Streaming Plat<strong>for</strong>m is a<br />
high-per<strong>for</strong>mance CEP engine that can absorb and process high-volume data streams,<br />
such as market data, in real time, applying custom logic to normalize and analyze the<br />
data.<br />
An industry briefing prepared by A-TEAM GROUP <strong>for</strong> <strong>Sybase</strong>, Inc., an SAP company 6
<strong>Data</strong> <strong>Management</strong> <strong>for</strong> <strong>New</strong> <strong>Trading</strong> <strong>Opportunities</strong><br />
For MUSI, the combination offered a number of important benefits. Because <strong>Sybase</strong><br />
RAP – The <strong>Trading</strong> Edition writes to disk frequently, if memory capacity is reached there<br />
is no data loss and the database can pick up where it left off, avoiding the disastrous<br />
scenario of losing a whole day’s data. Per<strong>for</strong>mance improved massively, allowing traders<br />
to process and manipulate market data instantaneously rather than waiting minutes <strong>for</strong><br />
updates. And the plat<strong>for</strong>m’s single, cohesive design allowed traders to develop their own<br />
models and spreadsheets in a centralized system, allowing sharing and collaboration.<br />
Since implementation, MUSI has doubled tthe number of currencies it tracks, allowing a<br />
richer view into market opportunities – without having to add more storage. With <strong>Sybase</strong><br />
RAP – The <strong>Trading</strong> Edition, it has also increased its data retention fourfold, from an initial<br />
estimate of six months to two years. With richer data stores bearing no overhead costs,<br />
MUSI has a solution that enables it to manage its business today and identify new opportunities<br />
into the future.<br />
Migration to Adjacent Markets<br />
Market practitioners are also seeking to apply the expertise they’ve garnered in the equities<br />
markets to other, adjacent markets. This involves both extending the in<strong>for</strong>mationintensive<br />
equities analysis to linked markets – where listed corporations may have also<br />
issued other types of financial security – and migrating those techniques to wholly new<br />
areas, such as over-the-counter securities.<br />
Nowhere has the migration occurred more swiftly than in <strong>for</strong>eign exchange. This shift has<br />
been driven by cross-border trading, particularly in Europe, where the fragmentation of<br />
markets under the 2007 EU Markets in Financial Instruments Directive (MiFID) encouraged<br />
the multiple listing of stocks in multiple currencies. This exacerbated the existing<br />
need to add a currency hedge to cross-border transactions, and encouraged a market <strong>for</strong><br />
FX speculation.<br />
The vast number of instruments – stock, warrants, bonds, futures, other derivatives – now<br />
being issued by a single corporation is <strong>for</strong>cing investors to consider a broader range of<br />
asset classes. It’s no longer enough to trade the stock in isolation; it’s becoming imperative<br />
to consider the impact on the stock of the behaviour of various other securities<br />
issued by the same corporation. This in itself greatly expands the amount of data required<br />
to create a comprehensive view of a particular security.<br />
As well as keeping track of listed instrument values, practitioners also need to understand<br />
how valuations <strong>for</strong> illiquid unlisted, or over-the-counter, issues are <strong>for</strong>med. And, possibly<br />
most challenging of all, they need to be sure they have a comprehensive view of the<br />
spectrum of securities – listed or otherwise – that a target corporation has issued, and to<br />
keep that view up to date.<br />
These and other factors originally pushed firms into the use of high-per<strong>for</strong>mance trading<br />
techniques in listed futures and options. Indeed, many of the major prime brokers now<br />
offer similar kinds of algorithms and other trading tools <strong>for</strong> listed derivatives as they do<br />
<strong>for</strong> plain-vanilla equities. The adoption of fast execution plat<strong>for</strong>ms by futures and options<br />
exchanges added to the potential. Now, a similar migration is occurring in cash bond<br />
markets, as interdealer brokers, broker-to-dealer plat<strong>for</strong>ms and other bond execution<br />
systems shift to higher-per<strong>for</strong>mance systems.<br />
Other cross-asset activity is more sophisticated. Practitioners report the use of arbitrage<br />
techniques between specific equity securities and specific commodities, <strong>for</strong> example,<br />
An industry briefing prepared by A-TEAM GROUP <strong>for</strong> <strong>Sybase</strong>, Inc., an SAP company 7
<strong>Data</strong> <strong>Management</strong> <strong>for</strong> <strong>New</strong> <strong>Trading</strong> <strong>Opportunities</strong><br />
based on the beliefs of a particular portfolio manager or analyst and incorporated into that<br />
manager’s trading strategy.<br />
In all cases, financial institutions are seeking to apply the real-time pre-trade analysis they<br />
use <strong>for</strong> equity markets, to other asset classes. As electronic trading moves into over-thecounter<br />
markets like <strong>for</strong>eign exchange and fixed income, firms are adopting many of the<br />
latency, algorithmic trading and analysis techniques from their equities businesses. This<br />
phenomenon is expected to take off over the next few years as OTC markets create the<br />
technology conditions necessary <strong>for</strong> model-driven and high-frequency trading.<br />
<strong>Data</strong> Analysis and <strong>Management</strong><br />
Challenges <strong>for</strong> Market Practitioners<br />
Mainstream market data, and the ‘standard’ analytical applications it drives, have become<br />
commoditised. With speed’s importance as a differentiator waning, access to similar trading<br />
models driven by the same data sources makes it difficult or impossible <strong>for</strong> trading<br />
firms to compete <strong>for</strong> alpha or executions, or differentiate the services they offer to clients.<br />
As we’ve seen, this trend is <strong>for</strong>cing the business side to turn to new data sources, particularly<br />
in listed markets. This has driven the growing popularity of certain types of fundamental<br />
data, text-based in<strong>for</strong>mation and new capabilities like machine-readable news. In<br />
all of these instances, trading firms believe they can create an edge through innovative<br />
use of new data sources.<br />
Machine-readable news is a case in point. MRN is being used <strong>for</strong> a host of different purposes:<br />
algorithmic execution, portfolio rebalance, real-time risk management and market<br />
abuse detection. It holds the promise of allowing a trader to react to an event in near real<br />
time, rather than wait <strong>for</strong> days <strong>for</strong> analysts to publish their assessment of a market or<br />
company-specific event.<br />
One way MRN is being used to improve pre-trade decision-making is in sentiment<br />
measurement. This builds on the belief that news analytics when combined with standard<br />
measures like implied volatility can provide better estimates of future volatility than<br />
implied volatility models alone. Specifically, MRN can be used to help traders adapt their<br />
models – which have traditionally looked to the past to predict future trends – essentially<br />
to respond to the current market situation.<br />
Through the analysis of MRN services, advanced sentiment measurement applications<br />
can determine whether news coverage is generally negative, positive or neutral <strong>for</strong> a given<br />
stock, industry, segment or geopolitical entity. While this may not be a new use of news –<br />
traders have manually assessed incoming news services <strong>for</strong> sentiment since their inception<br />
– the automation that MRN makes possible can allow models to respond much more<br />
rapidly. Some estimates reckon it can take 20 minutes <strong>for</strong> the market to settle on a new<br />
equilibrium price <strong>for</strong> a stock in response to a market event using manual techniques.<br />
Automated MRN services can also help understand what the quantifiable trend actually<br />
is. For example, in bull markets there are generally twice as many positive stories about<br />
any given company than negative ones. But when stocks are falling, research has shown<br />
there are almost equal amounts of positive and negative news; automated analysis of<br />
MRN feeds can lend credibility to such an assessment under market conditions humans<br />
may find difficult.<br />
MRN is being used in a number of nuanced ways. It can be used to differentiate a model’s<br />
An industry briefing prepared by A-TEAM GROUP <strong>for</strong> <strong>Sybase</strong>, Inc., an SAP company 8
<strong>Data</strong> <strong>Management</strong> <strong>for</strong> <strong>New</strong> <strong>Trading</strong> <strong>Opportunities</strong><br />
response to ‘expected news’ – a planned earnings announcement, perhaps, or the latest<br />
employment figures – from unexpected events – a CEO departing or a class-action lawsuit.<br />
Whatever the topic or event, MRN can be tapped to trigger a range of investment actions<br />
based on predetermined ‘what-ifs’ developed by the trader or investment manager.<br />
Indeed, unscheduled news can be based around ‘noise’ in the marketplace about a particular<br />
stock or sector. This can be monitored, assigned a positive or negative sentiment,<br />
and used to drive a trading model or algorithm that identifies opportunities based on such<br />
in<strong>for</strong>mation. A trading algorithm could be programmed to assign a higher weighting to<br />
negative sentiment than to positive sentiment, <strong>for</strong> example, based on the trading strategy.<br />
As data managers grapple with the integration of the burgeoning number of MRN services<br />
available into their internal clients’ trading models, their trading operations are looking<br />
beyond these relatively structured data sets to new unstructured sources, such as those<br />
we’ve already discussed.<br />
The pressure is on to be able to automatically process and analyse any kind of in<strong>for</strong>mation,<br />
not just that sourced from professional news organisations or research analysts,<br />
which come conveniently tagged with quantifiers of sentiment, <strong>for</strong> example. And increasingly,<br />
trading algorithms and other models will be required to incorporate unstructured<br />
data – from a quarterly financial statement, to the ‘noise’ generated by the blogosphere.<br />
This has implications.<br />
For one thing, the amount of data that needs to be managed by the application will rise<br />
significantly. For another, the complexity of data will grow, based on three factors: a)<br />
speed of incoming data, b) increasing volumes, both in terms of numbers of messages<br />
and the quantity of data per message, and c) complexity of analysis.<br />
Today’s trading algorithms are relatively simple, with many comparing one instrument to<br />
another (IBM relative to Microsoft, say) or to a basket of stocks, a segment or the market<br />
as a whole. But algorithms that once compared one stock to another will expand to take<br />
into account trends and correlations, based on more – and more-complex – data, much of<br />
it unstructured.<br />
Meeting the <strong>New</strong> <strong>Data</strong> <strong>Management</strong><br />
Challenge<br />
The drivers outlined above make it highly likely that financial institutions will be required to<br />
consume more data going <strong>for</strong>ward and from a broader range of suppliers in a multitude of<br />
different <strong>for</strong>mats.<br />
What are the major considerations <strong>for</strong> a data management strategy when handling this<br />
wide array of potentially unstructured data<br />
Take the handling of text-based intelligence like news, <strong>for</strong> example. Tagging and filtering<br />
on the source (supplier) side is one thing, but in order to rapidly take advantage of news<br />
events within a trading system several key considerations must be taken into account by<br />
the data manager.<br />
Be<strong>for</strong>e an automated trading application can consume news, news has to be defined. Any<br />
application must be supported by a data management infrastructure that can differentiate<br />
between true ‘news’ – an event happening in real time – and in<strong>for</strong>mation contained in a<br />
An industry briefing prepared by A-TEAM GROUP <strong>for</strong> <strong>Sybase</strong>, Inc., an SAP company 9
<strong>Data</strong> <strong>Management</strong> <strong>for</strong> <strong>New</strong> <strong>Trading</strong> <strong>Opportunities</strong><br />
news <strong>for</strong>mat – like a magazine article – that may not be news at all.<br />
Making this distinction, and generally assessing the relevance of real news as it arrives, is<br />
no mean feat. It may require the financial institution to define a kind of hierarchy of event<br />
types that are important to the trading philosophy, execution styles and operational capabilities<br />
in place.<br />
We’ve already identified the challenges of handling official, scheduled news vs. unofficial,<br />
unscheduled or breaking news. While MRN services can help in this respect, there remains<br />
the data management issue of integration with existing internal and external news<br />
sources to create and support the kind of comprehensive and consistent data model<br />
required <strong>for</strong> complex analysis. Integration may extend to incoming real-time market data<br />
services, reference data repositories and internal databases of holdings, clients and risk<br />
parameters.<br />
Drilling down further, data managers may be required to identify the context <strong>for</strong> an incoming<br />
news story. A negative story may in fact be ‘positive’ in the sense that market expectations<br />
were worse. Better-than-expected bad news can spell an opportunity, and allow a<br />
financial institution to differentiate by offering its unique ‘take’ on the situation.<br />
Understanding the context of this opportunity is another data challenge. Is it possible to<br />
understand whether the opportunity is itself a reaction to news or other incoming data<br />
event Or is it a result of a firm’s own analysis, allowing a trader to pre-empt the market<br />
More intelligence can be gleaned from market conventions and rules. For example, US<br />
exchanges routinely report whether an order was initiated as a buy or a sell. This can be<br />
useful to a trading model that, say, is seeking to establish whether a security is trending<br />
up or down in markets where such data is not available.<br />
And let’s not <strong>for</strong>get the language issue. It’s true that many financial in<strong>for</strong>mation services<br />
are available in English. But as practitioners cast their nets more widely, highly targeted<br />
<strong>for</strong>eign language services will become increasingly important. Natural Language Processing<br />
can help. But it’s important not to understate the ‘training’ it requires, and the ongoing<br />
resource required to keep it up to date.<br />
In their 2009 book, The <strong>New</strong> <strong>Data</strong> Imperative, <strong>Sybase</strong>’s Dr. Raj Nathan, Irfan Khan and<br />
Sinan Baskan outline 10 core infrastructure needs that must be fulfilled by financial institutions<br />
seeking to address the broad and varied analytical requirements of their trading<br />
operations:<br />
1. A multithreaded asynchronous playback engine that can precisely replicate the<br />
original timing between market events or accelerate and decelerate the data stream<br />
across multiple instruments simultaneously. <strong>Data</strong> playback should also be able to<br />
be paused/resumed and even reversed to step back in time <strong>for</strong> strategy debugging<br />
purposes.<br />
2. Being configurable to receive inbound order messages, match orders according<br />
to user-defined matching logic and trading rules (trading sessions, order types,<br />
validation, prioritisation logic) and deliver partial executions, fills, cancels, cancel/<br />
replaces or unfilled open orders.<br />
3. Support <strong>for</strong> complex market data messaging structures, such as full order<br />
books, object-based messaging, client-side caching and time-series data.<br />
4.<br />
Ability to aggregate multiple data sources into a centralised book or montage<br />
An industry briefing prepared by A-TEAM GROUP <strong>for</strong> <strong>Sybase</strong>, Inc., an SAP company 10
<strong>Data</strong> <strong>Management</strong> <strong>for</strong> <strong>New</strong> <strong>Trading</strong> <strong>Opportunities</strong><br />
while maintaining the ability to treat each data source differently with respect to its<br />
equality, timeliness and distribution latency harmonics.<br />
5. Scalability across grids/clusters to quickly test computationally intensive, highfrequency,<br />
high-volume trading strategies against massive high-frequency datasets.<br />
6.<br />
Ability to combine playback/simulation across multiple assets simultaneously.<br />
7. Granular control over the playback speed of historical data (i.e., playback at<br />
actual speed vs. multiples of real-time speed).<br />
8. A means of collecting real-time data live, as well as the ability to integrate/import<br />
historical market data (either collected internally or obtained from a third party).<br />
9. Ability to process historical data of different quality and granularity levels, from<br />
fully transparent (complete history of market depth and trades) to very coarse (best<br />
bids and offers or trades only).<br />
10. Capacity to run trading simulations in either live mode with real-time market<br />
data or historical mode with previously stored data.<br />
The list is useful as it helps make the case <strong>for</strong> a shared data management plat<strong>for</strong>m that<br />
can be extended from the trading environment to cover post-trade activities like transaction<br />
cost analysis, per<strong>for</strong>mance attribution and risk analysis. Ultimately, such a plat<strong>for</strong>m<br />
could be used to apply a broad range of data sets – real-time market data, <strong>for</strong>ecast estimates,<br />
fundamental data, transaction data, product data, emails, historical market data,<br />
customer data, index data, reference data and more – to the entire trading lifecycle, from<br />
pre-trade analysis through to back-office clearing and settlement.<br />
Indeed, the <strong>Sybase</strong> authors’ vision suggests: “When a continuous data flow is established<br />
through the organisation from internal and external data sources to a shared data services<br />
plat<strong>for</strong>m and access to data flow along the path is enabled on a request and real-time<br />
basis, all users can have a single view of the state of the markets.”<br />
How <strong>Sybase</strong> Can Help<br />
<strong>Sybase</strong> Complex Event Processing (CEP) enables rapid application development <strong>for</strong><br />
quickly building and deploying new applications that derive insight from streaming event<br />
data, enabling instant response to changing conditions. <strong>Sybase</strong> CEP technology users<br />
report that building event-driven data analysis with <strong>Sybase</strong>’s CEP tools cuts development<br />
time and ef<strong>for</strong>t by anywhere from 67% to 85%.<br />
It represents a third option to the perennial ‘buy vs. build’ debate, by providing the flexibility<br />
and fit of a custom solution at the lower cost and reduced implementation time of<br />
an off-the-shelf solution.<br />
<strong>Sybase</strong> RAP – <strong>Trading</strong> Edition is a unique consolidated analytics plat<strong>for</strong>m designed to<br />
manage high-volume real-time data feeds as well as massive historical and market data<br />
sets. It offers access to huge repositories of leaf-level market data, and can store additional<br />
data vital to capital markets analytics. The result is greater speed and more efficient<br />
data compression, which is combined with an in-memory database <strong>for</strong> the capture of<br />
real-time streaming data.<br />
Designed <strong>for</strong> real-time analysis of streaming data, <strong>Sybase</strong> Aleri Streaming Plat<strong>for</strong>m is a<br />
high-per<strong>for</strong>mance CEP engine that can absorb and process high-volume data streams,<br />
such as market data, in real time, applying custom logic to normalize and analyze the data.<br />
An industry briefing prepared by A-TEAM GROUP <strong>for</strong> <strong>Sybase</strong>, Inc., an SAP company 11
<strong>Data</strong> <strong>Management</strong> <strong>for</strong> <strong>New</strong> <strong>Trading</strong> <strong>Opportunities</strong><br />
About <strong>Sybase</strong><br />
<strong>Sybase</strong>, Inc., an SAP company, offers highly secure,<br />
high-availability software solutions that meet the<br />
trading industry’s needs <strong>for</strong> intensive transaction<br />
processing, advanced analytics and modeling. These<br />
solutions capitalize on cutting-edge technological<br />
advances, such as real-time risk analysis (<strong>Sybase</strong><br />
RAP - The <strong>Trading</strong> Edition) and complex-event<br />
processing (<strong>Sybase</strong> CEP), to allow financial services<br />
firms to gain a competitive edge in capital markets.<br />
Today, <strong>Sybase</strong> customers include a number of the<br />
world’s largest financial institutions. And every day,<br />
tens of millions of Wall Street trades run on <strong>Sybase</strong><br />
technology. For more in<strong>for</strong>mation, visit: http://www.<br />
sybase.com/capitalmarkets<br />
www.a-teamgroup.com<br />
A-Team Group, founded in 2001, provides a range<br />
of global online news, in-depth research reports,<br />
and events focused on the business of financial<br />
in<strong>for</strong>mation technology.<br />
A-Team Group serves its global client base of IT<br />
and data professionals within financial institutions,<br />
technology and in<strong>for</strong>mation suppliers, consultants<br />
and industry utilities with insight into the business<br />
of electronic trading, market data, low latency,<br />
reference data, risk management and the impact of<br />
regulation upon these industry segments.<br />
Our flagship news service is A-Team Insight,<br />
which delivers insight <strong>for</strong> Financial Technology<br />
Executives on the stories, trends and topics that<br />
matter to you, by leveraging our team of industry<br />
experts and our extensive network of contacts.<br />
A-Team Insight is delivered via regular updates on<br />
our website, in a monthly PDF digest <strong>for</strong>mat and with<br />
a quarterly printed magazine A-Team IQ.<br />
Find out if you qualify <strong>for</strong> a complimentary<br />
subscription and sign up <strong>for</strong> a free 30-day trial at:<br />
www.A-TeamGroup.com/complimentary-access.<br />
A-Team Group’s research division provides<br />
industry professionals with focused and in-depth<br />
research offerings to better understand the specific<br />
uses of data and technology in today’s trading and<br />
investment processes across the financial enterprise<br />
from front to back office. These include a series<br />
of topical white papers, survey-based research<br />
reports and focused directories (eg: algorithmic<br />
trading, valuations and alternative trading systems<br />
directories). Many of A-Team’s research publications<br />
are available <strong>for</strong> free at:<br />
www.A-TeamGroup.com/site/research.<br />
A-Team offers custom research solutions,<br />
commissioned by clients seeking answers to specific<br />
questions <strong>for</strong> in-house product development or<br />
marketing, or looking to support their marketing<br />
activities, promote thought leadership and generate<br />
sales leads. Find out how our custom research<br />
solutions can boost your marketing campaigns by<br />
contacting A-Team Group.<br />
A-Team Group’s events division produces a series<br />
of Insight Exchange events annually. These events<br />
combine A-Team’s expertise in financial markets IT<br />
with thought leadership from world-class technology<br />
innovators and practical experience from financial<br />
market practitioners. For a schedule and more<br />
in<strong>for</strong>mation, visit:<br />
www.A-TeamGroup.com/InsightExchange.<br />
A-Team also partners with customers to produce<br />
custom physical and webinar events.<br />
For more in<strong>for</strong>mation about A-Team Group, visit<br />
www.A-TeamGroup.com.<br />
An industry briefing prepared by A-TEAM GROUP <strong>for</strong> <strong>Sybase</strong>, Inc., an SAP company 12