REX

More documents

Recommendations

Info

Retours d’expériences Big Data en entreprise MAPR - COMSCORE COMSCORE RELIABLY PROCESSES OVER 1.7 TRILLION INTERNET & MOBILE EVENTS EVERY MONTH ON MAPR THE BUSINESS comScore is a global leader in digital media analytics and the preferred source of digital marketing intelligence. comScore provides syndicated and custom solutions in online audience measurement, e-commerce, advertising, search, video and mobile. Advertising agencies, publishers, marketers and financial analysts rely on comScore for the industry-leading solutions needed to craft successful digital, marketing, sales, product development and trading strategies. c omScore ingests over 20 terabytes of new data on a daily basis. In order to keep up with this data, comScore uses Hadoop to process over 1.7 trillion Internet and mobile events every month. The Hadoop jobs are run every hour, day, week, month and quarter, and once they’re done, data is nor- malized against the comScore URL data dictionary and then batch loaded into a relational database for analysis and reporting. comScore clients and analysts generate reports from this data; these reports enable comScore clients to gain behavioral insights into their mobile and online customer base. HADOOP REQUIREMENTS The comScore engineering team processes a wide variety of Hadoop work- loads and requires a Hadoop distribution that excels across multiple areas: Performance : As comScore continues to expand, the Hadoop cluster needs to maintain performance integrity, deliver insights faster, and also needs to produce more with less to minimize costs. Availability : comScore needs a Hadoop platform that provides data protection and high availability as the cluster grows in size. Scalability : comScore’s Hadoop cluster has grown to process over 1.7 trillion events a month from across the world, in the past comScore has seen increases of over 100 billion events on a month over month basis. Consequently, comScore needs a Hadoop platform that will enable them to maintain performance, ease of use and business continuity as they continue to scale. Ease of Use : comScore needs things to just work, and operating the cluster at scale needs to be easy and intuitive. BENEFITS MapR has been in continuous use at comScore for over two years. MapR has demonstrated superior performance, availability, scalability, ease of use, and significant cost savings over other distributions. Performance : Across various benchmarks, MapR executes jobs 3 - 5 times faster when compared to other Hadoop distri-butions and requires substantially less hardware than other distributions. Availability : MapR protects against cluster failures and data loss with its distributed NameNode and JobTracker HA. Rolling upgrades are also now possible with MapR. Scalability With architectural changes made possible by it’s no NameNode archi- tecture, MapR creates more files faster, processes more data faster, and produces better streaming and random I/O results than other distributions. comScore now runs more than 20,000 jobs each day on its production MapR cluster. Ease of Use : comScore’s Vice President of Engineering, Will Duckworth said, “With MapR, things that should just work, just work.” This means there is a lot less for comScore to manage with MapR. One of the advantages that Duckworth cites is that everything is a data node. This configuration results in much better hardware utilization from his perspective. With MapR, it is easy to install, manage, and get data in and out of the cluster. Speed : comScore is also able to use the MapR advanced capabilities to enforce parallel data allocation patterns. This enables key analyses to be performed using map-side merge-joins that have guaranteed data locality, resulting in a 10x increase in computation speed. “The specific features of MapR, such as volumes, mirroring and snapshots, have allowed us to iterate much faster,” said Michael Brown, CTO of comScore. ABOUT MAPR MapR delivers on the promise of Hadoop with a proven, enterprise-grade platform that supports a broad set of mission-critical and real-time production uses. MapR brings unprecedented dependability, ease-of-use and worldrecord speed to Hadoop, NoSQL, database and streaming applications in one unified big data platform. MapR is used by more than 500 customers across financial services, retail, media, healthcare, manufacturing, telecommunications and government organizations as well as by leading Fortune 100 and Web 2.0 companies. Amazon, Cisco, Google and HP are part of the broad MapR partner ecosystem. Investors include Lightspeed Venture Partners, Mayfield Fund, NEA, and Redpoint Ventures. MapR is based in San Jose, CA. Connect with MapR on Facebook, LinkedIn, and Twitter. Document réalisé par la Société Corp Events - Janvier 2015 44
Retours d’expériences Big Data en entreprise MARKLOGIC - DE GRUYTER 21ST CENTURY PUBLISHING EUROPEAN PUBLISHER TAKES UP TO TEN SYSTEMS OFFLINE WITH A SINGLE MARKLOGIC BUILT CUSTOMER-FACING APPLICATION COMPANY OVERVIEW Academic publishing is a long-standing and venerable industry—and De Gruyter is among the most established. Since 1749, this European publisher has been producing and distributing English and German content, focusing primarily on scientific, technical, legal, humanities, and social sciences. With an output of approximately 1,400 books per year—in addition to journals, reference materials, and databases—De Gruyter’s compiled catalog exceeds 65,000 titles. They also offer a program called e-dition, which is essentially e-books on demand. Unlike other industries, De Gruyter’s customers are not usually the end users: they sell to university libraries and other academic institutions, and it’s the scientists and educators at those universities who actually use the materials. CHALLENGES For many years, scientific publishing was focused on printed materials, but the last decade has seen an increasing demand for digital access. This industry-wide shift has forced De Gruyter to reassess—and redefine—their business. “With this change in content output, we’re not really a traditional publisher anymore,” Christian Kohl, Director of Information & Publishing Technology, explains. “We’re becoming more of a service provider, and also a hosting platform.” And customers’ needs are changing—both in terms of format and timeframe. As Kohl points out, they want more granular access to content, and they want it faster. “Purchasing is shifting from ‘just in case’ to ‘just in time’. We can’t afford to have a production time of several years. It needs to be quick, custom, and flexible.” Since 2006, De Gruyter has invested in a wide range of technologies and support staff. With so many different types of content and business models, they’ve relied on a variety of legacy databases, websites, and online platforms. Not only was this cumbersome and expensive to maintain, the complicated infrastructure prevented them from moving forward with new innovations. The writing was on the wall: in 2009, De Gruyter realized it was time to simplify. “We needed a new, consolidated platform that would bring increased flexibility, lower costs, and more automation,” Kohl says. THE SOLUTION Kohl and his team began searching for a solution that would support all of their various data types and adapt to constantly changing structures. Excellent searching and browsing capabilities were also high on the list. And with their limited budget, they needed a turnkey solution that wouldn’t require third-party technologies. In November of 2010, De Gruyter chose MarkLogic to consolidate up to ten of their legacy systems into one customer-facing application. The new platform would combine all of the existing systems on top of a MarkLogic database and search engine. Using an agile development approach, they were able to implement new iterations for key features. Just a little over a year later, the new platform was launched. WHY MARKLOGIC? Quicker Response, Higher Revenue: With previous systems, it could take up to six months to add new features or changes. With Marklogic ever-increasing requests can be met in a matter of days—or even hours. This has also enabled the company to increase acquisitions and partnerships with publishing partners. Reduced time to market: Database products and reference works used to take a year to develop and deploy. Now, it’s a matter of days or weeks. Seamless Integration & Automation: De Gruyter was able to set up XML feeds to send all data from their ERP system - which houses metadata, catalogs, pricing, and other information—directly into MarkLogic. The new platform also hooks into the company’s CMS and MAM systems, streamlining the flow of content. Streamlined acquisition process: Historically, acquiring more providers and content was a very painful process, particularly for the IT department. With the new platform, De Gruyter can ramp up data volume without impacting performance. Reduced Costs: With MarkLogic’s more simplistic structure, there is no longer a need to map between the SQL database and the application layer—which translates to lower maintenance costs. And by eliminating much of its hardware requirements, De Gruyter has cut its hosting costs in half. Out-of-the-box Search: With their limited budget, De Gruyter needed a solution that wouldn’t require a third-party tool—and MarkLogic delivered. “There’s no need to integrate with the SQL server,” Kohl says. “MarkLogic’s search is already powerful enough.” Document réalisé par la Société Corp Events - Janvier 2015 45
Page 1 and 2: REX Retours d’expériences Big Da
Page 3 and 4: Retours d’expériences Big Data e
Page 43: Retours d’expériences Big Data e
Page 95 and 96:
Retours d’expériences Big Data e
Page 97 and 98:
Page 99 and 100:
Page 101 and 102:
Page 103 and 104:
Page 105 and 106:
Page 107 and 108:
Page 109 and 110:
Page 111 and 112:
Page 113 and 114:
Page 115 and 116:
Page 117 and 118:
Page 119 and 120:
Page 121 and 122:
Page 123 and 124:
Page 125 and 126:
Page 127 and 128:
Page 129 and 130:
Page 131 and 132:
Page 133:
show all

REX

You also want an ePaper? Increase the reach of your titles

Delete template?

Save as template?