Ainda precisamos de um Data Warehouse

Gartner Business Intelligence & Information Management Summit
23-24 de junho 2015 | São Paulo, Brasil
Do We Still Need a Data Warehouse?
Donald Feinberg
This presentation, including any supporting materials, is owned by Gartner, Inc. and/or its affiliates and is for the sole use of the intended Gartner audience or other intended recipients. This presentation may
contain information that is confidential, proprietary or otherwise legally protected, and it may not be further copied, distributed or publicly displayed without the express written permission of Gartner, Inc. or its affiliates.
© 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Do You Actually Know What a Data Warehouse Is?
• Consolidated, integrated,
subject-oriented, time-variant
data management solution.
• It never was …
– A database.
– A DBMS.
– An appliance.
– Built in.
– … done.
Things are not always what you first see, first experience, first build,
first encounter.
1 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Key Issues
1. What aspects of a data warehouse methodology are still required?
2. How does the concept of a logical data warehouse balance the
needs of agility and governance?
3. What are your architectural options?
2 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Key Issues
1. What aspects of a data warehouse methodology are still required?
2. How does the concept of a logical data warehouse balance the
needs of agility and governance?
3. What are your architectural options?
3 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

What Is a Data Warehouse?
• Architectural Construct:
– Integrated
– Time variant
– Subject orientation
– Analytic orientation
– Service level driven
– Mission-critical
• Physical Implementation:
– Centrally stored data
– Modeled
– Mixed workload
– Optimized
– Servers
– Storage
4 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

The Data Warehouse Was a Bridge to "Near" Information
Shores … Now We Are in Orbit!
Evolution
Craftsmanship
5 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
Adaptation
The data warehouse crossed relatively simple information usage barriers using
binary dichotomy. Go "off world." Embrace more than local topography.

You Still Need All of the Analytics Data Management SLAs
But these SLAs are mostly incompatible!
• Compromise:
– The ".AND."
– Pervasive.
– Persistent.
– Latent.
– Source "write."
– Target "write."
– Static optimization
for all 5.
• Contender:
– The ".OR."
– Diverse.
– Transient.
– Native.
– Source "write."
– Target "read."
– Optimization
focuses on the
dominant.
• Candidate:
– ?
– Unique.
– Ambivalent.
– Contrary.
– Source "read."
– Target "read."
– Optimized by
processing.
Since you must continue to meet the demand of the data warehouse —
you must continue to support the data warehouse.
6 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

You Still Need Best Practices For …
• Source system identification.
• Determining to instantiate
data.
• Optimization by
end-user class.
• Optimization for
data processes.
• Requirements for static
reporting and "canned"
analytics.
• Information life cycle.
• Security management.
• Data governance & quality.
• Data preparation.
Moving data to a central location or giving a central point of access to many data locations
is the new warehouse. Moving data to a central location with less transformation actually
puts that job on the user.
7 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Funding the Need Somewhere Else Doesn't Mean
It's Not a Warehouse
• Appliances are platforms.
• Cloud? IaaS! PaaS! DBaaS!
• Models are customized.
• Source system access differs.
• Software and servers
are "pieces."
• Distributing cost among many in
"self-service" doesn't make it
disappear.
You need a different warehouse that includes the old warehouse —
but, you still cannot just "buy" an LDW.
8 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Key Issues
1. What aspects of a data warehouse methodology are still required?
2. How does the concept of a logical data warehouse balance the
needs of agility and governance?
3. What are your architectural options?
9 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

% of volume contribution
"New" Big data and "Rough" Data Volume Exceeds but
Does Not Eliminate "Curated" Data!
120
100
80
60
40
20
Curated data (traditional)
Rough data (virtual)
New (big) data
0
About 20% of the data warehouse market can be considered leading
implementations. By 2018, logical data warehouses in half of them will have
combinations of all three of these data management approaches.
10 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

% of volume contribution
The Notion of "Fit for Purpose" Has Evolved and Is
Supported in the LDW
100
90
80
70
60
50
40
30
20
10
0
2005 2010 2015 2020
Curated data (traditional)
Rough data (virtual)
New (big) data
The law of
diminishing
returns
Curated data for analytics was differentiating in 1995. In 2015, differentiation is
in rendering data in various states to various user types. Think about the value
of each data type to each user—not volume!
11 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Use-Case Access
Semantics
Traditional Warehouse Practices Are Augmented
by the EDW Evolution!
Can the Architecture
Perform as Required?
Do the Descriptions and
Categories Match?
Taxonomy/Ontology Resolution
SLA Requirements
Auditing and Management
Statistics
/=/~
DQ, MDM, Gov.
Metadata
Locate Data
Audit Data
Repositories
And/Or And/Or
Federation/
Virtualization
Distributed
Process
Optimize Data
Persist, Pervasive,
Latency, Optimized
Transient, Diverse
Usage, Native
Comprehensive,
Undefined
12 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
Schema
"Write to Write"
Schema
"Write to Read"
Schema
"Read to Read"

Key Issues
1. What aspects of a data warehouse methodology are still required?
2. How does the concept of a logical data warehouse balance the
needs of agility and governance?
3. What are your architectural options?
13 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Use-Case Access
Semantics
Semantics Form a Bridge to Data Science &
New Practices
Veracity
COMPROMISE
SLA Requirements
Repositories
80% of Analytics
Infrastructure Is the
Same as It Ever Was.
Taxonomy/Ontology Resolution
And/Or
/=/~
Federation/
Virtualization
Variability
Auditing and Management
Statistics
And/Or
CONTENDER
10% Need Flexibility and
Short Duration.
Distributed
Process
DQ, MDM, Gov.
?
Metadata
High Value
CANDIDATE
5% Converting
Structure on Read
and Integrated.
SCIENCE SANDBOX
5% Constant
Experimentation and
Exploration:
• 80/10/5 Rule Again!
A Semantic Tier Is How You Move From a Binary Design That Forces Unnatural Dichotomy and Creates
a More Flexible and Responsive Architecture.
14 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Semantic Interface
General Ratio in
Population
Single-Process
SME
Multiprocess
SME
Information/Data
Architects
Statistical Science
Frequent Simple
Support
Frequent
Advanced Support
Infrequent
Advanced Support
Even the Most Advanced Users Use That Curated
Repository — The Warehouse
Repository
Example
User Class
Typical Characteristics
Virtual
Apprentice
Reports, dashboards. Level 1 (L1)
support, maybe L2.
1,000
Journeyman
Create new reports. Needs
technical assistance. L2 and L3.
90
Distributed
Process
Master
Guild Master
Reliably utilizes knowledge of
systems and data to explore data
assets.
Modeling theory, graph theory,
mathematics, program languages. 1
5
In general, there is a disconnect. We tend to build toward either the "ease of use"
user or the "advanced analysis" user — and we build twice or more times than
that. The logical data warehouse says "build once, access many."
15 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Integration
Semantic Interface
Architecture Issue No. 1: Analysts Need a Unified Data
Delivery & Access Semantic
Apprentice
Reports, Dashboards. L1
Support, Maybe L2.
1,000
Compromise
Journeyman
Create New Reports. Needs
Technical Assist. L2 & L3.
90
Embedded/
Smart
(Stores)
Contender
Master
Reliably Use Knowledge of
Systems and Data to
Explore Assets.
5
Analytics
(Access)
Guild Master
Modeling Theory, Graph
Theory, Mathematics,
Program Languages.
1
Candidate
(Processes)
Logical Data Warehouse
16 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Integration
Semantic Interface
Architecture Issue No. 2: Need to Convert Discovery
Into Production — The Organization!
LDW
BICC
Casual
Analyst
Miner
Science
Reports,
Dashboards. L1
Support, Maybe L2.
Create New Reports.
Needs tech. Assist,
L2 & L3.
Ops. Process,
Systems Analyst,
Data Architect,
Reliable Tech.
Modeling Theory,
Graph Theory,
Mathematics,
Program Languages.
1,000
90
5
1
A "Bridge"
Embedded/
Smart
Analytics
Data Science Laboratory
Compromise
(Stores)
Contender
(Access)
Candidate
(Processes)
Information Architect, Systems Architect, Application Architect
17 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.
Data Services
Administrators
Data Modeling
Information
Administrators

Option No. 1: BI Platform as Semantic Tier
• Design Principles:
– Data scientist guild master
specifies processes and
identifies objects
– Business analyst journeyman
utilizes and reuses objects
– Architects work with BA
Journeyman to "productionize"
science
– Data analyst master is on
special projects
• Benefits:
– Familiar to users
– Encourages discipline on
business and data analysts
– Free's up commitments from
the data scientist
• Risks:
– Limits sharing data to BI
export process
– Challenging to add new BI
and data discovery tools —
must reinvent the model
"wheel" each time over
all 3 SLAs
19 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Option No. 2: DMSA (DBMS) as Semantic Tier
• Design Principles:
– Business analyst journeyman
assist data analyst master
– DA master works with
Architects and DBA team for
optimization
– Data scientist guild master
presents reinforcing and
dissenting analytics
– Architect team works with DA
and BA to prioritize and deploy
• Benefits:
– Familiar to DBAs and data
architects
– Leverages discipline of
analyst/DBA interaction
– Encourages workflow to
include data scientist with IT
and data analysts
• Risks:
– Most DBMS external calls
cannot extend optimization
– Channels data access through
a single "choke" point — but
can be managed
20 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Option No. 3: Virtualization as Semantic Tier
• Design Principles:
– Data scientist and data analyst
masters identify analytic data
patterns and propose.
– DA and business analyst
Journeyman determine top
contenders.
– Architecture team works with
DA on metrics to determine
promotion to compromise and
contender.
• Benefits:
– Encourages additional
technology for SLAs
– Creates semantic tier
accessible by all technologies
– Permits rapid prototyping and
re-modeling of data AND
quick-change between SLAs
• Risks:
– Heavily reliant upon internal
optimization and its limits
– Assumes knowledge of data
access and processing issues
21 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Recommendations
Keep your existing data warehouse as one engine beneath a
unified access tier.
Determine your best strategy for deploying a unified access tier
based on risks and benefits.
Use your ratio of analyst user types to determine a center of
gravity for using high value.
Gauge your rate of shift between "curated," "rough" and "new"
information asset types to develop priorities and timelines.
22 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.

Recommended Gartner Research
Focus on the 'Three Vs' of Big Data Analytics: Variability, Veracity
and Value
Alan D. Duncan (G00270472)
Making Big Data Normal Begins With Self-Classifying and
Self-Disciplined Users
Mark A. Beyer and Others (G00271691)
Avoid a Big Data Warehouse Mistake by Evolving to the Logical
Data Warehouse Now
Mark A. Beyer (G00252003)
Business Drivers of Technology Decisions for Healthcare Providers, 2015
Vi Shaffer and Others (G00272361)
For more information, stop by Gartner Research Zone.
23 © 2015 Gartner, Inc. and/or its affiliates. All rights reserved.