19CrAYS

“She's tidied up and I can't find anything!
all my tubes and wires
And careful notes
And antiquated notions
but - it's poetry in motion
She blinded me with science!"
She Blinded Me With Science - Thomas Dolby

Eric Kavanagh is the host of DM Radio and Information
Management's Webcasts. He is a veteran journalist and consultant
with 20 years of experience in print, broadcast and online media. For
the past 10 years, he has focused on enterprise technology and
information management. Recently, he helped spur adoption of the
Federal Funding Accountability and Transparency Act, which led to the
deployment of USASpending.gov, a Web-based portal that provides
detailed financial information about all federal contracts and grants.
Prior to joining the magazine, Eric was web editor for TDWI. He also
founded and runs Mobius Media, a strategic communications firm
based in New Orleans. You can email him at
dmradio@sourcemedia.com

Robin Bloor, President and Chief Analyst of The Bloor Group, has over 25
years experience in software development and IT analysis and consulting.
Robin is an influential and respected researcher and commentator on many
corporate IT issues and strategies. His recent research has focused on
virtualization, SOA, business intelligence, workload automation,
communications enabled business processes, and the evolution of software
tools.
In addition to Robin’s deep technical expertise, he has covered these topic
areas with a focus on end-use customer requirements. He is in great
demand as a presenter at conferences, user groups and seminars addressing
audiences across the world. Robin is a co-author of Service Oriented
Architecture for Dummies, published by Wiley in 2007 and Cloud Computing
for Dummies, published in 2009. He is also the author of The Electronic
Bazaar published by Brearly in 2000.
Robin is the founder of Bloor Research, an IT analyst company based in the
U.K. In 2003, he was awarded an honorary PhD in Computer Science by
Wolverhampton University in the United Kingdom, in recognition of
“Services to the IT Industry.”

Dr. Geoffrey Malofsky Ph.D,
Geoffrey Malafsky earned a PhD in Nanotechnology from The Pennsylvania State
University. He was a research scientist at the Naval Research Laboratory before
becoming a technology consultant in advanced system capabilities for numerous
Government agencies and corporate clients. He has over thirty years of experience
and is an expert in multiple fields including Nanotechnology, Knowledge Discovery
and Dissemination, and information engineering. He founded and operated the
technology consulting company TECHi2 prior to founding Phasic Systems Inc where
he is the CEO. He has authored numerous articles on semantic technology, agile
governance and other related topics and is regularly solicited to speak at industry
conferences, industry radio programs and is widely regarded as a leading
technologist in the semantic technology field

J. Kevin Moran
Vice Admiral J. Kevin Moran was the COO at The Investor Relations Group following a
highly distinguished thirty-two years in the U.S. Navy. He graduated from the United
States Naval Academy with a Bachelor of Science Degree in Oceanography/Physics
and went on to flight training where he was designated a Naval Aviator. He was
Deputy Chief of Naval Personnel where he led the effort to modernize and integrate
the Manpower, Personnel, Training, and Education organization, systems, and data
environments. He is a graduate of the Naval War College in Newport, Rhode Island,
where he earned a Master of Arts degree in International Relations and Strategic
Studies. He is also a graduate of the Advanced Management Program at the Harvard
School of Business. His personal awards include the Navy Distinguished Service
Medal, the Legion of Merit (gold star in lieu of fourth award), the Defense Meritorious
Service Medal, the Meritorious Service Medal (gold star in lieu of third award), the
Navy Commendation Medal (gold star in lieu of second award). Additionally, while he
was the Commander of the Naval Education and Training Command, he was awarded
the Elliot Masie Foundation Pioneer Award for visionary training and education
solutions; US News and World Report selected his command as one of the best places
to work in America; and he was awarded the Ellis Island Medal of Honor as a
distinguished first generation American.

Dr. Anand S. Rao is a Principal in PwC's Advisory practice, focused on the financial
services industry. Anand joined PwC as a direct-admit partner from Diamond
Management Technology Consultants in 2010. He has been with PwC/Diamond
since 2001 helping senior executives structure, solve and manage critical issues
facing their organizations. With over 21 years of industry and consulting
experience, Anand has worked extensively on both business and technology issues
across a wide range of industry sectors including, financial services (insurance,
retail, corporate, and investment banking, payments), telecommunications (mobile
and fixed-line), healthcare (payer), aerospace, retail, and resource sectors.
Anand's work has included behavioral economics, simulation modeling, global
growth strategies, marketing, sales, and distribution strategies, online, mobile,
social media strategies, segmentation and proposition development, customer
value management, multi-channel integration, risk management and compliance
(specifically, Basel II), large scale program mobilization and program management.
He has worked and lived in four countries spanning four continents and has
consulted to clients in fifteen countries across six continents.

Phasic Systems Inc.
www.phasicsystemsinc.com
703-945-1378

“Many CIOs believe data is inexpensive because storage has become
inexpensive. But data is inherently messy – it can be wrong, it can be
duplicative, and it can be irrelevant – which means it requires handling,
which is where the real expenses come in. „The cost of more data is the
application and the computing power and the processes to reconcile all
these things‟,”
"While there are a myriad of analytical tools that can be leveraged, a
recent study indicated that more than 70% of CMOs feel they are
underprepared to manage the explosion of data and „lack true insight.‟ “
1. Wall Street Journal, CIO‘s Big Problem with Big Data, 2012-08-02
2. Forbes, The CEO/CMO Dilemma: So Much Data, So Little Impact, 2012-07-18
9

Based on non-IT, well-established, technical field and methods
Has potential to solve intractable, complex challenges but also
vulnerable to same forces that blocked prior ‘movements’
Science is structured, highly analytical with a culture of
questioning assumptions and requiring objective proof
Science is based on big data and handles it every day in every
Science knows how to manage distributed sources, differing
semantics, quality problems, linking business needs to
operational data
But, this is a culture shift from traditional data management
New ideas: raw data, calibration transforms, uncertainty,
continuous assessment and improvement, willingness to admit
errors and immediately fix them 10

Example from DARPA Evidence
Extraction & Link Discovery
Today’s Situation: ~10k
messages/day from multiple
sources read by multiple
analysts and analyzed in
multiple manual non-integrated
tools
Similar to Social Network
Analysis
11

“We spent years and many millions of dollars using the best technology and
people and always failed to reconcile our data across major functional areas.
This seriously impeded our efforts to improve Navy training, personnel
placement, and planning to meet requirements for greater efficiency and
adaptability. Only the rationalization process solved this challenge and
amazingly did so with broad consensus across groups.” VADM (Ret) K. Moran
14

Challenge: Complicated environment with conflicting data
values, standards, business uses cases, and lack of
documentation. Data owned by 4 major organization, in multiple
Warehouses and data stores, redundant non-reconciled sets of
data
Requirement: Integrated, common, accurate data to enable new
Integrated workforce planning, training, management application
(“Sailor of the Future”) for 1 million people
Prior Activities: 10+ years of system integration, data warehouse,
data governance efforts no improvement, poor coordination
across organizations and systems
15

The US Navy struggled for years with creating a common,
trustworthy, governance directed warehouse of Human
Resources data.
The data was spread out in five major warehouses and dozens
of small databases supporting many applications.
The applications spanned decades of business logic and
technology, including old mainframe COBOL programs through
modern Services Oriented Architecture (SOA) web service
based tools.
Many projects and years were spent without success despite
extensive analysis, integrated project teams, and highly
experienced personnel.
16

This was the fundamental problem preventing all
previous efforts, namely the lack of business context
and the different perspectives among user groups and
applications on what each data element really meant.
This problem could not be overcome using traditional
governance, data modeling, and warehousing
techniques because they require knowledge of all
specific requirements, definitions, and use cases before
data modeling and engineering.
17

US Navy: VADM (Ret) Moran, executive
US Navy: Tim Traverso, former Technical Director
OPNAV, N6
US Navy: Jerry Best, project lead and SME
Contractor: Dr. Geoffrey Malafsky, chief technical
designer and development leader
19

With one million records covering over fifty years
of data, the data values supported many older
business processes and policies as well as new HR
standards.
This created the situation where data conflicted
within each warehouse and across warehouses
even when the data was supposedly the same
job title, salary, address
A significant number of people had multiple
records listed as currently active with wildly
different values.
21

The data encompassed all HR data including:
personnel, training, and manpower analysis and
planning.
It included critical job assignments, career path, job
performance, qualifications, field deployments, and
alignment to enterprise business capabilities.
It spanned decades of changing data models,
definitions, value spaces, and Navy strategies.
22

Yet, there were problems with the most basic
data fields, which for the Navy, include things
like
billet (effectively a job but also includes other
characteristics),
rank (similar to seniority but with formal rules that change
over time),
rating (similar to vocational ability but also with changing
rules),
and even the primary identifier of a person the Social
Security Number (SSN).
23

There were many types of conflicts.
The Personnel warehouse contained Training data because the
Personnel group did not trust the data from what was
supposed to the authoritative warehouse managed by the
Training group.
Similarly, the Training warehouse contained personnel data
because they did not trust the quality of the data managed by
the Personnel group.
These data sets did not agree preventing applications from
creating unified functions, and causing erroneous reports and
analytics that had to be manually corrected on a recurring
basis.
24

The solution was to use Phasic Systems Inc.’s Rapid
Rationalization Process.
The first major advance was to recognize that there
are multiple valid definitions of the key business
concepts (e.g. job, billet, address).
The method accepts and accommodates all variations
by placing them into the predefined semantic
framework, and identifying exactly how they are used
and why.
In this manner, organizational disagreements ceased,
key knowledge was captured, and common data
standards were defined in hours and days instead of
26
months and years.

Data Rationalization is the process of building and managing a
continuously adaptive data environment that fuels current and
future business needs for decision making and system
operations
It ensures data (i.e. not just metadata) is as accurate,
meaningful, and useful as possible while continuously adjusting
to improve and add capability
It provides collaborative management of data assets, the
designs governing who, why, and how of data , and the where,
when, how of data use in operational systems
It solves the great challenge of mapping all source values to
each target along the entire complex paths of enterprise data
use
Consolidated values when possible with continuous improvement
Simplified and adaptive mapping with Corporate NoSQL 28

Tailored for real environments with complex or
undocumented business/technical activities, models, rules
Use standards-based simple organization-process-technology
model to capture key corporate knowledge from all sources
at any stage of design & operation
List-based entry of intuitive concepts
People according to job and experience
Documents of any kind extracted and correlated
Reverse engineer systems and databases
Produce design and ready-to-implement results
30-60 days
Correlated, authoritative, consensus approved system
models, data models, codes, glossaries, rules
29

Design Rationalization Issues
• Multiple data models
• Conflicting definitions
• Similar, supposedly similar, operationally
distinct values
• Unknown business logic
• Multiple ETL mappings
Design Rationalization
• Consolidated, adaptive data models
• Standardized definitions
• Synchronized distinct operational values
• Managed business logic
• Coordinated ETL mappings
System Rationalization Issues
• Multiple database systems
• Conflicting formats
• Redundant storage
• Unsynchronized values
• Multiple integration points
• System performance
System Rationalization
• Consolidated, adaptive systems
• Common, interoperable formats
• Common storage
• Synchronized interfaces
• Coordinated integration
• Greater system performance
30

Corporate NoSQL
31

Suffix in source A, prefix in B, neither in C for same (part
number, title, …)?
Conflict syntactically (simplest case) and semantically (most
difficult)
Other tools & methods never solve this because they deal with
the obstacles independently or not at all
Data values are out-of-synch with metadata, data models, BI warehouse
Different Meanings (Legal and Business Activities)
NKY HomeSeekers Texas
32

The Ψ–KORS System Model
35

Key terms used for decades but debated across organizations and used
differently in data stores. PSI clarified, aligned to standards, and defined
distinct versions as well as common enterprise version
Rating – code of occupational specialty
Rate – rating with pay grade code appended
Billet – roughly associated with a job but used for projections in
manpower, salary in personnel, and classroom seats in training
Task – PSI identified and defined 3 unique varieties: 1)operational
mission, 2) job activities, 3) training curriculum segments
Key data entities stored with different and undocumented business
logic leading to conflicting values and organizational disagreements
Navy Enlisted Classification (NEC) – important value used in most HR
processes and systems. Ad hoc logic adopted over time as
authoritative but in fact not aligned to regulations and changed
36
frequently

Current data is disjointed and of low quality
Variable use and meaning among systems even for “same” data
elements
Undocumented definitions and data mgmt processes
Errors in data systems
Disagreement among data systems
Lack of existing descriptions for key readiness use cases
Legacy data systems have failed to overcome these
problems despite several years of new
marts/houses/brokers/IPTs/applications

Multiple Rate/Rating entries that conflict per person.
5 entries with 4 ending on the same date (1992-06-30); 2/5 have
start dates after they their end dates (e.g. 1992-07-01, 1992-10-
16); 2 start and end on the same days but have different rates
Some data systems confuse Rate and Rating
NEC is critical value shared across systems but logic varies
Local manager, “…When we apply an NEC, we look up the priority on a
table, and shuffle the 5 NECs around as needed.”
Manpower: A_RTABBR is empty for 11919 records.
R_RTABBR has values but it uses non-standard Rate codes
that are meant to show a range of paygrades

Complicated Mixture of Commercial, Custom, Legacy, Services Applications, Data Stores
39
Copyright Phasic Systems Inc 2013

Navy HPC
HR-XML
40

Bridge Organizations, Processes, Technologies to Data Concepts
41

Logical Models derive directly from conceptual and use business terms
43

The new data was widely acknowledged as the only
completely integrated, accurate data by all levels of
the organization.
The approach was so successful that the governance
board adopted the name of the new enterprise data
model (Position-Resume) as its own name.
Meets to this day on a monthly basis
Usually garners high consensus without friction
46

Different Meanings (Legal and Business Activities)
NKY HomeSeekers Texas
Example solution:
1. Create table – title aligned to business = Garage
2. Create vocabulary for distinct use cases system, value analysis, business use =
(spaces, spaces.description, spaces.national, spaces.state, listingservice, ….)
3. Define ETL logic
4. Merge in warehouse and process in virtualization layer
5. Change as needed
47

Costs
Business Alignment: Goal, Capability, Architecture
Data Assets: Systems, Owners, Use
48