Data Masking - Software AG

Data Masking for Adabas
Becky Albin
Chief IT Architect

Reporting, BI, Dashboards
JDBC ODBC
SQL
Adabas SQL Gateway
Adabas SOA Gateway
Other DBMSs
REST
SOAP
Data Access
Data Virtualization
Data Replication
Data Synchronization
Disaster Recovery
Event Replicator
for Adabas
Production
DB
Non-Production
Reporting DB
Real-time Dashboards, BI Clients
webMethods
EntireX, MQ Series
Data Warehouse
BI
RDBMS
Reporting DB
Disaster Recovery
Oracle
SQL Server
DB2
mySQL
Sybase
Teradata

Adabas Product Family
Administration
Adabas Manager
(LUW)
Adabas Online
Services (MF)
Adabas Transaction
Manager
Adabas Statistics
Facility
Adabas Utilities
Security
Adabas SAF Security
Adabas Security
Encryption for
Entire Net-Work
Monitoring
Adabas Review
Optimize for
Infrastructure
Windows
Client
Data Access
Adabas SQL Gateway
Entire Net-Work
Data Integration
Adabas VSAM & DL1
Bridges
Event Replicator for
Adabas
webMethods EntireX
webMethods
Integration Server
User Interfaces
Rich Internet
Applications
Web
Browser
VSAM
DL/1
Terminal
Linux, UNIX, Windows, Mainframe
Data Distribution
Applications
3GL
Services SQL
RDBMs
24x7 Availability
Cluster Services
Parallel Services
Optimization
Adabas Caching
Facility
Adabas Fastpath
Adabas Vista
Data Protection
Adabas Delta Save
Data Archiving
Data Masking

Data Masking - What the Analysts say
Forrester Group (Noel Yuhanna) states:
“All enterprises dealing with private data in test environments should
mask or generate test data to comply with regulations such as PCI,
HIPAA, SOX and European Union (EU)”
80% of all threats come from inside and 65% are undetected
Accenture and Information Week
Gartner
Security breaches are increasingly coming from inside an organization
70% of all security incidents come from insiders
Ernst & Young
An insider attack against a large company causes an average of $2.7
million US in damages, whereas the average outside attack costs only
$57,000

Data Masking – Why would you use it?
• Improve application quality - artificially generated test data is
usually insufficient
• Secure sensitive data in
Development environments
Test centers
Offshore activities
• Provide a real business data training environment without publishing
sensitive data
• Compliance with legal regulations such as HIPAA, SOX and others

Data Masking - Value Proposition
- Ability to consistently create reduced and secured test data
- Rapid masking of production data from across the enterprise to, deliver
“de-identified” data for testing
- Provides a repeatable and automated solution to reduce the resources
needed to create test data
- Easily create high quality training data with a low cost investment
- Facilitates an essential and safe training environment for end-users,
when using live production data for training
- Reduce risk of legal exposure for compromised data

Data Masking – Current Status
- General Availability end of 1 st quarter 2012
- Production shops
- Trial copies can be obtained; contact your Software AG Account
Representative
- Supported Source Databases:
Adabas
Oracle
DB2 (UDB and z/OS)
Microsoft SQL Server
MySQL
Sybase
Ingres
SQLAnywhere
Informix
Cache
VSAM
Flat Files

Data Masking – Hype or Requirement?
•Challenge
High quality test data is required for:
Improved application quality
Test and training environments
Most organizations use home-grown scrambling methods
or even Production data
Legal regulations do not allow use of production data
Scrambling methods do not always consider semantics
Applications do not always function using such data
Preparation is a time consuming and inconsistent manual process
Cross references are not often considered/maintained

Data Masking – Hype or Requirement?
•Business aspect
Business needs high quality applications
Online shops are open for 24 hours a day
The competition is one click away
Pressure to reduce cost
Creating artificial test data is expensive
Manual process
Each project team does more or less the same
Not all use cases can be built
References across tables are difficult to handle
Difficult to create the same values every time
Creating test data can delay projects which affects business negatively
Select
production
data
Copy
production
data
Copied
production
data
Mask data
According
to rules
Masked
production
data
Software ‘package’ needs to fulfill all requirements and have an early ROI
Define
rules

Data Masking – Software that fulfills the requirements
•Criteria for a solution
Ease-of-use
Almost no training needed on the software
Easy to exchange obfuscation rules with
non-IT staff
First results needed quickly (

Data Masking – Software that fulfills the Requirements
•Criteria for a solution
Data source coverage
One tool, not one per database type
Adabas, all market relevant RDBMS and flat
files need to be supported
Platform coverage
Mainframe
Distributed environments (LUW)
Define
rules
Select
production
data
Copy
production
data
Copied
production
data
Mask data
According
to rules
Masked
production
data

Data Masking – Software that fulfills the Requirements
•Criteria for a solution
A rich set of rules need to be available
Replacement
Custom functions/Seed tables
Hashing
Translation
Substitution
Multi-table columns
ZIP code
Credit cards number manipulation
Social security number manipulation
Random numeric/text
Etc.
Define
rules
Select
production
data
Copy
production
data
Copied
production
data
Mask data
According
to rules
Masked
production
data

Data Masking – Data Masking for Adabas
•A solution that fulfills the criteria
Supports the requested rules
Extended features are available
Cross reference masking beyond Referential
Integrity
Reference data can be used
Using “where” clauses
Ease to learn and run
Use a sophisticated user interface to define
rules and run-time option
Non-IT professionals understand rules easily
Run the masking process as a background task
Using a simulation before changing data in a database
Define
rules
Select
production
data
Copy
production
data
Copied
production
data
Mask data
According
to rules
Masked
production
data

Data Masking – Data Masking for Adabas
•A successful approach requires knowledge of your data
Which data is sensitive and need to be masked?
Which columns contain what?
Which relationship consists between data,
maybe across tables?
Are there invalid data in your data sources?
What is the goal?
Test a new part of an application
Achieve legal compliance
Ready to start?
Define
rules
Select
production
data
Copy
production
data
Copied
production
data
Mask data
According
to rules
Masked
production
data

Close a Gap
Provide what’s required – Hide what’s necessary
Provide high quality test data
Test
Application
Enhance
Production
Application
Masked Copy of Production
Use
Adabas Tools
to create
Production
Data
Masking
Rules
Run-time parameter
Meta-data
Data
Mapping
Protect sensitive production data

Data Masking for Adabas Architecture
Production
Application
Adabas
Nucleus
Production
Adabas Tools
to create
Test
Application
Adabas
Nucleus
Copy Masked
Production Production
Adabas
SQL
Gateway
Data Server
Adabas
SQL
Gateway
SQL Engine
Meta-data
Repository
Masking
Engine
Rules
Run-time Options
Mainframe / Distributed Environments Distributed Environments
Mapping
Tool

Prepare your Database Environment
• Create a copy of the ‘production’ Adabas database
• Define the file/table “GTSRC_XREF” for cross reference masking
Masking data consistently across different tables
• Prepare an Adabas SQL Gateway meta-data repository (CDD)

Getting familiar with the SDM Environment
• Structure after installation
Windows/Linux/Unix
Main directory contains
The software
“connect” file
A number of home-grown test cases
Sub-directories
Audits result file if defined
Backups backup of rule files
DDM Natural DDMs
Errorlogs
Logs run logs
Seedtables contains a number of reference data

Getting familiar with the SDM Environment
Start Mapping Process
connectAdabas
GTMAPPER is the tool to define masking rules and
run-time options
Start the Mapper, select the appropriate connect
file and “Connect”
“connect” parameter file
The connectAdabas file is used to
connect to the Adabas SQL Gateway
User name, password, default schema
are defined in the Adabas SQL Gateway
meta-data repository (CDD)
Host refers to the JDBC definition
made through the “DSNRegistry” tool

Simple Data Masking – Getting Started
Connect to the Meta data
repository of the Adabas SQL
Gateway
Open rules file if available
Define rules
Define run-time options
Save rules and options in files
Run the masking process

Getting familiar with the SDM Environment
Define Rules - Main Functions
Select a Table
Select a Column
Select a Rule
Define Values
or
Open an existing File

Getting familiar with the SDM Environment
Define Run-time Options
Define Audit Option
Define Reference File
Cross Connection File
Define Reference
Table
Specify update mode
or
Open an existing File

Getting familiar with the SDM Environment
Define Rules - Main Functions
Save Definitions
Close the GTMAPPER
Check Parameter
Run Masking
Check Results
The save operation creates
A file containing
- Rules
- Run-time Options
- Start Script

SDM – Auditing and Logging Options
Depending on the option
An audit file is generated
containing all actions along
with the original and the
new values
Log files are written which
contain information about
the masking run and possible
errors

Next Steps
Order a test copy of Data masking for Adabas
Get assistance if needed
Saving money by reducing project time
Reduce demands on application staff
Improve application quality
Achieve compliance

Data Archiving for Adabas

Data Archiving for Adabas
Terminology
• Source database
One or more Adabas production database(s) where the daily business runs on
• VAULT
Intermediate Store – an Adabas database either on the same computer or a
different one
Vault – a offline store – a set of intelligently managed files / data sets
This is part of the product. The only pre-requisite is disk space.
• User Interface - “The dashboard”
For defining, managing, running and monitor archiving tasks. It is part of the
product
| Data Archiving Insight |May 2010 | 27

Data Archiving for Adabas
More Terminology
• EXTRACT
Read selected data from the source database and file(s)
• ARCHIVE
Writes extracted data to either an intermediate store or a vault
Does NOT remove the records from the Source
• TRANSFER
Move or Copy to ADABAS
| Data Archiving Insight |May 2010 | 28

Dynamic Extraction Syntax Examples
-------------------------------------------------------------------------------
+++ Using integer variables
-------------------------------------------------------------------------------
/* Pick out all PEOPLE in Derby who have a car between 1980-82*/
INT[0] = 1980;
INT[1] = 1982;
EXTRACT PEOPLE( PEOPLE.AJ == "DERBY")
{
}
EXTRACT CARS (CARS.AC == PEOPLE.AA)
{
}
/* Archive all CARS registered between 1980 - 1982*/
IF(CARS.AG >= INT[1])
{
}
ELSE
{
}
ARCHIVE CARS[*];
/* Delete all CARS registered before 1980 */
IF(INT[0]

Dynamic Extraction Syntax Examples
-------------------------------------------------------------------------------
+++ REMOVE records from a file
-------------------------------------------------------------------------------
EXTRACT PEOPLE (PEOPLE.AJ == "DERBY")
{
}
EXTRACT CARS (CARS.AC == PEOPLE.AA)
{
}
/* Only process Ford ORION 1.6 GHIA's */
IF (CARS.AD == "FORD" && CARS.AE == "ORION 1.6 GHIA")
{
}
/* Archive the records for the CARS we are about to remove */
ARCHIVE CARS[*];
/* Remove record from the CARS file */
REMOVE CARS;
| Data Archiving Insight |May 2010 | 30

Thank You!

Data Archiving for Adabas
More Terminology
• REMOVE
Deletes the Archived records from the Source
• MODIFY
Change field(s) values in source records after Archive
| Data Archiving Insight |May 2010 | 32

Administration User Interface
• A browser based graphical tool for
administrators…
Define Vaults
The final secure store for your historical
data
Define Plans
A virtual folder for archiving tasks
Define Actions
The actual archiving task with all required
details
Administer the environment
• Search the archive history and content
• Recall content to its original form for
detailed search
• Observe archive activities across the
enterprise
| Data Archiving Insight |May 2010 | 33

Dynamic Extraction Syntax Examples
EXTRACT PEOPLE(PEOPLE.AJ == "DERBY" || PEOPLE.AJ == "PARIS" ||
PEOPLE.AJ == "DETROIT")
{
EXTRACT CARS(CARS.AC == PEOPLE.AA)
{
}
ARCHIVE CARS[*];
EXTRACT OTHER(OTHER.CA == PEOPLE.AA)
{
}
ARCHIVE OTHER[*];
ARCHIVE PEOPLE[*];
| Data Archiving Insight |May 2010 | 34

Dynamic Extraction Syntax Examples
-------------------------------------------------------------------------------
+++ Using MODIFY to change the field values of a record
-------------------------------------------------------------------------------
EXTRACT PEOPLE (PEOPLE.AJ == "DERBY")
{
}
EXTRACT CARS (CARS.AC == PEOPLE.AA)
{
}
/* Only process Fords, Vauxhalls and Austins */
IF (CARS.AD == ["FORD","VAUXHALL","AUSTIN"])
{
}
/* Archive the current CARS records */
ARCHIVE CARS[*];
/* Upgrade the employees car to a Rolls Royce by modifying the record */
MODIFY CARS.AD = "ROLLS ROYCE";
MODIFY CARS.AE = "PHANTOM";
| Data Archiving Insight |May 2010 | 35

Dynamic Extraction Syntax Examples
-------------------------------------------------------------------------------
+++ Using integer variables
-------------------------------------------------------------------------------
/* Pick out all PEOPLE in Derby who have a car between 1980-82*/
INT[0] = 1980;
INT[1] = 1982;
EXTRACT PEOPLE( PEOPLE.AJ == "DERBY")
{
}
EXTRACT CARS (CARS.AC == PEOPLE.AA)
{
}
/* Archive all CARS registered between 1980 - 1982*/
IF(CARS.AG >= INT[1])
{
}
ELSE
{
}
ARCHIVE CARS[*];
/* Delete all CARS registered before 1980 */
IF(INT[0]

Dynamic Extraction Syntax Examples
-------------------------------------------------------------------------------
+++ REMOVE records from a file
-------------------------------------------------------------------------------
EXTRACT PEOPLE (PEOPLE.AJ == "DERBY")
{
}
EXTRACT CARS (CARS.AC == PEOPLE.AA)
{
}
/* Only process Ford ORION 1.6 GHIA's */
IF (CARS.AD == "FORD" && CARS.AE == "ORION 1.6 GHIA")
{
}
/* Archive the records for the CARS we are about to remove */
ARCHIVE CARS[*];
/* Remove record from the CARS file */
REMOVE CARS;
| Data Archiving Insight |May 2010 | 37

Thank You!