28.04.2013 Views

Configuring datapump extract in Oracle GoldenGate (OGG) 11.2 1 ...

Configuring datapump extract in Oracle GoldenGate (OGG) 11.2 1 ...

Configuring datapump extract in Oracle GoldenGate (OGG) 11.2 1 ...

SHOW MORE
SHOW LESS

You also want an ePaper? Increase the reach of your titles

YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.

<strong>Configur<strong>in</strong>g</strong> <strong>datapump</strong> <strong>extract</strong> <strong>in</strong> <strong>Oracle</strong> <strong>GoldenGate</strong> (<strong>OGG</strong>) <strong>11.2</strong><br />

In the article you will have a look at an example of how to configure an<br />

additional <strong>datapump</strong> <strong>extract</strong> <strong>in</strong> <strong>OGG</strong>. How are <strong>OGG</strong> <strong>in</strong>stallation performed and<br />

basic classic CDC <strong>extract</strong> and replicat configured and verified is described<br />

here.<br />

The article will cover the follow<strong>in</strong>g topics<br />

What is a data pump <strong>extract</strong><br />

Example of sett<strong>in</strong>g a data pump <strong>extract</strong><br />

1. What is a data pump <strong>extract</strong><br />

With a s<strong>in</strong>gle <strong>extract</strong> and correspond<strong>in</strong>g replicats configuration as discussed<br />

previously here the <strong>extract</strong> was responsible for captur<strong>in</strong>g the changes from the<br />

transaction log on the source system, it was a classic <strong>extract</strong>, and shipp<strong>in</strong>g<br />

it to a remote trail on the target system via TCP/IP. The replicat(s) were<br />

read<strong>in</strong>g from the local trail files and apply<strong>in</strong>g the changes to the target<br />

database(s).<br />

Data pump <strong>extract</strong> is an optional <strong>extract</strong> that is created and started on the<br />

source system. Witch a data pump <strong>extract</strong> present the process<strong>in</strong>g changes as<br />

follows. The primary <strong>extract</strong> still captures the changes from the transaction<br />

log of the primary database but writes it to a local trail file on the source<br />

system, <strong>in</strong>stead of to a remote trail on the target system as without a data<br />

pump <strong>extract</strong>. Data pump <strong>extract</strong> reads data from the local trail on the source<br />

system and sends it to the remote trail on the target system us<strong>in</strong>g TCP/IP.<br />

The replicat <strong>in</strong> both cases, with or without a data pump, reads data from the<br />

local trail on the target and applies it to the target database.<br />

The advantage offered by data pump <strong>extract</strong> is that<br />

Network failure between source and target systems are tolerated without<br />

primary <strong>extract</strong> abend<strong>in</strong>g, <strong>in</strong> case of network failure, by hav<strong>in</strong>g data on<br />

a trail on the source system and a dedicated data pump <strong>extract</strong> for data<br />

shipment to the target system<br />

Data pump <strong>extract</strong> can perform extra process<strong>in</strong>g on the data such as<br />

filter<strong>in</strong>g, mapp<strong>in</strong>g etc and pass the data processed to the <strong>extract</strong>. The<br />

data pump can also operate <strong>in</strong> a passthru mode where only transmits the<br />

data as it is. Data pump <strong>extract</strong> does not require a log on to the<br />

source database.<br />

2. Example of sett<strong>in</strong>g a data pump <strong>extract</strong>


On the source system I have sales schema <strong>in</strong> RACD database. On the target<br />

system I have sales schema <strong>in</strong> RACDB database. In the sales schema on both<br />

sites there are two tables named sales and cust.<br />

create table sales<br />

(<br />

product_id number(20) primary key,<br />

product varchar2(50),<br />

channel_id number,<br />

cust_id number,<br />

amount_sold number(10,2),<br />

time_id date)<br />

partition by list (channel_id)<br />

(partition c0 values (0),<br />

);<br />

partition c1 values (1),<br />

partition c2 values (2),<br />

partition c3 values (3),<br />

partition c4 values (4)<br />

create table cust<br />

(<br />

cust_id number(20) primary key,<br />

name varchar2(50),<br />

address varchar2(4000),<br />

bank_account number,<br />

state varchar2(2),<br />

country varchar2(50));<br />

I am populat<strong>in</strong>g the source with the follow<strong>in</strong>g statements. The sales<br />

table is with 1 Mil records and cust table is with 20K records.<br />

<strong>in</strong>sert /*+ append */ <strong>in</strong>to sales<br />

select


ownum,<br />

'Someth<strong>in</strong>g ||mod(rownum,100000) as product,<br />

mod(rownum,5) as channel_id,<br />

mod(rownum,1000) as cust_id ,<br />

5000 as amount_sold,<br />

to_date<br />

('01.' || lpad(to_char(mod(rownum,12)+1),2,'0') || '.2010' ,'dd.mm.yyyy')<br />

as time_id<br />

from dual connect by level


Install <strong>OGG</strong> on both sites and configure managers – ( see the <strong>OGG</strong><br />

<strong>in</strong>stall )<br />

Set CDC – start CDC <strong>extract</strong><br />

Perform an <strong>in</strong>itial dataload (<strong>OGG</strong> based is discussed <strong>in</strong> the article)<br />

Start data apply – start the replicat to apply changes captured by CDC<br />

Set NOHANDLECOLLISIONS on the target replicats<br />

Note. Special thanks to Chris Tijer<strong>in</strong>a who po<strong>in</strong>ted out that HANDLECOLLISIONS<br />

(HC) is required only if us<strong>in</strong>g an <strong>OGG</strong> for the <strong>in</strong>itial dataload. However, if<br />

you use SCN based exp(dp)/imp(dp) or rman to build a po<strong>in</strong>t <strong>in</strong> time copy on<br />

the target from the source DB than you can skip sett<strong>in</strong>g the HC. For <strong>Oracle</strong><br />

source and target database imp(dp)/expdp) pump or rman is much faster way to<br />

<strong>in</strong>itially move the data than <strong>OGG</strong> native methods. <strong>OGG</strong> native <strong>in</strong>itial data load<br />

methods are very valuable <strong>in</strong> heterogeneous data transfers. I will blog about<br />

this no HC topic later but the idea is as follows:<br />

Start the primary <strong>extract</strong> gett<strong>in</strong>g CDC from the source database;<br />

Make sure that there is no long runn<strong>in</strong>g transaction. Query until none<br />

or kill the transaction.<br />

Query the database for the SQL and write the SCN down<br />

Make sure that exp/imp or rman restore on the target system is done<br />

Use the follow<strong>in</strong>g <strong>OGG</strong> syntax to start the replicat on the target<br />

system: start replicat afterscn <br />

When replicats start they will parse through the trail files and will start<br />

apply<strong>in</strong>g the data with next SCN right after the SCN for the imp(dp)/exp(dp)<br />

or rman restore or duplicate.<br />

For CDC capture from <strong>Oracle</strong> source database you will need to perform the<br />

follow<strong>in</strong>g steps <strong>in</strong> an orderly manner.<br />

a. Add supplemental logg<strong>in</strong>g at database level<br />

SQL> alter database add supplemental log data;<br />

Database altered.<br />

SQL><br />

b. Add supplemental logg<strong>in</strong>g at table level<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 34> dblog<strong>in</strong> userid ddl_ogg<br />

Password:<br />

Successfully logged <strong>in</strong>to database.


GGSCI (racl<strong>in</strong>ux1.gj.com) 35><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 36> add trandata sales.sales<br />

Logg<strong>in</strong>g of supplemental redo data enabled for table SALES.SALES.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 37> add trandata sales.cust<br />

Logg<strong>in</strong>g of supplemental redo data enabled for table SALES.CUST.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 38><br />

c. Add CDC <strong>extract</strong> extsale, create parameter file for extsale <strong>extract</strong>, add<br />

ext trail ./dir.dat/zz for extsale <strong>extract</strong> and start <strong>extract</strong> extsale<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 53> view param extsale<br />

<strong>extract</strong> extsale<br />

tranlogoptions asmuser sys@ASM, asmpassword sys1<br />

--checkparams<br />

--NODYNAMICRESOLUTION<br />

userid ogg_<strong>extract</strong>, password ogg_<strong>extract</strong><br />

exttrail ./dirdat/zz<br />

table sales.sales;<br />

table sales.cust;<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 54><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 47> add <strong>extract</strong> extsale, tranlog, beg<strong>in</strong> now, threads 2<br />

EXTRACT added.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 48><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 54> add exttrail ./dirdat/zz, <strong>extract</strong> extsale megabytes 500<br />

EXTTRAIL added.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 55><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 58> start <strong>extract</strong> extsale<br />

Send<strong>in</strong>g START request to MANAGER ...<br />

EXTRACT EXTSALE start<strong>in</strong>g


GGSCI (racl<strong>in</strong>ux1.gj.com) 59> <strong>in</strong>fo all<br />

Program Status Group Lag at Chkpt Time S<strong>in</strong>ce Chkpt<br />

MANAGER RUNNING<br />

EXTRACT RUNNING EXTCDC 00:00:00 00:00:09<br />

EXTRACT RUNNING EXTSALE 00:00:00 00:09:16<br />

REPLICAT RUNNING REPCDC1 00:00:00 00:00:08<br />

REPLICAT RUNNING REPCDC2 00:00:00 00:00:06<br />

REPLICAT RUNNING REPCDC3 00:00:00 00:00:04<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 60><br />

d. Add a pump <strong>extract</strong> (pumpsale) and start it<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 63> view params pumpsale<br />

<strong>extract</strong> pumpsale<br />

passthru<br />

rmthost racl<strong>in</strong>ux1, mgrport 7809<br />

rmttrail ./dirdat/yy<br />

table sales.sales;<br />

table sales.cust;<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 64><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 64> add <strong>extract</strong> pumpsale, exttrailsource ./dirdat/zz<br />

EXTRACT added.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 65><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 66> add rmttrail ./dirdat/yy, <strong>extract</strong> pumpsale megabytes 500<br />

RMTTRAIL added.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 67><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 67> start <strong>extract</strong> pumpsale<br />

Send<strong>in</strong>g START request to MANAGER ...


EXTRACT PUMPSALE start<strong>in</strong>g<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 68> <strong>in</strong>fo all<br />

Program Status Group Lag at Chkpt Time S<strong>in</strong>ce Chkpt<br />

MANAGER RUNNING<br />

EXTRACT RUNNING EXTCDC 00:00:00 00:00:02<br />

EXTRACT RUNNING EXTSALE 00:00:00 00:00:02<br />

EXTRACT RUNNING PUMPSALE 00:00:00 00:02:48<br />

REPLICAT RUNNING REPCDC1 00:00:00 00:00:06<br />

REPLICAT RUNNING REPCDC2 00:00:00 00:00:05<br />

REPLICAT RUNNING REPCDC3 00:00:00 00:00:03<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 69><br />

e. Create replicats resale and repcust. DO NOT START the replicats before<br />

complet<strong>in</strong>g <strong>in</strong>itial data load<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 71> view params repsale<br />

replicat repsale<br />

userid ogg_replicat, password ogg_replicat<br />

handlecollisions<br />

assumetargetdefs<br />

discarfile ./dirrpt/sale.dsc, append<br />

map sales.sales, target sales.sales;<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 72><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 73> view params repcust<br />

replicat repcust<br />

userid ogg_replicat, password ogg_replicat<br />

handlecollisions<br />

assumetargetdefs<br />

discarfile ./dirrpt/sale.dsc, append


map sales.cust, target sales.cust;<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 74><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 74> add replicat repsale, exttrail ./dirdat/yy<br />

REPLICAT added.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 75> add replicat repcust, exttrail ./dirdat/yy<br />

REPLICAT added.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 76><br />

f. Initial dataload ( for details see here). Create a task <strong>extract</strong> and a<br />

replicat with the follow<strong>in</strong>g parameters and options for data collection<br />

(SOURCEISTABLE) and no checkpo<strong>in</strong>t operation (SPACIALRUN).Start only the<br />

<strong>extract</strong> but not the replicat and wait for the completion.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 8> view params <strong>in</strong>itexts<br />

<strong>extract</strong> <strong>in</strong>itexts<br />

SETENV (ORACLE_SID = "RACD1")<br />

---tranlogoptions asmuser sys@ASM, asmpassword sys1<br />

userid ogg_<strong>extract</strong>, password ogg_<strong>extract</strong><br />

rmthost racl<strong>in</strong>ux1, mgrport 7809<br />

rmttask replicat, group <strong>in</strong>itreps<br />

table sales.sales;<br />

table sales.cust;<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 9> add <strong>extract</strong> <strong>in</strong>itexts, sourceistable<br />

EXTRACT added.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 10><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 8> view params <strong>in</strong>itreps<br />

replicat <strong>in</strong>itreps


SETENV (ORACLE_SID = "RACDB1")<br />

userid ogg_replicat, password ogg_replicat<br />

assumetargetdefs<br />

discardfile ./dirrpt/sales_schema.dsc, purge<br />

map sales.sales, target sales.sales;<br />

map sales.cust, target sales.cust;<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 9><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 9> add replicat <strong>in</strong>itreps, specialrun<br />

REPLICAT added.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 10><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 10> start <strong>extract</strong> <strong>in</strong>itexts<br />

Send<strong>in</strong>g START request to MANAGER ...<br />

EXTRACT INITEXTS start<strong>in</strong>g<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 11><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 40> view report <strong>in</strong>itexts<br />

2012-07-08 04:50:42 INFO <strong>OGG</strong>-01017 Wildcard resolution set to IMMEDIATE because<br />

SOURCEISTABLE is used.<br />

***********************************************************************<br />

<strong>Oracle</strong> <strong>GoldenGate</strong> Capture for <strong>Oracle</strong><br />

Version <strong>11.2</strong>.1.0.1 <strong>OGG</strong>CORE_<strong>11.2</strong>.1.0.1_PLATFORMS_120423.0230_FBO<br />

L<strong>in</strong>ux, x64, 64bit (optimized), <strong>Oracle</strong> 11g on Apr 23 2012 08:42:16<br />

Copyright (C) 1995, 2012, <strong>Oracle</strong> and/or its affiliates. All rights reserved.<br />

Start<strong>in</strong>g at 2012-07-08 04:50:42<br />

***********************************************************************<br />

Operat<strong>in</strong>g System Version:


L<strong>in</strong>ux<br />

Version #1 SMP Thu Sep 3 04:15:13 EDT 2009, Release 2.6.18 -164.el5<br />

Node: racl<strong>in</strong>ux1.gj.com<br />

Mach<strong>in</strong>e: x86_64<br />

soft limit hard limit<br />

Address Space Size : unlimited unlimited<br />

Heap Size : unlimited unlimited<br />

File Size : unlimited unlimited<br />

CPU Time : unlimited unlimited<br />

Process id: 14866<br />

Description:<br />

***********************************************************************<br />

** Runn<strong>in</strong>g with the follow<strong>in</strong>g parameters **<br />

***********************************************************************<br />

2012-07-08 04:50:42 INFO <strong>OGG</strong>-03035 Operat<strong>in</strong>g system character set identified as UTF-8.<br />

Locale: en_US, LC_ALL:.<br />

<strong>extract</strong> <strong>in</strong>itexts<br />

SETENV (ORACLE_SID = "RACD1")<br />

Set environment variable (ORACLE_SID=RACD1)<br />

userid ogg_<strong>extract</strong>, password ***********<br />

2012-07-08 04:50:42 INFO <strong>OGG</strong>-03500 WARNING: NLS_LANG environment variable does not match<br />

database character set, or not set. U<br />

s<strong>in</strong>g database character set value of AL32UTF8.<br />

rmthost racl<strong>in</strong>ux1, mgrport 7809<br />

rmttask replicat, group <strong>in</strong>itreps<br />

table sales.sales;<br />

Us<strong>in</strong>g the follow<strong>in</strong>g key columns for source table SALES.SALES: PRODUCT_ID.


table sales.cust;<br />

Us<strong>in</strong>g the follow<strong>in</strong>g key columns for source table SALES.CUST: CUST_ID.<br />

2012-07-08 04:50:43 INFO <strong>OGG</strong>-01815 Virtual Memory Facilities for: COM<br />

anon alloc: mmap(MAP_ANON) anon free: munmap<br />

file alloc: mmap(MAP_SHARED) file free: munmap<br />

target directories:<br />

/u02/stage_ogg112_ora11/dirtmp.<br />

CACHEMGR virtual memory values (may have been adjusted)<br />

CACHESIZE: 64G<br />

CACHEPAGEOUTSIZE (normal): 8M<br />

PROCESS VM AVAIL FROM OS (m<strong>in</strong>): 128G<br />

CACHESIZEMAX (strict force to disk): 96G<br />

Database Version:<br />

<strong>Oracle</strong> Database 11g Enterprise Edition Release <strong>11.2</strong>.0.3.0 - 64bit Production<br />

PL/SQL Release <strong>11.2</strong>.0.3.0 - Production<br />

CORE <strong>11.2</strong>.0.3.0 Production<br />

TNS for L<strong>in</strong>ux: Version <strong>11.2</strong>.0.3.0 - Production<br />

NLSRTL Version <strong>11.2</strong>.0.3.0 - Production<br />

Database Language and Character Set:<br />

NLS_LANG = ".AL32UTF8"<br />

NLS_LANGUAGE = "AMERICAN"<br />

NLS_TERRITORY = "AMERICA"<br />

NLS_CHARACTERSET = "AL32UTF8"<br />

Process<strong>in</strong>g table SALES.SALES<br />

Process<strong>in</strong>g table SALES.CUST


***********************************************************************<br />

* ** Run Time Statistics ** *<br />

***********************************************************************<br />

Report at 2012-07-08 10:32:26 (activity s<strong>in</strong>ce 2012-07-08 04:50:42)<br />

Output to <strong>in</strong>itreps:<br />

From Table SALES.SALES:<br />

# <strong>in</strong>serts: 1000000<br />

# updates: 0<br />

# deletes: 0<br />

# discards: 0<br />

From Table SALES.CUST:<br />

# <strong>in</strong>serts: 20000<br />

# updates: 0<br />

# deletes: 0<br />

# discards: 0<br />

REDO Log Statistics<br />

Bytes parsed 0<br />

Bytes output 175694478<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 41><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 44> view report <strong>in</strong>itreps<br />

***********************************************************************<br />

<strong>Oracle</strong> <strong>GoldenGate</strong> Delivery for <strong>Oracle</strong>


Version <strong>11.2</strong>.1.0.1 <strong>OGG</strong>CORE_<strong>11.2</strong>.1.0.1_PLATFORMS_120423.0230_FBO<br />

L<strong>in</strong>ux, x64, 64bit (optimized), <strong>Oracle</strong> 11g on Apr 23 2012 08:48:07<br />

Copyright (C) 1995, 2012, <strong>Oracle</strong> and/or its affiliates. All rights reserved.<br />

Start<strong>in</strong>g at 2012-07-08 04:50:47<br />

***********************************************************************<br />

Operat<strong>in</strong>g System Version:<br />

L<strong>in</strong>ux<br />

Version #1 SMP Thu Sep 3 04:15:13 EDT 2009, Release 2.6.18-164.el5<br />

Node: racl<strong>in</strong>ux1.gj.com<br />

Mach<strong>in</strong>e: x86_64<br />

soft limit hard limit<br />

Address Space Size : unlimited unlimited<br />

Heap Size : unlimited unlimited<br />

File Size : unlimited unlimited<br />

CPU Time : unlimited unlimited<br />

Process id: 14879<br />

Description:<br />

***********************************************************************<br />

** Runn<strong>in</strong>g with the follow<strong>in</strong>g parameters **<br />

***********************************************************************<br />

2012-07-08 04:50:57 INFO <strong>OGG</strong>-03035 Operat<strong>in</strong>g system character set identified as UTF-8.<br />

Locale: en_US, LC_ALL:.<br />

replicat <strong>in</strong>itreps<br />

SETENV (ORACLE_SID = "RACDB1")


Set environment variable (ORACLE_SID=RACDB1)<br />

userid ogg_replicat, password ************<br />

2012-07-08 04:50:57 INFO <strong>OGG</strong>-03501 WARNING: NLS_LANG environment variable is <strong>in</strong>valid or<br />

not set. Us<strong>in</strong>g operat<strong>in</strong>g system character<br />

set value of AL32UTF8.<br />

assumetargetdefs<br />

discardfile ./dirrpt/sales_schema.dsc, purge<br />

map sales.sales, target sales.sales;<br />

map sales.cust, target sales.cust;<br />

2012-07-08 04:50:58 INFO <strong>OGG</strong>-01815 Virtual Memory Facilities for: COM<br />

anon alloc: mmap(MAP_ANON) anon free: munmap<br />

file alloc: mmap(MAP_SHARED) file free: munmap<br />

target directories:<br />

/u02/stage_ogg112_ora11/dirtmp.<br />

CACHEMGR virtual memory values (may have been adjusted)<br />

CACHESIZE: 2G<br />

CACHEPAGEOUTSIZE (normal): 8M<br />

PROCESS VM AVAIL FROM OS (m<strong>in</strong>): 4G<br />

CACHESIZEMAX (strict force to disk): 3.41G<br />

Database Version:<br />

<strong>Oracle</strong> Database 11g Enterprise Edition Release <strong>11.2</strong>.0.3.0 - 64bit Production<br />

PL/SQL Release <strong>11.2</strong>.0.3.0 - Production<br />

CORE <strong>11.2</strong>.0.3.0 Production<br />

TNS for L<strong>in</strong>ux: Version <strong>11.2</strong>.0.3.0 - Production<br />

NLSRTL Version <strong>11.2</strong>.0.3.0 - Production<br />

Database Language and Character Set:<br />

NLS_LANG = ".AL32UTF8"


NLS_LANGUAGE = "AMERICAN"<br />

NLS_TERRITORY = "AMERICA"<br />

NLS_CHARACTERSET = "AL32UTF8"<br />

***********************************************************************<br />

** Run Time Messages **<br />

***********************************************************************<br />

MAP resolved (entry sales.sales):<br />

map "SALES"."SALES", target sales.sales;<br />

Us<strong>in</strong>g follow<strong>in</strong>g columns <strong>in</strong> default map by name:<br />

PRODUCT_ID, PRODUCT, CHANNEL_ID, CUST_ID, AMOUNT_SOLD, TIME_ID<br />

Us<strong>in</strong>g the follow<strong>in</strong>g key columns for target table SALES.SALES: PRODUCT_ID.<br />

MAP resolved (entry sales.cust):<br />

map "SALES"."CUST", target sales.cust;<br />

Us<strong>in</strong>g follow<strong>in</strong>g columns <strong>in</strong> default map by name:<br />

CUST_ID, NAME, ADDRESS, BANK_ACCOUNT, STATE, COUNTRY<br />

Us<strong>in</strong>g the follow<strong>in</strong>g key columns for target table SALES.CUST: CUST_ID.<br />

***********************************************************************<br />

* ** Run Time Statistics ** *<br />

***********************************************************************<br />

Report at 2012-07-08 10:32:31 (activity s<strong>in</strong>ce 2012-07-08 04:51:01)


From Table SALES.SALES to SALES.SALES:<br />

# <strong>in</strong>serts: 1000000<br />

# updates: 0<br />

# deletes: 0<br />

# discards: 0<br />

From Table SALES.CUST to SALES.CUST:<br />

# <strong>in</strong>serts: 20000<br />

# updates: 0<br />

# deletes: 0<br />

# discards: 0<br />

CACHE OBJECT MANAGER statistics<br />

CACHE MANAGER VM USAGE<br />

vm current = 0 vm anon queues = 0<br />

vm anon <strong>in</strong> use = 0 vm file = 0<br />

vm used max = 0 ==> CACHE BALANCED<br />

CACHE CONFIGURATION<br />

cache size = 2G cache force pag<strong>in</strong>g = 3.41G<br />

buffer m<strong>in</strong> = 64K buffer highwater = 8M<br />

pageout eligible size = 8M<br />

================================================================================<br />

RUNTIME STATS FOR SUPERPOOL<br />

CACHE Transaction Stats<br />

trans active = 0 max concurrent = 0<br />

non-zero total = 0 trans total = 0<br />

CACHE File Cach<strong>in</strong>g


disk current = 0 disk total = 0<br />

disk cach<strong>in</strong>g = 0 file cached = 0<br />

file retrieves = 0<br />

CACHE MANAGEMENT<br />

buffer l<strong>in</strong>ks = 0 anon gets = 0<br />

forced unmaps = 0 cnnbl try = 0<br />

cached out = 0 force out = 0<br />

Allocation Request Distribution<br />

< 128B: 0<br />

128B: 0 0 | 512B: 0 0<br />

2K: 0 0 | 8K: 0 0<br />

32K: 0 0 | 128K: 0 0<br />

512K: 0 0 | 2M: 0 0<br />

8M: 0 0 | 32M: 0 0<br />

128M: 0 0 | 512M: 0 0<br />

2G: 0 0 | 8G: 0<br />

Cached Transaction Size Distribution<br />

0: 0<br />

< 4K: 0<br />

4K: 0 0 | 16K: 0 0<br />

64K: 0 0 | 256K: 0 0<br />

1M: 0 0 | 4M: 0 0<br />

16M: 0 0 | 64M: 0 0<br />

256M: 0 0 | 1G: 0 0<br />

4G: 0 0 | 16G: 0 0<br />

64G: 0 0 | 256G: 0 0<br />

1T: 0 0 | 4T: 0 0<br />

16T: 0 0 | 64T: 0 0<br />

256T: 0 0 |1024T: 0 0


================================================================================<br />

CUMULATIVE STATS FOR SUPERPOOL<br />

CACHE Transaction Stats<br />

trans active = 0 max concurrent = 0<br />

non-zero total = 0 trans total = 0<br />

CACHE File Cach<strong>in</strong>g<br />

disk current = 0 disk total = 0<br />

disk cach<strong>in</strong>g = 0 file cached = 0<br />

file retrieves = 0<br />

CACHE MANAGEMENT<br />

buffer l<strong>in</strong>ks = 0 anon gets = 0<br />

forced unmaps = 0 cnnbl try = 0<br />

cached out = 0 force out = 0<br />

Allocation Request Distribution<br />

< 128B: 0<br />

128B: 0 0 | 512B: 0 0<br />

2K: 0 0 | 8K: 0 0<br />

32K: 0 0 | 128K: 0 0<br />

512K: 0 0 | 2M: 0 0<br />

8M: 0 0 | 32M: 0 0<br />

128M: 0 0 | 512M: 0 0<br />

2G: 0 0 | 8G: 0<br />

Cached Transaction Size Distribution<br />

0: 0<br />

< 4K: 0<br />

4K: 0 0 | 16K: 0 0


64K: 0 0 | 256K: 0 0<br />

1M: 0 0 | 4M: 0 0<br />

16M: 0 0 | 64M: 0 0<br />

256M: 0 0 | 1G: 0 0<br />

4G: 0 0 | 16G: 0 0<br />

64G: 0 0 | 256G: 0 0<br />

1T: 0 0 | 4T: 0 0<br />

16T: 0 0 | 64T: 0 0<br />

256T: 0 0 |1024T: 0 0<br />

QUEUE Statistics:<br />

num queues = 15 default <strong>in</strong>dex = 0<br />

cur len = 0 max len = 0<br />

q vm current = 0 vm max = 0<br />

q hits = 0 q misses = 0<br />

queue size q hits curlen maxlen cannibalized<br />

0 64K 0 0 0 0<br />

1 128K 0 0 0 0<br />

2 256K 0 0 0 0<br />

3 512K 0 0 0 0<br />

4 1M 0 0 0 0<br />

5 2M 0 0 0 0<br />

6 4M 0 0 0 0<br />

7 8M 0 0 0 0<br />

8 16M 0 0 0 0<br />

9 32M 0 0 0 0<br />

10 64M 0 0 0 0<br />

11 128M 0 0 0 0<br />

12 256M 0 0 0 0<br />

13 512M 0 0 0 0


14 1G 0 0 0 0<br />

================================================================================<br />

RUNTIME STATS FOR CACHE POOL #0<br />

POOL INFO group: <strong>in</strong>itreps id: p14879_BLOB<br />

trans active = 0 trans concurrent (max) = 0<br />

trans total = 0 (0 )<br />

flag = 0x00000030<br />

last error = (0=)<br />

Allocation Request Distribution<br />

< 128B: 0<br />

128B: 0 0 | 512B: 0 0<br />

2K: 0 0 | 8K: 0 0<br />

32K: 0 0 | 128K: 0 0<br />

512K: 0 0 | 2M: 0 0<br />

8M: 0 0 | 32M: 0 0<br />

128M: 0 0 | 512M: 0 0<br />

2G: 0 0 | 8G: 0<br />

================================================================================<br />

CUMULATIVE STATS FOR CACHE POOL #0<br />

POOL INFO group: <strong>in</strong>itreps id: p14879_BLOB<br />

trans active = 0 trans concurrent (max) = 0<br />

trans total = 0 (0 )<br />

flag = 0x00000030<br />

last error = (0=)<br />

Allocation Request Distribution<br />

< 128B: 0<br />

128B: 0 0 | 512B: 0 0<br />

2K: 0 0 | 8K: 0 0


32K: 0 0 | 128K: 0 0<br />

512K: 0 0 | 2M: 0 0<br />

8M: 0 0 | 32M: 0 0<br />

128M: 0 0 | 512M: 0 0<br />

2G: 0 0 | 8G: 0<br />

QUEUE Statistics:<br />

num queues = 15 default <strong>in</strong>dex = 0<br />

cur len = 0 max len = 0<br />

q vm current = 0 vm max = 0<br />

q hits = 0 q misses = 0<br />

queue size q hits curlen maxlen cannibalized<br />

0 64K 0 0 0 0<br />

1 128K 0 0 0 0<br />

2 256K 0 0 0 0<br />

3 512K 0 0 0 0<br />

4 1M 0 0 0 0<br />

5 2M 0 0 0 0<br />

6 4M 0 0 0 0<br />

7 8M 0 0 0 0<br />

8 16M 0 0 0 0<br />

9 32M 0 0 0 0<br />

10 64M 0 0 0 0<br />

11 128M 0 0 0 0<br />

12 256M 0 0 0 0<br />

13 512M 0 0 0 0<br />

14 1G 0 0 0 0<br />

================================================================================<br />

RUNTIME STATS FOR CACHE POOL #0


POOL INFO group: <strong>in</strong>itreps id: p14879_BLOB<br />

trans active = 0 trans concurrent (max) = 0<br />

trans total = 0 (0 )<br />

flag = 0x00000030<br />

last error = (0=)<br />

Allocation Request Distribution<br />

< 128B: 0<br />

128B: 0 0 | 512B: 0 0<br />

2K: 0 0 | 8K: 0 0<br />

32K: 0 0 | 128K: 0 0<br />

512K: 0 0 | 2M: 0 0<br />

8M: 0 0 | 32M: 0 0<br />

128M: 0 0 | 512M: 0 0<br />

2G: 0 0 | 8G: 0<br />

================================================================================<br />

CUMULATIVE STATS FOR CACHE POOL #0<br />

POOL INFO group: <strong>in</strong>itreps id: p14879_BLOB<br />

trans active = 0 trans concurrent (max) = 0<br />

trans total = 0 (0 )<br />

flag = 0x00000030<br />

last error = (0=)<br />

Allocation Request Distribution<br />

< 128B: 0<br />

128B: 0 0 | 512B: 0 0<br />

2K: 0 0 | 8K: 0 0<br />

32K: 0 0 | 128K: 0 0<br />

512K: 0 0 | 2M: 0 0<br />

8M: 0 0 | 32M: 0 0<br />

128M: 0 0 | 512M: 0 0


2G: 0 0 | 8G: 0<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 45><br />

g. Start all replicates (repsale and repcust)<br />

ggsci>start replicat repsale<br />

ggsci>start replicat repcust<br />

h. Verify that <strong>OGG</strong> setup capture data from source and replicate it to the<br />

target tables.<br />

I am go<strong>in</strong>g to perform a transaction on each of the tables.<br />

Let’s change all 20000 records on cust source table and see how <strong>OGG</strong><br />

behaves and propagates the data from source database to the target<br />

database.<br />

update cust set country='USA';<br />

commit;<br />

Check the stats on <strong>extract</strong> extsale, <strong>extract</strong> pumpsale and replicat<br />

resale to verify the architecture and that transactions are processed<br />

as expected.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 68> stats <strong>extract</strong> extsale<br />

Send<strong>in</strong>g STATS request to EXTRACT EXTSALE ...<br />

Start of Statistics at 2012-07-08 14:43:23.<br />

Output to ./dirdat/zz:<br />

Extract<strong>in</strong>g from SALES.CUST to SALES.CUST:<br />

*** Total statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

*** Hourly statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00


*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

End of Statistics.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 69><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 72> stats <strong>extract</strong> pumpsale<br />

Send<strong>in</strong>g STATS request to EXTRACT PUMPSALE ...<br />

Start of Statistics at 2012-07-08 14:43:54.<br />

Output to ./dirdat/yy:<br />

Extract<strong>in</strong>g from SALES.CUST to SALES.CUST:<br />

*** Total statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

*** Hourly statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

End of Statistics.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 73><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 75> stats replicat repcust<br />

Send<strong>in</strong>g STATS request to REPLICAT REPCUST ...<br />

Start of Statistics at 2012-07-08 14:46:11.


Replicat<strong>in</strong>g from SALES.CUST to SALES.CUST:<br />

*** Total statistics s<strong>in</strong>ce 2012-07-08 14:43:32 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:43:32 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

*** Hourly statistics s<strong>in</strong>ce 2012-07-08 14:43:32 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:43:32 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

End of Statistics.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 76><br />

Let change some (999K) records on sales table on the source database<br />

and see how <strong>OGG</strong> behaves and propagates the data.<br />

SQL> update sales set amount_sold=20000 where cust_id between 1 and 2000;<br />

999000 rows updated.<br />

SQL><br />

SQL> commit;<br />

Commit complete.<br />

SQL><br />

Check the stats on the <strong>in</strong>volved <strong>extract</strong> and replicat groups and verify<br />

that the changes are processed by the <strong>extract</strong> and replicat.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 73> stats <strong>extract</strong> extsale<br />

Send<strong>in</strong>g STATS request to EXTRACT EXTSALE ...<br />

Start of Statistics at 2012-07-08 15:23:39.<br />

Output to ./dirdat/zz:<br />

Extract<strong>in</strong>g from SALES.CUST to SALES.CUST:<br />

*** Total statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />

Total <strong>in</strong>serts 0.00


Total updates 40000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 40000.00<br />

*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 40000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 40000.00<br />

*** Hourly statistics s<strong>in</strong>ce 2012-07-08 15:00:00 ***<br />

No database operations have been performed.<br />

*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 40000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 40000.00<br />

Extract<strong>in</strong>g from SALES.SALES to SALES.SALES:<br />

*** Total statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 999000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 999000.00<br />

*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 999000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 999000.00<br />

*** Hourly statistics s<strong>in</strong>ce 2012-07-08 15:00:00 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 999000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 999000.00<br />

*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 999000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 999000.00<br />

End of Statistics.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 74><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 78> stats <strong>extract</strong> pumpsale<br />

Send<strong>in</strong>g STATS request to EXTRACT PUMPSALE ...


Start of Statistics at 2012-07-08 15:24:31.<br />

Output to ./dirdat/yy:<br />

Extract<strong>in</strong>g from SALES.CUST to SALES.CUST:<br />

*** Total statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

*** Hourly statistics s<strong>in</strong>ce 2012-07-08 15:00:00 ***<br />

No database operations have been performed.<br />

*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 20000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 20000.00<br />

Extract<strong>in</strong>g from SALES.SALES to SALES.SALES:<br />

*** Total statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 999000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 999000.00<br />

*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 999000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 999000.00<br />

*** Hourly statistics s<strong>in</strong>ce 2012-07-08 15:00:00 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 999000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 999000.00<br />

*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 999000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 999000.00<br />

End of Statistics.


GGSCI (racl<strong>in</strong>ux1.gj.com) 79><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 84> stats replicat repsale<br />

Send<strong>in</strong>g STATS request to REPLICAT REPSALE ...<br />

Start of Statistics at 2012-07-08 15:44:30.<br />

Replicat<strong>in</strong>g from SALES.SALES to SALES.SALES:<br />

*** Total statistics s<strong>in</strong>ce 2012-07-08 15:01:52 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 999000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 999000.00<br />

*** Daily statistics s<strong>in</strong>ce 2012-07-08 15:01:52 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 999000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 999000.00<br />

*** Hourly statistics s<strong>in</strong>ce 2012-07-08 15:01:52 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 999000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 999000.00<br />

*** Latest statistics s<strong>in</strong>ce 2012-07-08 15:01:52 ***<br />

Total <strong>in</strong>serts 0.00<br />

Total updates 999000.00<br />

Total deletes 0.00<br />

Total discards 0.00<br />

Total operations 999000.00<br />

End of Statistics.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 85><br />

i. Send NOHANDLECOLLISIONS and modify accord<strong>in</strong>gly the respective parameter<br />

files.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 85> send replicat repsale, nohandlecollisions<br />

Send<strong>in</strong>g NOHANDLECOLLISIONS request to REPLICAT REPSALE ...<br />

REPSALE No tables found match<strong>in</strong>g * to set NOHANDLECOLLISIONS<br />

.<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 86> send replicat repcust, nohandlecollisions<br />

Send<strong>in</strong>g NOHANDLECOLLISIONS request to REPLICAT REPCUST ...<br />

REPCUST No tables found match<strong>in</strong>g * to set NOHANDLECOLLISIONS<br />

.


GGSCI (racl<strong>in</strong>ux1.gj.com) 87><br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 91> view param repsale<br />

replicat repsale<br />

SETENV (ORACLE_SID = "RACDB1")<br />

userid ogg_replicat, password ogg_replicat<br />

--handlecollisions<br />

assumetargetdefs<br />

discardfile ./dirrpt/sale.dsc, append<br />

map sales.sales, target sales.sales;<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 92> view param repcust<br />

replicat repcust<br />

SETENV (ORACLE_SID = "RACDB1")<br />

userid ogg_replicat, password ogg_replicat<br />

--handlecollisions<br />

assumetargetdefs<br />

discardfile ./dirrpt/sale.dsc, append<br />

map sales.cust, target sales.cust;<br />

GGSCI (racl<strong>in</strong>ux1.gj.com) 93><br />

j. This concludes test<strong>in</strong>g and verify<strong>in</strong>g that pump <strong>extract</strong> operates<br />

properly<br />

Summary:<br />

In the article you had a look at the <strong>OGG</strong> feature data pump <strong>extract</strong>. You<br />

looked at an example of sett<strong>in</strong>g a data pump <strong>extract</strong> and methodology for<br />

verify<strong>in</strong>g that configuration works properly.

Hooray! Your file is uploaded and ready to be published.

Saved successfully!

Ooh no, something went wrong!