Configuring datapump extract in Oracle GoldenGate (OGG) 11.2 1 ...
Configuring datapump extract in Oracle GoldenGate (OGG) 11.2 1 ...
Configuring datapump extract in Oracle GoldenGate (OGG) 11.2 1 ...
You also want an ePaper? Increase the reach of your titles
YUMPU automatically turns print PDFs into web optimized ePapers that Google loves.
<strong>Configur<strong>in</strong>g</strong> <strong>datapump</strong> <strong>extract</strong> <strong>in</strong> <strong>Oracle</strong> <strong>GoldenGate</strong> (<strong>OGG</strong>) <strong>11.2</strong><br />
In the article you will have a look at an example of how to configure an<br />
additional <strong>datapump</strong> <strong>extract</strong> <strong>in</strong> <strong>OGG</strong>. How are <strong>OGG</strong> <strong>in</strong>stallation performed and<br />
basic classic CDC <strong>extract</strong> and replicat configured and verified is described<br />
here.<br />
The article will cover the follow<strong>in</strong>g topics<br />
What is a data pump <strong>extract</strong><br />
Example of sett<strong>in</strong>g a data pump <strong>extract</strong><br />
1. What is a data pump <strong>extract</strong><br />
With a s<strong>in</strong>gle <strong>extract</strong> and correspond<strong>in</strong>g replicats configuration as discussed<br />
previously here the <strong>extract</strong> was responsible for captur<strong>in</strong>g the changes from the<br />
transaction log on the source system, it was a classic <strong>extract</strong>, and shipp<strong>in</strong>g<br />
it to a remote trail on the target system via TCP/IP. The replicat(s) were<br />
read<strong>in</strong>g from the local trail files and apply<strong>in</strong>g the changes to the target<br />
database(s).<br />
Data pump <strong>extract</strong> is an optional <strong>extract</strong> that is created and started on the<br />
source system. Witch a data pump <strong>extract</strong> present the process<strong>in</strong>g changes as<br />
follows. The primary <strong>extract</strong> still captures the changes from the transaction<br />
log of the primary database but writes it to a local trail file on the source<br />
system, <strong>in</strong>stead of to a remote trail on the target system as without a data<br />
pump <strong>extract</strong>. Data pump <strong>extract</strong> reads data from the local trail on the source<br />
system and sends it to the remote trail on the target system us<strong>in</strong>g TCP/IP.<br />
The replicat <strong>in</strong> both cases, with or without a data pump, reads data from the<br />
local trail on the target and applies it to the target database.<br />
The advantage offered by data pump <strong>extract</strong> is that<br />
Network failure between source and target systems are tolerated without<br />
primary <strong>extract</strong> abend<strong>in</strong>g, <strong>in</strong> case of network failure, by hav<strong>in</strong>g data on<br />
a trail on the source system and a dedicated data pump <strong>extract</strong> for data<br />
shipment to the target system<br />
Data pump <strong>extract</strong> can perform extra process<strong>in</strong>g on the data such as<br />
filter<strong>in</strong>g, mapp<strong>in</strong>g etc and pass the data processed to the <strong>extract</strong>. The<br />
data pump can also operate <strong>in</strong> a passthru mode where only transmits the<br />
data as it is. Data pump <strong>extract</strong> does not require a log on to the<br />
source database.<br />
2. Example of sett<strong>in</strong>g a data pump <strong>extract</strong>
On the source system I have sales schema <strong>in</strong> RACD database. On the target<br />
system I have sales schema <strong>in</strong> RACDB database. In the sales schema on both<br />
sites there are two tables named sales and cust.<br />
create table sales<br />
(<br />
product_id number(20) primary key,<br />
product varchar2(50),<br />
channel_id number,<br />
cust_id number,<br />
amount_sold number(10,2),<br />
time_id date)<br />
partition by list (channel_id)<br />
(partition c0 values (0),<br />
);<br />
partition c1 values (1),<br />
partition c2 values (2),<br />
partition c3 values (3),<br />
partition c4 values (4)<br />
create table cust<br />
(<br />
cust_id number(20) primary key,<br />
name varchar2(50),<br />
address varchar2(4000),<br />
bank_account number,<br />
state varchar2(2),<br />
country varchar2(50));<br />
I am populat<strong>in</strong>g the source with the follow<strong>in</strong>g statements. The sales<br />
table is with 1 Mil records and cust table is with 20K records.<br />
<strong>in</strong>sert /*+ append */ <strong>in</strong>to sales<br />
select
ownum,<br />
'Someth<strong>in</strong>g ||mod(rownum,100000) as product,<br />
mod(rownum,5) as channel_id,<br />
mod(rownum,1000) as cust_id ,<br />
5000 as amount_sold,<br />
to_date<br />
('01.' || lpad(to_char(mod(rownum,12)+1),2,'0') || '.2010' ,'dd.mm.yyyy')<br />
as time_id<br />
from dual connect by level
Install <strong>OGG</strong> on both sites and configure managers – ( see the <strong>OGG</strong><br />
<strong>in</strong>stall )<br />
Set CDC – start CDC <strong>extract</strong><br />
Perform an <strong>in</strong>itial dataload (<strong>OGG</strong> based is discussed <strong>in</strong> the article)<br />
Start data apply – start the replicat to apply changes captured by CDC<br />
Set NOHANDLECOLLISIONS on the target replicats<br />
Note. Special thanks to Chris Tijer<strong>in</strong>a who po<strong>in</strong>ted out that HANDLECOLLISIONS<br />
(HC) is required only if us<strong>in</strong>g an <strong>OGG</strong> for the <strong>in</strong>itial dataload. However, if<br />
you use SCN based exp(dp)/imp(dp) or rman to build a po<strong>in</strong>t <strong>in</strong> time copy on<br />
the target from the source DB than you can skip sett<strong>in</strong>g the HC. For <strong>Oracle</strong><br />
source and target database imp(dp)/expdp) pump or rman is much faster way to<br />
<strong>in</strong>itially move the data than <strong>OGG</strong> native methods. <strong>OGG</strong> native <strong>in</strong>itial data load<br />
methods are very valuable <strong>in</strong> heterogeneous data transfers. I will blog about<br />
this no HC topic later but the idea is as follows:<br />
Start the primary <strong>extract</strong> gett<strong>in</strong>g CDC from the source database;<br />
Make sure that there is no long runn<strong>in</strong>g transaction. Query until none<br />
or kill the transaction.<br />
Query the database for the SQL and write the SCN down<br />
Make sure that exp/imp or rman restore on the target system is done<br />
Use the follow<strong>in</strong>g <strong>OGG</strong> syntax to start the replicat on the target<br />
system: start replicat afterscn <br />
When replicats start they will parse through the trail files and will start<br />
apply<strong>in</strong>g the data with next SCN right after the SCN for the imp(dp)/exp(dp)<br />
or rman restore or duplicate.<br />
For CDC capture from <strong>Oracle</strong> source database you will need to perform the<br />
follow<strong>in</strong>g steps <strong>in</strong> an orderly manner.<br />
a. Add supplemental logg<strong>in</strong>g at database level<br />
SQL> alter database add supplemental log data;<br />
Database altered.<br />
SQL><br />
b. Add supplemental logg<strong>in</strong>g at table level<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 34> dblog<strong>in</strong> userid ddl_ogg<br />
Password:<br />
Successfully logged <strong>in</strong>to database.
GGSCI (racl<strong>in</strong>ux1.gj.com) 35><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 36> add trandata sales.sales<br />
Logg<strong>in</strong>g of supplemental redo data enabled for table SALES.SALES.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 37> add trandata sales.cust<br />
Logg<strong>in</strong>g of supplemental redo data enabled for table SALES.CUST.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 38><br />
c. Add CDC <strong>extract</strong> extsale, create parameter file for extsale <strong>extract</strong>, add<br />
ext trail ./dir.dat/zz for extsale <strong>extract</strong> and start <strong>extract</strong> extsale<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 53> view param extsale<br />
<strong>extract</strong> extsale<br />
tranlogoptions asmuser sys@ASM, asmpassword sys1<br />
--checkparams<br />
--NODYNAMICRESOLUTION<br />
userid ogg_<strong>extract</strong>, password ogg_<strong>extract</strong><br />
exttrail ./dirdat/zz<br />
table sales.sales;<br />
table sales.cust;<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 54><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 47> add <strong>extract</strong> extsale, tranlog, beg<strong>in</strong> now, threads 2<br />
EXTRACT added.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 48><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 54> add exttrail ./dirdat/zz, <strong>extract</strong> extsale megabytes 500<br />
EXTTRAIL added.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 55><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 58> start <strong>extract</strong> extsale<br />
Send<strong>in</strong>g START request to MANAGER ...<br />
EXTRACT EXTSALE start<strong>in</strong>g
GGSCI (racl<strong>in</strong>ux1.gj.com) 59> <strong>in</strong>fo all<br />
Program Status Group Lag at Chkpt Time S<strong>in</strong>ce Chkpt<br />
MANAGER RUNNING<br />
EXTRACT RUNNING EXTCDC 00:00:00 00:00:09<br />
EXTRACT RUNNING EXTSALE 00:00:00 00:09:16<br />
REPLICAT RUNNING REPCDC1 00:00:00 00:00:08<br />
REPLICAT RUNNING REPCDC2 00:00:00 00:00:06<br />
REPLICAT RUNNING REPCDC3 00:00:00 00:00:04<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 60><br />
d. Add a pump <strong>extract</strong> (pumpsale) and start it<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 63> view params pumpsale<br />
<strong>extract</strong> pumpsale<br />
passthru<br />
rmthost racl<strong>in</strong>ux1, mgrport 7809<br />
rmttrail ./dirdat/yy<br />
table sales.sales;<br />
table sales.cust;<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 64><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 64> add <strong>extract</strong> pumpsale, exttrailsource ./dirdat/zz<br />
EXTRACT added.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 65><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 66> add rmttrail ./dirdat/yy, <strong>extract</strong> pumpsale megabytes 500<br />
RMTTRAIL added.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 67><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 67> start <strong>extract</strong> pumpsale<br />
Send<strong>in</strong>g START request to MANAGER ...
EXTRACT PUMPSALE start<strong>in</strong>g<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 68> <strong>in</strong>fo all<br />
Program Status Group Lag at Chkpt Time S<strong>in</strong>ce Chkpt<br />
MANAGER RUNNING<br />
EXTRACT RUNNING EXTCDC 00:00:00 00:00:02<br />
EXTRACT RUNNING EXTSALE 00:00:00 00:00:02<br />
EXTRACT RUNNING PUMPSALE 00:00:00 00:02:48<br />
REPLICAT RUNNING REPCDC1 00:00:00 00:00:06<br />
REPLICAT RUNNING REPCDC2 00:00:00 00:00:05<br />
REPLICAT RUNNING REPCDC3 00:00:00 00:00:03<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 69><br />
e. Create replicats resale and repcust. DO NOT START the replicats before<br />
complet<strong>in</strong>g <strong>in</strong>itial data load<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 71> view params repsale<br />
replicat repsale<br />
userid ogg_replicat, password ogg_replicat<br />
handlecollisions<br />
assumetargetdefs<br />
discarfile ./dirrpt/sale.dsc, append<br />
map sales.sales, target sales.sales;<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 72><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 73> view params repcust<br />
replicat repcust<br />
userid ogg_replicat, password ogg_replicat<br />
handlecollisions<br />
assumetargetdefs<br />
discarfile ./dirrpt/sale.dsc, append
map sales.cust, target sales.cust;<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 74><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 74> add replicat repsale, exttrail ./dirdat/yy<br />
REPLICAT added.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 75> add replicat repcust, exttrail ./dirdat/yy<br />
REPLICAT added.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 76><br />
f. Initial dataload ( for details see here). Create a task <strong>extract</strong> and a<br />
replicat with the follow<strong>in</strong>g parameters and options for data collection<br />
(SOURCEISTABLE) and no checkpo<strong>in</strong>t operation (SPACIALRUN).Start only the<br />
<strong>extract</strong> but not the replicat and wait for the completion.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 8> view params <strong>in</strong>itexts<br />
<strong>extract</strong> <strong>in</strong>itexts<br />
SETENV (ORACLE_SID = "RACD1")<br />
---tranlogoptions asmuser sys@ASM, asmpassword sys1<br />
userid ogg_<strong>extract</strong>, password ogg_<strong>extract</strong><br />
rmthost racl<strong>in</strong>ux1, mgrport 7809<br />
rmttask replicat, group <strong>in</strong>itreps<br />
table sales.sales;<br />
table sales.cust;<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 9> add <strong>extract</strong> <strong>in</strong>itexts, sourceistable<br />
EXTRACT added.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 10><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 8> view params <strong>in</strong>itreps<br />
replicat <strong>in</strong>itreps
SETENV (ORACLE_SID = "RACDB1")<br />
userid ogg_replicat, password ogg_replicat<br />
assumetargetdefs<br />
discardfile ./dirrpt/sales_schema.dsc, purge<br />
map sales.sales, target sales.sales;<br />
map sales.cust, target sales.cust;<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 9><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 9> add replicat <strong>in</strong>itreps, specialrun<br />
REPLICAT added.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 10><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 10> start <strong>extract</strong> <strong>in</strong>itexts<br />
Send<strong>in</strong>g START request to MANAGER ...<br />
EXTRACT INITEXTS start<strong>in</strong>g<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 11><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 40> view report <strong>in</strong>itexts<br />
2012-07-08 04:50:42 INFO <strong>OGG</strong>-01017 Wildcard resolution set to IMMEDIATE because<br />
SOURCEISTABLE is used.<br />
***********************************************************************<br />
<strong>Oracle</strong> <strong>GoldenGate</strong> Capture for <strong>Oracle</strong><br />
Version <strong>11.2</strong>.1.0.1 <strong>OGG</strong>CORE_<strong>11.2</strong>.1.0.1_PLATFORMS_120423.0230_FBO<br />
L<strong>in</strong>ux, x64, 64bit (optimized), <strong>Oracle</strong> 11g on Apr 23 2012 08:42:16<br />
Copyright (C) 1995, 2012, <strong>Oracle</strong> and/or its affiliates. All rights reserved.<br />
Start<strong>in</strong>g at 2012-07-08 04:50:42<br />
***********************************************************************<br />
Operat<strong>in</strong>g System Version:
L<strong>in</strong>ux<br />
Version #1 SMP Thu Sep 3 04:15:13 EDT 2009, Release 2.6.18 -164.el5<br />
Node: racl<strong>in</strong>ux1.gj.com<br />
Mach<strong>in</strong>e: x86_64<br />
soft limit hard limit<br />
Address Space Size : unlimited unlimited<br />
Heap Size : unlimited unlimited<br />
File Size : unlimited unlimited<br />
CPU Time : unlimited unlimited<br />
Process id: 14866<br />
Description:<br />
***********************************************************************<br />
** Runn<strong>in</strong>g with the follow<strong>in</strong>g parameters **<br />
***********************************************************************<br />
2012-07-08 04:50:42 INFO <strong>OGG</strong>-03035 Operat<strong>in</strong>g system character set identified as UTF-8.<br />
Locale: en_US, LC_ALL:.<br />
<strong>extract</strong> <strong>in</strong>itexts<br />
SETENV (ORACLE_SID = "RACD1")<br />
Set environment variable (ORACLE_SID=RACD1)<br />
userid ogg_<strong>extract</strong>, password ***********<br />
2012-07-08 04:50:42 INFO <strong>OGG</strong>-03500 WARNING: NLS_LANG environment variable does not match<br />
database character set, or not set. U<br />
s<strong>in</strong>g database character set value of AL32UTF8.<br />
rmthost racl<strong>in</strong>ux1, mgrport 7809<br />
rmttask replicat, group <strong>in</strong>itreps<br />
table sales.sales;<br />
Us<strong>in</strong>g the follow<strong>in</strong>g key columns for source table SALES.SALES: PRODUCT_ID.
table sales.cust;<br />
Us<strong>in</strong>g the follow<strong>in</strong>g key columns for source table SALES.CUST: CUST_ID.<br />
2012-07-08 04:50:43 INFO <strong>OGG</strong>-01815 Virtual Memory Facilities for: COM<br />
anon alloc: mmap(MAP_ANON) anon free: munmap<br />
file alloc: mmap(MAP_SHARED) file free: munmap<br />
target directories:<br />
/u02/stage_ogg112_ora11/dirtmp.<br />
CACHEMGR virtual memory values (may have been adjusted)<br />
CACHESIZE: 64G<br />
CACHEPAGEOUTSIZE (normal): 8M<br />
PROCESS VM AVAIL FROM OS (m<strong>in</strong>): 128G<br />
CACHESIZEMAX (strict force to disk): 96G<br />
Database Version:<br />
<strong>Oracle</strong> Database 11g Enterprise Edition Release <strong>11.2</strong>.0.3.0 - 64bit Production<br />
PL/SQL Release <strong>11.2</strong>.0.3.0 - Production<br />
CORE <strong>11.2</strong>.0.3.0 Production<br />
TNS for L<strong>in</strong>ux: Version <strong>11.2</strong>.0.3.0 - Production<br />
NLSRTL Version <strong>11.2</strong>.0.3.0 - Production<br />
Database Language and Character Set:<br />
NLS_LANG = ".AL32UTF8"<br />
NLS_LANGUAGE = "AMERICAN"<br />
NLS_TERRITORY = "AMERICA"<br />
NLS_CHARACTERSET = "AL32UTF8"<br />
Process<strong>in</strong>g table SALES.SALES<br />
Process<strong>in</strong>g table SALES.CUST
***********************************************************************<br />
* ** Run Time Statistics ** *<br />
***********************************************************************<br />
Report at 2012-07-08 10:32:26 (activity s<strong>in</strong>ce 2012-07-08 04:50:42)<br />
Output to <strong>in</strong>itreps:<br />
From Table SALES.SALES:<br />
# <strong>in</strong>serts: 1000000<br />
# updates: 0<br />
# deletes: 0<br />
# discards: 0<br />
From Table SALES.CUST:<br />
# <strong>in</strong>serts: 20000<br />
# updates: 0<br />
# deletes: 0<br />
# discards: 0<br />
REDO Log Statistics<br />
Bytes parsed 0<br />
Bytes output 175694478<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 41><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 44> view report <strong>in</strong>itreps<br />
***********************************************************************<br />
<strong>Oracle</strong> <strong>GoldenGate</strong> Delivery for <strong>Oracle</strong>
Version <strong>11.2</strong>.1.0.1 <strong>OGG</strong>CORE_<strong>11.2</strong>.1.0.1_PLATFORMS_120423.0230_FBO<br />
L<strong>in</strong>ux, x64, 64bit (optimized), <strong>Oracle</strong> 11g on Apr 23 2012 08:48:07<br />
Copyright (C) 1995, 2012, <strong>Oracle</strong> and/or its affiliates. All rights reserved.<br />
Start<strong>in</strong>g at 2012-07-08 04:50:47<br />
***********************************************************************<br />
Operat<strong>in</strong>g System Version:<br />
L<strong>in</strong>ux<br />
Version #1 SMP Thu Sep 3 04:15:13 EDT 2009, Release 2.6.18-164.el5<br />
Node: racl<strong>in</strong>ux1.gj.com<br />
Mach<strong>in</strong>e: x86_64<br />
soft limit hard limit<br />
Address Space Size : unlimited unlimited<br />
Heap Size : unlimited unlimited<br />
File Size : unlimited unlimited<br />
CPU Time : unlimited unlimited<br />
Process id: 14879<br />
Description:<br />
***********************************************************************<br />
** Runn<strong>in</strong>g with the follow<strong>in</strong>g parameters **<br />
***********************************************************************<br />
2012-07-08 04:50:57 INFO <strong>OGG</strong>-03035 Operat<strong>in</strong>g system character set identified as UTF-8.<br />
Locale: en_US, LC_ALL:.<br />
replicat <strong>in</strong>itreps<br />
SETENV (ORACLE_SID = "RACDB1")
Set environment variable (ORACLE_SID=RACDB1)<br />
userid ogg_replicat, password ************<br />
2012-07-08 04:50:57 INFO <strong>OGG</strong>-03501 WARNING: NLS_LANG environment variable is <strong>in</strong>valid or<br />
not set. Us<strong>in</strong>g operat<strong>in</strong>g system character<br />
set value of AL32UTF8.<br />
assumetargetdefs<br />
discardfile ./dirrpt/sales_schema.dsc, purge<br />
map sales.sales, target sales.sales;<br />
map sales.cust, target sales.cust;<br />
2012-07-08 04:50:58 INFO <strong>OGG</strong>-01815 Virtual Memory Facilities for: COM<br />
anon alloc: mmap(MAP_ANON) anon free: munmap<br />
file alloc: mmap(MAP_SHARED) file free: munmap<br />
target directories:<br />
/u02/stage_ogg112_ora11/dirtmp.<br />
CACHEMGR virtual memory values (may have been adjusted)<br />
CACHESIZE: 2G<br />
CACHEPAGEOUTSIZE (normal): 8M<br />
PROCESS VM AVAIL FROM OS (m<strong>in</strong>): 4G<br />
CACHESIZEMAX (strict force to disk): 3.41G<br />
Database Version:<br />
<strong>Oracle</strong> Database 11g Enterprise Edition Release <strong>11.2</strong>.0.3.0 - 64bit Production<br />
PL/SQL Release <strong>11.2</strong>.0.3.0 - Production<br />
CORE <strong>11.2</strong>.0.3.0 Production<br />
TNS for L<strong>in</strong>ux: Version <strong>11.2</strong>.0.3.0 - Production<br />
NLSRTL Version <strong>11.2</strong>.0.3.0 - Production<br />
Database Language and Character Set:<br />
NLS_LANG = ".AL32UTF8"
NLS_LANGUAGE = "AMERICAN"<br />
NLS_TERRITORY = "AMERICA"<br />
NLS_CHARACTERSET = "AL32UTF8"<br />
***********************************************************************<br />
** Run Time Messages **<br />
***********************************************************************<br />
MAP resolved (entry sales.sales):<br />
map "SALES"."SALES", target sales.sales;<br />
Us<strong>in</strong>g follow<strong>in</strong>g columns <strong>in</strong> default map by name:<br />
PRODUCT_ID, PRODUCT, CHANNEL_ID, CUST_ID, AMOUNT_SOLD, TIME_ID<br />
Us<strong>in</strong>g the follow<strong>in</strong>g key columns for target table SALES.SALES: PRODUCT_ID.<br />
MAP resolved (entry sales.cust):<br />
map "SALES"."CUST", target sales.cust;<br />
Us<strong>in</strong>g follow<strong>in</strong>g columns <strong>in</strong> default map by name:<br />
CUST_ID, NAME, ADDRESS, BANK_ACCOUNT, STATE, COUNTRY<br />
Us<strong>in</strong>g the follow<strong>in</strong>g key columns for target table SALES.CUST: CUST_ID.<br />
***********************************************************************<br />
* ** Run Time Statistics ** *<br />
***********************************************************************<br />
Report at 2012-07-08 10:32:31 (activity s<strong>in</strong>ce 2012-07-08 04:51:01)
From Table SALES.SALES to SALES.SALES:<br />
# <strong>in</strong>serts: 1000000<br />
# updates: 0<br />
# deletes: 0<br />
# discards: 0<br />
From Table SALES.CUST to SALES.CUST:<br />
# <strong>in</strong>serts: 20000<br />
# updates: 0<br />
# deletes: 0<br />
# discards: 0<br />
CACHE OBJECT MANAGER statistics<br />
CACHE MANAGER VM USAGE<br />
vm current = 0 vm anon queues = 0<br />
vm anon <strong>in</strong> use = 0 vm file = 0<br />
vm used max = 0 ==> CACHE BALANCED<br />
CACHE CONFIGURATION<br />
cache size = 2G cache force pag<strong>in</strong>g = 3.41G<br />
buffer m<strong>in</strong> = 64K buffer highwater = 8M<br />
pageout eligible size = 8M<br />
================================================================================<br />
RUNTIME STATS FOR SUPERPOOL<br />
CACHE Transaction Stats<br />
trans active = 0 max concurrent = 0<br />
non-zero total = 0 trans total = 0<br />
CACHE File Cach<strong>in</strong>g
disk current = 0 disk total = 0<br />
disk cach<strong>in</strong>g = 0 file cached = 0<br />
file retrieves = 0<br />
CACHE MANAGEMENT<br />
buffer l<strong>in</strong>ks = 0 anon gets = 0<br />
forced unmaps = 0 cnnbl try = 0<br />
cached out = 0 force out = 0<br />
Allocation Request Distribution<br />
< 128B: 0<br />
128B: 0 0 | 512B: 0 0<br />
2K: 0 0 | 8K: 0 0<br />
32K: 0 0 | 128K: 0 0<br />
512K: 0 0 | 2M: 0 0<br />
8M: 0 0 | 32M: 0 0<br />
128M: 0 0 | 512M: 0 0<br />
2G: 0 0 | 8G: 0<br />
Cached Transaction Size Distribution<br />
0: 0<br />
< 4K: 0<br />
4K: 0 0 | 16K: 0 0<br />
64K: 0 0 | 256K: 0 0<br />
1M: 0 0 | 4M: 0 0<br />
16M: 0 0 | 64M: 0 0<br />
256M: 0 0 | 1G: 0 0<br />
4G: 0 0 | 16G: 0 0<br />
64G: 0 0 | 256G: 0 0<br />
1T: 0 0 | 4T: 0 0<br />
16T: 0 0 | 64T: 0 0<br />
256T: 0 0 |1024T: 0 0
================================================================================<br />
CUMULATIVE STATS FOR SUPERPOOL<br />
CACHE Transaction Stats<br />
trans active = 0 max concurrent = 0<br />
non-zero total = 0 trans total = 0<br />
CACHE File Cach<strong>in</strong>g<br />
disk current = 0 disk total = 0<br />
disk cach<strong>in</strong>g = 0 file cached = 0<br />
file retrieves = 0<br />
CACHE MANAGEMENT<br />
buffer l<strong>in</strong>ks = 0 anon gets = 0<br />
forced unmaps = 0 cnnbl try = 0<br />
cached out = 0 force out = 0<br />
Allocation Request Distribution<br />
< 128B: 0<br />
128B: 0 0 | 512B: 0 0<br />
2K: 0 0 | 8K: 0 0<br />
32K: 0 0 | 128K: 0 0<br />
512K: 0 0 | 2M: 0 0<br />
8M: 0 0 | 32M: 0 0<br />
128M: 0 0 | 512M: 0 0<br />
2G: 0 0 | 8G: 0<br />
Cached Transaction Size Distribution<br />
0: 0<br />
< 4K: 0<br />
4K: 0 0 | 16K: 0 0
64K: 0 0 | 256K: 0 0<br />
1M: 0 0 | 4M: 0 0<br />
16M: 0 0 | 64M: 0 0<br />
256M: 0 0 | 1G: 0 0<br />
4G: 0 0 | 16G: 0 0<br />
64G: 0 0 | 256G: 0 0<br />
1T: 0 0 | 4T: 0 0<br />
16T: 0 0 | 64T: 0 0<br />
256T: 0 0 |1024T: 0 0<br />
QUEUE Statistics:<br />
num queues = 15 default <strong>in</strong>dex = 0<br />
cur len = 0 max len = 0<br />
q vm current = 0 vm max = 0<br />
q hits = 0 q misses = 0<br />
queue size q hits curlen maxlen cannibalized<br />
0 64K 0 0 0 0<br />
1 128K 0 0 0 0<br />
2 256K 0 0 0 0<br />
3 512K 0 0 0 0<br />
4 1M 0 0 0 0<br />
5 2M 0 0 0 0<br />
6 4M 0 0 0 0<br />
7 8M 0 0 0 0<br />
8 16M 0 0 0 0<br />
9 32M 0 0 0 0<br />
10 64M 0 0 0 0<br />
11 128M 0 0 0 0<br />
12 256M 0 0 0 0<br />
13 512M 0 0 0 0
14 1G 0 0 0 0<br />
================================================================================<br />
RUNTIME STATS FOR CACHE POOL #0<br />
POOL INFO group: <strong>in</strong>itreps id: p14879_BLOB<br />
trans active = 0 trans concurrent (max) = 0<br />
trans total = 0 (0 )<br />
flag = 0x00000030<br />
last error = (0=)<br />
Allocation Request Distribution<br />
< 128B: 0<br />
128B: 0 0 | 512B: 0 0<br />
2K: 0 0 | 8K: 0 0<br />
32K: 0 0 | 128K: 0 0<br />
512K: 0 0 | 2M: 0 0<br />
8M: 0 0 | 32M: 0 0<br />
128M: 0 0 | 512M: 0 0<br />
2G: 0 0 | 8G: 0<br />
================================================================================<br />
CUMULATIVE STATS FOR CACHE POOL #0<br />
POOL INFO group: <strong>in</strong>itreps id: p14879_BLOB<br />
trans active = 0 trans concurrent (max) = 0<br />
trans total = 0 (0 )<br />
flag = 0x00000030<br />
last error = (0=)<br />
Allocation Request Distribution<br />
< 128B: 0<br />
128B: 0 0 | 512B: 0 0<br />
2K: 0 0 | 8K: 0 0
32K: 0 0 | 128K: 0 0<br />
512K: 0 0 | 2M: 0 0<br />
8M: 0 0 | 32M: 0 0<br />
128M: 0 0 | 512M: 0 0<br />
2G: 0 0 | 8G: 0<br />
QUEUE Statistics:<br />
num queues = 15 default <strong>in</strong>dex = 0<br />
cur len = 0 max len = 0<br />
q vm current = 0 vm max = 0<br />
q hits = 0 q misses = 0<br />
queue size q hits curlen maxlen cannibalized<br />
0 64K 0 0 0 0<br />
1 128K 0 0 0 0<br />
2 256K 0 0 0 0<br />
3 512K 0 0 0 0<br />
4 1M 0 0 0 0<br />
5 2M 0 0 0 0<br />
6 4M 0 0 0 0<br />
7 8M 0 0 0 0<br />
8 16M 0 0 0 0<br />
9 32M 0 0 0 0<br />
10 64M 0 0 0 0<br />
11 128M 0 0 0 0<br />
12 256M 0 0 0 0<br />
13 512M 0 0 0 0<br />
14 1G 0 0 0 0<br />
================================================================================<br />
RUNTIME STATS FOR CACHE POOL #0
POOL INFO group: <strong>in</strong>itreps id: p14879_BLOB<br />
trans active = 0 trans concurrent (max) = 0<br />
trans total = 0 (0 )<br />
flag = 0x00000030<br />
last error = (0=)<br />
Allocation Request Distribution<br />
< 128B: 0<br />
128B: 0 0 | 512B: 0 0<br />
2K: 0 0 | 8K: 0 0<br />
32K: 0 0 | 128K: 0 0<br />
512K: 0 0 | 2M: 0 0<br />
8M: 0 0 | 32M: 0 0<br />
128M: 0 0 | 512M: 0 0<br />
2G: 0 0 | 8G: 0<br />
================================================================================<br />
CUMULATIVE STATS FOR CACHE POOL #0<br />
POOL INFO group: <strong>in</strong>itreps id: p14879_BLOB<br />
trans active = 0 trans concurrent (max) = 0<br />
trans total = 0 (0 )<br />
flag = 0x00000030<br />
last error = (0=)<br />
Allocation Request Distribution<br />
< 128B: 0<br />
128B: 0 0 | 512B: 0 0<br />
2K: 0 0 | 8K: 0 0<br />
32K: 0 0 | 128K: 0 0<br />
512K: 0 0 | 2M: 0 0<br />
8M: 0 0 | 32M: 0 0<br />
128M: 0 0 | 512M: 0 0
2G: 0 0 | 8G: 0<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 45><br />
g. Start all replicates (repsale and repcust)<br />
ggsci>start replicat repsale<br />
ggsci>start replicat repcust<br />
h. Verify that <strong>OGG</strong> setup capture data from source and replicate it to the<br />
target tables.<br />
I am go<strong>in</strong>g to perform a transaction on each of the tables.<br />
Let’s change all 20000 records on cust source table and see how <strong>OGG</strong><br />
behaves and propagates the data from source database to the target<br />
database.<br />
update cust set country='USA';<br />
commit;<br />
Check the stats on <strong>extract</strong> extsale, <strong>extract</strong> pumpsale and replicat<br />
resale to verify the architecture and that transactions are processed<br />
as expected.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 68> stats <strong>extract</strong> extsale<br />
Send<strong>in</strong>g STATS request to EXTRACT EXTSALE ...<br />
Start of Statistics at 2012-07-08 14:43:23.<br />
Output to ./dirdat/zz:<br />
Extract<strong>in</strong>g from SALES.CUST to SALES.CUST:<br />
*** Total statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
*** Hourly statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00
*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
End of Statistics.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 69><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 72> stats <strong>extract</strong> pumpsale<br />
Send<strong>in</strong>g STATS request to EXTRACT PUMPSALE ...<br />
Start of Statistics at 2012-07-08 14:43:54.<br />
Output to ./dirdat/yy:<br />
Extract<strong>in</strong>g from SALES.CUST to SALES.CUST:<br />
*** Total statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
*** Hourly statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
End of Statistics.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 73><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 75> stats replicat repcust<br />
Send<strong>in</strong>g STATS request to REPLICAT REPCUST ...<br />
Start of Statistics at 2012-07-08 14:46:11.
Replicat<strong>in</strong>g from SALES.CUST to SALES.CUST:<br />
*** Total statistics s<strong>in</strong>ce 2012-07-08 14:43:32 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:43:32 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
*** Hourly statistics s<strong>in</strong>ce 2012-07-08 14:43:32 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:43:32 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
End of Statistics.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 76><br />
Let change some (999K) records on sales table on the source database<br />
and see how <strong>OGG</strong> behaves and propagates the data.<br />
SQL> update sales set amount_sold=20000 where cust_id between 1 and 2000;<br />
999000 rows updated.<br />
SQL><br />
SQL> commit;<br />
Commit complete.<br />
SQL><br />
Check the stats on the <strong>in</strong>volved <strong>extract</strong> and replicat groups and verify<br />
that the changes are processed by the <strong>extract</strong> and replicat.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 73> stats <strong>extract</strong> extsale<br />
Send<strong>in</strong>g STATS request to EXTRACT EXTSALE ...<br />
Start of Statistics at 2012-07-08 15:23:39.<br />
Output to ./dirdat/zz:<br />
Extract<strong>in</strong>g from SALES.CUST to SALES.CUST:<br />
*** Total statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />
Total <strong>in</strong>serts 0.00
Total updates 40000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 40000.00<br />
*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 40000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 40000.00<br />
*** Hourly statistics s<strong>in</strong>ce 2012-07-08 15:00:00 ***<br />
No database operations have been performed.<br />
*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 40000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 40000.00<br />
Extract<strong>in</strong>g from SALES.SALES to SALES.SALES:<br />
*** Total statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 999000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 999000.00<br />
*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 999000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 999000.00<br />
*** Hourly statistics s<strong>in</strong>ce 2012-07-08 15:00:00 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 999000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 999000.00<br />
*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:22:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 999000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 999000.00<br />
End of Statistics.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 74><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 78> stats <strong>extract</strong> pumpsale<br />
Send<strong>in</strong>g STATS request to EXTRACT PUMPSALE ...
Start of Statistics at 2012-07-08 15:24:31.<br />
Output to ./dirdat/yy:<br />
Extract<strong>in</strong>g from SALES.CUST to SALES.CUST:<br />
*** Total statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
*** Hourly statistics s<strong>in</strong>ce 2012-07-08 15:00:00 ***<br />
No database operations have been performed.<br />
*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 20000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 20000.00<br />
Extract<strong>in</strong>g from SALES.SALES to SALES.SALES:<br />
*** Total statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 999000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 999000.00<br />
*** Daily statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 999000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 999000.00<br />
*** Hourly statistics s<strong>in</strong>ce 2012-07-08 15:00:00 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 999000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 999000.00<br />
*** Latest statistics s<strong>in</strong>ce 2012-07-08 14:43:31 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 999000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 999000.00<br />
End of Statistics.
GGSCI (racl<strong>in</strong>ux1.gj.com) 79><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 84> stats replicat repsale<br />
Send<strong>in</strong>g STATS request to REPLICAT REPSALE ...<br />
Start of Statistics at 2012-07-08 15:44:30.<br />
Replicat<strong>in</strong>g from SALES.SALES to SALES.SALES:<br />
*** Total statistics s<strong>in</strong>ce 2012-07-08 15:01:52 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 999000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 999000.00<br />
*** Daily statistics s<strong>in</strong>ce 2012-07-08 15:01:52 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 999000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 999000.00<br />
*** Hourly statistics s<strong>in</strong>ce 2012-07-08 15:01:52 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 999000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 999000.00<br />
*** Latest statistics s<strong>in</strong>ce 2012-07-08 15:01:52 ***<br />
Total <strong>in</strong>serts 0.00<br />
Total updates 999000.00<br />
Total deletes 0.00<br />
Total discards 0.00<br />
Total operations 999000.00<br />
End of Statistics.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 85><br />
i. Send NOHANDLECOLLISIONS and modify accord<strong>in</strong>gly the respective parameter<br />
files.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 85> send replicat repsale, nohandlecollisions<br />
Send<strong>in</strong>g NOHANDLECOLLISIONS request to REPLICAT REPSALE ...<br />
REPSALE No tables found match<strong>in</strong>g * to set NOHANDLECOLLISIONS<br />
.<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 86> send replicat repcust, nohandlecollisions<br />
Send<strong>in</strong>g NOHANDLECOLLISIONS request to REPLICAT REPCUST ...<br />
REPCUST No tables found match<strong>in</strong>g * to set NOHANDLECOLLISIONS<br />
.
GGSCI (racl<strong>in</strong>ux1.gj.com) 87><br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 91> view param repsale<br />
replicat repsale<br />
SETENV (ORACLE_SID = "RACDB1")<br />
userid ogg_replicat, password ogg_replicat<br />
--handlecollisions<br />
assumetargetdefs<br />
discardfile ./dirrpt/sale.dsc, append<br />
map sales.sales, target sales.sales;<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 92> view param repcust<br />
replicat repcust<br />
SETENV (ORACLE_SID = "RACDB1")<br />
userid ogg_replicat, password ogg_replicat<br />
--handlecollisions<br />
assumetargetdefs<br />
discardfile ./dirrpt/sale.dsc, append<br />
map sales.cust, target sales.cust;<br />
GGSCI (racl<strong>in</strong>ux1.gj.com) 93><br />
j. This concludes test<strong>in</strong>g and verify<strong>in</strong>g that pump <strong>extract</strong> operates<br />
properly<br />
Summary:<br />
In the article you had a look at the <strong>OGG</strong> feature data pump <strong>extract</strong>. You<br />
looked at an example of sett<strong>in</strong>g a data pump <strong>extract</strong> and methodology for<br />
verify<strong>in</strong>g that configuration works properly.