Showing posts with label replication. Show all posts
Showing posts with label replication. Show all posts

Tuesday, January 14, 2014

Reference: Clean up SharePlex Queues

We use SharePlex replication as our DR solution for couple of applicatins. Last Saturday night, we did a DR test for one of the very important applications, but we could not make replication working from DR site to other target. As we ran out of time in the window, we just brought up application back to the production site without fixing issues. Today I involved vendor support to test DR configuration (with a dummy configuration) again to make sure that if real DR situation happens replication will work. I have learned that the key is that we should clean up orphan or corrupted queues before activating a configuration in DR. ( note: when we issue deactivate config , suppose all associated queues with this particular conifg will be gone, if not, those left queues need to be cleaned up).


Below are the steps to clean up SharePlex queues for future reference:


1. shutdown or shutdown force at source and target
2. qview -i
3. qview> qsetup
4. qview> qstatus
5. qview> deleteq p   -- for post
   qview> deleteq x   -- for export
6. On target:  truncate splex.shareplex_trans;

Thursday, October 10, 2013

Can Oracle GoldenGate DDL replication support interval partition?

We have a fast-growing table that requires keeping only two months' worth of data. We plan to take advantage of the interval partition feature in Oracle 11g, in which Oracle automatically creates an interval partition as data for that partition is inserted. Therefore, we will not worry about data load failure situation due to the possibility of forgetting to add new partitions by DBA.

In our environment, this table will also be in Oracle GoldenGate replication configuration.   So I try to do tests to confirm whether the GG DDL replication supports the following two operations:

1. Create interval partition

2. Drop partition

I used a test GG environment, in which  a two-way GG replication with DDL replication enabled  is set up between two databases: WESTDB and EASTDB; and the tables are in different schemas called west and east respectively.

The following are the testing steps:

1. create a table named interval_tab in the source db
 denis@WESTDB>> CREATE TABLE west.interval_tab
   2  ( prod_id        NUMBER(6)
   3   , cust_id        NUMBER
   4   , time_id        DATE
   5  )
   6  PARTITION BY RANGE (time_id)
   7  INTERVAL(NUMTOYMINTERVAL(1, 'MONTH'))
   8   ( PARTITION p0 VALUES LESS THAN (TO_DATE('1-1-2013', 'MM-DD-YYYY')),
   9     PARTITION p1 VALUES LESS THAN (TO_DATE('2-1-2013', 'MM-DD-YYYY')),
  10     PARTITION p2 VALUES LESS THAN (TO_DATE('3-1-2013', 'MM-DD-YYYY')),
  11     PARTITION p3 VALUES LESS THAN (TO_DATE('4-1-2013', 'MM-DD-YYYY')) );

 Table created.

Verify it is replicated in target:
 denis@EASTDB>> desc east.interval_tab;
  Name                                                                          Null?    Type
  ----------------------------------------------------------------------------- -------- ---------------------
  PROD_ID                                                                                NUMBER(6)
  CUST_ID                                                                                NUMBER
  TIME_ID                                                                                DATE

2. create indexes and contraint At source:
 denis@WESTDB>> create unique index west.interval_tab_pk on west.interval_tab(prod_id, time_id) local;

 Index created.

 denis@WESTDB>> create index west.interval_tab_ix1 on west.interval_tab(cust_id) local;

 Index created.

 denis@WESTDB>> alter table west.interval_tab add constraint interval_tab_pk primary key (prod_id, time_id) using index;

 Table altered.

Verify index and constraints creation are replicated at target:
 denis@EASTDB>> @tabix
 Enter value for tabowner: east
 Enter value for tabname: interval_tab

 TABLE_NAME           INDEX_NAME           COLUMN_NAME             COL_POS UNIQUENES
 -------------------- -------------------- -------------------- ---------- ---------
 INTERVAL_TAB         INTERVAL_TAB_IX1     CUST_ID                       1 NONUNIQUE
 INTERVAL_TAB         INTERVAL_TAB_PK      PROD_ID                       1 UNIQUE
 INTERVAL_TAB                              TIME_ID                       2 UNIQUE


 denis@EASTDB>> select owner,table_name,  constraint_name from dba_constraints where table_name='INTERVAL_TAB';

 OWNER           TABLE_NAME           CONSTRAINT_NAME
 --------------- -------------------- --------------------
 EAST            INTERVAL_TAB         INTERVAL_TAB_PK

3. Insert data at source that will cause new interval partition created automatically

Before the insert, check the current partitions:
 denis@WESTDB>> select table_owner, table_name, partition_name, high_value from dba_tab_partitions where table_name='INTERVAL_TAB';

 TABLE_OWNER      TABLE_NAME           PARTITION_NAME       HIGH_VALUE
 ---------------- -------------------- -------------------- ---------------------------------------------
 WEST             INTERVAL_TAB         P0                   TO_DATE(' 2013-01-01 00:00:00', 'SYYYY-MM-DD
                  HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 WEST             INTERVAL_TAB         P1                   TO_DATE(' 2013-02-01 00:00:00', 'SYYYY-MM-DD
                  HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 WEST             INTERVAL_TAB         P2                   TO_DATE(' 2013-03-01 00:00:00', 'SYYYY-MM-DD
                  HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 WEST             INTERVAL_TAB         P3                   TO_DATE(' 2013-04-01 00:00:00', 'SYYYY-MM-DD
                                                                         HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')
Perform the following insert at source and check the partitions:

insert into interval_tab values (1004,1, sysdate);
 denis@WESTDB>> insert into west.interval_tab values (1004,1, sysdate);

 1 row created.

 denis@WESTDB>> commit;

 Commit complete.

 denis@WESTDB>> select table_owner, table_name, partition_name, high_value from dba_tab_partitions where table_name='INTERVAL_TAB';

 TABLE_OWNER                    TABLE_NAME           PARTITION_NAME       HIGH_VALUE
 ------------------------------ -------------------- -------------------- ---------------------------------------------
 WEST                           INTERVAL_TAB         P0                   TO_DATE(' 2013-01-01 00:00:00', 'SYYYY-MM-DD
           HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 WEST                           INTERVAL_TAB         P1                   TO_DATE(' 2013-02-01 00:00:00', 'SYYYY-MM-DD
           HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 WEST                           INTERVAL_TAB         P2                   TO_DATE(' 2013-03-01 00:00:00', 'SYYYY-MM-DD
           HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 WEST                           INTERVAL_TAB         P3                   TO_DATE(' 2013-04-01 00:00:00', 'SYYYY-MM-DD
           HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 WEST                           INTERVAL_TAB         SYS_P81              TO_DATE(' 2013-11-01 00:00:00', 'SYYYY-MM-DD
           HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')


 5 rows selected.

4. Check partitions at target database
 denis@EASTDB>> select table_owner, table_name, partition_name, high_value from dba_tab_partitions where table_name='INTERVAL_TAB';

 TABLE_OWNER            TABLE_NAME           PARTITION_NAME       HIGH_VALUE
 ---------------------- -------------------- -------------------- ---------------------------------------------
 EAST                   INTERVAL_TAB         P0                   TO_DATE(' 2013-01-01 00:00:00', 'SYYYY-MM-DD
          HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 EAST                   INTERVAL_TAB         P1                   TO_DATE(' 2013-02-01 00:00:00', 'SYYYY-MM-DD
          HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 EAST                   INTERVAL_TAB         P2                   TO_DATE(' 2013-03-01 00:00:00', 'SYYYY-MM-DD
          HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 EAST                   INTERVAL_TAB         P3                   TO_DATE(' 2013-04-01 00:00:00', 'SYYYY-MM-DD
          HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 EAST                   INTERVAL_TAB         SYS_P123             TO_DATE(' 2013-11-01 00:00:00', 'SYYYY-MM-DD
           HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')


 5 rows selected.


 denis@EASTDB>> select index_owner,index_name,partition_name from dba_ind_partitions where index_name='INTERVAL_TAB_IX1';

 INDEX_OWNER                    INDEX_NAME           PARTITION_NAME
 ------------------------------ -------------------- --------------------
 EAST                           INTERVAL_TAB_IX1     P0
 EAST                                                P1
 EAST                                                P2
 EAST                                                P3
 EAST                                                SYS_P123

 5 rows selected.

5. Drop a partition at source alter table west.interval_table drop partition P0;
 denis@WESTDB>> alter table west.interval_tab drop partition P0;

 Table altered.


 Verified, it is replicable


 denis@EASTDB>> /

 TABLE_OWNER                    TABLE_NAME           PARTITION_NAME       HIGH_VALUE
 ------------------------------ -------------------- -------------------- ---------------------------------------------
 EAST                           INTERVAL_TAB         P1                   TO_DATE(' 2013-02-01 00:00:00', 'SYYYY-MM-DD
           HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 EAST                           INTERVAL_TAB         P2                   TO_DATE(' 2013-03-01 00:00:00', 'SYYYY-MM-DD
           HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 EAST                           INTERVAL_TAB         P3                   TO_DATE(' 2013-04-01 00:00:00', 'SYYYY-MM-DD
           HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')

 EAST                           INTERVAL_TAB         SYS_P123             TO_DATE(' 2013-11-01 00:00:00', 'SYYYY-MM-DD
           HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')


 4 rows selected.


In conclusion, the tests confirmed that GG DDL replication support the interval partition automatic creation and drop partition operations. It is worth noting that when we use DBMS_METADATA package to get the definitions of interval partitioned table and its associated local indexes, we won't see the interval partitions that are created automatically. To verify the existance of the partitions, we shall use dba_tab_partitions and dba_ind_partitions views.

P.S.

Note from Oracle docs:

You cannot explicitly add a partition to an interval-partitioned table unless you first lock the partition, which triggers the creation of the partition. The database automatically creates a partition for an interval when data for that interval is inserted. In general, you only need to explicitly create interval partitions for a partition exchange load scenario.

update Apr 9, 2014 -

I may have misunderstood the relationship of DDL replication and interval partition creation.  DDL replication may have no relationship with interval partition creation at all. Even without DDL replication enabled. Interval partition will be created due to the data insertion DML statement. 

Monday, June 17, 2013

GoldenGate: Steps of Resynchronizing a Table

In the target, Replicat process was abended due to the following error:

2013-06-17 10:17:30  ERROR   OGG-01163  Oracle GoldenGate Delivery for Oracle, rtdnfrd.prm:  Bad column length (3) specified for column TAX_AUTHORITY_TYPE in table WAX_TDN.V_TAX_DETAILS, maximum allowable length is 1.

I checked TAX_AUTHORITY_TYPE colume, it is char(1) in both source and target databases, so not sure how this error comes up. I have to comment it out in the Replicat parameter file in order to re-start the process. As a result, the table is out-of-sync. This actually gives me a chance to pratice steps of resynchronizing a table in GoldenGate for the first time.

I took the following steps:

1. Comment out the table in the Replicat parameter file
Already did.


2. Stop Replicat and start it again so that it continues for unaffected tables
Already did.

3. Record the time stamp on the source system
2013-06-17 13:49

4 Start a copy of the source data for the affected tables
 
Note: before making the copy, try to resolve any long-running transactions

5. Import the copy to the target table

6. Create a new Replicat group for the out-of-sync table, using Begin to start at the source time stamp that you recorded earlier and using the existing train for ExtTrain

Add Replicat  rvtd, ExtTrail ./dirdat/rb, Begin 2013-06-17 13:49:00

7. Create the new parameter file so that it includes 
  HandleCollisions

replicat rvtd
SETENV(ORACLE_HOME="/apps/opt/oracle/product/11.2.0/db_1")
SETENV(ORACLE_SID = "tdnprdfd1")
SETENV(NLS_LANG=AMERICAN_AMERICA.AL32UTF8)
-- DBOPTIONS SUPPRESSTRIGGERS
userid
gg_owner@tdnprdfd1, password AADAAAAAAAAAAAHAJIKGFGKGRJTBJBCIZGUERJNHBFPBLCAEOBUADFWASJMJCDWEICWGBEGHOIRESCPA, encryptkey securekey1
discardfile ./dirrpt/RTDNFRD.dsc, Append, megabytes 1
handlecollisions
assumetargetdefs
MAP WAX_TDN.V_TAX_DETAILS,             TARGET WAX_TDN.V_TAX_DETAILS;


8. Start the new Replicat

9. View the new Replicat's lag until it shows "At EOF, no more records to process"
GGSCI> Send replicat rvtd, GetLag

10. Turn off HandleCollisions in the new replicate with
GGSCI> send replicat rvtd NoHandleCollisions

11. Edit the parameter file to comment out or remove HandleCollisions if you ever bounce the process later
(The next steps merge the table back with the others so that only one Replicat group is needed

12. Stop Extract at source

13. View both Replicats's lag until you see "EOF" again

GGSCI> Send replicat rtdnfrd, GetLag
GGSCI> Send replicat rvtd, GetLag

14 stop both Replicats

15. Uncomment the resynced table in the original Replicat parameter file

16. Start Extract

17. Start the original Replicat:
GGSC> start rtdnfrd

18. Delete the new Replicat that you created:
GGSCI> delete replicat rvtd

All done!

GoldenGate Replicat Process Abending due to Tablespace Full and Discard File Exceeding Max Bytes

We have a cron job set up to monitor errors in the GoldenGate ggserr.log. This monrining, in a target database, we recieved:

< 2013-06-17 02:58:17  ERROR   OGG-01172  Oracle GoldenGate Delivery for Oracle, rvasip.prm:  Discard file (./dirrpt/RVASIP.dsc) exceeded max bytes (1000000).
< 2013-06-17 02:58:17  ERROR   OGG-01668  Oracle GoldenGate Delivery for Oracle, rvasip.prm:  PROCESS ABENDING.


It appeared that Replicat process abended due to  Discard file exceeded max bytes.
Discard file is used by GoldenGate to log records it cannot proccess. The maximum size of the discard file can be specified by MAXBYTES or MEGABYSTS options,
the defaults are 1000000 or 1MB.  If the specified size is exceeded, the process will abend.

Further troubleshooting showed the reason for the discard file filled up was due to a tablespace filled up in this case. In the RVASIP.dsc files we can found:

OCI Error ORA-01653: unable to extend table PPOWNER.VZ_JNR_FEED_TRX_LOG by 8192 in tablespace PPOWNER_DATA_1 (status = 1653). INSERT INTO "PPOWNER"."VZ_JNR_FEED_TRX_LOG" ("ID","JOURNAL_ID"
,"TRX_EXT_ID","BILLED","DB_MODIFICATION_DATE","DB_CREATION_DATE","SUB_TRX_ID","TRX_ID","CHG_ATTR","ACCESS_TYPE") VALUES (:a0,:a1,:a2,:a3,:a4,:a5,:a6,:a7,:a8,:a9)
Aborting transaction on ./dirdat/rt beginning at seqno 21275 rba 3795007
                         error at seqno 21275 rba 5169654


To fix the problem, I renamed RAVSIP.dsc, changed the max bytes of discard file to be 10MB in the parameter file:

discardfile ./dirrpt/RVASIP.dsc, Append,megabytes 10

Then I stopped and started the Replicat process. I have verified that those discarded dmls recorded in the discard file have been applied after Replicat process re-started. no manual intervene is ndeed.

Tuesday, February 22, 2011

Tip - List all of the tables in the Shareplex replication

Some time ago a co-worker showed me how to do this with a query. Today I have a need to do it, but I forget. I have to get the table list from the configuration file. Now I blog this as a reference for future:

--- for port 2200 ---
select owner, object_name, objid
from splex.shareplex_objmap s, dba_objects o
where s.objid = o.object_id 
order by owner, object_name;

--- for port 2201 ----
select owner, object_name, objid
from splex_cpf.shareplex_objmap s, dba_objects o
where s.objid = o.object_id 
order by owner, object_name;

Thursday, March 26, 2009

Adjust Oracle sequence to use odd and even number respectively on a Production and DR database

Our production databases are replicated to DR databases through SharePlex replcation. Recently, We have a need to enable two-way replication. i.e. in addition to replication from production to DR, we also need DR to prodcution. I was thus assigned a task to adjust the sequences in production to use odd number and sequences in DR to use even number.

The sequneces in production and DR are out-of-sync currently, as we don't replicate sequence from production to DR. That means if on production a sequence's last number is 1000, whereas on DR, the number could be 1.

Testing for a while, I found that when generating the DDL for a sequence through DBMS_METADATA package, the number following the 'start with' clause is the last number of DBA_SEQUENCES view.

For example:



SQL> select sequence_name, last_number from dba_sequences
where sequence_owner=user;

SEQUENCE_NAME LAST_NUMBER
------------------------------ -----------
TEST2_SEQ 107
TEST3_SEQ 107
TEST_SEQ 175


SQL> ;
1* SELECT DBMS_METADATA.GET_DDL(upper('&OBJTYPE'), upper('&OBJNAME') , upper('&OWNER')) ddl_string from dual
SQL> /
Enter value for objtype: sequence
Enter value for objname: TEST_SEQ
Enter value for owner: abc

CREATE SEQUENCE "ABC"."TEST_SEQ" MINVALUE 1 MAXVALUE 1.00000000000000E+27
INCREMENT BY 98 START WITH 175 CACHE 20 NO
ORDER NOCYCLE ;



  

I thus developed the following plan:

1. On Prod, run script seq_odd.sql to change seq number to odd and increment by to 2
2. On prod, run script seq_ddl_gen.sql to generate sequence ddl script: seq_ddl.sql
3. SCP seq_ddl.sql to DR server
4. On DR, run script seq_drop_gen.sql and then seq_drop.sql to drop sequence
5. On DR, run script seq_ddl.sql generated in step 2 to re-create all sequences
6. On DR, run script seq_incr1_gen.sql, seq_incr1.sql to modify sequence increment by to 1;
7. On DR, run script seq_nextval_gen.sql, seq_nextval.sql to make the currval of sequences to be even number,
8. on DR, run script seq_incr2_gen.sql, seq_incr2.sql to modify sequence increment by to 2;

The above scripts are simple execept for seq_odd.sql, I wrote PL/SQL code for this:



---- seq_odd.sql -----
declare
seqown varchar2(30);
seqname varchar2(30);
sqlstmt varchar2(1000);
cval number;
incr number;
cursor seq_cur is
select sequence_owner, sequence_name
from dba_sequences
where sequence_owner is [some_condition];
begin
open seq_cur;
loop
fetch seq_cur into seqown, seqname;
exit when seq_cur%notfound;
sqlstmt := 'select ' seqown '.' seqname '.nextval from dual';
execute immediate sqlstmt into cval;

if ( mod(cval, 2) = 0 )
then
-- ensure the current val is odd number
-- first change increment by 1
sqlstmt := 'alter sequence ' seqown '.' seqname ' increment by 1';
execute immediate sqlstmt;

sqlstmt := 'select ' seqown '.' seqname '.nextval from dual';
execute immediate sqlstmt into cval;
sqlstmt := 'alter sequence ' seqown '.' seqname ' increment by 2';
execute immediate sqlstmt;
else
-- already an odd number
sqlstmt := 'alter sequence ' seqown '.' seqname ' increment by 2';
execute immediate sqlstmt;
end if;
end loop;
close seq_cur;

-- follwing code is to verify ----
---- end of seq_odd.sql --------

Saturday, February 14, 2009

Set up read-only materialized view replication

I was assigned a task to evaluate the feasibility of seting up materialized view replication for another team. Currently this team obtains the data from one of our reporting databases through export and import. So I started to understand materialized view replication. As the first step, I build a testing materialized view replication environment on my pc. Below are some notes as the result of this effort.

 

=========================================
== Set up read-only MVIEW replication ==
=========================================

Environment:
Master site (DBT920) : 9.2.0.4
Materialized view site (DBT10G) : 10.2.0.2

Reference:
1. Metalink Note 256235.1 Scripts to create Trusted / Untrusted ReadOnly MVIEW Replication Sites
2. http://www.hpfuchs.com/2008/02/06/materialized-view-replication/

Overview
~~~~~~~~~~
1. create users at master site
2. create users at mview site
3. create database link at mview site
4. create mview logs at master site
5. create mview at mview site
6. create mview groups
7. Some basic operations


Detailed Steps
~~~~~~~~~~~~~~~

1. create users at master site

-- run as system

create user mviewproxy identified by mviewproxy;

grant create session to mviewproxy;
grant create any table to mviewproxy;
grant comment any table to mviewproxy;
grant select any table to mviewproxy;

BEGIN
dbms_repcat_admin.register_user_repgroup(
username => 'mviewproxy',
privilege_type => 'proxy_mviewadmin',
list_of_gnames => NULL);
END;
/

Note: Not sure if this register_user_repgroup is necessary


2. create users at mview site

-- run as system

create user mvowner identified by oracle
default tablespace users temporary tablespace temp;
grant connect, resource to mvowner;
grant create materialized view to mvowner;
grant create database link to mvowner;

-- run as system

CREATE USER mviewadmin IDENTIFIED BY mviewadmin;
ALTER USER mviewadmin DEFAULT TABLESPACE users;
ALTER USER mviewadmin TEMPORARY TABLESPACE temp;

EXECUTE dbms_repcat_admin.grant_admin_any_schema('mviewadmin');
GRANT comment any table TO mviewadmin;
GRANT lock any table TO mviewadmin;

GRANT create any materialized view TO mviewadmin;
GRANT alter any materialized view TO mviewadmin;


3. create db link at mview site

-- run as system
CREATE PUBLIC DATABASE LINK DBT92.US.ORACLE.COM USING 'DBT92';

Note: not sure why this is necessary.

-- run as mvowner
create database link DBT92.US.ORACLE.COM connect to mviewproxy identified by mviewproxy
using 'DBT92';

4. create mview logs at master site

-- run as mviewproxy
create materialized view log on mstowner.big_table tablespace mviewlog;


Note: 1. mview logs residing in their own tablespace, i.e.
create tablespace mviewlog datafile '/u03/oracle/oradata/DBT92/mviewlog01.dbf' size 50M;
2. mviewproxy does not have the privs to alter/drop materialized view log

5. create mview at mview site
login as mvowner

-- run as mvowner
CREATE MATERIALIZED VIEW mvowner.t_mv REFRESH FAST AS SELECT * FROM mstowner.t@DBT92.US.ORACLE.COM;
CREATE MATERIALIZED VIEW mvowner.big_table_mv REFRESH FAST AS SELECT * FROM mstowner.big_table@DBT92.US.ORACLE.COM;

Note:
Oracle will create three objects when executing the following command:
CREATE MATERIALIZED VIEW mvowner.big_table_mv REFRESH FAST AS SELECT * FROM mstowner.big_table@DBT92.US.ORACLE.COM;

OBJECT_NAME OBJECT_TYPE
------------------------------ -------------------
BIG_TABLE_MV TABLE
BIG_TABLE_PK INDEX
BIG_TABLE_MV MATERIALIZED VIEW



6. create mview groups at mview site

-- create the refresh group for the mview to ensure transactional
-- consistency when refreshing nore than one mview in the group.

-- run as mviewadmin on the mview side

begin
dbms_refresh.make(
name => 'RG_BIG_TABLE',
list => 'mvowner.big_table_mv',
next_date => sysdate,
interval => 'sysdate + 1/24',
implicit_destroy => true,
lax => true);
end;
/


7. Some basic operations

(1) Refresh the complete group
execute dbms_refresh.refresh('RG_BIG_TABLE');
(2) Refresh a single snapshot
execute dbms_snapshot.refresh('mvowner.big_table_mv');


(3) Check materialized veiw refresh status


set linesize 120
set pagesize 100
col owner format a20
col table_name format a20
col name format a20
col master_owner format a20
col master_link format a20
col next format a20

select owner
,name
-- ,table_name
-- ,master_owner
-- ,master
-- ,master_link
,to_char(last_refresh, 'YYYY-MON-DD HH24:MI:SS') lst_rfrsh
,next
,status
from dba_snapshots
/


(4) views:
dba_snapshots
dba_mviews

Sunday, December 14, 2008

Testing Update Conflict Resolution in a Streams Environment

Before starting this experiment, a two-way replication has already been set up for a table called denis.strm_tab3 between databases TEST10G and TEST02DB.

The structure of the table is as follows:

Name              Null?    Type
----------------- -------- ---------------
ID                NOT NULL NUMBER
FIRST_NAME                 VARCHAR2(20)
LAST_NAME                  VARCHAR2(30)


To test the conflict resolution, I will update the same row of the table with different values at same time on the two databases. This will be archieved through scheduler jobs. First,the follwoing procedure is created:

create or replace procedure update_strm_tab3(l_id in  number, 
l_first_name in varchar2,
l_last_name in varchar2 )
as
begin
update strm_tab3 set first_name = l_first_name, last_name=l_last_name
where id= l_id;
commit;
end;
/



Second, the folliwng scheduler jobs that have same start date will be submitted:

At TEST10G:

begin
dbms_scheduler.create_job (
job_name => 'update_strm_tab3_job',
job_type => 'PLSQL_BLOCK',
job_action => 'begin update_strm_tab3(6,''fn6d_10g'', ''ln6d_10g''); end;',
start_date => '14-DEC-2008 06:35:00 PM',
enabled => true,
comments => 'Update strm_tab3 ');
end;
/

At TEST02DB

begin
dbms_scheduler.create_job (
job_name => 'update_strm_tab3_job',
job_type => 'PLSQL_BLOCK',
job_action => 'begin update_strm_tab3(6,''fn6d_02db'', ''ln6d_02db''); end;',
start_date => '14-DEC-2008 06:35:00 PM',
enabled => true,
comments => 'Update strm_tab3 ');
end;
/

Note: to check scheduler job, isssue the following statment:

col owner format a15
col next_run_date format a20
select owner, job_name, state, last_run_duration,
next_run_date
from dba_scheduler_jobs
where owner='DENIS';

I have tested the following cases:

Case 1 - No conflict resolution method is set up

Results:

At TEST10G
ID FIRST_NAME           LAST_NAME
---------- -------------------- ------------------------------
6 fn6d_10g             ln6d_10g


At TEST02G
ID FIRST_NAME           LAST_NAME
---------- -------------------- ------------------------------
6 fn6d_02db            ln6d_02db


The apply processes status became 'ABORT' on both database, for example
we can get something like:


++  APPLY PROCESS INFORMATION ++
APPLY_NAME                    : STRMADMIN_APPLY_2
MAX_APPLIED_MESSAGE_NUMBER    :
STATUS                        : ABORTED
STATUS_CHANGE_TIME            : 14-dec-2008 15:10:09
ERROR_NUMBER                  : 26714
ERROR_MESSAGE                 : ORA-26714: User error encountered while applying




Case 2 - Conflict resolution - OVERWRITE at TEST02DB

OVERWRITE - When a conflict occurs, the OVERWRITE handler replaces the current value at the destination database with the new value in the LCR from the source database.

According Oracle doc: You must specify a conditional supplemental log group at the source database for all of the columns in the column_list at the destination database

I issued the following statment at TEST10G and TEST02DB:

ALTER TABLE denis.strm_tab3 ADD SUPPLEMENTAL LOG GROUP log_group_jobs_cr (first_name, last_name);

I then set an update conflict handler using the SET_UPDATE_CONFLICT_HANDLER procedure in the DBMS_APPLY_ADM package, using the prebuilt method OVERWRITE.

I issue the following statement at TEST02DB:

DECLARE
cols DBMS_UTILITY.NAME_ARRAY;
BEGIN
cols(1) := 'first_name';
cols(2) := 'last_name';
DBMS_APPLY_ADM.SET_UPDATE_CONFLICT_HANDLER(
object_name => 'denis.strm_tab3',
method_name => 'OVERWRITE',
resolution_column => 'first_name',
column_list => cols);
END;
/


Note:

The resolution_column is not used for OVERWRITE and DISCARD methods, but one of the columns in the column_list still must be specified.


RESULTS:

1. We can see the following on both database.
ID FIRST_NAME           LAST_NAME
---------- -------------------- ------------------------------
6 fn6d_10g             ln6d_10g


2. The apply process is in abort status at TEST10G, and apply process at TEST02DB works fine

To prepare for the next test, I restart the apply process at TEST10G, and issue the
following statment at TEST02DB:

denis@TEST02DB> update strm_tab3 set first_name='first', last_name='last' where id=6;

1 row updated.

denis@TEST02DB> commit;

I verified that this row is updated at both databases.


Case 3 - Conflict resolution - OVERWRITE at TEST02DB, DISCARD at TEST10G

DISCARD - When a conflict occurs, the DISCARD handler ignores the values in the LCR from the source database and retains the value at the destination database.

I issue the following statement at TEST10G to set up the DISCARD handler:

DECLARE
cols DBMS_UTILITY.NAME_ARRAY;
BEGIN
cols(1) := 'first_name';
cols(2) := 'last_name';
DBMS_APPLY_ADM.SET_UPDATE_CONFLICT_HANDLER(
object_name => 'denis.strm_tab3',
method_name => 'DISCARD',
resolution_column => 'first_name',
column_list => cols);
END;
/

Then I submitted the scheduler job again.

Results:

This time the row is updated as expected and apply processes run fine at both database. So in this configuration, if there are update conflicts, the statment issued at TEST10G will take effect. The statment issued at TEST02G will be ignored.

Note: we can issue the following query to check information about update conflict Handlers.

COLUMN OBJECT_OWNER HEADING 'TableOwner' FORMAT A5
COLUMN OBJECT_NAME HEADING 'Table Name' FORMAT A12
COLUMN METHOD_NAME HEADING 'Method' FORMAT A12
COLUMN RESOLUTION_COLUMN HEADING 'ResolutionColumn' FORMAT A13
COLUMN COLUMN_NAME HEADING 'Column Name' FORMAT A30

SELECT OBJECT_OWNER,
OBJECT_NAME,
METHOD_NAME,
RESOLUTION_COLUMN,
COLUMN_NAME
FROM DBA_APPLY_CONFLICT_COLUMNS
ORDER BY OBJECT_OWNER, OBJECT_NAME, RESOLUTION_COLUMN;



The output looks like:

At TEST10G


Table                           Resolution
Owner Table Name   Method       Column        Column Name
----- ------------ ------------ ------------- ------------------------
DENIS STRM_TAB3    DISCARD      FIRST_NAME    LAST_NAME
DENIS STRM_TAB3    DISCARD      FIRST_NAME    FIRST_NAME



At TEST02DB

Table                           Resolution
Owner Table Name   Method       Column        Column Name
----- ------------ ------------ ------------- -----------------
DENIS STRM_TAB3    OVERWRITE    FIRST_NAME    LAST_NAME
DENIS STRM_TAB3    OVERWRITE    FIRST_NAME    FIRST_NAME



To explore more about conflict resolution, check Chapter 3 Streams Conflict Resolution of the Oracle online documentation.

Friday, December 12, 2008

Creating a Simple Two-Way Streams Replciation Environment

Before starting this game, my environment is as follows:

  • Two 10g databases on my PC: TEST10G and TEST02DB
  • One-way Streams replication has been set up for the tables: dept and strm_tab2 in scott schema
  • replciation from TEST10G to TEST02DB.

By the end of game, my new enviroment should be:

  • Two databases: TEST10G and TEST02DB
  • One-way Streams replication for table dept and strm_tab2 in scott schema, from TEST10G to TEST02DB
  • Two-way Streams replication for table strm_tab3 in denis schema, between TEST10G and TEST02DB

My experimenting steps are as follows:
(ref: Oracle document: Creating a New Streams Multiple-Source Environment )

Step 0 -- Stop current capture, propagation and apply processes


Step 1 -- Create table denis.strm_tab3 at TEST10G


sys@TEST10G> select * from denis.strm_tab3;

ID FIRST_NAME LAST_NAME
---------- -------------------- ------------
1 Denis Sun
2 Tom Kyte
3 Jack Smith


Step 2 -- Create database link between TEST10G and TEST02DB

connect STRMADMIN/STRMADMIN@test02db

CREATE DATABASE LINK TEST10G.world connect to
STRMADMIN identified by STRMADMIN using 'TEST10G.world';
select * from global_name@TEST10G.world;

Note: DB link from TEST10G to TEST02DB already exists.

Step 3 -- Specifying an Unconditional Supplemental Log Group for Primary Key Column(s) at TEST10G

ALTER TABLE denis.strm_tab3 ADD SUPPLEMENTAL LOG GROUP tab3_id_pk1 (id) ALWAYS;


Step 4 -- Performing the following steps at TEST10G


-- create queues

connect strmadmin/strmadmin

BEGIN
DBMS_STREAMS_ADM.SET_UP_QUEUE(
queue_name => 'STREAMS_Q_SRC',
queue_table =>'STREAMS_Q_SRC',
queue_user => 'STRMADMIN');
END;strm_tab3;
/

BEGIN
DBMS_STREAMS_ADM.SET_UP_QUEUE(
queue_name => 'STREAMS_Q_DST',
queue_table =>'STREAMS_Q_DST',
queue_user => 'STRMADMIN');
END;
/


-- Add table rules to caputure process


BEGIN
DBMS_STREAMS_ADM.ADD_TABLE_RULES(
table_name => 'DENIS.STRM_TAB3',
streams_type => 'CAPTURE',
streams_name => 'STRMADMIN_CAPTURE_2',
queue_name => 'STRMADMIN.STREAMS_Q_SRC',
include_dml => true,strm_tab3;
include_ddl => true,
source_database => 'TEST10G.world');
END;
/


-- Add rule to propagation

connect strmadmin/strmadmin
BEGIN
DBMS_STREAMS_ADM.ADD_TABLE_PROPAGATION_RULES(
table_name => 'DENIS.STRM_TAB3',
streams_name => 'STRMADMIN_PROPAGATE_2',
source_queue_name => 'STRMADMIN.STREAMS_Q_SRC',
destination_queue_name => 'STRMADMIN.STREAMS_Q_DST@TEST02DB.world',
include_dml => true,
include_ddl => true,
source_database => 'TEST10G.world');
END;
/


-- Add apply rules for the table

conn strmadmin/strmadmin

BEGIN
DBMS_STREAMS_ADM.ADD_TABLE_RULES(
table_name => 'DENIS.STRM_TAB3',
streams_type => 'APPLY',
streams_name => 'STRMADMIN_APPLY_2',
queue_name => 'STRMADMIN.STREAMS_Q_DST',
include_dml => true,
include_ddl => true,
source_database => 'TEST02DB.world');
END;
/

-- apply user
conn strmadmin/strmadmin

BEGIN
DBMS_APPLY_ADM.ALTER_APPLY(
apply_name => 'STRMADMIN_APPLY_2',
apply_user => 'DENIS');
END;
/

Step 5 -- Performing the following steps at TEST02DB


-- create q
connect strmadmin/strmadmin

BEGIN
DBMS_STREAMS_ADM.SET_UP_QUEUE(
queue_name => 'STREAMS_Q_SRC',
queue_table =>'STREAMS_Q_SRC',
queue_user => 'STRMADMIN');
END;
/

BEGIN
DBMS_STREAMS_ADM.SET_UP_QUEUE(
queue_name => 'STREAMS_Q_DST',
queue_table =>'STREAMS_Q_DST',
queue_user => 'STRMADMIN');
END;
/


-- Add table rules to caputure process

BEGIN
DBMS_STREAMS_ADM.ADD_TABLE_RULES(
table_name => 'DENIS.STRM_TAB3',
streams_type => 'CAPTURE',
streams_name => 'STRMADMIN_CAPTURE',
queue_name => 'STRMADMIN.STREAMS_Q_SRC',
include_dml => true,
include_ddl => true,
source_database => 'TEST02DB.world');
END;
/


-- Add rule to propagation

connect strmadmin/strmadmin
BEGIN
DBMS_STREAMS_ADM.ADD_TABLE_PROPAGATION_RULES(
table_name => 'DENIS.STRM_TAB3',
streams_name => 'STRMADMstrm_tab3;IN_PROPAGATE_2',
source_queue_name => 'STRMADMIN.STREAMS_Q_SRC',
destination_queue_name => 'STRMADMIN.STREAMS_Q_DST@TEST10G.world',
include_dml => true,
include_ddl => true,
source_database => 'TEST02DB.world');
END;
/


-- Add apply rules for the table

conn strmadmin/strmadmin

BEGIN
DBMS_STREAMS_ADM.ADD_TABLE_RULES(
table_name => 'DENIS.STRM_TAB3',
streams_type => 'APPLY',
streams_name => 'STRMADMIN_APPLY_2',
queue_name => 'STRMADMIN.STREAMS_Q_DST',
include_dml => true,
include_ddl => true,
source_database => 'TEST10G.world');
END;
/
-- apply user
conn strmadmin/strmadmin

BEGIN
DBMS_APPLY_ADM.ALTER_APPLY(
apply_name => 'STRMADMIN_APPLY_2',
apply_user => 'DENIS');
END;
/



Step 6 Export and import

$ exp denis/oracle file=tab3.dmp tables=strm_tab3
Export: Release 10.2.0.1.0 - Production on Fri Dec 12 20:45:29 2008
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
Export done in WE8MSWIN1252 character set and AL16UTF16 NCHAR character set
About to export specified tables via Conventional Path ...
. . exporting table STRM_TAB3 3 rows exported
Export terminated successfully without warnings.

---------------------------------------------

$ imp denis/oracle@TEST02DB file=tab3.dmp tables=strm_tab3
Import: Release 10.2.0.1.0 - Production on Fri Dec 12 20:49:10 2008
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to: Oracle Database 10g Enterprise Edition Release 10.2.0.1.0 - Production
With the Partitioning, OLAP and Data Mining options
Export file created by EXPORT:V10.02.01 via conventional path
import done in WE8MSWIN1252 character set and AL16UTF16 NCHAR character set
. importing DENIS's objects into DENIS
. importing DENIS's objects into DENIS
. . importing table "STRM_TAB3" 3 rows imported
Import terminated successfully without warnings.



Step 7 -- Setting Instantiation SCNs

At TEST10G
~~~~~~~~~~

connect STRMADMIN/STRMADMIN@TEST02DB
set serveroutput on
DECLARE
iscn NUMBER; -- Variable to hold instantiation SCN value
BEGIN
iscn := DBMS_FLASHBACK.GET_SYSTEM_CHANGE_NUMBER();
DBMS_OUTPUT.PUT_LINE ('Instantiation SCN is: ' iscn);
END;
/


connect strmadmin/strmadmin@TEST10G
BEGIN
DBMS_APPLY_ADM.SET_TABLE_INSTANTIATION_SCN(
source_object_name => 'DENIS.STRM_TAB3',
source_database_name => 'TEST02DB.world',
instantiation_scn => &iscn);
END;
/



At TEST02DB
~~~~~~~~~

connect STRMADMIN/STRMADMIN@TEST10G
set serveroutput on
DECLARE
iscn NUMBER; -- Variable to hold instantiation SCN value
BEGIN
iscn := DBMS_FLASHBACK.GET_SYSTEM_CHANGE_NUMBER();
DBMS_OUTPUT.PUT_LINE ('Instantiation SCN is: ' iscn);
END;
/


connect strmadmin/strmadmin@TEST02DB
BEGIN
DBMS_APPLY_ADM.SET_TABLE_INSTANTIATION_SCN(
source_object_name => 'DENIS.STRM_TAB3',
source_database_name => 'TEST10G.world',
instantiation_scn => &iscn);
END;
/



Step 8 -- Configure conflict resolution

(skip this step for future experimenting, check this doc for reference )


Step 9 -- Start apply process, propagation and capture


Note:

Oracle recommends that you use only one capture process for each source database. I have two capture at TEST10G.



Step 10 - Verifying the replication.

Working as expected

Sunday, December 07, 2008

Adding a new table to an existing Oracle Streams Replication environment - second attempt

I breifly read the Oracle document about this topic, and I have successfuly added a new table to the replication by taking the following steps:

1. Create the table at source and destination database.

create table scott.strm_tab2 (
id number,
name varchar2(20),
constraint id_pk primary key(id)
);


2. Turn on supplemental logging for the table at source

ALTER TABLE scott.strm_tab2 ADD SUPPLEMENTAL LOG GROUP id_pk1 (id) ALWAYS;

3. At source database, stop the capture process and propagation process

++ CAPTURE PROCESSES IN DATABASE ++
CAPTURE_NAME : STRMADMIN_CAPTURE
QUEUE_OWNER : STRMADMIN
QUEUE : STREAMS_QUEUE
CAPTURE_TYPE : LOCAL
STATUS : DISABLED
RULE_SET_OWNER : SYS
RSN : RULESET$_17
NEGATIVE_RULE_SET_OWNER :
RSN2 :
CHECKPOINT_RETENTION_TIME : 60
VERSION : 10.2.0.1.0
LOGFILE_ASSIGNMENT : IMPLICIT
ERROR_NUMBER :
STATUS_CHANGE_TIME : 07-dec-2008 14:18:13


++ PROPAGATIONS IN DATABASE ++
PROPAGATION_NAME : STRMADMIN_PROPAGATE
SOURCE_QUEUE_OWNER : STRMADMIN
SOURCE_QUEUE_NAME : STREAMS_QUEUE
SRC GLOBAL NAME : TEST10G.WORLD
DESTINATION_QUEUE_OWNER : STRMADMIN
DESTINATION_QUEUE_NAME : STREAMS_QUEUE
DESTINATION_DBLINK : TEST02DB.WORLD
QUEUE_TO_QUEUE : FALSE
STATUS : ABORTED
ERROR_DATE :
ERROR_MESSAGE :
-----------------


4. At destionation database stop the apply process
++ APPLY INFORMATION ++
APPLY_NAME : STRMADMIN_APPLY
QUEUE_OWNER : STRMADMIN
QUEUE_NAME : STREAMS_QUEUE
APPLY_CAPTURED : YES
STATUS : DISABLED
APPLY_USER : SCOTT
APPLY_TAG : 00
RULE_SET_OWNER : STRMADMIN
RULE_SET_NAME : RULESET$_35
NEGATIVE_RULE_SET_OWNER :
NEGATIVE_RULE_SET_NAME :
APPLY_DATABASE_LINK :
-----------------
++ APPLY PROCESS INFORMATION ++
APPLY_NAME : STRMADMIN_APPLY
MAX_APPLIED_MESSAGE_NUMBER : 10138270879363
STATUS : DISABLED
STATUS_CHANGE_TIME : 07-dec-2008 14:20:47
ERROR_NUMBER :
ERROR_MESSAGE :
-----------------


5. At destination database, add rules for an apply process


BEGIN
DBMS_STREAMS_ADM.ADD_TABLE_RULES(
table_name => 'SCOTT.STRM_TAB2',
streams_type => 'APPLY',
streams_name => 'STRMADMIN_APPLY',
queue_name => 'STRMADMIN.STREAMS_QUEUE',
include_dml => true,
include_ddl => true,
source_database => 'TEST10G.world');
END;
/

6. At source database, add rules for propagation process

BEGIN
DBMS_STREAMS_ADM.ADD_TABLE_PROPAGATION_RULES(
table_name => 'SCOTT.STRM_TAB2',
streams_name => 'STRMADMIN_PROPAGATE',
source_queue_name => 'STRMADMIN.STREAMS_QUEUE',
destination_queue_name => 'STRMADMIN.STREAMS_QUEUE@TEST02DB.world',
include_dml => true,
include_ddl => true,
source_database => 'TEST10G.world');
END;
/

7. At source database, add rules for capture process

BEGIN
DBMS_STREAMS_ADM.ADD_TABLE_RULES(
table_name => 'SCOTT.STRM_TAB2',
streams_type => 'CAPTURE',
streams_name => 'STRMADMIN_CAPTURE',
queue_name => 'STRMADMIN.STREAMS_QUEUE',
include_dml => true,
include_ddl => true,
source_database => 'TEST10G.world');
END;
/

8. At destination database, set the instantiation SCN for the table

8.1 Execute the following to get SCN at source

connect STRMADMIN/STRMADMIN@TEST10G
set serveroutput on
DECLARE
iscn NUMBER; -- Variable to hold instantiation SCN value
BEGIN
iscn := DBMS_FLASHBACK.GET_SYSTEM_CHANGE_NUMBER();
DBMS_OUTPUT.PUT_LINE ('Instantiation SCN is: ' iscn);
END;
/

8.2 Execute the following with the SCN obtained in 8.1


connect strmadmin/strmadmin@TEST02DB
BEGIN
DBMS_APPLY_ADM.SET_TABLE_INSTANTIATION_SCN(
source_object_name => 'SCOTT.STRM_TAB2',
source_database_name => 'TEST10G.world',
instantiation_scn => &iscn);
END;
/

9. At source database start capture process

10. At source database start propagation process


11. At destinationdatabase start apply

12. Test the replciation --- OK DML and DDL

It is still not completely clear to me what was wrong for the steps I have taken yesterday. But it looks like we must stop capture, propagation and apply processes if we add a rule to the rule sets they are using and the order matters - From the doc: "If you perform administrative steps in the wrong order, you can lost LCRs"

Saturday, December 06, 2008

Adding a new table to an existing Oracle Streams Replication environment - A failed attempt

Today I assigned myself a task, which is to add a new table to an existing streams replication. The source database is TEST10G and the destination database is TEST02DB. They both reside in my notebook pc.

The following are all the test steps for today, the task does not succeed.

1. Apply create table DDL on both databases:

create table denis.strm_tab1 (
id number,
name varchar2(20),
constraint id_pk primary key(id)
);


2. Check current capture, propagation processes on the source database


++ CAPTURE PROCESSES IN DATABASE ++
CAPTURE_NAME : STRMADMIN_CAPTURE
QUEUE_OWNER : STRMADMIN
QUEUE : STREAMS_QUEUE
CAPTURE_TYPE : LOCAL
STATUS : ENABLED
RULE_SET_OWNER : SYS
RSN : RULESET$_17
NEGATIVE_RULE_SET_OWNER :
RSN2 :
CHECKPOINT_RETENTION_TIME : 60
VERSION : 10.2.0.1.0
LOGFILE_ASSIGNMENT : IMPLICIT
ERROR_NUMBER :
STATUS_CHANGE_TIME : 30-nov-2008 21:07:57
ERROR_MESSAGE :
-----------------

++ CAPTURE PROCESS SOURCE INFORMATION ++
CAPTURE_NAME : STRMADMIN_CAPTURE
CAPTURE_TYPE : LOCAL
SOURCE_DATABASE : TEST10G.WORLD
FIRST_SCN : 10138270589233
START_SCN : 10138270589233
CAPTURED_SCN : 10138270859114
APPLIED_SCN : 10138270859114
LAST_ENQUEUED_SCN : 10138270865525
REQUIRED_CHECKPOINT_SCN : 10138270830473
MAX_CHECKPOINT_SCN : 10138270859114
SOURCE_DBID : 917147433
SOURCE_RESETLOGS_SCN : 534907
SOURCE_RESETLOGS_TIME : 665622764
LOGMINER_ID : 1
-----------------

++ PROPAGATIONS IN DATABASE ++
PROPAGATION_NAME : STRMADMIN_PROPAGATE
SOURCE_QUEUE_OWNER : STRMADMIN
SOURCE_QUEUE_NAME : STREAMS_QUEUE
SRC GLOBAL NAME : TEST10G.WORLDInstantiating
DESTINATION_QUEUE_OWNER : STRMADMIN
DESTINATION_QUEUE_NAME : STREAMS_QUEUE
DESTINATION_DBLINK : TEST02DB.WORLD
QUEUE_TO_QUEUE : FALSE
STATUS : ENABLED
ERROR_DATE :
ERROR_MESSAGE :



3. Check current apply process on the destioination database


++ APPLY INFORMATION ++
APPLY_NAME : STRMADMIN_APPLY
QUEUE_OWNER : STRMADMIN
QUEUE_NAME : STREAMS_QUEUE
APPLY_CAPTURED : YES
STATUS : ENABLED
APPLY_USER : SCOTT
APPLY_TAG : 00
RULE_SET_OWNER : STRMADMIN
RULE_SET_NAME : RULESET$_35
NEGATIVE_RULE_SET_OWNER :
NEGATIVE_RULE_SET_NAME :
APPLY_DATABASE_LINK :
-----------------

++ APPLY PROCESS INFORMATION ++
APPLY_NAME : STRMADMIN_APPLY
MAX_APPLIED_MESSAGE_NUMBER :
STATUS : ENABLED
STATUS_CHANGE_TIME : 03-dec-2008 15:26:13
ERROR_NUMBER :
ERROR_MESSAGE :
-----------------


4. Execute the following steps at source database
4.1 Turn on supplementapply_vt.sqlal logging for STRM_TAB1 table
ALTER TABLE denis.strm_tab1 ADD SUPPLEMENTAL LOG GROUP id_pk1 (id) ALWAYS;

4.2 Add capture rules the table STRM_TAB1 at the source database:

conn / as sysdba
BEGIN
DBMS_STREAMS_ADM.ADD_TABLE_RULES(
table_name => 'DENIS.STRM_TAB1',
streams_type => 'CAPTURE',
streams_name => 'STRMADMIN_CAPTURE',
queue_name => 'STRMADMIN.STREAMS_QUEUE',
include_dml => true,
include_ddl => true,
source_database => 'TEST10G.world');
END;
/

4.3 Add propagation rules for the table STRM_TAB1 at the source database.

BEGIN
DBMS_STREAMS_ADM.ADD_TABLE_PROPAGATION_RULES(
table_name => 'DENIS.STRM_TAB1',
streams_name => 'STRMADMIN_PROPAGATE',
source_queue_name => 'STRMADMIN.STREAMS_QUEUE',
destination_queue_name => 'STRMADMIN.STREAMS_QUEUE@TEST02DB.world',
include_dml => true,
include_ddl => true, Adding to a Streams Replication Environment
source_database => 'TEST10G.world');
END;
/


5. Execute the following steps at the destination database
5.1 Add apply rules for the table at the destination database

BEGIN
DBMS_STREAMS_ADM.ADD_TABLE_RULES(
table_name => 'DENIS.STRM_TAB1',
streams_type => 'APPLY',
streams_name => 'STRMADMIN_APPLY',
queue_name => 'STRMADMIN.STREAMS_QUEUE',
include_dml => true,
include_ddl => true,
source_database => 'TEST10G.world');
END;
/


5.2 Grant privs to scott

Note: The apply user in existing repliction is scott. It may be better to
have different capture, propagation and apply for different schema, will
explore this later

denis@TEST02DB> grant all on strm_tab1 to scott;

Grant succeeded.


6. Instantiating

6.1 Execute the following to get SCN


connect STRMADMIN/STRMADMIN@TEST10G
set serveroutput on
DECLARE Adding to a Streams Replication Environment
iscn NUMBER; -- Variable to hold instantiation SCN value
BEGIN
iscn := DBMS_FLASHBACK.GET_SYSTEM_CHANGE_NUMBER();
DBMS_OUTPUT.PUT_LINE ('Instantiation SCN is: ' iscn);
END;
/

6.2 Execute the following with the SCN obtained in 6.1

connect strmadmin/strmadmin@TEST02DB
BEGIN
DBMS_APPLY_ADM.SET_TABLE_INSTANTIATION_SCN(
source_object_name => 'DENIS.STRM_TAB1',
source_database_name => 'TEST10G.world',
instantiation_scn => &iscn);
END;


7. Test to see if the replication work.
Insert a row into denis.strm_tab1 at source to see what happen at the dest.
Results: At the dest the row did not get inserted.
Verified that the replication for scott.dept still works fine.

8. Troubleshoot:

Execute the health check script at source and destination and review the report. Fixed the instantiating error, however, it is still not working
Tried stop and start capture, propagation and apply process, does not help.


9. Next step: Read documentation, try to fix the problem tommorrow if possible.

Oracle Doc: Adding to a Streams Replication Environment

Wednesday, December 03, 2008

Set up a Simple Oracle Streams Replication

Based on the Metalink Note: 224255.1 : "Steps To Setup Replication Using Oracle Streams", I was trying to set up a streams replicatin for the scott.dept table from a soruce datbase TEST10G to a destination database TEST02DB. Both databases are 10g and on my notebook computer. However my first attempt was not successfuly. When I inserted a row into the source table, nothing happens in the destionation table.

Oracle Metalink Note: 273674.1: "Streams Configuration Report and Health Check Script" provides a script that can be used to retrieve all the infomation related to the Streams configuration and perform health check. I used this script to generate the report for the TEST10G and TEST02BD. As expected, the report shows that the setup for the Apply process in the destinaton database is not correct. The source database should use the global name. The global name of source database is TEST10G.world, however in the report the source database is specified by TEST10G.

Firstly, I issued the following statement in the destionatin database as strmadmin
BEGIN
DBMS_STREAMS_ADM.remove_streams_configuration;
END;
/

Then I redo the step 2.4, 2.5, 2.6 and 4.2 as described in the Note: 273674.1. After that I inserted a row in the source scott.dept table and I see the same row got inserted into the destination table also. Replication is now working ...

Friday, June 20, 2008

Speed up the optimizer analyze job in a replicated environment

In an application, the production database is replicated to a reporting database and a DR database through SharePlex replication technology. The analyze jobs for the optimizer stats run on three instances bi-weekly. The problem is that the analyze job takes as long as 24 hours in the reporting database, which impact other jobs seriously. I have proposed a new procedure in order to reduce the time needed for the analyze job and potentially allow collectng more accurate optimizer stats within the time window available. The basic idea is to distribute the job among three instances and then combine the results, kind of divide-and-conquer.

Below I describe the steps. And overall time to obtain the stats was about 7-8 hours from the first time implementation, which involvs some manual steps. I think it is possible to automate the whole procedure if needed.


Steps:


1. Backup the current stats in PROD, RPT and DR instances:

1.1 create_stat_table

begin
DBMS_STATS.CREATE_STAT_TABLE (
'DB_ADMIN',
'MYSCHMEA_STATS_052508'
);
end;
/

1.2 export_schema_stats

begin
DBMS_STATS.EXPORT_SCHEMA_STATS (
'MYSCHEMA',
'MYSCHEMA_STATS_052508' ,
NULL,
'DB_ADMIN');
end;
/

2. Analyze group A table stats in PRD

Note: all tables that under MYSCHEMA are divided into two groups: group A and group B

It took about 4.5 hrs

3. Analyze group B table stats in DR

It took about 4.5 hrs

4. Analyze additional index stats in rpt

Note: Some indexes only exist in the reporting database

It took about 5.5 hrs

5. Sync up stats among three database

5.1 In PRD: export group A table/index stats

a. create stats table

begin
DBMS_STATS.CREATE_STAT_TABLE (
'DB_ADMIN',
'TAB_A_STATS',
NULL);
end;
/

b. run script: exp_tab_a_stats.sql



5.2 In DR: export group B table/index stats

begin
DBMS_STATS.CREATE_STAT_TABLE (
'DB_ADMIN',
'TAB_B_STATS',
NULL);
end;
/

script: exp_tab_b_stats.sql

5.3 In PRD: import group B table/index stats
a.
create table db_admin.tab_b_stats as
select * from db_admin.tab_b_stats@DB_DR;

b.
run script: imp_tab_b_stats.sql

Verify:

select table_name, last_analyzed from dba_tables where owner='MYSCHEMA' and last_analyzed < sysdate -1;

select index_name, last_analyzed from dba_indexes where owner='MYSCHEMA' and last_analyzed < sysdate -1;

5.4 RRT: import group A table/index stats


create table db_admin.tab_b_stats
as select * from db_admin.tab_b_stats@DB_DR;

create table db_admin.tab_a_stats
as select * from db_admin.tab_a_stats@DB_PRD;

script: imp_tab_a_stats.sql

5.5 Rpt: import group B table/index stats
script: imp_tab_b_stats.sql

5.6 Dr: import group A table/index stats
create table db_admin.tab_a_stats
as select * from db_admin.tab_a_stats@DB_PRD_lnk;

script: imp_tab_a_stats.sql


Note: to generate the imp/exp table stats scripts, I used the AWK script: for example:


# exp_stats.awk
{
print "begin"
print " dbms_stats.export_table_stats("
print " 'MYSCHEMA', "
print " '" $1 "',"
print " stattab => 'TAB_B_STATS' ,"
print " statown => 'DB_ADMIN' ,"
print " cascade => TRUE, "
print " );"
print "end;"
print "/"
print " "
}


# imp_stats.awk
{
print "begin"
print " dbms_stats.import_table_stats("
print " 'MYSCHEMA', "
print " '" $1 "',"
print " stattab => 'TAB_B_STATS' ,"
print " statown => 'DB_ADMIN' ,"
print " cascade => TRUE "
print " );"
print "end;"
print "/"
print " "
}

Monday, June 16, 2008

Resolve an issue of duplicated rows in a Shareplex replication environment

Table SMC is one of the tables that are replicated from the production database to the reporting database. In production, id is the PK column and there is also a unique constraint (UK) on the columns (oid,eid,mid,st) of the table.

The problem of duplicated rows was identified in terms of there are mutiple rows having same (oid, eid, mid,st) in the SMC table in the reporting database.

The reason for the existing of the duplicated rows may be as follows:
(1) The PK and UK constraints have not been enabled
(2) In the reporting database, single post has been changed to multiple post queue.

Since there is no testing environment available, the actual reason for the problem has not been able to be confirmed by repeating the issue. However, it is obviously wrong that the PK and UK are not enabled in the SMC table in reporting database.

To fix this problem, I have to identify and remove the duplicated rows and then enable the PK and UK.

First of all, by comparing the row counts of the table in production and reporting using the following query, it can be known that the duplication only occurred during one day. ( Since the table is huge: > 100 GB, it is not wise to try to find the duplicated rows in all date range)

select count(*) from sf.smc
where
mc_date >= to_date('2008-05-21 00:00:00', 'YYYY-MM-DD HH24:MI:SS')
and
mc_date <= to_date('2008-05-21 23:59:59', 'YYYY-MM-DD HH24:MI:SS')
/
Then, I created a temporary table from the SMC table in the reporting database as follows:
create table temp_a  nologging
as 
select 
id, oid, eid, mid, st 
FROM
sf.smc
where 
mc_date > to_date('2008-05-21 00:00:00', 'YYYY-MM-DD HH24:MI:SS')
and
mc_date > to_date('2008-05-21 23:59:59', 'YYYY-MM-DD HH24:MI:SS')
/

execute dbms_stats.gather_table_stats(user, 'TEMP_A', null,estimate_percent=>10);

The duplicated rows as well as the baseline row ( i.e. the row to which any other rows compared in order to determine if they are duplicated )are displayed through the following sql:
select id,  oid, eid, st, mid from smc
where (oid, eid, st, mid ) in (
select  oid, eid, st, mid
FROM
temp_a A
WHERE
rowid >
(SELECT min(rowid) FROM temp_a B
WHERE
B.oid = A.oid
and
B.eid = A.eid
and
B.mid = A.mid
and B.st = A.st
)
)
order by 2,3,4,5

The following are the duplicate rows in RPT
ID      OID             EID                   ST              MID
---------- -------------------- -------------------- ----------- ----------
1037693787 CICN087494895        489740124            Change           8
1036151673 CICN087494895        489740124            Change           8
1036151856 CICN087494953        430282795            Change           2
1036152154 CICN087494953        430282795            Change           2
1037621342 ICOG087494888        503141443            Install         19
1036151603 ICOG087494888        503141443            Install         19
1036151617 IICN087494988        503141881            Install          8
1036183677 IICN087494988        503141881            Install          8

8 rows selected.

Note the column ID is different, to decide which row should be removed, we need to know which ID is presented in the produciton database: The following are the corresponding row in the production
ID     OID                EID                   ST             MID
---------- -------------------- -------------------- ---------- ----------
1037693787 CICN087494895        489740124            Change           8
1036152154 CICN087494953        430282795            Change           2
1037621342 ICOG087494888        503141443            Install         19
1036183677 IICN087494988        503141881            Install          8





As a result, the rows with the following id will be deleted in the reporting database:

'1036151673'
'1036151856'
'1036151603'
'1036151617'


Note: schema, table and column names have been modified to hide any possible confidential information


updated: Mar 29, 10 - When dealing with duplicated rows issue, it is obvious we should be very clear about what the meaning of  'duplicate' for a particular table, all  columns value are same or just some columns values are same?. If we only consider PK columns, for example (col1, col2) are supposed to be PK, but the constraint are not enabled and we want to delete the duplicated rows in order to create the PK. We can do:

create table tab_tmp
as
select a.rowid, a.col1, a.col2
   from tab a
  where a.rowid >
     ( select max(rowid)
       from tab b
      where a.col1 = b.col1
         and a.col2=b.col2);

We maybe want to create index on (col1 col2) if not there  first , non-unique of course. Then we delete duplicated row by:

delete from tab a
where a.rowid in ( select rowid from tab_temp);


Updated: Jul 29, 2010 -

In this post: http://viralpatel.net/blogs/2010/06/deleting-duplicate-rows-in-oracle.html , several ways of deleting duplicated rows are summarized. One interesting method is using analytic function, i.e.
DELETE FROM tbl_test
 WHERE ROWID IN (
         SELECT rid
           FROM (SELECT ROWID rid,
                 ROW_NUMBER () OVER (PARTITION BY ser_no, fst_nm, deptid, cmnt ORDER BY ROWID) rn
                 FROM tbl_test)
        WHERE rn <>1);

Tuesday, July 03, 2007

Experience errors in a test database refreshed from production

MYDBP is a production database (8i) with replication and in archivelog mode. MYDB3S is a test databe, intend to be standalone and in noarchivelog mode. I created database MYDB3S from RMAN backup of MYDBP using RMAN duplicate command. However, I have made two mistakes:


1. Dropped replication package, resulting in some triggers invalid

In some triggers, there is a condition check:

IF DBMS_REPUTIL.FROM_REMOTE
THEN
RETURN;
END IF;

While I executed the $ORACLE_HOME/rdbms/admin/catrepr.sql to remove replication catalog views, packages, etc, I dropped the DBMS_REPUTIL package among other things.

Solution: re-install replciation catalog views and package by issue:
$ORACLE_HOME/rdbms/admin/catrep.sql

2. Failed to turn off archivelog mode
I only set archive_log_start=false in the init.ora. I should have issued
'alter system noarchivelog' in the mount mode too to really turn off the archivelog mode

Wednesday, June 13, 2007

Replication and LOB

ISSUE
=====
During conversion test on MYDBQA/QB the SYSTEM tablespace got extended to ~ 5G. Try to understand the reason.


RESEARCH
=========

1. issue following query against MYDBP and MYDBQA

col owner format a14
col segment_name format a27
col table_name format a18
col column_name format a20


select a.owner, a.segment_name, a.bytes/1024/1024 "size (M)",
b.table_name, b.column_name
from dba_segments a, dba_lobs b
where a.tablespace_name='SYSTEM'
and a.segment_type='LOBSEGMENT'
and a.segment_name=b.segment_name
order by 3


system@MYDBQA> /

OWNER SEGMENT_NAME size (M) TABLE_NAME
COLUMN_NAME
--------------- --------------------------- ----------
------------------ --------------------
SYS SYS_LOB0000000270C00002$$ .0234375 KOTTD$
SYS_NC_ROWINFO$
SYS SYS_LOB0000000274C00002$$ .0234375 KOTTB$
SYS_NC_ROWINFO$
SYS SYS_LOB0000000278C00002$$ .0234375 KOTAD$
SYS_NC_ROWINFO$
SYS SYS_LOB0000000282C00002$$ .0234375 KOTMD$
SYS_NC_ROWINFO$
SYSTEM SYS_LOB0000002394C00012$$ .0625 AQ$_QUEUES
SUBSCRIBERS
SYSTEM SYS_LOB0000002590C00002$$ .0625 DEF$_TEMP$LOB
TEMP$CLOB
SYSTEM SYS_LOB0000002590C00003$$ .0625 DEF$_TEMP$LOB
TEMP$NCLOB
SYSTEM SYS_LOB0000002590C00001$$ .0625 DEF$_TEMP$LOB
TEMP$BLOB
SYSTEM SYS_LOB0000002581C00005$$ .0625 DEF$_LOB
NCLOB_COL
SYSTEM SYS_LOB0000002581C00004$$ 2.3125 DEF$_LOB
CLOB_COL
SYSTEM SYS_LOB0000002581C00003$$ 3241 DEF$_LOB
BLOB_COL


system@MYDBP> /

OWNER SEGMENT_NAME size (M) TABLE_NAME
COLUMN_NAME
---------- --------------------------- ---------- ------------------
--------------------
SYS SYS_LOB0000000270C00002$$ .0234375 KOTTD$
SYS_NC_ROWINFO$
SYS SYS_LOB0000000274C00002$$ .0234375 KOTTB$
SYS_NC_ROWINFO$
SYS SYS_LOB0000000278C00002$$ .0234375 KOTAD$
SYS_NC_ROWINFO$
SYS SYS_LOB0000000282C00002$$ .0234375 KOTMD$
SYS_NC_ROWINFO$
SYSTEM SYS_LOB0000002394C00012$$ .0625 AQ$_QUEUES
SUBSCRIBERS
SYSTEM SYS_LOB0000002590C00002$$ .0625 DEF$_TEMP$LOB
TEMP$CLOB
SYSTEM SYS_LOB0000002590C00003$$ .0625 DEF$_TEMP$LOB
TEMP$NCLOB
SYSTEM SYS_LOB0000002590C00001$$ .0625 DEF$_TEMP$LOB
TEMP$BLOB
SYSTEM SYS_LOB0000002581C00005$$ .0625 DEF$_LOB
NCLOB_COL
SYSTEM SYS_LOB0000002581C00004$$ 2.3125 DEF$_LOB
CLOB_COL
SYSTEM SYS_LOB0000002581C00003$$ 640.625 DEF$_LOB
BLOB_COL


Note: the last LOGSEGMENT has size of 3.2G in MYDBQA versus 640 M in MYDBP


CONCLUSIONS
===========
It can be inferred that there are significant DMLs on LOB type columns
during conversion

From Oracle Doc:
=================
( http://www.csee.umbc.edu/help/oracle8/server.815/a67791/ch9.htm#1420 )


DEFLOB View of Storage for RPC

Oracle stores internal LOB parameters to deferred RPCs in a side table that is referenced only by way of a synonym. This gives the you flexibility for storage parameters and the containing schema. The following shows the default storage table for LOB parameters.

CREATE TABLE system.def$_lob(
id RAW(16) CONSTRAINT def$_lob_primary PRIMARY KEY,
deferred_tran_db VARCHAR2(128), -- origin db
deferred_tran_id VARCHAR2(22), -- transaction id
blob_col BLOB,
clob_col CLOB
nclob_col NCLOB)
/
-- make deletes fast
CREATE INDEX system.def$_lob_n1 ON system.def$_lob(
deferred_tran_db,
deferred_tran_id)
/
-- use a synonym in case underlying table is moved
CREATE SYNONYM sys.def$_lob FOR system.def$_lob
/
CREATE OR REPLACE VIEW DefLOB AS SELECT * FROM sys.def$_lob
/
CREATE PUBLIC SYNONYM DefLOB FOR DefLOB
/

Thursday, May 24, 2007

Rebuild/Move Replication Base Tables

1. Check deferred transactions.

system - chk_def_tbls.sql


----
spool chk_def_tbls.log

select * from defcall;
select * from deftran;
select * from deferror;
select * from dba_repcatlog;

spool off
----

2. Check base tables.

system - chk_base_def_tbls.sql
----
spool chk_base_def_tbls.log

select * from system.def$_calldest;
select * from system.def$_defaultdest;
select * from system.def$_error;
select * from system.def$_origin;
select * from system.repcat$_repschema;
select * from system.def$_destination;

spool off
-----


3. Remove replication if it exist.

sys - $ORACLE_HOME/rdbms/admin/catrepr.sql

4. Drop the base deferred transaction tables.

system - dr_base_def_tbls.sql

----------------
spool dr_base_tbls.log

drop table system.def$_calldest;
drop table system.def$_defaultdest;
drop table system.def$_error;
drop table system.def$_origin;
--drop table system.repcat$_repschema; This is dropped in catrepr.sql
drop table system.def$_destination;

execute dbms_aqadm.drop_queue_table('SYSTEM.DEF$_AQCALL', TRUE);
execute dbms_aqadm.drop_queue_table('SYSTEM.DEF$_AQERROR', TRUE);

commit;

spool off
-------------


5. Create the replication tablespace and
set system's default tablespace to the rep tblsp.

system - cr_rep_tblsp.sql

6. Create the base deferred transaction tables.

system - $ORACLE_HOME/rdbms/admin/catdefrt

7. Create replication

sys - $ORACLE_HOME/rdbms/admin/catrep

8. Change system's default tablespace back to what it was.

Tuesday, May 22, 2007

Post RMAN Refresh tasks -- Remove Replication

Post RMAN Refresh tasks -- Remove Replication
==================================================================

Issues:
-------

When use RAMN duplicate command to create a test database from a
production database, which is a replication master definition site
database, we want to drop all replication stuff in the test database.

Post-refresh task
--------------------

1. drop replicaiton packages
login as system
------ gen_drop_rep_packages.sql ---
set feedback off;
set echo off;
set pagesize 1000;
set linesize 200;
set heading off;
set echo off;
spool drop_replication_packages.sql;
select 'spool drop_replication_packages.lst;' from dual;

select 'DROP PACKAGE ' owner '.' object_name ';' from
dba_objects
where object_type ='PACKAGE' and (object_name like '%$RP' or
object_name like '%$RL') ;

select 'spool off;' from dual;
select 'exit;' from dual;
spool off;
-- exit;
---------------

select 'DROP PUBLIC SYNONYM ' object_name ';' from dba_objects
where object_type ='SYNONYM' and (object_name like '%$RP' or
object_name like '%$RL') ;

2. login as sys, run catrepr.sql ( remove replciation-replciated catalog
views)
@?/rdbms/admin/catrepr.sql

3. remove replciation related jobs

login as repadmin
select 'exec dbms_job.remove(' job ');' from user_jobs;

'EXECDBMS_JOB.REMOVE('JOB');'
---------------------------------------------------------------
exec dbms_job.remove(98);
exec dbms_job.remove(78);
exec dbms_job.remove(44);
exec dbms_job.remove(79);
exec dbms_job.remove(80);
exec dbms_job.remove(81);
exec dbms_job.remove(82);
exec dbms_job.remove(83);
exec dbms_job.remove(84);
exec dbms_job.remove(85);
exec dbms_job.remove(86);
exec dbms_job.remove(43);
exec dbms_job.remove(87);
exec dbms_job.remove(88);
exec dbms_job.remove(89);
exec dbms_job.remove(90);
exec dbms_job.remove(141);
exec dbms_job.remove(142);
exec dbms_job.remove(144);
exec dbms_job.remove(145);

4. drop database links
repadmin@GENQB> drop database link GENB.WORLD
2 /
Database link dropped.

repadmin@GENQB> @conn system/xxx

system@GENQB> drop public database link GENB.WORLD;
Database link dropped.

drop GENARCHP.WORLD public link
drop public database link GENARCHP.WORLD;

5. Remove the REPADMIN user
DROP USER repadmin CASCADE;

Note: Before re-create replication, maybe need to execulte following:
EXECUTE Dbms_Defer_Sys.Unregister_Propagator(username=>'REPADMIN');
EXECUTE Dbms_Repcat_Admin.Revoke_Admin_Any_Schema(username=>'REPADMIN');

6. change password for all users

7. fix global_name
alter database rename global_name to genqb.world

8. remove statspack job too
conn perfstat/xxx
execute dbms_job.remove(119);

9. Add tempfile if not done so

10. Drop gen_rep_data tablespace:

rm the following file after drop the tablespace
system@GENQB> select file_name from dba_data_files where tablespace_name
='GEN_REP_DATA';

FILE_NAME
------------------------------------------------------------------------
----------------------------
/ora02/oradata/GENXQB/gen_rep_data_05.dbf
/ora01/oradata/GENQB/gen_rep_data_01.dbf
/ora02/oradata/GENQB/gen_rep_data_02.dbf
/ora01/oradata/GENQB/gen_rep_data_03.dbf
/ora02/oradata/GENQB/gen_rep_data_04.dbf

Saturday, May 19, 2007

Set up a two-site multimaster replication environment including databases MYDBQA and MYDBQB

Note: I had a task to create a testing replication environment. This is my note about this effort.

Purpose:
Create a test replication environment that is similar to the production environment for MYDBP and MYDBB

References:
MetaLink Note:117434. Initial Steps Required to Create a Multi Master Replication Environment v8.1/v9.x

Procedure:

1. Create MYDBQA and MYDBQB databases

a. Shutdown the MYDBQA
b. Delete all the datafiles, controlfiles, redologfiles
c. Recreate the password file at $ORACLE_HOME/dbs
orapw file=orapwMYDBQA.ora password=xxxx
d. Edit a simple initMYDBQA.ora (see note)
e. Start sqlplus, startup the instance in nomount mode (using the simple initMYDBQA.ora)
f. Create the database
Run the script: cr_mydbqa.sql ( only created system tablespace)
After succeed, shutdown, restart with original initMYDBQA.ora
Create some additional tablespaces and users
g. Add rollback segment
h. Run catalog.sql and catproc.sql ( as SYS)
@?/rdbms/admin/catalog.sql
@?/rdbms/admin/catproc.sql

As system

@?/sqlplus/admin/pupbld.sql

Note: keep an old copy of initMYDBQA.ora file. Create a simple initMYDBQA.ora, including only necessary parameters such as those dump dir, db_name, db_block_size etc in the file first. After successfully start up the instance using this simple init.ora, running the create database script. If directly use the old initMYDBQA.ora to start up the instance, it will encounter an error at the creating database stage: ORA-02084: database name is missing a component. Not sure why it happened. After that, shutdown the database, using the old initMYDBQA.ora,
,check it can be startup/shutdown normally.

2. Preparations for replication environment:

2.1 Check init.ora parameter of MYDBQA, MYDBQB database, compared with MYDBP, MYDBB
2.2 Check tablespace requirements

3. Install the Replication Catalogue

Note: Create an new tablespace mydb_rep_data, change the default tablespace for user ‘SYSTEM’ to mydb_rep_data, after install the replication catalogue, change the default tablespace back to system for user ‘SYSTEM’

3.1 Connect to the database as sys (mydb_rep_data is the tablespace used)
a. Change user SYSTEM’s default tablespace to MYDB_REP_DATA

ALTER USER SYSTEM DEFAULT TABLESPACE MYDB_REP_DATA;

b. change sys and system user’s temporary tablespace to temp (optional – not critical for setting up replication)

alter user sys temporary tablespace temp;

alter user system temporary tablespace temp;


3.2. Execute the script CATREP.SQL once the database has started. ( as SYS)

a. execute the script catdefer.sql ( as system, not sure if necessary, it does not hurt)
@?/rdbms/admin/catdefer
b. execute catrep.sql as sys
@?/rdbms/admin/catrep

SQL> SPOOL output.log
Once CATREP.SQL has completed, turn off the output spooling.
SQL> SPOOL OFF
Check OUTPUT.LOG for errors before proceeding.

3.3 Confirm that CATREP.SQL ran correctly by running a query on ALL_OBJECTS
where STATUS = 'INVALID'. For example,

SELECT OWNER, OBJECT_NAME, OBJECT_TYPE FROM ALL_OBJECTS
WHERE STATUS = 'INVALID' and OWNER=’SYS’;
2
OWNER OBJECT_NAME OBJECT_TYPE
------------------------------ ------------------------------ ------------------
SYS DBMS_OBFUSCATION_TOOLKIT PACKAGE BODY
SYS DBMS_PSP PACKAGE BODY
SYS DBMS_SNAP_INTERNAL PACKAGE BODY
SYS DBMS_UTILITY PACKAGE BODY
SYS UTL_TCP PACKAGE BODY

If you find that any of the SYS or SYSTEM package bodies have compiled
incorrectly, recompile them manually.

SQL> ALTER PACKAGE COMPILE BODY;

alter package DBMS_OBFUSCATION_TOOLKIT compile body;
alter package DBMS_PSP compile body;
alter package DBMS_PSP compile body;
alter package DBMS_SNAP_INTERNAL compile body;
alter package DBMS_UTILITY compile body;
alter package UTL_TCP compile body;

If CATREP.SQL has run successfully, a number of replication catalog tables are created in the MYDB_REP_DATA tablespace. The database is now setup for advanced replication.

3.4 Change the default tablespace of the user ‘SYSTEM’ back to system.
alter user system default tablespace system;

Note: Oracle9i creates the replication catalog as part of the CATALOG.SQL script,
so customers running Oracle9i 9.0 / 9.2 can skip this section. However it might be useful to check the catalog is valid. ( In this case we may want to move the replication packages from SYSTEM tablespace to MYDB_REP_DATA tablespace)


4. Configure Oracle Net
Configuration file: listner.ora and tnsname.ora, already exist, no changes needed

5. Using OEM to setup a sample replication environment
( just to get a feeling about replication, clean objects created in this once done. Skip this step normally!)

5.1 Create a user: nbk9lsj on both MYDBQA and MYDBQB database. Create table t1 and t2.

5.2 Using OEM to create a replication group
nbk9lsj_grp, t1, t2 are replicated objects in this group
In OEM, Click: Distributed -> Advanced Replication -> right click: Multimaster replication, then follow the wizard to set up multimaster replication environment

Click Setup Master Sites... to launch a wizard that helps you set up a multimaster replication environment.

5.3. Create Master Group.

Login as repadmin, follow the wizard.

Upon completion, the new group will appear in the navigator, and a request will be submitted to start the replication process.

From the navigator, select the master group you just created. Multimaster replication is running if the master group status is Started.

( undo everything in this step)

6. Import schema objects to the newly created MYDBQA and MYDBQB.

Do creating public synonym, grant, compile invalid object, analyze etc after importing

Schema: MYDB, DEV, GWH, MYDBRSV

/oracle/admin/MYDBQA/create/scripts/impmydb/MYDBQA/REFRESH_ALL/reload_obj.sh

Note: when imp from MYDBP backup, there are many replication packages in mydb schema remain invalid, ignore. ( drop those replication package as well in the future)

7. Implement the replication using script:
( script directory: …/create/rep)


1) Create the replication administrator (edit and execute the following at /create/rep/admin as system).[1 ]
cr_admin.sql

Note: the following should be issued first to avoid some error:
system@MYDBQA> @?/sqlplus/admin/pupbld.sql

2) Create directories (rg_repgroupname) under the …/create/rep directory for each replication group.[2]

3) Copy the files from the rg_x directory to each rg_repgroupname directory.[2]

4) Edit the files in each of the rg_repgroupname directory to correspond to the replication group.[2 ]

5) Synchronize the data in all databases, if necessary.

6) Execute each of these scripts for each replication group (as repadmin):[2]
While running these, check dba_repcatlog for admin request.
Make sure that all admin request have completed before executing the next script.

cr_rg - after running this one for a group, change the job interval!
mydb_cr_ro2
cr_ro
mydb_cr_rs2
cr_rs
cr_ms

Note:
(1) mydb_cr_ro2 and mydb_cr_rs2 are created to replace the original mydb_cr_ro and mydb_cr_rs. They use db link to obtain the replicated table names from MYDBP.

As repadmin, create the db link as follows:
SQL> create database link MYDBP.WORLD
connect to system identified by using ‘MYDBP.WORLD’

(2) The script doitall.sql contains all above scripts for convenience. (It seems not guaranteed that the rep group will be in normal status after applying this script. The status of rep group could be still quiesced. But using OEM to adjust after running doitall is usually easy.)

(3) Using OEM to change the interval of the dbms_repcat.do_deferred_repcat_admin job.
If you want to use command line, follow these steps:
a. Login as repadmin
b. Find the job id for the replication group, e.g.
SQL> select job from dba_jobs where what like '%RG_MYDB_CF_AC%';

JOB
----------
5


c. Change the interval

BEGIN DBMS_JOB.CHANGE(
job => 5,
next_date => sysdate,
interval => '/*10:Secs*/ sysdate + 10/(60*60*24)',
what => NULL);
END;
/

d. Run the job
execute dbms_job.run(5);
After the replication has been created.

7) Schedule the purge (as repadmin).[1]
sch_purge
8) Schedule the push (as repadmin).[1]
sch_push

(Note: 7,8 only need to execute once for the first group)

9) Resume master activity (as repadmin).[2]
rsum_master_act

10) Schedule jobs to monitor the dba jobs and deferred transactions.[1]

[1]= Execute the step on all sites.
[2] = Execute the step on the master definition site only.

Sunday, May 13, 2007

Oracle Multi-Master Replication - Remove Replication Support


1. To remove replication perform the following on the Master Definition Site:

CONNECT repadmin/repadmin@tsh1
-- Stop replication
EXECUTE Dbms_Repcat.Suspend_Master_Activity(gname=>'MYREPGRP');

-- Delete replication groups
EXECUTE Dbms_Repcat.Drop_Master_Repobject('SCOTT', 'EMP', 'TABLE');
EXECUTE Dbms_Repcat.Remove_Master_Databases('MYREPGRP', 'TSH2.WORLD');

2. Next do the following on all Master Sites:
CONNECT repadmin/repadmin@tsh1

-- Remove private databse links to other master databases
EXECUTE Dbms_Repcat.Drop_Master_Repgroup('MYREPGRP');
DROP DATABASE LINK TSH2.WORLD;

-- Remove any leftover jobs (see DBA_JOBS for job numbers)
EXECUTE Dbms_Job.Remove(62);
EXECUTE Dbms_Job.Remove(63);

CONNECT sys@tsh1

-- Remove the REPADMIN user
EXECUTE Dbms_Defer_Sys.Unregister_Propagator(username=>'REPADMIN');
EXECUTE
Dbms_Repcat_Admin.Revoke_Admin_Any_Schema(username=>'REPADMIN');
DROP USER repadmin CASCADE;

-- Drop public database links to other master databases
DROP PUBLIC DATABASE LINK TSH2.WORLD;