May 11, 2008

Database Replication with GoldenGate software

For a number of reasons - people replicate data to remote destinations. We do have our own needs. We have been running SharePlex for data replication purpose for the last 3 years. By this time, our system usages increased more than twice in just last year. Initially SharePlex was good for us but now a days we need more flexible configurations and different replication architecture for the system.

We are looking for an alternative database replication mechanism that would suit best for us. We are exploring new technologies available to serve replication purpose.

Now, we found two alternatives to be considered as we need more control for an "Active-Active" mode -

1. Oracle Streams
2. GoldenGate Software

I did several extensive experiments with Oracle Streams for about 1.5 months. You can guess, there are lots of outcomes. This post is for telling a bit about my GoldenGate experience!

GoldenGate Software (GGS) is a popular one used for data replication and keep several database in sync. They are supporting a number of database including Oracle, Teradata, DB2, Sybase, SQL Server. It seems that GGS are in market for long time, probably before Streams introduced by Oracle, now they are running version 9.5.

The basic concepts of GGS are - GGS has EXTRACT process which captures changes in source database. The changes are kept in files called TRAIL. The changes can be directly sent to the target/destination database's REPLICAT process to apply the changes in destination database. GGS EXTRACT reads data from oracle online redo log file - so it's independent of oracle process. I forgot to mention, GGS has several processes, MANAGER is one of the important process which managers all other stuffs.

My first "one way" replication test configuration (obvious these are not all for production db) with GGS were as follows -


Source Neptune: Create EXTRACT which will capture changes and send the changes to remote host's TRAIL files

EDIT PARAMS extprod
EXTRACT extprod
USERID ggs, PASSWORD ggs
RMTHOST saturn, MGRPORT 7820
RMTTRAIL /export/home/oracle/ggs/dirdat/rt
TABLE DEV.LOGIN;
**
ADD EXTRACT extprod, TRANLOG, BEGIN NOW
**
ADD RMTTRAIL /export/home/oracle/ggs/dirdat/rt, EXTRACT extprod, MEGABYTES 50

Destination Saturn: Create REPLICAT which will read changes from TRAIL files and apply the changes in current database

EDIT PARAMS repprod
REPLICAT repprod
ASSUMETARGETDEFS
DISCARDFILE ./dirrpt/REPSOE_DSC.rpt, PURGE
USERID ggs, PASSWORD ggs
MAP DEV.*, TARGET DEV.*;
**
ADD REPLICAT repprod, EXTTRAIL /export/home/oracle/ggs/dirdat/rt, NODBCHECKPOINT


So, these were the basic stuffs that I started with GoldenGate Software (GGS). They have thousand page documents and there are a lot more to test. Earlier, we created spreadsheet putting about 30 evaluation criteria against which we compared SharePlex and Oracle Streams. After experimenting with GGS, I will put the outcomes for GGS and would take the ultimate product decision.

Expecting comments if you have any good/bad experience with the product!

10 comments:

Anonymous said...

From our experience GoldenGate GGS is not efficient. Besides it is very expensive. There are couple of alternatives that do read Oracle redo log files and work better than goldengate. One is a DataMirror datamirror.com. Though they were bought recently by IBM, not sure if you want to deal with these gorillas. There is also WisdomForce - quite interesting company. WisdomForce had FastReader for a long time - very solid bulk data extract and load. Recently WisdomForce came up with DatabaseSync www.wisdomforce.com/dweb/index.php?id=1001
Database Sync can read from Oracle redo log files in real time and apply the changes to the destination that can be Oracle, Teradata or SQL Server. Recommended

Omar Faruq said...

Thanks for your valuable comments. We are in the evaluation mode - let's see :)

Anonymous said...

They had major bugs in their replication systems, specially with LOBs. I still prefer Oracle Streams over it. If you have a good database team to run and maintain Oracle Streams, you should not use anything else. Good luck!

Venu Kudupudi said...

Can you shed on some light on how your evaluation went? We are in same situation to choose between GG and streams. Your post was almost 2 years old, i guess by now you must have implememted replication using one of these products.

Also can you give some details on your eraluation criteria between the products?

Jared said...

Would you care to share your evaluation spreadsheet?

I am beginning to evaluate GoldenGate and SharePlex, and either need to find or create an evaluation spreadsheet.

Anonymous said...

Dear Omar,

Would you pls. share your evaluation results & Spread-sheets.

As we are looking for strong Replication Solutions for our Oracle Standard Edition One Database.

For Golden Gate, we have to buy extra options to upgrade the Database. Any feature benefit of GGS over SharePlex for Oracle ?

Anonymous said...

Dear Omar,

Would you pls. share your evaluation results & Spread-sheets.

As we are looking for strong Replication Solutions for our Oracle Standard Edition One Database.

For Golden Gate, we have to buy extra options to upgrade the Database. Any feature benefit of GGS over SharePlex for Oracle ?

Omar Faruq said...

Hello,

I am sorry, the evaluation spreadsheet which I did long long ago, was an internal document and specific to our company and business rules. I won't be able to share that. However, we have been using Oracle GGS. Thanks!

Omar Faruq said...

At the time back, while we looked at other options, GGS seemed more flexible, manageable and predictable than the other ones. Thanks!

Eric said...

Dbvisit has an interesting product in this space called Dbvisit Replicate.
Has the advantages of GG but easier to install and maintain and way more cost effective.
Check it out at www.dbvisit.com