Third party replication product for Oracle database!

Before I forget, I want to write a few words on this. It's my feeling, one might say "gut feeling" about the technology or technological world. Sometimes, what it seems, may not be the case for a product. In my opinion, people actually buy "trust" with money! It is fair as long as that meets the demands.

For replication, the product we chose, we had been told that the product were being used by many fortune 500 companies and wall street shops were really happy with the product.

The reality is, we found a number of problems time to time, which we did never expect from such a "high profile" product. It seems that they never had an extensive user like us or they were lagging for testing resources, sounds funny though!

* During the transition phase, one special feature of the product swapped our data between two servers and the databases got out of sync. It was really really hard to understand which data was more correct. To correct the problem, I had to go through about each and every 7000 records - one by one :-(

* Initially, the LOB replications were problematic - after replication, it inserted invalid and empty spaces in between each and every characters on destination database. We could not believe this!

* Again, after 2 months, I discovered another replication problem with CLOB data type. At destination server, after replication, some character got truncated. This problem specially happens with Multibyte character set. We reported the problem but "they" did not understand what was happening. On this issue, I had to spend 3/4 days to analyze, reproduce and pin-point the problem and I described why and how this is happening with a big explanation like a detective - the company were not able to find the cause of the problem initially.

* After few months, again discovered problem with BLOB replication. The product generates some internal files after capturing changes. For some bug in their product, what I saw is, for 20 MB data size in BLOB field, the product were generating internal files of size more than 20 GB! So, one night we caught for out of disk space!

It seems that we were doing extensive testing for their product and coming up with bugs to fix.

Well, I am not going to tell you which third party replication product we are using. But what I want to point out is - one should test any third party product (my recommendation) if they think that they are doing something vary important for their customers. We can't always rely on what other says or how big the product profile is!

Database Replication with GoldenGate software

For a number of reasons - people replicate data to remote destinations. We do have our own needs. We have been running SharePlex for data replication purpose for the last 3 years. By this time, our system usages increased more than twice in just last year. Initially SharePlex was good for us but now a days we need more flexible configurations and different replication architecture for the system.

We are looking for an alternative database replication mechanism that would suit best for us. We are exploring new technologies available to serve replication purpose.

Now, we found two alternatives to be considered as we need more control for an "Active-Active" mode -

1. Oracle Streams
2. GoldenGate Software

I did several extensive experiments with Oracle Streams for about 1.5 months. You can guess, there are lots of outcomes. This post is for telling a bit about my GoldenGate experience!

GoldenGate Software (GGS) is a popular one used for data replication and keep several database in sync. They are supporting a number of database including Oracle, Teradata, DB2, Sybase, SQL Server. It seems that GGS are in market for long time, probably before Streams introduced by Oracle, now they are running version 9.5.

The basic concepts of GGS are - GGS has EXTRACT process which captures changes in source database. The changes are kept in files called TRAIL. The changes can be directly sent to the target/destination database's REPLICAT process to apply the changes in destination database. GGS EXTRACT reads data from oracle online redo log file - so it's independent of oracle process. I forgot to mention, GGS has several processes, MANAGER is one of the important process which managers all other stuffs.

My first "one way" replication test configuration (obvious these are not all for production db) with GGS were as follows -


Source Neptune: Create EXTRACT which will capture changes and send the changes to remote host's TRAIL files

EDIT PARAMS extprod

EXTRACT extprod
USERID ggs, PASSWORD ggs
RMTHOST saturn, MGRPORT 7820
RMTTRAIL /export/home/oracle/ggs/dirdat/rt
TABLE DEV.LOGIN;

**

ADD EXTRACT extprod, TRANLOG, BEGIN NOW

**
ADD RMTTRAIL /export/home/oracle/ggs/dirdat/rt, EXTRACT extprod, MEGABYTES 50

Destination Saturn: Create REPLICAT which will read changes from TRAIL files and apply the changes in current database

EDIT PARAMS repprod

REPLICAT repprod
ASSUMETARGETDEFS
DISCARDFILE ./dirrpt/REPSOE_DSC.rpt, PURGE
USERID ggs, PASSWORD ggs
MAP DEV.*, TARGET DEV.*;

**
ADD REPLICAT repprod, EXTTRAIL /export/home/oracle/ggs/dirdat/rt, NODBCHECKPOINT

So, these were the basic stuffs that I started with GoldenGate Software (GGS). They have thousand page documents and there are a lot more to test. Earlier, we created spreadsheet putting about 30 evaluation criteria against which we compared SharePlex and Oracle Streams. After experimenting with GGS, I will put the outcomes for GGS and would take the ultimate product decision.

Expecting comments if you have any good/bad experience with the product!