Oracle CDC: Why you should care

Implementing CDC on Oracle is not a one-size-fits all solution, it varies according to specific business needs. There are multiple Oracle CDC options on the market from premium to cost-effective solutions. Each one comes with its own set of challenges and benefits. As the precarious global economy forces us to work under tighter budgets, we need to rethink our solutions and meet the demands of the current market.

In this article, we’ll explore just that, how we implemented Oracle CDC, in order to help our customers overcome these overwhelming economic constraints. 

Change Data Capture 101

Change Data Capture, or CDC as it’s commonly known, is a method that identifies and captures the changes made to the data, enabling seamless integration between a data source, such as an operational database and a target destination, such as a data warehouse. Instead of transferring the entire dataset every time a change is made, CDC only moves the changes, significantly reducing the volume of data transfer and enhancing efficiency. This distinction is critical for businesses that require near real-time data integration, without overwhelming their systems or wasting bandwidth.

One of the key benefits of CDC is its reduced strain on resources. Unlike traditional bulk data transfer methods which can burden systems, affecting performance, especially during peak usage times,CDC operates with minimal impact. It only transfers the altered data, not the entire dataset.

CDC also facilitates tracking changes in near real-time, helping businesses make timely decisions. It allows for a more agile response to data-driven insights which is pivotal in verticals like FinOps, healthcare and retail. Let’s dive in!

Making Change Data Capture Flexible

The complexities of data integration are undeniable. When it comes to simplifying data replication and data extraction processes, our ICP’s needs are always top of mind. So jobs to be done is our focus and that’s exactly why we offer support for both CDC and standard SQL extraction. It gives you total control, so you can decide which method best suits your business use case. You can easily compare the two methods here.   . 

Investing in flexibility and meeting our customers’ needs has been a top priority for us since day one. Our customers were seeking to optimize their Oracle-based data operations and to support the demand, we decided to invest in our CDC engine and extend our capabilities to support Oracle. Before we jump into that in more detail, let’s look at the Oracle CDC options on the market. 

Oracle CDC: What Are the Options?

Implementing CDC on Oracle isn’t a one-size-fits-all solution; there are multiple avenues you can explore. One prominent method is through Oracle’s proprietary software, Oracle GoldenGate, which provides real-time data integration. Oracle XStream and utilizing Oracle’s Redo logs are the other available options. Let’s take a close look:

• Oracle GoldenGate: This is Oracle’s premium solution for real-time data integration and replication. It facilitates the capturing, routing, transformation, and delivery of transactional data across heterogeneous environments in real-time. However, users have to keep in mind that it requires separate licensing. 
• Oracle XStream: Oracle’s XStream offers APIs that enable applications to consume and produce transactional data changes in the Oracle database. There are two primary components: XStream Out, which captures database changes, and XStream In, which applies changes to the database. However, Oracle announced the deprecation of XStream in favor of Oracle GoldenGate. Starting with Oracle Database 19c, Oracle began steering users away from XStream and started promoting GoldenGate as the preferred solution for data replication and change data capture.
• Redo logs with LogMiner: Redo logs record all changes made to the database, and the LogMiner utility allows these changes to be queried. This method is cost-effective and efficient, especially for those who may not have the resources or need for a premium solution like GoldenGate.

Rivery’s Approach to Oracle CDC 

There are significant licensing costs involved with Oracle GoldenGate and the price to pay is too high for most businesses. Businesses that are currently facing huge economic constraints, with data teams that are burnt out and need a solution that works for them not against them. 

That really forced us to rethink our current CDC capabilities. Instead of relying on off-the-shelf solutions, we adapted our CDC engine for Redo logs with LogMiner. So what does this mean for our customers and for you? 

It helps you achieve production-grade replication from Oracle databases without the need to pay the hefty GoldenGate license. It’s time for a change in the market and we want to shift things by offering fair CDC solutions without unnecessary vendor lock-ins and license fees. The focus should be on CDC agility and scalability, and a CDC engine that makes your life easier.

How Rivery’s Oracle CDC Works

Rivery’s Oracle CDC solution is designed to be both user-friendly and powerfully robust. At its core, it capitalizes on the potential of Redo logs and LogMiner to capture database changes, ensuring that data integration is as real-time as possible.

Setting up Rivery’s solution requires minimal prerequisites. The only essentials are:

• Enabling ARCHIVELOG mode within the Oracle database, which ensures that all changes to the database data files are saved and can be reapplied in a recovery scenario.
• Activation of supplemental logging in Oracle, a necessity to guarantee the complete capture of information from Redo logs for LogMiner.

With these prerequisites met, users are primed to initiate the Rivery Oracle CDC solution. They can be effortlessly set up via a source-to-target river, which will consistently monitor changes made to the chosen Oracle tables. Every DML change – be it an insert, update, or delete operation—will be captured and seamlessly propagated to target systems like Snowflake or Databricks.

Under the hood, Rivery handles the intricacies with precision. The platform manages LogMiner sessions, ensuring no data is overlooked. It keeps a vigilant eye on the rotation of archived logs in real-time, safeguarding against data loss. The solution also diligently tracks the System Change Number (SCN), ensuring the continuity of data capture and ensuring any changes are promptly relayed to the target systems.

Rivery’s Oracle CDC workflow:

As the visual workflow represents, every Oracle data value change triggered by a user is captured by Oracle’s Redo logs which are then archived every 5 minutes. Rivery’s CDC engine, will query the Redo logs via LogMiner, store the results within the customer file zone, parse the values and finally propagate the data to the target database. 

Build vs. Buy an Oracle CDC Solution

Implementing CDC replication on Oracle is not without its challenges. Users often grapple with:

• Complex Configuration: Setting up CDC can be intricate and time-consuming.
• Cost Concerns: Leveraging premium solutions might strain budgets.
• Real-time Replication: Achieving true real-time data replication can be elusive.

Beyond these primary concerns, there are several edge cases, network concerns, session management intricacies, and other unpredictable issues that can arise when working with CDC and Oracle. However, during our implementation phase, Rivery’s Oracle CDC solution has been battle-tested against these very challenges. Through rigorous QA tests, we have encountered, addressed, and rectified a multitude of potential issues, ensuring that our clients don’t have to.

Thus, when users adopt Rivery’s CDC solution, they are not only leveraging a platform that simplifies configuration, but one that has been hardened against the very edge cases and challenges they hope to avoid. By using Redo logs with LogMiner, we provide a cost-effective, resilient, and efficient method for real-time replication. This ensures our users can focus on data utilization rather than the intricacies of data streaming, confident in the reliability of the data flow.

Limitations of Oracle CDC

While CDC replication offers numerous advantages, it’s crucial to be aware of its limitations when implemented on Oracle. These include potential performance impacts and considerations around database versions and compatibility.

Some aspects to consider when working with Oracle CDC include:

• Data Availability: Reading from the archive logs as they transition from online to archive, could add a lag. It’s not uncommon for the Database to take up to 5 minutes to update the archived logs, which could introduce a slight delay in reflecting the most recent changes.

• Table and Column Names: LogMiner, which is pivotal for this CDC implementation, has a limitation when it comes to table and column naming conventions. Specifically, it requires table and column names to be 30 characters or less. Adhering to this constraint is crucial to ensure smooth CDC operation and avoid unexpected errors during the process.

• Data Types: Oracle CDC has certain restrictions concerning data types. Notably, it does not support the following data types: LONG, LONG RAW, BFILE, BLOB, CLOB, NCLOB, XMLTYPE, EMOJI types.

Oracle CDC Wrap Up 

The benefits of implementing CDC replication on Oracle with Rivery are manifold. It’s not just about data integration; it’s about efficiency. All it takes is a few clicks to reap the benefits. 

Ready to try it out for yourself? Get started with $1,200 in free credits 

Want to learn more about our CDC engine? Our docs are a great place to start.