X

The Oracle NoSQL Database Blog covers all things Oracle NoSQL Database. On-Prem, Cloud and more.

  • May 4, 2021

New Concepts in NoSQL: Introducing a conflict-free replicated data type

Michael Brey, and Dario Vega

Handle Real-Time Data in a Geo-Distributed Context

Business is changing quickly; customers are demanding a better application experience and increased satisfaction.  As a result, businesses are adapting to changing characteristics:

  • Data growth - doubling
  • Demand for different types of data - growing
  • Real-time data – rising
  • Secure access – a must-have

We showed you how Oracle addresses the speed and latency aspects of those characteristics in previous blog posts.

  • Speed matters: Customers enjoy and expect a fast browsing experience. Even slight page-loading delays can hurt business. Your site needs to be responsive and deliver page loads with low latency in today's global economy.
  • Low Latency: A NoSQL geo-distributed solution enabling predictable low latency and response time from anywhere in the world.

We demonstrated in this blog how to build a geo-distributed application easily with predictable low latency from a technical perspective. We return to that series, where we revisit the power of the Multi-Region Tables feature provided by Oracle NoSQL.

A Multi-Region table is a global logical table that eliminates the problematic and error-prone work of replicating data between regions, enabling developers to focus on application business logic.  All updates performed in one region are automatically propagated to all the other regions. But what about Conflict Detection and Resolution?

Conflict-Free Replicated Data Types

Implementing an active-active replication solution is not trivial. The key to success lies in real-time data movement, conflict detection and resolution, and support for heterogeneous environments. Of the three, conflict detection and resolution introduces the most complexity.

When multiple systems are processing transactions and the activity is shared across the systems, detecting and addressing conflicts becomes an essential requirement for any active-active replication configuration.

This blog presents a new feature called conflict-free replicated data type (CRDT).

CRDTs are a family of replicated data types with a common set of properties that enable operations performed on them to always converge to a correct and consistent common state among all replicas. To ensure that conflicts never happen and cause problems in your applications, operations on CRDTs must respect a specific set of arithmetic properties.

There are multiple types of CRDTs and the first type that we have released in Oracle NoSQL Database is the Positive-Negative (PN) Counter type. Other CRDTs may be built in the future.

To start, let’s go through an example based on our first blog of this series. We talked about a common telco use case - offering a "Family Plan." With this option, an account owner and their family share the data usage on the plan. The account is allocated a data usage limit for a month that the entire family collectively uses. When the total data usage across all family members sharing the account reaches 90 percent of the data limit, the telecom provider will send an SMS alert to the account owner.

For this example, let's say that there are four family members and these family members are in different geographic regions, separated by a large distance. The account owner gets an alert from the telecom provider when the account reaches the 90 percent threshold, notifying the account owner that overage fees may be charged if usage goes beyond the contracted limit. 

The data is replicated in different regions, so each family member gets a local view of the data improving latency, throughput, and overall performance. That means there are multiple regions, and each has a copy of the data containing the details of the customer's data usage. Their individual usage needs to be updated in different regions, and at any point in time, the total usage can be monitored.

A CRDT data type is ideal in this situation to do conflict-free tracking of the data usage across different regions. In the above example, an increment counter in a multi-region table will track the data usage in that region. The consolidated data usage is automatically computed for you when the CRDT value is read from Oracle NoSQL Database.  Oracle NoSQL will automatically merge the delta values from each region.

In our telco family plan use case, the telecom provider can simply read the value of the “dataUsed” CRDT column, compare this against the contracted amount for the account, and then decide to raise an SMS alert if necessary.  No complex book keeping or reconciliation is necessary for the telecom provider to implement.

This is all done for them by Oracle NoSQL Database!

MR_COUNTER data type  

Oracle NoSQL Database has introduced a new CRDT datatype, called MR_COUNTER, for use with multi-region tables. In a multi-region table setup, a CRDT is a data type that can be replicated across servers where regions can be updated independently.  It is always possible to converge the CRDT to a correct common state. Changes in the regions are concurrent and not synchronized with one another. In short, CRDTs provide a way for concurrent modifications to be merged across regions without user intervention. The MR_COUNTER datatype is a subtype of the INTEGER, LONG, or NUMBER data type.

To ensure that conflicts never happen, operations on CRDTs must respect the commutative properties of addition. That is, CRDT updates must only be via addition or subtraction. In the case of Positive-Negative (PN) Counter:

  • Only the DEFAULT value is authorized when inserting a new row.  Assigning an explicit value to the CRDT field during insert will throw an error because the commutative property will break.
  • Only increment and decrement operations are authorized when updating a row.

Conflict-Free Replicated Data Types - Example

Let's walk through a detailed example showing you how CRDTs work.

Creating the table

Create a table called users with an MR_COUNTER data type in both the regions, Frankfurt and London.

 

Describing the table

 

Insert Data

While data is inserted in a multi-region table with an MR_COUNTER column, the system generates a default value of 0 for the MR_COUNTER column value in the following two cases:

  • When the keyword DEFAULT is used in the insert_clause for the MR_COUNTER column.
  • When the MR_COUNTER column is skipped in the INSERT clause.

 

Insert data - Common Error

You cannot insert values into an MR_COUNTER column explicitly. If you try to insert values into the MR_COUNTER column using the INSERT clause or using API, an error is thrown, as shown below.

 

CRDT in action

MR_COUNTER ensures that though data modifications happen simultaneously in different regions, the data can always be merged into a consistent state. Just do a SELECT.

 

Add or Remove an MR_COUNTER column

You can alter an existing multi-region table to add an MR_COUNTER column. The existing records in the table will have a value the default value for the newly added MR_COUNTER column. You can also remove an existing MR_COUNTER column from a multi-region table.

 

Summary

In this blog, we discussed how vital conflict detection and resolution is in an active-active replication.

  • Oracle NoSQL Multi-Region solution enabling predictable low latency and response time from anywhere in the world
  • Oracle continues to improve the developer experience by introducing CRDTs that perform automatic merging and bookkeeping of value update across regions, alleviating the pain of multi-region reconciliation from your app development experience.

A Multi-Region table is a global logical table that eliminates the problematic and error-prone work of replicating data between regions, enabling developers to focus on application business logic.

Explore more

  • Check out the documentation
  • Read our next blog providing more use-cases for CRDT

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.