This blog covers the technical details on Global Active Tables’ underlying architecture, configuration, and operation. If you are interested in the overview, value propositions, and business use cases, please refer to this blog.

 

I love both words, “SIMPLE” and “COMPLEX”. I find them intriguing. They are antonyms semantically. Yet, I see they are deeply connected with the “SIMPLE” and “COMPLEX” overlapping strings hidden in both words, pairing well with my take on design challenges: “simplicity hides complexity” and “complexity hides simplicity.” Both are hard nuts to crack! You strive for one while unraveling the other.

Oracle NoSQL’s Global Active Table is the new feature with such a feat!

It unravels the underlying technical complexity and offers absolute simplicity to users, delivering instant agility for global business data access.

Introduction

In today’s competitive global business climate, enterprises must innovate fast and offer mobile-first responsive applications to users while operating 24/7 around the globe. Complexity and daunting efforts arise when deploying and maintaining global applications in multiple regions to deliver a highly responsive user experience.

Imagine a mobile application company that serves its customers in three different regions: Phoenix (U.S.), Johannesburg (South Africa), and Tokyo (Japan). The following diagram depicts a family with three members using mobile devices in three different regions. Two members are on extended business trips to South Africa and Japan separately. These users expect a fast and smooth online experience while using the mobile application, which offers predictable low latency by accessing the data stored in each local region.

When a member updates the user profile via a smartphone in Johannesburg, the new data is immediately replicated in Phoenix and Tokyo. All members see the latest change in their local region. Similar replication will be performed when a member update happens in Phoenix or Tokyo. Suppose all members’ updates happen in all regions at the same time. The cross-region data replication must handle the complexity of data conflict, ensuring that the correct data is present in all regions. Another challenge is what if an unlikely catastrophe affects one of the data centers, causing it to go down.

The above scenario merely scratches the surface of the complexity and daunting efforts facing businesses that offer such a service to their user base. Here are some additional challenges:

• Mitigate the impact of an unexpected downtime in a specific region.
• Implement multi-region data backup and restore measures. 
• Minimize the integration effort of different cloud services for such an enablement. 
• Resolve data conflicts when updates are performed in multiple regions simultaneously.
• Manage auto cleanup of expired data in all regions.
• Decide the suitable data model for multi-active global applications, keeping primary / shard keys uniform across regions.
• Manage variants of regional database operations, e.g., local writes/reads.
• Deliver write operations to multiple regions in the order that those write operations were performed at the source region. 
• Manage operational complexity of the underlying infrastructure, achieving 24/7 high availability and resiliency at scale.
• Globally replicate data with no data loss and no affect on the latency and throughput of ongoing operations.
• Struggle with trade-offs between investment costs versus reliability, performance, security, and efficiency.

Meeting all these challenges is a huge endeavor!

Global Active Table is a powerful feature with intelligent cross-region data replication architecture, hiding all the underlying complexity. Things happen “magically” with Global Active Tables for businesses implementing:

• Multi-active geo-distributed applications to serve global users with predictable single-digit millisecond latency experience.
• Business continuity and disaster recovery to minimize business disruptions during a catastrophe.
• Blue-green deployment for applications and data.

Endless possibilities!

 

The Beauty

It is simply slick! Hidden beneath Global Active Tables is an architecture called Cloud Data Pipelines, with workflows automating cross-region data replication.

Users Experience Simplicity

You configure a NoSQL table as a Global Active Table (GAT) in multiple regions with just a few clicks or lines of code. Voila! You made your business application data accessible globally right away.

One service, one billing, and all the complexity mentioned earlier are behind you! You can fully focus on innovations, business opportunities, and growth, meeting your customers’ needs. Doing more with less is the mantra in the current global business climate. Scaling up or down Global Active Tables is extremely simple, with the underlying infrastructure resource consumption automated transparently.

 

NoSQL Manages Complexity

NoSQL’s Global Active Tables unravel these complexities, offering absolute simplicity. Oracle NoSQL Cloud Service manages automated workflows replicating data between the sender’s and receiver’s regional table replicas, resolving complex challenges. Let’s peek into the underlying mechanism that conceals the complexity of users.

 

Global Active Table Architecture Overview

Cloud Data Pipelines 

This “magical wand” makes things simple at the user level, managing the complex coordination of data replication processes from the sender’s to the receiver’s region. The underlying architecture does all these wonders while meeting the challenges mentioned earlier.

Conceptual Components

• Global Active Table: A group of multi-active regional table replicas with identical schemas replicated across different OCI (Oracle Cloud Infrastructure) regions. Table operations performed at any regional replica are replicated to participating regions.  Each regional table replica in the group can act both as a source of changes and a receiver of changes.  You can select which table to configure as a Global Active Table, using the provisioned or on-demand capacities model. 

• Sender: The actor at the source region of the data replication stream where local changes occurred, managing the sending process.

• Receiver: The actor at the destination region of the data replication stream, managing the receiving process.

• Exchange Zone: The intermediate storage mechanism for table operations data and metadata during the replication.

• Replication Transports: A network of conduits between the sender and receiver regions, replicating the table data between regions.

 

Automated Workflow 

Cloud Data Pipelines ensures data changes from table operations at the sender’s region are replicated to the receiver’s region. The diagram above shows a bidirectional replication where a given region acts as a sender and a receiver, offering active-active cross-region replication. Let’s dive into the complexity of the workflows managed by NoSQL.

The replication workflows as depicted in the picture above:

• 1 – Global Active Table represents the source regional table replica.  It captures data changes from application local write operations (insert, update, delete) and serves local read operations. Those table operations trigger a stream operation.

• 2 – Sender monitors local data changes in the Global Active Table and prepares replication packages, storing them in the intermediate storage. Sender orchestrates the replication protocols with Receiver and Exchange Zone, tracks the replication streaming progress, and manages intermediate storage.

• 3 – Exchange Zone stores and secures key data and metadata that facilitate the replication process between Sender and Receiver.  The following are kept until the replication is completed:
  • Snapshot data, metadata, and status for initial replication of existing data in Global Active Table.
  • Cross-region requests, responses, and receivers consumed status.
• 4 – Replication Transports provide a network of securely encrypted conduits for replication streaming between the source and destination regions.

• 5 – Receiver monitors and fetches the replication data and metadata in Exchange Zone, replaying table operations at the receiver’s regional table replica. Receiver and Sender continuously communicate, update the status, and manage the housekeeping during the replication process.

• 6 – Regional Table Replica represents the receiving regional table replica.  It captures data changes by replaying table operations at the receiver’s region.

 

More Built-In Power Features

• Fault-Tolerant Regional Replication
  • Oracle NoSQL Database Cloud Service replicates NoSQL tables in three OCI Availability Domains / Fault Domains by default at each OCI region for high availability, including Global Active Tables. More details.

• Conflict Resolution Last Write Wins
  • In a multi-active application scenario, when concurrent update operations are performed simultaneously on the same record in Global Active Tables at multiple regions, the workflows automatically detect and resolve the conflict, replicating the latest changes at all regions. More details. 

• Conflict Free Replicated Data Type
  • Global Active Table offers the Positive-Negative Counter CRDT (Conflict-Free Replicated Data Type). This data type can be implemented in Global Active Tables across different regions, enabling applications to perform update operations independently in each region. Data changes in Global Active Tables are concurrent and not synchronized. The CRDT algorithm allows concurrent changes to be merged across regions, converging to a uniform common state among all Global Active Tables. More details.

• Time to Live
  • An intelligent auto-cleanup mechanism that enables applications to set a time frame on table rows, after which the rows expire and are removed automatically. When Time to Live (TTL) is enabled in Global Active Tables, all table replicas across regions have the same value as the table’s TTL in the sender’s region. When a table row is replicated to other regions, its expiration time is replicated as an absolute timestamp, enabling such row to expire and be removed in all the regions simultaneously, irrespective of when it was replicated. TTL process does not consume write/read capacity at any region. It is free of charge. More details. 

• Unique Record-Level Identity
  • Global Active Tables can use the UUID data type for creating a unique record-level identity, removing the complex effort of creating universal primary/shard keys across participating regions. Whenever a new record is inserted, a unique UUID-based identity is auto-generated for the regional table replica. More details. 

• On-demand and Provisioned Capacity Models
  • Global Active Tables build flexibility in capacity planning per your workloads at the table level. The provisioned capacity mode reserves the write and read capacity limits in advance per the expected application workloads. Billing is based on the write and read capacity reserved. The on-demand capacity model dynamically adjusts the write and capacities needed by your applications in real-time. Billing is based on the actual write and read capacities consumed. Depending on your business use cases, both models offer cost-effective options. More details.

 

Interfaces for Global Active Tables

Global Active Table offers a variety of interfaces:

• NoSQL OCI Consoles
• Programmatic interfaces
  • NoSQL SDKs 
  • OCI Terraform
  • OCI SDKs
  • OCI REST APIs

This blog showcases NoSQL OCI Consoles and Java SDK programmatic interface, offering a sneak peek at configuring Global Active Tables within two simple steps. This companion blog delves deeper into programmatic interfaces.

 

NoSQL OCI Consoles

This interface configures Global Active Tables without writing any code. In just 2 steps and a few clicks, you can experience Global Active Tables’ “magic” right away.

Configuration Flow Chart

Following is the process to create a Global Active table in two steps:  1- create a new NoSQL table in a local region 2- add the table replica to a remote region.   Once configured, all local update operations in either region will be replicated to the other region automatically. 

 

The following diagram shows the configuration steps on the left and the table state changes to Global Active Tables on the right. This illustration uses Ashburn, U.S. East as region 1 and Phoenix, U.S. West as region 2.

Step 1: Create a table

Proceed to Oracle NoSQL Database Cloud Service console. This step starts with an empty table list in Ashburn’s console.

Click Create table followed by clicking Advanced DDL input. Use the default Provisioned Capacity model, fill in Read capacity, Write capacity, Disk storage(GB).

In the DDL input box, enter the following to create a new table with schema frozen.

CREATE TABLE demoGlobalTable (id INTEGER,personInfo JSON,PRIMARY KEY(id)) with schema frozen

Click Create table to create a singleton table called demoGlobalTable with Schema state equals  “Frozen” and Global active equals “No” in the Ashburn region.

 

Step 2: Add replica

Click demoGlobalTable on the tables list to Table information tab. Click Replicas on the left sidebar to pop up Add replica dialog box.

Select us-phoenix-1 from the Replication region‘s drop-down menu.

Click Add replica to configure Global Active Tables for both Ashburn and Phoenix regions.

You can start inserting records in either region and watch data replicated to the other region. 

 

Programmatic Interface

The following Java SDK example configures Global Active Tables with just a few lines of codes. Using the same parameters in the console example above,

• tableName equals demoGlobalTable
• region equals us-phoenix-1

Step 1 uses the DDL statement to create a new singleton table in Ashburn region.

Step 2 uses the Java class AddReplicationRequest() to add a table replica in Phoenix region.

For more programmatic interfaces, refer to this companion blog.

 

Summary 

I n s t a n t   A g i l i t y .     A b s o l u t e   S i m p l i c i t y .    E n d l e s s   P o s s i b i l i t i e s .   

Once again, Oracle NoSQL Cloud Service pushes the boundary of business agility and data accessibility at the global scale. Global Active Tables enable business applications to deliver a highly responsive user experience and always-on availability to meet the demands of digital citizens wherever and whenever they choose. Oracle NoSQL Cloud Service manages all the complexity while enterprises reap the benefits of simplicity and cost efficiency.

In an increasingly competitive global climate, every second counts. Your business’s success and survival depend on innovations, solution delivery, and operation measured by speed, efficiency, reliability, and costs. When new opportunities emerge, businesses can’t afford complexity. With Global Active Tables, enabling cross-region data replication is just a snap at the speed of your business.

Imagine deploying the following data platform. Enterprise solution architects must tread through a maze of products and services, open or closed-source, and derive a framework. With Global Active Tables, deployment can be created, torn down, re-configured, rebuilt, or expanded to meet different scenarios in minutes. The companion blog covers different programmatic interfaces for making this agility possible.

Give Global Active Tables a try! Here’s the getting started guide.