Setup Single Sharded Oracle NoSQL Database: Quickly and Easily

This blog is part 2 of the series on using Oracle NoSQL Database on-premises. Last time we saw how to get started with KVlite can be accomplished in under 5 mins! In this edition, we’ll look at setting up a single Oracle NoSQL Cluster. In this blog, we will look at how to set up a single sharded cluster with 3 Replica Oracle NoSQL Database Cluster.  

Oracle NoSQL Cluster Architecture

Before we dive into setting up an Oracle NoSQL Database Cluster, it is essential to have a look at high-level architecture and various components that form its underpinnings.

Oracle NoSQL Database reads and writes data by performing network requests against an Oracle NoSQL Database data store (“store” for short), referred to as the KVStore. The KVStore is a collection of one or more Storage Nodes, each of which hosts one or more Replication Nodes. Data is automatically spread across these Replication Nodes by internal KVStore mechanisms.

The store can contain multiple Storage Nodes. A Storage Node is a physical (or virtual) machine with its local storage. The machine is intended to be commodity hardware. While not a requirement, each storage node is typically identical to all other Storage Nodes within the store. Every Storage Node hosts one or more Replication Nodes as determined by its capacity. A Storage Node’s capacity serves as a rough measure of the hardware resources associated with it. We recommend configuring each Storage Node with a capacity equal to the number of available disks on the machine. Such configuration permits the placement of each Replication Node on its disk, ensuring that Replication Nodes on the Storage Node are not competing for I/O resources. Stores can contain Storage Nodes with different capacities, and Oracle NoSQL Database ensures that a Storage Node is assigned a proportional load size to its capacity.

The store’s Replication Nodes are organized into shards. You can think of shard as a placeholder for a subset of the sharded data. A single shard contains multiple Replication Nodes. Each shard has a replica master node or a Paxos elected master node as shown in the figure below. The master node performs all database write activities. Each shard also contains one or more read-only replicas. The master node copies all new write activity data to the replicas. The replicas are then used to service read-only operations.

While there can be only one master node per shard at any given time, any of the other shard members can become a master node. An exception to this is for nodes in a secondary zone. 

When an application asks to retrieve the record for a given key, the Oracle NoSQL Database driver will hash a portion of the key (denoted as “the shard key” during table creation) to identify the shard that houses the data.  Once the shard is identified, the Oracle NoSQL Database driver can choose to read the data from the most optimal replica in the shard, depending on the requested consistency level.  With respect to write operations, the Oracle NoSQL Database driver will always route the write requests to the leader (Elected Leader in Figure 1) node of the shard.  Hence, from the perspective of workload scaling, the application can generally think of this architecture as being scaled by adding shards.  Oracle NoSQL Database supports online elastic expansion of the cluster by adding shards.

High Level Architecture Oracle NoSQL Database                           

Oracle NoSQL Database Cluster Deployment

 At a high level deploying a NoSQL database cluster requires the following steps : 

1. Bootstrap Storage Node (SN): The physical machine or the storage node that we are going to use needs to be bootstrapped first. This process will create a config file in the KVRoot Dir that holds the cluster deployment info.

2. Start Storage Node Agent (SNA): After bootstrapping, we start the Storage Node Agent or SNA that works like a listener that enables the communication between the admin node and other replication nodes (RN). We run bootstrapping and start SNAs on all the storage nodes. 

3. Configure the NoSQL Database store :

   • Name your Store

   • Create a Data Canter(Zone)

   • Deploy the storage nodes and start the admin nodes.

   • Create the storage node pool and assign the storage node

   • Create the deploy the replication nodes.

We are now set to start the deployment. It’s important that users read the installation prerequisites listed in the documentation.

Download the kv-ee-<version>.tar.gz (.zip)  server software from our OTN page and unpack it into a directory. Refer to our doc about Installing and Prerequisites. Set the following environment variables on all the storage nodes.

• KVHOME points to the directory where you unzipped the latest Oracle NoSQL Database binaries
• KVROOT where you would like metadata files to be stored.
• KVDATA to the storage directories.

Bootstrap the Storage Nodes:

In this step, we bootstrap the storage nodes that deployed as part of the Oracle NoSQL Database cluster. We ship a utility called makebootconfig that will do the bootstrapping job for us. I suggest reading the utility document to understand the various options that are available. Remember, we are setting a single sharded NoSQL DB cluster with a Replication Factor of 3 spread across three different storage nodes (kvhost01, kvhost02, kvhost03) with a capacity of 1 each. We are setting up an unsecured NoSQL Cluster, but you can setup a secured cluster by configuring the security related configuration parameter in the makebootconfig. We bootstrap the storage nodes one by one.

Bootstrap storage node 1

Bootstrap storage node 2

Bootstrap storage node 3

Start Storage Node Agent(SNA)

Once the Storage Node is bootstrapped the next step is to start the Oracle NoSQL Database Storage Agent(SNA). Notice that we are passing KVROOT of the SN to the start command.

This step needs to be done on each of the storage nodes. At this stage, users can use the ping to verify if that Oracle NoSQL Database client library can contact the SNA. If the Storage Nodes do not start up, you can look through the adminboot and snaboot logs in the KVROOT directory to identify the problem.

Configure the Oracle NoSQL Database store

Assuming the Storage Nodes all started successfully, you can configure the KVStore. To do this, you use the CLI command interface. Start runadmin:

Follow the steps below:

• Name your KVStore

The name of your store is essentially used to form a path to records kept in the store. Here we are calling it mystore: 

• Create a Zone

Once you have started the command line interface and configured a store name, you can create at least one zone (or datacentre). In this example we are creating zone called “Boston” with replication factor of 3 using plancommand

• Deploy all the storage node to the zone and start the admin nodes

Every Oracle NoSQL Database store has an administration database. You must deploy the Storage Node(SN) to which the command line interface is currently connecting to, in this case, “sn1”, “sn2” and “sn3” and then deploy an Administration process on that same node.

• Create the storage node pool and add storage nodes to the pool

Once the storage nodes are deployed we create a storage node pool. This pool is used to contain all the storage nodes in the store. In this example we call this pool “BostonPool”  and then join storage nodes to the pool

• Create and Deploy the Replication nodes

The final step in your configuration process is to create Replication Nodes on every nodes using the topology create and deploy-topology commands. 

As a final sanity check, you can confirm that all of the plans succeeded using the show plans command.

Having done that, you can exit the command line interface and you are done.