Review these blogs before optimizing TimesTen synchronous replication on Oracle Database Appliance:
This blog will show you:
The Oracle Database Appliance X8-2-HA has two compute nodes, each of which has both external and internal network interface cards [NICs]. The internal/private network interface cards are designed to be used as a dedicated cluster interconnect for Oracle Real Application Clusters and Oracle Clusterware.
When the Oracle Database Appliance X8-2-HA is used with Oracle TimesTen for a highly available Active Standby Pair replication configuration and read/write caching for an Oracle database, the private network interface cards should be used for TimesTen replication.
The following TimesTen replication configuration for Oracle Clusterware will automatically use the private network addresses:
Even though only the public hostnames [eg odoemoda0 and odoemoda1] are specified, the TimesTen software will use the Oracle Clusterware C API to determine what the interconnect network addresses are and use those addresses. This will result in the following TimesTen replication scheme:
This replication scheme uses default values for the store attributes. If you want to explicitly define the replication routes and/or store attributes, then you need to use the RepDDL syntax in the cluster.oracle.ini config file:
This RepDDL will create the following TimesTen repscheme:
The above RepDDL syntax does the following:
The RepDDL syntax give the full power of the TimesTen create active standby pair syntax within the cluster.oracle.ini config file.
TimesTen Replication enables many different configurations and behaviors.
If you cannot afford to lose any transactional data [eg financial transactions] then your should use two safe synchronous replication.
TimesTen Replication for an Active Standby Pair is logically transmitting the committed SQL Inserts, Updates and Deletes from the active database to the standby database. With TimesTen two safe synchronous replication, the SQL commits actually occur first on the standby database and then on the active database.
TimesTen replication uses several components:
If the transmitter thread can always read from the transaction log buffer and the receiver thread can always read from the transaction log buffer, then replication can have both great throughput and latency as it is not doing any disk reads or writes.
The goal of TimesTen Replication tuning is to configure the transaction log buffer and transaction log files so that no (or minimal) disk reads and writes occur. By default the transaction log files are written to asynchronously from the transaction log buffers.
It turns out that sequentially writing to a single socket for replication is a bottleneck for the TimesTen In Memory Database. Instead, TimesTen enables parallel replication:
TimesTen Parallel Replication does the hard stuff for you:
TimesTen Parallel Replication is configured via setting attributes in the sys.odbc.ini file. Attributes of interest are:
These values are all interrelated.
The LogBufMB attribute defines how large the Transaction Log Buffer should be in Megabytes:
The LogBufParallism attribute defines how many logical log buffers there should be:
The LogFileSize attribute should always be the same value as LogBufMB!
The TempSize attribute defines an area of memory used for things like SQL ORDER BY and GROUP BY operations. It also is used as a work area to determine the commit dependencies.
The CacheAWTParallism attributes defines how many Oracle Net Services connections are used to write committed SQL Insert, Update and Delete statements to an Oracle Database [single instance, RAC or Exadata]. Depending on the workload, TimesTen uses either OCI batching or PLSQL blocks to send the writes to the Oracle Database.
The ReplicationParallism attribute defines how many Replication transmitter and receiver threads can be used concurrently:
Using more transmitter threads [each with their own log buffer and TCP socket] should provide linear scalablity for TimesTen Replication right? Unfortunately there is a pesky thing called Amdahl's Law.
It basically says that a parallel algorithm cannot go faster than its sequential tasks:
Your workload's commit dependencies will determine the effective parallelism for TimesTen Replication.
Complex business logic will tend to reduce the effective Replication Parallelism. Simple workloads tend to enable greater replication Parallelism.
The following example sys.odbc.ini file is used for two safe synchronous parallel replication. It uses:
Disclaimer: These are my personal thoughts and do not represent Oracle's official viewpoint in any way, shape, or form.