X

Welcome to All Things Data Integration: Announcements, Insights, Best Practices, Tips & Tricks, and Trend Related...

Recent Posts

GoldenGate Technical Features

Connect to Oracle Cloud Infrastructure GoldenGate Using a Private IP and Oracle Cloud Infrastructure Bastion

Oracle Cloud Infrastructure (OCI) GoldenGate is a fully managed cloud service that customers can use to ensure that their valuable data is always available and analyzable in real-time.  Best of all, Oracle Cloud Infrastructure GoldenGate is built on the industry leading, highly trusted real time data technology – Oracle GoldenGate. By default, OCI GoldenGate deployments are only accessible using a private endpoint from machines running inside the OCI network, or through a bastion host which secures access to OCI resources. You can create your own bastion host using a compute instance and manage it yourself, or you can use the new OCI Bastion service! OCI Bastion provides restricted and time-limited access to target resources that don’t have public endpoints. Bastions are essential in tenancies with stricter resource controls. Targets can include resources like Compute instances, Bare Metal and Virtual Machines, MySQL, ATP, OKE v2 and any other resource which allows SSH access. In just a few easy steps, you can use OCI Bastion to connect to your OCI GoldenGate Deployment Console using a private IP. What You Need Ensure that you have the following in place: A free trial or paid Oracle Cloud Infrastructure account Access to OCI GoldenGate and OCI Bastion Required IAM policies added for both OCI GoldenGate and OCI Bastion VCN and subnet details for the target OCI GoldenGate deployment Create the bastion Log in to OCI Console. From the navigation menu, click Identity & Security, and then select Bastion. In Bastion, select the appropriate compartment, and then click Create Bastion. In the Create Bastion screen: Enter a Bastion Name. Use the same VCN as the target OCI GoldenGate deployment you want to access. Pick a subnet which has access to the subnet in which the OCI GoldenGate deployment resides. In the CIDR Block Allowlist, enter a CIDR block that includes the IP address of the machine you will be using to access the Bastion and connect to the deployment console in OCI GoldenGate. It could be your laptop IP address for example. Click Create Bastion Create the session After the bastion is active, create an SSH port forwarding session: Click Create Session, complete the fields as follows, and then click Create Session: For Session Type, select SSH port forwarding session. Optionally, you can give it a name. For IP Address, enter the OCI GoldenGate deployment's private IP. You can find the private IP on the OCI GoldenGate deployment's Details page. For Port, enter 443. For Add SSH Key, provide the public key file of the SSH key pair to use for the session, or create a new pair. After the session is created, from the session's Actions (three dots) menu, select Copy SSH Command. Paste the command into a text editor, and then replace the <privateKey> and <localPort> placeholders with the OCI GoldenGate deployment's private IP and port 443. You can also add -v to get some debugging information when connecting, this can be useful for troubleshooting. Run the command using the command line interface to create the tunnel. Open a web browser and go to localhost:443. Accept the security exception, if prompted, and then continue to the Deployment log in page. Enter your OCI GoldenGate Deployment credentials and proceed! Learn more Setting up an OCI Bastion and session to access a secured OCI GoldenGate deployment is quick and easy. If you want to learn more about OCI GoldenGate or OCI Bastion, check out: OCI GoldenGate OCI GoldenGate Documentation OCI Bastion Simplify secure access with OCI Bastion OCI Bastion Documentation Contributor: Jenny Chan, Database User Assistance

Oracle Cloud Infrastructure (OCI) GoldenGate is a fully managed cloud service that customers can use to ensure that their valuable data is always available and analyzable in real-time.  Best of...

Data Integration

Get Going with Oracle Cloud Infrastructure Data Integration

We hope you have been finding the Oracle Cloud Infrastructure Data Integration blogs helpful as you get started with the service: Workspace in Oracle Cloud Infrastructure (OCI) Data Integration Understanding VCN Configuration for Oracle Cloud Infrastructure (OCI) Data Integration Data Asset in Oracle Cloud Infrastructure (OCI) Data Integration Project Setup in Oracle Cloud Infrastructure (OCI) Data Integration Data Flow overview in Oracle Cloud Infrastructure (OCI) Data Integration   Integration Tasks in Oracle Cloud  Infrastructure (OCI) Data Integration More are coming!   We also wanted to point you to some related exploratory Oracle Cloud Infrastructure Data Integration blogs from David Allan.  Thanks David! Oracle Cloud Infrastructure Data Integration and Python SDK Explores the first Oracle Cloud Infrastructure Data Integration API in action to list workspaces. The example see here is how to list workspaces in a compartment. Executing Tasks using Python SDK in Oracle Cloud Infrastructure Data Integration Tasks Uses the Oracle Cloud Infrastructure Data Integration Python SDK to execute a task which has been published to an application. Oracle Cloud Infrastructure Data Integration and Fn Shows how to use Functions when you want to focus on writing code to meet business needs; the example is the ‘hello world’ example for Data Integration.  This example will be extended in subsequent posts illustrating integration with other services in OCI such as the Events Service and Notification Service. Automate Loading Data to a Data Lake or Data Warehouse Using OCI Data Integration and Fn Explains how multiple services from Oracle Cloud Infrastructure work together to load data into the Data Lake or ADW, leverage Oracle Cloud Infrastructure Data Integration, Fn and Events Service to automate the load.   Happy reading!

We hope you have been finding the Oracle Cloud Infrastructure Data Integration blogs helpful as you get started with the service: Workspace in Oracle Cloud Infrastructure (OCI) Data Integration Understandi...

Oracle Cloud Infrastructure Data Integration

Understanding VCN Configuration for Oracle Cloud Infrastructure (OCI) Data Integration

Let's learn more about Oracle Cloud Infrastructure Data Integration. Today's blog will help you understand and teach you Virtual Cloud Network (VCN) configuration for Oracle Cloud Infrastructure Data Integration. Check out the previous blog written on Oracle Cloud Infrastructure Data Integration about Workspaces. Overview of Virtual Cloud Network (VCN) A virtual cloud network (VCN) is a customizable and private network in Oracle Cloud Infrastructure. Just like a traditional data center network, the VCN provides complete control over the network environment. This includes assigning own private IP address space, creating subnets, route tables, and configuring stateful firewalls. VCN resides within a single region but can cross multiple Availability Domains. Once users, groups, and compartments are created then start with VCN creation.  By default, there are two subnets in the VCN (Region Specific).  Private Subnet - Instances contain private IP addresses assigned to Virtual Network Interface Card (VNIC) Public Subnet - Contains both private and public IP addresses assigned to VNICs For more understanding of VCN can refer to - https://docs.cloud.oracle.com/en-us/iaas/Content/Network/Concepts/overview.htm Oracle Cloud Infrastructure Data Integration and Virtual Cloud Networks Now coming to the main topic "Understanding VCN with Oracle Cloud Infrastructure Data Integration". Oracle Cloud Infrastructure Data Integration is in the Oracle Tenancy which resides outside the user tenancy. For Data Integration to access the resources in the user tenancy and get the information related to VCN and subnets the following policy needs to be set at the compartment level/tenancy level i.e. policy set at default root compartment level. allow service dataintegration to use virtual-network-family in tenancy (or) allow service dataintegration to use virtual-network-family in compartment <vcn_compartment> Different Options when Creating a Workspace While creating workspaces there are two options provided i.e. Enable Private Network or using Public Network. Oracle Cloud Infrastructure Data Integration only supports regional subnets i.e. subnet across all Availability Domains. Regional subnets are used for high availability purposes.  While in the process of creating a Workspace using "Enable Private Network", Oracle Cloud Infrastructure Data Integration VCN gets extended with the user-selected VCN. When the option is not selected then Oracle Cloud Infrastructure services like Object Storage get accessed through Service Gateway defined at the tenancy level and the rest of the resources like Database are accessed through Public Internet. Let us consider multiple Scenarios to understand the Oracle Cloud Infrastructure Data Integration with VCN by selecting Private/Public subnet and accessing its resources. Before testing multiple scenarios following are pre-requisites created in the environment: Created VCN with the name "VCN_DI_CONCEPTS" in the respective compartment.   Created four subnets within the mentioned VCN. Oracle Cloud Infrastructure Data Integration only supports regional subnet. For more information on the regional subnets, refer to https://docs.cloud.oracle.com/en-us/iaas/Content/Network/Tasks/managingVCNs.htm Below is the list of resources created belonging to Subnet and Region while Testing   For Autonomous Data Warehouse (ADW) to be in private instance Network Security Group (NSG) needs to be defined. In NSG defined two ingress rule for PUBLIC_SUBNET_DI (10.0.2.0/24) and PRIVATE_SUBNET_DI (10.0.1.0/24)   For DB Systems in Private subnet, the following rules in the ROUTE table are included   For Service Gateway, select the option "All IAD Services in Oracle Services Network". To understand more about this option, refer https://docs.cloud.oracle.com/en-us/iaas/Content/Network/Tasks/servicegateway.htm   Scenario 1 - Accessing ADW, Object Storage, and Databases in the same Region using DI workspace in Private Subnet Oracle Cloud Infrastructure Data Integration workspace was created in PRIVATE_SUBNET_DI (10.0.1.0/24) Service Gateway used in the PRIVATE_SUBNET_DI     Scenario 2 - Accessing ADW, Object storage in different regions, and accessing Database Systems residing in a public subnet. To access ADW in different regions and DB Systems in public subnet a NAT Gateway is required.  Service Gateway is required for Object storage along with NAT Gateway for cross traffic. Route Rules screenshot(added NAT Gateway with the existing Service Gateway):     Scenario 3 - Accessing ADW, Object Storage, and Database in the same Region using DI workspace in Public Subnet OCI DI Workspace in Public Subnet "PUBLIC_SUBNET_DI" (10.0.2.0/24) Depending on the requirement if Oracle Cloud Infrastructure Data Integration Workspace has been assigned in a VCN and wants to connect resources residing in another VCN which might be in the same region or different region then Local or Remote peering is required accordingly. To understand more about Local or remote peering, refer https://docs.cloud.oracle.com/en-us/iaas/Content/Network/Tasks/VCNpeering.htm. If the resources are having Public IP then NAT or Service Gateway can be used accordingly. Scenario 4 - ADW, Object Storage, Database systems residing in the public subnet and all these resources are in different tenancy, different region, and different VCN To test this scenario we have created the following resources in the Mumbai region and different tenancy. The workspace of Oracle Cloud Infrastructure Data Integration is in Public Subnet (10.0.2.0/24). DI Workspace is created in the Ashburn region.   Scenario 5 - Connecting ADW, Databases and Object Storage using DI workspace with "Enable Private Network" Disabled While creating workspace if the option "Enable Private Network" is not selected   This non - enabling option means public connectivity option is selected where the Oracle Cloud Infrastructure Data Integration can access all the public services using Service Gateway and NAT Gateway from Oracle Tenancy. Here, Oracle Cloud Infrastructure Data Integration can't access private resources since for the workspace no VCN is assigned. In this example, Oracle Cloud Infrastructure Data Integration is enabled in the Ashburn region.     Scenario 6 - Connecting Oracle Cloud Infrastructure Data Integration with On-Premise DB There are two methods where Oracle Cloud Infrastructure Data Integration can connect to On-Premise DB IPSec VPN FastConnect Below are the details on how using FastConnect Oracle Cloud Infrastructure Data Integration can access the Database. To understand more about FastConnect, refer https://docs.cloud.oracle.com/en-us/iaas/Content/Network/Concepts/fastconnect.htm   Oracle Cloud Infrastructure Data Integration workspace should be in the same subnet where FastConnect is configured.   In the below example, VCN is created by Oracle as part of FastConnect with the name "####-iad.vcn"   Regional Public subnet is created within the VCN   Dynamic Route Gateway (DRG) is configured which is used as a virtual router that provides a path for private traffic (that is, traffic that uses private IPv4 addresses) between user VCN and networks outside the VCN's region. For more information on DRG, refer - https://docs.cloud.oracle.com/en-us/iaas/Content/Network/Tasks/managingDRGs.htm   DRG can be configured with IPSec or Oracle FastConnect   Within the DRG two virtual network have been configured using FastConnect   Route Rules defined in the VCN   OCI DI workspace created in the subnet   Under Data Asset Create and Test the connection   Summary - We can observe that Scenario 1 and Scenario 2 are the same irrespective of Subnet allocated to the workspace.  Since the secondary VNIC extended to the users VCN/tenancy is always Private. Oracle Cloud Infrastructure Data Integration Workspace is assigned to Public or Private Subnet Oracle Cloud Infrastructure Data Integration Workspace is not assigned any network - Disabled "Enable Private Network" Option We just recently announced the general availability of Oracle Cloud Infrastructure Data Integration. With a series of upcoming blogs, we look forward to introducing various concepts. This concludes our blog on how to use VCN in Oracle Cloud Infrastructure Data Integration. To learn more, check out some Oracle Cloud Infrastructure Data Integration Tutorials and the Oracle Cloud Infrastructure Data Integration Documentation.

Let's learn more about Oracle Cloud Infrastructure Data Integration. Today's blog will help you understand and teach you Virtual Cloud Network (VCN) configuration for Oracle Cloud Infrastructure Data...

Oracle Cloud Infrastructure Data Integration

Workspace in Oracle Cloud Infrastructure (OCI) Data Integration

Oracle Cloud Infrastructure Data Integration is a fully managed, serverless, native cloud service that helps you with common extract, load, and transform (ETL) tasks such as ingesting data from different sources, cleansing, transforming, and reshaping that data, and then efficiently loading it to a target system on Oracle Cloud Infrastructure. Before you get started, the administrator must satisfy connectivity requirements so that Oracle Cloud Infrastructure Data Integration can establish a connection to your data sources. The administrator then creates workspaces and gives you access to them. You use workspaces to stay organized and easily manage different data integration environments. The workspace is the preliminary component of Oracle Cloud Infrastructure Data Integration. The workspace acts as an environment provided where the user can work on multiple Projects, Publish/Run Tasks, and Define Data Assets. The administrator must define the policies for the users/groups to start with this data integration solution. Creating and Editing a Workspace: Pre-requisites - All the necessary compartments and VCN have been created for Data Integration activities.  To understand more about VCN for Oracle Cloud Infrastructure Data Integration, refer to https://docs.cloud.oracle.com/en-us/iaas/data-integration/using/preparing-for-connectivity.htm Create a group for users in charge of workspaces and then add users to the group. All the policies have been set up by the administrator so that the user can access the Oracle Cloud Infrastructure Data Integration. If the administrator wants to limit activities within the network, "inspect" permission for VCNs and subnets within the compartment has to be provided instead of "manage". Below is the list of policies required to access Oracle Cloud Infrastructure Data Integration Give permissions to the group to manage Oracle Cloud Infrastructure Data Integration allow group <group_name> to manage dis-workspaces in compartment <compartment_name> Give permission to the group to manage network resources for Workspaces allow group <group_name> to manage virtual-network-family in compartment <compartment_name> Give permission to the group to manage tag-namespaces and tags for Workspaces allow group <group_name> to manage tag-namespaces in compartment <compartment_name> Oracle Cloud Infrastructure Data Integration is located in Oracle Tenancy which is outside user Tenancy. Data Integration sends a request to user tenancy. In return, the user must give the requestor(DI) permission to use the virtual networks set up for integration. Without a policy to accept this request, data integration fails. These policies can be defined at the compartment level or the tenancy level i.e. at the root compartment level allow service dataintegration to use virtual-network-family in tenancy allow service dataintegration to inspect instances in tenancy   Select the Data Integration link from the main menu of Oracle Cloud Infrastructure   Select the corresponding compartment and click on "Create Workspace".   Provide necessary information i.e. Name for the Workspace, VCN details, and other information like DNS, Tag Names.   Click on create for creating the workspace in the corresponding compartment. You're returned to the Workspaces page. It may be a few minutes before your workspace is ready for you to access. After it's created, you can select a Workspace from the list.   You can see the status of a Workspace creation or startup using View Status. It is available while creating or starting a Workspace from Stopped Status   Workspace can be accessed in two ways as shown in the below picture   After accessing the workspace New Projects, Data Assets or Applications can be created through the main console of the Workspace   You can edit the Workspace details, such as a name or description. You can't make changes to the identifier, compartment, VCN, or subnet selections. To edit the tags applied to a Workspace, select Add Tags from the Workspace's Actions (three dots) menu. In the Console, you edit a workspace from the Workspaces page. Select Edit from a workspace's Actions (three dots) menu. Edit the fields you want to change, and then click Save Changes Terminating/Stopping a Workspace -  Only Workspaces with an Active status or in Stopped status can be terminated. When you terminate a workspace, all the associated objects and the resources are removed. Below is the list of resources: Projects Folders Data Flows Tasks Applications Task Runs Data Assets All executions within a Workspace must be stopped before you can terminate the Workspace. Any open tabs associated with the Workspace you're terminating are closed upon termination. Once terminated, a Workspace cannot be restored. Be sure to carefully review the Workspace and resources before you commit to a termination. To terminate the Workspace click on the workspace action(three dots) and then click on Terminate   We just recently announced the general availability of Oracle Cloud Infrastructure Data Integration. With a series of upcoming blogs, we look forward to introducing various concepts. This concludes our initial blog on how a Workspace can be created and used in Oracle Cloud Infrastructure Data Integration.  To learn more, check out some Oracle Cloud Infrastructure Data Integration Tutorials and the Oracle Cloud Infrastructure Data Integration Documentation.

Oracle Cloud Infrastructure Data Integration is a fully managed, serverless, native cloud service that helps you with common extract, load, and transform (ETL) tasks such as ingesting data from...

Data Integration

Oracle Named 2019 Gartner Peer Insights Customer Choice for Data Integration Tools

We are pleased to announce that Oracle has been recognized as a 2019 Gartner Peer Insights Customer Choice for Data Integration Tools.  This distinction is especially important to Oracle because it is based on the direct feedback from our customers.  Thank you all for your support! Oracle Data Integration provides an enterprise class, fully unified solution for building, deploying, and managing real-time data-centric architectures. It combines all the elements of data integration—real-time data movement, transformation, synchronization, data quality, data management, and data services—to ensure that information is timely, accurate, and consistent across complex systems.  By using Oracle Data Integration, customers can experience significant cost savings, and efficiency gains are critical to leverage in today's challenging global economic climate.  They are delivering real-time, enriched, and trusted data from disparate cloud and on-premises sources to enable insightful analytics.  “We are honored to receive Gartner Peer Insights Customers’ Choice designation for the Data Integration Tools market. We thank our customers for their support,” said Jeff Pollock, Vice President Product Management for Oracle. “Over the past 20 years Oracle Data Integration has evolved into an industry leading platform used by thousands of companies across every industry. Working together with our customers, Oracle is committed to driving the innovation necessary to solve the industry’s most challenging data integration issues.”  Find out more! Gartner Peer Insights is an enterprise IT product and service review platform that hosts more than 300,000 verified customer reviews across 430 defined markets. In markets where there is enough data, Gartner Peer Insights recognizes up to seven vendors that are the most highly rated by their customers through the Gartner Peer Insights Customers’ Choice distinction. According to Gartner, “The Gartner Peer Insights Customers’ Choice is a recognition of vendors in this market by verified end-user professionals.” To ensure fair evaluation, Gartner maintains rigorous criteria for recognizing vendors with a high customer satisfaction rate. We at Oracle are deeply proud to be honored as a 2019 Customers’ Choice for the Data Integration Tools Market. To learn more about this distinction, or to read the reviews written about our products by the IT professionals who use them, check out the Customers’ Choice Data Integration Tools for Oracle landing page on Gartner Peer Insights. Here are some excerpts of what Oracle Customers are saying: “Using GoldenGate, it is possible to carry out operations in high data volumes in a much faster and uninterrupted manner. It is also very easy to use. One of the most effective abilities is to manage transactional processing in complex and critical environments. It is very important that data, costs and ongoing transactions are regularly secured to bring the risk to near zero." Software Engineer, Finance Industry “ODI is a very good product. It is lightning fast (which really comes handy when we have to transform massive amount of data), It ability to support heterogeneous databases, big data, JMS, XML, and many other flavors.” Senior Manager - MIS & Middleware, Service Industry A big thank you to our wonderful customers who submitted reviews, and those customers who continue to use our product and services and help shape the future.     The GARTNER PEER INSIGHTS CUSTOMERS’ CHOICE badge is a trademark and service mark of Gartner, Inc., and/or its affiliates, and is used herein with permission. All rights reserved. Gartner Peer Insights Customers’ Choice constitute the subjective opinions of individual end-user reviews, ratings, and data applied against a documented methodology; they neither represent the views of, nor constitute an endorsement by, Gartner or its affiliates.

We are pleased to announce that Oracle has been recognized as a 2019 Gartner Peer Insights Customer Choice for Data Integration Tools.  This distinction is especially important to Oracle because it is...

GoldenGate Solutions and News

Oracle GoldenGate Plug-in for Oracle Enterprise Manager v13.2.3.0.0 is now available

We have released GoldenGate OEM Plug-in 13.2.3.0.0. The release's primary focus was to support the monitoring of Oracle GoldenGate 18.1, 19.1 Microservices (MA) Instances. In the earlier GoldenGate OEM Plug-in 13.2.2.0.0 release, we started supporting our first GoldenGate 12.3 Microservices Instance. In the new release, we have certified the latest of GoldenGate releases 18.1 and 19.1 Microservices and Classic. Along with the certification, we are supporting the new metrics for coordinated and parallel replicats. We have provided more services(Administration Service and Service Manager) and Deployments support in the plug-in. You may discover the new targets in the discovery module and promote the targets of your choice. Finally, we have certified the OEM 13.3 in the release.   Once you discover targets, you can select the processes(Extract, Replicat, etc) while promoting the targets. The selected processes(aka targets) and its parent process would get promoted automatically. For example, if you select the Extract process under Admin Server, the OEM PlugIn will promote the selected Extract process, Admin Server (which is Extract’s Parent), and Service Manager (which is Admin Server’s Parent). On a similar line, if you select the parent process, all the children will be selected by default and then you may choose to de-select the particular child.       Once you promote the processes or targets, you may notice the changes in Dashboard User Interface for Microservices processes. All the processes are shown in the tree structure. The Service Manager is the parent process, which shows one or many GoldenGate Deployments, all the extracts and Replicats are part of Admin Server. You may see all the services status on the screen. Along with the Microservices Instance, you may monitor the Classic Instance on the same dashboard. We have given each process(Target) a specific type name as per GoldenGate terminologies. It will be helpful when you want to know about what type of Extract or Replicats you are monitoring (Classic or Integrated Extract, Coordinated or Parallel Replicat).     When you click on the Service Manager on the dashboard, it will direct you to the below-mentioned page. The page shows all the Deployments and the details of its services, such as Port and Status. In the future, you should be able to search across a particular Deployment. The admin Server page will show all the extract and Replicats processes and its detailed metrics. When you click on individual Extract, Replicat you will be able to see the Metrics, Logs, and Configuration of the process.     For the Parallel and Coordinated replicat(PR/CR), you can see the accumulated metrics in the Parent process. The children process of the PR/CR is not visible on the screen. In the future, we will provide options to select the child process and then you would be able to monitor those children as well.   The GoldenGate OEM Plug-in has upgraded infrastructure to be compatible with the newer version of Enterprise Manager (EM) to 13.3.0.0.0. As mentioned earlier, we have certified the GoldenGate 18.1, 19.1 Classic and Microservices and added few metrics related to Parallel Replicat and Coordinated replicat.    Just to recap the communication between EM Agent and GoldenGate MA, and Classic Instances. You would not require to setup GoldenGate jAgent(Monitor Agent) to communicate with GoldenGate OEM Plug-in for GoldenGate Microservices Instances. The GoldenGate MA architecture provides the RESTful APIs to monitor and manage the GoldenGate MA Instances. The GoldenGate OEM Plug-in uses these RESTful APIs to communicate with GoldenGate MA Instances. For your GoldenGate Classic Instances, you would still need to setup GoldenGate jAgent 12.2.1.2.0+ for the communication purposes. The latest Monitor Agent was released on May, 19 (12.2.1.2.190530).   You can get more details of the release from the documentation.    We are working to get more features around monitoring the GoldenGate Microservices and Classic architecture in future releases. Please stay tuned for further updates.  

We have released GoldenGate OEM Plug-in 13.2.3.0.0. The release's primary focus was to support the monitoring of Oracle GoldenGate 18.1, 19.1 Microservices (MA) Instances. In the earlier GoldenGate OEM...

Release Announcement for Oracle GoldenGate 19.1

This post was authored by Bobby Curtis, Director of Product Management, Oracle What’s New in Oracle GoldenGate 19.1 To succeed in today’s competitive environment, you need real-time information. This requires a platform that can unite information from disparate systems across your enterprise without compromising availability and performance. Oracle GoldenGate 19c is a high-performance software application for real-time transactional change data capture, transformation, and delivery, offering bidirectional data replication. The application enables you to ensure that your critical systems are operational 24/7, and the associated data is distributed across the enterprise to optimize decision-making. GoldenGate 19.1 Platform New Features For the Oracle Database ➢ Oracle Database 19c Support Capture and Delivery support for Oracle Database 19c, cloud and on-premises. ➢ Centralized Key Management Service Use Oracle Key Vault to centralize and manage encryption keys for the replication environment. ➢ Target-Initiated Paths Distribution paths enabled from the Receiver Service to pull trail files. ➢ New REST API Endpoints Retrieve active transactions and current system change number (SCN) details using REST API endpoints. ➢ New Heartbeat Table Command The UPGRADE HEARTBEATTABLE command upgrades the Heartbeat table from prior versions of Oracle GoldenGate to the 19.1 version. ➢ Cross-Endian Support for Remote Integrated Extract Automatically enabled when the server where the Integrated Extract is running is different from the server where the Oracle Database is running. For MySQL ➢ MySQL 8.0 Support for Capture and Delivery Capture and Delivery support for MySQL 8.0 has been added. ➢ MySQL SSL Connection Support Extract and Replicat can now connect to a MySQL database via SSL. For DB2 for i ➢ Enhanced TIMESTAMP Support Supports all valid TIMESTAMP precisions ➢ New Datatype Support Support for DECFLOAT datatype. ➢ New DBOPTIONS USEDATABASEECODING Parameter Allows Extract to store all text data in the trail file in native character encoding. ➢ Improved Extract Performance Enhanced throughput while reducing overall processing. ➢ Security Improvements Availability of AES Encryption. Credential Store, and Oracle Wallet. ➢ Long Running Transaction (LRT) Support Support for LRT features showtrans, skiptrans, forcetrans. For DB2 z/OS ➢ Enhanced TIMESTAMP Support Supports all valid TIMESTAMP precisions ➢ Online Schema Change Support Support for online TABLE CREATE, DROP and ADD, ALTER, DROP COLUMN commands. ➢ Long Running Transaction (LRT) Support Support for LRT features showtrans, skiptrans, forcetrans. For DB2 LUW ➢ Enhanced TIMESTAMP Support Supports all valid TIMESTAMP precisions ➢ New Datatype Support Support for DECFLOAT datatype. ➢ Long Running Transaction (LRT) Support Support for LRT features showtrans, skiptrans, forcetrans. Other Information ➢ In the initial release, OGG 19.1.0.0.0, Linux builds will be available for most Database/OS combinations that are supported, followed by tiered releases for other supported platforms. ➢ GoldenGate for SQL Server will be released for both Windows and Linux soon, in a 19.1.x release. Docs, Downloads, and Certification: • Documentation is available at: https://docs.oracle.com/en/middleware/goldengate/core/19.1/index.html • Downloads are available through OTN at: https://www.oracle.com/middleware/technologies/goldengate.html • Certification Matrix (19.1 Cert Matrix to be posted soon): https://www.oracle.com/technetwork/middleware/ias/downloads/fusion-certification-100350.html Join us in upcoming events: • Stay up to date by visiting our Data Integration Blog for up to date news and articles. • Save the Date! Oracle OpenWorld is September 16th through the 19th. Don’t hesitate to contact us for any special topics that you might like to discuss. 

This post was authored by Bobby Curtis, Director of Product Management, Oracle What’s New in Oracle GoldenGate 19.1 To succeed in today’s competitive environment, you need real-time information. This...

GoldenGate Solutions and News

Zero Down Time (ZDT) Patching for Oracle GoldenGate

  This document explains how to apply a patch or upgrade an OGG environment without taking any downtime.  This assumes that OGG is already up and running and that the user is already very familiar with how OGG works, and the actual upgrade process.  Like any mission critical, 24x7 environment, this expectation is that the user takes the necessary precautions to test this process prior to implementing it in production, and is aware of any differences between versions.  All of these items are covered in other documents.  Terminology “New” – This refers to the new OGG installation.   This “new” environment is where everything will be running once you have completed the procedure. “Old” – This refers to the old OGG installation.  This “old” environment is the existing OGG installation that you want to upgrade.  After the process is completed, you will be removing this installation. Patching OGG Homes where there are Extract(s) running. Install the new OGG version into a new directory. This location will be referred to as the “new” OGG installation. In the new installation Apply any necessary patches to bring the releases to the most recent bundle patch, and then apply any required one-off patches on top of that. Create new Extract process(es) with different names than the old OGG environment. Create a new set of trail files (different names than the old OGG installation. Copy the parameter files from the old installation into the new one.  Modify them to account for new directories, names, and address any deprecated / modified parameters. On the target Create a new Replicat to receive data from the new OGG installation. In the new Installation Start the Extract Start the Extract pump (if necessary) In the old installation Wait.   How long to wait for?  It depends.  When you started the new Extract in step 4a, it will not process any transactions that were open when it was started.  You will want to wait until any open transactions during that time are closed.  SEND EXTRACT … SHOWTRANS may help in this case. Stop the Extract On the target If the old Replicat is not using a checkpoint table ,add one for it. Once the Replicat from the old installation is at EOF (SEND REPLICAT … GETLAG) stop the old replicat. Start the new replicat using START REPICAT … AFTERCSN [scn].  Where the [scn] is the log_cmplt_csn column from the checkpoint table for the old replicat. This will tell the new replicat to pick up right where the old replicat left off. In the old installation Stop the extract pump (optional) Clean up the old installation and remove it.   Patching OGG Homes where there are Replicat(s) running. Install the new OGG version into a new directory. This location will be referred to as the “new” OGG installation. In the new installation Apply any necessary patches to bring the releases to the most recent bundle patch, and then apply any required one-off patches on top of that. Create new Replicat process(es) with different names than the old OGG environment.  The new replicat will read from the existing trail files. Copy the parameter files from the old installation into the new one.  Modify them to account for new directories, names, and address any deprecated / modified parameters. In the Old installation. If the old Replicat is not using a checkpoint table ,add one for it. Stop the Replicat when it is at EOF (SEND REPLICAT … GETLAG) In the New Installation Start the new replicat using START REPICAT … AFTERCSN [scn].  Where the [scn] is the log_cmplt_csn column from the checkpoint table for the old replicat. This will tell the new replicat to pick up right where the old replicat left off. In the old installation Clean up the old replicat and remove it.  

  This document explains how to apply a patch or upgrade an OGG environment without taking any downtime.  This assumes that OGG is already up and running and that the user is already very familiar with...

Demystifying Oracle Cloud Infrastructure

Oracle has a longstanding reputation for providing technologies that empower enterprises to solve demanding business problems. Oracle has built a cloud infrastructure platform that delivers unmatched reliability, scalability, and performance for mission-critical databases, applications, and workloads. Oracle Cloud Infrastructure is the first cloud built specifically for the enterprise. With the latest high-end components, support for open standards and multi-cloud strategies, and an unwavering commitment to protecting sensitive business data, Oracle Cloud Infrastructure is perfectly suited to meet the needs—and exceed the expectations—of today's enterprise IT teams. Oracle Cloud Infrastructure represents a fundamentally new public cloud architecture and serves as the foundational layer for Oracle Cloud. The infrastructure is designed to provide the performance predictability, core-to-edge security, and governance required for enterprise workloads. Oracle supports traditional, mission-critical, and performance-intensive workloads typically found in on-premises environments, including artificial intelligence (AI), machine learning (ML), and high-performance computing (HPC), as well as cloud-native applications. Oracle Cloud Infrastructure combines the benefits of public cloud (on-demand, self-service, scalability, pay-for-use) with those benefits usually associated with on-premises environments (governance, predictability, control) into a single offering. Here is a good example of how Alliance Data Saves $1 Million Annually Running Critical Applications on Oracle Cloud Infrastructure Learn more about Oracle Cloud Infrastructure here.

Oracle has a longstanding reputation for providing technologies that empower enterprises to solve demanding business problems. Oracle has built a cloud infrastructure platform that delivers unmatched...

Enabling Analytics with Oracle data integration and Oracle Analytics Cloud on Oracle Autonomous Database

Enabling global analytics is one of the most common use cases among customers who build and maintain a data store. In this post, we shall identify the critical components that are required for an end-to-end analytics solution, the characteristics of a great analytics solution, and how Oracle Analytics Cloud, Oracle data integration, and Oracle Autonomous Database, together combine to provide a platform for great analytics. Any chosen analytics solution should bring together and balance the requirements of two major stakeholders, those in the Information Technology (IT) departments and those in the line-of-business functions. Fig 1: IT and Business dictates priorities that need to be balanced in an analytics solution Achieving this balance between the scalability requirements of IT and the user experience focus of a visual tool is critical to the success of any visualization solution. Oracle Data Integration -  The IT Component Oracle data integration solutions help solve key requirements for a successful IT deployment of an analytics solution. Oracle data integration Provides the latest data, both in real-time and bulk, from various sources to be delivered into the data warehouse that is built on top of Oracle Autonomous Database to power analytics, Helps govern data and provide transparency to the data that underpins the analytics visualizations, for easy lineage and impact analysis, thus increasing trust in the data, and Enables true global analytics, by making data available both on-premises and on the cloud. Oracle Analytics Cloud - The Business Component Oracle Analytics Cloud provides the features and benefits that satisfy the requirements of a business user. Oracle Analytics Cloud Provides powerful data flows and enrichment features to enable sharable and traceable business data transformations, Avoids Excel clutter and empower analysts to enhance data with no coding skills required, and Enables augmented data enrichment, through Machine Learning driven enrichment and data transformations. Oracle Autonomous Database - The Platform  Oracle Autonomous Database forms the third component of this analytics solution, along with Oracle data integration and Oracle Analytics Cloud. Oracle Database Provides a robust self- driving, self-securing, and self-repairing data store, providing autonomous datawarehousing capabilities, Watch this video to understand how these three components come together to provide end to end analytics on an Oracle platform. Fig 2: Oracle data integration video  Oracle Data integration, along with Oracle Autonomous Data Warehouse and Oracle Analytics accelerates speed to insight and innovation while enabling fast access to a sophisticated set of analytics and accelerates data preparation and enrichment. Watch this webcast to learn more about how to focus on growing your business and drive innovation with an end-to-end analytics solution.

Enabling global analytics is one of the most common use cases among customers who build and maintain a data store. In this post, we shall identify the critical components that are required for...

Data Integration

Loading Data Into Oracle Autonomous Data Warehouse Cloud with Oracle data integration

Oracle offers the world’s first autonomous database. Oracle also offers tools that helps customers get data into the autonomous database.  In this blog, we will go through what is an Autonomous Database and the capabilities that Oracle data integration provides that helps adopt the Autonomous Data Warehouse Cloud Service (ADWCS). What is an Autonomous database? An autonomous database is a cloud database that uses machine learning to eliminate the human labor associated with database tuning, security, backups, updates, and other routine management tasks traditionally performed by database administrators (DBAs). Autonomous database technology requires that enterprise databases be stored in the cloud, using a cloud service. Being autonomous in the cloud allows the organization to leverage cloud resources to more effectively deploy databases, manage database workloads, and secure the database. A database cloud service makes database capabilities available online, when and where those capabilities are needed. Watch Senior Vice President Juan Loaiza introduce the Oracle Autonomous Database for a deeper insight into the technology. What is Oracle data integration? Oracle’s data integration encompasses a portfolio of cloud-based and on-premises solutions and services that helps with moving, enriching, and governing, data. Oracle data integration has the following capabilities that make it the logical choice when looking to migrate and move data to Oracle Cloud. Oracle data integration Has integrated APIs that allow easy access to Oracle’s underlying tables without affecting source system performance for real-time data access through change data capture, Can automate repeated data delivery into Oracle Datawarehouse Cloud Service by easily surfacing ADWCS as a target system, Brings together real-time data replication, data streaming, bulk data movement, and data governance into a cohesive set of products that are seamlessly integrated for performance. Watch this video to get a quick glimpse of our latest product and how it functions with Oracle Data Warehouse Cloud and Oracle Analytics cloud. Moving Data Into Oracle Data Warehouse Cloud Service Oracle data integration solutions bring together some key technological and user benefits for customers. Managed by Oracle – Engineered and built by teams that have a shared vision, the different solutions and technologies incorporate the best of scientific advances, as well as, seamless integration between the solutions. Unified Data Integration – Provides a single-pane-of-glass to control the various components of data integration like bulk data movement, real-time data, data quality, and data governance. Simplify Complex Integration Tasks – Groups together functions that build up to a business or technology pattern, so that often repeated scenarios can be executed with efficiency. Flexible Universal Credit Pricing – Oracle’s pricing tracks usage, and can be applied across technologies, allowing customers access to all participating Oracle cloud services, freeing customers from procurement woes and providing customers with a truly agile and nimble set of solutions. Here are some scenarios that Oracle data integration helps to solve. Extraction & Transformation - Execute bulk data movement, transformation, and load, scenarios, Data Replication – Change data capture helps replicate data into Oracle Autonomous DataWarehouse and Kafka, for data migration and high availability architectures, Data Lake Builder - Create a comprehensive, fully governed, repeatable data pipeline to your big data lakes, Data Preparation - Ingest and harvest metadata for better data transparency and audits, and Synchronize Data - Seamlessly synchronize two databases together. Fig1: A sample architecture of moving data from source to analytics For a deeper understanding of moving data into Oracle Autonomous Data Warehousing Cloud, watch the below webcast.

Oracle offers the world’s first autonomous database. Oracle also offers tools that helps customers get data into the autonomous database.  In this blog, we will go through what is an Autonomous...