Wednesday Jul 20, 2016

Oracle GoldenGate Application Adapters 12.2.0.1.2 Released

Oracle GoldenGate Application Adapters 12.2.0.1.2 for JMS and Flatfile was released on 19/July/2016. 

New Features in this release:

  • DDL propagation for heterogeneous relational databases
  • Metadata in trail for both Extract and Replicat processes
  • Oracle Java standard based Java Package names

You now can download the software from OTN or eDelivery.

Oracle GoldenGate Application Adapters 12c (12.2.0.1)  Documentation

Oracle GoldenGate Adapter for Confluent Kafka Connect

Oracle GoldenGate Adapter/Handler for Kafka Connect (open-source) is released on 07/Jul/2016.

Summary

Kafka Connect is a tool for streaming data between Apache Kafka and other data systems in a scalable and reliable way. Kafka Connect can ingest entire databases or collect metrics from all your application servers into Kafka topics, making the data available for stream processing with low latency. An export connector can deliver data from Kafka topics into secondary indexes like Elasticsearch or into batch systems such as Hadoop for offline analysis. (Source: Confluent Website)

This Kafka Connect handler lets you to integrate using the Kafka Connect API which can be managed using Confluent Platform. The Kafka Connect Handler takes change data capture operations from a source trail file and generates data structs (org.apache.kafka.connect.data.Struct) as well as the associated schemas (org.apache.kafka.connect.data.Schema). The data structs are serialized via configured converters then enqueued onto Kafka topics. The topic name used corresponds to the fully qualified source table name as obtained from the GoldenGate trail file. Individual operations consist of inserts, updates, and delete operations executed on the source RDBMS. Insert and update operation data include the after change data. Delete operations include the before change data. A primary key update is a special case for an update where one or more of the primary key(s) is/are changed. The primary key update represents a special case in that without the before image data it is not possible to determine what row is actually changing when only in possession of the after change data. The default behavior of a primary key update is to ABEND in the Kafka Connect formatter. However, the formatter can be configured to simply treat these operations as regular updates or to treat them as deletes and then an insert which is the closest big data modeling to the substance of the transaction.

Difference between official GoldenGate Kafka Handler and Kafka Connect Handler

The Kafka Handler officially released in Oracle GoldenGate for Big Data 12.2.0.1.x is slightly different in functionality than the Kafka Connect Handler/Formatter included in this opensource component.

  • The officially released Kafka Handler interfaces with pluggable formatters to output the data to Kafka in XML, JSON, Avro, or delimited text format.
  • The Kafka Connect Handler/Formatter builds up Kafka Connect Schemas and Structs.  It relies on the Kafka Connect framework to perform the serialization using the Kafka Connect converters before delivering the data to topic.


Compatibility Matrix

  • Oracle GoldenGate for Big Data 12.2.0.1.1
  • Confluent.io Kafka/Kafka Connect 0.9.0.1-cp

Download Location

 It is available for download at java.net

Tuesday Jun 21, 2016

Oracle GoldenGate Studio 12.2.1.1.0 Release is Available

Oracle GoldenGate Studio (OGGS) 12.2.1.1.0 is released on June 21st, 2016. You can now download the software from OTN and eDelivery. This is the second release after we launched the OGGS at the start of this calendar year. In this release, we have certified two new heterogeneous databases (MySQL and Microsoft SQL Server) that you can use to design and deploy in high-volume, real-time database replication.

Key Features

  • Design and deploy high-volume, real-time replication across heterogeneous platforms in just a few clicks
  • Best practice solution templates and wizards
  • Automap tables and drag and drop custom maps
  • Deploy the same solution to multiple locations
  • Monitor active deployments with many monitoring points and track deployment history
  • Refresh the metadata in mapping editor

You can refer GoldenGate Studio 12.2.1.1 Data Sheet  for brief information about Studio, new features and benefits. The detailed information can be read from Oracle GoldenGate Studio 12.2 documentation.

Hope to see you guys using new Studio release. Feel free to reach out to me for your queries by posting in this blog or tweeting @nisharsoneji

Thursday Apr 21, 2016

Oracle GoldenGate for DB2 for i 12.2.1.0.2 Release is Available

Oracle GoldenGate for DB2 i 12.2.0.1.2 was released on 4/15/2016. You now can download the software from OTN and eDelivery.[Read More]

Oracle GoldenGate for DB2 z/OS 12.2.1.0.2 Release is Available

Oracle GoldenGate for DB2 z/OS  12.2.0.1.2 was released on 4/15/2016. You now can download the software from OTN and eDelivery. [Read More]

Wednesday Mar 23, 2016

Getting Started with Oracle GoldenGate Cloud Service

GoldenGate Cloud Service (GGCS), is a cloud based real-time data integration and replication service, which provides seamless and data movement from various on-premises relational databases to databases in the cloud with sub-seconds latency while maintaining data consistency and offering fault tolerance and resiliency. This blog provides an overview.[Read More]

Thursday Mar 17, 2016

Oracle GoldenGate Cloud Service Executive Webcast

We are proud to announce the availability of the much awaited Oracle GoldenGate Cloud Service.

Oracle GoldenGate Cloud Service builds on top of Oracle’s on premises GoldenGate solution, the industry leading data replication and real time streaming engine. Oracle GoldenGate Cloud Service combines the flexibility of a scalable cloud service that can be consumed on a subscription basis along with the powerful data streaming capability of an enterprise solution.

fig 1:Oracle GoldenGate Cloud Service Overview

Oracle GoldenGate Cloud Service 

Oracle GoldenGate Cloud Service

  1. Enables migration to the cloud from heterogeneous databases to the Oracle Cloud platform. This includes moving data from Amazon RDS for Oracle among other data sources. Oracle GoldenGate Cloud Services ensures security and reduces risks to the data streaming process through its high-throughput encryption based data delivery. Oracle delivers all this with minimal negative impact on the critical source and business application systems.
  2. Is critical for well performing global real time reporting and analytics. Oracle GoldenGate Cloud Service delivers real time data into the cloud allowing the population and maintenance of cloud based data centers and data warehouses. A cloud based global data reserve allows intensive business analytics queries to be offloaded to avoid impacting source data systems. Globally located cloud data centers allow these queries to be offloaded to data the geographically closest location to the user.
  3. Helps easy dev/test in the cloud with live production data. Oracle GoldenGate Cloud service enables fast, elastic and web driven provisioning to create sandboxes in the cloud complete with real time data to help replicate production like environments for development and testing. Among other advantages, this allows customers to test and validate time sensitive business decisions with production data.

Fig 2: Take advantage of existing skills and architecture to extent data centers into the cloud. 

Learn More

 Here is a quick product overview video about Oracle GoldenGate Cloud Service. For more information around Oracle GoldenGate Cloud Service visit our home page here.

GoldenGate Cloud Service Executive Webcast

Join us for our executive webcast featuring Amit Zavery, Senior Vice President, Oracle Cloud Platform and Integration Products and Jeff Pollock, Vice President of Product Management at Oracle discuss Oracle's latest cloud offering. Joining them will be Paul Stracke from Paychex to provide more customer perspective.

Mark your calendars today and join us on 10 AM PST on the 7th of April. Register here.

Thursday Mar 10, 2016

Oracle GoldenGate Plug-in for Oracle Enterprise Manager 13.1 is released

Oracle GoldenGate Plug-In 13.1.1 for Oracle Enterprise Manager Cloud Control 13.1 is released.

Oracle GoldenGate Plug-In 13.1.1 is certified to work with Enterprise Manager 13.1 and Oracle GoldenGate Core 11.2.1.0.20+, 12.1.2.0.1, 12.1.2.1.0 and 12.2.0.1.0(latest) versions.

You may download the software for online or offline upgrade from the following locations

Please refer to the following links for more information:

  • Documentation Oracle GoldenGate for Enterprise Manager 13.1.1.0.0

  • Documentation Oracle Enterprise Manager Cloud Control 13.1.1.0.0

  • Learn more about Oracle GoldenGate Management Pack

  • Learn more about Oracle Enterprise Manager Cloud Control

Friday Feb 19, 2016

Oracle GoldenGate Adapter for MongoDB

Oracle GoldenGate can now integrate with Mongo DB using the GoldenGate Adapter for MongoDB. GoldenGate Adapter for MongoDB can run on the latest Java based Replicat available with Oracle GoldenGate for Big Data 12.2. It uses high performance native Java driver to MongoDB. It can handle automatic DDL replication and supports inserts, updates and deletes into MongoDB.

This project has converted to an open-source project for increased contribution from the Java community. The downloadable bundle contains source code, binary (jar) files, sample configuration files and instructions to use (readme.txt). You can download the bundle from URL https://java.net/projects/oracledi/downloads/directory/GoldenGate/MongoDB%20Adapter

Compatibility Matrix:

  • GoldenGate for Big Data: 12.2.0.1.0 (All platforms)
  • MongoDB Server : 2.4, 2.6, 3.0 3.2
  • MongoDB Java Driver: 3.2.2


Monday Feb 01, 2016

Oracle GoldenGate 12.2 Release

In keeping with our foundation principles, the focus for this release has been Usability, Manageability, Heterogeneity, and Performance.  

What’s New in Oracle GoldenGate 12.2 (Replication Simplification)

GoldenGate Platform Features –for Oracle, SQL Server, Sybase, Teradata, Informix, TimesTen, DB2 LUW, NonStop SQL/MX, PostgreSQL, and MySQL

Automatic Heartbeat Table
Record end-to-end replication lag and view details with a database view. Valid for heterogeneous configurations as well.

Metadata in Trails
No need to generate a definitions file or use SOURCEDEFS/ASSUMETARGETDEFS.

Parameter Insight
Lookup parameter definitions from GGSCI with INFO PARAM, get the list of running parameters for a process with SEND … GETPARAMINFO, and pre-validate parameter files with the new checkprm utility.

One Billion Trail Files
Trail file sequence length has increased to nine digits; a 1000x increase and effectively one billion trail files.

For the Oracle Database

  • DL replication between more than 2 Oracle databases
  • Seamless DML and DDL replication between tables of differing structures
  • Replicat integration with the Oracle Data Pump
  • Filters past table data loaded to target tables from expdp/impdp: DBOPTIONS ENABLE_INSTANTIATION_FILTERING
  • Integrated Capture support for invisible columns
  • Offload fetching to Active Data Guard
  • Oracle Clusterware command and control integration

For SQL Server

  • Remote ALO mode Capture
  • SQL Server Standard Edition Capture and Delivery
  • SQL Server 2014 Compressed Log backups
  • Column level character sets

For Teradata

  • Replicat can process truncates
  • Column level character sets
  • Capture from Teradata is no longer supported from this version onward

For DB2 LUW

  • Column level character sets
  • Temporal Tables

More information on Oracle GoldenGate 12.2

Downloads: http://www.oracle.com/technetwork/middleware/goldengate/downloads/index.html

Documentation: http://docs.oracle.com/goldengate/c1221/gg-winux/index.html

Certification Matrix: http://www.oracle.com/technetwork/middleware/data-integration/goldengate12-2-x-cert-matrix-2769360.xls

Tuesday Dec 22, 2015

Oracle GoldenGate for Big Data 12.2.0.1 is Generally Available Now!

Much awaited Oracle GoldenGate for Big Data 12.2 is released today and it is available for download at OTN.

Let me give you a quick recap on Oracle GoldenGate for Big Data. Oracle GoldenGate for Big Data streams transactional data into big data systems in real-time, raising the quality and timeliness of business insights. Oracle GoldenGate for Big Data offers also provides a flexible and extensible solution to support all major big data systems.

Oracle GoldenGate for Big Data

  • Same trusted Oracle GoldenGate architecture used by 1000’s of customers
  • Data delivery to Big Data targets including NoSQL databases
  • Support for Polyglot, Lambda and Kappa architectures for streaming data

Key Benefits

  • Less invasive on source databases when compared to batch processing such as Sqoop or ETL processes
  • Simple ingestion for 1:1 data architecture for populating “raw data” zones
  • Real-time data delivery for streaming analytics/apps
  • Reliable, proven at scale with high performance


Architecture – GoldenGate for Big Data 12.2 versus 12.1

New Features in 12.2.0.1:



New Java based Replicat Process 

The advantages of using Java based Replicat process are the following:

    1. Improved performance with Java based adapters
    2. Declarative design and configurable mapping
    3. Transaction grouping based on Operation count & Message size
    4. Improved check pointing functionality
      E.g.: CHECKPOINTSECS 1 (default 10 seconds)

Dynamic Data Handling

You no longer require to define SOURCEDEFS. DDL changes are automatically replicated to target. For example, if a new column named “mycolumn“ is added on the source database, it will be automatically replicated to the target without stopping and reconfiguring Oracle GoldenGate.

Pluggable Formatters

Oracle GoldenGate for Big Data can write into any Big Data targets in various data formats such as delimited text or XML or JSON or Avro or custom format. This can save users cost and time for staging data in ETL operations.

Example: gg.handler.name.format= <value>
values supported are delimitedtext”, “xml”, “json”, “avro” or “avro_row”, “avro_op” or Custom Format. Extended class path needs to be included in the config file. <com.yourcompany.YourFormatter

Security Enhancement

Native Kerberos support is available in the 12.2.0.1 binaries.

Example of configuration:
gg.handler.gghdfs.authType=Kerberos
gg.handler.gghdfs.kerberosKeytabFile=/keytab/file/path
gg.handler.gghdfs.kerberosPrincipal=user/FQDN@MY.REALM

Declarative Design

Oracle GoldenGate for Big Data is able to provide mapping functionally between source table to target table and source field to target field for HDFS/Hive, HBase, Flume and Kafka. The metadata is also validated at Hive or using an Avro schema to ensure data correctness.

Example:
MAP GG.TCUSTOMER, TARGET GG.TCUSTMER2, COLMAP (USEDEFAULTS, "cust_code2"=cust_code,"city2"=city);

Kafka as target

Oracle GoldenGate for Big Data can write Logical change records data to a Kafka topic. Operations such as Insert, Update, Delete and Primary Key Update can be handled. It can handles native compression such as GZIP and Snappy in Kafka.

Example of defining Kafka Handler Properties:
gg.handlerlist=ggkafka
gg.handler.ggkafka.type=kafka
gg.handler.ggkafka.topicName=gg_kafka
gg.handler.ggkafka.mode=tx

Other Enhancements

  • Partition data by Hive Table and/or column. Partitioning into new file based on designated column values
    Example:
    • gg.handler.{name}.partitionByTable =true | false
    • gg.handler.{name}.partitioner.{fully qualified table name}={colname}
    • gg.handler.{name}.partitioner.{fully qualified table name}={colname1},{colname2}
    • gg.handler.<yourhandlername>.partitioner.dbo.TCUSTORD=region, rating
  • Configurable File Rolling Property for HDFS (file size, duration, inactivity timer, metadata change)
  • Configurable file output encoding into HDFS
  • Automatically create HBase table if it does not exist
  • Ability to treat primary key updates as a delete and then an insert in HBase
  • HBase row key generation
  • Treat Primary Key updates as delete and insert in Flume and HBase
  • New Time stamping functionality to include micro second precision as ISO-8601
  • Availability on additional OS platforms: Windows and Solaris
  • Certification for newer versions: Apache HDFS 2.7.x, Cloudera 5.4.x, Hortonworks 2.3, Kafka 0.8.2.0 and 0.8.2.1


For more details about new product features, you may refer to Oracle GoldenGate for Big Data 12.2.0.1 Release Notes and User Documentation.

For more information about Oracle GoldenGate for Big Data.

Feel free to reach out to me for your queries by posting in this blog or tweeting @thomasvengal

Happy Holidays ! 

Tuesday Oct 06, 2015

Catch Up on Oracle GoldenGate at Oracle OpenWorld

This year Oracle OpenWorld brings many opportunities for Oracle GoldenGate users to learn the latest features and best practices. If you will be at OpenWorld later this month, I recommend you check out the following key Oracle GoldenGate sessions:

On Monday October 26th, Chai Pydimukkala from the GoldenGate product management team will present Oracle GoldenGate Product Update and Strategy (CON9720) session and share the new features of Oracle GoldenGate and its strategic direction. Along with Chai, Eric Schneider and Andrew Yee from our customer TicketMaster will present their GoldenGate deployment and best practices

On Tuesday Oct 27th, in the Enabling Real-Time Data Integration with Big Data (CON9724) Chai will discuss GoldenGate's offering for big data environments. With Chai,  Janardh Bantupalli from LinkedIn, will present their solution that uses Oracle GoldenGate for Big Data to optimize the data warehousing environment and achieve operational insights with lower costs.

On Wednesday October 28th you can get a preview to the anticipated Oracle GoldenGate Cloud Service by attending Accelerate Cloud On-Boarding using GoldenGate Cloud Service (CON9717). You will also learn about using GoldenGate in public, private, and hybrid cloud deployment and hear from Michael Pape, CTO Database Operations for Intuit, talk about their GoldenGate best practices.  

Other key GoldenGate sessions to attend:

You can find the full list of Oracle GoldenGate and Oracle Data Integration sessions in our Focus On Document, and use the Schedule Builder to build your personal schedule for OpenWorld. While at OpenWorld, follow us via @OracleDI and @ORACLEBigData and using the hashtags #ODI12c,#OGG12c, #OEMM, #OEDQ and #OBDPCS to join the conversation.

Friday Jun 19, 2015

Oracle GoldenGate Certified on Teradata Unity

Oracle GoldenGate 12.1.2.1.1  is now certified with Unity 14.11.  With the certification, customers can use Oracle GoldenGate to deliver data to Teradata Unity which can then automate the distribution of data to multiple Teradata databases.  This joined effort from Oracle GoldenGate and Teradata extends the Oracle GoldenGate and Teradata ecosystem for real-time data integration. 

Wednesday Apr 08, 2015

Oracle GoldenGate for DB2 z/OS Supports DB2 11

With the release of Oracle GoldenGate 12.1.2.1.4 release, Oracle GoldenGate for DB2 z/OS provides the support for DB2 11. This release also includes the fix to make Oracle GoldenGate z/OS Extract compatible with  IBM APAR PI12599 for DB2 z/OS. [Read More]

Thursday Feb 19, 2015

Introducing Oracle GoldenGate for Big Data!

Big data systems and big data analytics solutions are becoming critical components of modern information management architectures.  Organizations realize that by combining structured transactional data with semi-structured and unstructured data they can realize the full potential value of their data assets, and achieve enhanced business insight. Businesses also notice that in today’s fast-paced, digital business environment to be agile and respond with immediacy, access to data with low latency is essential. Low-latency transactional data brings additional value especially for dynamically changing operations that day-old data, structured or unstructured, cannot deliver.

Today we announced the general availability of Oracle GoldenGate for Big Data product, which offers a real-time transactional data streaming platform into big data systems. By providing easy-to-use, real-time data integration for big data systems, Oracle GoldenGate for Big Data facilitates improved business insight for better customer experience. It also allows IT organizations to quickly move ahead with their big data projects without extensive training and management resources. Oracle GoldenGate for Big Data's real-time data streaming platform also allows customers to keep their big data reservoirs up to date with their production systems. 

Oracle GoldenGate’s fault-tolerant, secure and flexible architecture shines in this new big data streaming offering as well. Customers can enjoy secure and reliable data streaming with subseconds latency. With Oracle GoldenGate’s core log-based change data capture capabilities it enables real-time streaming without degrading the performance of the source production systems.

The new offering, Oracle GoldenGate for Big Data, provides integration for Apache Flume, Apache HDFS, Apache Hive and Apache Hbase. It also includes Oracle GoldenGate for Java, which enables customers to easily integrate to additional big data systems, such as Oracle NoSQL, Apache Kafka, Apache Storm, Apache Spark, and others.

You can learn more about our new offering via Oracle GoldenGate for Big Data data sheet and by registering for our upcoming webcast:

How to Future-Proof your Big Data Integration Solution

March 5th, 2015 10am PT/ 1pm ET

I invite you to join this webcast to learn from Oracle and Cloudera executives how to future-proof your big data infrastructure. The webcast will discuss :

  • Selection criteria that will drive business results with Big Data Integration 
  • Oracle's new big data integration and governance offerings, including Oracle GoldenGate for Big Data
  • Oracle’s comprehensive big data features in a unified platform 
  • How Cloudera Enterprise Data Hub and Oracle Data Integration combine to offer complementary features to store data in full fidelity, to transform and enrich the data for increased business efficiency and insights.

Hope you can join us and ask your questions to the experts.

About

Learn the latest trends, use cases, product updates, and customer success examples for Oracle's data integration products-- including Oracle Data Integrator, Oracle GoldenGate and Oracle Enterprise Data Quality

Search

Archives
« July 2016
SunMonTueWedThuFriSat
     
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
21
22
23
24
25
26
27
28
29
30
31
      
Today