In this tutorial we deal with columns and tables. Specifically how to map Oracle columns to specific data fields in HDFS files or to Hive columns. Previously...
In this tutorial we deal with columns and tables. Specifically how to map Oracle columns to specific data fields in HDFS files or to Hive columns. Previously in Part 3, we discussed the methods for loading Oracle tables for relatively simple cases. The simplifying assumptions included the following: When loading delimited text into Oracle tables, the physical order of the delimited text fields reflects the declared order of the columns in the Oracle table When loading Hive...
In this tutorial we deal with columns and tables. Specifically how to map Oracle columns to specific data fields in HDFS files or to Hive columns. Previously in Part 3, we discussed the methods for...
In Part 3 we described how to use OHSH to load Oracle tables with content living in either HDFS files or in Hive tables. In this tutorial we focus on how to...
In Part 3 we described how to use OHSH to load Oracle tables with content living in either HDFS files or in Hive tables. In this tutorial we focus on how to use OHSH to do the opposite: create, replace, and incrementally load Hive tables with content living in Oracle tables. To use OHSH to do this you need to download and configure Copy To Hadoop feature of Big Data SQL (CP2HADOOP) as described in Part 1 and Part 2 of this series of tutorials. When launching OHSH, the...
In Part 3 we described how to use OHSH to load Oracle tables with content living in either HDFS files or in Hive tables. In this tutorial we focus on how to use OHSH to do the opposite: create,...
In this tutorial we are going to talk in depth about loading Oracle tables with content from HDFS files or from Hive tables. The focus will be on content that...
In this tutorial we are going to talk in depth about loading Oracle tables with content from HDFS files or from Hive tables. The focus will be on content that is stored as delimited text, but we will talk about loading other forms of content (e.g. parquet). The Oracle target table and sources of data The Oracle table we will be loading lives in a schema called "MOVIEDEMO" and is called MOVIE_RATINGS with various attributes relating customers to movies they saw and other...
In this tutorial we are going to talk in depth about loading Oracle tables with content from HDFS files or from Hive tables. The focus will be on content that is stored as delimited text, but we will...
In this "eat your vegetables before having dessert" tutorial we will talk about how to configure OHSH. We will also discuss groups of OHSH commands that you...
In this "eat your vegetables before having dessert" tutorial we will talk about how to configure OHSH. We will also discuss groups of OHSH commands that you will need to know about before you start using the tool to perform loading tasks. Configuration and SQL*Plus and Hadoop/Hive Dependencies Configuration of OHSH will differ depending upon where you want to run it. The short story is that OHSH expects to run in a Linux environment with Bash shell available, and have...
In this "eat your vegetables before having dessert" tutorial we will talk about how to configure OHSH. We will also discuss groups of OHSH commands that you will need to know about before you start...
After following the tips below your Oracle Loader for Hadoop jobs should fly! Summary Check your network bandwidth Use OCIOutputFormat if possible...
After following the tips below your Oracle Loader for Hadoop jobs should fly! Summary Check your network bandwidth Use OCIOutputFormat if possible (OCIOutputFormat requires the target table to be partitioned, if target table is not partitioned, see whether you can make it a partitioned table) Check parallelism (this is the number of reduce tasks when loading into a partitioned table) Use SDP (in particular when using InfiniBand) Confirm that balancer is enabled, a...
After following the tips below your Oracle Loader for Hadoop jobs should fly! Summary Check your network bandwidth Use OCIOutputFormat if possible (OCIOutputFormat requires the target table to be...
