Information, tips, tricks and sample code for Big Data Warehousing in an autonomous, cloud-driven world
-
Big Data,
Big Data SQL
May 2, 2020
Oracle Big Data SQL 4.1 is now available
I'm pleased to announce that Oracle Big Data SQL 4.1 is now available. This is a significant release that enables you to take advantage of new Oracle Database 19c capabilities plus core Big Data SQL enhancements.
Big Data SQL enables organizations to easily analyze data across Apache Hadoop, Apache Kafka, NoSQL, object stores and Oracle Database leveraging their existing Oracle SQL skills, security policies and applications with extreme performance. Oracle Database 19c introduces the following features to make this support even better:
- Enhanced Information Lifecycle Management with Hybrid Partitioned Tables a single partitioned table can now source data from both Oracle data files and external stores (object storage, hive, HDFS and local text files). Store the external data in native big data formats (parquet, orc, csv, avro, etc.) - enabling that data to be shared with Spark and other data lake applications
- In-Memory External Tables: load frequently accessed external data in-memory to dramatically improve performance
Core Big Data SQL reader functionality for object storage has also been enhanced - offering support for new sources, file types and complex columns:
- ORC columnar file type support has been added to existing parquet, avro and text file support
- Complex data types (e.g. arrays, records, etc.) can now be queried using JSON functions
- Vastly improved support for text file attributes
- Azure Blob Storage added to existing Oracle Object Storage and Amazon S3 support
Apache Kafka support has also been enhanced with Oracle SQL Access to Kafka. This new capability removes the requirement for the Oracle Kafka hive storage handler and provides a more systematic approach to querying Kafka streams.
Enjoy the new release!