Information, tips, tricks and sample code for Big Data Warehousing in an autonomous, cloud-driven world
-
The Data Warehouse Insider
March 12, 2018
Roadmap Update: What you need to know about Big Data Appliance 4.12
As part of our continuous efforts to ensure transparency in release planning and availability to our customers for our big data stack, below is an update to the original roadmap post.
Current Release
As discussed the 4.11 release delivered the following:
- Released with updates CDH and now deliver 5.13.1
- Updates to the Operating System (OL6) with security updates to said OS
- Java updates
The release consciously pushed back some of the features to ensure the Oracle environments pick up the latest CDH releases within our (roughly) 4 week goal.
Next up is BDA 4.12
As part of the longer development time we carved out by doing 4.12 with features we are able to schedule a set of very interesting components into this release. At a high level, the following are planned to be in 4.12:
- Configure a Kafka cluster on dedicated nodes on the BDA
- Set up (and include) Big Data Manager on BDA. For more information on Big Data Manager, see these videos (or click the one further down) on what cool things you can do with the Zeppelin Notebooks, ODCP and drag-and-drop copying of data
- Full BDA clusters on OL7. After we enabled the edge nodes for OL7 to support Cloudera Data Science Workbench, we are now delivering full clusters on OL7. Note that we have not yet delivered an in-place upgrade path to migrate from an OL6 based cluster to an OL7 cluster
- High Availability for more services in CDH, by leveraging and pre-configuring best practices. These new HA set up steps are obviously updated regularly and are fully supported as part of the system going forward:
- Hive Service
- Sentry Service
- Hue Service
- On BDA X7-2 hardware 2 SSDs are included. When running on X7, the Journal Node metadata and Zookeeper data is put onto these SSDs instead of the regular OS disks. This ensures better performance for highly loaded master nodes.
Of course the software will have undergone testing and we do run infrastructure security scans on the system. We include any Linux updates that available when we freeze the image and ship those. Any violation that crops up after the release can, no should, be updated using the official OL Repo to update the OS.
Lastly, we are looking to release early April, and are finalizing the actual Cloudera CDH release. We may use 5.14.1, but there is chance that we switch and jump to 5.15.0 depending on timings.
And one more Thing
Because Big Data Appliance is an engineered system, customers expect robust movement between versions. Upgrading the entire system, where BDA is different from just a Cloudera cluster, is an important part of the value proposition but is also fairly complex. With 4.12 we place additional emphasis on addressing previously seen upgrade issues and we will be doing this as an ongoing priority on all BDA software releases. So expect even more robust upgrades going forward.
Lastly
Please Note Safe Harbor Statement below:
The preceding is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
- CDH will be 5.14.2
- HA components will be there
- Big Data Manager will be in (this one caused a bit of the delays)
Should all be coming in the next week or so (no promises of course!)