Tuesday Mar 07, 2017

NUMA effects and "Empty Homes"

A really good blog article "Empty Homes" by Joerg Moellenkamp illustrates effects of non-local memory access (CPU and memory on different sockets of the same server) in an Oracle VM Server for SPARC environment. Non-Uniform Memory Access (NUMA) affects all multi-socket servers and can reduce performance by increasing latency when CPUs access memory.

Details in his article are specific to Solaris, SPARC, and Oracle VM Server for SPARC, which together provide tools for measuring NUMA effect, determining the underlying CPU and memory assignments that can lead to it, and avoiding it in the first place.

The principles apply elsewhere, including Oracle VM Server for x86 which also attempts to prevent NUMA latency. Avoiding NUMA effects, and otherwise balancing memory and CPU allocations has a powerful benefit. As Joerg says in the article "memory and CPUs don't exist independently in modern systems when you are doing your configuration".

Thursday Feb 23, 2017

Oracle VM Server for SPARC 10th anniversary

This year marks the 10th anniversary of Oracle VM Server for SPARC, formerly called Logical Domains, providing virtual machine capability for SPARC systems.

At that time, it was a feature of horizontally scaled T1000 and T2000 class SPARC servers, but now is available throughout the SPARC product line. It retains its original benefit of low-overhead virtualization, but is now suitable for every application workload category, and able to support the most demanding requirements for performance, scale and flexibility.

A lot has been added over time, as shown in the What's New page that goes all the way back to Logical Domains 1.0. For more perspective, please see Honglin Su's blog Happy Anniversary - Oracle VM Server for SPARC.

Also, we've just announced the release of a new maintenance release: Oracle VM Server for SPARC, shipped with Oracle Solaris 11.3 SRU 17. I just installed this in my lab, and it was as simple and straightforward as previous releases.

Sunday Feb 12, 2017

Private Cloud Appliance (PCA) Tenant Groups

This blog article describes the "tenant group" feature of the Oracle Private Cloud Appliance (PCA) used to ensure isolation between different tenants (clients) of a PCA.[Read More]

Wednesday Nov 02, 2016

MDB commands for Virtual HBA on Oracle VM Server for SPARC

One of the very cool recent features in Oracle VM Server for SPARC 3.4 is virtual HBA (vHBA) which adds I/O generality and function for logical domains.

I discussed this in previous blogs, and am pleased to announce a technical blog entry by Andrew Rutz (so it's authoritative!) on using MDB commands to introspect on vHBA: see MDB commands for vHBA (Virtual HBA)

Thursday Sep 29, 2016

OpenStack for Oracle VM Server for SPARC

Last week during Oracle Openworld, Oracle announced the availability of the OpenStack Nova driver for Oracle VM Server for SPARC (logical domains). This means that SPARC servers hosting guest domains (SPARC virtual machines) can be administered by OpenStack to create and run SPARC-based clouds.

The announcement link above gives an overview of the capabilities and installation requirements, but some additional details are:

  • Support for OpenStack Kilo
  • Support for Solaris 10 and Solaris 11 instances
  • Network boot support, and ability to boot from a volume or a volume snapshot
  • Support for SAN, NAS and local storage (FC, iSCSI, local vdisk files, NFS vdisk files, and local ZFS volumes "ZVOLs")
  • Flat and VLAN networking
  • Dynamic attachment and detachment of storage volumes, and network interfaces. Virtual disks can be added or removed while an instance (guest domain) is running)
  • VNC and serial console support
  • Historical console log
  • Image (nova) and volume (cinder) based OS deployments
  • Whole and max core constraints to control CPU allocation
  • SSH key, Password, and file injection into guests without use of a network metadata service (network agnostic)
  • Parallel guest deployment
  • Multiple, physically isolated virtual switches, with guests attached to many networks (multi-homing), with varied MTUs
  • Multiple MAC addresses supported for guest domains - so guests can run Solaris network virtualization and exclusive IP zones
  • Nova Evacuate support with distributed lock management protection, allows redeployment of a logical domain on another node if hardware fails

Several demo videos have been produced to show it in action:

To get started, see the documentation, which guides you through setting up a Nova compute node on a SPARC server, and shows how to install a sample controller node running the typical OpenStack Kilo services (horizon, neutron, keystone, and so on). Note that the controller node can be a guest domain or a physical server. If a guest domain (1 core and 16GB of RAM will be plenty), don't try to run it on the infrastructure it controls as that would be a "chicken and egg" situation" :)  The controller node can be based on Solaris running on SPARC or on x86, and with some configuration setups could be a non-Solaris controller environment.

I've installed this, created images and fired up "instances" (the generic name for a virtual environment which might be a VM or a container). There is a learning curve after installation since there are a lot of concepts and tools to assimilate. That said, the demonstration controller setup runs in a few minutes with little setup effort, so is a quick way to start delving into OpenStack.

Wednesday Sep 28, 2016

Upgrading Oracle VM Manager to version 3.4.2

A few months ago I blogged on upgrading an Oracle VM environment from 3.3.4 to 3.4.1, and in that spirit (and with a few variations) I just did an upgrade from 3.4.1 to 3.4.2.

As before, I upgraded my lab systems (one Oracle VM Manager controlling an x86 server pool and a SPARC server pool) using the Upgrading Oracle VM instructions. The Manager upgrade was done first and that was completely by the book (mount the ISO image, run ./runInstaller.sh, and so on) with nothing exciting to report. After a few minutes I had the Manager at 3.4.2 level.

However, I used some variations for the servers. Before upgrade, the Oracle VM Manager instance was at the public 3.4.1 version, but the x86 servers were at an earlier internal test build (yes, I know: I should have synced it up long ago. My bad). To simulate customer experience I upgraded the servers to the official Oracle VM Server 3.4.1 level and then upgraded them to Oracle VM Server 3.4.2. This was the regular, documented process - I just did it twice with different Yum repository contents. A slight difference is that I created the repositories on one of my SPARC servers running Solaris instead of on x86 running Linux. No real reason except that I have those machines handy. The commands were almost the same as in the documentation, except for the slight differences for the mount command and the directory where I put the files.

Finally, I upgraded the SPARC portion of this lab. Instead of doing the method I've used in previous upgrades (update the server update repository, run the update from the Oracle VM Manager user interface), for a change I deleted a server from the Manager, uninstalled all presence of Oracle VM Manager's OVS agent, and then did a fresh install using the current OVS agent and Distributed Lock Manager (DLM). That worked just fine, and then I took another minute or two to re-present network definitions and repositories.

The net result is that this lab environment is now running Oracle VM Manager 3.4.2 with Oracle VM Server for x86 3.4.2, and Oracle VM Server for SPARC 3.4 (the release numbers vary slightly) and with the OVS agent corresponding to the Manager version.

Tuesday Sep 06, 2016

Security Best Practices for Oracle VM Server for SPARC

I was recently asked for a list of 'security best practices' for Oracle VM Server for SPARC, so I'll share the list I came up with:

  1. Start with the Oracle VM Server for SPARC Security Guide.
  2. Lock down access to the control domain, any service domains, and root domains hosting physical functions for SR-IOV virtual functions.
  3. Keep management and storage networks separate from guest domain networks.
  4. Separate guest domains in different security categories by using different virtual switches, VLANs, or PVLANs.
  5. Use etherstub-based virtual switches for private inter-guest networking, or virtual switches without any net-dev.
  6. Set allowed-ips, allowed-dhcp-cids, and allowed-mac on a vnet in conjunction with setting protection=mac_nospoof, ip_nospoof and dhcp_nospoof. This requires recent versions of Oracle VM Server for SPARC.
  7. Set maxbw value on vnet to prevent denial of service attack from a VM.
  8. Implement secure wanboot / verified boot to ensure use of signed binaries at boot time.
  9. Deploy control and service domains (and when possible, guest domains) with Solaris 11.3 minimal server profile plus necessary IPS packages. The minimal server does not even include snoop(1M), so is very tight with regard to attack surface.
  10. Stay current on firmware and Solaris versions to get the latest security fixes.
  11. Don't use clear-text communication for anything that has a potential security implication.

That's not meant to be comprehensive list, but should be a good start for making sure your Oracle VM Server for SPARC environment is configured for security.  Thanks to Steffen Weiberle for several helpful suggestions.

Tuesday Aug 30, 2016

Updated whitepaper: Optimizing Oracle VM Server for x86 Performance

I'm pleased to announce an update to the whitepaper Optimizing Oracle VM Server for x86 Performance.

This update adds information about aligning virtual disk partitions on a 4K boundary for performance and references the KM note describing this in further detail. The update also adds text to clarify that Oracle VM does not buffer file writes, so there's no risk of data loss due to buffering.

Tuesday Jun 21, 2016

Oracle VM Server for SPARC - virtual HBA (vHBA) enhancements

Oracle VM Server for SPARC 3.4 was released in May. This blog entry describes enhancements to the virtual HBA (vHBA) feature, and (bonus!) describes how NPIV can be used to control LUN assignments.[Read More]

Wednesday Jun 01, 2016

Oracle VM Server for SPARC 3.4 - scalability improvements

Oracle VM Server for SPARC 3.4 has been released, with improved capabilities for scale, security, availability, and advanced networking support. This blog entry describes some of the enhancements that increase scalability.
[Read More]

Friday May 06, 2016

Upgrading Oracle VM Manager to version 3.4.1

Last month I blogged on Oracle VM upgrades: one blog entry upgrading Oracle VM Manager to version 3.3.4. and the other blog upgrading to the corresponding SPARC OVS agent. Today, I upgraded the same systems to Oracle VM Manager 3.4.1 and the ovs-agent 3.4.1

[A note on release numbers: Oracle VM Manager's release is 3.4.1, and so is the ovs-agent that runs on the SPARC server. The actual hypervisor, Oracle VM Server for SPARC, formerly called logical domains (and frequently still called by the original name), is currently at release 3.3. Close, but not exactly aligned.]

Today's process was exactly the same as before, and proceeded without any excitement. I upgraded the Manager, and then the server component. If anyone asks via comment I'll post the commands I used, as I did last time, but it will look exactly the same except for file names and release numbers.

This was uneventful. I won't say "boring", because people might find it interesting ;) but it proceeded as expected without any excitement. System upgrades should be uneventful! Now my lab systems are at the current of Oracle VM Manager, ovs-agent, and logical domains.

Friday Apr 22, 2016

New whitepaper: Optimizing Oracle VM Server for x86 Performance

I am very pleased to announce publication of a new whitepaper "Optimizing Oracle VM Server for x86  Performance". This article contains material previously posted on this blog, plus additional technical information and features newly introduced with Oracle VM 3.4.

Wednesday Apr 20, 2016

Upgrading Oracle VM Server to version 3.3.4

This blog entry shows the step-by-step procedure I used to upgrade Oracle VM Server from version 3.3.3 to 3.3.4, corresponding to the upgrade performed last week for Oracle VM Manager.[Read More]

Tuesday Apr 12, 2016

Upgrading Oracle VM Manager to version 3.3.4

This blog entry shows the step-by-step procedure I used to upgrade Oracle VM Manager from version 3.3.3 to 3.3.4. No muss, no fuss.[Read More]

Thursday Mar 24, 2016

Oracle VM 3.4.1 and new performance features

Oracle VM 3.4.1 has just been released, with important new features and improved performance and scalability. This blog describes a new feature than can be used to further improve disk device performance on Oracle VM 3.4.1, and also on the recent maintenance release Oracle VM 3.3.4.[Read More]



« March 2017