Sunday Jul 20, 2008

Sun Virtual Desktop Connector - Insides on Recycling

originally blogged by Tino Rachui

Recycling is not only an excellent way to protect our environment but also a good way to balance resources in your VDI environment. A couple of interesting postings about this topic has been publish already by Dirk G. in his blog White men can't jump.

I assume you have read them in addition to the official Sun Virtual Desktop Connector 1.0 (SVDC) documentation in order to have the necessary context to understand the topic this blog post is about.

So why am I writing about recycling again? The answer is that question about this area keep coming from many people using the SVDC.

Answering all of these questions individually takes a lot of time so I decided to use a blog as means to address these questions in particular and future technical questions around SVDC in general. I don't aim at duplicating the information that have been given on the topic of recycling elsewhere I rather try to complement them with a little more focus on in-depth technical details on how it is working and how to diagnose issues that you might be facing along the way.

The explanations that will follow assume that you are running Windows XP as guest Operating System (OS) inside your Virtual Machines (VM). There exist some limitation for Window Vista please have a look into the official SVDC 1.0 documentation or find some details on White men can't jump

One crucial point in order to understand how VM recycling with SVDC 1.0 works or how to diagnose possible problems in this area is to understand the differences as well as relationships between the VM power options and the guest OS power options (Windows XP assumed).

With the Windows XP  "Power Options" control panel you can influence how Windows should behave when a certain period of inactivity has elapsed. On this control panel there exist a 'System standby' drop down box enabling you to specify a value between 'never' and 'After 5 hours'. When you have chosen a value not equal to 'never' Windows will go into a low power consumption mode after the specified time of inactivity (not mouse moves, keystrokes etc.) has elapsed. The state that Windows then enters is often also referred to as 'standby' mode.

The Virtual Machine Power Options on the other hand determine "How the VM should respond when the guest OS is placed on standby". In other words what should VMware Virtual Center (VC) do with a VM when the guest OS running inside that VM went into the 'standby' mode. With VC 2.5 there are two possibilities (please consult the official VMware VirtualCenter/ESX documentation for details):

  1. Suspend the virtual machine. This options probably makes sense if you want to save resources. If the guest OS inside the VM has entered the 'standby' mode VC will put the VM into suspend mode so that it doesn't consume CPU and memory resources for instance.
  2. Put the guest OS into standby mode and leave the virtual machine powered on. This option could make sense if you want to be able to enable something like Wake On LAN (WoL) for instance.

A second point that is essential in order to really understand how recycling is working with SVDC is to know and grasp the conditions that SVDC uses in order to determine if a VM can be recycled or not. I will explain later how SVDC actually identifies each of these conditions. But first let me name the conditions that must be met so that a VM is subject to recycling (please note that these conditions are combined by 'AND' and 'ORs'):

  • The VM is assigned to a user (for SVDC 1.0 this means it resides in the 'used' folder of the corresponding pool) AND
    • the VM is in power state 'suspended' OR
    • (the VM is in power state 'running' AND no user is logged in to the guest OS) OR
    • (the VM is in power state 'running' AND the guest OS is in 'standby' mode)

When a VM will really be recycled after the aforementioned conditions have been met and what happens to it then can be controlled with the pool settings "Idle Time" and "Recycle policy" (again please consult the official SVDC documentation for details).

These are the preconditions in order to understand VM recycling. Large blog entries are tedious and this one is quite long already so I will break down the whole topic into multiple postings. Next time I will explain how SVDC identifies the conditions mentioned above in order to come to a decision if a VM has to be recycled or not.

Sun Virtual Desktop Connector - Insides on Recycling Part 2

originally blogged by Tino Rachui

With my previous posting I've tried to set the stage in order to understand how VM recycling works with SVDC 1.0. I've listed the conditions that SVDC checks to decide whether or not a VM should be recycled. So how does SVDC identify these conditions?

It is easy for SVDC to detect that a VM is in power state "suspended" and get the time stamp when it actually went into that state. VC maintains this information and it can be access using the VMware SDK (alas VC 2.5 doesn't provides the suspend time of a VM correctly anymore, this seems to be a regression to previous versions).

It's not possible though to determine the rest of the conditions using the VMware SDK so the another component of SVDC - the VDA Tools - are necessary to help. The VDA Tools are a Windows service that need to be running on the guest OS. They detect if the guest OS enters the 'standby' mode or the user logs off his Windows session. If either of these conditions is met they will write a time stamp that SVDC uses for its recycling decisions.

It might help the understanding to imagine what would happen without the VDA Tools?

Assume a user did log off his Windows session. In that moment you could consider this VM as being 'unused'. If you have configured your guest OS power options to "Never" go into 'standby' mode VC would never put this VM into power state "suspended" obviously and consequently this VM would never be recycled meaning it'll be assigned to the last user forever. That would be identical to a static VM assignment and somehow contradicts the idea of a dynamic pooling.

A similar situation arises with the combination VM power options "leave VM powered on when guest OS enters standby" and guest OS power options have been set to lets say "System Standby after 5 min".  Again the VM would always be in power state 'running' and there would be no chance for SVDC to find out that the guest OS is in 'standby' hence the VM would never be recycled.

If everything is setup correctly (and I hopefully don't need to mention to this audience again that time synchronization across your VDI environment is of paramount importance for reliable behavior) recycling with SVDC is working great. But alas this world is not perfect and things sometimes don't work as expected what to in this case, how to find out asap what is going wrong?

Stay tuned for my next posting.

Sun Virtual Desktop Connector - Insides on Recycling Part 3

originally blogged by Tino Rachui

This is the last posting on how recycling works with SVDC for the time being. The previous posting of this small series ended with the question "What if things are not working as expected and you need to know why not as soon as possible?".

There exist a couple of tools and sources of information to help you tracking down possible root causes:

    \* VMware Infrastructure Client (VIC)
    \* ESX Managed Object Browser
    \* Windows Event Log Viewer
    \* VDA Agent log files

Before you start using any of these tools or information reflect and understand on what conditions SVDC is actually supposed to recycle your VMs.

Ask yourself "What are the power settings of your guest OS?" and "What are the power settings of the VM hosting your guest OS?". Review the conditions that SVDC takes into account for recycling to get a clear picture of what actually should lead SVDC to recycle a specific VM. When you have done this it makes sense to actually start using the above listed tools and information sources for further investigation.

Lets go through some prominent cases and see how to check what is going on (this is how I investigate recycling problems before resorting to use the debugger).

  • You have a VM that is assigned to a user and the VM is in power state 'suspended' but the VM will not be recycled after the idle time has elapsed. You probably have had a look at the VM status already using the VIC in order to know that it is actually 'suspended'.

    If a VM is in use and suspended SVDC makes the recycling decision based on the suspend time stamp that VC has set for that VM (see my previous postings for details).

    Probably the first thing to do would be to have a look into the VDA Agent log files which you will find in the 'log' subdirectory of the installation directory of the VDA Agent on your VC host. If you find a warning therein similar to

    "Virtual Machine <name> suspended without suspend time"

    then you know that you've come across the VC 2.5 problem I've mentioned in my previous posting. Otherwise you might want to check if the VDA Agent is actually running. Use the "Services" control panel on Windows to verify that. Finally you should check if something in the logs suggests communication problems between VDA Agent (running on the VC host) and VDA Service (running on your Sun Ray host(s)) or even more subtle issues like problems to connect to VC via the VMware SDK etc.
  • You have a VM that is assigned to a user, the VM is in power state 'running' but no user is logged in to the guest OS anymore.

    The VM will not be recycled after the idle time has elapsed. In this current case the VM should actually be recycled based on the time stamp written by the VDA Tools.

    If you have multiple ESX servers in your setup use the VIC to identify which one is running your VM. Open a web browser and browse to the Managed Object Browser of this ESX host


    Log in using the root credentials of your ESX host. Follow these links


    on this page you'll find a list of 'childEntity' entries (a bunch of numbers) referring to the VMs running on that ESX host. Pick the one of the machine your're interested in. You have to look for it by clicking on the links and find that VM with the name of the one you are seeing when using the VIC.

    After you've found it click on the 'config' link near the top of the page. In the section 'extraConfig' you'll find a 'extraConfig["guestinfo.logoffTime"]'. Click on this link to see what the value of this property is. A value of "0" means the VM will not be recycled by SVDC any other value means this VM is subject to recycling.

    If everything is working correctly in the case we are talking about the value you find should not be "0". If it is nevertheless than you need to dig deeper. Connect to the guest OS via RDP using the Sun Ray Windows Connector (SRWC) or the console in the VIC. Verify that the VDA Tools are actually started. Open the Windows "Event Viewer" (in the control panel under Administrative Tools) and go to the 'Application' section. See if you can find any error messages written by the VDA Tools suggesting that they were having problems setting the 'guestinfo.logoffTime' property.
  • You have a VM that is assigned to a user, the VM is in power state 'running' and the guest OS has entered the 'standby' mode. The VM will not be recycled after the idle time has elapsed. Use the same procedure as suggested in the second case to investigate possible causes for the problem.

These are some problems I've heard about quite frequently. The list is certainly not complete nor will all possible root causes be tracked down using the methods and tools I've described it's a good start however.

Feedback appreciated!



« July 2016