Thursday Mar 12, 2009

Bug investigation (part 1 - tale of sir VIF)

Last year at VirtualBox was fun, but provided such a heavy load that I almost had no time to blog. Now let's fix this a bit. There are several almost detective stories of recent bug hunting in VBox core I'd like to share. They show level of complexity of modern virtualizers, and how bugs in system software may manifest in rather unexpected ways to applications (and guest kernel is an application for us :)). Let's start with this little bugger. Almost 2 years old, and it is a lot for VBox bug. Bug manifested itself as
I'm receiving the following message when certain commands are run in FreeBSD 6.2 and VirtualBox 1.4.0:

   sigreturn: eflags 0x80247

This error was one of nasty blockers to run FreeBSD reliably as the guest, and as I thought it's a good idea to support it, I looked on that bug. As it was pointed out in the bug, message is triggered by:

                if (!EFL_SECURE(eflags & ~PSL_RF, regs->tf_eflags & ~PSL_RF)) {
                       printf("sigreturn: eflags = 0x%x\\n", eflags);
                       return (EINVAL);
in FreeBSD kernel's sys/i386/i386/machdep.c. This check essentially means, that sometimes, in EFLAGS CPU register some bit FreeBSD considered insecure was toggled. Bit which is of interest is VIF bit, which has rather convoluted story behind itself.

When Intel, in early 90s wanted to keep compatibility with legacy 16-bit DOS code, while running protected mode OSes, they introduced so called VM86 mode. This mode was first take of Intel on virtualization, and call it clumsy is kind of compliment. VIF (and VIP) flags are exactly part of this extension. VIF represents virtualized version of IF flag (interrupts enabled flag). If DOS application would be allowed to modify real IF - it could disable interrupts and render whole system unusable. Thus instead, cli instruction in VM86 mode affected only VIF flag. At the same time, pushf and popf instructions which are (almost) only way to access EFLAGS, were modified in such a way, that value of VIF bit was placed to IF bit. And as VIF is 19th bit of EFLAGS, it's not visible in 16-bit version of pushf.

So now the bug reasons: sometimes FreeBSD executes BIOS calls in VM86 mode, which may modify VIF flag on the CPU. When taking protected mode interrupts (such as timer used for task scheduling), if it happens in the wrong moment (when VIF flag value was toggled) our dynamic recompiler wasn't clearing VIF flag (as according to Intel/AMD instruction manuals it shouldn't). All following EFLAGS accesses has VIF flag setting masked, thus to OS it looked like VIF bit toggled at random. Fix was not that hard: just mask out VIF and VIP bits in EFLAGS when taking interrupts in VM86, as those bits makes no sense outside of VM86 mode.

Next post will cover story of most time consuming bug I ever worked on (about 80 hours of continuous hacking).

About

nike

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today