A Simple VNC Server and GDM Configuration Example for OpenSolaris 2008.05

My requirement was to be able to connect my VNC client to a system running OpenSolaris 2008.05 and to be able to login as root. I have now done this successfully on a  system running the original OpenSolaris 2008.05 binary distribution and on a system running OpenSolaris 2008.05 after I ran a full image update to snv_91.

Update September 19th 2008: This procedure does not work if you have updated the image to snv_97 but does work if you update the image to snv_98. The upgrade from snv_97 to snv_98 wiped out the entries I had made in /etc/X11/gdm/custom.conf so I had to make those again. I have added an extra step at the end, based on Chris Drake's comments, to make the VNS server session persist if you exit the client.

1. Check that the VNC Server is Installed

This should be present as it is part of the 2008.05 binary distribution, but I checked anyway.

# pkg info SUNWxvnc
          Name: SUNWxvnc
       Summary: X11/VNC server
         State: Installed
     Authority: opensolaris.org (preferred)
       Version: 4.1.2
 Build Release: 5.11
        Branch: 0.91
Packaging Date: Fri Jun 13 17:49:25 2008
          Size: 6.3 MB
          FMRI: pkg:/SUNWxvnc@4.1.2,5.11-0.91:20080613T174925Z

2. Add this line to /etc/services

vnc-server      5900/tcp                        # Xvnc

3. Edit /etc/X11/gdm/custom.conf as below


4. Enable the Services

# svcadm enable xvnc-inetd
# svcs xvnc-inetd
STATE          STIME    FMRI
online         16:22:30 svc:/application/x11/xvnc-inetd:default
# svcadm enable gdm
# svcs gdm
STATE          STIME    FMRI
online         14:43:13 svc:/application/graphical-login/gdm:default

5. Connect to the Display with a VNC Client

You should now be able to connect to <hostname>:5900 and you should see the gdm login screen.

If you cannot connect, try stopping & starting the services:

# svcadm disable xvnc-inetd gdm
# svcadm enable xvnc-inetd gdm

6. Making the Session Persist

This may or may not be desirable for you, but if you want the VNC session to persist if you exit the VNC client then do the following:

# svccfg -s xvnc-inetd

svc:/application/x11/xvnc-inetd> editprop

This take you into a vi session. Look for the line...

#setprop inetd/wait = boolean: false

Copy the line, uncomment it and set it to true. Save the file, exit svccfg and run the command...

# svcadm refresh xvnc-inetd

Connect again with you VNC client. Now, when you exit/kill the VNC client, the session on the server will persist and you will be able to connect to it again.

You may now want to add an extra level of security to enable password protection on your VNC server. That is something that I have been unable to make work...and from searching around, it seems that others have a similar problem.

References: 1, 2


Starting a GUI such as GNOME using root is quite stupid. It'd be better to integrate gksu instead.

Posted by James Cornell on July 26, 2008 at 09:36 PM BST #

And why is that so? I can understand disabling remote root login, but for the local system I see no reason to not be able to have a root X session availble to use. It's a lot more usable then having to use gksu or su (depending on if you are on a terminal emulator or not)

Posted by P Massey on August 01, 2008 at 09:36 AM BST #

Tim, these instructions were very helpful. Thank you. :-)

With regards to the login session closing whenever you close the vncviewer. You can fix this by edditing the xvnc-inetd svc config and changing the 'inetd/wait' property to 'true'. Use svccfg -s xvnc-inetd and editprop to change the value.

You should probably also add a password to the vnc session by editing the 'inetd_start/exec' property after you do this to prevent someone from accessing your login session without permission.

Posted by Chris Drake on August 28, 2008 at 07:12 AM BST #

Hi Tim,
Very useful information. I am trying the same on domU imahe to get this done but doesn't seem to work? I can see gdm and xvnc services up, but I cannot connect to the server at 5900 port?

Any reason?


Posted by Vijay Upreti on August 31, 2008 at 08:35 AM BST #

Very helpfull and works fine with the original 2008.05 stuff.
After an upgrade to the latest version something seems to be fschked up around librfb.so.

sven@marvin:~$ xvncviewer client13
Not a valid VNC server
sven@marvin:~$ telnet client13 5900
Connected to client13.home.hoaxter.de.
Escape character is '\^]'.
ld.so.1: Xvnc: fatal: librfb.so.0: open failed: No such file or directory
ld.so.1: Xvnc: fatal: relocation error: file /usr/X11/bin/Xvnc: symbol __1cDrfbYSSecurityFactoryStandardRvncAuthPasswdFile_: referenced symbol not found
Connection closed by foreign host.

Will try to find out what happend to librfb.so tomorrow. Any hints to shorten my search appreciated.

Posted by Sven on September 13, 2008 at 03:42 PM BST #


This is the corresponding bugreport.

Posted by Sven on September 13, 2008 at 04:07 PM BST #

Hi, thanks for all the comments on this article. The process I described work fine for the initial 2008.05 binary release but I also hit problems when I upgraded to the snv_97 based build. I have now done a full image update to the snv_98 packages and it all works (again) but I did have to put the lines back into /etc/X11/gdm/custom.conf after the upgrade. Chris Drake's tips on making the session persistent worked. I will update the blog entry. Rgds, Tim

Posted by Tim Thomas on September 19, 2008 at 03:31 AM BST #

Thanks for posting this its been a great help to me :-)
Very clear and well writen.

Posted by BrianC on September 21, 2008 at 04:54 PM BST #

"You may now want to add an extra level of security to enable password protection on your VNC server."

You can just lock the screen when you are finished. When you connect again you wil be asked for password. Not perfect solution but it is some security.


Posted by Brian on September 28, 2008 at 11:24 AM BST #

VNC authentication works fine with 2008.11. Your inetd_start/exec parameter will look something like:

/usr/X11/bin/Xvnc -inetd -query localhost -once -securitytypes vncauth -rfbauth /etc/X11/.vncpasswd -geometry 1280x1024

To set the VNC password just use the vncpasswd command:

pfexec vncpasswd /etc/X11/.vncpasswd

I'd suggest changing the VNC startup user to one dedicated for this purpose. I use 'xvnc' with a group, also named 'xvnc'. Just set the inetd_start/user and inetd_start/group properties to whatever user and group you create.

Then change the permissions on the VNC password file:

pfexec chown xvnc:xvnc /etc/X11/.vncpasswd
pfexec chmod 400 /etc/X11/.vncpasswd

Stop and restart the service:

svcadm disable xvnc-inetd gdm
svcadm enable xvnc-inetd gdm

Posted by Justin Thomas on February 04, 2009 at 03:09 PM GMT #

I noticed that logon screen appears only after svcadm disable gdm

Posted by Giacomo on February 05, 2009 at 05:54 AM GMT #


Excellent Procedures.

I was up and operational in two minutes. I am moving from fedora to opensolaris and this is the information I needed!


Posted by Scott on February 08, 2009 at 07:08 AM GMT #

Tim, the instructions worked flawlessly. Thank you very much.

I'm not sure what I did but now I get a gray screen. I have not made any changes to the configuration files.

Looking for ways to debug this.


Posted by MarkE on March 07, 2009 at 10:35 PM GMT #

thank you for this. i know making these tutorials kinda sucks (i have to do it sometimes where i work) but i'm sure you are helping 1000s of people, including me!

best regards from the states,


Posted by dave on March 29, 2009 at 06:29 PM BST #

Thanks a lot Tim for this helpful post -- it has helped me get VNC up and running on OpenSolaris 2009.06 very quickly.


Posted by Simon Breden on June 08, 2009 at 02:20 PM BST #

great, thanks!

Posted by guest on June 18, 2009 at 07:14 PM BST #

I have followed all the procedure above but when I connect the server from my Windows XP using TightVNC Viewer I only got a blank gray screen? What happened? Please help

Posted by paulus on June 19, 2009 at 03:52 AM BST #

Post a Comment:
Comments are closed for this entry.

Tim Thomas


« July 2016