Adventures with MKS and CYGWIN on Windows

Adventures with MKS and CYGWIN on Windows

Using unix toolkits on Windows machines can be a frustrating experience. I just thought I note a few items I've discovered trying to make things work with either kit.

The JDK uses makefiles and shell scripts in it's build process and has historically used MKS on Windows. Recently the use of CYGWIN has become important and we have been trying to make the choice of Windows unix utilities an option and not force the adoption of either. We may find it difficult to police that both always work but our intention is to allow for a variety of developers. Getting shell scripts and makefiles to work with both has been interesting.

In addition, on Windows we have been trying to make the builds a bit smarter by having it detect or find various components installed on the system with fewer interventions on the user's (builder actually) part, e.g. automatically finding the shorter windows paths to things instead of insisting on the user providing the shorter paths or changing the default install locations of components.

  • PATHNAMES: Try and stick with the windows style of directory names, but using a / instead of a \\ character as the file separator character. With few exceptions, most windows applications seem to accept the C:/ style and it makes life easiest if you can avoid the \\ characters.

  • NMAKE: One exception to the C:/ rule is NMAKE, anything passed into NMAKE needs to be transformed into the C:\\ style.

  • MKS 8.7: Try and use MKS 8.7 or newer. It has a utility dosname that seems to work well, without dosname, or a reliable one, things will be difficult.

  • 64BIT: Supposedly you need at least MKS 8.1 for 64bit Windows, but I've heard reports that even MKS 6.1 can work on 64bit Windows. I've used 8.7. I recently tried to install CYGWIN on a 64bit Windows machine and finally gave up. A co-worker infoirmed me that if the packages are downloaded on a 32bit machine, then copied over to the 64bit machine and installed, it should work. Something is wrong with the CYGWIN install process on Windows 64bit.

  • SPACES: With MKS try and use dosname, and in particular use 'dosname -s'. The dosname utility helps considerably in dealing with changing directory names with spaces to their non-space equivalents (so called short names) and also changing all the backslash characters (\\) to slash (/), which is essential inside makefiles and shell scripts. With CYGWIN use cygpath, and in particular 'cygpath -s -m' to get the short names in the 'mixed' mode, e.g. the "C:/" style of pathname. It's easy in GNU makefiles to check to see if a directory has spaces, just use:

    
    ifneq ($(DIR),$(word 1,$(DIR)))
      # DIR contains spaces
    endef
    
  • DOSNAME and CYGPATH: Want directory names and filenames that exist. If the entry doesn't exist you will get undefined results. Also, if the pathname contains spaces, you will need to quote it.

  • MIXED: You want the mixed mode of paths because many executables will not understand the cygwin style of directory names, e.g. /usr/bin, or "/cygdrive/c/Program Files". The MKS dosname returns mixed mode, and the 'cygpath -m' will too.

  • BACKSLASH: Inside a shell script or in a makefile, deal with the \\ -> / transformation first. If something processes your pathnames in a unix way before you deal with this, you will end up with no file separators at all, they will all assumed to be character escapes.

  • DOUBLE SLASH: In the process of transforming pathnames, you sometimes end up with double slashes, make sure you process these out, sometimes they can cause certain utilities problems or make them behave differently, like 'jar'.

  • DRIVES: Watch out for the difference between "C:" and "C:/", they are different. With windows it's usually wiser to always end directory names with /, but inside makefiles that can be a problem, causing the DOUBLE SLASH problem. In general you just need to be careful in Makefiles which variables contain a prefix path (ending with a /) vs. a pathname used like $(DIR)/foobar.c.

  • UNAME: CYGWIN's uname returns "CYGWIN_NT-5.0", or something similar, effectively the version of CYGWIN, not really the version of the system or machine OS. The MKS uname returns something like "Windows_NT". In general CYGWIN is it's own system, and MKS is just a set of Windows utilities.

  • UMASK: MKS seems to have problems with file/directory permissions, usually leaving permissions wide open or ill-defined. CYGWIN does better.

  • Administrator: MKS has had some problems being used by anyone other than the one who installed it, I'm not sure what this problem is. CYGWIN may or may not have this problem.

  • PATH: MKS PATH is the same as Windows PATH, using the ';' path separator character, and must be quoted if it has any embedded spaces. It can contain a mix of \\ and / file separator characters apparently. With CYGWIN PATH is unix style, using : path separator characters and NOT using any C: path names, all pathnames need to be in the cygwin style like /cygdrive/c.

  • ROOTDIR: If MKS is installed on the system, the environment variable ROOTDIR will refer to the installation location. I'm not sure if CYGWIN has any way to know where the base install location is.

  • SHELL: To detect if the shell you are in there are various ways. I usually assume that the MKS or CYGWIN basic utitities are in your search path, so I assume they have fired up a MKS or CYGWIN shell of some kind. This seems to make sense so far. One way is to assume the proper 'uname' is in the search path and use it:

        if [ "$(uname -a | fgrep Cygwin)" = "" -a -d "${ROOTDIR}" ] ; then
            unix_toolset=MKS
            mkshome=$(dosname -s "${ROOTDIR}")
            fullpathprocessor="${mkshome}/mksnt/dosname -s"
            unixcommand_path="${mkshome}/mksnt"
        elif [ "$(uname -a | fgrep Cygwin)" != "" -a -f cygpath ] ; then
            unix_toolset=CYGWIN
            fullpathprocessor="cygpath -a -m -s"
            unixcommand_path="/usr/bin"
        else
            echo "WARNING: Cannot figure out if this is MKS or CYGWIN"
        fi
    Notice that I added the 'cygpath -a' option to get an absolute path. The MKS dosname always returns an absiolute path, so adding the -a option to cygpath makes it do the same thing. Inside a makefile (GNU make) you might:
    ifeq ($(shell uname), Windows_NT)
      PLATFORM = windows
      OS_VERSION := $(shell uname -r)
      WINDOWS_NT_VERSION_STRING=Windows_NT
    endif
    ifneq (,$(findstring CYGWIN,$(shell uname)))
      PLATFORM = windows
      OS_VERSION := 5
      USING_CYGWIN = true
      export USING_CYGWIN
      WINDOWS_NT_VERSION_STRING=CYGWIN_NT
    endif
  • MAKEFILES: The trick to keeping your makefiles sane is to try and stick with the mixed mode paths as much as possible, and isolate the pathname transformation to a make macro:

    ifdef USING_CYGWIN # CYGWIN
    CYGPATH_CMD=cygpath -a -s -m
    define FullPath
    $(subst //,/,$(shell $(CYGPATH_CMD) "$1"))
    endef
    else # MKS
    DOSNAME_CMD:=dosname -s
    define FullPath
    $(subst //,/,$(shell $(DOSNAME_CMD) "$1"))
    endef
    endif
    Note that there is no leading spaces or indenting above. The define macros replicate all spaces, so be careful with these things. Also, note that we add quotes around what we pass dosname or cygpath. To use the above macro you would:
    
    ABS_DIR:=$(call FullPath,"$(subst \\,/,$(DIR))")
    
    Note that the first thing I do is a substitute of \\ with /, then I quote that result and send it to FullPath.
  • PREFIX PATHS: If you need a setting that is a prefix path (ends with /) that could also be empty, I created this macro:

    
    define PrefixPath
    $(if $1,$(subst //,/,$1/),)
    endef
    
    Which also takes care of the // characters, or the fact that what it was given already had a trailing /.
  • EXISTS: Getting dynamic in makefiles is a bit tricky but using the $(shell) you can just about do anything. Here is a macro that picks the first directory that exists or a default value:

    
    define DirExists
    $(shell \\
      if [ -d "$1" ]; then  \\
        echo "$1"; \\
      elif [ -d "$2" ]; then \\
        echo "$2"; \\
      else \\
        echo "$3"; \\
      fi)
    endef
    
  • ENVIRONMENT VARIABLES: It appears that CYGWIN inherits Windows system variables with their names mapped to all uppercase, so things like windir, SystemDrive, SystemRoot, and MSDEVDIR, turn into WINDIR, SYSTEMDRIVE, SYSTEMROOT, and MSDEVDIR. I don't know why. The MKS shells don't seem to have this behavior. Makefiles and scripts may need to check for both names.

  • VCVARS32.BAT: For some reason, it appears that VC6 (Visual Studio 98?) somehow managed to setup a system variable MSDEVDir during installation, regardless of using vcvars32.bat. So after VC7 (Visual Studio .NET 2003) is installed, VC71COMNTOOLS is in the system variables now, but MSDEVDir still refers to the old MSDEV98, at least until the VC7 vcvars32.bat script is run, then MSDEVDir refers to the new .NET 2003 version. So be careful.

Hope this information was helpful, it was rather painful to learn, maybe by posting this I've saved someone some time. Many thanks to my Sun co-workers, especially Martin, Vijay, Tim, Graham, Jim, and others for the input and hints they have provided.

If there are mistakes or errors in the above, please post a comment. Or if you have more to add, please add a comment.

Comments:

Post a Comment:
Comments are closed for this entry.
About

Various blogs on JDK development procedures, including building, build infrastructure, testing, and source maintenance.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today