Sunday Nov 23, 2008

JMake source repository (formerly called javamake)

The Mercurial repository for the source to the Sun experimental tool "jmake" formerly called "javamake" is now available at the kenai.com site at kenai.com/projects/jmake.

I've noticed some confusion about tools called "jmake", "javamake", and "jmk". I'm pretty sure these projects are all unrelated.

The jmake wiki page is mostly copied from the Sun experimental site with a few added bits of information from me.

A Mercurial repository can be obtained by using:

hg clone https://kenai.com/hg/jmake~mercurial jmake

-kto

Monday Nov 17, 2008

OpenJDK and bugs.sun.com

Yeah yeah, I haven't done a blog in a while.

For the most part I work on build related bugs in most of the JDK releases, so I thought it might be helpful for people to see what kind of build bugs we are talking about. Granted this is via bugs.sun.com and is a bit of a "one way street" for the open source community, but the content is still valid. Once we have a true open source bugzilla instance working for OpenJDK, the mechanism changes (becoming more open of course).

In the current bug tracking system you see via bugs.sun.com we are using has a category ("java" in my examples) a subcategory ("build" in my example), and I'm using the bugStatus selection which is a number between 1 and 11.

So here are the JDK Build Change Requests (CRs) (CRs cover both bugs and rfes) for all releases, listed based on the state of the bug:

  1. Dispatched (Submitted but not officially accepted yet)
  2. Incomplete (Some kind of missing information)
  3. Accepted (Someone has at least taken a quick look at it)
  4. Defer (No work being done on it for some reason, rare state to be in)
  5. Cause Known (Generally it's understood what the cause is)
  6. Fix Understood (The necessary fix is understood)
  7. Fix in Progress (The fix is in progress)
  8. Fix Available (The fix has been implemented somewhere, access may be limited)
  9. Fix Failed (The fix does not work, rare state, usually goes back to Fix In Progress)
  10. Fix Delivered (The fix has been pushed into an official repository forest, official is public for OpenJDK)
  11. Closed (All activity on this CR has stopped, could be a verified fix, could be closed as not a bug)

The general rules about the bug states have a little flexibility between teams and products, but the above is pretty close for most of them. It's normal that when a bug is Accepted, that it gets an engineer assigned to it, but that is not a hard and fast rule. Also, many bugs reach the "Fix Delivered" state and remain there forever, never being changes to "Closed" as a verified bug, not exactly sure why that happens.

-kto

Sunday May 11, 2008

OpenJDK Bug States

Until the OpenJDK project converts to Bugzilla, I thought some more information about how our internal bug tracking system works might help people watching the current situation.

The OpenJDK change requests (CRs) or bugs are currently visible at bugs.sun.com.

There are 11 states for a CR and the normal change of state is the following:

  1. Dispatched: The initial state for a new CR.
  2. Incomplete: Something critical is missing from the CR.
  3. Accepted: CR was accepted, the first step in being investigated and fixed.
  4. Defer: Work on this CR is on hold.
  5. Cause Known: A basic understanding of the problem is known.
  6. Fix Understood: A basic fix is now understood.
  7. Fix in Progress: The assigned engineer is actively working on the fix or getting it integrated.
  8. Fix Available: A changeset or the actual fix has been made available in a team area.
  9. Fix Failed: Something went wrong with the fix.
  10. Fix Delivered: The changeset or fix has been integrated into the master area and will show up in the next build promotion.
  11. Closed: There are many reasons why a bug will be in the closed state. It might be verified as fixed, closed as a duplicate, closed as 'not a bug' or closed as 'will not fix'.

The states 1-9 are considered "unresolved", so until a bug becomes "10 - Fix Delivered" or "11 - Closed", it is still considered unresolved. So being "unresolved" may mean that a fix is available, just not in a position to be made part of a formal build promotion.

The bugs.sun.com interface is read-only and somewhat crude, but you can query the bug information, for example the Unresolved JDK Build Subcategory Bugs and RFEs. Hope this helps explain things.

-kto

Wednesday May 07, 2008

Sun Studio Compilers on LINUX?

With the magic of Mercurial, you can see changesets, like this one: http://hg.openjdk.java.net/jdk7/jdk7/hotspot/rev/485d403e94e1. Which Serguei Spitsyn integrated recently.

But wait, what does this changeset actually mean? Sun Studio on Linux? Does that make sense? YES! It does and it's true. Mind you, it's just Hotspot that can be built with the Sun Studio compilers on Linux right now, but it's an important piece, the Hotspot C++ code is not a trivial pile of code to compile and optimize correctly. My hat is off to the Sun Studio team in making this all happen. What should be interesting now is how well the rest of the tools work like dbx and the analyzer/collector.

More can be read about the Sun Studio Linux Compilers at the Sun Studio Site.

Humm, I guess I'm now on the hook to see if the rest of the OpenJDK can be setup to build with Sun Studio on Linux.... Back to work...

-kto

P.S. No, we are not abandoning gcc/g++, just providing choices on how the OpenJDK can be built.

InfoQ Article on Git, Mercurial, and Bzr

A while back S├ębastien Auvray asked me some questions about the OpenJDK Mercurial conversion. His article was recently published at http://www.infoq.com/articles/dvcs-guide.

Has some interesting information and stats about Git, Mercurial, and Bzr.

-kto

Wednesday Apr 30, 2008

OpenJDK7, Solaris, and Sun Studio 12 Compilers

Updated 4/30/2008: Added more configuration information

We are starting work to update the compilers and build OS releases used to build OpenJDK7 and JDK7. The implications of a build system OS change may mean that the JDK7 binary bundles may support fewer OS releases. So this is a bit of a status report, and a bit of a heads up. Hopefully most people will cheer over this news, but we'll see.

Solaris

On Solaris we have already moved to Solaris 10 as the base OS, which means that the build bundles will no longer run on Solaris 8 or Solaris 9 systems. We will also be moving from the Sun Studio 11 compilers to the Sun Studio 12 compilers. Sun Studio 12 is FREE for SDN members (registration is free too). Keep in mind that this doesn't mean that you can't build with Sun Studio 11, just that we will focus on using Sun Studio 12. At some point using older tools or OS releases to build the OpenJDK may become an issue, but it is not our intent to create these issues. There are some places in the JDK where we would like to depend on some newer features of the OS being available, so there are no promises that the OpenJDK7 sources will stay buildable on Solaris 8 or 9, but currently they are.

The Solaris Makefile changes for Sun Studio 12 (SS12) will include changing to new -xarch options as described in the "What's New In The Compilers" document. The old -xarch options will continue to work fine, but generate warning errors, and so the change to the new options is to avoid the warning errors.

Note that since we still use the assembler that comes with the Solaris system at /usr/ccs/bin/as, we still have to supply it with the old option spellings.

We did run into two build issues so far:

  • Using -xO4 built .o files with dtrace is giving us some problems. We suspect the optimizer may have tossed some C++ methods that exist only for dtrace probes. We had to change -xO4 to -xO3 -g on three files in hotspot.
  • We had to revert the C++ debug format from Dwarf2 to stabs with the option -xdebugformat=stabs. We were getting some ld errors on some Elf sections during link time. We suspect this is a known problem with Elf COMDAT sections and will be watching for a fix.

It may be a while before we push these Makefile changes to the OpenJDK repositories. Changing compilers with the JDK on any platform is not as trivial a task as people might think. With Solaris and the Sun Studio compilers, we have an advantage, knowing the team doing the compilers. But with any compiler change, our biggest issue becomes performance and correctness. Not so much the performance and correctness of the C/C++ generated code, but the combination of this C/C++ code with the Hotspot generated code. It's not unusual to have bad runtime interactions between the C/C++ generated code, and the Hotspot generated code, on any platform and with any compiler revision.

So the next steps will involve running formal benchmarks to verify that we haven't regressed in performance. A somewhat difficult task given the moving target that OpenJDK7 is right now.

Linux

We already know that building the OpenJDK with different Linux releases and gcc compiler versions is working, so with Linux we just need to advance to a slightly newer Linux release as the base OS, and one that gives us the largest set of deployed targets. We still run the risk of performance and correctness issues like with Solaris, so the same benchmark exercises will need to be done. This will mean that the JDK7 binary bundles will no longer run on some older Linux systems. That doesn't mean it can't be built for these versions, just that the binary bundles we supply may be limited in the Linux releases it supports.

Windows

There is some efforts going on to see about changing to the Visual Studio .NET 2008 compilers, and potentially Windows XP as the base OS for 32bit. Again, dito on the performance and correctness issues. This will mean that the JDK7 binary bundles will no longer work on Windows 2000.

Official Build Configurations

The current list of build machines for JDK7 is documented in the JDK7 Build README:

  • Solaris 10 SPARC
  • Solaris 10 SPARCV9
  • Solaris 10 X86
  • Solaris 10 X64
  • Linux X86 Redhat Enterprise Advanced Server 2.1 update 2
  • Linux X64 Suse 8 Enterprise Server - AMD64 Edition
  • Windows 2000 X86
  • Windows 2003 X64

Official Supported Configurations

The Official JDK6 supported configurations are those configurations that we have tested the JDK6 builds on. The official supported list for JDK7 will need to be adjusted of course, certainly Solaris 8 and Solaris 9 support for JDK7 will be removed.

Other Configurations

The OpenJDK sources can be built on many more systems as documented in the OpenJDK Build README. The intent with the OpenJDK is to allow for it to be built on as many configurations as possible, but that does not necessarily mean that a configuration will become any kind of officially supported JDK7 configuration. The OpenJDK project is an open-source project, and to date, binary builds have not been provided.

Other architectures like IA64 or PowerPC are or may be considered "ports", depending on the need for changing the Hotspot VM to generate code for a different architecture. There are several "ports" being talked about on the OpenJDK porters alias, see the OpenJDK Mailing Lists. Recently, changes to Hotspot were pushed into the OpenJDK source repositories that allowed for Linux SPARC builds of Hotspot, see the changeset Open-source hotspot linux-sparc support. Email exchanges on the distro-pkg alias indicate people are building on Linux IA64, not sure if they have been successful or not, I have no idea how IA64 this would be. Of course, successfully building is only step 1, the next step is to see if it works, and works reliably. The point being that many people are taking the OpenJDK sources or IcedTea and building on many configurations.

The funding needed to take on and support a new official configuration is considerable. We hope that the OpenJDK developers contribute back the changes made to port or build on any configuration. But adding a new configuration to the official JDK7 build configuration list is a high level decision.

-kto

About

Various blogs on JDK development procedures, including building, build infrastructure, testing, and source maintenance.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today