Setting up an OpenGrok Server
By ford on Jan 05, 2007
In a new job, I have to learn my way around about 800 megabytes of mostly unfamiliar source code — a perfect application for OpenGrok. I had never done anything with J2EE and I had to get it running on a vanilla RedHat system that didn't even have Java installed. It turned out to be pretty easy but I couldn't find a step-by-step guide. Now there is one:
OpenGrok consists of three main components:
- The indexer, which spiders a source tree and builds a cross-reference index database. It can be run automatically, in the background, in a cron job, etc.
- The web application (servlet), which presents a browser-based interface to search and view the source. It runs inside an "applet container" — basically a Java web server.
- The query application, which is a GUI that you can run interactively and works more like "cscope"; i.e. it pops up its own source viewing window and/or your editor.
All three components require access to the same two directories: SRC_ROOT and DATA_ROOT. SRC_ROOT is just a plain tree of source code (or other) files to be cross-referenced. DATA_ROOT is where OpenGrok keeps its cross-reference database. The indexer updates DATA_ROOT incrementally so it goes fast when only some of the files in SRC_ROOT have changed. If you, for example, create a new file in SRC_ROOT, it will show up in the web view immediately, but the symbols it contains won't be in the index and the file can't be found by searching until you re-run the indexer.
Pick locations for these two directories and create them. SRC_ROOT can be a tree that already exists or you can checkout a special sandbox/workspace just for OpenGrok to index. Note that if you index a tree that has had a build run inside it, you will probably have various object files and things of that sort showing up in the browser index (and OpenGrok will index the symbols and strings it finds in .o files, which may or may not be what you want). I find it best to establish a dedicated sandbox for the purpose of grokking the current, checked-in source code, unpolluted by build side-effects. Remember that you need to keep this source sandbox around for use by the web app and the query application. It's OK if you let SRC_ROOT get a little bit out of sync with DATA_ROOT, OpenGrok will still be able to find and display things most of the time.
Requirements and Downloads
As the OpenGrok project page states, the requirements are:
- Latest Java
- A servlet container like GlassFish or Tomcat
- Exuberant Ctags
- Subversion 1.3.0 if subversion support is needed
I didn't attempt to install Subversion support.
It turns out that the system I was using (RedHat Enterprise Linux WS release 3) already had a suitable version of exuberant ctags installed as /usr/bin/ctags so I didn't even bother to install the slightly newer ctags rpm from the SourceForge site.
I installed the RPM version of JDK 5.0 Update 9.
Both Tomcat and OpenGrok are written in Java and download with all class files already built, so no compilation is required (so you could use the JRE instead of the JDK, but having the JDK around is a good thing anyway). Just pick someplace to unpack the tar files for Tomcat and OpenGrok, and they can be run in place; no particular "installation" is required.
Tomcat is essentially a standalone web server that primarily serves J2EE servlets. It has one admin command ("catalina.sh") in its bin subdirectory that will start or stop the daemon (you don't even need to run it as root if you're not using a priviliged TCP port). By default it binds to port 8080 and will automatically find and serve any web applications placed in its "webapps" subdirectory.
I didn't have to do any configuration of Tomcat other than to add two lines near the beginning of catalina.sh:
JAVA_HOME=/usr/java/jdk1.5.0_09; export JAVA_HOME CATALINA_HOME=/...mydirectory.../apache-tomcat-5.5.20; export CATALINA_HOME
For convenience, I made a symlink from catalina.sh to ~/bin/catalina .
Then I ran:
This starts the daemon in the background and it will persist after logout (but not across reboots). I have not attempted to use or understand the rest of the command-line and browser-based administration tools for Tomcat; it works with OpenGrok well enough right out of the box.
Next, it's time to "install" OpenGrok. The easiest way to do that is to copy the pre-built "war" file (web application archive) from the OpenGrok directory into Tomcat's webapps direcory. It's called source.war by default, which results in a Tomcat web application with the URL http://yourhost:8080/source/ . You can rename the war file to something else or even create multiple copies, which would create multiple instances of OpenGrok which could be configured to show different projects/branches, etc.
As soon as you access (from a browser) the URL for the first time, Tomcat unpacks the war file into a subdirectory containing all the files that implement the application. Then you can go in and customize things in that subdirectory (.../webapps/source). I customized as follows:
WEB-INF/web.xml mandatory: This is where you tell the servlet the location of SRC_ROOT and DATA_ROOT. index.html optional: customize the text, logos, etc. help.html optional: customize the text, logos, etc. style.css optional: customize the text, logos, etc.
The web application (http://yourhost:8080/source/) will probably work right away (as soon as you populate SRC_ROOT) for simply viewing the directory hierarchy and individual files. But search queries won't find anything.
To run the indexer, use the program "run.sh" in the OpenGrok directory. You need to make a few small edits to run.sh to set SRC_ROOT, DATA_ROOT, and EXUB_CTAGS. You might want to set PATH right in this file to include the JRE location (/usr/java/jdk1.5.0_09/bin). Also, I had to increase the JRE's heap size for running the indexer: Add "-Xmx200m" as the first argument to the first invocation of java in run.sh (it wouldn't hurt to add it to both). Now just "./run.sh" will run the indexer and populate DATA_ROOT.
Then you can start using the search features, either via the web application or the standalone query application. To run the standalone application, invoke the jar file as a Java program with no arguments:
java -jar .../opengrok-0.3.1/opengrok.jar
You can also re-run the indexer from within the query application.