Thursday Oct 31, 2013

Clustering in GlassFish with DCOM on Windows 7

I've discovered that Windows 7 makes it very difficult to use DCOM and, mainly, the GlassFish clustering commands that rely on DCOM.  I spent a few days trying to solve the problems.  I don't yet have a cookbook for making DCOM work on Windows 7.  But here are a few tips and advice I've found.

  1. run asadmin setup-local-dcom -- It now comes automatically with the open source GlassFish 4.  It will write some critical registry entries for you.
  2.  run asadmin validate-dcom to test dcom

3.   When I ran validate-dcom on my Windows 7 network I saw the problem below:

Successfully resolved host name to: gloin/10.28.51.10 Successfully connected to DCOM Port at port 135 on host gloin.

Successfully connected to NetBIOS Session Service at port 139 on host gloin.

Successfully connected to Windows Shares at port 445 on host gloin.

Can not access the remote file system.  Is UAC on? : Access is denied.

 

I discovered the actual problem is that Windows 7 no longer has the "C$" Administrative file share available by default. If "C$" isn't available then nothing will work. Here is how to expose the "C$" share:

  • Registry Change -- this change allows “C$” to be accessed.  As soon as I set it -- the file copying started working!  [1]

    • regkey: HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System

    • create this key, 32 bit word, with value == 1

  • LocalAccountTokenFilterPolicy

    4.  Turn on the Remote Registry Service -- This is critical and it's easy to do. Windows 7 has it turned off by default. MyComputer-right click, manage, services, then turn on Remote Registry Service and set it to start automatically in the fture.

    5. Turn off UAC: %systemroot%\system32\UserAccountControlSettings.exe

    6. This is where I discovered that McAfee virus scanner blocks all the NetBios shares!  It has to be disabled.

 

Monday Dec 10, 2012

Tip #19 Module Private Visibility in OSGi

I hate public and protected methods and classes.  It requires so much work to change them in a huge project like GlassFish.  Not to mention that you may well have to support those APIs forever.  They are highly overused in GlassFish.  In fact I'd bet that > 95% of classes are marked as public for no good reason.  It's just (bad) habit is my guess.

private and default visibility (I call it package-private) is easier to maintain.  It is much much easier to change such classes and methods around. 

If you have ANY public method or public class in GlassFish you'll need to grep through a tremendous amount of source code to find all callers.  But even that won't be theoretically reliable.  What if a caller is using reflection to access public methods?  You may never find such usages.

If you have package private methods, it's easy.  Simply grep through all the code in that one package.  As long as that package compiles ok you're all set.  There can' be any compile errors anywhere else.  It's a waste of time to even look around or build the "outside" world.

 So you may be thinking: "Aha!  I'll just make my module have one giant package with all the java files.  Then I can use the default visibility and maintenance will be much easier.  But there's a problem.  You are wasting a very nice feature of java -- organizing code into separate packages.  It also makes the code much more encapsulated.  Unfortunately to share code between the packages you have no choice but to declare public visibility.

What happens in practice is that a module ends up having tons of public classes and methods that are used exclusively inside the module.  Which finally brings me to the point of this blog:

 If Only There Was A Module-Private Visibility Available

Well, surprise!  There is such a mechanism.  If your project is running under OSGi that is.  Like GlassFish does!  With this mechanism you can easily add another level of visibility by telling OSGi exactly which public you want to be exposed outside of the module.  You get the best of both worlds:

  • Better encapsulation of your code so that maintenance is easier and productivity is increased.
  • Usage of public visibility inside the module so that you can encapsulate intra-module better with packages.

How I do this in GlassFish:

  1. Carefully plan out at least one package that will contain "true" publics.  This is the package that will be exported by OSGi.  I recommend just one package.
  2. Here is how to tell OSGi to use it in GlassFish -- edit osgi.bundle like so:
    -exportcontents:     org.glassfish.mymodule.truepublics;  version=${project.osgi.version}
  3. Now all publics declared in any other packages will be visible module-wide but not outside the module.

There is one caveat:

Accessing "module-private" items outside of the module is controlled at run-time, not compile-time.  The compiler has no clue that a public in a dependent module isn't really public.  it will happily compile it.  At runtime you will definitely see fireworks.  The good news is that you don't have to wait for the code path that tries to use the "module-private" items to fire.  OSGi will complain loudly when that module gets loaded.  OSGi will refuse to load it.  You will see an error like this:

remote failure: Error while loading FOO: Exception while adding the new configuration : Error occurred during deployment: Exception while loading the app : org.osgi.framework.BundleException: Unresolved constraint in bundle com.oracle.glassfish.miscreant.code [115]: Unable to resolve 115.0: missing requirement [115.0] osgi.wiring.package; (osgi.wiring.package=org.glassfish.mymodule.unexported). Please see server.log for more details.

That is if you accidentally change code in module B to use a public that is really a "module-private" in module A, then you will see the error immediately when you try to test whatever you were changing in module B.


Tuesday Jul 17, 2012

Tip #18 Debug Mode in GlassFish

GlassFish -- both client (asadmin) and server look for an environmental variable.  If it is set to true then you get more detailed verbose information at runtime.  This is not related to the reporting level in the loggers.

AS_DEBUG=true

The environmental variable has to exist AND it has to be set to a String that Boolean.parseString() will see as "true". 

Examples:

Asadmin will dump out much more information.  E.g. if you have it set and run "asadmin start-domain" you will see how that command actually verifies that the domain has started.

Flashlight's job is to instrument classes.  It does this by creating new source code with ASM, compiling it and then replacing the existing class bytes with the new ones.  If AS_DEBUG is true -- it will save the new .class files to disk in an obvious location:

<install-root>/flashlight-generated

There are several Server-side commands that look at AS_DEBUG.  Which ones and what they do is left as an exercise for the reader!

Sunday May 20, 2012

Tip #17 Don't Be Lazy #2

I was debugging some GlassFish code today.  This one Admin Command needed to find out if a given node was local or remote.  It is (obviously, clearly)  impossible for the node to change this attribute during this command call.  The code calls this method:

node.isLocal()

Now if you write a final getter method this is perfectly OK because the compiler will simply inline the code so that it becomes a simple variable access.

But NO WAY is that happening in this case.  The "node" object is a proxy. The value needs to get processed through code that resolves tokens, etc.  The stack shows 8 calls to get this value.  So it is an expensive call.

 I had a breakpoint in this method.  It was called over TWENTY times by the same command!  That's 152 or so pointless method calls.

What it needed was just ONE line of code:

boolean isLocal = node.isLocal();


Saturday May 19, 2012

Tip #16 Don't Be Lazy

Hmmm.  What's wrong with the code below?

I don't know if I've ever seen code screaming out louder for re-factoring.  Obviously there should be ONE method with the 4 lines that are repeated over and over and over and over.  What happens when one of the lines changes?  Well, then FOUR lines will have to change.  Of course if these copy&pasted lines are spread around in a huge file you'll probably miss a few. 

The only reason this can possibly happen is because copy&paste exists.  If we had to type in those identical blocks again and again -- THEN we would re-factor into a method call!

       if (!env.isDas()) {
            String msg = Strings.get("notAllowed");
            logger.warning(msg);
            report.setActionExitCode(ActionReport.ExitCode.FAILURE);
            report.setMessage(msg);
            return;
        }

        // Make sure Node is valid
        theNode = nodes.getNode(node);
        if (theNode == null) {
            String msg = Strings.get("noSuchNode", node);
            logger.warning(msg);
            report.setActionExitCode(ActionReport.ExitCode.FAILURE);
            report.setMessage(msg);
            return;
        }

        if (lbEnabled != null && clusterName == null) {
            String msg = Strings.get("lbenabledNotForStandaloneInstance");
            logger.warning(msg);
            report.setActionExitCode(ActionReport.ExitCode.FAILURE);
            report.setMessage(msg);
            return;
        }       

        if (demo != null) {
            String msg = Strings.get("demoOnly");
            logger.warning(msg);
            report.setActionExitCode(ActionReport.ExitCode.FAILURE);
            report.setMessage(msg);
            return;
        }

Thursday Apr 12, 2012

Tip #15: How To Debug Unit Tests During Maven Builds

It must be really really hard to step through unit tests in a debugger during a maven build.  Right?

Wrong!

Here is how i do it:

1) Set up these environmental variables:

MAVEN_OPTS=-Xmx1024m -Xms256m -XX:MaxPermSize=512m
MAVEN_OPTS_DEBUG=-Xmx1024m -Xms256m -XX:MaxPermSize=512m  -Xdebug (no line break here!!)  -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=9999
MAVEN_OPTS_REG=-Xmx1024m -Xms256m -XX:MaxPermSize=512m

2) create 2 scripts or aliases like so:

 maveny.bat:

set MAVEN_OPTS=%MAVEN_OPTS_DEBUG%

mavenn.bat:

set MAVEN_OPTS=%MAVEN_OPTS_REG%

 


 To debug do this:

  1. run maveny.bat
  2. run mvn install
  3. attach your debugger to port 9999 (set breakpoints of course)
  4. When maven gets to the unit test phase it will hit your breakpoint and wait for you.

When done debugging simply run mavenn.bat

Notes

  1. If it takes a while to do the build then you don't really need to set the suspend=y flag.
  2. If you set the suspend=n flag then you can just leave it -- but only one maven build can run at a time because of the debug port conflict.



Friday Jan 06, 2012

Tip #14 Symbolic Links on Windows?!?

Windows does have real UNIX-like symbolic links.  They are called junctions.  There are no built-in Windows tools for using them though. 

SysInternals has a great tool for using them.  Note that Microsoft owns SysInternals now.

Try it: http://technet.microsoft.com/en-us/sysinternals/bb896768


Thursday Nov 10, 2011

Tip #13 java.io.File Surprises

There is an assumption that I've seen in code many times that is totally wrong.  And this assumption can easily bite you.  The assumption is:

File.getAbsolutePath and getAbsoluteFile return paths that are not relative.

 Not true!  Sort of.  At least not in the way many people would assume.  All they do is make sure that the beginning of the path is absolute.  The rest of the path can be loaded with relative path elements.  What do you think the following code will print?

public class Main {
    public static void main(String[] args) {
        try {
            File f = new File("/temp/../temp/../temp/../");
            File abs  = f.getAbsoluteFile();
            File parent = abs.getParentFile();
            System.out.println("Exists: " + f.exists());
            System.out.println("Absolute Path: " + abs);
            System.out.println("FileName: " + abs.getName());
            System.out.printf("The Parent Directory of %s is %s\n", abs, parent);
            System.out.printf("The CANONICAL Parent Directory of CANONICAL %s is %s\n",
                        abs, abs.getCanonicalFile().getParent());
            System.out.printf("The CANONICAL Parent Directory of ABSOLUTE %s is %s\n",
                        abs, parent.getCanonicalFile());
            System.out.println("Canonical Path: " + f.getCanonicalPath());
        }
        catch (IOException ex) {
            System.out.println("Got an exception: " + ex);
        }
    }
}

Output:

Exists: true
Absolute Path: D:\temp\..\temp\..\temp\..
FileName: ..
The Parent Directory of D:\temp\..\temp\..\temp\.. is D:\temp\..\temp\..\temp
The CANONICAL Parent Directory of CANONICAL D:\temp\..\temp\..\temp\.. is null
The CANONICAL Parent Directory of ABSOLUTE D:\temp\..\temp\..\temp\.. is D:\temp
Canonical Path: D:\

Notice how it says that the parent of d:\ is d:\temp !!!The file, f, is really the root directory.  The parent is supposed to be null.

I learned about this the hard way! getParentXXX simply hacks off the final item in the path. You can get totally unexpected results like the above. Easily.

I filed a bug on this behavior a few years ago[1].  

Recommendations:

(1) Use getCanonical instead of getAbsolute.  There is a 1:1 mapping of files and canonical filenames.  I.e each file has one and only one canonical filename and it will definitely not have relative path elements in it.  There are an infinite number of absolute paths for each file.

(2) To get the parent file for File f do the following instead of getParentFile:

File parent = new File(f, "..");

[1] http://bt2ws.central.sun.com/CrPrint?id=6687287



Wednesday Oct 12, 2011

Tip#12 Waste of time setting instance variables to their default.

I see this code frequently:

class foo {

private String s1 = null;
private boolean b1 = false;
int i1 = 0;
....
}

This is a waste of time.  Those variables are NOT on the stack.  They are always guaranteed to be equal to the default that is defined for that data type.  By setting them to a value, special code has to be written and run automatically when the instance is instantiated.  E.g. the String is initially set to null automatically.  Then code is run which once again sets it to null.

 

class foo {
private String s1;
private boolean b1;
int i1;
....
}

==============

On the other hand you have to set initial values for local variables.  There storage space is allocated on the fly and they are on the stack.  Therefore they are initialized to whatever garbage happens to be on the stack.


Tuesday Oct 04, 2011

Tip #11 Subversion Low-Tech Safety Net

Have you ever lost a significant  amount of source code?  If you have you probably already have multiple layers of safety nets.  This stuff is just too hard to do to take a chance on hardware failures, hickups, etc.  

E.g. I remember once I thought I was in my temp directory.  I entered this command:

rm -rf *

Oops.  I was in the root directory!  I.e. stuff indeed happens. 

Today's excruciatingly simple tip:

When you are actively developing code but it is not yet ready to be checked-in.  Yet you have spent a significant chunk of time on it.  I do this:

svn status | sort | tee dome.bat

I edit the dome.bat file and change this:

?       comm-util-3.1.2.save.jar
?       d
?       diffs.txt
?       dome.bat
?       q
A       src\main\java\com\sun\enterprise\universal\process\WindowsCredentials.java
A       src\main\java\com\sun\enterprise\universal\process\WindowsException.java
A       src\main\java\com\sun\enterprise\universal\process\WindowsRemoteAsadmin.java
A       src\main\java\com\sun\enterprise\universal\process\WindowsRemotePinger.java
A       src\main\java\com\sun\enterprise\universal\process\WindowsRemoteScripter.java
A       src\main\java\com\sun\enterprise\util\io\WindowsRemoteFile.java
A       src\main\java\com\sun\enterprise\util\io\WindowsRemoteFileCopyProgress.java
A       src\main\java\com\sun\enterprise\util\io\WindowsRemoteFileSystem.java
M       pom.xml
M       src\main\java\com\sun\enterprise\universal\glassfish\TokenResolver.java

to this:

       jar cvfM comm-util-3.1.2.save.jar src\main\java\com\sun\enterprise\universal\process\WindowsCredentials.java src\main\java\com\sun\enterprise\universal\process\WindowsException.java src\main\java\com\sun\enterprise\universal\process\WindowsRemoteAsadmin.java src\main\java\com\sun\enterprise\universal\process\WindowsRemotePinger.java src\main\java\com\sun\enterprise\universal\process\WindowsRemoteScripter.java src\main\java\com\sun\enterprise\util\io\WindowsRemoteFile.java src\main\java\com\sun\enterprise\util\io\WindowsRemoteFileCopyProgress.java src\main\java\com\sun\enterprise\util\io\WindowsRemoteFileSystem.java pom.xml src\main\java\com\sun\enterprise\universal\glassfish\TokenResolver.java

Now  I simply run the script and email the jar file to my work email account.  My company will be happy to guarantee safety for my jar file. 


Friday Sep 30, 2011

Tip#10 Technique for making your class Immutable

Immutable classes are great.  You never have to worry about concurrency problems.  It also makes the code more readable - if a variable is final you don't have to look around for the other places where it gets modified -- there aren't any. Also the compiler will help you.  If you forget to assign a final variable it's a compile error.  It's also an error to try and reassign it.

Here is a simple trivial technique for setting final variables from a constructor where you need to do processing first.  The point is that you use temp variables to get your final values all lined up.  Then you assign the final values once.

I'll just let the code write the rest of this blog:

public final class DcomInfo {
    private final WindowsCredentials credentials;
    private final Node node;
    private final String password;
    private final String host;
    private final String user;
    private final String windowsDomain;
    private final String remoteNodeRootDirectory;

    public DcomInfo(Node theNode) throws WindowsException {
        node = theNode;

        if (node == null)
            throw new WindowsException(
                    Strings.get("internal.error", "Node is null"));

        if (!isDcomNode(node))
            throw new WindowsException(Strings.get("not.dcom.node",
                    getNode().getName(), getNode().getType()));

        SshConnector conn = node.getSshConnector();
        if (conn == null)
            throw new WindowsException(Strings.get("no.password"));

        SshAuth auth = conn.getSshAuth();
        if (auth == null)
            throw new WindowsException(Strings.get("no.password"));

        String notFinal = auth.getPassword();
        if (!ok(notFinal))
            throw new WindowsException(Strings.get("no.password"));

        password = DcomUtils.resolvePassword(notFinal);

        notFinal = node.getNodeHost();
        if (!ok(notFinal))
            notFinal = conn.getSshHost();
        if (!ok(notFinal))
            throw new WindowsException(Strings.get("no.host"));
        host = resolver.resolve(notFinal);

        notFinal = auth.getUserName();
        if (!ok(notFinal))
            notFinal = System.getProperty("user.name");
        if (!ok(notFinal))
            throw new WindowsException(Strings.get("no.username"));
        user = resolver.resolve(notFinal);

        notFinal = node.getWindowsDomain();
        if (!ok(notFinal))
            notFinal = host;
        windowsDomain = resolver.resolve(notFinal);

        notFinal = node.getInstallDirUnixStyle();
        if (!ok(notFinal))
            throw new WindowsException(Strings.get("no.lib.dir"));

        if (!notFinal.endsWith("/"))
            notFinal += "/";

        notFinal += SystemPropertyConstants.getComponentName();
        remoteInstallRoot = StringUtils.quotePathIfNecessary(notFinal);
        notFinal += "/lib";
        notFinal = StringUtils.quotePathIfNecessary(notFinal);
        notFinal = notFinal.replace('/', '\\');
        nadminParentPath = notFinal;
        nadminPath = notFinal + "\\nadmin.bat";

        String notFinal2 = node.getNodeDirAbsolute();

        if (notFinal2 == null) {
            // no special nodedir -- use the defaults
            notFinal2 = remoteInstallRoot;  // e.g. "d:/glassfish3/glassfish"
            notFinal2 += "/nodes";
        }
        notFinal2 = notFinal2.replace('/', '\\');

        if (!notFinal2.endsWith("\\"))
            notFinal2 += '\\';

        remoteNodeRootDirectory = notFinal2 + node.getName();

        credentials = new WindowsCredentials(getHost(), getWindowsDomain(),
                getUser(), getPassword());
    }

Wednesday Sep 28, 2011

Tip #9 - Advanced Debugger Attach

GlassFish has server-side commands, for instance create-instance, that in-turn call the client side.  It generally does this by calling asadmin's cousin:

glassfish/lib/nadmin[.bat]

with the command. 

But what if you need to debug the spawned client JVM?  Impossible?  No!  Trivial!

Simply do this:

edit nadmin (or nadmin.bat if you are using Windows) and add the java debugging args.

e.g.

-Xdebug -Xrunjdwp:transport=dt_socket,server=y,suspend=y,address=1234

 Now simply run the server-side command and then leisurely attach a debugger to port 1234 (make sure you have a breakpoint waiting or you'll miss out). 

If you use NetBeans you can easily debug both server and client at the same time.

Don't forget to undo nadmin when you are done! 


Tuesday Sep 27, 2011

Tip #8 Watch Those RegEx calls -- they are S-L-O-W

I see this construct quite a bit:

String someString ....
someString.replaceAll("\\\\","/");

This is probably a huge waste of time (probably because I didn't time it).  Not only does it do much much much more complex regular expression parsing, for no good reason, it is also much less readable then the "correct" way to do it:

String someString ....
someString.replace('\\','/');


Monday Sep 26, 2011

Tip #7 - Absolute Filenames in Java

I'm busy reading lots and lots of code lately because I need to implant DCOM support.  I just saw this:

  if (nodeDirFile.isAbsolute()) {
                return nodeDir;
            }

Would this surprise you?

File f = new File("/foo/../foo/.././././././foo/././././././../foo/goo");

f.isAbsolute()
will return true.  That is an absolute path as far as Java IO is concerned.  I.e. if it BEGINS with something absolute -- then it is absolute.  Even if it's really really ugly.

What was probably desired is getCanonical().  There is one and only canonical filenames per file.  There are infinite "absolute" filenames per file.


Monday Aug 15, 2011

Tip #6 Windows Script to change directory QUICKLY

GlassFish's source tree, like any huge software has enormously long directory paths.  My goal is to try to avoid ever typing them in. Even with the shell auto-filling it still takes too much time, IMHO.

I created 2 scripts on my Windows systems:

1)  "cdn.bat"

cd \gf\main\nucleus\*%1*\*%2*

2) "cda.bat"

cd \gf\main\appserver\*%1*\*%2*

==============

If I want to change my directory to,say flashlight-framework I just do this:

cdn fl fr

This command instantly moves me to cluster-cli

cdn clu cl



About

ByronNevins

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today