Real-World Bytecode Handling with ASM

Both monitorenter and monitorexit bytecodes consume the top of the stack and lock or unlock the object that they find there. So if these opcodes were replaced with a basic pop, this would strip the synchronization out of any method body that is encountered.

The resulting code is represented by the following two simple classes: an UnsynchronizingClassVisitor and an UnsynchronizingMethodVisitor, both of which extend ASM framework classes:

public class UnsynchronizingClassVisitor extends ClassVisitor {

    public UnsynchronizingClassVisitor(int api, ClassVisitor cv) {
        super(Opcodes.ASM5, cv);
    }

    @Override
    public MethodVisitor visitMethod(int flags, String name, 
      String desc, String signature, String[] exceptions) {
        int maskedFlags = flags & (~ACC_SYNCHRONIZED);

        MethodVisitor baseMethodVisitor = 
            super.visitMethod(maskedFlags, name, desc, 
                              signature, exceptions);

        return new UnsynchronizingMethodVisitor(baseMethodVisitor);
    }
}

The UnsynchronizingClassVisitor class uses a Decorator pattern: it takes the baseMethodVisitor and wraps it by adding functionality that is called only when a no-argument opcode is encountered in the body of the method, as shown in this code:

public class UnsynchronizingMethodVisitor extends MethodVisitor {
    public UnsynchronizingMethodVisitor(MethodVisitor mv) {
        super(Opcodes.ASM5, mv);
    }

    @Override
    public void visitInsn(final int opcode) {
        switch (opcode) {
            case Opcodes.MONITORENTER:
            case Opcodes.MONITOREXIT:
                super.visitInsn(Opcodes.POP);
                return;
        }

        super.visitInsn(opcode);
    }
}

I use the following bit of code to drive this transformation:

try (InputStream in = 
        Files.newInputStream(Paths.get(fName))) {
    ClassReader classReader = new ClassReader(in);
    ClassWriter writer = 
       new ClassWriter(classReader, ClassWriter.COMPUTE_FRAMES);

    ClassVisitor unsynchronizer = 
       new UnsynchronizingClassVisitor(writer);
    classReader.accept(unsynchronizer, 
       ClassReader.SKIP_FRAMES | ClassReader.SKIP_DEBUG);

   Path newClazz = Paths.get(transformName(fName));
   Files.write(newClazz, writer.toByteArray());
} catch (Exception ex) {
    System.err.println(
        "Exception whilst reading class: " + fName);
    ex.printStackTrace(System.err);
}

Now, if I take a synchronized version of the Counter class, I can run it through the OfflineUnsynchronizer, and the resulting transformed class will suffer the lost-update problem even though the original code was safe.

Ruling Out Native Code

Java bytecode is platform-independent, so it cannot call operating system libraries directly (for example, to handle I/O operations). Instead, Java programs (including the JDK) call out to native methods (written in C) that in turn call the relevant parts of the operating system.

Suppose you have a use case where you want to allow users to execute unknown code as part of a framework or container. Such a capability has obvious security concerns, so you might want to reduce the risk by disallowing certain actions—such as running native methods—in the users’ classes. Fortunately, the Java security model relies on class loading, and it allows you to hook into the loading process to customize how (and whether) new code is loaded. [For more on how class loading works, see the article “How the JVM Locates, Loads, and Runs Libraries” by Oleg Selajev, which you can download as a PDF. —Ed.]

The overall scheme could look like this:

• Write a custom class loader.
• During class loading, inspect every “call site” where a method is called.
• Check to see whether the metadata for the method indicates that the method is native.
• If it is, reject the class and fail class loading.
• If you reach the end without failing, the class is good and can be loaded.

Here’s how to write a class loader that will reject any non-pure Java classes it is asked to load:

public final class PureJavaClassLoader extends ClassLoader {
    private final List<String> auxClasspath = new ArrayList<>();

    public PureJavaClassLoader(ClassLoader parent) {
        super(parent);
    }

    public void setupClasspath(final String auxiliaryClassPath) {
        for (String entry : auxiliaryClassPath.split(":")) {
            if (entry.startsWith("/")) {
                auxClasspath.add(entry);
            } else {
                System.err.println(
                    "Bad classpath entry seen: " + 
                     entry + ", ignoring");
            }
        }
    }

    Path findClassFile(String qualifiedClassName)throws IOException {
        final String fileName = 
            qualifiedClassName.replaceAll("/", "\\.") +".class";
        for (String s : auxClasspath) {
            Path trial = Paths.get(s, fileName);
            if (trial.toFile().exists())
                return trial;
        }

        throw new IOException("Class "+ qualifiedClassName +
                              " not found on classpath");
    }
}

For simplicity, I’ll manage an auxiliary class path of directories that I want to search for classes to load, rather than using the main class path. The findClassFile method is a helper that locates the file corresponding to a qualified class name. The real action is in the findClass method to which class loaders delegate from loadClass(). This is where I implement the check for native code:

@Override
public Class<?> findClass(final String qualifiedClassName) throws
  ClassNotFoundException {
    
    Class<?> cls = null;
    try {
        return super.findClass(qualifiedClassName);
    } catch (ClassNotFoundException ignored) {
        try (final InputStream in = Files.newInputStream(
                findClassFile(qualifiedClassName))) {
                    final byte[] allClassBytes = in.readAllBytes();
                    final ClassReader classReader = 
                        new ClassReader(allClassBytes);
                    final PureJavaCheckingClassVisitor 
                        classVisitor = 
                          new PureJavaCheckingClassVisitor();

                // If there's debug info in the class, 
                // don't look at it
                    classReader.accept(
                        classVisitor, ClassReader.SKIP_DEBUG);

                    if (classVisitor.containsNative()) {
                        throw new ClassNotFoundException(
                       "Class cannot be loaded - contains native code");
                    } else {
                        return defineClass(null, allClassBytes, 0,
                                           allClassBytes.length);
                    }
        } catch (IOException e) {
                throw new ClassNotFoundException(
                    "Error finding and opening class", e);
        }
    }
}

In the previous code, I call in.readAllBytes() directly, rather than passing in to the ClassReader constructor. This is because the ASM class ClassReader consumes input streams, so I can’t reuse in after it’s been used to create a class reader.

Next, I create an instance of our custom class visitor, PureJavaCheckingClassVisitor. This visitor simply visits the metadata for each method in the class being considered and records whether any method is native. It is defined as the following:

public class PureJavaCheckingClassVisitor extends ClassVisitor {
    private boolean containsNative = false;

    public PureJavaCheckingClassVisitor() {
        super(Opcodes.ASM5);
    }

    @Override
    public MethodVisitor visitMethod(int flags, String name, 
      String desc, String signature, String[] exceptions) {
        if ((flags & ACC_NATIVE) > 0) {
            containsNative = true;
        }

        return new MethodVisitor(Opcodes.ASM5) {};
    }

    public boolean containsNative() {
        return containsNative;
    }
}

If the class visitor ever sees a native method, it sets a flag. The flag is read by the PureJava-ClassLoader, which rejects the class with a ClassNotFoundException if the flag has been set. This exception is used, rather than the alternative natural choice (SecurityException), because the contract of ClassLoader (which is the supertype of this class) uses the checked exception ClassNotFoundException. In this circumstance, use of a runtime exception (such as Security- Exception) could violate some expectations of clients of the classloader.

Assuming that an exception has not been thrown, the bytes of the class file are fed to defineClass(), which is a protected method defined on ClassLoader so it is accessible only to subclasses—effectively custom class loaders. This returns the Class<?> object that I return from findClass(), and the class is successfully loaded.

Conclusion

A word of caution: the previous example will indeed prevent any classes with native methods from being loaded. However, in a real environment, you would also have to take into account other cases, such as the following:

• Code that calls a native method of an already-loaded class (the transitive case)
• Reflective access to native methods
• Invocation of native methods via the MethodHandles interface

Not only that, but some native methods are essential for proper functioning of virtually all Java programs (such as getClass() or Object::hashCode).

A full discussion of what would be required to fully restrict native code from running is too far afield for this article. In practice, some sort of approved list of core native methods within the JDK would have to be used. Nevertheless, note the things I did with ASM in the example: I read through bytecodes for a given release of Java, skipped over debugging data, and identified specific bytecodes. And earlier, I transformed bytecodes on the fly.