Saturday May 17, 2008

How Does "Mark Occurrences" Work?

Yesterday's code generator was made possible by Editor Library 2: "The NetBeans editor infrastructure provides an implementation of the Swing Text package APIs as well as some additional features such as syntax coloring, code folding, braces matching, etc."

The Code Generator SPI is in that module, but so is the Highlighting SPI. There's several usecases that relate to this SPI, as you can read in that link. Yesterday I came across a very cool tutorial that describes one of them: Extending the C/C++ Editor in NetBeans IDE 6.0 to Provide Mark Occurrences Highlighting. That's an excellent tutorial by Sergey Grinev from the NetBeans C++ team in St. Petersburg. Let's get to know this SPI while doing something pretty useful: we'll create a mark occurrences plugin for HTML files on the NetBeans Platform (i.e., typically, in NetBeans IDE, but also applicable to any other HTML file open in an application on the NetBeans Platform):

Above, I selected one instance of "you" and then all matching instances were highlighted. Try it. Doesn't work, does it? That's because this functionality is not part of the IDE, but part of my plugin. If you're interested in this functionality as a user, get it here from the Plugin Portal:

http://plugins.netbeans.org/PluginPortal/faces/PluginDetailPage.jsp?pluginid=9441

To create the above, I simply copied Sergey's code, except that I read the content of HTML files rather than C++ (or C) files and then changed the layer.xml accordingly. My MarkOccurrencesHighlightsFactory is identical to Sergey's:

package org.netbeans.modules.markoccurrences;

import javax.swing.text.Document;
import org.netbeans.spi.editor.highlighting.HighlightsLayer;
import org.netbeans.spi.editor.highlighting.HighlightsLayerFactory;
import org.netbeans.spi.editor.highlighting.ZOrder;

public class MarkOccurrencesHighlightsLayerFactory implements HighlightsLayerFactory {

    public static MarkOccurrencesHighlighter getMarkOccurrencesHighlighter(Document doc) {
        MarkOccurrencesHighlighter highlighter = (MarkOccurrencesHighlighter) doc.getProperty(MarkOccurrencesHighlighter.class);
        if (highlighter == null) {
            doc.putProperty(MarkOccurrencesHighlighter.class, highlighter = new MarkOccurrencesHighlighter(doc));
        }
        return highlighter;
    }

    @Override
    public HighlightsLayer[] createLayers(Context context) {
        return new HighlightsLayer[]{
                    HighlightsLayer.create(
                    MarkOccurrencesHighlighter.class.getName(),
                    ZOrder.CARET_RACK.forPosition(2000),
                    true,
                    getMarkOccurrencesHighlighter(context.getDocument()).getHighlightsBag())
                };
    }

}

Here's the layer.xml registration:

<folder name="Editors">
    <folder name="text">
        <folder name="html">
            <file name="org-netbeans-modules-markoccurrences-MarkOccurrencesHighlightsLayerFactory.instance"/>
        </folder>
    </folder>
</folder>

And, finally, the CaretListener, which picks up whatever is selected and then finds matching occurrences:

package org.netbeans.modules.markoccurrences;

import java.awt.Color;
import java.lang.ref.WeakReference;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import javax.swing.JEditorPane;
import javax.swing.event.CaretEvent;
import javax.swing.event.CaretListener;
import javax.swing.text.AttributeSet;
import javax.swing.text.Document;
import javax.swing.text.JTextComponent;
import javax.swing.text.StyleConstants;
import org.netbeans.api.editor.settings.AttributesUtilities;
import org.netbeans.modules.editor.NbEditorUtilities;
import org.netbeans.spi.editor.highlighting.support.OffsetsBag;
import org.openide.cookies.EditorCookie;
import org.openide.loaders.DataObject;
import org.openide.util.RequestProcessor;

public class MarkOccurrencesHighlighter implements CaretListener {

    private static final AttributeSet defaultColors =
            AttributesUtilities.createImmutable(StyleConstants.Background,
            new Color(236, 235, 163));
    private final OffsetsBag bag;
    private JTextComponent comp;
    private final WeakReference weakDoc;

    public MarkOccurrencesHighlighter(Document doc) {
        bag = new OffsetsBag(doc);
        weakDoc = new WeakReference((Document) doc);
        DataObject dobj = NbEditorUtilities.getDataObject(weakDoc.get());
        EditorCookie pane = dobj.getCookie(EditorCookie.class);
        JEditorPane[] panes = pane.getOpenedPanes();
        if (panes != null && panes.length > 0) {
            comp = panes[0];
            comp.addCaretListener(this);
        }
    }

    @Override
    public void caretUpdate(CaretEvent e) {
        bag.clear();
        scheduleUpdate();
    }
    private RequestProcessor.Task task = null;
    private final static int DELAY = 100;

    public void scheduleUpdate() {
        if (task == null) {
            task = RequestProcessor.getDefault().create(new Runnable() {

                public void run() {
                    String selection = comp.getSelectedText();
                    if (selection != null) {
                        Pattern p = Pattern.compile(selection);
                        Matcher m = p.matcher(comp.getText());
                        while (m.find() == true) {
                            int startOffset = m.start();
                            int endOffset = m.end();
                            bag.addHighlight(startOffset, endOffset, defaultColors);
                        }
                    }
                }
            }, true);
            task.setPriority(Thread.MIN_PRIORITY);
        }
        task.cancel();
        task.schedule(DELAY);
    }

    public OffsetsBag getHighlightsBag() {
        return bag;
    }
}

The only difference from Sergey's tutorial is the bit in bold above. Nothing magical, not using any NetBeans APIs, except that whenever a matching pattern is found, it is added to the "bag" (hello, lookup, I knew you well), and highlighted. The only difference between my implementation of "mark occurrences" and the typical implementation is that in my case one actually needs to select something, rather than simply putting the caret in the middle of some word. But that's fine, I think this does the job pretty well. Not only is this a nice introduction to this SPI, but now there is something I've been missing for a while—mark occurrences for HTML files. I've written to Vladimir Voskresensky to ask him what he did to make navigation between marked occurrences possible (as described here) because that would be very cool to add to this plugin. It works pretty well together with Ctrl-F (i.e., search), so that you can search for a word while using mark occurrences for another word (so, you have a double-edged search, as in Java files), as you can see here (orange is from Ctrl-F, while yellow is from mark occurrences):

I'm looking forward to exploring the Highlighting SPI in other contexts too, now that I have this example as an entry point.

Code Generator SPI Officially Available

Sometime ago this blog provided a preliminary example of a code generator. However, at the time, all that info wasn't official yet and you had to set an implementation dependency on one of the APIs. In the meantime, if you look at the NetBeans API Changes since Last Release document (which you should keep tabs on religiously, if you're a NetBeans Platform developer), you'll see this notification: "Code Generation SPI added":

The Code Generation SPI consists of two interfaces. The CodeGenerator implementations registered for various mime types serve for creating code snippets and inserting them into documents on the Insert Code editor action invocation. The CodeGeneratorContextProvider implementations registered for the mime types could provide the respective CodeGenerators with an additional context information.

Fine. What does all that mean? Firstly, interestingly, it means that the CodeGenerator class (and its supporting classes) are now officially supported and are exposed to the NetBeans API Javadoc. So, read the description to get an overview of it all.

Secondly, guess what? You can create code generators for any MIME type you want. So, you could add one/more to your HTML files... such as here:

So, the above appears when I press Alt-Insert in an HTML source file. What happens when I select the code generator item is up to me (i.e., the implementor of the API). In the case of Java source files, you can make use of the rather cool (though very cryptic) Retouche APIs. Let this document be your friend. I have found it pretty tough going, but gradually things become clear. Below, step by step, is my first implementation of a code generator. In the process, you'll see the Retouche APIs in action.

  1. Get a very recent post 6.1 development build and create a new NetBeans module.

  2. Set dependencies on Editor Library 2, Javac API Wrapper, Java Source, and Utilities API.

  3. Let's start very gently:

    import java.util.Collections;
    import java.util.List;
    import org.netbeans.spi.editor.codegen.CodeGenerator;
    import org.openide.util.Lookup;
    
    public class HelloGenerator implements CodeGenerator {
    
        public static class Factory implements CodeGenerator.Factory {
    
            @Override
            public List create(Lookup context) {
                return Collections.singletonList(new HelloGenerator());
            }
        }
    
        @Override
        public String getDisplayName() {
            return "Hello world!";
        }
    
        @Override
        public void invoke() {
        }
        
    }

  4. Register it in the layer.xml file like this, in other words, register the Factory that you see in the Java code above:

    <folder name="Editors">
        <folder name="text">
            <folder name="x-java">
                <folder name="CodeGenerators">
                    <file name="org-netbeans-modules-my-demo-HelloGenerator$Factory.instance"/>
                </folder>
            </folder>
        </folder>
    </folder>

  5. Now you're good to go. Just install the module and invoke the code generators as always, Alt-Insert, and then you'll see the new one added:

  6. OK, now we'll do something useful. We'll start by getting the JTextComponent from the Lookup. That JTextComponent is the Java editor and if we use getText, we can get the text of the Java editor. We can also get the Document object, which is all that we need in order to get the JavaSource object, via JavaSource javaSource = JavaSource.forDocument(doc);, which, in turn, is our entry point into the Retouche APIs. So, here we go:

    public class HelloGenerator implements CodeGenerator {
    
        private JTextComponent textComp;
    
        private HelloGenerator(JTextComponent textComp) {
            this.textComp = textComp;
        }
    
        public static class Factory implements CodeGenerator.Factory {
    
            public List<? extends CodeGenerator> create(Lookup context) {
                Item<JTextComponent> textCompItem = context.lookupItem(new Template(JTextComponent.class, null, null));
                JTextComponent textComp = textCompItem.getInstance();
                return Collections.singletonList(new HelloGenerator(textComp));
            }
        }
    
        @Override
        public String getDisplayName() {
            return "Hello world!";
        }
    
        @Override
        public void invoke() {
        }
        
    }

    I don't know whether the above is the optimal way of doing this, but at the end of the day we now have a JTextComponent. In the next part, I've simply taken code from the Java Developer Guide, referred to earlier, and implemented the invoke exactly as described there, so see that document in order to understand the code below:

  7. Define the invoke method like this, which as stated above is completely taken from the Java Developer Guide:

    @Override
    public void invoke() {
        try {
            Document doc = textComp.getDocument();
            JavaSource javaSource = JavaSource.forDocument(doc);
            CancellableTask task = new CancellableTask<WorkingCopy>() {
                @Override
                public void run(WorkingCopy workingCopy) throws IOException {
                    workingCopy.toPhase(Phase.RESOLVED);
                    CompilationUnitTree cut = workingCopy.getCompilationUnit();
                    TreeMaker make = workingCopy.getTreeMaker();
                    for (Tree typeDecl : cut.getTypeDecls()) {
                        if (Tree.Kind.CLASS == typeDecl.getKind()) {
                            ClassTree clazz = (ClassTree) typeDecl;
                            ModifiersTree methodModifiers = make.Modifiers(Collections.<Modifier>singleton(Modifier.PUBLIC), Collections.<AnnotationTree>emptyList());
                            VariableTree parameter = make.Variable(make.Modifiers(Collections.<Modifier>singleton(Modifier.FINAL), Collections.<AnnotationTree>emptyList()), "arg0", make.Identifier("Object"), null);
                            TypeElement element = workingCopy.getElements().getTypeElement("java.io.IOException");
                            ExpressionTree throwsClause = make.QualIdent(element);
                            MethodTree newMethod = make.Method(methodModifiers, "writeExternal", make.PrimitiveType(TypeKind.VOID), Collections.<TypeParameterTree>emptyList(), Collections.singletonList(parameter), Collections.<ExpressionTree>singletonList(throwsClause), "{ throw new UnsupportedOperationException(\\"Not supported yet.\\") }", null);
                            ClassTree modifiedClazz = make.addClassMember(clazz, newMethod);
                            workingCopy.rewrite(clazz, modifiedClazz);
                        }
                    }
                }
                @Override
                public void cancel() {}
            };
            ModificationResult result = javaSource.runModificationTask(task);
            result.commit();
        } catch (Exception ex) {
            Exceptions.printStackTrace(ex);
        }
    
    }

  8. Excellent. At this point, check that you have this impressive list of import statements:

    import com.sun.source.tree.AnnotationTree;
    import com.sun.source.tree.ClassTree;
    import com.sun.source.tree.CompilationUnitTree;
    import com.sun.source.tree.ExpressionTree;
    import com.sun.source.tree.MethodTree;
    import com.sun.source.tree.ModifiersTree;
    import com.sun.source.tree.Tree;
    import com.sun.source.tree.TypeParameterTree;
    import com.sun.source.tree.VariableTree;
    import java.io.IOException;
    import java.util.Collections;
    import java.util.List;
    import javax.lang.model.element.Modifier;
    import javax.lang.model.element.TypeElement;
    import javax.lang.model.type.TypeKind;
    import javax.swing.text.Document;
    import javax.swing.text.JTextComponent;
    import org.netbeans.api.java.source.CancellableTask;
    import org.netbeans.api.java.source.JavaSource;
    import org.netbeans.api.java.source.JavaSource.Phase;
    import org.netbeans.api.java.source.ModificationResult;
    import org.netbeans.api.java.source.TreeMaker;
    import org.netbeans.api.java.source.WorkingCopy;
    import org.netbeans.spi.editor.codegen.CodeGenerator;
    import org.openide.util.Exceptions;
    import org.openide.util.Lookup;
    import org.openide.util.Lookup.Item;
    import org.openide.util.Lookup.Template;

  9. Now install the module again. Then open a Java source file. Let's say it looks like this:

    package org.netbeans.modules.my.demo;
    
    public class NewClass {
    
    }

    Press Alt-Insert anywhere in the source file, choose "Hello world!", and now you will see this instead of the code above:

    package org.netbeans.modules.my.demo;
    
    import java.io.IOException;
    
    public class NewClass {
    
        public void writeExternal(final Object arg0) throws IOException {
            throw new UnsupportedOperationException("Not supported yet.");
        }
    
    }

Hurray. You have your first code generator. In an HTML source file, the above invoke method could be as simple as this, which would print <h2>hello</h2> at the caret:

@Override
public void invoke() {
    try {
        Caret caret = textComp.getCaret();
        int dot = caret.getDot();
        textComp.getDocument().insertString(dot, "<h2>hello</h2>", null);
    } catch (BadLocationException ex) {
        Exceptions.printStackTrace(ex);
    }

}

The very cool thing about these code generators is that you can keep typing, i.e., your fingers don't leave the keyboard in order to go to a menu item via the mouse, etc. You're typing as normal, then you press Alt-Insert, you select something, and then you go on typing. Being able to make your own contributions to the list of Java code generators (plus, being able to make them for other MIME types) is a really powerful enhancement to NetBeans IDE.

About

Geertjan Wielenga (@geertjanw) is a Principal Product Manager in the Oracle Developer Tools group living & working in Amsterdam. He is a Java technology enthusiast, evangelist, trainer, speaker, and writer. He blogs here daily.

The focus of this blog is mostly on NetBeans (a development tool primarily for Java programmers), with an occasional reference to NetBeans, and sometimes diverging to topics relating to NetBeans. And then there are days when NetBeans is mentioned, just for a change.

Search

Archives
« May 2008 »
SunMonTueWedThuFriSat
    
2
5
7
16
20
22
25
28
       
Today