Interview: Author of New D Programming Language Editor

Recently I've been in touch with Björn Lietz-Spendig. Björn is German and lives in the south of France. He's using the Schliemann Framework to create an editor for the D programming language and also for its scripting language MiniD (written by Jarrett Billingsley). I asked him a few questions about this development work he's doing. He had some problems getting code folding to work and in this blog entry I will show him (and others who might be interested) how to iteratively develop the grammar rules of a Schliemann-based editor.

Here's a pic of Björn at work:

Why are you creating an editor for the D programming language and its scripting language MiniD?

Well, the answer is a bit weird. I don't like C++, especially the preprocessor and template language. But, even as a database developer who is used to working with RAD tools, I have a need for low-level programming. And now we have a competitive stable D 1.x, including a module system, single inheritance, interfaces, garbage collector, Design by Contract, Class Invariants, and—last but not least—templates as a natural part of the language. D templates make C++ programmers cry... in other words, C++ on steroids.

But D is quite young and I simply want a professional, multilingual, platform-independent IDE for Java, D, and MiniD with all the bells and whistles. Furthermore, thanks to the DBus technology, and to TioLink, Java and D can work together in harmony. So, my IDE can use a handcrafted D-parser written in D to enable, for example, advanced semantic analyses.

You live in the south of France... that sounds great!

Well, I live quite isolated in the mountains. Fortunately, there exists a wonderful, and unfortunately pretty unknown, NetBeans plugin, called "Developer Collaboration", for live on-line communication and code sharing with others in a distributed environment. This tool enables me to setup a "virtual company", and maybe even more imortant, it enables the D programming community to share ideas, collaborate, and to develop together.

So, are you already using developer collaboration for a virtual company or do you want to do so in the future?

A year ago I brewed my own developer collaboration tool based on a Client/Server DB, using a proprietary asynchronous socket protocol, saving sources as a BLOB. At the time I thought it was a stroke of a genius. In fact, it was not. An eye opener for me was Allan Kay's Croquet Project and then I found that there is a NetBeans plugin that enables me to do something similar in Java! I am very enthusiastic about that plugin and I will definitely use it in the upcoming NetBeans-based D / MiniD IDE. I will definitely set up an XMPP Server a.s.a.p. to allow interested D programmers to collaborate. In terms of what I said about "virtual company", what I meant was that it is easier to set up the technical resources than to keep a crowd of "high end" programmers together. However, realtime collaboration is in my opinion THE next big thing.

How long have you been using NetBeans IDE?

Since 4.0 and, sorry, I was not satisfied at all. But now we have NetBeans IDE 6.0 coming up and I think this tool is really outstanding.

And why and how did you start using NetBeans IDE in the first place?

To teach myself Java.

And what are the strengths and weaknesses of NetBeans IDE, from your perspective?

Weak points? Seriously, more speed, less resource hunger. I am using NetBeans IDE 6.0 builds daily already, and I am pretty sure that we'll have acceptable performance by Beta 2.

And strengths? Well, the NetBeans Platform, the Schliemann project, Matisse, the developer collaboration plugin, the visual library, and the NetBeans community will, together, enable me to create an outstanding IDE in a reasonable timeframe. In the end, my MiniD NBS file will be an excellent "give it a go" starting point for people who want Lua and Squirrel support based on Schliemann.

What is your experience with NetBeans plugin development? Is the editor you are writing for D and MiniD the first plugin you're developing for NetBeans IDE? How are you finding the experience of doing this?

The editor I'm working on is indeed the first plugin I am developing for the NetBeans Platform. I find it very interesting and I have a a lot of fun (well, mostly). Thanks to the excellent documentation on netbeans.org, the community, YOUR blog, and the RCP book (I hope I'll get it within the next few days), it is a doable task. The NetBeans IDE itself... I think everybody who has used the NetBeans IDE for a while will never look back. So what's my experience so far? I feel comfortable.

Let's now have a look at where things are with your MiniD Editor.

Okay, so when I install the editor and open some MiniD script, I see this:

Nice!

Yes, but the code folding isn't working as I expected. I defined the code folds like this:

However, as you can see in the earlier screenshot, the multiline comments are in a code fold (in the screenshot above, just visible at the bottom of the screenshot), while the function declaration is not.

OK, let's debug your editor. To do so, we need to set some things up first. In this preparatory phase, take the steps that follow...

  1. Install your editor.
  2. In the IDE where you installed the editor, go to the Plugin Manager and find the 'Generic Language Framework Studio' plugin, as shown below:

    Once you've found the plugin, install it.

  3. Next, open a file of the MIME type to which you have assigned your Schliemann implementation. Then go to this menu item in the Window menu:

    Now you should see a new window, called "AST", which visualizes the abstract syntax tree defined by the grammar section of your Schliemann implementation:

  4. Now we need to analyze the information provided by the AST View. For example, above you can see that the selected node has ERROR as its label and that in the editor the word 'class' is highlighted in yellow. Whenever you click a node in the AST window, the text in the editor that corresponds to the grammar rule defined by the node is highlighted. This makes it easy to identify problems in the grammar rules. Then we need to make changes to our NBS file and, after that, install the module again. This way, we can iteratively develop our language support module.

Without the above AST View tool, you will have a very hard time developing your Schliemann implementation. You really need it, it makes life a lot simpler. Let's now fix some of the problems in your grammar rules, by using the AST View tool. Take the steps below...

  1. Expand some of the nodes in the AST View, and look at the highlighting on the right hand side, to see the effect of your grammar rules. For example, take a look here:

  2. What does the above situation tell us? The 'ExpressionStatement' grammar rule has kicked into action. Hmmm. I wonder if that is as intended, seeing as there doesn't seem to be a 'ClassDeclaration' or a 'TypeDeclaration', which I would expect at this point. So, let's look at the grammar rule section of the NBS file. The parser interprets the NBS file from top to bottom, so we should start investigating at the top of the grammar rule section of the NBS file. This is what it looks like, in this case:

    So, why does the parser imagine that the beginning of my class declaration in the MiniD script is actually an expression? Because... 'ExpressionStatement' comes before 'DeclarationStatement' above. And also because, after investigating the rest of the NBS file, the 'ExpressionStatement' seems to be confusing and not correctly defined. So, for now, let's simply delete 'ExpressionStatement' in the Statement declaration above. So, now a 'Statement' is the same as before, except that 'ExpressionStatement' is now no longer a part of 'Statement'. This means that... a class declaration cannot be confused with the 'ExpressionStatement' anymore. (We can put it back later but, for now, let's remove it, so that we can fix the problem with the 'ClassDeclaration'.)

  3. Install the editor again to see the results of your debugging. Now look again in the AST View:

    Yes! The class declaration is now correctly handled by the 'DeclarationStatement' grammar rule! That's cool. When I select the node, the entire class is highlighted, which shows me that the 'ClassDeclaration' declaration is working as I would expect. The AST View shows there are other errors (plus, a code fold seems to have suddenly appeared), but at least we've fixed this one. We need to fix errors from top to bottom, otherwise we'll be creating new problems for ourselves, if we start fixing problems randomly all over the place.

  4. Next, let's investigate why the code fold isn't working for 'FunctionDeclarations'. As before, expand a few of the nodes in the AST View and then notice the highlighting in the editor. Continue doing so until you get to a node that covers what you think should be a function, such as below:

    Okay, so clearly we're dealing with a function and clearly we have a node in the AST that provides the grammar rule for a function. However... look at the labels of the nodes. Is there one labeled 'FunctionDeclaration'? Nope. So, no big surprise, then, that the code fold for the 'FunctionDeclaration' doesn't work. Instead, there is a node labeled 'SimpleFunctionDeclaration'. So, we have at least two ways to fix this—either fix the definition of 'FunctionDeclaration' so that the correct piece of code is captured... or... change the definition of the code fold so that the code fold applies to ''SimpleFunctionDeclaration', rather than 'FunctionDeclaration'. The latter is easier, so, for now, let's just do that. Now our code fold section is defined as follows:

    # code folding
    FOLD:md_comment: {
        expand_type_action_name:"LBL_ExpandComments";
        collapse_type_action_name:"LBL_CollapseComments";
        fold_display_name:"MiniD multi-line comment";
    }
    FOLD:SimpleFunctionDeclaration: {
        expand_type_action_name:"LBL_ExpandFunction";
        collapse_type_action_name:"LBL_CollapseFunction";
        fold_display_name:"MiniD function";
    }

    The only change is that we no longer have a code fold for 'FunctionDeclaration'. Instead, we have one for 'SimpleFunctionDeclaration'.

  5. Install the editor again and notice that we now have code folds for our function declarations:

Hurray! From this example, you can see that the basis of debugging the grammar rules in your Schliemann implementation is very simple and effective with the AST View tool in NetBeans IDE 6.0. So now, just continue in this way, tweaking, fiddling, fixing, reinstalling, checking, and so on, until you have the editor of your dreams!

Finally, how do you see your D Editor helping people?

The D (not MiniD) implementation will open new horizons to NetBean IDE developers. Can you imagine being able to use a C++ class directly from Java... and vice versa? With the D language plugin (based on TioLink) you can call D classes and D can call Java classes. Imagine the power of being able to create platform-independent eye-candy applications, based on the NetBeans Platform, and having seamless integration with D, just in case that you need mission critical high speed performance and reliability? Furthermore, thanks to Schliemann and the developer collaboration plugin, the IDE and the D language plugin will be part of a collaborative process.

Good luck with the D Editor, Björn! Looks like you're off to a great start.

In other news. It's official—I am a rock star! The JavaOne Content Team sent me an e-mail yesterday that starts off like this: "Congratulations! Your session titled 'JFugue: Making Music with Java MIDI and Illustrating API Usability' was a Top Session at the 2007 JavaOne Conference. The people in attendance at your session surveyed you as an exceptional speaker and we thank you for contributing so much to the conference program. With this honor, you are now considered one of the JavaOne Conference Rock Stars and we would like to send you a commemorative award as recognition." Hurray for me and Dave Koelle, my co-speaker! Hope this means I'll get extra points when the selection process begins for the next JavaOne!

Comments:

Thats awesome Geertjan! Now you are a rock star.. Congratulations.

Posted by Vadiraj on September 05, 2007 at 11:50 PM PDT #

Here is the learning material for D Programming Language
http://docs.sun.com/app/docs/doc/817-6223/chp-intro?a=view

Posted by Solaris10 on September 06, 2007 at 12:41 AM PDT #

Regarding the Learning tutorial posted by Vadiraj...
The D programming language is a successor of C++, written by Walter Bright. Your D trace link is, in this context, completely irrelevant.

Kind regards, Bjoern

Posted by Bjoern Lietz-Spendig on September 06, 2007 at 01:00 AM PDT #

Geertjan, what can I say..... Simply, Thank you. I think a lot of Schliemann noobs, like me, will participate, and make more progress thanks to your awesome Block.
Bjoern

Posted by Bjoern Lietz-Spendig on September 06, 2007 at 01:07 AM PDT #

Hi Bjoern, thanks for liking the interview, thanks to you for the participation in it! I hope my debugging tips help. By the way, Vadiraj didn't post that learning trail... someone called 'Solaris10' posted it. I hope to hear more from you as you continue developing your editor!

Posted by Geertjan on September 06, 2007 at 01:25 AM PDT #

Björn, keep your good work, i love netbeans and D, so i'm very happy to see your progress. I've read your post on digitalmars' forum too. I would like to know what Descent guys thinks about it :)

Posted by ASpanishGuy on September 06, 2007 at 05:26 AM PDT #

> I would like to know what Descent guys thinks about it :)

Always nice to have some healthy competition!

Posted by Robert Fraser on September 06, 2007 at 06:15 AM PDT #

I can't find "Generic Languages Framework Studio" plugin in NB6 M10 (or M9).

Should I be able to?

I have written a schliemann file for AutoLISP, without the AST view, but it sure would have been easier with it.

What do I need to do to be able to get the AST view on M10?

Posted by Bruce Chapman on September 10, 2007 at 07:45 AM PDT #

Hi Bruce, it might have been removed for a while, remember that all this stuff is under development, so things may come and go, until the 6.0 release. Anyway, if you get a recent build (which you should anyway with Schliemann, because things get fixed all the time) you will find it as described here. On another note, it would be great if you could drop me a note at geertjan DOT wielenga AT sun DOT com, to see if you want to do an interview about your experiences or share your thoughts for an article I am writing on early implementers of Schliemann, such as yourself. Your views would of course be extremely valuable!

Posted by Geertjan on September 10, 2007 at 07:54 AM PDT #

Awesome! I really look forward to this .

Posted by charles on September 11, 2007 at 08:30 AM PDT #

The D programming language is a successor of C++, written by Walter Bright. Your D trace link is, in this context, completely irrelevant.

Posted by DEBT on October 08, 2007 at 03:57 PM PDT #

Not my link, DEBT, but a comment that someone else left here/

Posted by Geertjan on October 08, 2007 at 04:01 PM PDT #

PS: But you don't need to install "'Generic Language Framework Studio' plugin" anymore. The windows described here are now available under Window|Others, I believe. Drop me an e-mail if you continue having problems.

Posted by Geertjan on October 25, 2008 at 02:08 PM PDT #

Isn't there a D plugin for Netbeans 6.7 so I can download it and use it.

Posted by B.klass on June 13, 2009 at 06:12 PM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Geertjan Wielenga (@geertjanw) is a Principal Product Manager in the Oracle Developer Tools group living & working in Amsterdam. He is a Java technology enthusiast, evangelist, trainer, speaker, and writer. He blogs here daily.

The focus of this blog is mostly on NetBeans (a development tool primarily for Java programmers), with an occasional reference to NetBeans, and sometimes diverging to topics relating to NetBeans. And then there are days when NetBeans is mentioned, just for a change.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
12
13
14
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today