Geertjan's Blog

  • November 27, 2007

Where's the AST View? And the Tokens View?

Geertjan Wielenga
Product Manager
If you, inspired by Heinrich Schliemann's gift for languages, are bringing that gift to NetBeans IDE, you will by now know about the AST view and Token view that the IDE provides to help you. Many have cited these views as just about the most useful thing provided by the Schliemann project. Put a different way, if you are using Schliemann without the AST view and Token view... your life must suck. :-) At least, it sucks a lot more than it should, because you're driving your car with your eyes closed. These views show you the impact of your grammar rules and tokens on a file of the type for which you're creating the editor. Here is an example of the AST view in action:

The power of the AST view in debugging your grammar rules is discussed in detail here.

However, these two views are potentially confusing for end users (i.e., for people who are not module developers or, even, those who are module developers but do not need to write a Schliemann editor). Hence, in the RC releases, certainly in RC2, these views are not easy to find. We don't want the unwary user to stumble across them and get confused or worried or something similar. So, now, if you want to see these views... you must have an NBS file. You'd probably begin creating your Schliemann editor via the Language Support wizard in the New File wizard. So, now you have an NBS file (i.e., "NetBeans Scripting", which defines your tokens, grammar, and feature mappings). Now you can right-click this file and choose "Tokens View" or "AST View". And then these views open and now... they show you the AST and tokens of your NBS file.

But you don't want that! You want to see the AST and tokens of your own file. No worries. Just leave the window open, now that you've opened it. Leave it open! Then... select your own file. Thanks to selection management in NetBeans IDE, the views show the AST and tokens of the currently selected file. So now you can go ahead and select your own file and the windows will be populated with the correct AST and tokens. Drag the window somewhere else if that will make working with it more comfortable for you. You can even undock it (Alt-Shift-D) and then it will be freed from the NetBeans IDE frame.

Yes, maybe slightly cumbersome (but a lot less than this blog entry makes it seem). However, since these views are intended for a microscopically tiny fraction of developers, when compared to the entire NetBeans user base, it is provided in such a way that only those specifically needing them can can get to them. An alternative might have been to only show the views in the Window menu if an NBS file is created, but that would be contrary to all design principles. And the problems that this solution would cause would make the solution totally meaningless. Hence, under the circumstances, it all makes complete sense. However, it is good to know what happened and... now you know too. I will add this blog entry to our fantastic NetBeans Developer FAQ for future reference.

Join the discussion

Comments ( 8 )
  • kawazu Tuesday, November 27, 2007

    Thanks for pointing that out - stumbling across the Schliemann tutorial I indeed was searching for that for quite a while (and initially gave up on it as I had other things to be done :) ).


  • Casper Bang Wednesday, November 28, 2007

    Hmm... why not just make the AST view generic instead of requiring all these hoops. It strikes me as allround benneficial to be able to inspect the AST no matter the language. It would assist in resolving tricky scopes, dangling elses etc. and refactoring options on these nodes rather would certainly be more user friendly than having to carefully select content in the editor?!

  • Geertjan Wednesday, November 28, 2007

    I agree with that, but my understanding is that we want to not overload the end user with too many menu items on nodes. We only want to provide what is necessary, not all things that are possible. But, personally, I would enjoy seeing the AST and tokens of any/every file type too. Just, for new users, they'd be overwhelmed with too much information and the list of menu items must be kep to a minimum.

  • Casper Bang Wednesday, November 28, 2007

    Ok Geertjan, that makes sence I guess except it sounds a little like a usability issue which should simply be solved.

    For instance, I don't understand why NetBeans doesn't use JCheckBoxMenuItem in the Window menu. Being able to see the toggled state of what's shown and what's not in this huge list would be right up this usability ally.

  • ChessMess Friday, November 30, 2007

    I had wanted to create support for a specific language (based on C++) using NB6 but not understanding AST or how to set that up and the prospect of doing everything from scratch and trying to find usable info difficult, we'll I put it on the back burner. I still want to do it but until better tutorials and organized resources for doing this become available it will be a larger barrier of entry for many.

  • Geertjan Friday, November 30, 2007

    AST is not something you will ever find a lot about in NetBeans documentation. It is a general concept, "Abstract Syntax Tree", completely unrelated to NetBeans. Rather than documentation being a barrier to many, it has been welcomed by many -- there's lots of documentation on Schliemann already. Largely, you can look at other examples of Schliemann, which are mostly self explanatory. There's also a project site with information(http://languages.netbeans.org/), this blog and other blogs, plus the mailing list.

  • Matthew Monday, January 26, 2009

    Why not make the AST and Token views into a plugin? Then those who need AST and Token views for their own files can easily have it choosing a normal Menu item instead of having to go through a kludged process using an nbs file first. The current setup is certainly more cumbersome than the previous one, particularly when you're trying to debug your own files and don't need to look at an nbs file. Thanks for showing how to see those views again!

  • Ben Saturday, May 12, 2012

    In newer NetBeans distribution (currently 7.2 dev) the AST tree is available via the "Navigator"-view. At the top this view there is a dropdown where you can select "Trees (NetBeans Development)".

Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.