New Test Runner UI

Lately I've been working on a new UI for displaying test results in the NetBeans Ruby IDE. This is now available in the latest daily builds, so please give it a try and let me know whatever issues you encounter. And of course, any ideas for improving it further are welcome too. I've already filed a couple issues myself that I haven't yet had time to address. Below are some screenshots, should look pretty familiar for those of who you have run JUnit tests in the IDE - the UI is pretty much the same with some Ruby specific modifications.

Here's a screenshot of the new UI, Test::Unit test run:

Another screenshot using a different layout. You're free to drag the results window anywhere in the IDE, the iconless button on the left side of the runner switches the layout from vertical to horizontal and vice versa (note that this will be done automatically once I get to fix this issue).

And finally a screenshot of an RSpec test run:

For those of you who are not familiar the JUnit results window, it might be worth noting a couple things. The results window can be found in Window -> Output -> Ruby Test Results. You can display only failures by using the filter button on the left side of the statistics panel. Double clicking the failed method nodes jump to the failure line in the test file, and you can of course double click the stack trace lines as well for the same effect.

Currently the new UI is invoked only if you specifically select 'Test' or 'RSpec Test' from the project context menu or run a single test file, meaning that running the test or spec rake tasks using the Rake runner in the IDE don't display the results using this UI. This will change of course, just a matter of time - see the link to the issue list in the beginning of this entry.

So, please go grab a fresh build and let me know whatever issues you encounter or how would you enhance it further!

Comments:

looks really cool ! thanks for adding this

Posted by guest on May 26, 2008 at 04:07 PM CEST #

The RSpec screenshot looks very promising!
I cannot see the 'RSpec Test' in my project's context menu, though - is there anything I have to enable? I'm running the 200805291203 build right now, and the only way to see test output is through the rake task.

Feature suggestion:
Really cool would be to integrate rcov-like coverage information.

Posted by Clemens on May 31, 2008 at 11:35 AM CEST #

Hi Clemens,
in case you had a Rails project, you were likely facing a bug that I just fixed (will take a couple of hours before it is reflected in the downloadable build). Thanks a lot for the catching this! As for the rcov integration, there is an open issue for that, http://www.netbeans.org/issues/show_bug.cgi?id=119707, if there is a lot of demand for that we'll try to provide something for 6.5.

Posted by Erno Mononen on June 02, 2008 at 08:09 AM CEST #

Very exciting. But when I run a rails 2.1 test from nb using apple-f6 I get the following consistently:

System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require': no such file to load -- test_helper (LoadError)
from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `require'
from /Users/stu/projects/origin/iter1-stu/test/unit/lead_definition_test.rb:1
from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `gem_original_require'
from /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/rubygems/custom_require.rb:27:in `require'
from /Users/stu/Downloads/netbeans/ruby2/nb_test_mediator.rb:97:in `add_to_suites'

I can run the tests fine from rake.

Even a simple test like this fails:

require 'test_helper'

class LeadDefinitionTest < ActiveSupport::TestCase
should_have_many :questions

end

Any ideas?

Thanks from the great work. Netbeans is a very slick ide for ruby!

Posted by stu on June 05, 2008 at 10:58 AM CEST #

Hi stu,
the problem you encountered should be fixed in the latest build (just put in the fix a few hours ago), let me know if you still face this.

Posted by Erno Mononen on June 06, 2008 at 09:03 AM CEST #

This looks great! Does "Currently the new UI is invoked only if you specifically select 'Test' or 'RSpec Test' from the project context menu or run a single test file..." mean that the UI is not activated from an autotest run?

Posted by Frank Luke on June 06, 2008 at 03:10 PM CEST #

I can invoke rspec now from the context menu, thanks, but cannot actually run tests as my rspec and rspec for rails plugins are not compatible. Do you have an ETA for a fix? Also, can you run all rspec tests by default if no single file is selected? currently no tests are being run.
Looking forward to your enhancements and fixes!

Posted by Clemens on June 07, 2008 at 11:02 AM CEST #

Hi Frank,
Currently the UI is not activated for autotest runs, but I do want to make it work with autotest as well. I filed http://www.netbeans.org/issues/show_bug.cgi?id=136746 to keep track on that.

Posted by Erno Mononen on June 07, 2008 at 01:50 PM CEST #

Hi Clemens,
so you are hitting http://ruby.netbeans.org/issues/show_bug.cgi?id=136483? I hope to address that in time for 6.5 M1 (should be out in about two weeks). In the meantime you can try the workarounds described in the issue, if possible in your case.

Doesn't invoking RSpec Test from the project context menu run all rspec tests in your project for you? That is a bug definitely, could you please file an issue for this (or just post here and I'll file it for you) with the following details:
- Rails, Rspec and Ruby/Jruby versions
- system info from the Help -> About dialog
- does it happen with all projects or just with some certain one (in which case I'll need to ask more details on that project)

Of course, if you can provide steps to reproduce from scratch that would be perfect..:) Thanks in advance!

Posted by Erno Mononen on June 07, 2008 at 02:13 PM CEST #

This looks great, but is it possible to use it with keyboard only? (i hate to switch hands between keyboard and mouse when coding)

Posted by Adam Ciganek on June 17, 2008 at 04:27 AM CEST #

Hi Adam,
you can navigate in the UI using tab and arrow keys, so you don't necessarily need a mouse to use it. You can also run tests using predefined shortcuts and assign a shortcut for activating the result window (in Options -> Keymap -> Window -> Ruby Test Results), currently there is no predefined shortcut for that. Let me know if that's not enough for you.

Posted by guest on June 18, 2008 at 02:45 AM CEST #

Hi 192.9.112.196 :)

Thanks for the tip about Ruby Test Results keyboard shortcut. It's fine, but still not comfortable enought for me.

What would be ideal for me, is this: when i run the test (using Ctrl+F6 for example) and there is a failure, then there should be some way (keyboard shortcut) to jump right into the test method that caused the failure. Right now, i have to hit tab several times and then find the right file/line in the backtrace and then hit enter.

So, what i suggest, is this:

1. When i run the test and there is some failure, move the keyboard focus right to the first test result. Look at this image to see what i mean: http://picasaweb.google.co.uk/adam.ciganek/Whatever/photo#5213169172523193362
Also do this when i hit the shortcut for "Ruby Test Results" window.

2. Also when i jump back to the editor and then go back to test results, the focus should stay on the same item it was last time.

3. And when there is no failure or error, the focus should remain in the editor.

Do you know what i mean? Does it make sense? What do you think about it?

By the way, should i put this stuff also into netbeans bugzilla (as a request for enhancement)?

Posted by Adam Ciganek on June 18, 2008 at 05:48 AM CEST #

Thanks, great ideas! Please do file them into issuezilla too, ruby/testing would be the right component. Automatically moving the focus to the first test failure might not be what everybody wants (some people tend to be irritated when the focus is "magically" lost), so there would need to be an option for that - either that or maybe focusing the failure when you hit the shortcut for the test results window would be enough? Also, I was thinking that maybe a shortcut for jumping to the next/previous failure would be good to have too - would you find that useful?

Posted by Erno Mononen on June 18, 2008 at 06:51 AM CEST #

Ok, let me see:

>> Automatically moving the focus to the first test failure might not be what everybody wants (some people tend to be irritated when the focus is "magically" lost)

I agree with that, but that's actualy (almost) the current behaviour. The only difference is, that it does focus the Ruby Test Result window frame (or something like that), not the first failure. So i need to hit tab several times to get there.

But i probably agree that it may not be that great idea. But focusing first failure when the test results shortcut is hit would be great anyway.

>> Also, I was thinking that maybe a shortcut for jumping to the next/previous failure would be good to have too - would you find that useful?

That may be great too. Also when you decide to do this, make it so activating "jump to next failure" while the test result window is \*not\* focused, will focus it and jump to the first failure.

So, to summarize:

1. Running the test should not change the focus in any case.
2. Invoking "Ruby Test Results" action should move the focus to the first failure.
3. Add "Jump to next/previous failure" action
4. When returning to test results window again (if the test wasn't run again), the focus should stay on the item where it was before

Do you agree? If yes, i will put this into issuezilla.

Posted by Adam Ciganek on June 18, 2008 at 08:28 AM CEST #

Hi Adam,
Sorry for the late reply, I never got a notification on your latest comment. I agree with this, please do file it in IZ. Thanks!

Posted by Erno Mononen on June 27, 2008 at 05:48 AM CEST #

Thanks for the improvements!

When I run Ruby on Rails tests in 6.5M1 I get lots of output like this, instead of seeing the actual console output (the dots and the E's and F's):

%SUITE_STARTING% AccountTest
%SUITE_STARTED% 0 tests, 0 assertions, 0 failures, 0 errors
%TEST_STARTED% test_truth(AccountTest)
%TEST_RESULT_CHANGED% 0 tests, 1 assertions, 0 failures, 0 errors
%TEST_RESULT_CHANGED% 1 tests, 1 assertions, 0 failures, 0 errors
%TEST_FINISHED% time=1.547 test_truth(AccountTest)
%SUITE_FINISHED% 1.547
%SUITE_SUCCESS% true
%SUITE_FAILURES% 0
%SUITE_ERRORS% 0

This output is too verbose for my taste. I prefer the ...E...F... -style output.

Is this just because it's not a final release? From the screen shots posted above, I don't see anything that looks like this.

Posted by Brian Morearty on July 21, 2008 at 11:21 AM CEST #

Hi Brian,
It's a bug, those lines should not be printed to the output. It might be fixed already in dev builds, it be great if you could try the latest build from http://deadlock.netbeans.org/hudson/job/trunk/lastSuccessfulBuild/artifact/nbbuild/dist/zip/ and let me know if you still see that (please specify also your OS and Rails/Ruby versions then). Or in case you don't want a dev build, please try with M2 when it will be out, should be in a week or so I think. Thanks!

Posted by Erno Mononen on July 21, 2008 at 04:30 PM CEST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

emononen's blog

Search

Top Tags
Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today