How many files does your project have?

I have played again with performance of the NetBeans PHP IDE. For this purpose I have created "PHP code generator" that generates php files with classes, which create large class structure. Still it's somehow artificial, but for testing purposes the generated sources are good and valid.

I did some measurements on Core 2 Duo 2 GHZ machine. I generated php files for four projects as you can see from the result table. I have switch off the Task list window and measured the time of the creating project with the initial indexing of the php files in the project. The initial indexing parses all files in the project and store information about the files in a cache. Basically the features like code completion, navigation etc. don't work directly with the files, but with the information from the cache. The time of the initial indexing is in the second column in result table. It can be increased when you allow to NetBeans to use more memory than is the default value. In such case the garbage collector doesn't have to run so often. For example when I have increased the memory for NetBeans to  512MB then the indexing of the project with 15 000 files was about one minute shorter.

Number of PHP files
Initial project indexing
Opening indexed project
Responsiveness
 2 500  34 s
 2 s
 OK
 5 000  75 s
 3 s
 OK
 10 000  258 s
 4 s
 sluggish
 15 000
 395 s
 5 s
slow

The third column contains times that the IDE needs for opening a project. Because the cache is stored on HDD, the IDE check during opening project, whether the files were not changed externally to keep informations in the cache up-to-date.  As you can see the times are very small.

The last column contains my feeling, when I tried to add a method into a class and navigate through the sources. Working with project with 5 000 files was OK and the slowdown of responsiveness wasn't significant. When I work with the project with 10 000 files, then the IDE was slow sometimes during opening files and I had to wait a while (1 - 3 seconds) for code completion in the top level context. Working with this project wasn't so comfortable like with the project with 5 000 files. Similar situation was, when I try to work with the project that has 15 000 php files. The hyperlink functionality still works great, but using code completion in the top level context needs some patience.

I was evaluating why the IDE is slower with big projects and I  think that there is chance for improvements. If you work with such big project, I recommend to switch off the Task window, which can slow the parsing and opening projects.

I have one question for you. How many php files do you have in your projects? I would like to know, how big are the real projects.

Comments:

Where can I change the assigned memory for Netbeans? I just looked through the settings and couldn't find it.

The project I have open now (and which I am mostly working on) has 2589 files of which 932 are .php files

Posted by Dominik Jungowski on November 07, 2008 at 05:39 AM CET #

@Dominik

[NetBeans Installation]/etc/netbeans.conf

best regards,
josh.

Posted by Aljoscha Rittner on November 07, 2008 at 06:16 AM CET #

Our PHP projects have 10-500 our own files, without third-party libraries, e.g. AdoDb, TCPDF, Smarty templating engine or some MVC framework (CodeIgniter, ZendFramework). I have tried to open a big Joomla project in Netbeans 6.1 some time ago. It was really slow.

Petr, could you cooperate your work with other SUN's programmers working on nbdrupalsupport project? Ask them how big is a Drupal project.
https://nbdrupalsupport.dev.java.net/files/documents/8560/110298/file_110298.dat/Project%20Plan.pdf

Posted by jozef on November 07, 2008 at 06:29 AM CET #

Main project:
- 1197 files
- 714 php files (including libs)

The feeling is pretty good, hope it stays the same as the project is growing...

Posted by soso on November 07, 2008 at 07:43 AM CET #

I currently have three php projects in netbeans right now. All three are wordpress based. The first project, which is Wordpress MU with a lot of customization has just over 1000 files in ti. My own personal website (which is really three sites and a wordpress blog) weigh in at just over 2000 pages. The third project which is a straight up wordpress blog contains just under 800 files.

Posted by David Duggins on November 07, 2008 at 09:19 AM CET #

My project itself does not have more than 200-500 php files, but ezcComponents or ZF (or even both) are in my include path, and these have lots of files, in the regions of more than 10k-15k

Posted by beberlei on November 07, 2008 at 11:01 AM CET #

Around 700 php files in my project.

Posted by Daniel Brice on November 07, 2008 at 01:59 PM CET #

@Aljoscha: Thanks! I use the zip file though, but there is a etc folder which contains a netbeans.conf aswell ;-)

Posted by Dominik Jungowski on November 10, 2008 at 12:57 AM CET #

Hi Jozef,

I need to contact people behind the Drupal integration project. The support is based on NB 6.1, but the php support for NB 6.5 is written from scratch and doesn't have almost nothing common with the previous one.

Thanks,
Petr

Posted by Petr on November 10, 2008 at 07:04 AM CET #

Thanks to all for the answers.

Posted by Petr on November 10, 2008 at 07:07 AM CET #

@Petr: Any news about Drupal support, https://nbdrupalsupport.dev.java.net/ ?

Posted by Jozef on November 14, 2008 at 02:27 AM CET #

Hi Jozef,

I haven't got any answer yet. I will try to write to the team again.

Thanks,
Petr

Posted by Petr on November 14, 2008 at 02:40 AM CET #

I have 10761 files in my current PHP project. However, I'd estimate that only 50% of that consists of PHP files.

Does the presence of non-PHP files slow indexing down?

Posted by Adam Jorgensen on November 20, 2008 at 05:34 AM CET #

Hi Adam,

> Does the presence of non-PHP files slow indexing
> down?

It depends. For the PHP features we index the php files. For the JavaScript editor the JavaScript files are indexed. So if you will have 1 PHP file and 100 JS files, then the indexing spend the bulk of time in JS files.

In NB 6.5 only php and js files are indexed. In the next release we want to index the css files too, to offer some better integration to the html and php.

Regards,
Petr

Posted by Petr on November 20, 2008 at 06:09 AM CET #

Hi
I work in a very big php project, and I can't use no one ide with it.

either eclipse, netbeans, komodo, and others code inspecting ide don't handle well this project, so I finish using gedit/textmate or any other text editor.

if you improve netbeans to go fine with it, your will be the Man!

the numbers(2 months ago)
Files:
PHP: 7786
PLSQL: 1323
SQL: 133
JS: 562
JAVA: 5
SH: 11
AGT: 17
RPT: 143
Manuais( HTML ): 414
------------
Total: 10251 files

Lines:
PHP: 1466497
PLSQL: 208868
SQL: 21905
JS: 62676
JAVA: 576
SH: 2314
AGT: 6455
RPT: 289732
Manuais ( HTML ): 55932
--------------
Total: 1825223 lines

Posted by lucas stephanou on November 26, 2008 at 10:57 AM CET #

Hi Lucas,

7800 files is not so much. Sure there is still things which can be improved. Could you describe exactly the reason, why NetBeans doesn't fit you? For such project I would recommend to switch off the Task list window.

Thanks,
Petr

Posted by Petr on November 27, 2008 at 05:43 AM CET #

Hi Petr,

Hi think that the big reason is the number subdirectories.

After a morning of use, the memory usage get peaks of 150/160 mb, after that the responsiveness of netbeans and so on the machine down very much.

Actually, this project indexed start in netbeans quickly, but the problem is the open/close, open again/ rotine.

Posted by Lucas Stephanou on November 27, 2008 at 06:15 AM CET #

Hi Petr,

I'll test again today this project with netbeans and give you more feedback.

can I ask for a feature?
SnapOpen: hitting Ctrl+Shift+o open a input box where you while putting letters appears a list above with files matching the letters.
gedit and textmate has this feature, and will be nice to have in netbeans to.

Posted by Lucas Stephanou on November 27, 2008 at 06:21 AM CET #

Hi,

is there a way to know what is the result of the Indexing process in terms of classes and methods?

Actually I have a project with a large folder structure, but code completation of my classes doesnt work. I've registered the include paths in project's properties, restart the IDE, but classes are not recognised, however the idexing process takes a lot of time.

Posted by Oscar Cala on February 26, 2009 at 04:53 PM CET #

It was a very nice idea! Just wanna say thank you for the information you have shared. Just continue writing this kind of post. I will be your loyal reader. Thanks again.

Posted by links of london on October 29, 2009 at 08:16 PM CET #

It was a very nice idea! Just wanna say thank you for the information you have shared. Just continue writing this kind of post. I will be your loyal reader. Thanks again.

Posted by links of london on October 29, 2009 at 08:17 PM CET #

I am running a Joomla 1.5 project with 7800 files. A fresh Joomla 1.5 project seems to have about 4500 files and this number seems to be smaller (like 3500) in the upcoming Joomla 1.6

All of them are very slow at scanning and code completion. It used to be faster in earlier versions of Netbeans.

My suggestion to turn of the task pane, do you simply mean close it - or do you have to somehow disable it?

Posted by Soren Beck Jensen on September 15, 2010 at 03:36 AM CEST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

This blogs is written by NetBeans developers who contribute to the PHP support mainly.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today