Wednesday Nov 26, 2008

Web Stack Updates

It has been an impossibly busy 3 months, something had to drop so blogging was it. Time to pick it up again!

Since it's been a while, I'll revisit what has been going on with OpenSolaris Web Stack in the last few months. This blog may have been quiet but things have been happening!

  • Lighttpd is now included in Web Stack! Amanda has some entries on lighttpd integration and scalability
  • Ruby has been updated to 1.8.6-p287
  • Ruby also integrates some new performance improvements on SPARC
  • Tomcat has been updated to 5.5.27
  • Apache httpd is now at 2.2.9
  • As part of the Apache 2.2.9 work, APR and APR-util have been split into separate packages. These libraries are consumed by some other apps, not just Apache so now OpenSolaris users don't end up installing all of Apache if they only needed APR.
  • Also on the Apache front, while not integrated yet, check out Nick's recent work on Apache and privileges
  • MySQL updated to 5.0.67. Previously we were hoping to go straight to 5.1 instead of doing another 5.0.\* update but 5.1 release moved out a bit, to the point where it wasn't possible to get it into 2008.11 so we ended up going to 5.0.67 to benefit from accumulated bug fixes for 2008.11. The plan is to have 5.1 in Web Stack (including dtrace support!) very soon after it gets released.
  • PHP memcached extension has been upgraded to 2.2.4
  • PHP has support for imap/imap-ssl now
  • PHP xdebug extension updated to 2.0.3
  • PHP suhosin upgraded to 0.9.27
  • PHP tcpwrap is now 1.1.3
  • PHP also now has support for PEAR/PECL
  • And finally on PHP, the packaging has been refactored a bit to be more sensible. Instead of being named after the full version (SUNWphp524) they are now just SUNWphp52. This allows bugfix updates without constantly renaming packages. Some unnecessarly splits were also combined, although that improvement is more visible to Nevada users than OpenSolaris users (because IPS combines some packages during post-production anyway).

So there's your whirlwind review of recent Web Stack changes...

You can get access to all these and the rest of Web Stack by updating your OpenSolaris installation to the latest, a good way to spend a long Thanksgiving weekend ;-0

(And OpenSolaris 2008.11 should be out real soon with many other improvements beyond the world of Web Stack, but if you're mainly interested in Web Stack components all of the above updates are already there in build 101 so you don't have to wait.)


Wednesday Aug 06, 2008

Web Stack at Immersion Week

Thanks to those of you who attended the Web Stack session at Immersion Week (Sun field/sales meeting) this morning! I'm glad to see the drupal-demo package was well received.

As I mentioned, for any comments/feedback please contact us at webstack-discuss@opensolaris.org. Thanks!


Wednesday Jul 30, 2008

IPS updates

Good news on the IPS front, the bug which prevented pkg from installing from additional repositories such as Web Stack has been fixed and with the recent availability of build 94 bits for OpenSolaris the IPS fix is now available.

So as soon as you upgrade your OpenSolaris image to b94 or later, it will no longer be necessary to use the workaround I documented earlier in order to install any of our Web Stack packages.


Drupal demo package

I decided to tweak the delivery of the evaluation/demo/quickstart script included in the Drupal package a bit. Since the script (/usr/bin/drupal_evaluation_init) is for various reasons not suitable for setting up a real production site I wasn't very happy that it was in the 'drupal' package. After all, it shouldn't be installed on a production site. So I created a new package called 'drupal-demo' which now delivers the quickstart script (and a manpage for it).

There's another benefit to splitting it out like this; I made 'drupal-demo' have a dependency on the MySQL package and removed that dependency from the core 'drupal' package. This makes more sense: for a production site you can install only 'drupal' package and since you're more likely to point it at a MySQL instance on a different server there is no need to forcefully install MySQL locally. If you want to play around, test or demonstrate Drupal you can install 'drupal-demo' which gets you not only the quickstart script but also MySQL on localhost which is perfect for test/demo purposes.

(Unfortunately this benefit is only theoretical at the moment. That's because all of MySQL, including the client libraries, is in one big package right now. This means that for PHP to have access to the MySQL libraries it needs (which is required for Drupal), PHP ends up bringing in all of MySQL locally anyway. However, we consider this a bug and there should be some refactoring of the MySQL package coming up.)

I'll be doing the same for other packages which include this kind of demo/quickstart setup (right now, phpmyadmin) both to separate the non-production files and to manage the dependencies better.

If you have any preference on the naming of these packages let me know. I'm not entirely happy with the \*-demo suffix but it's the most concise one I could think of at the moment.


Thursday Jul 24, 2008

Web Stack source packages

In a previous article I mentioned how easy it was to build one component in Debian by just grabbing the source package and its dependencies.

Earlier this week I was cleaning up the script which builds and produces the Web Stack repository packages and it occurred to me it'd be really easy to enhance it to also publish source packages, so why not. If you've looked at the repository in the last couple days you may have already noticed several \*-source packages.

So let's say you'd like to build nginx on OpenSolaris yourself instead of using the nginx package from the Web Stack repository (maybe you want to change a build flag or something). So now you have one more option on how to do it:

% pfexec pkg install pkg://webstack/nginx-source
% pfexec zsh
# cd /opt/webstack-source/nginx
# make install

If you have your own local IPS repo running you can also easily publish the freshly rebuilt package into it by simply:

# export PKG_REPO=http://myips.example.com:10000
# make publish

(where, of course, the value you set for PKG_REPO is actually the URL to your own local IPS repository)

Then you (or others in your group) could simply try out your rebuild package by installing it from your $PKG_REPO.

Notice that the \*-source package sets all its dependencies such that you can simply type 'make' right away and all the packages (such as compilers and needed libraries) are already in place. You don't have to go through any READMEs to figure out what other packages you need before compiling. Pretty cool! ;-)

Now, admittedly, in some ways this is just a gimmick due to IPS limitations today. For one thing, I had to pick an install location (so I chose /opt/webstack-source/$PKGNAME) because unlike apt, pkg doesn't know abot source packages so there's no convenient way to drop the source into "." which is probably where you really want it. So it also means you then had to become root to build under /opt/webstack-source/ (I might change this to use a dedicated user but still it's not optimal). (Installing the source package into a user image might seem like a workaround but it won't do what you want since then all the dependencies get installed in the wrong place.)

You could of course simply check out the source directly with mercurial and get the same content with the benefit of placing the source tree wherever you want and owned by your uid. The drawback is that in that case you will need to figure out and manuall install all dependencies needed to actually build the source.

So, there's benefits and drawbacks either way. Nonetheless, an interesting experiment!


Wednesday Jul 23, 2008

nginx package updates

Tonight was a moment to play with the nginx package which has been in the Web Stack repository for a while now. I added a working default configuration, smf support and a stub man page. While I was there I also updated the version to 0.6.32.

So you know how it goes.. give it a try and send suggestions on how to improve it for OpenSolaris to webstack-discuss@opensolaris.org!

% pfexec pkg set-authority -O http://pkg.opensolaris.org/webstack webstack
% pfexec pkg refresh
% pfexec pkg install pkg://webstack/nginx
% man nginx
% pfexec svccfg import /var/svc/manifest/network/http-nginx.xml

(the above is just to work around a bug in IPS, it does 
not yet import the manifest automatically)

% svcadm enable nginx


Tuesday Jul 22, 2008

Updated phpMyAdmin package

As most of you know the Web Stack repository has had a phpMyAdmin package for a long while now. It was in fact the first package to show up on the project repository. It was very much an experiment and it delivered its files in the simplest way possible, by simply dumping everything under the Apache htdocs directory.

With the release of the drupal OpenSolaris package recently, I decided to go back and rearrange the phpMyAdmin package to follow the same model and I just pushed it out to the repository. Changes include:

1. File layout similar to drupal (and also similar to the upcoming phpPgAdmin package): static files live under /usr/phpmyadmin/ and the editable configuration is in /etc/phpmyadmin/

2. The package delivers sample Apache configuration in /etc/apache2/2.2/samples-conf.d/phpmyadmin.conf

3. A stub man page is included (man phpmyadmin) and a quick-start evaluation script is also included (/usr/bin/phpmyadmin_evaluation_init).

Also, not specific to phpMyAdmin, I improved the package publishing so now the version numbers are preserved, so you'll find the package list more informative than before. For example by looking at "pkg:/phpmyadmin@2.11.7,5.11-1:20080722T221304Z" you can tell I updated the version of phpMyAdmin to 2.11.7.

Finally, while bug 2253 has recently been fixed, unless you're running a handbuilt version of IPS you'll still need to use the workaround in order to install Web Stack packages:

% pfexec pkg set-authority -O http://pkg.opensolaris.org/webstack webstack
% pfexec pkg refresh
% pfexec pkg install pkg://webstack/phpmyadmin
% man phpmyadmin

As always, please try it out and send your feedback to webstack-discuss@opensolaris.org. Better yet, if you're inspired to improve it, let me know.


Friday Jul 11, 2008

Web Stack Repository Updates

Some good news at the end of this week, the Web Stack package repository is back online tonight!

The IPS regression which prevents it from accessing packages in non-primary repositories is still not fixed though, so you will not be able to install normally from the Web Stack repository. Fortunately the workaround is working now so you can do 'pkg install pkg://webstack/packagename' instead of 'pkg install packagename' (assuming 'webstack' is the name you gave to the authority when adding it).

To summarize:

% pfexec pkg set-authority -O http://pkg.opensolaris.org/webstack webstack
% pfexec pkg refresh

I've posted some new content into it including packages for nginx and drupal so check it out...

Both of these new packages are in the early stages of development so they have not been tested much yet. As you run into any issues let us known at webstack-discuss@opensolaris.org!

Drupal

I included a small stub man page in the Drupal package to get you started so just type 'man drupal' after installation for some info. I also threw in a simple script to set up your system for a quick start with Drupal. It is definitely not for production use but if you'd simply like to quickly get a demo going it'll save you a few minutes (with this, I can reduce some of my "5 minutes setup" presentation demos down to "1 minute setup" demos ;-)

% pfexec pkg install pkg://webstack/drupal
% man drupal
% /usr/bin/drupal_evaluation_init

nginx

The nginx package is less polished at the moment and it is missing both a man page and an smf manifest. But you can just manually run /usr/nginx/sbin/nginx to get going...


Monday Jul 07, 2008

Web Stack Repository Temporarily Not Available

Just wanted to mention, for those wondering, that the Web Stack project repository (at http://pkg.opensolaris.org/webstack) is indeed temporarily out of service.

The repository itself is not accessible due to a setup problem on the pkg.opensolaris.org server which should hopefully be resolved in the near future.

The bigger problem is that IPS has developed a bug lately which prevents additional repositories from being accessible. You can follow the discussion on pkg-discuss@opensolaris.org and/or check bug 2253 (there are also bugs 2325 and 2347 which seem closely related). Hopefully these can all get fixed soon so we'll be back on the air! Unfortunately all these issues are outside of my control so I don't have an ETA for repairs but I'll be sure to update my blog and webstack-discuss once things get going again.


Sun Tech Days Demo Transcripts

Earlier I mentioned I'd post the transcripts from the three demos I gave at both the Mexico City and Manila Sun Tech Days. Each demo was just under 5 minutes. Here's the transcript of the commands I ran, taken from my recording (I added only brief commentary in the text, enclosed in square brackets):

Installing Apache and PHP on OpenSolaris

[check that apache2 is not yet installed, then install it]

% cd /etc/apache2
% pfexec pkg install SUNWapch22
% cd /etc/apache2
% cd 2.2
% ls

[bug in pkg, it does not import the smf manifest, so do it manually
and then enable apache]

% ls /var/svc/manifest/network/http-apache22.xml
% pfexec svccfg import /var/svc/manifest/network/http-apache22.xml
% svcadm enable apache22

[run firefox: load http://localhost to show Apache is working]

[check that php5 is not yet installed, then install it]

% cd /etc/php5
% pfexec pkg install SUNWphp524 SUNWphp524man
% cd /etc/php5
% ls
% cd 5.2.4
% ls -la

[let's look at apache docs dir and create a simple php script in there]

% cd /var/apache2
% ls
% cd 2.2
% ls -la
% cd htdocs
% ls -la
% su
# cat > test.php
<?php
phpinfo();
?>
\^D

[go to firefox: load http://localhost/test.php
doesn't work, shows source, because apache2-php5 connector not yet installed;
so install the connector package and show that it also installs its config]

# pkg install SUNWapch22m-php5
# cd /etc/apache2/2.2/conf.d
# ls
# more php5.conf
# svcadm restart apache22

[go to firefox, reload page, now it worked!]

Web Stack Project Repository

[show currently known (default) authority; add webstack repository and show it]

# pkg authority
# pkg set-authority -O http://pkg.opensolaris.org/webstack webstack
# pkg authority

[refresh package list and check what's available from webstack]

# pkg refresh
# pkg list -a | grep webstack

[let's install and take a quick look at two packages (varnish and lighttpd)

# pkg install SUNWvarnish
# cd /usr/varnish
# more README

# pkg install SUNWlighttpd14r SUNWlighttpd14u
# cd /usr/lighttpd
# cd 1.4
# ls -la
# ls -la sbin

Install and Configure Wordpress

Thanks to Sriram

[check that neither apache nor mysql is yet installed, then install them]

# cd /usr/apache2
# cd /usr/mysql
# pkg insall amp-dev

[bug in pkg, it does not import the smf manifests, so do both manually
and then enable them]

# svccfg import /var/svc/manifest/application/database/mysql.xml
# svccfg import /var/svc/manifest/network/http-apache22.xml
# svcadm enable apache22
# svcadm enable database/mysql:version_50

[create database for wordpress in mysql]

# /usr/mysql/bin/mysql -u root
mysql> create database wordpress;
mysql> grant all privileges on wordpress.\* to wp@localhost identified by 'wp';
mysql> flush privileges;
mysql> quit;

[retrieve the latest wordpress code, install and configure it]

# cd /tmp
# wget http://wordpress.org/latest.zip
# cd /var/apache2/2.2/htdocs/
# unzip /tmp/latest.zip
# cd wordpress
# cp wp-config-sample.php wp-config.php
# vi wp-config.php
   DB_NAME = wordpress
   DB_USER= wp
   DB_PASSWORD = wp

[go run firefox; http://localhost/wordpress]


Wednesday Jul 02, 2008

Adventures in Building a Web Stack Component

[Read More]

Wednesday Jun 18, 2008

Sun Tech Days Manila

An energetic Sun Tech Days event in Manila is now over. Yesterday I presented the OpenSolaris Web Stack project to a full room, probably close to 400 people, it was really cool to see so much interest! Then today I gave Jim Walker's OpenSolaris Testing presentation (since Jim couldn't attend). The network on stage cooperated and I was able to demo both the Self Service Testing and Test Farm services.

Whether at the talks or later at the booth, I hope most of your questions got answered. As any other issues come up I encourage you to contact the various opensolaris.org discussion lists:

webstack-discuss@opensolaris.org for any Web Stack questions or discussions.

testing-discuss@opensolaris.org for testing community interaction.

indiana-discuss@opensolaris.org for general OpenSolaris (code name: indiana) installation and usability questions.

Thanks to everyone who took time to attend!


Thursday Jun 12, 2008

Web Stack in Manila

As you may have read, I was recently at the Sun Tech Days event in Mexico City talking about the Web Stack, OpenSolaris and how you can be part of it all.

Next week I'll be at the Philippines Sun Tech Days conference, again bringing you the latest on OpenSolaris Web Stack activities. If you're interested (either using or participating in creating) web tier (such as AMP - Apache, MySQL, PHP) technologies on OpenSolaris, come check out my session and/or find me at the booths any time during the conference.

I will also be presenting the OpenSolaris Testing session for Jim Walker who isn't able to attend. While not my immediate area of expertise, I've been playing with the nice Self Service Testing and Test Farm (this is particularly cool) systems and I attended this same talk given by Joaquim Rosell (ok so he's not the most active of bloggers) last month in Mexico so I feel I'm fairly familiar with their work now.

See you in Manila!


Wednesday Jun 11, 2008

Endless Nights

How long is your night?

Or, how long does your nightly build take?

It's been about six months since my first article on Unconsolidating (as opposed to 'Consolidations', the very unique practice of the OpenSolaris organization of hosting the source code of all known applications in a single source tree instead of the more commonly accepted engineering practice of having individual applications in their own source repositories) in which I looked at the then-available packages in SFW and how long it took to build it (see the article for more).

Seems like a good time to do a refresh of the numbers and a sanity check on the predictions. Back in December I just assumed a linear increase in build time as package counts went up. Clearly there is a huge variation on build times of individual packages but I figured it would largely average out over a large number of packages.

Today (actually, last week when I looked), SFW is producing 205 packages and it took 3 hours 37 minutes to build (on the same dual Opteron w/2GB RAM which I used in December). That's not exactly on the line predicted in December, but it's actually closer than I thought it would be, so the linear approximation is not far off at all, so far.

So, once we have 20,000 packages we can expect the nightly build to take over two weeks. I guess we'll need to change the SFW release cycle since currently it pushes bits out every two weeks!

Even if we only get to 5000 packages in the foreseeable future, you'll only get to do one build per work week, so better make sure nobody broke anything ;-(

At least back in the days of batch processing and punchcards one could expect the results back the next day!

(Of course, from what I'm seeing, most would-be contributor to OpenSolaris are running within VirtualBox, so your builds will take much longer than this, I'm afraid.. I've been meaning to do a current SFW build within VirtualBox on my laptop, but I know it'll be painful so haven't found the time yet...)

Future problems aside, this is certainly a problem already. As the build time closes in on 4 hours, we can already only get a single build in during a work day, which tends to make even the simplest changes take multiple days instead of a few hours as they would in more normal circumstances.

Here's a graph showing current and predicted build times. The two lines are the predictions using the numbers from December'07 and from now, June'08. As you can see, they're identical for all practical purposes.

If we still haven't managed to solve this problem by December'08, I'll revisit the numbers and graph once again at the end of this year...


Wednesday May 28, 2008

Mexico City Sun Tech Days

Gracias a todos ustedes quienes tuvieron la oportunidad de asistir al evento de Sun Tech Days en la Ciudad de Mexico la semana pasada. Fue bueno ver tanto entusiasmo e interés en OpenSolaris y especialmente en mi proyecto, el Web Stack de OpenSolaris!

Aunque nos quedamos cortos de los LiveCD en los primeros dias espero que ya para el viernes todos hayan conseguido su copia del CD, pero si no, lo pueden bajar directamente de http://www.opensolaris.com/ y VirtualBox lo consiguen en http://virtualbox.org/.

En los próximos dias pienso añadir un articulo aquí con una transcripción de los comandos usados en las demostraciones que presenté en mi charla para aquellos quienes me pidieron esa información (creo que sera más útil incluirlo como texto en vez de poner el video, asi ustedes pueden hacer 'cut and paste' directamente a su terminal).

Recuerden que para cualquier pregunta o sugerencias (o si desean participar) al proyecto de Web Stack, la lista de discusión es webstack-discuss@opensolaris.org. Lo único malo es que la lista es en inglés, por el momento no tenemos foros en español ni otros idiomas.



About

jyri

Search

Top Tags
Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today