WebSphere from the ground up - part two

A week or so ago I posted on how to do an 'assume nothing' installation of WebSphere in order to run the OSGi Alpha Feature pack. Today the Open Beta of the OSGi Feature Pack was released and the requirements and installation process differ slightly from those required by the Alpha.

The basic steps are pretty much the same, differences are shown in this color:

1. Install the free-for-developers WebSphere Application Server Version 7
2. Install the 7.0.0.7 Fix pack and an update to the Java SDK for WAS
   
3. Install the Open Beta OSGi Feature pack using the IBM Installation Manager.

Following the instructions for the Alpha works up to half way through step 2. But here, as well as downloading the 7.0.0.7 Fix pack you will need to download an SDK update from here. The file will be called something like 7.0.0-WS-WASSDK-LinuxX32-FP0000007.pak. Put this file in the same directory as you put the 7.0.0.7 Fix pack, then run the UpdateInstaller as directed in the previous post to install both fixes.

To install the Open Beta you will need to start with this link . Scroll down till you find these two links:


The first link (Web install...) is the one you want. You will download a preconfigured version on the installation manager which already has links to the repositories containing the Beta download.

Click on the Web install link and follow through the next screens till you find an option to download the IBM installation manager. It is about 86MB and will be called something like iim.linux.x86_20100128.zip. Download it and unzip it in a temporary directory, then run the install script in that directory.

The installation manager will start up and ask if you want to install Version 1.3.4 of the installation manager. Continue with the installation of V1.3.4 through to the end when you will be asked if you want to restart, accept this option. When the installation manager restarts you will be prompted for an ID and password, these will be the same ones as you would have used to download WAS in the first place.

The next two steps are to 'Import' your existing WAS installation into the installation manager. After that you will be able 'Install' the Beta code from a remote repository which has already been configured in the Installation Manager.

At the end if the Beta installation the profile management tool will be launched, you will need to augment your profile with the OSGi feature pack. Again, the screens are easy to follow.

This is a fairly high level view of the instructions, there are excellent and very detailed steps available from here in the getting started guide.

WebSphere Application Server - from the bottom up.

Recently I installed the IBM WebSphere Application Server from scratch so that I could look at the OSGi support for enterprise applications available in WebSphere. Google was not my friend in this instance - I couldn't find a quick summary of the instructions for installing the WebSphere Application Server anywhere. It is actually all quite easy, not to mention free. In this post I'll provide a brief overview of the steps and links to download sites. If you want more detail, look here for information on every supported operating system. I used an unsupported operating system (Ubuntu**) and for the purposes of experimenting with the OSGi support this works just fine. Although I use Linux commands throughout this, the same thing will work on Windows with the appropriate modifications to paths and exchanging .bat for .sh files.

There are three steps:

1. Install the WebSphere Application Server Version 7
2. Install the version 7.0.0.7 Fix Pack
   
3. Install the Alpha OSGi Feature Pack
   

1. Install the WebSphere Application Server Version 7

This is free for development use and all you have to give IBM is your name and an email address, which they will not use unless you check the boxes saying that you want to be contacted. You will also have to read and accept some license terms. The link you need is here, there are Windows and Linux downloads, no Mac version (sorry). You will need version 7, and the download is 800 MB so you need to allow plenty of time and make sure you have enough disk space.

Installing is straightforward. Extract the file you downloaded (it will be called something like was.cd.7000.wasdev.nocharge.linux.ia32.tar.gz). In the top level directory you will find a 'launchpad' file. Launch the installer like this:

sudo ./launchpad.sh

The screens are fairly self explanatory, just make a note of where you install the AppServer (default on my system is /opt/IBM/WebSphere/AppServer) and, if all you want is an experimental installation, I suggest not configuring security. I'll refer to the place that you installed the AppServer as WAS_HOME for the rest of this post.

Skip the final screen asking if you want to launch the AppServer, it's better to get used to the command line. Change directory to WAS_HOME/bin, then run

sudo ./startServer server1

this will start up the server. To shut it down again,

sudo ./stopServer.sh server1

2. Installing the 7.0.0.7 Fix Pack.

This is a two step process. You will need to download both the fix pack and an installer. The steps are detailed here. If this is a new installation all you need are steps 5 and 6.

The installer that you download will be called something like 7.0.0.7-WS-UPDI-LinuxIA32.tar.gz and the fix pack will be 7.0.0-WS-WAS-LinuxX32-FP0000007.pak. Extract the installer somewhere temporary, then

cd UpdateInstaller
sudo ./install

The screens which follow will take you through installing the installer and then onto installing the fix pack. The screens are easy to follow - you will just need to point the installer to the location of the fix pack at some point.

After installing, try stopping and starting WebSphere again to make sure everything has gone according to plan.

3. Installing the Alpha OSGi feature pack

You will need to download the feature pack from this site. Select the 'Download' tab then scroll down till you find 'Download OSGi Applications Alpha'. You will have to register and accept another license, then you should find a screen that allows you to download 'OSGi-Applications-Open-Alpha-20100111.zip'. Copy the .zip file into the WAS_HOME directory and then extract it. To complete the installation you will need to run:

cd WAS_HOME/feature_packs/aries/bin/
sudo ./complete_installation.sh
sudo ./augmentProfile.sh AppSrv01

This assumes that 'AppSrv01' is the name of the profile you want to use; if this is stand alone test installation using AppSrv01 will be fine.

Finally - restart the AppServer, and point your web browser at http://localhost:9060/ibm/console, which will give you the WebSphere admin console, like this:



If you didn't set up security you will just be able to click the 'Login' button, if you did you will have to enter an ID and password.

You should now be in a position to experiment with the OSGi Alpha as described here

Finally...

This is all quite easy to work through but there are some additional things that it's helpful to know:

You will find the log files here:

WAS_HOME/profiles/AppSrv01/logs/server1

the one you are most likely to want if things don't go according to plan is SystemOut.log.

OSGi bundles are cached here:

WAS_HOME/profiles/AppSrv01/config/bundlecache

It can be useful to look and see what is in there.

The OSGi samples (and their source code) can all be found here:

WAS_HOME/feature_packs/aries/

** If you do use Ubuntu to run WebSphere you will need to make sure that /bin/sh is linked to /bin/bash, not /bin/dash.

Unconferences

A year of hosting weekend events at IBM's Southbank location culminated last Saturday with the London Java Community Unconference. This was the most successful event so far - all the others were good, but this was the highlight of the year. Have a look at twitter for comments.

Having gained some experience over the past year I thought it was worth writing a few words on hosting unconferences from the point of view of a $BIGCORP. In the future I hope we will see many more companies welcoming technology groups into their offices at weekends.

Here are my suggestions for a successful event:




1. Remember your place

We ($BIGCORP) are really only providing a location - the event is all about the group that wants to run it and about what their members want. The LJC - led by Barry Cranford - has a clear mission, they are passionate about all things Java - all we are doing is providing a place to meet. This is not the sort of event that can be used to promote commercial products - leave that to the marketing teams.


2. Work with a good team

In fact, three good teams in my case. The IBM team that turns up on the day (Robin Fernandes, Cath Hope, Mark Hindess, Ant Phillips) - we are all developers from IBM's labs in Hursley and have worked with each other for a few years; I know I can trust them to do whatever they say they will. The second IBM team, led by the Hursley Innovation Centre, are people you don't see at the event - they work hard in the weeks running up to the event to ensure that we have catering, security, networks, projectors etc. Finally, the user group team. In this case Martijn Verburg and Barry Cranford from the LJC who displayed an exceptional level of trust in our ability to pull this off, especially given Barry's inclination to plan everything.


3. Keep it small

One of the great things about last Saturday was watching people talk to each other over lunch and in breaks. This is something that I feel is sadly missed at big conferences, at least half the point of these events is to encourage people to talk to each other.


4. What not to organise

Resist the temptation to organise the speakers in advance. Whatever happens on the day it will work out right. It really is up to the attendees to provide the content and I have yet to go to an event where there were too few talks. Perversely I also find that the quality of talks is much higher at unconferences than at mainstream conferences - I don't really understand why - perhaps because the audience are involved the atmosphere tends to be friendly and supportive towards presenters.


5. What to organise

Registration, sponsors, badges, projectors, wireless network, flip charts, white boards, pens, food (40% vegetarian, 10% vegan), directions, cleaning equipment, twitter tag, 'the grid'. That's it really. Oh, and some signs:



Thanks to Richard Dallaway for the image.

The other side of IBM

Last week was full of 'firsts' for me. It started with my first ever commit to an Apache project . This was a career milestone for me, I will be writing a lot more about that project in the future, but not in this post.

The other 'firsts' were all associated with BarCamp London which ran from Saturday morning through to Sunday evening at IBM's Southbank location. It was the the first time I had ever been to BarCamp and I was enormously impressed by the level of organisation. I've been to a few big conferences and they could learn a lot from the BarCamp organisers who pulled the whole event together in their spare time.

The other side of IBM
Image courtesy of Adewale Oshineye - check out his other photos of the event here

It was also the first time that IBM UK has ever hosted a BarCamp and it's hard to describe the level of nervousness that was permeating the corporate psyche prior to the event. As it happens, apart from one small(s) problem, the weekend went entirely smoothly. So many people helped to pull this together that I can't list them all. I will however mention three companies that I bet no-one else will talk about, G4S, Johnson Controls and Compass . Their employees were on site for much of the weekend ensuring that the building was safe to sleep in, kept clean and comfortable and that people were fed. Well done!

I think IBM gained a great deal of respect for trusting BarCamp to use their premiere London location knowing that they would be entertaining clients there on Monday. The developers from IBM who attended also gained from meeting a different and diverse set of people. When you work for a big company it's too easy to develop a language and way of communication that only makes sense inside the company - getting out once in a while at least makes you realise you are becoming institutionalised.

Someone asked me why I got involved with organising BarCamp - it's a simple answer. I've seen so many complex problems solved by bringing highly technical and creative people together and allowing them time to communicate. Sadly there aren't that many places where 200+ people can meet and share ideas. IBM has a great location which is not often used at weekends, how could I not have got involved?

Does my PHP app run on Project Zero?

As the PHP implementation in Project Zero gets more complete it's possible to run quite a few significant applications. It would be nice to know in advance how likely it is that a PHP application would run on Project Zero - without investing a whole lot of time in trying to run it. Well, now you can.

There is a new Project Zero application which will scan your application and tell you whether it thinks it will run on Project Zero. The way it does this is to extract information from the Project Zero PHP parser - for instance it finds out the names of functions that are declared in the application and compares them with the names of functions that are invoked. If the application scanner finds that a function is invoked by the application but is not declared by it, it uses function_exists() to figure out if it is implemented in Project Zero's PHP. Similar analysis is carried out for classes and methods in the application.

The instructions for extracting and running the scanner are here. They assume that you already have a working version of Project Zero.

One of the reasons that I wrote this was that I wanted to figure out if PHPUnit would run on Project Zero. This led to a horrible chicken and egg situation in that I can't write PHP without PHPUnit and most of the scanner is written in PHP. In the end I had to inch forward on both tasks simultaneously, with the final benefit that I worked out how to use PHPUnit at the same time as writing the scanner. The instructions for running PHPUnit are here. I'm not claiming full support for PHPUnit in Project Zero - but there is enough to be able to test PHP classes as you write them.

The application scanner can't say definitely whether an application will run or not, this is because it is using static analysis. However it gives enough information to be able to tell whether it's worth trying or whether you should mail the Project Zero team with your scan results and ask them when they think they will have support for your app :-)

On no! Not more tests!

One of the problems with the incredible level of success of the PHP TestFest is that PHP gets more tests, 887 more to be precise.

Well, isn't that the point? Yes, but it's beginning to take *forever* to run them which is a bad thing and as TestFests get bigger and better this will only get worse.

A year or so ago I started worrying about this, then I met Stefan Priebsch at IPC and made him worry about it too. Together we started a project to re-factor the code that runs tests (run-tests.php) so that it could run tests in parallel.

In May we had a prototype going, it ran most of the tests in sequence. It still doesn't have all of the capability of run-tests.php but it was enough for our PHP GSOC student (Georg Gradwohl) to start writing and integrating code that would run groups of tests in parallel.

Yesterday I ran some tests on my dual core lap top - this isn't a proper benchmark and I had all sorts of other stuff going on too...but here they are:



The chart shows the time taken to run 7700 tests with the current code (old), new code sequentially (seq) and the new code with increasing numbers of parallel tasks (para). As you can see we are able to run this sample in about half the time taken by the code that is used at the moment. There is still a long way to go with this but I'm happy to say that a couple of people have started to check out the code and try and run it. At the moment this only works on *ix, supporting Windows is on the TODO list.

With the help of the 'many hands' of the PHP community I'm quietly confident that by the time PHP gets all the tests it needs we will have something that will run them efficiently :-)

PHP TestFest - one more month to go!

Groups from all over the world have contributed 554 new tests for PHP since April 1st 2009. Of these, 327 have already been moved into PHP's CVS (thanks Felipe!). The results of running tests on a Linux 32 bit platform can be seen here (sorry about the colour scheme), the user groups who have contributed are also listed. We also run the tests on a 64 bit Linux platform here, but this only runs PHP5.3 and PHP5.2.

I also ran some coverage tests recently to try and work out how much of a difference we were making overall. This turned out to be far more problematic than I had thought it would be. The main issue was one that Felix had already found, ccache and lcov do not coexist happily. It turns out that there is a check for this in the php configure script - but unfortunately the debian way of installing ccache is simply to link it to gcc, which by-passes the check in configure. Oh well.

The before TestFest coverage data shows an overall line coverage of 70.4%, a run from the 28th May shows a line coverage of 72.3% in total. There are minor differences in the number of lines but I think this is to be expected. It's interesting to look at the areas where I know groups have been concentrating their efforts - for example the line coverage of ext/ldap has increased 8.5% to 89.1%! Check curl, date, dom, ftp, gettext, intl, posix, reflection, sockets, spl, sqlite3, xmlwriter and xsl for other TestFest related increases in coverage.

I'll rerun the coverage tests regularly in future. I should perhaps mention that I know that ext/imap statistics are missing and that's because we don't yet have the ability to run the ext/imap tests on the system that is being used for TestFest. I will fix this in the next week or so - if you see anything else that doesn't look right please email me and I'll fix it, or better still, email php-qa@lists.php.net :-)