It was also interesting to see that XP has been adopted in about half as many organizations as Scrum. Of course, I’d guess that many of the Scrum shops are using at least some XP software development practices.

XPlanner was the fourth most widely used APM tool and the only open source product in the top 10 most used tools. (There is a factual error in the survey results that says ScrumWorks is also open source. The ScrumWorks Basic product is free but not open source). XPlanner is also the most used tool focusing specifically on XP. It surprised me a little to see that XPlanner is adopted somewhat more by large organizations than small organizations. It’s possible that this is an indication that the licensing fees for the commercial tools are high enough to cause some pain for large organizations. I know at least one large organization that chose XPlanner over VersionOne for this reason.

As the founder^[1] of the XPlanner project, I’m very happy to see it so widely used even with no marketing or sales resources.

As a side note, I’ve had several people request the XPlanner on Rails code. It would be very interesting to have someone pursue that effort.

According to Foote and Yoder…

“THE SELFISH CLASS pattern examines how the sociobiological notion that evolving artifacts tend to behave in the interests of their own survival applies to evolving code. The radical shift in perspective that Dawkins proposed was that from the standpoint of a gene, the organism itself was just a convenient vehicle the gene employed to propagate itself. Our perspective is that programmers stand in just this sort of relationship to evolving code artifacts. The [following] six patterns examine specific strategies that code artifacts can employ to attract programmers.”

So the memes attract the artifacts and the artifacts attract the programmers? It sounds a bit far fetched. However, I do believe the Selfish Class patterns do have a role to play in how application usage spread.

It’s difficult to know exactly how many teams use XPlanner. There have been 60,000+ downloads on Source Forge (as of June 2005), which is many more than other similar projects whose download statistics are also tracked by Source Forge. Using the number of links on del.icio.us as a rough indicator of interest gives similar results. I’ve been surprised by the rate of adoption of XPlanner over other open source and commercial agile planning tools. I believe these results are related to the Selfish Class patterns. Let’s look at these patterns in the XPlanner context.

WORKS OUT OF THE BOX

Designers are more likely to reuse an object if it is easy to try it out and see how it works. A good initial impression can motivate the designer to spend the additional time to develop a detailed sense of an object’s reuse potential. When the designer can actually see that an object works, he or she develops the confidence that a more detailed exploration will be time well spent. Conversely, if an artifact, such as a class, framework, component, or application, can’t be made to work at all, or requires elaborate preparation in order to work, the designer may become discouraged, and look to other options.

If you substitute ‘user’ for ‘designer’ in the quote above, then these statements are also relevant to application adoption. They can apply in at least two different ways. XPlanner is strong in one and weak in the other. XPlanner’s strength is that it’s easy to use once it’s been installed. My original goal was to make it easy enough to use that I’d never have to write a user manual. Even though the XPlanner community has added many features since the original release, we’ve been able to keep it simple enough that users can easily discover how to use it without a manual (who reads the manual anyway?).

Where XPlanner is currently weak is that it can be difficult to install the application. It’s a relatively easy installation for an individual or team with a J2EE servlet container and MySQL already running. Just drop in the JAR file and go. However, it can be difficult for teams with limited Java/J2EE experience. XPlanner needs an installer program.

WORKS OUT OF THE BOX doesn’t mean that difficult tasks are as easy as with more complex tools and associated user interfaces. It’s more about the application should be easy to use, right from the start. More about that later.

LOW SURFACE-TO-VOLUME RATIO

Objects that allow a user to control a large volume of complex machinery with a small, simple interface are more likely to flourish than those that don’t. An object with a simple interface relative to its internal complexity may be more likely to WORK OUT OF THE BOX.

There’s a fair amount of complexity inside XPlanner. However, most of this complexity related to security, internationalization, web service interfaces, persistent data access, and so on has little or no impact on the user interface and it’s ease of use. However, under the hood exist the hooks that developers need for integrating XPlanner with other project tools.

GENTLE LEARNING CURVE

“Complex interfaces can overwhelm beginning users.

In order to be Flexible and Adaptable, artifacts may provide a variety of Customizable and Tailorable interfaces. The variety and complexity of these interfaces can be confusing and intimidating to users who are unfamiliar with an artifact. These users are precisely the ones an artifact must attract if it is to broaden its mind-share. Artifacts that exhibit high Utility without imposing an up-front learning burden on the programmer have an advantage over those that do not.

An important force here is Time. Given unlimited time, a programmer might elect to learn a complex but powerful artifact as an investment in his or her skills. However, it is now far more likely that such a programmer, faced with the overwhelming array of choices that the marketplace now, will opt for the easier to Comprehend artifact every time.”

There are two ways to look at the pattern when applied to XPlanner. From the user interface perspective, the effect is similar to WORKS OUT OF THE BOX. It doesn’t take much time for a new user to start working with an installed XPlanner application. For developers extending XPlanner, the situation isn’t quite so good. The architecture is relatively clean but has become more complex over the years as new features have been added. We’ve attempted to make XPlanner easy to extend by adding various plugin extension points, but there is definitely an opportunity for improvement here.

I’m not sure how much this pattern has influenced XPlanner adoption compared to similar tools.

PROGRAMMING-BY-DIFFERENCE

“You want to adapt an artifact to address new requirements while maintaining the artifact’s integrity.”

This is one of XPlanner strengths compared to most commercial offerings. Without source code or an extremely well designed and documented extension API, it’s difficult to adapt an application. Many installations of XPlanner have been adapted to work with existing tools. The many tests at the unit and system level also make it easier to adapt XPlanner while remaining confident that you haven’t completely broken the application.

FIRST ONE’S FREE

“In order to survive, an artifact must become widely Available.

No matter how good an artifact is, it will have no chance of proliferating if other programmers never see it. In order to survive, an artifact must gain a wide audience. One of the forces that may limit an artifact’s Availability is it’s Cost. A countervailing force is Bankruptcy. With this strategy, there is always the risk of giving away the store.

Therefore, give the artifact away.”

The first XPlanner is free, and so is every one after that. This is definitely a big “selling” point for the application. Many of the XPlanner competitors are overpriced, in my opinion. A team is likely to try XPlanner and see how it works for them rather than spend large sums of money on a commercial product. Often, XPlanner does most of what they need. In other words, it provides 80%+ of the functionality of the expensive commercial products at 0% of the cost. That’s a good trade off.

WINNING TEAM

In order to survive, an artifact must become widely available.

No matter how good an artifact is, it will have no chance of proliferating if other programmers never see it. In order to survive, an artifact must gain a wide audience. One way an artifact can become widely Available is to hitch a ride on a popular platform. The way for an artifact to win big is to have its code universally included with every copy of a system that ships. One drawback to this strategy is that the artifact loses its Autonomy. It’s fate becomes tied to that of it’s platform.

Therefore, strive to become bundled with a popular platform.

Several of XPlanner’s commercial competitors have chosen the .NET platform for their application and I believe this has hurt their adoption. I’m not saying that .NET is not a winning team, but in the context of wide application adoption, especially on the server side, I believe Java (or other cross platform languages like PHP, Ruby, Python, Perl, …) has an edge. A large number of XPlanner installations run on Linux and .NET applications are a hard sell for that crowd.

I’ve always thought that XPlanner might have had even wider adoption if it had been written in PHP or Ruby on Rails instead of J2EE. A few years (before Rails) I asked the XPlanner community about moving to PHP and there was strong resistance to it. Maybe there were just more Java programmers than PHP programmers on the list at the time. I don’t know, but I thought it was interesting.

The WINNING TEAM effect for XPlanner is more about flexibility in deploying the application across a wide variety of platforms than it is about J2EE per se.

The following list is ordered by how what I believe is the effect of each Selfish Class pattern on XPlanner’s popularity.

1. FIRST ONE’S FREE
2. WINNING TEAM (cross platform)
3. WORKS OUT-OF-THE-BOX
4. GENTLE LEARNING CURVE
5. PROGRAMMING-BY-DIFFERENCE
6. LOW SURFACE-TO-VOLUME RATIO

I believe the Selfish Class patterns are a great way to evaluate the forces that encourage application adoption, whether the application is open source or commercial.

The JdbcLDAP Bridge does provide some basic JDBC access to LDAP. However, it has some serious limitations when used with Hibernate. The getMetaData() function returns null in the released version of the JAR although this appears to be corrected in the JdbcLDAP CVS HEAD. After building the CVS version of the bridge, the next problem I encountered was that the bridge does not support table or select field aliases. Hibernate makes extensive use of these aliases. I modified the bridge to do some rudimentary handling of the aliases and I was actually able to map XPlanner Person objects from LDAP. That was cool. However, there was one other serious problem and this one is with XPlanner. The domain objects in XPlanner use an integer as the object identifier. I don’t know of any way to map the LDAP information to an integer ID. This is a good example of the wisdom of the Hibernate team’s advice to use objects as identifiers. That way the LDAP DN could be used as the Person identifier. XPlanner’s O/R mapping was designed before Hibernate and the integers are a legacy from that time. Modifying the identifier representation would mean changing all the domain classes. In other words, it’s not a trivial change.

Another approach might be to use a Hibernate interceptor to map LDAP DNs to an XPlanner identifier as an object is loaded. Sounds like quite a hack, but it might work.

I didn’t get far enough to try the C-JDBC driver for combining LDAP and MySQL.

I’ve read that Hibernate 3 can map a single object to multiple tables. Using that feature, it might be possible to have part of the Person object in the relational database and the other part in LDAP.

Here’s a wild idea… one that I haven’t tried yet, but it would be an interesting experiment.

XPlanner uses Hibernate. Hibernate uses JDBC connections to access relational data. What if we made the LDAP data look like a relational database, accessible through JDBC? There’s an open source tool called the JDBC-LDAP Bridge that provides access LDAPv3 repository using a Type 4 JDBC driver. OK, so that’s one problem potentially solved.

How do we use the new JDBC data source with Hibernate. Well, unfortunately, we have to use two Hibernate SessionFactory objects, one for the JDBC-LDAP Bridge and one for the relational database. This makes the partitioning of the data less transparent than I’d like. What if we could treat both JDBC connections as a single virtual JDBC connection?

We can. There’s an open source JDBC driver called C-JDBC that provides JDBC clustering. It has many interesting features, but for our purpose it will allow us to combine the JDBC-LDAP Bridge and MySQL (or other relational database) JDBC driver into a single virtual JDBC connection.

At least theoretically, if we configure Hibernate to use the C-JDBC driver it should see a unified view of the XPlanner domain model although it’s actually partitioned across LDAP and a relational database.

Like I said, I haven’t tried this yet. If you know of problems with the approach let me know.

The web-based, thin client approach used by XPlanner makes it very useful for a distributed team. I’ve participated in distributed planning games where most of the team was collocated and one or more team members were at other locations. We typically displayed XPlanner at the central site using a video projector and used a speakerphones and conference calls for verbal communication. As we clarified stories and created estimates, anybody at any location could make related changes in XPlanner. This worked very well.

Status tracking is equally facilitated by XPlanner’s web-based interface. Developers enter their actual hours and task status. Any authorized user can monitor the status of an iteration and see who is working on what tasks.

The ability to attach notes and documents to plan elements like stories and tasks can also be useful as a simple means of distributed communication. However, it practice I’ve seen this used much less than IM, email, phone calls, and wikis.

Our distributed team used a single CVS repository for development. We used an integration token (sometimes called an integration “hat”) to control integration activities. Some of the tokens we used included a toy light saber, a skeleton on a stick, a hard-hat, and so on. This became more problematic with remote developers. Initially the remote developer would call someone at the central location to hold the token for them until the integration was complete. The web-based integration token eliminated the need for those calls and, as a bonus, recorded a summary of recent integrations. The integration history was useful to see who was integrating what types of functionality. This type of transparency compensates partially for not having the whole team in one room.

As I mentioned earlier, XPlanner was only a part of a larger tool suite for supporting distributed development. The following is a summary of the tools I’ve used on distributed XP projects. The order of the tools list for each phase is an indication, more or less, of each tool’s usage frequency. Other tools like blogs may be useful for distributed teams but I haven’t personally used them yet for that purpose.

During story definition:

• XPlanner
• issue tracker (Jira)
• email
• telephone
• wiki

During the planning game:

• XPlanner (video projector at central location)
• issue tracker (Jira)
• telephone (speakerphone)

During the iteration:

• XPlanner (time tracking, integration tracking)
• issue tracker (Jira)
• wiki
• instant messaging
• email
• telephone (standup meetings and as needed)

If the backlog is viewed as a prioritized story queue, then some of the stories may become starved for attention (similar to starvation in prioritized task queue systems). One technique for preventing starvation is to set priorities of stories as a function of how long they’ve been in the backlog. The longer the stories have languished in the backlog, the higher the priority.

I don’t like this approach. The priorities have meaning to the team that set them and modifying the priorities automatically will risk losing that meaning. There’s also the problem of comparing priorities across different roles and types of stories. It can be difficult to compare the priority of a technical debt story to a new feature story, for example.

Another approach is to create multiple backlogs. In XPlanner, this means creating a pseudo-iteration for each customer role or story category (depending on how you want to group the stories). Given an overall effort budget for an iteration, the customers decide on how to partition that budget to the various roles. For example, new feature development might get 75% of the budget, technical debt reduction is allocated 15%, and system administration features get 10%. The stories are then planned from each backlog, in priority order according to the specific backlog, until the budget has been allocated to stories.

This technique reduces the risk of starvation for categories of stories without requiring dynamic story priority modification or attempting to compare priorities between the story categories.

I experienced a few minor challenges at the start of this effort. I was working with an existing database schema which is a little trickier than creating a schema from scratch using Rails naming conventions. It wasn’t difficult to support the existing schema but some inaccuracies in the documentation led me astray temporarily.

When I’m learning a new framework I like to be able to easily browse the source code. This exploration fills in the information gaps in the documentation and in some cases resolves inaccuracies or ambiguities in the documentation. This type of exploration was very beneficial even back when my development environment was Emacs and ctags or the info application. However, I’ve been spoiled by modern IDEs with powerful cross referencing and source code navigation features. I used the Ruby Eclipse plugin and it was nice to have some basic syntax highlighting but it has no source code navigation support (version 0.5). I felt like I was somewhat crippled compared to exploring a Java framework. Obviously this is not a criticism of Ruby or Rails per se, but it’s a consideration in the total productivity evaluation. The next version of the Ruby Eclipse plugin will address some of these issue so I expect the situation to improve.

The Rails MVC architecture was similar to the XPlanner internal architecture so porting the core functionality was not difficult. There’s definitely a serious increase in fun when developing a scripted web application. I enjoyed Ruby’s dynamic features and how they’ve been used in Rails. It’s not hype that there is no XML configuration is required with Rails. This is a real time save compared to a Java Struts application using Tiles.

Many people are evangelizing Rails as providing a 10x productivity improvement. I wouldn’t say I experienced that level of productivity increase compared to a Java web application but that could well be due to my newbie status with Rails compared to extensive experience with Java applications. To fully port XPlanner will require charting and advanced HTML table components (sorting, pagination, filtering).

I wasn’t able to find portable charting code (pure Ruby, no native libraries) at the time I wrote this article. The charting code that I did find didn’t appear as capable as open source Java charting packages like JFreeChart. I’m also concerned about packaging and installation issues when a web application requires native libraries.

I didn’t find any advanced HTML table components. It’s not extremely difficult to write one of these from scratch, but the time spent writing it offsets much of the benefits of the more productive language and web framework. I believe this situation will improve with Ruby and Rails over time and probably very quickly, but again it’s a consideration for now when considering productivity. If portability is not an issue and advanced web components are not needed for your application, then Rails may be a very good choice.

Note: I’m not currently planning to complete this experimental XPlanner port. However, if someone is interested in finishing the port and supporting it, I’d be happy to provide the source that I’ve developed so far. The current experimental code is compatible with existing XPlanner databases (as of XPlanner 0.6.2 and the upcoming 0.6.3 versions).