My favorite talk was Glen Smith’s overview of Grails plugins for Rich UI support, search, Ajax and more. It was a very informative and fun presentation.

The speaker’s panel discussion was one of the best I’ve seen. One of the most controversial questions was about Groovy, JRuby and the relative impedance mismatch with the Java Platform. Neil Ford of Thoughtworks initially took the position that there was no impedance mismatch with JRuby. Graeme Rocher and others listed numerous examples of where JRuby was less integrated with Java compared to Groovy. Graeme commented that although JRuby can access compiled Java classes, it is much more difficult (impossible?) for Java to use JRuby classes. Groovy classes are Java classes and so they can be used relatively transparently in Java programs.

Speaking from experience, I’ve tried to use Java libraries (QuickFIX/J) using JRuby and it has proven much more difficult than with Groovy. One issue was insufficient (and somewhat outdated) documentation about how to integrate JRuby with Java classes outside the “java.*” packages. This was a few months ago so maybe the documentation has improved. However, with Groovy I was able to use the Java libraries within a few minutes.

I agreed with a point that Neil made that having some competition in the JVM dynamic language arena helps everyone. I highly respect the work the JRuby team is doing although I believe that Groovy is a better and more pragmatic dynamic language choice for Java developers.

There’s no question that open source technology is used widely in financial applications, especially for middle components. However, there was also a discussion about how willing the financial companies are to open source their own technology (where “technology” might include algorithms and research rather than just software).

I don’t believe any company intends to sacrifice a competitive advantage by converting their technology to open source. For that reason, I doubt that we’ll see financial companies releasing algorithms or proprietary research. However, financial companies are involved in open source in a variety of roles. The user role is probably most common today, but a few financial companies have contributed to open source. One example is the Advanced Message Queuing Protocol (AMQP) effort started by JP Morgan and others. This technology benefits financial software (and software from other domains) by provides a programming language-independent and vendor-independent wire protocol for message queues. For this technology to be successful it must be widely used. An open source strategy has a clear advantage here.

Another example is the Open Financial Market Platform (OFMP) which has recently become an Eclipse Foundation project. In this case, I’m not so clear about the benefits to the company contributing the initial code. Maybe someone from the project will comment.

Tim O’Reilly closed the panel with a suggestion that open source developers consider projects that make it easier to capture and process the vast amounts of unstructured data available on the net.

import java.util.HashMap

describe "An empty", HashMap do
 before :each do
  @hash_map = HashMap.new
 end
 it "should be able to add an entry to it" do
  @hash_map.put "foo", "bar"
  @hash_map.get("foo").should == "bar"
 end
 it "should return a keyset iterator that throws an exception on next" do
  proc do
   @hash_map.key_set.iterator.next
  end.should raise_error(java.util.NoSuchElementException)
 end
end

What is the point of this example? Is there really a significant difference between this example and the following JUnit4 test class?

import static org.hamcrest.core.Is.is;
import static org.junit.Assert.assertThat;

import java.util.HashMap;
import java.util.NoSuchElementException;

import org.junit.Test;

public class HashMapTest {
    private HashMap<String,String> hashMap =
            new HashMap<String,String>();

    @Test
    public void shouldBeAbleToAddAnEntryToIt() {
        hashMap.put("foo", "bar");
        assertThat(hashMap.get("foo"), is("bar"));
    }

    @Test(expected = NoSuchElementException.class)
    public void shouldReturnAKeysetIteratorThatThrowsAnExceptionOnNext() {
        hashMap.keySet().iterator().next();
    }

}

Sure, the syntax is a bit different and the Java version is slightly more verbose with it’s access specifiers and type declarations. However, the Java version runs fast enough that you don’t need to set up a Ant server to get reasonable performance from your tests (as recommended in the article for JTestR).

I’m not questioning that scripting can be useful in unit tests. However, I’d think that Groovy’s superior Java integration would make it a better choice for this purpose than JRuby. Does anybody know of some compelling, pragmatic examples of the relative benefit of JRuby over Groovy for Java test scripting purposes?

Installation

You can use JUnitFactory through a web interface or as an Eclipse plugin. I have only used the Eclipse plugin . It was very easy to install using the JUnitFactory Eclipse update site (more info). You can also download a file if you prefer manual installation. The JUnitFactory plugin is currently only available for Eclipse, but may be available for other IDEs in the future.

Generating Tests

Warning: Carefully read the JUnitFactory FAQ before generating any tests!

I decided to generate tests for a specific class I had been modifying. When I ran the test generator, it first required me to set up a user name and password and agree that I’ve read their legal statements about security and other usage risks.

Do not generate tests for company code with this tool unless you have permission to upload your company’s proprietary source to a nonsecure remote server, possibly to be stored there and used for unspecified purposes. In some companies, you might lose your job for this type of security breach.

This wasn’t a problem for me since I was generating tests for an open source code. I set up an account and then initiated the test generation again. Since I hadn’t read the FAQ first, I was surprised to see that hundreds of files were being uploaded to their server. JUnitFactory uploads every source and class file in your Eclipse project when it generates tests. In fact, if your project depends on other projects it will also upload all the sources and classes for those projects.

When test generation begins, a JUnitFactory view is displayed in Eclipse and it shows the progress of the test generation. My first impression of the Eclipse plugins are that they are well designed. I didn’t encounter any bugs when using them.

The test generation completed after a few minutes (the source project was small, about 50-60 classes). The Eclipse view for my test generation results is shown below.

As you can see the coverage wasn’t that great. There are several reasons for this poor coverage. The number after “Done” in the status column is a count of the number of suggestions from JUnitFactory about testing problems and how to improve the test coverage. The screen shown below contains typical warnings and suggestions.

Clicking in the second column of the JUnitFactory view will display the generated test code. Being the eager person I am, I immediately ran the tests in Eclipse JUnit test runner… and they all failed. Fortunately, the situation was quickly resolved by reading the test error messages which told me that I must run the test in the Agitar-specific JUnit test runner. I do so and all the tests passed. I’m not sure why a proprietary test runner is required. It doesn’t appear to be compatible with other plugins I have installed, like the EclEmma coverage analyzer.

First Impressions

One very important fact to remember is that the tests generated by JUnitFactory are not intended to be used in the way most people use unit tests. They don’t test correct behavior, for example. They test what the code does, not what it’s supposed to do. To replace Test Driven Design (TDD) and handwritten unit tests with JUnitFactory-generated would be a huge mistake.

Agitar suggests that the tests are best used as characterization tests. A characterization test can be thought of as executable documentation of the behavior (actual, not necessarily correct, behavior) of the tested software. If you are redesigning existing (legacy) software that doesn’t have tests, a characterization test suite can help you determine if the new software behaves the same as the previous implementation.

This sounds great in theory, but there are many practical problems. For example, it may be difficult to effectively use the legacy-based characterization tests if the OO design changes in the new code. The JUnitFactory tests are at the class-level, so if even just the class name changes in the new code you’ll need to make manual modifications to the characterization tests to run them against the new code. With a highly modified OO design, you’ll probably not be able to effectively use the legacy tests at all or only with significant effort. This problem is aggravated if the legacy software has problems like undisciplined use of method visibility. The exposed methods will generally trigger unit test generation although they are really an internal implementation detail. Of course, JUnitFactory can’t do anything about that, but it’s a practical problem to consider. If you have thousands of classes with issues like these, it would be a major effort to keep the characterization test suite consistent between legacy code and new implementations.

JUnitFactory looks inside the implementation of the class when generating the tests. This can result in some tests that are arguably overly implementation-dependent and will fail even with a correct reimplementation of the code. For example, it may assert that a NPE is thrown for a null method argument. This isn’t required behavior of the class. A new implementation might handle the null argument without a NPE (by returning a empty result, for example). This can be easily corrected in any specific test suite. However, in a large legacy system there could be many of these false alarms.

Conclusions

To quote one of the JUnitFactory announcements from a few months ago:

Remember, it’s still experimental and mostly for fun so don’t expect too much or bang on it too hard.

Overall, I thought JUnitFactory was interesting but I’m not convinced it would be useful for my day-to-day work. If I weren’t required to upload the code to their server, I’d love to try it on some work-related legacy code that we are currently reimplementing. The legacy code is EJB-based (circa 2000), has few abstractions for testing, and makes heavy use of stored procedures. My guess is that JUnitFactory wouldn’t be able to help me much in this situation.

JUnitFactory does provide various types of hooks for fixing testing-related problems and improving coverage through writing extra testing support code. Based on the coverage and suggestions for the project shown earlier in the JUnitFactory view, most of the classes in that project would need custom test framework extensions to get decent coverage. This was relatively well-written code. I’m guessing that practically 100% of the classes in the legacy system I described would need custom test extensions to get acceptable test coverage. That’s a huge job. And I’m wondering if the whole approach is fatally flawed anyway.

When reimplementing a legacy system, characterization tests at the class level are generally going to be far too low level (too sensitive to behavior-preserving design changes). Another strategy is to do characterization testing closer to the system or subsystem level and verify that externally visible behavior is consistent. JUnitFactory will not help with this type of testing.

Another use for JUnitFactory is to find holes in a handwritten unit test suite. However, be prepared for studying numerous, somewhat obfuscated tests, and then doing some deep reasoning about whether what you learn represents a hole in your current test suite or not.

I’m not sure about the practical payoff, but if you are interested in automated test generation from a research perspective, definitely try out JUnitFactory.

Related Links:

• Fun With Automated Characterization Test Generation
• Characterization: Beyond Pure Unit Testing

There’s been intense online discussion recently about ScrumMaster certification. The opponents claim it is damaging the agile community and they are pressuring the Scrum Alliance to eliminate it. Although the CSM certification is clearly described on the Scrum web site, some people believe the word ‘certification’ is misleading and will cause managers to make false assumptions about the significance of being a certified ScrumMaster. This is a reasonable concern. However, the certification opponents have not shown that this is actually a widespread problem. Certification has benefits and potential risks, but no one has shown any net damage to the agile community. The most evidence they have currently produced to support their position are a few anecdotal stories of busy, misinformed managers who thought ScrumMaster certification meant something different than it does.

As an outside observer, I see no reason why the Scrum Alliance would change their position on certification based on the unsubstantiated claims by certification opponents. Although some people confuse Scrum-specific certification with agile certification in general, these are quite distinct issues. Given the lack of evidence of any widespread problem, I believe the Scrum community should be able to make their own decisions about certification specific to their methodology. There are people who believe that Scrum is becoming synonymous with “Agile”, but I don’t believe this is true. However, I can see how this belief could cause some people to feel threatened by Scrum’s rapidly increasing adoption and associated certification programs. This could be especially true for potential competitors with their own Agile companies, services and products.

Some of the most intense discussion of this issue was in a long off topic thread on the Industrial XP mailing list. Joshua Kerievsky, founder of Industrial Logic, was one of the most outspoken opponents to ScrumMaster certification. Industrial Logic describes themselves as “XP leaders since 1999″. In 2003, Industrial Logic created a fork of the XP methodology called Industrial XP (IXP). Ironically, the creation and naming of IXP has been criticized by XP thought leaders as being deceptive, divisive, and damaging to the XP community. These are similar arguments to those being directed at the Scrum certification process. However, Industrial Logic continues to market the IXP brand. Obviously, they believe they are doing more good than harm, and they probably are. It’s surprising to me that Kerievsky doesn’t recognize that this could also be the perspective of the Scrum leadership and community.

Another example of this, typically unsuccessful, pattern of argumentation is the opposition to the word “extreme” in Extreme Programming. Although the meaning of the extreme nature of XP has been misleading and confusing for some managers, the name has never been changed. Some believe that the name slowed down early agile adoption because some managers were concerned about having “extreme”, undisciplined, anarchic, out of control programmers on their projects. Again, I think the XP community had the right to keep the name even if it caused some minor problems. In the long run, I believe that if the name is a significant problem, then XP will fade away (or be subsumed by a more acceptably packaged methodology). It will be an organic process rather than a forced change.

The thought leaders involved in this debate often present themselves as experts in change facilitation. I haven’t seen much demonstration of those skills in this debate. Instead, the discussion quickly degraded into a stream of unfounded accusations, negative characterizations and frequent sarcasm. It’s understandable human behavior, given the emotional intensity of the discussion, but I would have preferred a better demonstration of respect-oriented principles of the Agile Community.

Eventually the discussion on the IXP list spread to the Scrum Development mailing list. The purpose of the Scrum Development list is to support new ScrumMasters rather than debate certification. However, several people attempted to continue the off topic certification discussion and were subsequently banned from the list. This resulted in the creation of two new mailing lists within a few day period. One of these lists, Agile Tangents, is officially for respectfully discussing Agile topics that are off topic on other lists. It remains to be seen if this list will serve a long term purpose or will primary be a place for people banned from the Scrum Development list to vent their frustrations. As of the time of this writing, the list activity appears to be slowing down rapidly.

The discussion also has prompted the Agile Alliance to publish a position statement on agile certification. I thought it was a very fair and balanced position. It didn’t denounce the ScrumMaster certification or ask for it’s elimination, but provided objective information that can help people make their own decisions about the significance of the certification.

There will always be people who are confused and misinformed. That fact alone is not a compelling argument for change in this case. There was even significant confusion about the Agile Alliance position statement although it was written very clearly.

My recommendation is to allow the situation to evolve organically rather than attempting forceful change. If Scrum certification does not provide value it will fade away.

Dave Thomas: I think agile methodologies have adopted their own venom. And to that extent, I’m pulling back, away from, the individual agile methodologies. I believe very, very strongly in agility. I’m an author of the agile manifesto and I think the underlying values of agility fit far better with my world view of how software should be built. I think some of the implementations of agile methodologies are, frankly, scarey in their kind of…

Bob Payne: dogmatic?

Dave Thomas: dogmatic, yeah that’s a good word, that’s a mild word for some it. It’s kind of like “my way or you’re not doing it properly”, a lot of blackmail, a lot of absolutes. And to me that’s not what agility is all about. So I pull back from that. I don’t want to be associated with that.

Bob Payne: In the past I’ve felt a number of pulls in different directions and I’ve started to realize that it is not necessarily the particular practice but what the resulting outcome is, the rhythm that it sets up.

This perspective sounds very similar to what I’ve been calling “agile without a name” or “effective software development” or what Tim Beck calls “pliant software development“.

P.S. I highly recommend the Agile Toolkit Podcast. It’s a great resource to hear candid discussions about agile topics.

This problem is not unique to eBay. For example, CNet News reports that “a Google engineer has warned that if the performance per watt of today’s computers doesn’t improve, the electrical costs of running them could end up far greater than the initial hardware price tag.” The Gartner group reports that IT energy costs will increase from 25-30% to 40% within 5 years. A report from Lawrence Livermore Laboratories says that data centers will be using more electricity than the 275 million televisions in the United States. The Wall Street Journal reports that one large data center can consume as much power as a small city of 30,000-40,000 people.

There are several ways to respond to these challenges. The most common approaches are to buy more efficient hardware or to place data centers near cheap and abundant power. This is reportedly one reason for Google’s interest in creating a data center in North Carolina. However, Dan Pritchett spoke of another option: writing more energy efficient software. He estimates that inefficient programming techniques may increase eBay’s energy consumption by 25-30%. This limits scalability of their data center given the limits of the power companies to provide energy, but it also results in huge amounts of wasted money for large data centers.

It’s a very interesting idea. Computationally efficient software would also generally be more energy efficient. What other techniques can we use to optimize power efficiency for “green software development”?

It was also interesting to see that XP has been adopted in about half as many organizations as Scrum. Of course, I’d guess that many of the Scrum shops are using at least some XP software development practices.

XPlanner was the fourth most widely used APM tool and the only open source product in the top 10 most used tools. (There is a factual error in the survey results that says ScrumWorks is also open source. The ScrumWorks Basic product is free but not open source). XPlanner is also the most used tool focusing specifically on XP. It surprised me a little to see that XPlanner is adopted somewhat more by large organizations than small organizations. It’s possible that this is an indication that the licensing fees for the commercial tools are high enough to cause some pain for large organizations. I know at least one large organization that chose XPlanner over VersionOne for this reason.

As the founder^[1] of the XPlanner project, I’m very happy to see it so widely used even with no marketing or sales resources.

“The results of this experiment do not support the hypotheses that pair programming in general reduces the time required to solve the tasks correctly or increases the proportion of correct solutions. On the other hand, there is a significant 84 percent increase in effort to perform the tasks correctly.”

In specific contexts, like junior programmers pairing on complex tasks, pair programming results in significant increases in quality. However, in other contexts the increase in quality, if any, comes at a high cost. The study doesn’t appear to support the conclusions described in earlier XP research. Specifically, that a team could expect “a 15% increase in development cost for a 15% improvement in quality”. This earlier study used students rather than professional programmers and the strength of some of the researchers’ techniques and conclusions have been questioned.

My own personal experience is that I enjoy pair programming for some activities and in some contexts, but not in all contexts. On complex tasks, I enjoy pairing with other senior developers for collaborative problem solving. It’s not as enjoyable, but I also see benefit for the team when pairing with junior or less skilled developers for some tasks. The less skilled developer may provide the occasional useful input often I feel the collaboration is a net loss in productivity. For example, let’s say we’re working on functionality that uses several advanced technologies. If my pair is not skilled in these technologies then the pairing will probably not be a net productivity increase. You could say that it’s valuable for training the less skilled developer in these technologies. However, I’m speaking about the case where the less skilled developer has already had extensive opportunities to learn the technologies in question and has not been successful. “Training” by pair programming will generally not solve this problem, in my experience.

When developing simpler functionality, I see more benefit from testing and specifically from test-first and TDD techniques than pair programming. For the either/or types of people I generally enjoy collaboration and I’m not the type that wants to work in isolation from the team. Pair programming is only one specific form of collaboration. I’ve worked exclusively in open work areas for the last 7-8 years and worked in those environments several times before then. I like that work environment very much and although I had the option to have an office I choose to sit with the team.

I like Martin Fowler’s perspective on pair programming. He says “my view is that you should try it and the team should reflect on whether they feel they are more effective with pairing that without”. I’d also recommend that the team discuss in what contexts they feel they are more effective or not.

I like the Wikipedia description of effectiveness. The description defines positive effectiveness in terms of efficacy and efficiency. Efficacy is the ability to produce a desired effect or achieve a goal. In a software development context, that might mean the ability to produce timely business value. Efficiency is the amount of effect that can be produced with a given amount of resources such as time or number of developers.

I think of efficacy as a primary component of effectiveness and efficiency as a secondary component. In fact, it’s sometimes desirable to intentionally reduce efficiency to increase efficacy. This is the idea behind having “slack” resources. The demands on a team are often dynamic and having slack resources for the peak demand times can improve the team’s overall efficacy. The challenge here is to know the right amount of slack to allow when future demands are uncertain. Of course, being highly inefficient will often not be efficacious. For example, wasting time on unnecessary or valueless activities might cause us to miss schedule milestones.

How do I know if I or my team are efficacious? I have to identify the desired effects I want to produce or the goals I want to achieve. A few examples might be:

• I want to produce something useful or valuable for my employer
• I want interesting work (learning new technologies, continually improving my skills)
• I want to generate sufficient income for my non-work goals like hobbies, travel, or retirement.
• I want balance between work, play, self development, and other activities

Of course, this list could become quite large and would be significantly different for different people. Sometimes the goals conflict and trade offs are required. Sometimes the trade offs are guided by values, but often observing the trade offs we make is a great way to discover our deeper and possibly unconscious and conflicting values. Create your own list. Are you achieving achieving those goals? How do you know? Do you judge your efficacy based on your feelings or by some quantitative method or some combination of the two?

So, are you as effective as you’d like to be? If so, that’s great. Ask yourself again every a few weeks. If not, what are you able and willing to do to improve your effectiveness? You might consider whether you have sufficient feedback. Are you evaluating yourself often (think of it as a personal retrospective)? It’s difficult to accurately predict the future effects of our actions so it’s good to get feedback frequently and adjust accordingly. Rapid feedback is a form of communication that allow us to courageously take action knowing we be able to quickly adjust our actions. This is starting to sound like the values of the first edition of Extreme Programming Explained. I don’t believe that’s a coincidence.