A Rapidly Increasing Frustration: Ken Moore's Law (January 6, 2004)

My friend Ken Moore, a programmer like myself (but with much longer experience) writes to me periodically about his ongoing frustrations with programming. He's currently a Java programmer, but has also worked with COBOL (I think), C and C++, Perl, etc., as well as some 4GLs. He's been around, if you know what I mean. What he often writes about is the frustration not just of learning one new API, but working with several new ones in a project. I have to say I agree with him, which is what I want to write about today.

The issue isn't just the difficulty of learning new APIs, but also in integrating them with others (a combination possibly new to you and to the API). It seems that each additional API we pull into our project not only slows down our development effort incrementally, it increases frustration exponentially. Thus, what I call Ken Moore's Law: With each different API we integrate into our system, the likelihood of absolute frustration approaches 1.

The Java API Problem

When Java was introduced, the concept of packages and JavaDoc combined to powerful effect. We now had both a unit of distribution (the package) along with package documentation, plus a fussy compiler that made sure we used the new distribution correctly. Packages could be nested, and a top-level package could represent not just a set of classes lumped together, but a whole Application Programming Interface. Asking someone if they knew AWT was asking them if they understood the API defined by the java.awt package. A good distribution would also include a manual and plenty of examples. You could ship around APIs, or point someone to them on the web, and they could add a completely unforseen level of functionality to their application. Later, the arrival of a ClassLoader for jar and ZIP files improved the distribution problem measurably.

Within a short time, the JDK itself had expanded to include two GUI APIs, web page support, XML, XSL and other X-APIs, sound, cryptography and etc. Other development efforts led to the massive Apache Jakarta API set, etc. The support for XML grew to about 6 different, incompatible APIs, and there are as many command-line parameter parsers out there as ships in New England.

What was worse, with our goal of supplying the rapidly expanding demands for functionality in our applications, we all turned to learning new APIs. The promise was that we could "easily" add sound, graphs, dynamic interpreted languages, CSS 2 and yadayadayada to our apps in no time at all. Our classpaths grew beyond readability. We added trailers to our cars to carry all the new JavaDoc printouts around, plus all those books from O'Reilly.

But, at the end of the day, what we found was rarely easy times, but rather, a rapidly increasing frustration. Each new API required us to not only learn the interfaces, classes, methods, factories and configuration involved, but also the order of execution for using API components, odd workarounds, configuration of configuration factories for dynamically generated runtime deployment configuration subsystems, and etc. And many times one API (like EJB) required us to integrate another, completely separate one (like JNDI), which itself had a horrible amount to learn. And then there were bugs, inter-version dependencies, and code forks. My hair has grown grey before its time, and people often now ask me, at 36, how old my grandchildren are.

And so?

The worst thing is that I see no immediate relief to this frustration. We can, for example, try to make API installation easier, integrate JavaDocs, perform automated cross-version checking, and so on. Certainly that would help. But we are stuck with, simply put, too much to learn.

In a sense, Java programmers suffer a diet too rich. We have the riches--Java APIs are so varied that you can program almost anything with Java these days, and often have multiple choices for API implementations, because there is an API for it. But the problem is that our poor human brains have a limited capacity (diminuishing over time) for absorbing new information, especially richly structured, hierarchically organized information, etc. And, worse, it would be one thing if every API was well-documented, worked perfectly, and was bug free. The fact that it is not so just adds to the aggravation, more so if we find we just cannot use the Xerces XML 2.0 parser with version 1.5 of our favorite web container.

I think we should look for easier deployment mechanisms, and better documentation systems. We also need more working example code. I think there is also better use to be made of "simplifying" APIs, using a Facade pattern, which reduce the number of steps required to get something useful done.

Beyond that, I'm just expecting a heck of a lot more frustration in my life. The probability is rapidly approaching 1.

You can comment on this on the jRoller Website site, the the host for the blog entry above.