'Tools for X': Frustrations of a Tool Architect
By toddfast on Feb 09, 2005
A Common Situation
As a tool architect, I often hear from various teams or other engineers that they want me to build "tools for X", where X is some specific technology. Such requests are well-intentioned but often misguided. In such requests, I believe there are two factors at work:
- Most engineers are impressed by tools
- Tool concepts are unknown to most engineers
Engineers generally love tools and understand their appeal, even if they can't always articulate what makes one tool more useful, easier-to-use, or more productive than another. I tend to think that this is the result of being primarily tool users and not tool authors; tools are a specialized domain like any other and require a certain amount of experience from an author's perspective in order to fully understand and articulate the opposing forces that must be balanced when designing a good tool.
Yet, the perspective of the non-tool experts is entirely understandable. In the broadest sense, a tool is just a piece of software specialized for making some task easier, and in this sense, all software qualifies as a tool to someone. Every professional technologist has used thousands of tools, and written perhaps hundreds of them in some form or another. Tools come in many distinct flavors—administration tools, analysis tools, application development tools, compiler tools, programming tools, Mars rover path plotting tools, and many more. However, for those unaccustomed to speaking about these specific variations, they are all just "tools". This leads to a fundamental disconnect between those immersed in tools and others who are not, and this is the primary motivation behind requests for tools for X.
I believe the main gap in understanding is that one can only understand a tool by understanding what general problem its user is trying to solve. This problem definition and the use cases behind it are usually implicit assumptions of someone when they ask for tools—they have a hard time solving some problem P, in problem domain D, using X. Naturally, they want a tool to help with X, but P and D often go either unmentioned or under emphasized. The hard part for those not accustomed to it is in understanding and judging the applicability and relevance of D, the problem domain, to a broader audience.
Let's take an example. I encounter a common perspective in my interactions with various Java technology groups at Sun. For years, Sun has been pumping out great specs and technologies at a phenomenal rate, and we have armies of brilliant people relentlessly expanding the boundaries of the state of the art. But most of these people are not tools experts (nor should they be). They are experts in other areas, but they all sense the need for great tools that enhance the value of their various technologies. But what are tools for X when X is JAX-RPC, JMS, or JBI? Is it possible to write tools for these technologies? Yes, absolutely. But the question we must ask is: should we?
A $4,000,000 Hammer?
How do we answer the question of whether we should build a tool for a specific technology? There are any number of equally valid axes on which we could measure value, but there is one constant goal for tools: adoption. Tools must be adopted in order to be useful; indeed, the only true value of a tool is in codifying a solution to something that it is otherwise hard to do over and over again into a repeatable, easy to replicate form. The value of a tool grows in direct proportion to the number of times it will be used. Thus, we can make a decision on the attractiveness of a tools-for-X request by evaluating whether the resulting tool will be of use to a sufficient number of users such that its development and maintenance costs are justified. Few people can afford to build a $4,000,000 hammer to pound $0.01 nails (though some part of me can't resist thinking what an awesome hammer it would be).
So, using this simple yardstick, do tools for, say, JAX-RPC the technology make sense? Given its current state and the way it is used in the industry, my answer is no. This isn't to say that tools for JAX-RPC wouldn't be of terrific value to some people, but it would be hard to argue that they would be terrific value to most people.
The primary reason is that JAX-RPC is just one technology out of hundreds that developers will use to build something that will solve a broader problem they face. The real problem that developers have is not in using JAX-RPC per se; although that may be difficult in itself, it's just a speed bump on the much-longer road to something bigger—the application. JAX-RPC will be just one small part of an application. The meat is not in tools for JAX-RPC, but in tools for using JAX-RPC and a large number of other technologies together in order to build an application.
The Reason Behind The Request
I tend to think that Microsoft panders to users when it uses the slogan, Where do you want to go today?™, but this question is actually the beginning of wisdom when applied to tools. Tool writers—and Microsoft is arguably at the top of the heap—must always start with this question when confronted with a request from someone asking for tools. It's dangerous to take such a request and execute on it blindly, as the result will almost certainly be a failure because the underlying reason behind the request for tools was not discovered.
In nearly all cases, people that ask for tools for X are actually asking for tools for application development, where application is a fluid concept. An application is really a solution to a heterogeneous, recurring set of problems, independent of any particular problem domain. An application is the bringing to bear of a solution to an instance of a recurring problem set. When this solution is canned for reuse, we call it a tool.
As an example, to engineers building say, JAX-RPC, one recurring problem they encounter is likely to be the building of JAX-RPC services (perhaps in order to test that all the other one-off, non-recurring problems they've been working through in the creation of JAX-RPC have been actually solved). This then becomes their problem domain, and naturally they gravitate toward thinking of tools for JAX-RPC as the specific problem domain which they need help solving repeatedly. However, JAX-RPC will be just one small part of another, broader type of application that includes many more parts, and many more technologies.
I'm not trying to be hard on the JAX-RPC people here; they are just serving as a convenient technology example. But the disconnect I've hopefully illustrated is at the core of quite common misunderstandings around what a tools organization is, and what it should be doing when building tools.
Give 'Em What They Want
As a part of the Enterprise Java Tools group at Sun, I believe our ultimate goal is to give people the tools that they want. And what do they want? Well, this is the enduring question in the tools business, and knowing the answer is a key part of running one. But, there are some pretty obvious clues to the right answer, which I think I can illustrate with a few specific examples: most people don't want administration tools, because administrators are few and far between; most people don't want JBI deployment tools because integration architects are rare (at least today); most people don't want disk format tools because the need to reformat a disk is infrequent; etc.
No, what most people want are application development tools. But what type of application? For the Java Studio Creator group, application means the traditional, Web-based, business application where information is collected from users via forms and processed information is then reported back to them. This is arguably the most prevalent type of application today, and the one that most people refer to implicitly when they use the term "application". This type of application and its related patterns are quite well-understood in the tools industry now, and the stratospheric ease-of-use and capability that tools for this application type have achieved is quite remarkable.
However, in the Java Studio Enterprise group, the definition of application is significantly different. Although there will always be a need for corporate applications, developers are beginning to focus less on building the same kinds of corporate applications over and over again. Instead, focus in the industry is now moving quickly toward Web services and service-oriented architectures used to solve advanced integration and B2B requirements. I contend that this is a new era, and now that corporate application development has been solved well within the Java realm, attention has turned to the next frontier: distributed applications that consist of heterogeneous, loosely-coupled services which collaborate in ways more complex than traditional client-server, human-to-computer interactions. These new types of applications are an order of magnitude more complex, and all the more difficult to build good tools for. For me, it's an exciting time to be building application development tools, and there's no time to waste building "tools for X".