Thursday May 21, 2009

Introducing Twitter Data

My good friend and colleague, Jiri (Jirka) Kopsa, and I recently formulated a means of embedding semi-structured data in Twitter. We call it Twitter Data.

From the site,

Twitter Data is a simple, open, semi-structured data representation format for embedding machine-readable, yet human-friendly, data in Twitter messages. This data can then be transmitted, received, and interpreted in real time to enable powerful new kinds of applications to be built on the Twitter platform.

Here is an example Twitter Data message:

I love the #twitterdata proposal! $vote +1

We'd love to hear your feedback on our proposal. Catch us on, or on Twitter: @toddfast, @jirikopsa. If you tweet about Twitter Data, please add the #twitterdata hashtag so we can track your comments.

Friday Jun 06, 2008

zembly is here!

I've been working in stealth mode for the last 18 months, and today I'm able to show the results to everyone: is now live!

zembly lets you collaboratively create and host social applications of all shapes and sizes, including Facebook apps, meebo apps, OpenSocial apps, iPhone apps, Google Gadgets, embeddable widgets, and other social applications, using just your browser and your creativity.

More info later, but head on over to to sign up for the private beta before the invitations run out!

Monday May 12, 2008

Applications for the Masses by the Masses: Why Engineers Are An Endangered Species

I did a session last week at JavaOne 2008 where I talked about how application development, and the role of application developers, is changing. Here's the abstract:

When we engineers normally think of applications, we think of elaborate technology that takes highly trained developers weeks, months, or years to develop and debug. But our days are possibly numbered. In this era of light-speed boom and bust, the demand for technology is higher than ever and engineers and their traditional development techniques simply can’t keep up. What’s needed is “disposable applications,” applications so quick and easy to write that they are cheaper to throw away than to maintain and which the increasing numbers of casual technologists can consume and even create themselves.

Fueling this demand in recent months has been the release of high-profile web platforms (such as Facebook, Ning, Meebo, and others) that increasingly enable nontechnical users to compose mashups and other social and situational applications out of widgets and RESTful-style web services, all built by use of lightweight technologies and composed right from the browser.

This session describes how these factors are coming together to produce a new paradigm of application development in which hordes of 16-year-olds are in charge and software engineers are overwhelmed by the flood of applications created by tech-savvy novices. It also examines the roles of high technology versus technology for the masses and shows that they are actually complementary and a boon to engineers and nonengineers alike.


I presented to about 500 people over two sessions on Wednesday and Friday, and despite the fact that I could not show a demo, it was a rapt audience. Since then, coverage has been popping up around the Web.

Here are slides and links to articles and other material:



Wednesday Oct 31, 2007

Google OpenSocial and the perils of abstraction

Google has announced the imminent release of something they call OpenSocial. My understanding from the press coverage, though not necessarily correct, is that Google is trying to abstract away (using a framework) the specifics of developing apps for certain social platforms. It sounds interesting, but we in Sun tools know what a dangerous path this is; abstraction is a religion among software developers for dealing with complexity. Sadly, this religion is founded on a lie.

My opinion for the last couple of years, as a tool provider, has increasingly been that we need to stop thinking like software architects ("abstract everything", "there can be only one") and instead deal with the real, messy, heterogeneous world. We need to dive into the chaos and find other strategies for dealing with it (or to reflexively use my argument above, there is not just one general strategy, abstraction, for dealing with complexity).

There is usually one overriding reason for abstraction in software: productivity. Abstraction gives people a conceptual handle to a larger number of specific things than is otherwise possible to manage in a fixed amount of time. This works well when the level of complexity is low and when specific instances fit well into a single conceptual framework. Unfortunately, most things don't--oh, they look like they do at first, but quickly the edge cases the abstraction cannot deal with become more of the problem than the original productivity problem. This is when a new paradigm shift is born, the old technologies and tools are swept away, and a new wave of technology and tools emerges (usually built around a different, but simpler, abstraction that is easier to grasp, until it too becomes top-heavy).

Abstraction is inherently limited in what it can do precisely because it ignores specifics; this is why abstraction is a useful, but not an ideal, strategy in the real world. I think we are well past the days on the Web when abstraction alone can help us be more productive. Now, it's up to more powerful methods of expression plus domain-tailored tools that will really help us deal with complexity without the religion of abstraction.

However, there is another way to meet the same productivity goal as abstraction, at least in the medium range where there are more than a few and less than a lot of specific instances: raise productivity through other means, like the use of disposable software. Thus, one could argue that you don't need a metaplatform like OpenSocial in order to plug into various platforms if instead plugging into those platforms is essentially as easy as a couple lines of code. Plus, when you have something this productive, you can still take advantage of platform specifics and better tailor the integration and user experience.

Wednesday Aug 29, 2007

There is no chicken. There is no egg.

We spend a lot of time in the tools group at Sun thinking about users. In short, we want to get more. We talk a lot about the "chicken and egg" problem of community building, which states that you need community to get more users, but you need users to get community. How do you actually build community? The answer may be pretty simple: you don't.

I've been spending a lot of time lately working on social computing models with several brilliant colleagues, a couple of which are bona fide sociologists. One colleague, Chun Xia, has been applying elements of Marshall McLuhan's media theories to our problem, and has articulated an elegant thesis describing the relation of technology and culture.

The short form of this thesis is that technologies engender social interactions, thus becoming forms of media, which then ultimately produce culture, which then influences technology, ad infinitum. To paraphrase McLuhan: we shape our tools, and thereafter our tools shape us.

The technologies we are producing at Sun, including NetBeans, have few social aspects to them. They begin and end with the technology. Any social interaction, and eventually culture change, occurs outside the technology through a medium that is related but disconnected. For example, culture forms around use of APIs like Java EE, or around collaboration on open source projects via mailing lists and version control; culture does not form around APIs, projects, or technologies like NetBeans or Eclipse per se, but where the social interactions are happening around the usage and modification of them.

It's interesting to think of the IDE as a testbed for social computing. Thinking in this way, we can surmise that Eclipse is not popular (merely) because of its technology, but because it is a tool that has engendered a vibrant culture. IDEs, unlike much other software, are extremely malleable and oft-changing and so are fertile soil for growing cultures around them. (We can also attribute the rampant success of the modern IDE to the culture around Java, not the technology itself.)

This also fits what we see with many open source projects—the project serves merely as a substrate on which a culture is founded, and frankly, the substrate doesn't really matter much at the end of the day. The usage of an open source technology is ultimately correlated more to the essence of the culture around it than to the technology itself. Take Linux, Apache, Struts, Eclipse, Ruby on Rails—the list goes on and on—none of these are the best technologies in their class, but they are among the best cultures in their class. Good technology is relatively easy; good cultures are one in a million.

This principle also helps explain why good technologies fade into obscurity, while mediocre ones take off—vibrant culture breeds voluminous usage. Where there is an ailing or missing culture, usage languishes, and technologies wither.

Perhaps these reasons are why some of my colleagues at Sun grapple with the problem of community, yet never make much headway. To most people at Sun, "community" is a walled garden cut off from the real conversation, a case of we know better, shut up and listen. Sun is hardly alone in holding this attitude, but particularly prone to it because of its technology-centric perspective. From that perspective, community is not something that is self-modifying nor self-sustaining, rather it is something to be moderated, controlled, and ultimately held at arm's length. We talk about gathering community in order to drive usage, but we're actually grasping at a shadow of the thing we should want instead—culture.

If we foster culture, then community will come easily, for free...or not, if we don't foster a vibrant culture. I won't pull any punches: with a few notable exceptions, we don't have vibrant cultures around much of our technology at Sun, and this is at the root of the problem in driving more usage, not the technology itself.

When we talk about ways of increasing usage of a particular product like NetBeans, we need to think less about changing the technology for technology's sake, and instead think about changing the technology as a means to improve the human culture around it. When it comes to the competition, feature parity is not what we should be focusing on, but rather culture parity. We don't necessarily need better technology, but we absolutely need better culture.

So now for the part where I should be standing on a balcony:

There is no chicken. There is no egg. Community emanates from culture, and the culture begins with us. We are the grain of sand at the center of the pearl. Change ourselves, let go, and we will grow.

Tuesday Jan 09, 2007

Introducing Bug-Driven Development

Bug-driven development, or BDD, is a lightweight, software-development methodolgy based around using a project's bug tracking system to track not just just defects and RFEs, but also software features. The central concept in BDD is that from a user's perspective, the biggest defect in a piece of software is that a necessary feature is missing. Whether that feature works as it should is critically important, but actually is a secondary concern in the project lifecycle. Therefore, I argue that the distinction between missing features and bugs is, in many ways, artificial.

The primary advantage of BDD is that it addresses the need for both project governance and transparency, which are generally quite difficult to achieve without rigid project lifecycle methodologies and considerable overhead, usually at the cost of productivity. Because BDD uses a project's ubiquitous bug tracking system that developers already use, but in a novel way, it is a lightweight methodology that can be easily adopted without the need to introduce significant new processes or tools.

I formulated this approach because, like many of you, my engineering teams work under very demanding circumstances including wide geographic distribution and native language differences, and we needed better ways to track project progress and feature implementation. Development teams, particularly the NetBeans team and other projects working in open source (for example, Mozilla), have used elements of this methodology for many years. However, to my knowledge, the particular formulation of BDD I describe below is novel in one or two ways, though the name is not (see examples below).


I've had several comments on why I've called this "Bug-Driven Development" and not "Feature-Driven Development" or "Issue-Driven Development". The initial impression of many people is that "bug" refers only to defects, or that they are negative in some way, and so calling it BDD gives a bad impression.

Those other names are fine, but I have three reasons for preferring to stick with the "Bug-Driven Development" name. The first reason is that from a user's perspective, the lack of a feature really is a defect in the software, or at least the result is the same: the user is frustrated and demotivated from using the software. Does the distinction really matter? I think it's novel to think of software in this way; to think of filling in a hole by fixing bugs rather than building a mountain by implementing features. The advantage here is that it compels us to think of users' needs as drivers for what we do, rather than the fun and interesting things we'd like to do just because we can. This is what most projects lack, particularly those led by technologists. We should think of bugs as good things that help us achieve what users really want and need.

My second reason is that informally, we call the issue tracking systems we use "bug tracking systems" most naturally. We put RFE's in our bug-tracking system, and we talk about reviewing and closing bugs all the time. So, I think it's a very natural name on one level, until someone begins to think a little more about it. That's where the next reason comes in.

My third reason is that the name is provocative, and even non-sensical at first. This is useful in making any new concept not only marketable, but palatable; it's often necessary to first shock people out of their current thinking before supplanting it with something else. The idea that bugs could drive software development sounds fanciful (or misguided) at first, but when examined, it indeed makes sense. Changing the way people think about the role of bugs in software development--meaning, not as annoying little tasks that come in after all the fun stuff is over--is the first step to changing real engineering practice.

Similar Proposals and Names

I am not the first person to use the name "bug-driven development" or the acronym "BDD", nor am I describing an all-new methodology.

  • Dan North uses "BDD" as an acronym for Behavior-Driven Development. A very interesting proposal which I recommend you read, but it is not a coincident methodology.
  • Christian Robottom Reis and Renata Pontin de Mattos Fortes identified bug-driven development in their paper, An Overview of the Software Engineering Process and Tools in the Mozilla Project from 2002. While they identify that Mozilla development is driven by issues and RFEs (like many open source projects), they don't outline a generalized usage of BDD to track feature implementation, reviews, or other non-code deliverables, particularly in a hierarchical fashion, which makes the methodology as I describe it more applicable for ensuring feature delivery in a time-scheduled engineering organization. Consequently, this formulation of BDD has more applicability to non-open source software development, which is the majority of software development happening today.


The BDD concept evolved from a generalization of test-driven development, or TDD, which is a popular development strategy today. TDD can be a valuable part of engineering practice, but it's inherently limited in that it's very low-level and focuses exclusively on the implementation of code. What drove the implementation to be written in the first place--the feature list, derived from business requirements--is excluded from this process, and is left to be filtered through the developer's translation of requirements to implementation (which is often imperfect). It's just as easy to write tests that test a poorly designed feature as it is to write them to test a well-designed feature. Therefore, unit tests can be used to track compliance to a feature, but coming to agreement on how a feature should work, much less what features should be present, is a necessary step before testing becomes applicable and useful.

I begain thinking about this problem when considering how to improve the execution of the Java Enterprise Tools development team at Sun. Because my colleagues and I have more to do than we can possibly keep track of manually, I began thinking about an application that could be used to track not just low-level tasks, but also features and designs. I needed a way to transmit designs to engineers and project leads who would then take those designs and embody them, while also being able to track and remember the many design decisions that were made along the way. In addition, the team needed a way to transmit requirements for visual design, icons, docs, and other auxiliary deliverables to the appropriate people on other teams. Traditionally, this has all been done using email, or some mishmash of project tracking tools, which vary from team to team.

The application I wanted was starting to look big and complicated. Like all such systems, it's easy to devise a grandiose application to do everything under the sun, but they seldom get implemented, and when they do, they take on a life of their own. It seemed that building such a custom application was infeasible, and besides, I'd be starting from scratch and inventing some new process which would then need to be accepted by a large number of people. Unlikely on both counts.

The leap to conceptualizing BDD happened because of a simple interteam communication problem. Recently, we realized that three teams had built three different IDE wizards in three different ways, and none of them were consistent. Clearly, the teams had not communicated sufficiently to make these wizards consistent; there wasn't a single design or designer driving the implementation, nor were the individual craftsmen building them trained to do things in a consistent style.

When looking at the product from a holistic perspective, it's obvious that these wizards should be consistent, but no one noticed the scope of the problem until after all three wizards had been implemented. I promptly filed a bug on the problem and assigned a lead to hammer out a common design that could then be implemented by the three teams.

There were several issues with just that, however:

  • How do I and the managers track that the lead has completed a design and communicated it?
  • How do we record the design so that it can be consumed by all three teams?
  • How do we track progress on implementing the design across three different teams?
  • How do we remember to check later that the wizards were in fact implemented consistently?
  • How do we remember later that a fourth wizard should be consistent with the existing three?

I began thinking about when I could have filed this bug and begun this process. Obviously, I could have filed the bug the moment the three wizards were implemented or any time after that, which is exactly what happened (I think we found the consistency issue about a month or two after the last wizard was written). But it's likely that someone (me!) knew that these three wizards should have been consistent, and I could have told the teams this before they completed the wizards, or even before they began. In fact, I probably did tell two teams, or two different teams at different times, but I had no way to track it or remember that design decision. It was lost in the overwhelming load of design decisions and action items flowing through my inbox every day.

In a small team, these sorts of inconsistencies would have been found immediately by the implementing engineers, and resolved without having to think about it, especially if they were users of the product themselves. However, in a large team, trying to implement something to help customers that are not the same kind of technologists, in different areas of the world and with limited windows of communication, it's simply not possible to rely on this sort of ad hoc review process.

I realized that if I knew that consistency between these wizards was necessary, I could have filed the bug even before the features were implemented. Not only would this have been a clear way to communicate this aspect of the design, but it prevents the discomfort that comes with the typical review process where someone's work is critiqued and then must be reworked. By putting what I know as a design owner into an easily tracked form preemptively, I circumvent many of the worst aspects of the implementation process. By logically following this train of thought, I came to the realization that I could have filed this bug before development even began, in the planning stages of the project. Not only would I have captured the design in a concrete, living form (as opposed to functional specs, or PRDs), but it would have been open for everyone throughout the extended team to audit and track without any additional overhead on my part, or on the program manager's part.

Of course, all these things are possible without BDD. A good team that works well together can certainly turn out a superb product without using such a methodology. However, this is very much dependent on the makeup of the team and its leadership. I make an analogy to politics here: Democracy as a political form isn't unique in terms of its being able to produce positive results. Certainly, an elightened dictator or monarchy can achieve the same or higher quality of life for a country's citizens. But, these instances are very much dependent on the quality of that leadership, and history is littered with stories of thrones being passed to madmen or tyrants after death of a benevolent monarch. What democracy has done that is unique in history is in creating a process by which to guarantee a generally good result for its citizens by means of checks and balances, independent of a particular government at a given time.

BDD is just that: a system for checks and balances by allowing for governance and transparency of the development process. And best of all, doing that by using a system that already exists in every project, which developers are comfortable with, and with minimal change in the normal development process. Ultimately, I realized that the grandiose application I wanted at the beginning of all this already exists, that it covers perhaps 95% of my requirements, and we are already using it--it is the bug tracking system that we rely on day-in and day-out.

Assumptions and Prerequisites

There are a few assumptions and prerequisites needed to make BDD work. On the whole, I think the list is quite short, and covered by most bug tracking systems in widespread use:

  • Project uses a bug-tracking system
  • Developers are familiar with and committed to using the bug-tracking system
  • The bug-tracking system allows for assignment of bugs to an individual, who can then track his assigned bugs easily
  • The bug-tracking system allows bugs to be designated as dependent on or blocking another bug, and can automatically help users traverse dependency graphs of bugs
  • Preemptive notification (for example, email) of changes to bug status or addition of new bugs

Note that NetBeans IssueZilla meets all of these requirements.

How It Works

A feature owner is a person (or persons) that have defined a feature that should be included in the product. They define this feature by entering it into the bug tracking system in a way that makes it clear that the issue is a feature, not a defect or RFE. In NetBeans IssueZilla, this means simply changing the issue type to "Feature".

Generalizing the idea of a missing feature as a bug from the user's perspective, the root bug for any project is that there is no product that does X, where X is a business or customer requirement. For example, for the NetBeans IDE when it was started years ago, the root bug might have been "Implement a modular, all-Java IDE". Let's call this bug B1. This bug should be assigned to the feature owner, and can only be closed with his consent.

Bug Description Owner
B1 Implement a modular, all-Java IDE Jaroslav

Below this root bug are other sub-bugs, each describing a coarse-grained feature set within the root bug. By sub-bug, I mean that these bugs are marked as blocking dependencies on the root. For example, for NetBeans, one or more feature owners may have added bugs B2-B4:

Bug Description Owner
B1 Implement a modular, all-Java IDE Jaroslav
+-B2 Need a Java editor Petr H.
+-B3 Need a way to build Web applications Petr J.
+-B4 Need a project system Jesse

We call B1 an umbrella bug, in that it offers a way to track the sub-bugs (sub-tasks really). However, it also gives Jaroslav and others a convenient location to discuss the overall direction of the IDE, to upload prototype and mockup screenshots, functional specs, customer requests, and more. The set of relationships between B1 and the other bugs now has some interesting properties:

  • Because bugs B2-B4 are blocking B1, Jaroslav can't close his bug until the other three feature owners close their bugs. Assuming Jaroslav wants to close his bug, perhaps because his manager is insistent he do so, this fact gives him a natural reason to "bug" the other three people to get their work completed.
  • In order to close his bug, Jaroslav is kept involved with his other colleagues, no matter where they are located, because they appear as blocking providers to his bug. This is a particularly useful property if the inclination of project engineers is not to talk to others and instead wall themselves off from the rest of the team.
  • It gives Jaroslav's manager an easy way to see where he's at and that he's waiting on others to complete their work; this helps his manager pursue closure of the bug through different channels.

Now let's say that Jesse, owner of B4, decides his feature is large enough that he needs to assign multiple owners to pieces of the feature. He then adds new sub-bugs, assigns them to owners, and blocks B4 on those:

Bug Description Owner
B1 Implement a modular, all-Java IDE Jaroslav
+-B2 Need a Java editor Petr H.
+-B3 Need a way to build Web applications Petr J.
+-B4 Need a project system Jesse
+---B5 Ant-based project Engineer1
+---B6 Java project Engineer2

Now Jesse realizes that for B6, he needs to coordinate with both HIE and docs, as well as Engineer2 as the implementor. He does the following:

Bug Description Owner
B1 Implement a modular, all-Java IDE Jaroslav
+-B2 Need a Java editor Petr H.
+-B3 Need a way to build Web applications Petr J.
+-B4 Need a project system Jesse
+---B5 Ant-based project Engineer1
+---B6 Java project Jesse
+-----B7 Java project - Implementation Engineer2
+-----B8 Java project - UI design HIE1
+-----B9 Java project - Documentation Writer1

By assigning bugs B8 and B9 to HIE and docs people, he's now involved them in the development process in the same way as Engineer2. He also changed the responsible person for B6 back to himself. This illustrates the ability to reassign bugs as needed to appropriate owners, and makes sure that there is a concrete person assigned to assuring that bugs B7-B9 are completed to the feature owner's satisfaction.

Now HIE1 wants to break down B8 into several sub-bugs, such as a list of icons that need to be delivered. He does the following:

Bug Description Owner
B1 Implement a modular, all-Java IDE Jaroslav
+-B2 Need a Java editor Petr H.
+-B3 Need a way to build Web applications Petr J.
+-B4 Need a project system Jesse
+---B5 Ant-based project Engineer1
+---B6 Java project Jesse
+-----B7 Java project - Implementation Engineer2
+-----B8 Java project - UI design HIE1
+-------B10 Icon 1 HIE1
+-------B11 Icon 2 HIE1
+-----B9 Java project - Documentation Writer1

This illustrates the fact that the HIE1 person, and not just engineers, can add bugs and use this methodolgy to track deliverables. This is extremely powerful, not only because there is a tool to do this, but that it's a common tool across groups and domains.

One thing that's not illustrated above is that B7 is clearly dependent on B8. This relationship can be expressed by making B8 a blocking issue for both B7 and B9, like so (omitting B10 and B11 for clarity):

Bug Description Owner
B1 Implement a modular, all-Java IDE Jaroslav
+-B2 Need a Java editor Petr H.
+-B3 Need a way to build Web applications Petr J.
+-B4 Need a project system Jesse
+---B5 Ant-based project Engineer1
+---B6 Java project Jesse
+-----B7 Java project - Implementation Engineer2
+-------B8 Java project - UI design HIE1
+-----B8 Java project - UI design HIE1
+-----B9 Java project - Documentation Writer1
+-------B8 Java project - UI design HIE1

This is important to illustrate that a bug can block several others, and not just those at bugs the same level. This is useful if there are particular deliverables from a bug that feed into several other bugs.

Let's talk more about bug B8, the UI design. There is certainly going to be a large amount of debate and design for this, and that would proceed as usual through email or face-to-face meetings. However, when the times comes that a design has been decided upon, rather than leave that in email, the owner for B8, the HIE person, would update the bug with that result, and possibly some screenshots or mockups to show how it should look.

If any questions arise about the design, the bug can be used to discuss and track those, so that when the doc writer comes in to write documentation, there is a wealth of information available, and all the design decisions have been recorded. In the traditional development model, were he not on the email alias where the discussions happened, or if he did not remember the conclusion of the discussions, he would then need to go back to someone like Jesse and ask for a recap, which is overhead for both of them. While this may still be necessary even with BDD, having as much information captured as possible makes the need for this less frequent.


At this point, the general concept of BDD should be clear. In summary:

  • BDD is not really very different than the process most projects already follow. The difference is primarily in timing of and perspective in using the bug tracking system: The bug tracking system is used from the very beginning of the development process to track features.
  • Clear owners are defined for each deliverable, whether that's an overall feature, code, designs, icons, docs, or something else relating to the project
  • Because bugs are dependent on other bugs, owners have a natural reason to talk to colleagues and resolve issues rather than go off on their own
  • Bugs can be refactored into sub-bugs as needed in a right-sized manner for the project. Sometimes, a feature can be wholly implemented by one person, so only one bug is needed. But in the cases where multiple people are involved, that can be expressed with clear ownership.
  • The information in the bug tracking system can be mined to create a dashboard of overall project status
  • Defects filed by customers, QA, and others can be related back to the parent feature. This means that both unimplemented features and defects appear in the same dashboard. This is critical for getting an overall picture of the completeness of the project.
  • Bugs can be assigned and reassigned as needed to gather more information, or to transition work from one person to another, even across widely separated time zones and organizational boundaries. The overall project status remains the same, however.


Check out an existing usage of BDD in NetBeans IssueZilla, bug 88122:

Differences from Current IssueZilla Process

Lifecycle of issues in IssueZilla:

  • Considerable use of umbrella bugs
  • Feature owners close feature bugs, not QA


Overall bug counts start high and go down

The number of bugs starts high as feature owners add bugs to the system, and as those bugs are refactored into sub-bugs to better track individual deliverables. Of course there are upward blips as defects are found, but these are small compared to the large spike in bug counts that normally occurs some significant way into development as QA begins filing defects.

Processes become uniform

We break the current asymmetry between "before bug filing" and "after bug filing". For example, traditionally, implementation team meetings go through three phases. The first phase, during initial development, focuses on high-level TOIs between QA, docs, and engineering. For new development, the QA team is instructed not to file defects because things are changing too rapidly to make them worthwhile.

At some point in the project, perhaps halfway, there is a transition phase, during which QA begins filing defects. The difficult part of this phase is in communicating what is sufficiently stable enough to be tested by QA. This requires a great deal of communication between engineering and QA, and a common definition of when a feature is "done" versus really done.

The third and final phase comes as most features are completed and QA and engineering reach an iterative balance of opening, fixing, and closing bugs, and the implementation team meetings primary function becomes tracking bug status. However, these bugs revolve around feature quality, and are measured according to the test specs designed by QA, which are based on functional specs provided by engineering. What those specs contain, and how a feature measures up to it in the final design, are very fluid, and require a great deal of teamwork between QA and engineering to negotiate. In good teams, this works well, though it comes with overhead. In poor teams, the result is disaster and a general breakdown of any quality assurance.

With BDD, however, the process is the same from the beginning. All i-team meetings become bug tracking meetings. It is just the content of the bugs that change over time, from features to defects. However, everyone is kept on the same page, in the same way throughout the project lifecycle.

Full project lifecycle dashboard

QA normally develops a bug dashboard as the product gets close to a release, so that management can track progress on bugs. This is a primary tool used in "bug council" meetings, in which engineering managers and QA work together to actively reduce the number of bugs. (In fact, this property of engineering managers--that they are relentless in tracking and getting bugs closed--is an important aspect of why BDD works; it gets them involved in the process earlier, in more consistent way.)

Rather than wait to develop the dashboard at the end of the development process, in BDD, the dashboard is developed at the beginning. Everyone, from architects to managers to engineers, all track the project from the beginning using the same technology and the same process. The health of the project is immediately available to anyone, from the time that the feature owners enter features into the system.

Consistency of implementation

Because features and defects are in the same place, the developer's primary task list is the set of bugs assigned to him. This puts him on the same pages as his manager, architect, and tech lead; there is no sudden changeover between feature implementation and bug fixing, as they are aspects of the same thing. And because the features in the tracking system really describe constraints and deliverables, engineers are still free to innovate like before. The bug tracking system gives him a place to capture his thinking, ask questions, and communicate about what he's doing beyond a very loose notion of "done" or "not done".

Full involvement of all parties

Feature owners, like architects and customers, must be sure that their features are being implemented properly. When an engineers says "done", there must be an agreement between him (code is written), QA (can begin testing, or there are no defects), and tech lead or architect (feature does what it should). By default, without a formal definition or process that clarifies the meaning of "done", it's difficult to communicate the real status of a feature. With BDD, the status of the feature is the status of the bug. In IssueZilla, bugs can be closed but then reopened if not completed satisfactorily, and there is a way to track why a bug has been reopened and its current status. When doing this verbally or in email, there is no guarantee that all parties agree or have even had notification of a change in status.

Better engineering and QA relationship

In traditional engineering organizations, the engineering/QA relationship is adversarial in nature, or tends to become that way, because the roles are distinct and responsibility is tightly bounded. Part of this comes from the fact that developers may feel caged by having to work on bugs instead of new features, and QA engineers may feel disenfranchised when voicing user concerns that are ignored.

BDD has the potential to remove the adversarial component from this relationship in two ways. First, bugs are part of the standard development process from the beginning, rather than coming in near the end. Developers have less reason to react negatively to issues filed there because they have been socialized to it from the beginning of the project, and they are accustomed to the idea of both missing features and defects as contributing to user dissatisfaction.

Second, QA engineers can be much more involved in the ongoing feature work because it's well-documented and there is a means by which to be heard. They can comment, add new features and requirements to the set, and even work on developing features rather than feeling powerless until a feature has been implemented. They can feel that their voice has been heard throughout the process, and so there is less reason for them to have to say, "I told you so" when something is implemented or designed poorly.

More user advocacy

Open source projects succeed (when they do) for many reasons, but few of these have much to do with the fact that the source is actually openly available. Making source available is one possible means to achieve a better product, but a product is not better because its source is available (a common refrain from open source advocates; in fact, one can easily argue exactly the opposite--open source success stories are actually few and far between, especially where user interface is concerned).

One of the properties that helps open source projects become successful is that developers and QA are generally one in the same; there is no formal QA organization. However, this is largely possible because developers of open source projects are almost invariably users of that project ("for us, by us", or FUBU). This works very well for software that is targeted at developers or technologists but doesn't work nearly as well for software targeted to a broader user base, which is the norm for commercial software ("by us, for you", or BUFY).

In these cases, formal QA organizations play the role of user advocates--they look for problems so that users don't have to. QA is a barometer of the health of a product. If QA is concerned about a product, then users will also be concerned. Where a formal QA organization exists (and I consider it an open question of whether it should), by bringing QA into the process from the very beginning, it brings many more people to bear on not just defects, but also lack of features and general user advocacy. In Linus Torvald's words, "Given enough eyeballs, all bugs are shallow." The same logic applies to missing features, also bugs in BDD terms: Given enough users, design flaws are obvious.

There are some problems in achieving this, however. In traditional engineering organizations, features are treated as unassailable truths, thereby relegating QA to the role of mere defect-finders. This is valuable to a point, but like someone restoring an aging, cracked painting, there are limitless flaws to be found if one looks closely enough. However, users don't generally care about the cracks and chips; they care about the overall picture.

Where the danger lies in treating features as golden is that it it gives QA justification to look for problems that, after a point, aren't really relevant to users. QA as a defect-finding mechanism can be very effective at finding problems, but that diminishes their ability to act as user advocates. However, in order for user advocacy to work, there must be a feedback loop to feature owners, and features cannot be treated as givens being passed down from on high.

By making features transparent to developers, QA, and management, and futhermore by giving a concrete means to discuss features and design openly before completion, BDD can offer a formal channel for user advocacy to flow into the design process, not only from QA but from anyone with access to the bug tracking system (docs, HIE, evangelists, marketing, product management, etc.). This helps achieve the same ends as open source projects that are highly user-focused, while working within the traditional engineering structure.


There are numerous advantages to the BDD approach:

  • Development transparency from the beginning of the development process. As features are completed, the bugs are closed. As defects are filed, they can be related back to the feature using the dependency mechanism, and the feature can be reopened as a visible indicator of status.
  • Issue and task tracking are automated. IssueZilla and other bug tracking systems have excellent methods for assigning bugs to developers in an easy-to-track way and notifying interested parties of changes in state.
  • Project status tracking. We can use a dashboard from the beginning of the development process to inform managers, architects, and other engineers about the status of the project on a day-by-day, hour-by-hour basis. The trick here is to develop this dashboard at the beginning of development rather than near the end.
  • Full organizational involvement. Engineering managers are good at making sure bugs get closed; the key to this is giving them something they can track. Furthermore, docs and QA need a way to track their deliverables as they relate to project features. Because anyone can either add themselves as interested parties to a bug, or can file their own bugs with dependencies, this becomes much easier.
  • Cross-organizational coordination. When working across teams and geos with large numbers of features and designs, it becomes imperative to track design decisions and status in a more formal way. Emails are easily lost, and if the right aliases are not used (and archived), then design decisions are all too easy to remain detached from other organizations.
  • Better team coordination. I-team meetings become bug status meetings at the beginning of development instead of just the end of development. QA and docs can easily track ongoing design decisions without always needing to be present at engineering design meetings. Engineering teams can lessen the burden of keeping functional specs up to date with the latest thinking (which seldom happens anyway) while the information is still preserved in a transparent way.
  • Better design tracking. All features have a unique bug number, and all discussion (or summary of discussion) can be added to one place for all to see. Design discussion will still occur face-to-face and over email, but relevant decisions can be recorded and then easily tracked in the bug.
  • Easy prioritization. All bug tracking systems have the concept of a priority which can be used to indicate the importance of a feature in the final product.
  • Efficient code reviews. The output of code reviews will be a mix of defects, featues, and RFEs, which can then be cross-referenced to features and other code review issues. This eliminates the common occurrence of going to a code review, making lots of comments, and then promptly forgetting them because there is no way to track them. It is left to an individual developer to track feedback from reviews, and generally it is assumed that the developer is diligent in this task, though this is seldom a guarantee. By tracking feedback as bugs assigned to a particular engineer, architects and managers can be sure that these are not being lost.
  • BDD is very effective when used in iterative development. For example, the umbrella bug for each iteration can contain all the tasks which need to be done before that iteration is complete. When a feature task is complete, QA can add blocking bugs on the iteration, so that the release iteration will be stable and closed, thus providing a usable set of functionality that has been tested.


At this point, BDD is being used by the Java Enterprise Tools organization at Sun for development of the NetBeans Enterprise Pack, with promising success. Chris Webster previously blogged about our use of BDD. The NetBeans team has used aspects of BDD for quite some time, but I don't know if it has ever been used in such a top-down manner as I've proposed. I am looking to Jesse Glick and Jaroslav Tulach for feedback on this proposal and anecdotal evidence of how well aspects of it have worked for the NetBeans team. I am also looking for feedback from various open source teams on how much or how little of this approach has been used on their projects. Again, my sense is that the top-down nature of requirements gathering and tracking using the bug tracking system has not been used to its full extent in these projects, though these projects are precisely where it would be exceptionally useful because of the overall disconnected and chaotic process under which they are developed.


Integration with Version Control

Before going open source, the Java Enterprise Tools team used something called the Integration Notice Facility (INF) to track CVS commits and relate them to bugs. The INF also provided the ability to indicate additional metadata for checkins, but that metadata was infrequently used.

The missing killer feature in implementing BDD is integration with the version control system so that commits are automatically tracked back to bugs (either features or defects) along with some metadata. Some tracking systems may have this, but others like IssueZilla require developers to manually paste CVS output into bugs. The manual approach is unwieldy, but very useful if adopted in conjunction with BDD, making it easy to track the change history of a feature and for feature owners to track commits in a consistent way. Therefore, I highly recommend this practice.

Additional Useful Features

In a bug tracking system used for BDD, these would desirable features:

  • Set a deadline for feature completion in a bug and be able to query based on time interval before deadline (<7 days, <3 days, 1 day, overdue)
  • Let bug assignee set a status value for percent complete once a bug is started
  • Construct bug dependency trees in a direct fashion. For example, be able to click a link on a bug to create a sub-bug with all the main details filled in. (IssueZilla is deficient in this regard.)
  • A separate axis of priority for blocking issues
  • Version control integration as described above

Friday Jan 05, 2007

Five things most people don’t know about Todd Fast

Andi Egloff and Trey Spiva have blog-tagged me; looks like I'm it! So, five things most people don't know about me, and may not want to:

1. I grew up near Dallas/Ft. Worth, Texas, USA. When I first moved to the Bay Area, everyone was surprised I didn't have a Texas accent for some reason. Two of my close colleagues are also from Texas. We like to compare boots, though I prefer Kenneth Cole over Tony Llama.

2. The first computer I owned was a Commodore 64, when I was 13 years old. I've always loved computer and console games, so I spent most of my spare time between 1983-84 programming games in BASIC and 6502 assembly. A few years ago, I bought a cable to hook up my old 1541 disk drive to my PC and I imported all my old C64 files as disk images, which I can now run in CCS64 and other Commodore 64 emulators. Pretty interesting to see what the hell I was programming at age 13.

3. When I was 16 years old, I burned most of the skin off my lower left leg and foot while screwing around with some extremely flammable chemicals I "borrowed" from the school's chemistry lab.

4. When I was in college at the University of Texas at Austin, I scored the music for a student film by Robert Rodriguez (director of Spy Kids, Sin City) called Bedhead. Previously, it was only available on the El Mariachi Special Edition DVD. By the way, "bedhead" is a slang term describing someone's messy hair after getting out of bed in the morning.

The short was made for Robert's Film 1 class at the UT Film School. As for my part, I was dating his film partner at the time, and she connected me with Robert. After they heard some of my music, they asked me to score the film. Don't be too hard on me; I only had one 8-bit synthesizer (an Ensoniq ESQ-1, which I still have and play) that could play a maximum of 8 voices at once, and couldn't do drums worth a damn. I worked on this over a long Thanksgiving holiday at my parents house, and then completed it in my dorm room with Robert the weekend after. I did it for free.

Two years after the short made its rounds to film festivals worldwide and won numerous awards, Robert signed a multi-movie deal with one of the big Hollywood studios. Not long after, he called me out of the blue and asked to come by my apartment. We talked for a bit until he asked me how much I'd charge to score another movie like Bedhead. I was humble and uninterested, but after he badgered me awhile I stupidly said, "Oh, 75 bucks.", to which he responded "A minute?". He wrote me a check for $700 bucks on the spot, and asked me to sign a release form (probably for the right to distribute the short, though I don't recall). At the time, I was happy to have such a windfall, but I don't want to think how much I could've actually hit him up for--I'm sure he was worth well over a million at that point, and now look at him.

Of all Robert's films, I like Bedhead the most. I've never really connected with his work on a personal level, though he's certainly talented. At the time it was made, it was a very surprising accomplishment for someone in his early 20s, especially for a first film, even at the UT film school, which was one of the best in the country. Now, YouTube and have shown just how unremarkable that sort of talent at that age has become.

5. In 1997, I inadvertently crashed all of Microsoft's web servers and brought down for a few hours. Yes, that Microsoft. And no, I didn't use that on my resume to Sun (but I should have).

Now for my five victims:

Tuesday Dec 19, 2006

Undermining Web 2.0

It's come to light that Google has shut down their SOAP-based search API. David Megginson has an interesting post on the impact of this:
Over on O’Reilly Radar, Brady Forrest mentioned that Google is shutting down its SOAP-based search API. Another victory for REST over WS-\*? Nope — Google doesn’t have a REST API to replace it. Instead, something much more important is happening, and it could be that REST, WS-\*, and the whole of open web data and mash-ups all end up on the losing side.
Many of the more interesting ideas around Web 2.0 and mashups assume that the data being mashed up will be available from somewhere, in a pure form that is consumable by the end user from the browser. So far, a few leaders like Google have made this possible, but there are strong economic incentives for others not to do this, as data is really the only "thing" of value anymore.

For example, Yahoo has a geocoding service (Google does not), but they have to pay a lot of money to buy that data from a provider and then make it free. Same goes for Google map data--they didn't generate that data, they just lease it from someone else. This is actually a pretty strong argument for server-side mashups (which may or may not be "enterprise mashups"), where partner agreements (i.e. restrictions on use and payment for service) can be enforced. I think in economic terms, we have a very long period ahead of us where only a very few large companies or governments will be able to or be interested in making large, useful databases available. There will be free alternatives, but like much of open source software, free data will be of lower quality than commercial versions.

To me, this issue is a major hurdle that undermines a lot of the interesting ideas that Web 2.0 visionaries (and zealots) are promoting. The socialist ideas underpinning open source software have been easy for corporations to accept because many of them recognize that the value is in something else besides the bits, like data, or usage, or support. To me, unless data is free, Web 2.0 is largely reduced to just some cool UI widgetry. Without data APIs, mashups, at least on the client, are not possible.

Now, having said all this, there is at least one alternative if data APIs are not available: screen scraping. This time-honored approach has taken on new life with Dapper, which in a nutshell is "Screen Scraping 2.0". While I'm guessing the Dapper guys are probably not happy about being referred to as screen-scrapers given their mottos like "It's like Legos" and "Unleashing the World's Creativity", that's effectively what they do, though in a way that's elegant and powerful, and geared precisely to Web 2.0 mashup use cases.

While technology like Dapper is very interesting, it has at least two limitations: first, I don't believe it does a good job of extracting data from sites with complex UIs (especially AJAX UIs), and second, armies of lawyers will probably sue them out of existence. If Google's strategy is really to replace direct data APIs with UI components which support their business model by serving advertising along with the data, it's unlikely that they will tolerate extraction of that data by third parties (in fact, their new terms of use explicitly forbid that). While I really like Dapper, and it may be useful for departmental or hobbyist use, I doubt it would be able to support Web-scale business.

Thursday Jul 20, 2006

Jury Duty - 12:03 PM - Excused!

The judge has come down to personally thank us for coming today. He told us that although none of us had been seen, we were definitely felt, and that because we were here, the attorneys in the case were working all morning to reach an agreement and have now come to a resolution. We are now excused with our duty complete, and I am now immune to jury service for 1 year.

It's been emotional.

Jury Duty - 11:51 AM - Good News?

The attendent just came on the speaker and said that she's waiting for instructions from the judges upstairs, but that she thinks she has good news for us in about 10 minutes. I assume that means that we will be released at lunchtime. Everyone is chattering happily now, as if the danger has passed. Fingers crossed!

Jury Duty - 11:25 AM - Pre-Lunch Anticipation

I'm here listening to Sirius satellite radio over WiFi on my laptop. You can feel the anticipation in the room that no one else will be called before lunch. Everyone is being quiet like they're trying to be as inconspicuous as possible.

Jury Duty - 10:51 AM - Break

The judges have given us a 20 minute break, though the cases are still being discussed.

Jury Duty - 9:55 AM - Courthouse Singles

The people in the room are very diverse, comprised of all ages and backgrounds. The makeup of the selectees seems skewed toward women when I glance across the room, but actually I think it's about 50/50 men/women when I consider it more carefully. My perception is more likely a result of working in an industry that is 95% men. It's not often I'm in a mixed crowd.

Because of the wide range of people, it occured to me that jury duty could be a good place to hold speed dating. I bet they'd get a lot more willing participants if they had something like a Courthouse Singles function as a side benefit of showing up. I can imagine the pitch, "Join Courthouse Singles today! Sure, there may be a 10% chance of serving on a jury, but there's a 20% chance of hooking up with a hot, fellow juror!" OK, 20% is a tad optimistic...

Jury Duty - 9:30 AM - Patriotic Video

They called people up to the front desk who have a "language problem". I assume that anyone that actually has such a thing would not really understand what that euphemism means. They also passed out vouchers for parking at the Performing Arts Garage on Grove street; my parking fee is only $5 for the whole day.

They started playing a video to pump us up about serving on a jury and to explain a little bit of the overall process. It's accompanied by lots of patriotic music and heartfelt testimonies.

I've been involved in a trial in the past, and it is true: it's a compelling and emotional experience. However, I have no wish to repeat that now, and I certainly doubt that a civil trial would be either of these.

Jury Duty - 8:30 AM

I've been summoned to report for jury duty this morning at San Francisco's Civil Court. I'll be honest--I'm not enthused about it. I don't mind doing my duty, but given that this is a civil court and it's a major conflict with my busy schedule at a critical time, I'm not happy about having this as (yet another) distraction.

I arrived at 8:15 AM, and grabbed a spot at the laptop desks along the wall. They provide numerous power plugs, phone ports, and wireless Internet. It's actually nicer here than most airport business lounges, just without the free food (which usually sucks anyway). There are about 100 people here now, all reading their papers, books, or magazines, or working on their laptops. Everyone seems quite calm, but there's little conversation.

Shortly after 8:30, about 6 people were called to follow someone to the next room. Sounds like they're starting at the end of the alphabet. If so, it bodes well for me. I'll let you know.




« July 2016