Friday Feb 22, 2008

More = Better ?

I had a discussion yesterday that touched briefly upon how agile our project is (or is not). The answer was that we're probably more agile than some other projects I've been on, but that we could be more agile. The underlying assumption was that things would be better if we were more agile. I think this assumption happens to be true, but it's not fundamentally true. Let me explain.

A former manager once said to me that he thought I was process-oriented. My response was, quite emphatically, "I'm not process-oriented; I'm results-oriented!" (I swear this is true. It just popped out.) The reason I'm a proponent of agile techniques is that I think we can achieve better results using them. Otherwise, what would be the point? If we made some change that was more agile, but our results didn't improve, there wouldn't be any benefit. Or if we made some change that did improve our results, I'd be in favor of it regardless of whether or it's considered "agile".

Note also that I'm using "results" in a very generalized way. For instance, if the team were to deliver the same product, feature set, quality, etc. but with less overtime and stress, and more time spent relaxing with families, and so forth, I'd consider that to be an improved result.

The default in my area seems to be for projects to be very plan-driven. Frankly, I don't think they've worked well. For this reason I've looked to alternatives, including many agile techniques. But the point isn't to be agile for the sake of being agile; it's to get better results. Let's make sure we don't forget this.

Monday May 14, 2007

Retrospectives and Regret

David Carlton mentioned the Retrospective Prime Directive. I think the prime directive is a great idea. A retrospective is not a place to deal with firing of slackers or whatever. If somebody is truly neglecting their job, this problem should be dealt with through other means.

I've found that many developers are defensive about their work, and deny that they've written a bug or made a mistake. They reject constructive criticism, etc. for fear that admitting they've done something wrong will be held against them. This attitude gets in the way of learning and improvement. So, in a retrospective, one of the key values needs to be a sense of personal safety, so that people don't fear being blamed for something. This lets the group focus on improving itself.

I'm not sure that "regret" is the right word. To me, "regret" means "I wish I hadn't done that." It has a very negative connotation. Perhaps "critical self-examination" is more descriptive term, though it's also more verbose. If you had a chance to go back and do it all over again, would you do it differently? If the answer is "yes" then that should be a positive: it means you learned something. Capturing what you've learned and using it to improve yourself and your group is the whole point of a retrospective. If you haven't learned anything, then something is seriously wrong.

Friday Mar 23, 2007

Mike Cohn at bayXP

I attended Mike Cohn's talk at bayXP on Tuesday (March 20, 2007) entitled "Planning and Tracking Agile Projects." Mike is quite open about all the stuff he presents. His presentation is available at along with a bunch of other stuff.

This wasn't so much a presentation as a workshop. A couple times Mike asked the audience to form small groups and perform some planning activities. In one instance he passed out decks of "planning poker" cards and had us play planning poker with (that is, give estimates for) various made-up tasks. The dynamics of planning poker are quite interesting. It really helps you flush out unstated assumptions and disagreements about what a task involves.

The event was hosted at Google, which provided excellent food (hummus, baba ghanoush, pita, spinach in phyllo, falafel, etc. plus an excellent selection of beer, wine, and soft drinks). Combined with Mike's excellent talk, and the low, low price of admission -- $0! -- this evening was an exceptional value.

Thanks to Mike, the bayXP organizers, and Google.

Tuesday Mar 13, 2007

Is The Triple Constraint Agile?

Let's revisit this "triple constraint" business I blogged about in my previous blog entry. What's this doing in the "Agile" category? Isn't the triple constraint that thing from the Project Management Body of Knowledge (PMBOK), which is about classical non-agile waterfall stuff? In a word, no. I don't know what much about the PMBOK, but I believe the triple constraint applies to any project, regardless of how it's planned.

The triple constraint is a model for doing reasoning about planning. The style in which planning is done, whether waterfall or agile or something else, doesn't affect the fundamentals of the model. The style certainly affects the outcome of the plan, but that's a different story.

Consider a waterfall, plan-everything-then-execute approach for managing a project. The triple constraint certainly applies here. You're given a certain resource (cost) budget, a desired scope for what to deliver, and a target date for delivery. It's usually the case that you develop time estimates for everything and the end date that pops out is far later than the target date. So you have to add resources or cut features to pull in the date. You have to make the tradeoffs the model implies before you can formulate a credible plan for delivering. Then (theoretically) you execute according to plan.

Now consider an agile process. It doesn't matter which, really, since they pretty much all use an incremental planning model. During each planning cycle, you have to make the same tradeoffs among cost, scope, and time. The difference is that you do this incrementally instead of all up-front. For example, consider an urgent customer request that comes in the midst of a release cycle. At the next iteration planning meeting, you reshuffle the project backlog to put the customer's requests at the top, which pushes everything else down. This implies that you can deliver the original set of features at a later date, or you can deliver a reduced set of features at the same date as originally planned: a classic triple constraint tradeoff.

The key point is that incremental planning provides a structure for making these tradeoffs on a regular basis. By contrast, in a waterfall approach, making changes of this sort is viewed as a project planning error: you have to replan everything. This example embodies the line from the Agile Manifesto, "Responding to change over following a plan." The alternative is not to make the change at all, which gives rise to the idea that waterfall approaches tend to resist change.

Agile planning approaches embrace the triple constraint wholeheartedly as a normal part of the planning process, whereas waterfall approaches make all the tradeoffs up front and hope they stick.

Wednesday Mar 07, 2007

The Triple Constraint

Here's a bit of elementary project management: the triple constraint. This was explained to me by an experienced project manager colleague some time ago, who drew a diagram similar to the one below on my whiteboard.

The "Q" in the middle stands for quality. The point is that the variables are interrelated: changing one affects both of the others. If you don't account for the changes to the others, you "break the triangle" and it's quality that suffers.

Now I don't really have any formal project management experience, but I've been on a lot of projects. I've been on projects where the end date and the size of the team ("cost") were fixed, but scope was increased. And what do you know, when we got to the end date, there was a huge pile of bugs. When my colleague presented this model to me it seemed blindingly obvious. Yet, it seems that many people just don't get it.

(Some literature considers "triple constraint" to be an obsolete term, preferring instead to consider cost, quality, scope, and time as a set of four variables. This goes against the Received Wisdom of Quality, which states that Quality shall not be a variable. But I digress.)

Upon further reflection, it's not that people don't "get" the triple constraint. The real problem is that people are unwilling to confront the results of thinking through the model.

Now, like any model, there are some qualifications. I'm thinking of two in particular: "in the long run" and "all other things being equal." I'll cover these in separate blog posts.

Thursday Aug 17, 2006

Tests have a better memory than you do

My past several blog entries have been about testing, more precisely about unit testing or test-driven development. I suppose I should talk about how I got started on all of this.

Up until few years ago, I developed code in a "conventional" style, that is, without unit tests. I kept hearing about unit testing and test-driven development but I didn't know what to make of it. I read a couple articles and it seemed sorta weird, but I didn't really catch on. But, as my experience shows, you can't catch on to something just by reading about it. You have to do something and experience it.

So, I decided that when I next had a couple spare hours free (naturally, this was a major source of delay) I assigned myself a simple programming task. The actual task isn't relevant, but it was simple enough that I could implement something of interesting complexity within a few minutes and then add features incrementally.

But I did eventually get the time to do this. I downloaded JUnit from and followed the procedure of writing the test for the code before writing the actual code. I felt really stupid at first. I couldn't even compile the test, since I hadn't written any code yet, not even a stub! But I kept going: I added the code, and then the test passed. I added another test (which failed, of course) and then I added the code to make the test pass. And so forth. I even started taking shortcuts by writing the test and the code at the same time. It's not strictly by the book, but what-the-heck. After a while, I had a couple hundred lines of code and a dozen or so unit tests. Things were going pretty smoothly.

But then something really interesting happened. I had implemented features A, B, C, and D and their corresponding unit tests. I added a test for feature E and then code to implement feature E. I ran the tests, and the test for the new feature E passed. But the test for feature B failed!

This came as quite a surprise to me. After programming for only a couple hours, and with only a couple hundred lines of code, my unit tests knew more about my code than I did. When I implemented feature E, I thought it would work -- and it did -- but I also thought that it wouldn't break anything else. I was wrong. It just goes to show that your unit tests have a better memory than you do.

I've been test infected ever since.

Wednesday May 03, 2006

Tests Embody the Results of Thinking

I was talking to one of the engineers on my team the other day about bugfixing and unit tests.

We've reached the bugfixing stage in our project. To fix a bug, one must do a lot of thinking about the case that fails. Clearly, one must change the behavior in a particular case from incorrect to correct. But one must also ensure that the correct behaviors that occur in all the other cases are preserved. So, the goal isn't just to make some test change from fail to pass; it's to make sure that all the cases are handled correctly at the same time.

It's surprising how often people forget this. Sometimes a developer will make a change, and if the failing test succeeds, they think they're done. If they haven't ensured that all of the other cases are still correct, they're being irresponsible. (A greater irresponsibility is with managers who don't understand this.)

When dealing with code -- whether writing it initially or fixing bugs in it later -- a developer has to do a tremendous amount of reasoning, analysis, and thinking about all the cases the code has to handle. Unfortunately, the results of this thinking are rarely written down anywhere.

Sometimes these results are written into design documents. These are usually separate from the code, they get out of date, and nobody reads them anyway. Sometimes these results are written into comments. People sometimes reads the comments, but they're so often out of date that they aren't to be believed anyway.

“When the comments and the code differ, both are probably wrong.” -- attributed to Dennis Ritchie

The best place to write down these results is in unit tests. A good suite of unit tests acts as a reviewer who checks all the "other" cases for you automatically. If you run the unit tests all the time, they don't get out of date, either.

Next time you're reading a piece of code and you gain a significant understanding of it, don't throw away that understanding. Write it down in the form of a unit test.

Wednesday Apr 19, 2006

Insights from Bob Martin's Agile Toolkit Podcast

Thanks to David Carlton for posting a link to the first Agile toolkit podcast in one of his comments.

In this podcast, Bob Martin restated some of the critical XP practices, from which the other practices will tend to follow. I thought this was OK but not particularly enlightening. I did find some interesting insights in some other comments that Martin threw in offhandedly.

First, there's the statement that the purpose of testing is to determine when we're done, not to find bugs. This probably isn't original, but it's an important point. I constantly run across people who think that the purpose of testing is to find bugs.

Second, there's the notion that there is a QA group whose jobs is to be specifiers for acceptance tests, instead of executing tests on the finished product. I hadn't heard this before. This seems to work well if you're using something like FIT for acceptance tests. I'm not sure how one would apply this to platform software products (which is what I work on). Something to think about.

Finally, there was the idea that QA can get ahead of developers in mature organizations, getting the tests ready in advance of the product being ready. This is TDD in the large, and it makes sense. Unfortunately I've never seen this in practice. It sounds like something worthwhile to strive for, however.

Saturday Apr 15, 2006

Increase the Ratio of Thought to Lines of Code

More on pair programming...

Nearly two years ago, Rich Berlin blogged some observations about pair programming. He mentioned an idea that he and I came up with: Increase the ratio of thought to lines of code.

Turns out that a similar piece of paper from that meeting I had with Rich is still posted outside my office. I think the meeting was all the way back in 2001. It wasn't actually that much of a meeting, as I recall, it was more of a whiteboard conversation. But it did result in a few insights that I thought were worth writing down.

One topic in particular is the number of bugs programmers write. Dijkstra observed that “If debugging is the process of removing bugs, then programming must be the process of putting them in.” My observation is that programmers do write lots of bugs, most of which are easily preventable. <lament>If only programmers were more careful!</lament>

This is really a counsel of perfection. Simply telling programmers to be more careful or to write fewer bugs is ineffective. One way to let programmers be more careful is to take some pressure off and not overload them. Programmers under pressure often subconsciously or willfully ignore potential issues, since they are apparently obstacles to completing the task at hand. With less pressure, programmers can take more time to think about their code and to investigate unresolved issues instead of just checking it in and moving to the next task.

But that doesn't mean they will take more time to think about their code. That's where pair programming comes in.

One aspect of pairing is keeping each other honest. A senior engineer paired with a junior engineer can instill greater discipline in the junior engineer, encouraging him or her to fix the problems now instead of postponing them until later. It works the other way, too. As a senior engineer, if I know a junior engineer is watching me, I feel compelled to set a good example. I'm always on my best behavior. (And boy, is that tiring.)

Another aspect of pairing is, as Rich said, “Two heads are better than one.” Two people will be looking at different things. The union of problems they'll find is greater than either one of them alone. This applies for senior-junior pairing or pairing between peers. Nobody is so perfect that they never make mistakes.

There are a few other topics from that discussion with Rich that I'll save for future blog entries. Stay tuned.

Friday Apr 14, 2006

Let's Look at the Code

Like many programmers, when I first heard of pair programming (one of the eXtreme Programming practices) I thought “Ewwww!”

Pair programming can be difficult and tiring. We don't do XP in my project and we're not likely to start soon. However, pair programming can be extremely effective in certain cases, such as for teaching and mentoring, for diagnosing obscure and difficult bugs, or working through really hairy code.

But how can you get people to start pair programming if they don't want to? You pretty much can't dictate “henceforth, everyone must start pair programming.” Well, you can dictate, but people won't follow. Or if they follow, they'll do so in a reluctant, grumbling fashion. Not the best way to improve productivity.

Yet I do find pair programming valuable in many circumstances, and people often don't pair program even when it's useful to do so. How to start?

I'm a senior engineer, and engineers on the team often drop by my office to ask questions about how to solve some problem they're working on. It's part of my job. I used to answer questions by drawing on the whiteboard or by waving my hands and describing how I thought a solution would look. More recently, however, I'll say something like “Hmm, well, let's go look at the code.”

I'll then wander over to the engineer's office and ask them to show me what's going on in the code. They'll pull up source files and start showing them to me in an editor. Pretty soon we'll run across some code that's not behaving quite right. We'll talk a little bit about how the code should be behave, and what changes need to be made to fix it, and so forth. I'll say, “Well, why don't you check out the file and we'll make those changes right now?” And we do. And pretty soon, we're pair programming.

Some time ago we had started advocating pair programming in my team, but nobody was really doing it. At one point I used this technique on one of the engineers. At the end our the session, I asked her, “Did you realize we've just been pair programming for the last hour?” The grin on her face was priceless.

So maybe we should pair program but not say we're pair programming. Just say, “Let's look at the code.”




« July 2016