Scorecarding & Tollgating: Is It Safe?
By asparks on Sep 29, 2008
Is it safe? At many points in a project you (as a professional Project Manager), or the customer, will be asking if it is safe to move to the next phase of the project phase (or to go live).
How do you know the answer? How do you convince the customer? Sometimes the situation might feel like this scene from a classic film.
In this entry I will look at how simple approaches to scorecarding and tollgating can help you answer these two basic questions and improve your results in project phase reviews.
I think it is worth reviewing why scorecarding and tollgating are worthy of a fresh look.
First and foremost Oracle programs & projects are getting more complex as our applications product portfolio broadens. In the meantime the methodologies are always going to be playing catch-up. So, frequently control & reporting often have to be developed from first principles rather than relying on a methodology to provide a cut and dried solution.
Secondly, today's business environment places more emphasis on compliance and auditability. So documentary evidence to support project phase reviews is becoming a must.
Let me define my terms for the sake of clarity.
Scorecarding: A structured project progress/risk/issue reporting using simple scorecard concepts
Tollgating: A formal project phase review & control process. Entry to the following phase is gated by formal review and acceptance (by the business) based on objective criteria. This process is supported by scorecards. Note for non-English speakers: traditionally a tollgate is the barrier where you have to stop to pay money (the toll) before you are allowed to proceed on a road or a bridge.
In this article I am addressing the role of the Project Manager. Where I mention “the customer” or “the business” the role of the PM can be either as an external consultant PM or an internal – the customer concept still remains valid as the party who has requested the project and will receive the business benefits from it.
Context: Dentist or Personal Trainer?
In looking at scorecarding and tollgating project phase reviews I think the first place to start is for you to consider the context of your role as Project Manager and your attitude to it.
What is your attitude regarding your assignment with the customer? How does the customer perceive it versus how you perceive and act on it?
Do you play the PM role like a Dentist or Personal Trainer?
Frequently the customer expects that we take on a role of a Dentist. We’ll do all the work and the customer just has to lie back and get a bit of corporate root canal work performed on them. The underlying assumption here is that their effort will be low – after all that is what they are buying our services for…Our own view of the role more often resembles that of the Personal Trainer. The customer has a (very) active role in doing the project work together with our project team. The key is we guide and direct the activity – but the customer has to put in the kilometers.
When it comes to project phase reviews understanding the role you are adopting is key to understanding your own behavior during the review.
As a Dentist you may feel you have complete control over the situation, but examining true readiness to move forward may be clouded by your own vested interest in success and the customer (or patient?) can opt out: after all, they were just getting an operation performed on them, right? In this situation phase reviews can feel very stressful as if you are being placed in the dentist’s chair and being subjected to a thorough examination….Is it safe?
As a Personal Trainer, you are much more able to keep objective focus on the customer’s performance in the phase review rather than focus on your own performance..
To keep an objective focus we need to build up a body of objective evidence and review this in a formal way.
Tollgates: Hard Edges, Firm Centers
Having the right attitude to your role of PM is just the start. You need to put hard edges around your project phases with formal phase reviews or tollgates.
- Set the dates of the tollgate with the customer and agree in advance who is expected to attend.
- Don’t move to the next phase without an explicit “Go” decision
Give the tollgate process a firm center with score cards.
- Use structured status scorecards for key deliverables
- Explicitly set expectation on deliverable availability for review before the tollgate (don’t only rely on the scorecard to run the tollgate). For example specify that all deliverables for review must be available and frozen at least 48 hours before the formal formal tollgate meeting. Last minute fudging is a time honored tradition in Project Management. It is not a good one, however.
- Explicitly set expectation & enforce business participation and signoff. In other words do not accept that IT folks can sign off on behalf of the business.
These two elements are components of your overall governance framework. They are all inter-dependent. There is no point in formalizing the project review process with tollgates if you do not also structure the content that you will review. Conversely, there is no point structuring the content (deliverables and scorecards) if you do not set a formal schedule with the customer to review the status and make a decision based on that review (a tollgate). The two elements complement each other.
One detail point is what to do if you get a No Go tollgate decision on moving forward.
- It is always good practice to build some contingency time into the schedule in order to allow for issue resolution and a rerun of the tollgate without major schedule impacts.
- If the problems are deep enough you can also retain the option to agree to proceed to the next test event – but confirm that the next event cannot be considered as a deliverable for the next phase: the project still needs to formally complete the current phase. For example – you get a Non Go on the CRP phase and a UAT is scheduled for the following week. The event can be allowed to proceed – but it cannot be considered as a UAT for the purpose of being a key deliverable in the next (UAT) phase. The CRP is not closed and the project is not in UAT yet.
Ruthlessly Prioritize & Simplify: Less is More
Many professional PMs are great at managing detail and the nuts and bolts elements of project. This can be both a strength and weakness in project phase reviews.
In many cases more information or detail does NOT make the situation clearer or change the basic truth (that the customer is/is not ready to move to the next phase).
In order to drive clear focused decisions, particularly with regard to phase reviews, you need to ruthlessly prioritize and simplify the information.
In your scorecards and reports simplify by layout and visual keys.
- Be binary (Done/Not Done) about (deliverable) status
- Be tertiary (Red/Amber/Green) about phase status and issues (but binary is better)
- Combine colour and letter coding. Use conditional formatting in Excel )for example) to get the colour coding automatic. But also use the letters (R/G and maybe Y) to drive this (remember – colour prints are expensive and you’ll normally print in black and white!)
Use other governance elements (e.g. your core team) to protect against “get-there-itis”
“Get-there-itis” is a phenomenon commonly noted in the airline industry. Pilots may be so intent on meeting their flight schedule for takeoff or arrival (and under management pressure to do so) that they ignore other factors such as bad weather of heavy traffic in order to “get there”. This can have literally fatal consequences for all involved if critical elements are overlooked and they crash the aircraft.
Similarly in project management situations. As PM’s we can be so under pressure that we ignore warning signals in our drive to “get there”.
Other governance elements can help to address this:
- A climate of brutal truth with in the project (governance) organization. Empower other team members to challenge you to acknowledge warning signals
- Broad access and group review of deliverables and scorecards and the knowledge that they will be scrutinized can avoid “window dressing” – putting a nice front on the deliverables without having the core decisions made
Drive Ownership by Simplification: Rule by Excel and PowerPoint
The best tool for the job of project progress reporting is often one the customer can understand and use themselves.
Simpler scorecarding or reporting with commonly used tools encourages customer (i.e. the business, not IT) ownership. This is a good thing.
MS Project and other tools (e.g. Collabnet, Test Director) are great for the PMs & IT guys but don’t expect the customer’s business folk to love or even understand them. So while these technologies make put all of the right information at your fingertips (as PM), you may still need to rework it for customer consumption. Sorry, there are no shortcuts.
Insist on Consistency: Build Your Own Safety Net
Iron discipline in maintaining project scorecards builds up objective data to support the tollgate. By the way, this never feels comfortable particularly for newer PMs. But being a nag about this pays off.
The scorecard evidence you build up as the project progresses can hold up a mirror both to yourself as PM and to the customer to help you deal with the reality of the situation rather than relying on your own personal perception. In this respect that scorecard evidence can act as a safety net.
Knowing that you have a body of relatively objective evidence on project progress and status helps you keep touch with reality in the project, and in project management reality is always the safest (if not always the most comfortable) place to be.
In large rollout programs consistent use of scorecards & tollages has two additional positive effects.
Over time you build up common language & understanding on pass/fail criteria & tolerances in larger group of participants as you run though multiple repetitions of tollgates/phase reviews together. This contributes greater understanding and ownership of the tollgating process both in your program organization and with your customer.
The increased experience pool also can improve the consistency of execution and results of phase reviews as positive and negative pattern detection improve.
Referring back to the original questions “Is is safe?”, “How do we know?” and “How do we convince the customer?” it is clear that formalizing the project phase review process is key.
Formalizing the phase review process puts hard start/stop edges around each phase. You also have a firm center of content based on well defined scorecard of milestone and deliverables status.
The scorecards also help you ruthlessly prioritize (not filter!) the information that you are asking the business to make a decision on. Remember: too much information and detail often results in poorer decisions (or none at all) rather than better ones.
The technology you use for project scorecarding also can drive customer ownership to a degree. Technology that helps you as a PM may not help the business own the issues when they are not comfortable using the same tools as you. Simplify tools to encourage the customers people to use them not just the IT folks.
But at the end of the day consistent use of the scorecards in a framework of project phase tollgate reviews really can be used as a lens to get a clear view of the true state of the project. The objective evidence you build up with the scorecards then means that a decision to move to the next project phase (or not!) is based on a common understanding of readiness that is supported by data. In this case you are no longer having to sell the arguments to move forward or not based on your own opinion.
This can help make phase review meetings feel a little less like a visit to the dentist.