What's I Look For in a QA/QE Engineer: A Program Manager's Perspective
By melinchina on May 23, 2006
One of our recruiters asked me to write an article about what Program Managers look for in QE or QA engineers. It's gonna go into some recruitment magazine we're putting out. I was surprised to discover that I actually had a lot to write and I thought I should post it in my blog. It's a long read but hopefully helpful for anyone who's applying for a QE or QA job at a technology company and expects to be interviewed by a program manager.
A Program Manager's Perspective
As a Program Manager at Sun Microsystems I have worked in many capacities and managed many kinds of projects. I started as a Localization Program Manager then went on to be the Release Program Manager for one of Sun's newest and largest consolidations: the Java Enterprise System. Today I'm the chief-of-staff for Michael Hayden, the engineering director for the Operating Platforms Group here in Beijing. In all of these roles I worked closely with Quality Assurance Engineers and I've come to know what I can expect from a good one, at least from my perspective as a Program Manager. When I interview QE or QA engineers (for purposes of this article I'll refer to them collectively as QE) here's what I'm looking for.
QE engineers are the first customers of a product and they can be the strongest advocates for the end users. When QE engineers are writing their test plans, executing the tests, reporting results and following up on bug fixes, the customer's experience should be their main concern. They need to advocate for the customer at all times.
Good high-level communication
When a QE engineer escalates an issue to a Program Manager he or she should have a good high-level description of the issue and where he or she needs help, if at all. Here's an example of a bad way to approach a Program Manager when a QE engineer has encountered an issue:
QE engineer (comes into Program Manager's office, looking very frustrated): “I was just testing the build and I clicked on the 'send' button but when I sent the mail all the characters are garbled. Prasad said in the bug report that he fixed it but it's not fixed! What are we going to do?”
The following questions will start running through the Program Manager's mind: “What product are you testing? What build are you testing? Is that the latest build? What 'send' button are you talking about? Who is Prasad? Have you talked to Prasad about this? Are there any differences in his environment and yours? Are you able to verify fixes for any of the bugs that are supposedly fixed in this build? What can I do to help in this situation? How much longer do I have to work until I can retire....?”
Here's how that could have been better:
QE engineer (comes into Program Manager's office looking very calm and in control): “I've been verifying bug fixes this afternoon in the latest build of product X. Most of them are fixed, but I've got an issue with bug 637xxxx. The responsible engineer said in the bug report that it would be fixed in this build but I'm still seeing it. I'm going to talk to Prasad now to find out why he thinks it's fixed but I wanted to give you a heads-up that if this bug really isn't fixed, it might be cause for a re-spin. I'll get back to you in an hour or so.”
Here's what's running through the Program Manager's mind now: “I'm so glad this guy is in control of the situation. I know he'll come to me if it gets out of control. I'm gonna send good feedback to his manager when the project is over. I love this job.”
Good peer-to-peer communication
A good QE engineer has to have good written and spoken communication and at Sun we have so many global, remote teams that written communication becomes especially important. The importance of good written communication is obvious when a QE engineer writes a bug report. If the responsible engineer has to follow up with questions in order to reproduce the bug, the original bug report wasn't thorough or detailed enough. I won't try to give a class in QE 101 here, but everyone who has studied QE knows the kind of info that makes a bug report successful. The summary needs to be clear, the priority and severity need to be appropriate and justified, the steps to reproduce the bug must be included. Any details about other environments where the bug is not reproducible are also very helpful. Basically, remember what you learned in your QE classes at university and apply it on the job.
While written communication skills are obviously key, spoken communication is also crucial. QE engineers must be able to give updates at team meetings and at times they'll need to call other QE engineers or development to talk through a problem live. Without good spoken communication skills you can leave your audience more confused than you found them.
QE engineers don't need to be able to have philosophical debates in English but they do need to have a good English vocabulary and decent pronunciation. Don't worry about impressing people with your stellar vocabulary - stick to simple words and phrases and get to the point as quickly and efficiently as you can. Don't forget some of the most useful phrases that a member of a software development team will ever use:
“Sorry, I don't understand.”
“I need help.”
“It's under control.”
It's not enough to identify a bug; good QE engineers need to know how to get bugs resolved. Quite often this means you'll have to go to development engineers to help them understand the issue and to help them understand the importance or urgency of a fix. Never assume that the ball is in someone else's court and you can sit back and wait for the resolution. Program Managers notice who is moving things along in a project, and they make sure those people's managers also know.
An outstanding QE engineer knows the product they're testing, the best way to test it, and the best way to report the results. I've run across QE engineers who couldn't do simple calculations like tests passed/tests executed or tests executed/tests planned. These skills are what I like to call hard skills and they are something each QE engineer should bring with them to the job.
Ability to identify risks
A good QE engineer can identify risks early and identify ways to avoid or mitigate them. For example, if a test team is going to need a certain kind of hardware for a particular release, that information needs to come to the project team and the Program Manager at the very beginning of the release when there's still time to order and set up the hardware. If a test team highlights too late that they're lacking hardware or resources, there's generally no way to avoid the risk. The best the team can do is try to mitigate it.
I've heard QE engineers say, “This bug just has to be fixed” however they can't explain why the fix is so critical and they aren't willing to talk about alternate solutions like implementing a workaround or releasing a patch or documenting something in the release notes. None of these solutions are perfect but there is also no perfect software. Good QE engineers know how to partner with the Program Manager and the rest of the team to find the best solution for the customer, they don't stonewall or shut down when their opinion isn't taken at face value.
In summary, I hope these points will be useful to you as you search for a job as a QE engineer. And if your search brings you to Sun I look forward to working with you!