Thursday Apr 17, 2008

Data Driven Personas

A week ago, I was in Florence Italy, giving a talk on a paper that a colleague and I wrote. The paper was a CHI Note, which described a new way to create "personas": fictional characters who represent user groups who will use the products that we develop. It's a lot like creating user profiles, but personas have names, photos, and other details added to them, so that designers and developers can meet the needs of, say, "Sarah" rather than a nameless, faceless "system administrator".

The process for creating personas has been around for about ten years, and usually starts by brainstorming user characteristics. Ideally, the details of the personas are drawn from facts that have been gathered through user contact. Sometimes, once the personas have been created, they are then validated, by conducting a survey or focus groups or by communicating with representative users in some way. Unfortunately, there are several drawbacks to the traditional method, usually revolving around trust in the accuracy of the personas and acceptance by the team that is supposed to use them. But it's a numbers game, too: if you interview 10 or 15 or 20 users, how can you ensure that they are truly representative of the larger population?

My partner in this paper, Nalini Kotamraju, is a user researcher at Sun, and it was she who initially proposed that we turn the traditional method around: rather than interview users first, and then validate the personas through a survey, why not conduct the survey first, and then interview them? Brilliant! Not only would the personas be an artifact of the data, but we'd have statistically significant numbers of users to base them on! Then, when we conducted the user interviews to add details to the personas, they would be "the right" (read that as "representative") 20 or 30 users, because we'd have done the stats first.

Our new persona creation process went well, but there were some surprises along the way... perfect for a paper submission :) Fast forward a little more than a year, and there I was in lovely Firenze, sharing our work with a receptive group. My favorite part was at the end, after the talk was over, when I was able to share more in-depth details with friends and colleagues. I even got a question and compliment from John Pruitt! But there were a lot of friends and colleagues who weren't there, so if you're one of them, I'll be giving another talk in May :)

Jen McGinn is an interaction designer in xDesign who is working to improve the user experience with software registration and Solaris system administration. She has an MS in Human Factors in Information Design and works out of Sun's campus in Massachusetts.

Thursday Apr 10, 2008

Calum's CHI Photoblog - Part 4


(My cold wasn't too bad today, as it turned out... throat much better, somewhat bunged-up otherwise but better that than having a runny nose all day, IMHO...)

Started the day at the Character Development session where Jen was presenting her data-driven persona talk—good to meet her in person at last! Also bumped into a few old (and a couple of new) faces there, including Robin Jeffries, now at Google, and Matthias Müller-Prove, with whom I co-authored a paper and presentation for CHI in Vienna.

Unfortunately I was paying so much attention to Jen's presenation that I forgot to take a photo, so you'll have to make do with this out-of-focus, mostly-people's-backs effort of her (left) chatting to Andrea Knight from Google, Christoph Noack from Bosch, and Robin (mostly obscured by Christoph):

Jen McGinn (Sun), Andrea Knight (Google), Christoph Noack (Bosch), Robin Jeffries (Google, obscured)

Another of the talks, on identifying personas using latent semantic analysis (essentially looking for similarities in the textual patterns of questionnaires completed by the subjects), was interesting too, although some of the resulting groupings presented in their example seemed a little dubious. Probably another one I ought to go and read the paper for...

I ducked out of the session after Jen's presentation to see Evangeline Haughney's talk on how she's used comic strips at Adobe to present user interview findings. This is obviously something we've done some work on at Sun as well, so it was interesting to see how her approach differed from ours. The most obvious difference to me was that she didn't use the traditional chronological comic strip format, but an information-rich montage of comic strip elements on each page. And also that she delivered it to the stakeholders as a printed comic book, rather than just in an email or on a website. (I really want to try this on one of my projects soon!)

Next up was the Collaboration and Cooperation session, which included a couple of talks on "co-located collaborative web searching" (aka "one person Googling while one or more others look over their shoulder trying to help") . One paper described a novel solution where the 'observers' who aren't sitting at the keyboard can contribute by suggesting search terms to the 'driver' via their bluetooth phone, on which they can also browse results pages independently of the other participants.

I went for lunch with Christoph Noack from Bosch, who's part of the OpenOffice user experience team, and Mike Terry, Ed Lank and Christine Szentgyorgyi from the University of Waterloo in Ontario—Mike gave the InGimp talk I attended on Tuesday. We chatted about his plans for analysis on the data collected via InGimp, whether the instrumentation might be abstractable to GNOME applications (since GIMP and GNOME are both built on gtk+), and about open source usability in general.

(Having spent some time trying to remember where he'd heard my name before, Mike suddenly pulled out his laptop halfway through lunch to show me a paper he was writing that referenced one of mine!)

Got back just in time to hear a case study from Intuit on the advantages (and pitfalls) of using users' real-life data, such as bank and tax details, in their usability tests. They cited a couple of examples where features that had been readily discovered by participants in 'fake data' tests were missed when users worked with their real data instead. To counter that, they also gave the example of the user associating a memory-jogging phrase with certain data to aid recollection months or years down the road—it was impossible to test the effectiveness of that feature, because the user naturally had to choose the phrase just before the test.

So to Bill Buxton's closing plenary, where he spoke about "not what can we do, but what should we do" with the expertise that the CHI community has. He showed how how great design borrows from and extends the great designs of the past, and how a product's development needs to include a consideration of the social and cultural implications of its success, rather than evaluating it in isolation. A suitably thought-provoking finale.

Bill Buxton

It's been a busy week, with some interesting sessions and some not so interesting ones, as always. Was good to meet up with the Sun folks, who I don't see in person very often, and I was particularly glad to meet some other proponents of open source usability—there still aren't many of us around! It's probably fair to say the only real disappointment this week has been the weather...

And with that, I'm back off to Dublin late tomorrow afternoon to catch the last few hours of my wife's birthday, so I guess I'd better try to find her something nice here in the morning :)

Wednesday Apr 09, 2008

Calum's CHI Photoblog - Part 3


Woke up this morning with a very sore throat and all the other tell-tale signs of an impending nasty cold. Still made it to my full compliment of sessions today, but might struggle a bit tomorrow... hope not though as at the very least I want to go along to Jen's talk!

Started today by attending the "25th birthday party" of The Psychology of Human Computer Interaction, the seminal book by Stuart Card, Thomas Moran and Allen Newell that more or less coined HCI as a phrase. Stu and Tom were on hand to talk about how the book came about (Allen having passed away in 1992), and a panel discussion led by Bonnie John reflected on the progress we've made since then with the likes of Soar, ACT-R, and various GOMS derivatives.

Stuart Card

My other session before lunch was Am I Safe?, where three papers were presented. The first looked at a novel way of visualising firewall alerts (traditionally jargon-heavy popups presented by applications like Zone Alarm), which amongst other things drew a connecting line between the window of the process generating the alert to a world map showing the location and details of the server generating the request. A novel idea, although the visualisation still looked over-complex to me and could use some work.

The next speaker presented a system that drew graffiti on the user's desktop when it determined that one of their installed applications needed updating to patch a security vulnerability—the more prominent the graffiti, the more serious the vulnerability. Hopefully, we wouldn't have quite as much use for that on Solaris as the Windows users in the study seemed to have :)

Finally, we heard from a team who investigated the effectiveness of the "phishing" alerts generated by IE7 and Firefox 2. Solaris users will be glad to know that none of the Firefox users in the study were fooled into giving their details to a phishing site set up for the study—many of the IE users did though!

I went poster-perusing again at lunchtime (bumping into former Sun colleague Nancy Frishberg in the process). Here are a couple that caught my eye today—click for the legible version:

Poster-3 Poster-2 Poster

In the afternoon, I went to:

  • Branding the Feel: Applying Standards to Enable a Uniform User Experience: A panel from Adobe, Microsoft, Google and SAP talked about the process of writing user interface guidelines, and providing supporting materials, in the context of a corporation for whom their user interface is part of their brand. Thought I might come out of it with some insights that might be useful in the many branding discussions we have at Sun, but in fact I came away with more ideas about refreshing the GNOME Human Interface Guidelines (an exercise which is long overdue at this point!)
  • Menu and Command Selection: First, a chap from Autodesk presented their "PieCursor" concept—a new take on tracking pie menus in which the cursor is the menu (or more specifically, the tool palette, to which it's really more suited). Their results showed it to be more effective than using a toolbar, but little mention was made of the fact that they also seemed to suggest it was less effective than their existing pie menu. (Unfortunately I didn't get a chance to ask why not...)

    Next up was a presentation from a Chinese team about their "tilt menu", in which the user summons a pie menu by tilting the pen or stylus (or it may be shown automatically, if the context demands). They select from the menu by tilting the pen further in the direction of the appropriate menu item. It looked like it might be a little awkward to use, to me, and as you might intuitively expect, their study showed that some segments around the pie were easier to select than others.

    Adaptive activation area menus were next, attempting to address the 'steering problem' of cascaded menus by manipulating the 'hot zone' for accessing the submenu once the pointer landed on a parent menu. I could be wrong, but the basic method described (two other variations were also shown) was awfully similar to what I thought GNOME already did, but I wasn't sure enough to raise that point at the time :)

    Finally, an algorithm was presented for automatically improving the layout of hierarchical menus based on decision time, item category and frequency of use. The speaker was pushed for time, and was not presenting in his native language, so it was not entirely clear to me whether this was intended to be used during the design phase, or on the fly on a per-user basis in the real product. Think I'll have to go and read that paper tonight...

Tuesday Apr 08, 2008

Calum's CHI Photoblog - Part 2


Okay, slight confession—I skipped the first session this morning to go shopping for some essentials that I forgot to pack! I did take a few snaps on the way though, and since I didn't take any other conference-related photos today, I at least ought to show you some of those instead.

Copy of Michelangelo's David, and Hercules and Cacus, Piazza della Signoria

Ponte Vecchio

Santa Maria Novella

So to the sessions I did attend:

  • Beyond end-user programming. I particularly wanted to go to this session because of the InGimp talk, to hear how Michael Terry et al. instrumented the open source GIMP image editor (available on a Solaris desktop near you) in such a way that all the information collected is publicly available. Since the GIMP uses the same widget toolkit (gtk) as the rest of the GNOME desktop, I was interested to hear how easy it might be to abstract this technique to instrument any GNOME application, perhaps even making it a standard feature (for development releases at least). I did specifically ask this question, but received a fairly non-committal response :) I'd still like to follow up on this later, though.
  • Meta-CHI was part of this year's alt.chi programme (a track causing some controversy in itself), and the three talks here concerned how to improve the way we do 'interaction criticism', a re-appraisal of the three classic principles of 'early focus on users and tasks, empirical measurement, and iterative design', and a comparison of the way that quality criteria are applied in the fields of science and design. As with the opening plenary, all a bit introspective and academic for my tastes, but at least the clue was in the name this time so I could go prepared!
  • Friends, Foes and Family. Social networking is a big theme at this year's conference, and also concerned a couple of the papers in this session. Have to say I was disappointed overall, very little made me think "oh, I never thought of that."

    The first talk was about how people assess 'attractiveness' on online dating profiles (conclusion: it's mostly about the photo—no surprise there), but without any real explanation as to how the research might be useful. "Maintaining friendships after a residential move" had the most potential out of this session, but again there were no real surprises; the one interesting snippet perhaps being that decreasing email frequency after a move does seem to have a negative impact on perceived 'quality of friendship', but that increasing the frequency doesn't have a positive impact. The other two shorter talks were about whether friends on social networking sites are more persuasive in spreading content than casual acquaintances or 'foes', and how and why parents tend to record personal information on their online calendars at work, and the sort of problems this can create. The latter (a Microsoft Research project) was particularly surprise-free, and seemed to me to highlight shortcomings in Outlook's calendaring capabilities compared to other online calendars as much as anything :)

Calum's CHI Photoblog - Part 1

Calum Benson is a usability engineer in the desktop team in Dublin, Ireland. He has worked at Sun for about eight years, primarily contributing to the open source GNOME Desktop project's usability efforts, to the usability aspects of Sun's integration of that project into Solaris, and currently, to the design of various GNOME-based applications for OpenSolaris.

Maya asked me if I'd file a few blog entries while I was at CHI 2008 in Florence... so here goes.


I arrived at my hotel around lunchtime, but although I was only coming from Ireland, I'd been up since 3.30am to catch the first leg of my flight to Amsterdam. So I was too zonked to do much other than wander up to the conference centre to register:

fortezza de basso

sculpture outside fortezza

sunlight canopy

tree in bloom

welcome to chi


Everything kicked off for real at 8.30am in something resembling an aircraft hangar, with the opening plenary given from my compatriot, Irene McAra-McWilliam, from the Glasgow School of Art—perhaps best-known outside the UK for being housed in Charles Rennie Mackintosh's finest building. Irene talked about the evolution of design, structuring her talk around a common feature of Florentine architecture: the Rose Window. Personally I did find it a bit of an introspective talk for an opener, perhaps because I don't have any formal training in design so I couldn't immediately connect with the content. But given the theme of this year's conference ("Art Science Balance"), that was perhaps partially the point...

Opening plenary

The sessions I attended today were:

  • Interactive Image Search, where Cue Flik, a Microsoft research project, was demonstrated. Here, the user could identify image search results that were most like the ones they'd been looking for (the example used was a 'product shot', which often shows a product on a white background), to not only refine the results but to apply to future searches. Thus, in this example, you could apply the 'product shot' rule when searching for anything from shoes to suitcases, and expect to find the top results showing those items on a white background.
  • Usability Evaluation Considered Harmful? Bill Buxton and Saul Greenberg presented their paper highlighting the dangers of blindly applying usability evaluations, to some extent because academia has come to demand them, sometimes at the expense of rejecting research into more challenging areas of usability and/or resulting in the mis-application of "weak science". Followed by a panel discussion (which naturally concluded that both usability evaluations and non-empirical methods have their place, and that the key is in choosing appropriately for the problem at hand).
  • Improved video navigation and capture. Three projects were discussed here, the first being research into automatically capturing meetings for live or delayed webcasting in a more engaging fashion, with appropriate cuts, tracking, close-ups etc. depending on who's talking—could tie in nicely with some of the telepresence projects at Sun Labs :) The other two projects, DimP and DRAGON (demos available by following the links) both provided the user a means to navigate through video by directly manipulating the objects in the frame, rather than the traditional means of dragging a slider. Both showed useful productivity gains for certain types of task, although I have to admit they weren't the sort of tasks I'd necessarily expect most people to encounter in their daily interactions with videos.

Inbetween sessions, I had a wander round the posters (which, this year, are changing every day):


In the evening, we had the official opening ceremony, with a plethora of Tuscan wine and edibles...

Table decoration

a string quartet...

String quartet

... and a few exhibits to play with! This is Snap and Grab, a video wall that you can interact with from your cellphone:

Snap and Grab

And this is Remote Impact from DistanceLab, in which you can punch (and be punched by) a remote opponent!

Remote Impact


xDesign is a software user experience design group at Sun.
Follow us on Twitter : Flickr : Blog (see feeds below)


« April 2014