My Oracle Support

This morning, I logged on to My Oracle Support and immediately dragged the “Knowledge Articles” region back to the top because I knew I would be using it often. I moved the“Getting Started” region to somewhere in the bottom since I wasn’t really using it anymore. Then I started wondering: how do other people organize the regions on their Support Dashboard? How more efficient does this make us?

A friend of mine here at Oracle says he keeps Knowledge and Targets (filtered to show only his “Favorite” Targets) on the top of his page since he rarely logs actual SR’s and already knows enough about his systems.

How do you organize/customize your dashboard? How does it make you more efficient? What’s your favorite (or least favorite) customization feature?

On August 30, a few hundred thousand CRM OnDemand, Siebel, Hyperion, PeopleSoft and JD Edwards customers and partners migrated to My Oracle Support. Welcome aboard!

I hope you have had the chance to access the My Oracle Support Welcome Center (Document # 873313.1) from the Getting Started region. This is your single resource for information, training and reference materials that will help you get started using My Oracle Support.

The My Oracle Support product management team values your feedback.

Your general feedback and suggestions for improvement are taken very seriously and help us make My Oracle Support better for you. Click the Contact Us link on the top right corner of the Dashboard page to tell us about your usage of the personalized dashboard; knowledge, service request, and configuration management; and the results you are seeing.

All feedback is welcome...please tell us about your experience using My Oracle Support!

Summary
Customers have asked about how we deal with customer feedback. Customers who have participated in feedback sessions are especially interested in understanding how their feedback was incorporated into My Oracle Support. This series should answer that question for you. Thanks Charles for bugging me to put this out!

The My Oracle Support design team reaches out to customers in a variety of ways. This includes one on one interviews, group meetings, user group sessions, reading feedback via the site, watching Oracle and other blogs or forums, and doing larger scale surveys. Via these processes we generate feature requests, bugs, or enhancements, which in turn are given to a designer to design. The most feedback comes from places like the Feedback form, but typically these are "bugs" (items which need to be fixed but do not require design consideration). The focus on this series is on design. What features do customers want? How do they want the features to work? Who will use the features and how often? And will the feature make sense for a new user and be efficient for an experienced user?

Design is the process of understanding customer needs and providing a user experience which achieves customer needs and goals while focused on "ease of use". Design is not development. If one writes code to solve a problem it may or may not be "easy to use". Design generally puts more complexity into the code and the development process so that the user doesn't have to deal with the complexity. We are focused on understanding customer needs (we answer to the customer not to development) and how customer interacts with Oracle Support. So one can imagine that the time and energy to design something well takes more time up front, but would require less effort by development in the long run to "get it right" and provide the customer with what they need and want. Of course, we don't always get it right, so we iterate and refine until we do.

A series of user research studies using one-on-one web conference sessions were conducted covering dozens of My Oracle Support customers. These sessions covered basic customer needs (how do you do your business), review of designs to cover those needs (typically screen shots), and up through the use of a demo by the customer, while myself and other watched and asked questions. All of these were done prior to the release of features. Up to this point, we have not yet conducted any in field testing of shipping products, other than the Service Request process, but more on that hairy issue, later!

The results of these tests are written up into design briefs which are used to guide the design and development of the product. Part One will cover the details of how we gather this feedback. Part Two will go into the details of the feedback received. We recognize there are a variety of key issues which still remain in My Oracle Support. This will also be discussed in Part Three. And hopefully all of this will serve as a mechanism to get your input into the design process. I for one, listen to customer feedback and work very hard to make the feedback results appear in the product via improved usability and the introduction of important features.

Part One: Gathering Feedback from Recent Usability Research and Studies
Part Two: Actual User Feedback
Part Three: Special Areas of Feedback: Content, Quality, and Performance

Back to Summary
Part One: Gathering Feedback from Recent Usability Research and Studies

In Part One I covered how we gathered feedback from customers for the new My Oracle Support. In this part, we will discuss the detailed results of those studies by looking at two studies we have done in the last year. One done last summer, which was a large study of twelve customers, covering basic navigation and usability. The second a detailed study of the design of the health check features found in production for former Metalink3 customers and coming soon for Oracle customers (November of 2009).

In the first study, participants spent about an hour doing basic tasks mostly focused on SR creation in My Oracle Support. The problems and issues were documented by the usability tester and then reviewed and triaged by the team. Twelve customers participated. Each session was video recorded with their permission. The videos are used to both clarify issues and when filing a bug, a link to the video can be provided to the designer or developer to exactly see the issue.

Results
12 Customers
88 Actionable Issues, by category:

Create SR (42)

General Issues (5)
Delete, Saving, Breadcrumbs, Templates (4)
Wizard: General Information Step (11)
Wizard: Knowledge Step (5)
Wizard : Upload (4)
Wizard: Problem Details (7)
Wizard: Product and Problem (2)
Wizard: Review (4)

View SR (15) Draft SRs (5) Knowledge (8) General Usability, Dashboard, Filtering, PowerView, Customize, Timezones, etc.. (15) Bug Region (3)

For each of these issues, we cataloged how many of the customers experienced the problem during the session, and then rated the issues to help prioritize them. This was done in a spreadsheet and includes details of when the problem occurred, for who, and screen shots. Later bugs could be filed and pointing the developer to a link to the video to see for themselves the issue.

Let's look at some examples of the issues found, that are fixed for the current or upcoming release.

So from this study about 30% of the bugs are currently fixed in production releases. I highlight this study because this has a low fix rate. This is something we should work to fix in upcoming releases. While in the next study done for the Health Check area, 70 of the 72 issues were resolved prior to the first release of the Health Check user interface for the MetaLink3 users in August of 2009. Those results were interesting for a few reasons.

The Health Check region for My Oracle Support was effectively designed starting with customer feedback. While the SR form has been around forever, Health Checks are reasonably new and don't have a lot of baggage associated with their design. So we went to about 6 customers without any concept of what we should build and asked them how they want health checks to work and what features would be useful to them. This eventually created a roadmap of features. We then took the key features and design a series of design ideas and floated those by customers. Here are some examples of the evolution of those screens over time.

First this is the screen where we just mapped content up into three regions and called it a day. If you want to see details for a check, you have to click on the finding and go to another page. In this example, you can click on the bars and it will change the health checks shown below. This was a hack we did to help moving between Critical, Warning and Informational. Prior to that you had to return to the home page to change between Critical, Warning and Informational. Ouch!

We heard a few key things which made it into the design showed below. This is the most recent release of Health Checks, but still not the end-state. Notice the ability to Suppress a check. This was one of the key communications. Customers wanted to suppress based on a variety of options. We currently allow for the three shown (the issue, an entire target, and an entire check). Some customers also want other ways of suppressing, but more options yields more complexity. So we started with the "big three".

Notice also the ability to see the details of the check without navigating. This improves performance. You can also group by findings and multiple select rows and then suppress. Also notice it is clearer how to navigate to the different classes of checks. And a short cut to view any suppressed items. We also know we had issues with what to call the "Suppression" function (Hide?, Archive?, Put in Trash?), and did our best but also notice how we extended the Suppress menu with a Help link to "Learn about Suppression". So if you don't get it right away, you might explore and learn from the Help system.

But we know we were not done. In other studies concerning Patch Recommendations we were asked to; bring the advances from the Patch Recommendation UI into Health Recommendations, and to further extend the features of Health Checks to support a broader array of checks, systems to check and features. Thus the future should see us handle larger numbers of checks for our biggest customers, view the check in more detail, provide feedback to Oracle and the Oracle community about their use of this check (or issues with the check) and other similar features we have now released for Patch Recommendations (sorry can't give you a timeline for this).

Overall, we took in 72 customer issues. Each resulted in a design, assigned to developers and we fixed 70 of the issues prior to shipping the first release of this in Spring of 2009 for the customers on support.oracle.com (Nov 2009 for metalink.oracle.com customers). Try it out for yourself and see what you think.

And in the meantime, take a look at an "idea" I am working on based on recent customer feedback. As with anything this is just a working concept. No guarantee it would ever be released... and I will let the picture do the talking... Want to get involved? Let us know via the Blog.

So, this is a look into some of the feedback we have received for usability issues. We get more than we can discuss here. It hurts to hear complaints, but it worse to know we have issues. So do provide us feedback, this makes it easier to focus attention on fixing issues which you really want addressed. And one tip... Don't just tell us, "My Oracle Support is slow!", tell us what is slow and provide details, "When I load My Oracle Support via VPN from my office in China it takes 45 seconds before my dashboard comes up, and that is too slow!". Speaking of "slow", performance has been on our mind for a while. In Part Three I will cover some specific areas of special interest to customers, including Performance.

Next: Part Three: Special Areas of Feedback: Content, Quality, and Performance

Back to Summary
Part One: Gathering Feedback from Recent Usability Research and Studies
Part Two: Actual User Feedback

In the previous post, we talked about general feedback to the product and showed examples of the feedback customers provided and how we worked to resolve those issues. In this part, I want to focus on a few key areas which get a lot of talk; Content, Quality, and Performance.

Content Issues
Content is the information inside of My Oracle Support. It is the "text" of your Word Processing document, the image in your photo gallery and for My Oracle Support the knowledge-base, the questions asked in a Service Request, the health or patch recommendation and so on are content. It is not the user interface per se. My Oracle Support, the software relies on a variety of back-end services to provide this information. Let's talk about these content issues and what can be done to improve them.
Content issues and bugs are super important, but they tend to not be something that the design or front-end development team can fix. They require many teams across Oracle, and tend to take a long time to fix compared to the fixes we can typically do in the UI or in the database. Content issues customers reported include;

• Knowledge returning poor results


• SR templates asking the same questions over (and over, and over) again


• Patch or Health Check recommendations not being correct


• The list of products is different when searching Knowledge, filing an SR,searching for a configuration, or finding a patch. Ouch!

This is not an excuse. But the team I work with need to work with many other teams to resolve these issues. Darn right these issues need to be fixed!

For example, tuning the knowledge engine behind the knowledge articles is a challenging task. And long-time Oracle customers probably got used to the old search results. The new engine does things differently, and it should be better for most searches. The knowledge team is dedicated to continuously improve the search and browse experience of our knowledge base. They continually mine the search logs to learn and identify ways to improve search.

Tip: If you have trouble finding an article, is to go to the Knowledge page and use the product search in the top left. Once you have filtered down to the specific product, THEN type in a search in the search field in the middle of the screen to further refine your search. And try using the filters provided on the right side of the screen to also filter down to a manageable list. Now see if you can find your article. This product filter is also available from the Knowledge advanced search link to the right of the search field.

There some usability issues with first time users of the knowledge search, especially with the use of facets (the drill down options on the right of the search). Some design improvements are being worked right now for an upcoming release in that area. Also look at the short training video on how to get the most out of the knowledge search available in the Video Training Blog entry.

I know the knowledge team is constantly working to provide more precise results, faster queries, and improvements to the user interface. So continue to provide feedback. Tell them what you were trying to find and tell them about your search via the Feedback mechanism. This can help them improve the results from knowledge search. And when you can, try providing feedback on the articles you read. Recently viewed articles on the Knowledge page have a link to provide a review.

I can attest that some issues in the Service Request templates are being fixed, but I think that the templates are the same coming from Classic MetaLink as they are in the new user interface, and they would need to be modified to take into account the new UI. This has not happened yet, which is partially why you see the same question being asked over and over again. I do know your complaints about the process being too long for "simple" problems, is being addressed.

Patch and Health recommendations went through a big over-hall in early 2009 and that should have resulted in customers seeing much better results with less "silly" recommendations. Sometimes a recommendation is tagged for applying to say 30 or 40 releases of a Database, but gets "over-tagged". Then we wind up with a recommendation on one release which just doesn't make sense. Typically those should be fixed quickly, and I have noticed a large reduction in issues in that area in the last few months. If you find that a recommendation just doesn't make any sense on the system you are looking at, let us know! Improvements and fixes to the recommendation engine are done frequently.

And finally my personal pet peeve is the naming of Oracle products. In one place you look for "Oracle Server - Enterprise Edition", in another "RDBMS Server", a third is "Oracle Database". So each part of My Oracle Support is asking you for the same product, but using different terminology. I am so sorry! In addition, when Oracle changes the product name you get stuck because even though you are on an "old" release which uses the old name, Oracle wants to refer to it by the new name, no matter what release. We saw this in early user testing and created an alias list. So that, you might type "Database" and our UI would find the product. We added aliases to E-Business Suite products (so you could type "GL" for General Ledger" and find that product). This shipped with the 3.0 release of My Oracle Support. And we are working for a release in the future to have this alias list used everywhere and base it on a single common table of product names. This feature is not available yet in the MetaLink3 (support.oracle.com) site and will not be available when the Oracle customer migration occurs in Nov '09. But we know this is an issue and are working to get all of the development and support teams to work from a single list of products. By the way Oracle has more than 5000 products, and with each new company we buy this list grows. We are also working on ways to hide the 95% of this which you don't want or typically need to see. If you have some thoughts on this, do post to the blog! We would love to talk about it.

Quality
What is quality to you? For me quality has many dimensions. Does the product crash? Run slow? Provide me accurate results? Can I do what I expect when I expect it?

All of these probably matter to you, they certainly matter to our team. When doing usability research, or testing with early code, we tend to not deal with most quality issues, because we expect to fix them prior to production. And typically we are not running on the typical high-performance hardware, so it hard to gauge actual performance in the field. Like how fast a query returns. And even accurate results we don't necessarily catch in usability sessions. We might have just test or sample data, so again this type of information is not captured. So how do we capture this information? A special environment is setup to mimic real-world settings, but even then we don't always have the right data or real data in the environment. It is a combination of our quality engineers to find these issues. But even today we don't have all of the tests in place to verify every possible situation you would experience. Are you seeing a slow down when you PowerView, do a Group by, and then filter by name? Humm... sometimes we can only catch this on a case by case basis, so file a bug! But when it comes to user experience quality, we can find the issues typically in our tests. The real question comes, if we can fix them before you see them. Not everything you see goes through the same quality filters. Health Checks might be reviewed by one team, while the knowledge article about the issue and service request questions for that product are from two other teams. And sure enough all of these are for the same issue! So we do have some work to do to provide you a consistent, accurate and quality product. Inconsistency like this is a quality issue, a usability issue, and a problem worth solving.

Performance Issues
I would guess we saved the most contentious issue for last. It is true there are places in My Oracle Support where performance is slow. And we have heard this loud and clear. Not all of the performance issues are from the use of Flash, but we are working them all, when possible. We have heard the following:

• Takes too long to load - get rid of the loading screen


• The dashboard comes up but then take a long time to load the content


• SR details load slowly


• Memory footprint of my browser grows and then slows the whole experience


• Delays when loading pages, PowerView, or other features

• Reducing the size of the initial download (deferring the loading of some data)


• Placing the application on edge servers to allow faster downloading across the Internet from non-US locations


• Doing more "just-in-time" and server-side queries


• Allowing collapsed regions to wait until opened to get data (deferred loading)


• Tuning queries to return data faster


• Compressing data across the data connection

This is a true client application running in a browser. So once the application is loaded it should be very very fast. But when it has to wait on the back-end to return data, and sometimes we return a lot of data, you wait. It is a tough tradeoff. For example, if I took the Inventory report and made it non-interactive, it would be a lot faster. But then you would have to create a new report for the equivalent of every single click or drill down. When you have a 1000 or so collected systems (when using the collector) an interactive report like the one provided can answer tons of questions with only a click or two. But the cost is loading all of that data ahead of time. Maybe this is an ok tradeoff? But for a Service Request, it needs to load quickly... every time. Right? I can honestly say a lot of folks are working very hard to continue to improve performance. And even though the application is probably now at least 50% larger in features and size than at Oracle World last year, I think you will still see performance improvements in the upcoming releases. If you don't use a region, collapsing it will improve performance...

You should notice some performance fixes right now if you have an account on support.oracle.com. For customers using metalink.oracle.com, these improvements should appear when your migration occurs in November.

And one final thought. You might ask, well just do it in straight AJAX, that will solve the issues! I know, that was one of our thoughts too. But the testing matrix for AJAX is huge, and the javascript code can also be quite large to load, and in the end some of our biggest performance hits have nothing to do with the front-end technology per se, they have to do with how we access the content and how much context exists. A list of 5000 products is the same no matter what the technology. How and when we access it still needs to be addressed.

Performance Test

Go to the Dashboard and click Reload from the browser. How long did this take? Now collapse all of the regions on the screen so they are just one row tall. Now reload again. Did performance improve? How long does it take you to load the application for the first time from a browser where the application is not in your cache? Click on SR on the Dashboard. How long did it take to load? Click the "next" icon in the top right. How long for the second and third SR? Do tell, post to the Blog, let us know where you are from and how long it takes you. Inquiring minds do want to know!

So, in conclusions; we hear you: Performance needs work. Content should be improved, the site should work with all browsers, and don't don't make customers "beta testers" with a buggy site. I don't like the idea of customers filing Service Requests because the Service Request system is not working! And we, everyone on the My Oracle Support team, is working to fix these issues. But don't be shy. Do provide that feedback. And if you can, when asked, do participate in feedback sessions or usability studies. I know I am listening and so are many folks on our team.

I hope this series explained how we gather customer feedback, how we work to resolve the issues, what some or the key issues are, and what we are doing about it. Thanks for getting this far in the blog!