Part Three: Special Areas of Feedback: Content, Quality, and Performance
By RichMiller-Oracle on Oct 26, 2009
Back to Summary
Part One: Gathering Feedback from Recent Usability Research and Studies
Part Two: Actual User Feedback
In the previous post, we talked about general feedback to the product and showed examples of the feedback customers provided and how we worked to resolve those issues. In this part, I want to focus on a few key areas which get a lot of talk; Content, Quality, and Performance.
Content is the information inside of My Oracle Support. It is the "text" of your Word Processing document, the image in your photo gallery and for My Oracle Support the knowledge-base, the questions asked in a Service Request, the health or patch recommendation and so on are content. It is not the user interface per se. My Oracle Support, the software relies on a variety of back-end services to provide this information. Let's talk about these content issues and what can be done to improve them.
Content issues and bugs are super important, but they tend to not be something that the design or front-end development team can fix. They require many teams across Oracle, and tend to take a long time to fix compared to the fixes we can typically do in the UI or in the database. Content issues customers reported include;
- Knowledge returning poor results
- SR templates asking the same questions over (and over, and over) again
- Patch or Health Check recommendations not being correct
- The list of products is different when searching Knowledge, filing an SR,searching for a configuration, or finding a patch. Ouch!
This is not an excuse. But the team I work with need to work with many other teams to resolve these issues. Darn right these issues need to be fixed!
For example, tuning the knowledge engine behind the knowledge articles is a challenging task. And long-time Oracle customers probably got used to the old search results. The new engine does things differently, and it should be better for most searches. The knowledge team is dedicated to continuously improve the search and browse experience of our knowledge base. They continually mine the search logs to learn and identify ways to improve search.
Tip: If you have trouble finding an article, is to go to the Knowledge page and use the product search in the top left. Once you have filtered down to the specific product, THEN type in a search in the search field in the middle of the screen to further refine your search. And try using the filters provided on the right side of the screen to also filter down to a manageable list. Now see if you can find your article. This product filter is also available from the Knowledge advanced search link to the right of the search field.
There some usability issues with first time users of the knowledge search, especially with the use of facets (the drill down options on the right of the search). Some design improvements are being worked right now for an upcoming release in that area. Also look at the short training video on how to get the most out of the knowledge search available in the Video Training Blog entry.
I know the knowledge team is constantly working to provide more precise results, faster queries, and improvements to the user interface. So continue to provide feedback. Tell them what you were trying to find and tell them about your search via the Feedback mechanism. This can help them improve the results from knowledge search. And when you can, try providing feedback on the articles you read. Recently viewed articles on the Knowledge page have a link to provide a review.
I can attest that some issues in the Service Request templates are being fixed, but I think that the templates are the same coming from Classic MetaLink as they are in the new user interface, and they would need to be modified to take into account the new UI. This has not happened yet, which is partially why you see the same question being asked over and over again. I do know your complaints about the process being too long for "simple" problems, is being addressed.
Patch and Health recommendations went through a big over-hall in early 2009 and that should have resulted in customers seeing much better results with less "silly" recommendations. Sometimes a recommendation is tagged for applying to say 30 or 40 releases of a Database, but gets "over-tagged". Then we wind up with a recommendation on one release which just doesn't make sense. Typically those should be fixed quickly, and I have noticed a large reduction in issues in that area in the last few months. If you find that a recommendation just doesn't make any sense on the system you are looking at, let us know! Improvements and fixes to the recommendation engine are done frequently.
And finally my personal pet peeve is the naming of Oracle products. In one place you look for "Oracle Server - Enterprise Edition", in another "RDBMS Server", a third is "Oracle Database". So each part of My Oracle Support is asking you for the same product, but using different terminology. I am so sorry! In addition, when Oracle changes the product name you get stuck because even though you are on an "old" release which uses the old name, Oracle wants to refer to it by the new name, no matter what release. We saw this in early user testing and created an alias list. So that, you might type "Database" and our UI would find the product. We added aliases to E-Business Suite products (so you could type "GL" for General Ledger" and find that product). This shipped with the 3.0 release of My Oracle Support. And we are working for a release in the future to have this alias list used everywhere and base it on a single common table of product names. This feature is not available yet in the MetaLink3 (support.oracle.com) site and will not be available when the Oracle customer migration occurs in Nov '09. But we know this is an issue and are working to get all of the development and support teams to work from a single list of products. By the way Oracle has more than 5000 products, and with each new company we buy this list grows. We are also working on ways to hide the 95% of this which you don't want or typically need to see. If you have some thoughts on this, do post to the blog! We would love to talk about it.
What is quality to you? For me quality has many dimensions. Does the product crash? Run slow? Provide me accurate results? Can I do what I expect when I expect it?
All of these probably matter to you, they certainly matter to our team. When doing usability research, or testing with early code, we tend to not deal with most quality issues, because we expect to fix them prior to production. And typically we are not running on the typical high-performance hardware, so it hard to gauge actual performance in the field. Like how fast a query returns. And even accurate results we don't necessarily catch in usability sessions. We might have just test or sample data, so again this type of information is not captured. So how do we capture this information? A special environment is setup to mimic real-world settings, but even then we don't always have the right data or real data in the environment. It is a combination of our quality engineers to find these issues. But even today we don't have all of the tests in place to verify every possible situation you would experience. Are you seeing a slow down when you PowerView, do a Group by, and then filter by name? Humm... sometimes we can only catch this on a case by case basis, so file a bug! But when it comes to user experience quality, we can find the issues typically in our tests. The real question comes, if we can fix them before you see them. Not everything you see goes through the same quality filters. Health Checks might be reviewed by one team, while the knowledge article about the issue and service request questions for that product are from two other teams. And sure enough all of these are for the same issue! So we do have some work to do to provide you a consistent, accurate and quality product. Inconsistency like this is a quality issue, a usability issue, and a problem worth solving.
I would guess we saved the most contentious issue for last. It is true there are places in My Oracle Support where performance is slow. And we have heard this loud and clear. Not all of the performance issues are from the use of Flash, but we are working them all, when possible. We have heard the following:
- Takes too long to load - get rid of the loading screen
- The dashboard comes up but then take a long time to load the content
- SR details load slowly
- Memory footprint of my browser grows and then slows the whole experience
- Delays when loading pages, PowerView, or other features
The development team has looked for any and all solutions. Some of the solutions implemented or slated for releases include the following:
- Reducing the size of the initial download (deferring the loading of some data)
- Placing the application on edge servers to allow faster downloading across the Internet from non-US locations
- Doing more "just-in-time" and server-side queries
- Allowing collapsed regions to wait until opened to get data (deferred loading)
- Tuning queries to return data faster
- Compressing data across the data connection
This is a true client application running in a browser. So once the application is loaded it should be very very fast. But when it has to wait on the back-end to return data, and sometimes we return a lot of data, you wait. It is a tough tradeoff. For example, if I took the Inventory report and made it non-interactive, it would be a lot faster. But then you would have to create a new report for the equivalent of every single click or drill down. When you have a 1000 or so collected systems (when using the collector) an interactive report like the one provided can answer tons of questions with only a click or two. But the cost is loading all of that data ahead of time. Maybe this is an ok tradeoff? But for a Service Request, it needs to load quickly... every time. Right? I can honestly say a lot of folks are working very hard to continue to improve performance. And even though the application is probably now at least 50% larger in features and size than at Oracle World last year, I think you will still see performance improvements in the upcoming releases. If you don't use a region, collapsing it will improve performance...
You should notice some performance fixes right now if you have an account on support.oracle.com. For customers using metalink.oracle.com, these improvements should appear when your migration occurs in November.
Go to the Dashboard and click Reload from the browser. How long did this take? Now collapse all of the regions on the screen so they are just one row tall. Now reload again. Did performance improve? How long does it take you to load the application for the first time from a browser where the application is not in your cache? Click on SR on the Dashboard. How long did it take to load? Click the "next" icon in the top right. How long for the second and third SR? Do tell, post to the Blog, let us know where you are from and how long it takes you. Inquiring minds do want to know!
So, in conclusions; we hear you: Performance needs work. Content should be improved, the site should work with all browsers, and don't don't make customers "beta testers" with a buggy site. I don't like the idea of customers filing Service Requests because the Service Request system is not working! And we, everyone on the My Oracle Support team, is working to fix these issues. But don't be shy. Do provide that feedback. And if you can, when asked, do participate in feedback sessions or usability studies. I know I am listening and so are many folks on our team.
I hope this series explained how we gather customer feedback, how we work to resolve the issues, what some or the key issues are, and what we are doing about it. Thanks for getting this far in the blog!