Metrics for Mix
By tbonnema on Jan 05, 2009
Aside from evaluating new feature ideas, we've also started to look at numbers more closely. We wanted to know where Mix stands in terms of its size, adoption rate, user activity levels etc.
For purposes of this high-level assessment, we looked at the following five key areas:
This includes measuring the total number of registered members as well as that portion that can be considered active.
Mix has been growing steadily in 2008 to currently a little over 31,000 members. Growth peaked around Oracle OpenWorld '08 and has slowed since. About one third of registered members are Oracle employees. As of November, when we took a snapshot, the number of active users (as defined as those who have logged in at least once over a 30-day period) was at a fairly low 7%. Definitely something we need to work on going forward.
Your basic web metrics: page views, session visits, time spent etc.
Thanks to some of the integration that happened leading up to Oracle OpenWorld '08, we saw a big spike in traffic, reaching almost 250,000 page views in September. Things have slowed down again since then. Traffic per 1,000 users has been trending downward all year, another thing we'd like to reverse in 2009.
Right now, the main two items people can contribute on Mix are questions and ideas. But they can also leave comments and answers, vote on ideas, share notes within their groups or send messages to people in their networks.
The activity levels here correspond to what we see in terms of traffic: a significant spike around OpenWorld '08, some slowing down since then, and a continuing downward trend in contributions per 1,000 users.
In order to understand how "networked" our members are, we looked at the number of connections per user, the number of group memberships per user as well as the groups themselves (size, activity etc.).
There's definitely a lot of power law stuff going on. A small number of users account for a large number of connections, while the majority of users don't have any connections at all. Similarly, a very small number of groups account for a large number of group memberships while many groups have only ten or fewer members (not that small groups are a bad thing per se, necessarily, but it may point to some issues with our matchmaking capabilities).
Last but not least, we looked at how well-behaved the Mix community is. Things have been very good so far (no spam, no abuse etc.). That's a great asset, in my view, and we'll try to further build on that as we grow Mix.
* * *
As we build out the Oracle Mix roadmap for 2009, more metrics will have to be added in order to measure progress against specific goals. To the extent that these numbers are ok for public consumption, we'll try to share them as freely and as regularly as possible.
If you're a metrics buff yourself or would like to learn more about the inner workings of Mix, drop us a line in the comments and we'll see if we can dig something up for you.