Oracle AI & Data Science Blog
Learn AI, ML, and data science best practices

4 Things to Remember When Defining KPIs

No matter the company size, maturity of team, or complexity of project, driving alignment on KPIs (key performance indicators) is always challenging. The metrics and indicators must point to a company’s overall strategy and objectives in the long term while also creating actionable items in the short term. Furthermore, the rapid increases in data prevalence and organizational data literacy can make measuring success even more difficult.

I imagine that before the big data generation, measuring success focused more on KPIs like “the # of users purchasing a given product each month” because that’s the only metric that could be easily tracked. Today, we can easily measure something like “the # of users who open the app, then take action X within 5 seconds, then invite their friend to the app, but that friend must not have opened the app within the last 7 days.” So we should use that as our KPI, right? Wrong!

Having led analytics projects of all shapes and sizes, I’ve found that the most underestimated and overlooked challenge for data professionals is aligning leaders on answering the question, “What does success look like?” There is no perfect solution, but here are some things I’ve learned from establishing and managing KPIs.

1. People are smart, but don’t assume they get it.

I once joined a company where “Active Users” was the top KPI, but when I polled key stakeholders, there were different interpretations of what that actually meant. One definition was that someone opened up the app. Another was that someone received a push notification. Still, another was that the app was running in the phone’s background. Each definition had their rationale, but yielded different results that had material impacts on decision-making. This KPI was the primary metric being used to drive company strategy, and there was a clear issue of misalignment.

To rectify this issue, we ended up having to call an executive meeting to collectively answer the question, “What does success look like?” The conversation kicked off with qualitative discussion, but soon transitioned to the topic of quantifying this success with “Active Users.” Each definition was discussed, with stakeholders explaining their preferences. Eventually, everyone agreed that “Active” would mean someone who has opened up the app. This definition was the easiest to understand, simplest to instrument, and best for comparing with other industry benchmarks. No KPI is perfect, but having alignment is imperative.

2. Evangelize, highlight, and educate.

After KPIs are established, you then must operationalize them. Data-savvy people will pick them up quickly, but KPIs are for the entire team. Teams have higher performance when everyone understands the primary objective. Moving from larger to smaller companies, I’ve learned that TV Dashboards may be the most undervalued strategic tool for organizational alignment. Additionally, leverage all comms channels (e.g. meetings, emails, town halls) to regularly socialize results and remind everyone of definitions. Pop quizzes during company town halls are a fun way to improve everyone’s understanding and get people talking about KPIs. It’s crucial that teams know whether they are winning, so KPI programs should be actively managed.

3. Find a “lower common denominator” KPI.

A common issue with defining KPIs for relatively technical products is going down the rabbit hole of finding a KPI that measures the exact behavior of the technical components. Yes, there are cases where success measures should be extremely technical, but for the majority of situations, simpler metrics are more effective. It’s not very useful if the technical expert understands the complex metric, but the rest of the team is lost. When you start noticing confusion, try to find a “lower common denominator” KPI. This involves focusing on a core behavior that happens as a result of technical component’s success. The feelings during an overly complex KPI conversation are palpable, and that’s when you should KISS (keep it simple, stupid!).

4. Lather, rinse, repeat.

Like everything else, measuring success should evolve over time. Shifting company strategy, new product features, and technology changes are all reasons why measuring success may need to be adjusted. It’s important that teams revisit whether existing metrics are still the best way to measure success. It depends on business rhythm, but I encourage reassessing KPIs every other quarter or whenever there is a major change. Hold regular meetings and ask the critical question, “What does success look like?”

Establishing KPIs is crucial to the success of any project or company. Whether launching a simple website or deploying machine learning applications, leaders must align stakeholders around measuring success. I’ve outlined a frameworks with the aforementioned tips, but I encourage the development of your own style that best helps you and your team win!

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.