Friday May 10, 2013

Enterprise API Management

There is a lot of buzz these days about API management. At Oracle we are major proponents of API Management and I am often in the field helping customers understand how to best leverage this new tool . The “API” stands for “Application Programming Interface”. I’m not sure exactly when the phrase was coined, but I suspect it was in the mid 1970s when the C language was starting to take hold. The need for applications to share information in a mainframe was increasing and the API was born.

That was fine for the 20th century, but in the 21st century we need to update our thinking. In my opinion, API Management is misnamed. It should be called “EPI Management” for “Enterprise Programming Interface Management”. Let me elucidate.
I’ve spoken with numerous customers and attended a number of API Management sessions hosted by public API Management companies where the same antiquated thinking is expressed:

API Management is about publishing my application interfaces.

This thinking is expensive and dead wrong.

I believe that the main problem with the latest API Management craze is that it is mis-named. EPI Management would be a much more appropriate term. What’s in a name? The core philosophy is in the name. The core philosophy in the name, “API Management”  leads enterprises into thinking that publishing their applications is the correct path forward. It is not, and here is why.

The name for directly integrating applications with each other is “Point-to-Point” Architecture. We know from decades of experience that this approach leads to building brittle IT systems that quickly take more time and money to maintain, leaving less money to innovate. Furthermore, API Management brings a new dynamic to the game of IT that you may not have considered. To better illustrate this, let’s examine a very simple enterprise with only 6 different applications.

Figure 1 - Simple Point-to-Point architecture example

Examine figure 1. It will be our starting point for our architecture as we embark on the API Management adventure. The figure shows 6 different systems and their integration dependencies. Implicit in this set of applications and integrations are numerous business rules. For example, the Customer Self-Service portal enforces the following business rules:

• Only customers registered in our Customer Service application may place orders with us.
• Customers must be in good standing with regard to their billing status before we allow them to place an order with us.

Similarly, business rules are built into the Order Management Application:

• When an order is placed the customers current billing status is verified
• A Bill of Materials is issued to the Inventory Application to assemble the order.
• Once the order is assembled, the Order Fulfillment application is invoked to handle the loading and shipping of the order

These are just a few samples of the dozens or hundreds of business rules that could be enforced by these applications. Now let’s examine the effects of naively publishing these application interfaces via an API Management service.

Figure 2 Simplistic API Management approach

 The blue circles represent API definitions in your API management layer. They are connected with a blue line to the respective applications whose functionality they publish. Let’s assume that this company has embraced using JSON messages over REST interfaces when they publish the APIs in the API Management tool. At the top of the figure I have drawn 3 different consumers of the APIs. I have only drawn  the API consumption of Consumer 1 to keep the diagram from getting too cluttered, but everything we are about to learn with Consumer 1 will apply to all other consumers.

Consumer 1 is developing a mobile application that allows its users to access their information and also to place orders with our company. The mobile app is essentially replicating the functionality expressed in our Customer Self-Service Portal. The only difference is that the mobile application is being developed by someone or some company that simply registered for access to these APIs.

As a direct result of this approach, we must trust and rely on the mobile application developer to both understand our business rules for the interaction of these applications and take the time to properly implement those business rules. Are you seriously going to trust external companies to execute this perfectly? We don’t even trust our internal developers to do this properly, which is why we employ QA testers, business analysts and others to help ensure that these details are properly handled.
Oh, and occasionally our business rules change. Now we have to give all of our external developers notice of the change and trust that they will implement that change in a timely manner.


That’s just part of the overall problem. Our internal applications also change over time. Our billing system will have upgrades applied over time. Can we be certain that these newer versions have not changed their current API definitions? If you are performing a major version upgrade, the vendor usually reserves the right to make whatever API changes are necessary to deliver the new and improved functionality.

The same problem exists if you decide to replace an existing application. Imagine that your billing system is antiquated, or the company is out of business and due to regulatory requirements you need to change to a new billing system that is supported. In this case you are guaranteed that the new application will have a totally different API and you will be forced to re-work your billing API in the API Management layer, as well as trust all consumers of that API to upgrade correctly and quickly.

More madness.

The core problem with this approach to API Management is that you are relinquishing almost all control over your business rules and integration logic. If you want to be successful with API Management you need to keep control over your business rules and integration logic. You need to provide application neutral APIs to the API Management system so that when your business changes, you can handle those changes without relying on external API consumers to do the right thing at the right time.

What is an application neutral API? It’s a logical API that makes sense to your company and how it does business. Each operation in the API needs to be as atomic as possible. By that I mean that each API operation needs to receive all of the data it will require to perform its work in a single call.

Figure 3 shows a slightly altered architecture that gives us the independence we need from application specific APIs. It also keeps our company in control over the APIs that we publish and it simplifies our API offering, thereby reducing our costs in teaching API consumer’s how to use our API and reducing our costs around maintaining a more complex API. Remember, the more you write, the more you have to maintain with your limited resources (time, people and money). That in turn reduces the resources you have available for new initiatives.

Figure 3 - API Management the right way

All we have really done in figure 3 is to create two logical views of information that our API consumers really want access to: Order Management Services and Customer Management Services. These service layers are used to convert between application specific interfaces of the lower level systems to the higher level, more logical and more business oriented functions that we want to publish in our API Management layer.
This is only one possible approach. The Order Management Services layer leverages the existing business rules that are implemented in the current Order Management Application. But it converts between the native API of the Order Management Application (which could be batch files, EJB, FTP or any one of a multitude of legacy technologies) into a more modern and easily consumable service like SOAP or REST.

Warning: Some API Management Vendors only consume SOAP services. Some of them cannot even consume REST services. Examine any potential vendor with care.

Using this approach we are trying to leverage our existing work as much as possible. There is no point in rewriting integrations if the existing work is already paid for, tested and deployed successfully. However, there are times where it makes good sense to rewrite some existing integrations to improve re-use and reduce duplicate business rules and code. I show this in figure 4.

Figure 4- Leveraging service-oriented integrations

In figure 4 all we have done is to re-create the integration and business logic that were contained in our Customer Self-Service Portal into our Customer Management Services integration layer. What’s the benefit of this approach? Well it can be rationally argued that the business logic behind those integrations need to be reused by multiple portals. Perhaps our partners want to be able to order from us using their portals. This approach keeps us in control of our business rules and only exposes logical services to our API consumers.

With this architecture in place, when I need to upgrade one of my existing applications to a new version, or replace the entire application with a new one, my existing Order Management and Customer Management service layers protect my API consumers from being affected by this change. It is in those layers that I can reliably define reusable business rules and convert between logical interfaces for my company and the application specific interfaces of my back-office applications.
So what happens when we publish only logical, application neutral services through our API Management layer? Well it turns out that we will naturally be publishing APIs that make sense to our enterprise. We will be building an interface to our enterprise. Hence the term API Management is fundamentally incorrect and out-dated. We should be calling it EPI Management. We will be doing all of our customers a favor by doing so, since it has a greater chance of setting them on the right path from the very beginning.


A site for thoughts, tutorials and more on the cloud, SOA and Oracle


« August 2016