Standards vs. Proprietary APIs, take the standard for the long haul

We're currently nearing Alpha / Beta for the StorageTek Management Portal 2.0 (previously the StorEdge Management Portal 1.0). The jump in the version number from 1.0 to 2.0 is necessitated by a jump in adoption of the SMI-S 1.1 standard for collecting information from the storage ecosystem as opposed to our previous "agent" technology that was a mash of SNMP and proprietary agent libraries. SMI-S is, basically, a blend of the three important pieces that make up a Service Oriented Architecture (granted, SMI-S is not currently a Web Services model, but that is in the works with WS-Management). SMI-S provides:
  • Lookup of agents via Service Location Protocol (SLP)
  • Communication with agents via Web-based Enterprise Management (WBEM) over HTTP(s)
  • Navigation of software/hardware elements via a standard model
Jumping to standards provides a variety of benefits for our team:
  • We have been able to rapidly inspect and gather content from devices we have not encountered before
  • We manipulate all agents with about 90% of the same code (maybe 10% unique code is necessary due to vendor anomalies in compliance to SMI-S)
  • We do not have to create extension points for new protocols (WBEM over HTTPS vs. SNMP vs. RPCs vs. RSH vs. etc...) to communicate with new devices
  • We are given a standard data model for our information and we don't have to spend much time creating an internal data model that is an abstraction of all of the proprietary APIs.
Moving to SMI-S is not exactly easy. With all standards, there is a certain level of over-design and obvious points where things may have been negotiated too long for naught. Further, with SMI-S, we mine only a small percentage of the information from the model. Advocates of leveraging proprietary protocols often hold up examples of how easy it is to mine such a small percentage of data by using proprietary libraries and, in many ways, they are correct. Prototyping with proprietary libraries is often very focused, quick and easy. Tackling the full SMI-S standard is like entering a world with TOO MUCH information and an often unwieldy model. The learning curve to get a small amount of information with SMI-S is long, as opposed with proprietary libraries with which the learning curve to get a small amount of information is, well, short (SMI-S 1.1 is approximately 1,500 pages...typical proprietary API documents are around 100-200 pages). But here's where things get interesting, we often attend SNIA Plugfests where we bring SMP 2.0 and plug it in (this is easy since we run on a Solaris x86 laptop now...being based on Java ES, it was a quick and easy move for us with only a couple of glitches). At the plugfests, information from systems we have not tested with starts flowing into the portal. This would be impossible if we were based on the proprietary APIs of customers. For example, we can acquire performance information about some EMC arrays without adding code. Not only is device compatibility easier for information collection, once our team mastered their section of the model, it became easier and easier to collect more and more information. Different customers have different information needs and mining that information from 10 vendor's proprietary APIs you end up needing 10 experts manipulating the libraries. And on top of the libraries, the architect layers a domain model, something that SMI-S already provides a large percentage of. With SMI-S client compliance has come a wealth of information and capabilities at our fingertips. This isn't really a turtle vs. hare race that prototypers want you to believe we are in. The turtle was persistent but slow and the hare was arrogant and fast and could have actually won the race, there is NO WAY that proprietary APIs, protocols and models can win this race. Maintaining a proprietary API if you're a vendor is a burden to your adopters and to yourself as you are reinventing the wheel. Adopting proprietary protocols if you are a client is asking for a heavily staffed agent team that largely duplicates each others work but with different vendors. This is more like having a race across the USA, I get a 747 and you get a Subaru Outback. If you are taking bets after 10 miles, chances are the Subaru is ahead. Talk to me when we get to the Mississippi River. I'm in Broomfield Colorado, the Subaru is unlikely to be out of Colorado and I will have a 2 state lead with my 747. I like my odds with SMI-S. There are many downsides to SMI-S. A lot of information is only avaialable in vendor "extensions" of the model. To this I answer, least I don't have to navigate a different protocol and add another lookup API and, further, the extension points to the model are standard even if the contents of the extension points are not. There is not complete device support, especially where legacy devices are supported. This issue will likely gnaw away at us. Many companies have broad SMI-S support, but we also realize there are many companies with older devices that are still a valuable part of their storage ecosystem. As SNIA says, customers MUST be part of the solution, you will get better and more standard management tools with SMI-S adoption. Pressure your suppliers and vendors to provide SMI-S support for older arrays, it is worth their effort to keep you as a customer. Standards will make your ecosystem more manageable. There is no in-band management. Well, this is a choice of the implementers of the SMI-S agent. There is no requirement that SMI-S agents talk out of band to the devices manipulated by the agent, the only requirement is that the management station talks to the SMI-S agent out of band. This can be tricky with high security networks. BUT, the more involved clients are with adopting SMI-S solutions, the more pressure we will have to show device vendors that this versatility is necessary. Working together with clients with the high-security networks can help build acceptable deployment architectures that maintain the integrity of the customer networks while leveraging the standard in the management tools. Footprint...SMI-S agents tend to be heavier than a proprietary agent. This is largely true, I have to admit. It doesn't HAVE to be true but it must be fixed in agent implementations and across the industry before SMI-S can deliver a knock-out blow to proprietary vendor libraries. Finally, my big issue with SMI-S is the blending of the lookup, protocol and model. SMI-S should be about the model only. Adoption of WS-Management will be a huge step to standardizing tools and capabilities with the rest of the industry on a Web Services base and we will be able to retire the WBEM is CIM-XML and not SOAP debate...and its not soon enough. The lesson with SMI-S that we are learning on the StorageTek Management Portal 2.0 is one that should resonate in all areas where there is a standard way to do things and a proprietary way to do things. If you have a one-off, go ahead, use the proprietary way...but if you are in it for the long-haul, learning the standard, working with the standard and participating in the standard will provide velocity through breadth and depth of information as well as unexpected vendor reach that simply cannot be achieved by pursuing each vendor's API separately. There will be turbulence along the way, I guarantee it. But are you positioning yourself for the future, or are you positioning yourself for next week?

Post a Comment:
Comments are closed for this entry.



« June 2016