David Chappell Blog

It's always great to hear about successful adoption of SOA, and its particularly exciting when its being used for what you'd think. :)

MOVE, Inc. has recently finished up a really cool SOA project to integrate together and streamline a set of processes a variety of externally facing web sites with internal CRM solutions.  The solution involves the use of BPEL, Web Services, ESB Mediation, application adapters, and canonical data models.

The business benefit to all this from a high level is that MOVE, Inc. was able to improve customer service by improving visibility of customer information across all their systems and improved processes while helping reduce the cost of ownership for its existing applications including Oracle's Siebel CRM and Oracle's PeopleSoft Enterprise Financial Management.

Before I get into how they did it, first a little bit about MOVE, Inc. and the challenges they faced -

Move Inc., formerly Homestore, Inc., is the leader in online real estate.  Through it's network of sites (below) Move, Inc provides information on real estate property listings (homes and apartments), moving, home and garden and home finance.  . 

Move's network of websites:

• 
  
  REALTOR                                www.realtor.com
  
  
• 
  
  Welcome Wagon                    www.welcomewagon.com
  
  
• 
  
  Moving                                    www.moving.com
  
  
• 
  
  Senior Housing Net                www.seniorhousingnet.com
  
  
• 
  
  Home Plans                            www.homeplans.com
  
  
• 
  
  Home Insight                         www.homeinsight.com
  
  
• 
  
  Factory Built Housing             www.factorybuilthousing.com

In the midst of rapid growth, their challenge was they needed to consolidate disparate applications across two of its business units into an enterprise standard CRM solution that could scale to meet its evolving needs and provide a single, accurate view of customer data to improve efficiencies and automate auditing, billing and fulfillment processes. 

Move's business units had custom applications built on the .Net framework for supporting their customer relationship functions and order creation. There were numerous disparate systems and manual touchpoints built to support various business functions for customer relationship management. In an effort to consolidate these applications into an enterprise standard CRM solution and also provide an application which could scale to the growing needs of the business, a CRM solution consisting of Oracle Customer Data Hub, Siebel and PeopleSoft integrated with Oracle Fusion Middleware was proposed to enable the following.

• 
  
  Consolidated Customer Database
  
  
• 
  
  Support for Existing Sales Activities
  
  
• 
  
  Order Capture
  
  
• 
  
  Integration with Fulfillment Systems
  
  
• 
  
  Integration of customer usage from web sites into Siebel
  
  
• 
  
  Integration to PeopleSoft Billing

It sounds like a pretty tall order.

As part of the implementation MOVE worked with CSC (Computer Sciences Corp).  CSC built a SOA that focused on re-usability of services. CSC also ensured that the business processes were at least as efficient if not better than what the current systems offered.  Using standard WSDL interfaces, Oracle BPEL PM and Oracle ESB were used to extend the reach and normalize the integration across Siebel CRM, PeopleSoft Enterprise Financial Management and other systems both internal and external (Figure 1).

Figure 1: Using BPEL, ESB, and Web Services to help implement their SOA, Move extends the visibility of customer data.<o:p></o:p>

How They Got There

The first step in the design process was to model the future business process and identify the functionality that would be enabled through Siebel and the key integration points with the various systems. Move and CSC worked together to define canonical data models for customer, order, listing, usage and billing information and created service contracts for each of the Web services that were used in the integration.  CSC then build new streamlined processes as BPEL orchestrations using Oracle BPEL Process and Oracle Enterprise Service Bus (Figure 2) -

Figure 2: Canonical data models allow processes to be constructed that generically orchestrate a common set of canonical services to without the need for specialized point-to-point data transforms

Service contracts were defined for each of the web services that would be used in the integration.  The customer information was then consolidated from the multiple systems that the two business units had into the Oracle Customer Data Hub. Customer Data Hub also serves as the hub of customer information and provides visibility of customer information across the various systems in real-time including Siebel and PeopleSoft.

Integrations were developed as BPEL orchestrations for processes such as Order Fulfillment and Account/Customer Information creation across the various systems. Oracle Customer Data Hub is used to maintain customers for other business units. Oracle ESB is used to receive real-time outbound business events from Oracle Customer Data Hub and synchronize the customer information with PeopleSoft Enterprise (Figure 3). 

Figure 3: ESB combines Event Driven Architecture (EDA) to synchronize customer information between Customer Data Hub and PeopleSoft

So, What's the ROI?

BTW, this all took them 6 months to implement and deploy.  The net results are that Move was able to achieve the following benefits of this CRM Solution.

• 
  
  Real-time integration across these systems
  
  
• 
  
  Process Improvement and improved efficiency
  
  
• 
  
  Visibility of Customer Information across all the systems
  
  
• 
  
  Reduction in cost of ownership on existing applications
  
  
• 
  
  Quicker integration of new products into back office applications

From a long term perspective, this solution allows Move to

• 
  
  Leverage and Protect their existing investments with their .Net applications
  
  
• 
  
  Integrate back office applications from future acquisitions into this CRM Solution efficiently
  
  
• 
  
  Extend and Evolve this CRM Solution and mitigate the need to enhance their existing applications to support business needs

Dave

At this week's RSA Conference 2008, Thomas Kurian laid out Oracle's vision for what is being coined Service-Oriented Security.  Service-Oriented Security is a way of using a service orientation to enable applications with a complete set of the common security capabilities and security processes such as authentication, authorization, user administration, role management, identity virtualization and governance, and entitlement management, as well as audit and control.  Tony Baer also discusses the topic in his blog: http://www.onstrategies.com/CURRENT-NEWS/Oracle-Releases-Role-Manager-Pushes-Service-Oriented-Security-Strategy.html. 

Typically an application may hard wire security capabilities, which makes for a brittle architecture that is not conducive to change.  With Service-Oriented Security, security capabilities and processes are decoupled from applications and more centralized via SOA, making them more available, manageable and consistent across an enterprise.  So in a sense, we are using SOA to enable and strengthen your SOA projects by making them more agile.

To back this up, there have been some key deliverables to date which enable this vision, which cover security issues spanning deployment, governance, administration, and development

• 
  
  GA of Oracle Role Manager, which provides a service for roles-based access control, provisioning and approvals across business applications.
  
  
• 
  
  GA of Oracle Application Access Controls Governor 8.0, which is control monitoring software that provides segregation of duties analysis and enforcement for heterogeneous enterprise application environments
  
  
• 
  
   A controlled beta preview release of Oracle Fine Grained Authorization.  This is software designed to externalize hard-coded authorization policies from heterogeneous enterprise applications, and nicely complements Oracle's Identity and Access Management offering.
  
  
• 
  
  Identity Governance Framework - a multi-vendor standard proposal, spearheaded by Oracle, that provides a service-oriented privacy-aware architecture for developers to access identity data while adhering to usage policies.  Oracle, in conjunction with the Liberty Alliance, has delivered the first open source component of the proposed standard.  

For more information, check out these links -

White paper on Service Oriented Security  -

http://www.oracle.com/technology/products/id_mgmt/pdf/serv_oriented_sec.pdf

Other related announcements -

Oracle Releases Oracle Application Access Controls Governor 8.0

Liberty Alliance and Oracle Team to Advance Identity Governance Framework

Dave

For the past several years, I have been involved in many healthy discussions centered around the benefits of adopting technology and its supporting tools and infrastructure.  Never once had I ever thought of measuring the benefits in terms of tonnage of hardware or in kilowatts per hour (kW/h).

Until now.

In David Linthicum's recent blog on "Is SOA Green" http://weblog.infoworld.com/realworldsoa/archives/2008/03/is_soa_green.html?source=NLC-SOA&cgd=2008-03-25, he makes the point that "the more efficient a business is, the fewer resources, natural and otherwise it will consume. Thus, the better the business is automated, and business processes optimized, the more efficient it will be. The primary of objective of an SOA is both efficiency and agility through architectural rejuvenation"

I couldn't agree more.  BTW today is Earth Day.  This week is also the Web2.0 expo in San Francisco http://en.oreilly.com/webexsf2008/public/content/home, and the Green Expo in New York http://gogreenexpo.com

In light of all these things coming together, and in the spirit of "going green", I would like to tell you about a concrete example in an exciting account of one of Oracle's customers, Verizon Wireless, who is in the process of going green by rewriting their fraud detection application using SOA, EDA, and Web 2.0, and as a result plan to eliminate 6 tons of hardware from their datacenter, and reduce power consumption by 99.5% from 200 kW/h down to 1100 W/h.

From a software and architecture perspective, this is an exciting story about how Verizon Wireless is building a new Fraud Detection application which uses Event-Driven Architecture (EDA), BPEL process management, a business rules engine together with a Web 2.0 style Flash/Flex UI to build what would be referred to in financial services as a straight-through processing application. 

In the process of doing this, they are dramatically reducing the amount of code that has been written, and the amount of data that needs to be stored locally.  As such, they plan to eliminate 6 E-class Sun boxes using ~192 processors and replace them with a single 8 core processor on a Sun UltraSPARC T1 using the Niagara chip architecture.

The old application is based on J2EE using what was available in the mid to late 90's.  It happened to replicate the entire data warehouse of call detail records for use by the fraud detection application. It also had a lot of procedural custom code that was hand written by 5 FTE over 2 years, some stuff that was ported from Forte to J2EE, and 100s of JSPs feeding (circa 1995) html 3.0 pages with data.

I plan to provide more detail about their service architecture in the future, but here is the basic outline of the new application structure:

• 
  
  Data coming from the switches is analyzed and checked for business exceptions.  Examples of business exceptions include detection of excessive data thresholds, which could be an indicator of a customer using their laptop to run a streaming video website, or the practice of phone cloning to make phone calls using someone else's account, or more ambitious fraud such as masquerading as a third party roaming partner and attempting to charge back to Verizon. (this is the EDA part of the architecture)
  
  
• 
  
  When such an exception is detected, an event is generated and sent to a BPEL process
  
  
• 
  
  The BPEL process invokes a number of services, which includes going out directly to the source of the call detail records to get the information necessary to enrich the event data.  It is then fed into a rules engine to check for violations, make decisions based on policy, and then on to generate more detailed reports.  (This is the SOA part of it)
  
  
• 
  
  The supporting SOA technologies involved include Oracle BPEL Process Manager, Oracle's rules engine, and Oracle AQ.
  
  
• 
  
  The UI interface is RIA/Flex based, and runs mostly in the browser (this is the RIA/Web 2.0 part of it)

So, back to the "going green" part - They are working with a third party energy consultant to produce the final number on energy reduction once they go into production (they are in UAT stage now), but here is their best estimate given what is known today. 

• 
  
  6 TONS of equipment reduced to ~200 pounds (production, qa, dev-test).  I just love those numbers!
  
  
• 
  
  192 processors reduced to a single, 8-core processor (production)
  
  
• 
  
  20+ Terabytes reduced to < 64 Gigabytes storage
  
  
• 
  
  100 kW/h reduced to 540 W/h (production)

I'm no expert at cooling system requirements, but I have heard the rough calculation for power consumption of cooling systems range from 1.3 to 2 X the amount of power consumption of the hardware, so the numbers should really be 200 kW/h reduced to 1080 W/h.  Its still a 99.5% reduction in power usage.

Of course I can't attribute 100% of the impressive results to adopting SOA.  There are lots of things that contribute to the reduction in footprint, such as the massive rewrite and the introduction of more powerful hardware and chip technology (I'll leave that writeup for  SUN to have their own day in the...sun).  However, I can point out some things that are obvious and significant -

It is often said that the best line of code is one that you never have to write.  The new implementation is 0.5% of its original size.  This is directly attributable to using Oracle  BPEL Process Manager and the rules engine instead of their custom code.   They also went from 5 FTE down to 1 consultant who built 40 - 50 BPEL processes over the course of a year.  The result is more flexible and resilient to change as it is declarative and UI driven rather than being hard wired into custom code.  Their presentation tier code also went down to 0.5% of its original size by moving to 1 JSP and 1 SWF for rich UI. 

The other significant contributor to the reduction in footprint is due to the nature of modern SOA standards and supporting infrastructure such as BPEL and ESB, which allows Verizon to extend the reach of their systems so that they don't have to store their "golden record" call detail data locally.  They can get the data remotely and on the fly, and use enrichment services along the way to get the data in the proper form to be make decisions about fraud and overage, and generate detailed reports about the business exceptions that are identified.  This is huge for them in that they no longer have to replicate the data warehouse, but are able to pull call records and other information directly from external heterogeneous systems.

BTW, in David's blog he also points out - "Thus, with gas approaching four bucks a gallon, I made the shift from an SUV that get's 11 MPG to a crossover that gets 30 MPG".   I'm still in the process of shopping around to make that shift...but I have some time since summer is here and my Harley gets 44mpg!

Dave

Following a recent posting on a SOA success story http://blogs.oracle.com/davidchappell/2008/04/soa_helps_you_move.html , I got some important feedback http://tech.groups.yahoo.com/group/service-orientated-architecture/message/10092 on how the writeup was a bit slanted towards highlighting the use of the tools and infrastructure such as BPEL and ESB, and didn't talk enough about the services themselves.   The point being made that Service Oriented Architecture is about service orientation, and all too often the folks who live and breath the supporting infrastructure tend to gloss over the part about the services themselves.

I plead guilty.

So as part of my retribution, I would like to provide some nuggets of wisdom on service design that came out of the Move project -

The Prospect to Cash business process includes a number of very large applications (PeopleSoft, Siebel, Oracle Customer Data Hub, and a variety of fulfillment applications).  CSC used the following criteria to identify service candidates:

- Is there similarity in the type of data that needs to be passed to various applications?  If so, then the business logic in each respective application that operates on that data is a candidate for service enablement.  The resulting collection of related services are also candidates for participating in a canonical data model.

- Does the business require part of an application to be used by both internal as well as external customers and partners - what are those parts and can they be broken into smaller atomic functions?

- Here is one that is bound to be a subject of discussion - A key service design consideration is that there is no direct interaction between two services.  They are self sufficient atomic functions that can be used to build orchestrations by the business e.g. - customer data needs to be created in Siebel, PeopleSoft, Oracle Customer Data Hub and the fulfillment system. Each one needs a different component of the data.  This raises a question - is "Create Customer" a service or should it be broken into "Create Customer, Create Address, Create Contact, Create Fulfillment Profile"?  The answer largely depends on whether each of these services is complete in itself, and depends on the nuances of the business.  On first thought, for instance, it may appear that an address without a customer may not be very useful. But if you are someone like Move the rules are different in this case. When they display a house on their website, they don't display the name of the person, they only display the address and the fulfillment profile. So for Move it makes sense to have them as separate services and business can decide how to string them together via BPEL to achieve business results they are intending to achieve.

A similar example can be given about the external customer who needs to consume these services.  If you want to update some information about your house, there is only specific information you can update.   So that would help determine what the service interface would look like.

The overall experience from the team about building the SOA at Move, Inc is that it is very easy to get carried away into JBOWS (Just a Bunch Of Web Services) mentality, without having proper checkpoints in place in order to ensure that the right services are being built for the right reasons.  To do it right really requires proper analysis and a "business case" for each logical function to make it candidate for being Service. If you follow this practice judiciously, you will not end up with JBOWS.

Dave

I'm presenting a keynote at the next International Association os Software Architects (IASA) IT Architect conferences http://www.iasahome.org/web/itarc/nyc/topics#Keynotes on May 22 - 23 in New York City.

I was looking through the agenda and I came across this - Interesting Real-world Architectures and the Handbook of Software Architecture presented by Grady Booch via Second Life

I checked with the conference organizers, and sure enough, Grady is going to be at his home base (which is usually Hawaii), and broadcasting the presentation via 2nd Life, and conference goers will be witnessing his avatar giving the presentation on the big screen.

How cool is that!?  I want some!

Dave