WebLogic Integration (WLI), Web Services and SOAP and REST (Part 1)
By jeffreyawest on Apr 20, 2011
RESTful web services have been around for some time, but it seems that the adoption of REST has increased significantly in the recent past. When I was a consultant at Accenture, I had heard that people were interested in REST, but I was part of the SOA group at Accenture, and I didn’t really pay much attention to REST because I wasn’t quite sure what I would do with it. WebLogic 11gR1 PS3 (10.3.4) now supports JAX-RS 1.1 using the Jersey reference implementation, and in order to learn more about Java EE 6 and how it is being integrated with WebLogic I decided to see what it was all about.
Before I dive into REST I felt I needed to give you a bit of context on how I see the world of web services. I could simply proffer an opinion on REST and SOAP but without this context and perspective how would you know if you should be interested in what I have to say? (especially with so many opinions on the matter already published)
My First ‘Web Service’
My first ‘Web Service’ was a Java Process Definition (JPD) in WebLogic Integration 8.1. I thought WLI was one of the best integration tools available and I still look at Enterprise Architecture projects in WLI constructs. To me, WebLogic Integration represented the ‘right’ way to do SOA. It forced you to perform an analysis on your business objects, model them as XML and put them in a ‘Schemas’ project and then create your request and response XML schemas that were associated with a JPD. Coming from a background of core Java development, this made perfect sense to me and it made it easy to get started with Web Services. If you needed to create a WSDL to share with a consumer of your service you could right-click, choose ‘Generate WSDL’ and BOOM – you were done. I could even deploy the process to my local server and have other developers use my service while I was still building it. It was great, both as a developer building a service and as a team lead assigning responsibilities for web service build.
However, there was this SOA thing that I was supposed to know, being in the ‘SOA Group’ and all. I had made a ‘Web Service’, but was this really a ‘Service’? Then I learned that you could expose multiple JPDs via a Web Service Control with a JWS. This allowed you to create a ‘Customer’ service where you could aggregate your Web Services like getCustomer, updateCustomer, createCustomer, etc.. This seemed to be what SOA was all about – I had a resource, ‘Customer’, and all operations that were relevant to this resource could be found on the ‘Customer’ Service. If I wanted, I could even look up difference services in a Registry and find one that suited my needs. Fabulous! I was told that everything that I built should be stateless and synchronous because stateful was harder to work with (Me: Sure, I can do that…).
Everything I Built was Synchronous
One thing that I didn’t use enough in WLI was the Message Broker Channels. This was simply (or not so simply) an optimized JMS system within a WebLogic cluster that allowed for Asynchronous messaging (which as I learned later was essential for scaling applications, or at least accepting more messages with the same resources). My first experience with Message Broker was a JPD that was initiated via a Message Broker receive (this was essentially an MBD). As a Developer, I didn’t really understand what that meant, but I had my XML request bean that I was still fascinated with and I could write my JPD. My processes never really went to sleep, except in my naiveté as a ‘green’ developer when I put in Thread.sleep() statements (not good) so everything I built was synchronous.
It wasn’t until I wanted to place a ‘Control Invocation’ after I had sent my XML response that I began to learn want ‘Asynchronous’ meant. To me, as a Developer, it was a pain in the butt. I just wanted to print a log message after I returned my response, which the IDE let me do – so I did it. Then, I had the WebLogic admin ask ‘Who created another asynchronous process?’ and I sheepishly raised my hand. I was admonished that I needed to let them know when I did this so they could create the corresponding Database tables and JMS Queues (Me: Sure, whatever – I can do that). But I still didn’t appreciate the value of asynchronous and long-running processes.
Long Running Processes & Threads
After I progressed in my career and did more team-leading and architecture type roles I was ‘educated’ on what it meant to have Asynchronous and Long-Running processes as part of your Enterprise Application architecture. If I had a synchronous process then I needed a thread available to service the request and then provide the response. That meant if I had 1,000 requests come in and only 50 available threads then some of the service consumers would just have to wait. And, what if we only had a database connection pool that was limited to 25 connections? At that point we have 950 requests waiting for a thread and 25 waiting for a database connection. Again, in my naiveté I asked: So what? Make it bigger!
It was at that point that I began to learn the relationship of Threads, JVM Heap and CPU. If you needed lots of threads, then you needed the Heap and CPU power to support them (Me: Yeah, and…?). These are not cheap. You can’t just add another server in the datacenter when you want, plus, what happens if we have a really big spike in load? How could we support it? Servers can get hot and heavy, but they don’t produce offspring…
Everything I Built was Stateful (almost)
At this point, I was told that I should make as many of my processes Asynchronous, and only make them Synchronous when I really needed to (Me: Yawn… sure, I can do that). It wasn’t until I was involved in load testing that I figured it out. If we sent in 1,000 requests for a process at a time, we could then see how they flowed through the system. We had a certain number of threads receiving requests and then a separate set of threads that processed the requests. We could then tune the number of consumers or simultaneous process instances to reach a reasonable CPU utilization, and we could also match the number of database connections to the number of threads in the thread pool such that they never waited for a connection. It was at that point that SOA and the role of an Enterprise Architect started to make sense.
Stateful and Stateless Together
I was working at an MVNO (like Virgin Mobile) where the WLI application was responsible or integrating with a service provider to provision new mobile phone subscribers. There were two steps: 1) reserve a telephone number and 2) activate the account (this is the part where Sprint, AT&T or whoever tells you that it may take 4 hours to activate your service). Step 1 was fast – 1-2 seconds and step 2 took several minutes. Part of it had to be synchronous because the salesperson at the register needed to program the telephone with the lock code for that specific phone, the Mobile Device Number (MDN, or phone number) and the Mobile Identification Number(MIN). The faster that we could make that part the happier the customer and salesperson were and the more customers we could process.
We started out with the whole process being synchronous, and when we did load testing it was clear that this just wouldn’t work. To complicate matters, we could only have five simultaneous active connections to our service provider that we had to use to reserve and activate accounts, so a peak-load scenario would crush our system. We ended up splitting the provisioning into two parts – step 1 was synchronous, stateless and fast (customer was waiting at the register) and step 2 was stateful (customer was driving home). We could then reserve 2-3 connections for the requirements of our synchronous step and only have 1-2 for the asynchronous part. Not only that, we could develop algorithms that changed the balance depending on the load of each type. It was pretty slick.
This is not REST
No, its not. This is the context for my opinion on REST and its place in Enterprise Applications .
Looking around on the internets there are plenty of opinions on REST and how SOAP is dead, so is SOA, REST is going to replace SOAP and so-on, and so-forth. For the record: SOAP is not dead, and REST is not the answer. SOAP is not going away any time soon, and at this point REST does not offer what SOAP does, nor should it.
What is REST, anyway? It is am architectural style of exposing URI’s that represent resources, accessing them with a synchronous, stateless protocol (HTTP) using simple verbs: GET, PUT, DELETE, etc. That’s it - its really that simple. REST is an approach to exposing URIs that represent RESOURCES.
REST is an additional tool in the toolbox of an Enterprise Architect. A good Enterprise Architect and team will look at the different tools that are available to be used, understand their value & their place and then use them in the appropriate manner when the situation calls for it. Let’s look back at my ‘Customer’ service that I built. The only thing that I did there was create CRUD methods on a resource. Is this the right place for REST? By the letter of the law for a RESTful approach, sure! ‘Customer’ is a RESOURCE that I want to expose externally over the network and I can do so with a URI. Done! (but not really).
What about the mobile phone subscriber provisioning problem? Well, you could choose to ‘GET’ a phone number in order to reserve it, but what about the Lock code for the phone and the MIN? I suppose I could create a ‘Resource’ that represented a Reservation, but do an MDN, MIN and Phone Lock Code all really belong in the same object? Am I ever going to do a PUT operation on that ‘Resource’, and does the ‘GET’ operation on that ‘Resource’ result in complex application logic, or does the operation simply return a copy of an Object? Seems like business logic to me…
But Hacking is FUN!
Like every new, shiny tool or approach REST is already being over-applied in scenarios where it’s not welcome. You can take any architecture or programming language and hack the crap out of it to bend it to your will. That is, after all, why we’re developers right? Plus, by doing this we’re driving innovation! We know firsthand that users never use the applications or tools we provide in the way we expect them to, so why should we?. It is THIS – this rebellion – that drives the industry forward, creates new opportunities and necessitates that new frameworks be created to be mis-used again (until we reach the ultimate goal of engineering software that doesn’t exist). The question is not whether or not the technologies behind RESTful web services will be applied in a way that breaks the ‘RESTful’ approach (I’m sure they already have), but ‘when’ & ‘how’ it will de done and whether or not we will be better off for doing so.
My advice is this: Innovate and push the boundary on your own time. Doing so in the context of an enterprise project puts the project team at risk and has the potential to risk hundreds of thousands of dollars in project delays and poor Reliability, Availability, Scalability and Performance. I’ve had to clean up behind too many cowboys that thought they were going to create something beautiful only to see the dung pile left in their wake. Yes – we need the innovation but we also need to minimize the risk!
What’s Part 2?
Part 2 will be coming in the next week or two, but I think REST challenges some traditional assumptions for enterprise architecture and I’ll be blogging about some of these challenges from my perspective going forward.