Tuesday Sep 23, 2008

Book review: Symmetry, A Journey into the Patterns of Nature

Last month I was set to spend two weeks along the headwaters of the Saint Lawrence River, on holiday. Of course, this required suitable reading material. On impulse I grabbed Marcus du Sautoy's book on Symmetry from the "new books" section of one of my local libraries. The title intrigued me, although I knew nothing of the author.

My impulse was a very good one. This book is a wonderful exploration of the mathematics of symmetry. It has two strong threads, historical and personal, wonderfully interwoven. For the most part it is light on the actual mathematics, which I found somewhat disappointing (I am an electrical engineer by training, so I have a strong background in applied maths), but is very clear on the key mathematical elements, without being complex, confusing, or boring.

For me the strongest thread was historical. du Sautoy relates the past history of his discipline with great skill, and genuine enthusiasm. A close second was the more recent history, mixed with the author's own personal involvement with key figures in recent advances in the field. The combination was gripping, and a thoroughly enjoyable read. It also is very suggestive of how God has structured the universe in ways that are both understandable and incredibly creative. A highly recommend read!

OpenESB about to hit a major milestone

Any parent will tell you there are major milestones that you, as a parent, mark in your child's life, like when she gets her first tooth, or first goes off to kindergarten (or college!). These moments can fill you with pride, and excited anticipation for what's next.

It is with similar parental feelings that I can announce that Project OpenESB is about to hit a major milestone in the life of an open-source enterprise software project: a commercially supported version is to be introduced. This indicates the the project is both mature enough and stable enough to be widely "used in anger." After about three years of development, project OpenESB has "grown up" enough to mark this important occasion.

Sun will be introducing a branded version of OpenESB, to be called GlassFish ESB (running atop the GlassFish v2 application server, so the name is quite descriptive). GlassFish ESB is essentially Open ESB (including most of its components), fully supported by Sun, with the 24/7, one-throat-to-choke type of support a lot of enterprises require for key production systems.

I know some OpenESB users have, in the past, requested this kind of support, and some have even gone into production without it, so I've known for quite a while OpenESB was ready for milestone. Congratulations to the whole OpenESB community! You make me proud. And I know this is only the beginning.

Monday Jun 30, 2008

Is JBI an ESB?

Does the JBI standard define an enterprise service bus? In a word, no.

In his book, Enterprise Service Bus, JBI expert group member Dave Chappell presents a very useful model of what an ESB is, and what it does. One of the major features of his model is the service container, a node on the bus where services are provided and consumed. The service container includes integrated applications, of course, presenting them to the bus as services invoked by XML message exchange. In Dave's model JBI is a technology for creating such service containers from standardized pieces (namely the NMR and the plug-in components).

JBI was deliberately crafted to support multiple approaches to building an ESB. This has resulted in some quite different approaches. Some ESBs use multiple JBI instances linked by JMS message queues. Others use multiple replicated JBI instances running together in a cluster, providing in essence a large, highly available and scalable virtual JBI instance. Still others approaches exploit more elaborate ESBs, using JBI to create highly flexible, componentized service containers.

Okay, this is all stuff that was published three years ago, so why am I bothering to write about it today? I have seen recent reports of certain ESB vendors running around claiming that JBI is a terrible ESB, saying things like "the NMR is a single point of failure," and "it is a hub-and-spoke architecture, not a bus". This either reflects a genuine misunderstanding of what JBI is (which seems unlikely after three years), or an attempt to spread self-serving lies and old-fashioned FUD.

So if you are contemplating buying an ESB that supports JBI, please look closely, and educate yourself about the benefits of using open standards, especially in avoiding vendor lock-in. And please, don't take too seriously any "technical advice" about JBI coming from a company that doesn't support JBI. The chances are, they just want to lock you in to their proprietary ESB, and they won't let the truth stand in their way.

Monday Apr 28, 2008

JavaOne 2008

JavaOne is coming soon: next week! This annual gathering in San Francisco has been a tradition I've kept for more years than I care to remember. This year I am presenting, along with Scott Morrison of Layer 7 Technologies, a technical session entitled "The Third Leg of SOA: Identity." (IIRC, it is scheduled for Thursday May 8th, at 1:30 PM.) Scott has incredible expertise in SOA security and governance issues, and very deep experience solving these problems using Java. Together we'll explore some of the issues and solutions needed in securing and governing SOA systems in general, using Java technologies and some very interesting architectural patterns. If you've got an existing SOA system (or at least some pieces you are assembling), or are contemplating the jump in service orientation, and you happen to be attending JavaOne, please check out our session. You'll learn some important lessons about SOA security and governance, and get to admire our Really Cool Graphics :-).

Thursday Feb 07, 2008

BPEL4People and WS Human Task submitted to OASIS

If you've used BPEL in anger, you know that it has limitations. One of the major ones that I've encountered is the lack of work flow support. Real business processes often involve human actions, and humans don't really look like web services! Also, web services don't perform roles in the same way humans do; there is no question about assigning a work item to a web service, whereas work item assignment for people can involve some complex considerations, including organizational structures.

BPEL4People and WS-Human Task are a pair of specifications that have recently submitted to OASIS for standardization. These important standards address many of the work flow-related problems related to using WS-BPEL, providing a standard way of modeling and executing business processes that include so-called human tasks.

One of my first comments to the WS-BPEL TC, when I joined it way back in 2003, was that BPEL needed to accommodate assignment of work items to humans. Unfortunately this was dismissed as "out of scope" (and that is probably a good thing, or the TC would still be active today!). I am gratified that other participants in the BPEL community realized that this was a real problem, and I'm grateful today to join them in submitting these two new specifications for standardization.

Friday Sep 07, 2007

Security leads to insecurity?

My spiffy new XP box came with all the security knobs & dials set to "on" (or, perhaps, "11" :-) ). The net effect of this well-meant reconfiguration was the opposite of what was intended -- an extremely insecure PC![Read More]

Thursday Aug 02, 2007

SOA lessons learned from business process automation

Business process automation taught us to separate business process logic from execution of business activities. Today, it teaches us about reuse of services in SOA-based systems.[Read More]

SOA services != RPC

So where do services come from?

That's actually a more profound question than it appears at first blush, especially if you are a hands-on, code-first sort of developer like me. Profound, for it goes directly to the value (potential value, perhaps) of a system built using a service-oriented architecture.

What is the value of SOA? Reuse. That has always been its strength, or at least its promise. But this is not the same as code reuse, the holy grail of software design since the invention of the subroutine.

Where then, do services come? We must look not to code, or other artifacts of service implementation. Instead, we must look to business processes. The elements of reuse in a business process are business functions. Reusable business functions are what enable the quick, reliable creation (or modification) of business processes possible. Business agility (I hate the phrase; it sounds like market-speak) depends on such services. Shorter time to market, high quality, avoiding any code creation: all things that help create business value.

How do you create such services? Simply by looking at business processes, mining the common activities that comprise them. Do some processes involve a credit check of a customer? Sounds like a candidate for a service. Notable SOA thinkers have developed nuanced models of services in a layered fashion (John Crupi's "lasagna" model, for instance), but the basic principle applies: the high-value reuse occurs at the business process level. It's what makes modern SOA worthwhile, and not simply a language-neutral way of packaging libraries: reuse at the service composition level.

This is in sharp contrast to how a lot of services are identified and implemented today. Many developers, with a very code-centric world view, very naturally look to their available reusable code assets (libraries, APIs to useful applications, and the like) when then look to service creation. After all, these are proven candidate for reuse already, and wrapping them with more technology-neutral interfaces (web services and the like) should make them even more reusable. Very logical.

Logical, but off target. This completely misses the value of reusable services, as discussed above. Repackaging code assets is really an exercise in RPC. Does it create the benefits of reusable services that can be used for process composition? No, for such code assets are the fine-grained stuff from which such business functions \*may\* be created, but aren't those business functions themselves. It is rather pointless to use a lot of RPC to write somethng that is more easily, and efficiently, realized in native languages (C++, Java, etc.).

So what is the lesson here? As the title of this piece says, RPC is not equal to SOA services. Services must be designed from a different centre than code reuse. Process composition is the proper centre of service design.

This is, in many ways, what the business process automation folks have been telling us for years, but that is for another entry.

Monday Jun 04, 2007

ZapThink should rethink SCA and JBI

Jason Bloomberg has made some rather unkind remarks about SCA and JBI. While he is entitled to his opinion, his remarks are not well supported by any arguments or statements of fact that puts forward. Although he asserts that there must be a better way to do SOA, he is remarkably silent about what that would look like. He should share any constructive ideas he has on this subject. He should also take a closer look at the specification; he will find many things, including the fact that JBI and SCA are complementary technologies.[Read More]

Tuesday May 15, 2007


Okay, I confess. I am very interested in computer language parsing. I've written parsers by hand, including one for a full-blown macro assembler. It's still something of a hobby of mine, and has been since the 70's.

So it wasn't surprising that, way back in 1989, I read with great interest an article in the ACM SIGPLAN Notices: the user manual for something called the Purdue Compiler Construction Tool Set, PCCTS 1.00B. This described a compiler generator that differed from things like LEX/YACC in a very important way: it generated vertical-descent parsers, just like a Real Human Being would (and like I had been doing for years). This made understanding the generated code very easy, and debugging the input grammar (which was converted into the parser code in a straightforward way) much easier than with YACC.

In 1989 I used PCCTS to create a mini-language that was used to describe high-level behaviours in a GUI framework I was creating at the time. (The language was actually part of the UI resources, allowing the developer to change UI behaviours without touching his C++ code.) PCCTS turned out to be an invaluable tool, since it let me create my mini-language without having to hand-craft the parser, which is a tedious, error-prone activity. Instead, I maintained a simple grammar description. As we gained more experience with the UI framework (used to create a family of interactive graphic language editors), I was able to adapt and extend the language to add more high-level behaviours, allowing the UI developers to specify what they wanted in a declarative fashion. It was very cool, and very effective.

Since that first exposure I've been following PCCTS, which, when it migrated to a Java-based implementation, became known as ANTLR (ANother Tool for Language Recognition, pronounced "antler"). ANTLR 2.0 was a major leap forward in capability (LL(k) parsing, as opposed to the LL(1) parsing that PCCTS 1.0 supported). I've used ANTLR periodically over the years. One interesting use was to take a text-based memory dump (from the Forte 4GL run-time), and convert it into HTML pages, making it much, much easier to explore the memory (hyperlinks are a lot easier to use than grep!) Of course, roll-your-own mini-languages have been rather fun to create, with the aid of ANTLR, over the years, and have crept into my work when need. (The Open ESB WSDL 2.0 API was generated from a high-level API description.)

Now, nearly 20 years after 1.00B was published, ANTLR 3 is nearly ready. This introduces LL(\*) parsing, another leap forward, allow arbitrary lookahead while parsing, while still allowing the parser to be efficient at run-time. I've been beta-testing a new book describing ANTLR 3, and it is a very nice exploration of language theory, practical parsing (and parser generator) issues, and the ins-and-outs of using ANTLR 3.

So if you're like me, and have an interest in language parsing, check out antlr.org. Tools, grammars, full source code, you name it. Enjoy!

Tuesday Apr 24, 2007

What makes a service reusable?

A typical scenario: a department in a company deploys a service into its production environment. It turns out the service is quite useful, and gets reused in a variety of business processes supported by the department. We have a success story, a poster child for SOA adoption, right? All too often the answer is "wrong!"

Why wrong? Just wait for the second shoe to drop... the service is too successful, and is reused by consumers far and wide in the company. Suddenly it slows down, adversely affecting business processes. People get grumpy, and fingers are pointed, and emergency plans must be hatched to scale the service to cope with its own success.

This scenario plays itself out every day, and it points out a lot of different deficiencies that all point to the need for both design- and run-time governance of any SOA system. I'll save design-time governance for another day, and dwell on some aspects of run-time governance of the system.

First, a definition, then an assertion. A policy enforcement mechanism (PEM) is a mechanism in the SOA infrastructure where run-time policies (security, SLA and the like) are enforced. The PEM can enforce access controls, ensure schema compliance, guard against over-use of a service, load balance, do content-based routing and much more.

Now the assertion: a service isn't really reusable unless it is offered through a PEM. The PEM gives you operational control of the service that is completely unrelated to the logical function provided by the service. SOA enables reuse, but unregulated reuse can result in an inability to control that reuse in any fashion. In some environments that might be just fine, but in most commercial environments such a collegial situation is rare, and the risks of suffering a meltdown (as illustrated in the little scenario above) quite real, and very costly (not to mention career-limiting :-) ).

This bears restatement: a service offered without run-time policy enforcement mechanisms is not truly reusable.

Regulating reuse is just the tip of the iceberg. It is an instance of a more general problem of ensuring that services comply to all applicable policies within the production environment. Examples include auditing requirements, privacy controls (e.g., all SSNs are to be encrypted), and even service mediation (versioning, or perhaps protocol). And just to make things worse, compliance is a moving target: legal requirements change (both over time, and by legal jurisdiction), as do internal policies.

This sounds like it is pointing to a massive application maintenance headache. How it the world do I write services that comply to all these extra, changing requirements, when it is tough enough just getting the core service functions right in a timely fahsion? How do I respond to what are operational issues and needs from within the normal software development life cycle?

Fortunately, there is a better way than brute-force, code-first solutions. The good old trick of "separation of concerns" is very handy here. Our PEM can be completely removed from service providers, and placed into a Policy Enforcement Point (PEP) in the SOA infrastructure. This separates the concerns neatly: the service provider concentrates on supplying the core service, and nothing more. The PEP "protects" that core service by acting as an intermediary between the service provider and the service consumer, enforcing all applicable policies so that the provider will only receive messages that are compliant. Further, the policies enforced by the PEP can be changed without affecting the service provider, allowing policies to be changed quickly and frequently, and completely outside the service's software development life cycle.

The service consumer, obviously, deals not with the service provider, but the PEP. The service offered by the PEP is often referred to as a "virtual service".

There is a lot more to be said about this. If you are interested in practical, policy-guarded service reuse (and that should be most of you SOA folks), and you are attending JavaOne in May, please attend TS-8459, titled Service Virtualization: Separating Business Logic from Policy Enforcement, Wednesday May 9th, at 6:35 PM. I'll be co-presenting this talk with Scott Morrison, from Layer 7 Technologies, who has a lot of experience with service virtualization in SOA systems. I hope to see you there!

Wednesday Apr 11, 2007

Relationship of SCA and JBI

There is an interesting white paper at the OSOA web site entitled "Relationship of SCA and JBI". Given the confusion about this subject in the recent past (which I have commented on), I'm glad to see something that is more informative on this subject, published by the Open SOA collaboration, which includes Sun, IBM, Oracle, SAP, BEA, and others. (Check out the Open SOA site for details.

Some interesting bits:

"Both SCA and JBI are focused on helping developers construct and compose services so it is natural to assume that they are competitors; however, this is not the case. While both can be used separately, they are also effective to use together because they focus on different aspects of the larger service composition problem.

"The reason this is possible is that SCA and JBI target different audiences. The SCA specifications target end users. They describe how to implement, assemble and deploy applications composed from services. As such, the target audience for these specifications is people working in the roles of developer, assembler and deployer respectively."

That pretty well summarizes the situation, as it exists with JBI 1.0. One of the possible outcomes of JSR 312 (JBI 2.0) is an improvement in the alignment between the two technologies. Such an improvement would allow the technologies not only to co-exist (as is the case today), but also would eliminate what small "impedance mismatches" exist in the areas of deployment and metadata.

Wednesday Apr 04, 2007

JBI 2.0 JSR Filed

Just a quick update -- the JBI 2.0 Java Specification Request (JSR) has been filed with the JCP, and is being voted on by the JCP executive committee in charge of Java SE and EE. (There is a separate EC for Java ME.)

If you are involved with JBI, especially as a user of the technology, this JSR is for you! Any 1.0 technology needs to grow and improve, and those improvements are directed by the experiences of users, not implementers, of the technology. That's one of the best features of the Java Community Process -- the whole community can get involved, and help shape the technologies. And that is much better than the alternatives...

Wednesday Mar 14, 2007

How to learn WS-BPEL 2.0?

Nearly four years ago I joined the WS-BPEL 2.0 technical committee, and from nearly the first day people have been asking me, "how do I learn BPEL?" At long last I have a good answer -- the NetBeans 5.5 enterprise pack (EP). The EP includes an integrated version of Open ESB, including its WS-BPEL 2.0 service engine.

The EP includes an intuitive interactive graphical process diagram editor. If you really like the XML language the TC crafted, there is two-way editing (between the graphic and text views of the process). This makes writing process definitions easier, since it handles a lot of the "boilerplate" XML, and makes it much harder to make a mistake. This can make learning the language syntax less painful, and shorten the learning curve. (The EP includes WSDL and XML Schema editors with similar capabilities, making these related document types easier to learn and create.)

Syntax isn't the whole story, though. What about the run-time dynamics of WS-BPEL? How can you learn them? How can you gain insight into the internal workings of the language? The best way I've seen to date is to use the EP's BPEL debugger. This allows you to do all the things you'd expect: stopping process instances, inspecting variables, setting breakpoints, etc. This lets you see what is happening "under the hood" of the BPEL service engine, answering the common question "why did it do that?" in a way that is very instructive.

If you are curious about WS-BPEL 2.0, check out the Enterprise Pack, and take the BPEL debugger for a test drive.

FYI, the WS-BPEL 2.0 service engine itself comes from Open ESB 2.0.

Tuesday Mar 13, 2007

Open ESB 2.0

It is with considerable pride that I can announce that the open-source JBI project Open ESB 2.0 beta is now available. This release includes a bunch of JBI plug-in components that provide a lot of interesting functionality. This also includes extensive design-time tool support in the form of NetBeans 5.5 modules (including the NetBeans Enterprise Pack).

The list of components that come with this release is impressive:

  • Intelligent Event Processing (IEP) service engine. Very sophisticated business activity monitoring.
  • WS-BPEL 2.0 service engine. Stateful business process automation. Very cool interactive debugging capabilities.
  • Java EE 5 service engine. Two-way integration of Java EE technologies, by means of JAX-WS. Your Java EE assets can provide services to other JBI components, as well as consume services they provide.
  • XSLT service engine. XML transformation.
  • JDBC service engine. Database access via service wrappers. Good for resource-layer services.
  • HTTP binding component. SOAP and REST protocols using a very high throughput HTTP implementation.
  • JMS binding component.
  • File binding component.
  • FTP binding component.
  • MQSeries binding component.
  • SMTP binding component.
In addition there are tools for creating composite applications and administering the ESB; everything you need to create, deploy, monitor and control your SOA.

This release represents a huge contribution to the open-source integration & SOA communities.




« July 2016