BPM, E2.0 @Oracle

This week’s post brings us a republication of a post from GeraldK.

In a recent article, economist Tim Harford summarized some of the key ideas in a paper by Paul A. David, "The Dynamo and the Computer: An Historical Perspective on the Modern Productivity Paradox". While I'll further distill it, I really encourage you to read one or both of the pieces.

I want to start with the idea that new technologies, though they may have the potential to massively and positively transform institutions, and although it may even be envisaged how they may do so, nevertheless take far longer to do so than their proponents expect, even in the face of years of high levels of investment in the new technologies. Professor David's paper was written in 1990 as an investigation into the question of why, as the Nobel Laureate economist Robert Solow remarked, "we see computers everywhere but in the productivity statistics". (This problem, the "productivity paradox", persists today; in the last handful of years we have seen some increases in productivity that are thought by some to be attributable to technology investments of the preceding decades.)

David gives us a history of the electric dynamo and the electrification of factories (you really should read the paper; at least read section II) as an analogy to help us think about the possible future of computers. While he talks about some of the specific changes that electrification wrought on factories, he doesn't go into specifics of what computer technology may do and how it might do it. Well, he's an economist writing 17 years ago, and I'm a (hmm...let's go with...) technologist and visionary with a blog today, so I'll be your huckleberry.

Electrification did little to change factories and work practices for decades, and there were many reasons why. The one that that is most analogously relevant is this: The major benefits of electricity didn't come from replacing steam with electricity per se; they came because electricity allowed "unit drive", which could let you run a factory in a cheaper, more efficient, higher-quality way than was possible with "group drive", which was steam engines limited you to. Electricity allowed you to go from a steam Factory 1.0 to an electric Factory 2.0. (Two examples of the differences: 2.0 buildings could be smaller, lighter, single-story structures because shafts and belts could be replaced with wiring, which was both lighter and allowed more flexible placement of powered machinery; and 2.0 factories could be more efficient because machines could be laid out according to flow of materials and tasks, and furthermore easily rearranged according to process changes.)

But even though the first central electric plant in the country had been operating since 1881, it wasn't until the 1920s, four decades later, that factory productivity growth started to improve. What happened? Well, there were all these existing buildings and machines and training and instructions and processes and institutions and relationships and habits that were all based on the steam-powered Factory 1.0. Even when they electrified an old steam factory, and even when they built new electric factories, they would simply replace the steam engine with an electric dynamo, and would end up with an electric Factory 1.0, not a Factory 2.0.

We've done similar with computer technology. Yes, we have email, and email has substituted as a better and cheaper memo and mail and bulletin board and meeting and runner and pneumatic tube. And yes, we have web sites, and they are a lot better than bulletin boards and forms and catalogs and brochures and manuals and calling up a rep to ask a question. But they do the same thing. Yes, better and faster and cheaper and with much greater reach, but it's the same thing. Going further back, pre-networking, computer reporting and printing was more efficient than accountants with (paper) spreadsheets and ledgers and typing and stenography and typesetting, but it was again the same thing. (Now I'm not discounting that at some point, speed and efficiency are quantitatively so much greater that in itself induces qualitative change--email has arguably reached that point--but that's an emergent rather than a contemplated change.)

I say then that Web 2.0 consists of those applications that not only use the underlying technologies of computers and networks and HTTP and HTTP-stuff; but furthermore and more saliently do so in ways that are particularly well-matched to the routines and practice of Org 2.0 organizations. (I know people sometime use "Enterprise 2.0", but I actually want to include organizations that may not be enterprises.) What's Org 2.0? Well, let's talk first about Org 1.0. Org 1.0 has a "traditional" corporate form: driven by standardized operating processes and organized around departments dedicated to performing those processes. Many organizations (or parts of organizations) at least aspire to this form, analyzing and optimizing business processes, doing TQM and ISO9001 and Six Sigma, installing whatever software brings them closer to the ideal. The 1.0 organization has persistent, determined outcomes, expected inputs, and reasonably well-known tools and practices; the main challenge in Org 1.0 is coordinating the inputs and tools and practices to achieve those goals effectively and efficiently. Configuring and ordering a PC on Dell's website, assembling a car, getting the right amount of beer and iPhones to the retail stores, checking out at the supermarket; those are the kinds of things that Org 1.0 is good at doing.

The key is that Org 1.0 is dedicated to repeating its outcomes, perhaps better each time, and with parameterized variations, but repeating. In contrast, Org 2.0 might be doing something new all the time; that something may be similar to what it may have done before, but not in any easily-formalizable way. I made a chart.

Table 1: I need to fancy this post up for you people with short attention spans.

Now, it's often assumed that what I've called Org 1.0 is actually just how you have to run a large organization, to be able to scale up without screwing up (after all, "repeating" is pretty much how a company scales up); and what I'm calling Org 2.0 is characteristic of small organizations, who just don't have the scale or the institutional knowledge to implement structured repeatable processes. Small firms that grow become more structured and more process-oriented: because they can, as they learn from what they do and standardize that into repeatable processes; and because they must, as informal mechanisms of communications break down when there are lots of people, and formal processes can ensure that individuals get the information they need to do their jobs.

That's true, and Org 1.0 represents the pinnacle of industrial achievement. The only way the world economy can work is with organizations dedicated to scaling up, in predictable and efficient and repeatable ways. Much of what Org 2.0 does is as part of or in direct service to a bigger Org 1.0.

But the thing is, in gaining all the scale advantages of Org 1.0, an organization loses the great advantages of Org 2.0: flexibility and adaptability, resulting from the high level of communication and overall situational awareness of each member of the organization. Yes, Org 1.0 realizes economies of scale, but at the finer grain, Org 1.0 is in fact much less efficient than Org 2.0, and the economies of scale have to offset the diseconomies. (That's something a lot of small and mid-sized companies often forget, btw. In the drive to formalize to be able to grow, they smother their own advantages.) That's why a lot of companies that work in Org 2.0 ways (architecture firms, law firms, creative agencies, research labs, medical specialists) are either small, or decentralized collections of small groups. That's why large organizations often break off smaller, less formal groups for innovation.

Ah, and here's where Web 2.0 comes in. Web 2.0 applications are those applications that are meant to help Org 2.0 to scale up, take advantage of specialization, and stop being so small. (Here I'll add something to Gerald's post and say that Web 2.0 applications can also help the already large ones i.e. Org 1.0 reap the benefits of Org 2.0). They help with Org 2.0-specific activities and share those characteristics: informal communications and collaboration, ad hoc inclusion of people and information, structured in ways that are easy for humans to understand (say, by faces, or by event, or by conversation, or by relationship) and synthesize. They do this so that the efficient working habits of a small group can be used by larger groups of people who are farther apart, so that we can get economies of scale, without the heavy diseconomies. They try to do this by improving communications and the sharing of information that works in small localized teams, so that it can work across larger dispersed groups of individuals. The point is to take advantage of what computer and networking technologies do to remake how and organization can operate, rather than to continue to operate it the same way, just with more speed or more cheaply, just as electrification and unit drive remade how a factory worked.

And so that's what we're trying to do. Just as animals and steam and electricity augmented human muscle, and earlier computer software and processes augmented human logical thinking and computation, Web 2.0 applications are meant to augment those unique human abilities that can't be automated. We're trying to produce applications that increase the power of human judgement, cognition, social understanding, inspiration, and adaptability.

I’m sure a lot of you remember the movie Dude, where's my car? and one of its famous lines:

- Q: What is the Continuum Transfunctioner?
- A: It is a very mysterious and powerful device and its mystery is only exceeded by its power.

While researching BPM on the Internet, I came across a very interesting challenge set by Ismael Ghalimi of Intalio. The whole thing started when Ismael argued that BPEL should be the language of choice for BPMN because it is based on Pi-Calculus. After which followed quite a few heated arguments, resulting in Ismael challenging vendors who serialize in anything other than BPEL (read XPDL) to show how they can model this particular process, failure of which everyone must accept that BPEL is the superior language of choice. I presume that the reasoning behind this is that only if a BPMS based on the Pi-Calculus theory (does the Pi-Calculus process engine sound like the Continuum Transfunctioner?) will indeed be able to model and execute this. To which Bruce Silver responded that a few tools can do it, including Oracle BPM (aka ALBPM) and to drop the Pi-Calculus claim. A few responses to his challenge later, Ismael goes on to say that Oracle BPM can do it because it relies on BPMN and BPEL.

So let me start with a couple of disclaimers first to avoid the kind of responses typical of "religious" wars:

1. I don’t know anything about Pi-Calculus so I will refrain from embarrassing myself by talking about it
2. I’m not a BPEL-basher and I don’t view the world in terms of BPEL-lovers vs. BPEL-haters; I take the view such as the one very elegantly described by Sandy Kemsley in reply to Arthur ter Hofstede:"it doesn't really matter whether it's BPEL or XPDL or a proprietary execution language, as long as the model can directly translate to execution".

Now, to clarify one important thing: Oracle BPM (aka ALBPM which Bruce was referring to) serializes the process definitions in XPDL (at least as of 10gR3). As Manoj Das, our BPM Product Manager, noted in the BPMN 2.0 roundtable, in response to a question on the relationship between BPMN and XPDL:

“Since BPMN 1.1 did not specify a persistence format, products, including ours, used XPDL as the persistence and inter-change format for it.”

Which brings us to the challenge itself. Since we don't normally talk about Pi-Calculus, I would have tackled this using both Oracle BPEL PM and Oracle BPM, but I think that 1) the proof that Oracle BPEL can do this has already been provided by Ismael himself (i.e. by saying that BPEL can do it) 2) the gauntlet was really thrown to vendors whose tool(s) serialize in XPDL. I’ll therefore limit my response to Oracle BPM.

Here we go then. Exhibit A, Process P1:

Exhibit B, Process P2:

I’ll now explain the processes. P1 starts with a “Global Creation” activity called “Receive Input from user to start P1”. Now, for some of you who might find that activity not linked to the rest of the process, rest assured. The “Global Creation Activity” has an implicit link to the Begin Activity (indicated by a circle). If that activity is successful, then a process instance is created. If not (e.g. if the user decides to abandon), no instance is created.

The “Start P2” activity is an asynchronous sub-process call to P2. The “Wait for P2” is a notification wait activity. The notification wait forces a calling (parent) process to wait until a message notification is received from the called (child) process through, say a “Send Message” activity (there are other methods to do this but that’s not the point). The remainder of the other activities in P1 are just simple displays of data (without any pretty UI but you'll get the idea).

Process P2 starts when called by P1 and allows a user to enter some data. It then sends a notification message to P1 and P2 ends.

For the purpose of execution, I’ll be using 3 variables, to show their use and modification in the various processes. So, I start by executing the "Receive Input from user to start P1" activity.

After submitting, the process is now in the activity “Add P1 Specific Data”:

I then execute “Add P1 Specific Data” and make some changes:

Now, being a lazy hacker, I created just one role and gave it to the only participant I used in this example but of course one can create more process roles and assign the roles to many more participants. Because of this, I now have 2 work items in my queue:

Please note their respective next activity: “Something is happening here” and “Add P2 Specific Data”. I now execute “Add P2 Specific Data”, followed by “Something is happening here ” (you can execute it in any order):

I finally execute the “Final display” activity:

You can notice that the value of the 3rd variable is passed from Process P2 as well. I believe I have now demonstrated the solution to the challenge.

Continued in Part 2.

Cheers,
Ali

While the increasing attention given to formal mathematical theories by BPM vendors is a welcome development in this field, it is not possible for everyone to understand, much less use them, least of all business analysts or process developers. As a result, it is quite understandable to get confused, especially when seminal luminaries and vendors in the BPM field trade such articles on the blogosphere in what is essentially a tit-for-tat exchange (not in chronological order):

• Why XPDL is essentially useless (Ghalimi)
• BPEL:Who needs it anyway? (Swenson)
• Why BPEL is not the holy grail for BPM (Vigneras)
• Why BPEL matters (Ghalimi)

Before I go on, let me first say that it is great to see the passion from these heavy weights and I personally learned a lot from these exchanges. The objective of this post is therefore to

1. Show that Oracle BPM can actually model parallel business processes which are stored in formats other than BPEL (in this case XPDL) and execute them as well (in response to Ghalimi’s challenge)
2. Reassure existing BPM customers (both Oracle BPM and BPEL PM) who may have become concerned of having chosen the “wrong” engine and therefore being denied the mysterious power of a Pi-Calculus process engine
3. Provide a historical perspective and briefly analyze the various claims made in some of the articles above for potential BPM customers so that they can make an educated choice
4. And finally to provide some words of comfort for business analysts/process developers, who are understandably worried about having to learn Pi-Calculus to before embarking on BPM projects

As such, I feel it is important to provide a historical perspective. In an influential paper written a few years ago, van der Aalst issued seven challenges for those advocating the use of Pi-Calculus. van der Aalst wrote that the “big discrepancy between the Pi-hype and reality will not only limit the applicability of WSCLs (Web Service Composition Languages) but also discredit a beautiful scientific framework like Pi-Calculus”. van der Aalst further went on to propose that a more mature evaluation of languages ought to consider how they support workflow patterns. Here, we have excellent literature to demonstrate Oracle’s richness of offering:

• Pattern-based evaluation of Oracle BPEL (Done by independent party)
• BPM Process Patterns (written by the brilliant Dan Atwood)

However, let’s stick to WSCLs. In another paper, van der Aalst made a comparison of the different languages. In this, he showed a clear distinction of what patterns the languages themselves (i.e. irrespective of products) support. If you compare the standards’ (BPEL, XPDL) support for the patterns vs. what Oracle BPEL or Oracle BPM can support, you’ll find that the products can actually support more than the standards themselves support and there’s a very good reason for this. The products and their process constructs are driven by customers’ requests which generally tend to be more complex than standards and for that we provide extensions to meet our customers' requirements. Additionally, the comparison was done using BPEL 1.0 and XPDL 1.0 beta which are currently at versions 2.0 and 2.1. While I haven't seen any study based on the newer versions of the standards or products, we can reasonably expect improved support for the patterns. So, now that we have established both standards’ support (to a varying degree) for patterns, what are we to make of the strident hype for a process engine based on Pi-Calculus vs. one based on Petrinets? Again, van der Aalst has already provided a most illuminating answer. In this, van der Aalst challenged those trying to link BPML and BPEL to Pi-Calculus and asked "what concepts in these languages can be attributed to things specific for Pi-Calculus?". In the same paper, van der Aalst wrote that “Pi-Calculus [..] is a solid foundation for modeling and analyzing processes. However, I’m not convinced that the features present in Pi calculus are vital for Business Process Management (BPM) systems." Arthur ter Hofstede added in his response to Sandy that “if anything, the most feature-complete semantics of BPEL have been defined in terms of Petrinets rather than Pi-Calculus". You can access Arthur’s work on the mapping of BPEL to Petrinets here. van der Aalst also added in his paper "that there is no clear evidence that Pi-Calculus supports patterns in a better way than more traditional languages like Petri nets”.

The point I'm trying to make is that whether your BPMS of choice uses XPDL or BPEL is irrelevant as both can be mapped to Petrinets and can support the patterns. Furthermore, as Manoj Das, our product manager explained (regarding our future BPM platform) that “there will be different use cases for which the two approaches - BPMN models executed as BPEL and native BPMN execution - will be better suited”. At any rate, as Arthur wrote, there’s so much more to a BPM System than execution languages.

In characteristic scientific humility, Milner himself said that “Petrinets were extraordinarily important development in understanding concurrency”. It is generally accepted that the progress of scientific and human thought in general does not occur spontaneously but rather builds on what others have done previously. As Newton himself wrote in a letter to his contemporary and rival Hooke, "If I have seen further, it is by standing on the shoulder of giants". Claiming that a BPM product that uses BPEL is better than another that uses XPDL because Pi-Calculus is better than Petrinets for BPM is not only a disingenuous way to confuse the market place but also does not match scientific reality. To quote van der Aalst again, "others advocating the use of BPEL and BPML, have not been able to demonstrate that any of these patterns requires Pi-Calculus to be supported".

Whether processes are serialized in XPDL, BPEL (or BPMN 2.0 in the near future) or whatever mathematical theories underpin these is irrelevant to business analysts and process developers. What matters most to them is the use of rigorous process modeling techniques, in particular, process patterns, to solve business problems and for that, we can all be grateful that we are indeed able to stand on the shoulder of giants. To quote Paul Strassman again (quoted by Nicholas Carr in his book "Does IT Matter?") as I did in my first post, "it's economics [..] technology has to be taken for granted". For BPM to become more accepted and not just dismissed as another fad as BPR (Business Process Re-engineering) came to be, it seems counterintuitive to make the choice and selection of BPMS as hard as having to understand the precise link between mathematical theories and software products and standards.

I neither have the extraordinary credentials nor the wealth of experience of luminaries such as Ismael in the BPM field. And nor do the brilliant engineers in our BPM labs request for butter over their toast at breakfast using formal Pi-Calculus semantics. However, making unsubstantiated and repeated claims (especially after these have been repeatedly debunked) linking formal BPM products or standards mathematical theories to formal mathematical theories will certainly not advance the latters' cause for their adoption by BPM vendors. As Arthur eloquently wrote: "I fear that these statements (by Ismael) may lead to a backlash against formal methods in BPM. Once people conclude that the emperor does not have any clothes (in the case of BPEL and the pi-calculus) they may conclude that there isn't any benefit at all in the use of formal theory."

I hope this provides some useful clarification.

Cheers,
Ali