There is confusion in the classification of software used to manage business processes:

• Enterprise Architecture (EA) tools target the following aspects of enterprise architecture:
  1. Business architecture - business goals, organizational structure, functions, processes, roles, etc.
  2. Application Architecture - enterprise systems and interfaces
  3. Data architecture - logical and physical
  4. Technology infrastructure - hardware, software, networks
• Modeling abilities of Business Process Analysis (BPA) tools cover a subset of EA: business architecture. But in addition to modeling BPA may include specialized tools like process simulation or statistical analysis.
• Business Process Management Suite (BPMS) must include business processes modeling, execution (process engine) and monitoring/analysis. Optionally it may include simulation, business rules and much more.
• Some vendors define their product as “BPM Software”. Usually this means that the system doesn’t fit into BPMS category - for example there is no engine - but marketing wants it to be labeled with BPM.

BPMS enthusiasts (including myself) believe in the power of executable business processes, in “what you model is what you run” principle. The process engine is a must for them so they can’t live with EA tools only.

On the other hand, enterprise architects have no idea how to handle business processes apart from the organizational structure, data and applications architecture. While many BPMS allow modeling the organizational structure, roles and external systems, it is fragmented to the level of “project” - that is, a logical business process (there may be several physical processes in the project) and do not cover the entire organization. Therefore BPMS doesn’t fit architect’s needs.

Now what shall we do about it? Maybe it’s a temporary issue and BPMS will eventually grow up to EA or EA will include the BPMS functionality some day?

I’d be glad to be wrong but I won’t expect this. Let’s forget about processes for a moment and look at the data architecture. DBMS is much more mature technology compared to the BPMS yet the divide between EA and DBMS is not going to disappear. Enterprise architecture deal with databases but their internal structure is modeled by specialized tools.

Getting back to the processes: they can be modelled both in EA and BPMS. As for today BPMN may be used in both. Typical business analyst knows EA/BPA tools better so the natural way to go for him is:

1. Draw BPMN diagram within EA
2. Two possible options:
   • If BPMS directly executes BPMN, export-import via XPDL
   • If BPMS supports BPEL, translate BPMN->BPEL

For example, model in ARIS and export to WebMethods. Or model in Casewise and translate to Oracle BPEL.

This workflow is simple, logical, but… not working well. Both options work only if we consider a single pass: draw - handover - execute. Yet BPM is not a one-time process automation excercise but management of ever-changing business processes.

What does it mean in practice? Once the model is loaded into BPMS, the developer edits it to specify certain details required by the execution engine. At the same time the original process model in EA tool doesn’t remain unchanged because analyst improves it continually. (I strictly recommend a series of posts started with “Model Strategy: Preserving vs. Transforming” at Keith Swenson’s blog to anyone interested in the matter.)

Now we must either transfer the details of physical implementation back to EA and find a room in the repository for them (so-called “round-trip” problem) or merge changes made in both models (like branch merging in version control systems). In theory the problem can be solved but as a matter of fact it wasn’t for years. Shall we wait more?

I propose to accept that:

1. EA-BPMS divide exists and will remain for the foreseeable future.
2. There will be no satisfactory automatic artifacts passing between them in the foreseeable future.

So where should we draw this division? There is certain freedom because the tools overlap. It’s better indeed to minimize the volume of information transmitted.

Suggestion: let’s divide at the level of a single process -

• The value chain analysis and inter-process communication (via events and/or data) modeling is done within EA tool
• The internal process logic is modeled within BPMS

This way only the list of processes and their interfaces will be transferred from EA to BPMS. Automatic export-import is not necessary for this task, we may utilize “export-import via the printer”.

Every business analyst should obey the red line: never use EA tools to model business process internals if BPMS is present.

There is better application for BPMN capabilities of EA tools. BPMS mostly focus on modeling the internal logic of the processes (orchestration), it’s poorly suited for interprocess communication modeling (process choreography). But the orchestration alone can only solve the tasks at the “office automation” level; as soon as you target the end-to-end business processes - i.e. the processes of real value - you can’t do much without choreography. (See process anti-pattern “End-to-End Process Orchestration”.)

Interprocess communications can be modeled by EA tools using the so-called “black box BPMN diagram” - modeling technique presenting each process as a blank BPMN pool. Communications through messages is modeled by message flow, communications through the data is modeled by associations.

BPMN diagram of interprocess communication (black box diagram) produced by EA tool may look like this:

For each process of the above diagram a BPMN diagram is created in BPMS. It deals with the internal logic of the process (white box diagram):

An example:

We can draw whatever we wish of course but as soon as we tried to make this diagram executable we’d find out that in the real world the payment doesn’t always arrive. As well as in the case of “one-way activity” antipattern, we communicate with an independent subject having a free will (the buyer) and we can’t assume that it will follow our prescriptions.

We should consider at least three outcomes in this example:

1. payment arrived
2. buyer informed about refusal to pay
3. neither payment nor refusal arrived in due time

There is a special BPMN construct exactly for such a case called “exclusive event gateway”:

The beauty of this construct is that the process executes in parallel after passing the gateway, expecting all three events simultaneously. But as soon as one event happens, the other branches stop executing automatically and immediately.

The big regret is that BPMS vendors tend to consider the event gatway as luxury. I know several systems being declared as supporting BPMN yet not supporting this construct.

This is mistake I’m afraid because alternative messages processing can’t be avoided! The only possible way to solve the task in the absence of the event gateway is parallel gateway. But here we hit the problem of stopping other routes as soon as one of possible events happens. It can only be resolved with ugly diagram constructs and/or program code. Of course the process visibility and standard compiance become lost on the way.

This antipattern is special because it comes not from process analysts but from BPMS developers. On the other hand, it’s BPMS vendors will to eliminate this antipattern - they only need to change their attitude towards the event gateway.

As long as particular BPMS does not support the exclusive event gateway, one cannot effectively utilize the message flow. The software supports orchestration but not choreography. An old good workflow to me, whether it has the BPMN label or not.

Alternative title: “Micromanagement”.

The typical first BPM project issue: how deep to dig into a business process details?

We were engaged into a process called “Purchase order by the three-tier agreement” in one of our first projects. Here is the brief:

• A three-tier agreement is concluded by and between the buyer, the manufacturer and the supplier - manufacturer’s authorized partner.
• The agreement is designed as a framework: it specifies prices, terms and conditions, but not the items to be purchased nor the volume. Each purchase shall be specified by a separate specification containing items and quantities. When signed such a specification enforces contractual obligations to all parties.
• The process is managed by the supplier and there were no intention to directly attach the buyer or manufacturer to BPMS. The whole process runs inside supplier’s organization yet there were such activities as “send the specification to manufacturer for signature” or “ship the goods to buyer”.

The process consists of four majour phases:

1. Agreeing the order.
2. Signing the specification.
3. Delivery.
4. Payments and deal closure.

Let’s consider the first phase “Agreeing the order”. Despite terms and conditions are set by the agreement, the bargaining may happen here, e.g. the buyer may request an additional discount for a large order or tougher delivery terms tied to the internal project schedule. In such cases the account manager should agree on the conditions within the organization and with the manufacturer; if the buyer requested too much then a compromise acceptable for all parties should be found. Several iterations may be necessary lasting for months. It’s a delicate work highly depending on account manager’s skills. And this is the key point of the process - the remaining activities are pure routine.

The first version of the process diagram looked like this:

Then it became more compliacated. First, if the manufacturer made a counter-offer that we found acceptable then we should only agree it with the buyer on the next round; the same is true with respect to buyer’s counter-offers. Then it was rightly pointed out that the process may be speed up if negotiations with the manufacturer and the buyer was made in parallel. And so on.

Let me skip the evolution of the process and proceed directly to the resulting diagram:

The key point of this process is that there is a single performer: account manager. No one else cares about where we are inside. Agreeing started / agreeing ended successfully / agreeing failed - that’s all the business (the process owner) is interested in.

I’ve seen process diagrams similar to the first one several times. For example, there was customer’s process of accepting goods to the warehouse with about twenty activities, all performed by a  storekeeper. It doesn’t make sence. You get cumbersome scheme prone to frequent changes yet the details do not add value to the process.

Target BPM on the overall process performance and cross-functional problems which are responsible for poor performance in most cases.

It is not easy - you must be ready to find yourself under crossfire if your studies affect existing borders between organization departments. But don’t go the path of least resistance and don’t use BPMS to document a sequence of activities performed by a single person.

Somehow I got involved into the long discussion with Rick Geneva about best practices of process modeling and execution. Now after we’ve spent a lot of bytes at his blog it’d be fair to allocate some of my provider’s. Besides, after initial verbal firefight Rick resorted to the heavy artillery of process diagrams. Being unable to retaliate in comments at his blog I had to retreat to my own turf.

I spoiled Rick’s intention to remain vendor-neutral and eventually we came to studying Intalio’s way of using pools and swimlanes which I believe is… well… not quite obvious and not widely used. Yet it’s great for me to know how things are done with Intalio because I know several specialists who seem to be happy with it but couldn’t get the idea.

Rick presented a diagram of a simplified sample process where a manager wants to hire someone:

Rick comments:

The diagram style is a hybrid, which shows both the swimlane approach and the single pool approach that shows only the BPM automation system. In this case the primary participant is the automation system. However I could easily reverse this to have any one of my people participants to be the primary.

Now this is how I’d model the same process:

It differs from Rick’s diagram in many aspects:

• Pools are used for external entities and/or processes. “Candidate” is such an entity: we don’t control his process, all we have is a messaging protocol.
• There is no such thing as “primary participant”. There is a single swimlane in “Hiring Process” pool but it’s only because we simplified it to a bare minimum. In reality there would be also “Functional Manager”, “Security Officer” and maybe others. From my point of view, the idea of primary participant is bad: the process is supposed to break the walls between department walls. More participants, more separated they are in terms of geography and organization structure - more value one may expect from process management and BPMS.
• There is no such thing as “executable system”. For me it’s like drawing “DBMS” on a database diagram. 

Rick argues:

Systems that only support one pool with many lanes are implying that they are human centric workflow, with some system capability.

Sorry Rick, I have enough of this: one party call the other “some legacy workflow vendors” and they in turn call the opponents “webservices orchestration vendors”. OK, I draw this diagram in a modeler that doesn’t have execution engine yet I can see how it can be executed in Tibco, Interstage or Oracle aka Fuego. The key is engine’s capability to process messages flow - workflow can handle only control flows.

Anyone wants more pools? No problem, I can draw internals of the Candidate’s process:

Is the diagram better now? I don’t think so. The beauty of process choreography is loose coupling: the changes in one process does not affect the other as long as messaging interface is followed. One cannot achieve this with process orchestration as I noted in my previous post and I believe this is Rick’s point too.

Previous diagram lacks loose coupling so I’d rather draw the candidate’s process in a separate diagram:

I share completely Rick’s concern:

I suggest the collaborative approach where IT and BA work together.

But sorry Rick - I didn’t meet a business analyst that would accept your style, it’s too technical. So we fall back to the traditional way: an analyst draws some vague diagram then throws it over the wall to developers and they convert it into executable code that the analyst is unable to read.

And I don’t believe it must be that technical like it is in Intalio.

How people activities look in various BPMN implementations? Let’s assume purely for illustration purposes that we have an “Inquiry to Order” process containing three activities: “Do This” (system), “Negotiate Contract” (human), “Do That” (system) and diagram it with several popular tools:

» read the rest

As mentioned at “End-to-End Process Orchestration anti-pattern“, business doesn’t work like “one-two-three”: shop floors don’t switch on/off by every client’s order even at produce-to-order scenario; material purchases are arranged for a production program rather than for a single finished product. Hence an end-to-end business process is modelled not by a sequence of activities within a single process (so called process orchestration) but by several processes executed independently and exchanging data and/or messages.

In BPMN each process is modelled by a separate pool. A process chain “Order to Cash” - “Manufacturing” - “Material Supply” for example may look as follows:

“Order to Cash” process is initiated by an incoming message - an order submitted by a client. “Manufacturing” is triggered by a timer, e.g. at the end of each working day, and handle queued orders within a cycle. If necessary materials are not available then “Manufacturing” waits for a signal from “Material Supply” that they are delivered. “Material Supply” is scheduled by a timer too and arranges purchasing for materials having the projected stock balance below the limit. (The internal orders chaing may have more links than in this example, e.g. a metallurgical plant may have inter-shop orders.)

Such asyncronous inter-process communication implies buffers between them. The buffers accumulate orders passed from process-requestor to process-provider. In the example above these are “Manufacturing Orders” and “Planned Stock Balance”. Technically such buffers may be implemented by a number of ways: message queue, database records, ERP objects at some specific state. Of course the buffer internals better to hide by wrapping them with services like “insert”, “traverse”, “extract”.

Modern BPM Suites are able to model and execute diagrams like above and this is the major BPMS advantage over traditional workflow systems. However such diagrams turned out to be difficult for analysts as Bruce Silver noted at “BPMN to Requester: Get Outta My Pool“. The major issue isn’t the notation but “asynchronous thinking”. One should develop an ability to extract separate asynchronous process from what business present to you as a single process. The answers to following questions should help: 1) what business object corresponds to a business process instance; 2) which events correspond to process instance start and stop.

For example, even in the relatively simple process of hiring an employee we can find a set of business objects: 1) a headcount item; 2) manager to HR request to fill the item; 3) a vacancy passed to a specific recruiting channel; 4) a candidate; 5) hired employee. They do not correspond to each other as 1:1 and their lifecycles aren’t synchronous. E.g. a candidate may submit his resume not caring about whether we have a vacancy for him - it’s HR task to assess which vacancy (or vacancies) it’s worth to consider for him. So a single process will hardly suffice; it depends on your business how much process there will be at the end of the day.

Worth to note that employee hiring is a classic business process example that BPM vendors love to use for their products demonstration. Yet they try to do it with a single process! Obviously such demos are made by developers, not consultants.

Final warning: please don’t consider the above as a call-up to breed too many asynchronous processes. In fact, the choice between synchronousness and asynchronousness is a non-trivial managerial decision but we’ll talk about it next time.

Definition:

• “Enterprise Process” (equivalent term “End-to-End Process”) is a business process connected to external customer at both ends and going through more than one top-level company’s departments.

Axiom:

• A BPM initiative would pay for itself only if you target end-to-end processes.

Otherwise you’ll be guilty of suboptimization sin according to Lean methodology or will “shoot sparrows with a cannon” according to Russian proverb. Yet it doesn’t mean you must begin with such a process - you may use something else for training but you won’t win a medal for training only.

Typical errors:

1. BPM vendors love to illustrate their products by supporting processes like “Employee Onboarding” or “Expenses Report” thus provoking prospects to do the same. It’s OK for training but there is simply not enough money beghind these processes to justify enterprise-wide project which what BPM project is by definition.
2. Many prospect wish to go for “Negotiating a Contract” process. There are money behind it but is this a process really? For me it’s rather a fragment of end-to-end process “Sales of Goods or Services”. Customer’s concern being the end result - i.e. performance of the process as a whole - it may happen that the bottleneck wouldn’t be this particular fragment. Narrowing a scope leads to suboptimization.
3. And here is the worst case: “We have a process here, well-studied and alredy automated, now let’s implement it in BPM for comparison”. In other words, let’s make a race between good and better. For your project to be acknowledged by the management you must not only implement some process in BPM but also significantly improve it. You’d hardly make it with a process that was well worked on already.

OK, let’s assume that we do everything right and consider “Order to Cash”, a classic example of end-to-end process. (Good questions would be: “Why exactly this process?” or “How to pick up the best one from many company’s processes?” These deserve detailed answers however so let’s leave them for the next time.)

“Order to Cash” process in “produce-to-order” business scenario may consist of prepayment, manufacturing, delivery and closure subprocesses:

What’s wrong with the above diagram? It assumes (since we have only one pool) that manufacturing works synchronously with sales: no order - manufacturing is idle, order arrives - manufacturing starts working on it. If we got deeper into manufacturing subprocess, we would probably found materials and/or parts purchasing subprocess, sychronized with manufacturing in turn.

But businesses just don’t work like “one-two-three”!

Manufacturing does not switch on/off by every client’s order even if we produce to order. Clients’ orders are accumulated and picked up on regular basis, say daily by production scheduling process. Similarly, purchasing is not triggered by a single order usually but rather runs regularily or is triggered by stock level going below a limit.

Technically such work is implemented in BPM not by a singe sychronous process like shown on the diagram above but by several asynchronously executed processes that communcate with each other by data and/or messages. Such scheme can be drawn with BPMN and executed by BPMS - which, from my point of view, is a big advantage of BPMS over workflow. But let’s consider the correct diagram next time - what I’m going to say here is that asychronous execution is a core feature not only of this particular but of every end-to-end process.

Definitions:

• “Process Orchestration” means tasks execution sequence and logic within a single process frame.
• “Process Choreography” means the logic of several processes asynchronous execution coordinated by data/message flows.

Theorem:

• End-to-end process should be modeled by the choreography, not by the orchestration.

It was confirmed by practice and could be proved by the following consideration: since end-to-end process by definition goes through several top-level departments, you’ll have to take into account the working rhytm of each one of them, which means - modeling asynchronous execution.

An attempt to model it by pure orchestration is nothing else but following an anti-pattern which I will call “End-to-End Process Archistration”.

Bruce Silver posted an article “BPMN’s Three Levels, Reconsidered” on his blog. (It’s a fllow-up to the earlier post on the matter: “Three Levels of Process Modeling with BPMN“.) From his two-years experience of giving BPMN classes Bruce noticed that many of students (he even says “most” couple of paragraphs forward) are simply trying to document, analyse and improve their processes and don’t bother about executable models. Bruce calls this Level 1 of BPMN usage. Level 2 also covers activity flow model suitable for direct execution inside BPMS that includes conditional logic, exceptions, events, messaging (process choreography assumed yet not mentioned).

But is it about BPMN really?

I like the statement Mark McGregor made on the cover page of his new book “Winning With Enterprise Process Management” (freely available at markmcgregor.com):

…process thinking takes many forms - Business Process Management, Continuous Process Improvement, Six Sigma, Lean Sigma, Business Process Reengineering and many others…

Mark is right: it’s about thinking. Process thinking. Different kinds of process thinking, to be precise.

Do you remember the times when object-oriented programming was just invented? It was noticed that it’s not about programming languages. One could write object-oriented software even with Fortran (and some people did when there were no decent C++ compilers) if he catched the idea. And of course you can (and many people actually do) write 100%-functional code on C++.

An interesting observation was made at that time: it’s much easier to teach C++ to a newbie than to experienced C programmers. The reason - it’s about installing a certain kind of mindset (object-oriented in that case) or changing it. The latter turns out to be much harder than the former.

Now I have absolutely the same experience with people trying to understand what BPMN (BPM, BPMS - you name it) is about. Those who deal with business processes for years and have strong background in BPR, ISO9000 etc. can’t grasp what’s so cool about executable process models. They always considered execution to be “implementation details”, something IT should care about. Some of them become irritated enough to say or write that BPM is nothing new, it’s pure marketing, it’s an “umbrella concept” etc.

By contrast, every student and/or junior consultant becomes excited about possibilities that this concept opens. When you just draw a process diagram you can make a dozen of them, all being valid and all being different. That’s no good. When execution is involved - even in it’s simplest form, with simple automatically generated screens and zero integration - you go from “diagrams” to “the diagram”. The analyst isn’t in position to draw an unconsistent diagram any more: if he does, the diagram returns back to him with developer’s note “sorry, can’t be executed this way, please correct”.

Getting back to BPMN Training - Bruce uses Process Modeler for Microsoft Visio by itp commerce for his classes. It could be a perfect choice: high level of BPMN compliance and strong simulation capabilities. But there is no execution. And you just can’t explain what is the execution by words and slides, without showing it.

When we realized this fact several years ago, we recorded and published a simple demo video showing BPMS modeling and execution. And it became a hit. Many people said “thank you” because it helped them understand the directly executable process model concept, regardless of BPMS used.

So Bruce, if you wish more people moved from Level 1 to Level 2, you must show them how a model can be executed in BPMS and how iterational modeling and execution is done. Don’t leave it implicit and don’t assume people know it already. Because as you say it yourself - most of them don’t and it’s the key assumption behind BPMN.