Alternative title: “Micromanagement”.

The typical first BPM project issue: how deep to dig into a business process details?

We were engaged into a process called “Purchase order by the three-tier agreement” in one of our first projects. Here is the brief:

• A three-tier agreement is concluded by and between the buyer, the manufacturer and the supplier - manufacturer’s authorized partner.
• The agreement is designed as a framework: it specifies prices, terms and conditions, but not the items to be purchased nor the volume. Each purchase shall be specified by a separate specification containing items and quantities. When signed such a specification enforces contractual obligations to all parties.
• The process is managed by the supplier and there were no intention to directly attach the buyer or manufacturer to BPMS. The whole process runs inside supplier’s organization yet there were such activities as “send the specification to manufacturer for signature” or “ship the goods to buyer”.

The process consists of four majour phases:

1. Agreeing the order.
2. Signing the specification.
3. Delivery.
4. Payments and deal closure.

Let’s consider the first phase “Agreeing the order”. Despite terms and conditions are set by the agreement, the bargaining may happen here, e.g. the buyer may request an additional discount for a large order or tougher delivery terms tied to the internal project schedule. In such cases the account manager should agree on the conditions within the organization and with the manufacturer; if the buyer requested too much then a compromise acceptable for all parties should be found. Several iterations may be necessary lasting for months. It’s a delicate work highly depending on account manager’s skills. And this is the key point of the process - the remaining activities are pure routine.

The first version of the process diagram looked like this:

Then it became more compliacated. First, if the manufacturer made a counter-offer that we found acceptable then we should only agree it with the buyer on the next round; the same is true with respect to buyer’s counter-offers. Then it was rightly pointed out that the process may be speed up if negotiations with the manufacturer and the buyer was made in parallel. And so on.

Let me skip the evolution of the process and proceed directly to the resulting diagram:

The key point of this process is that there is a single performer: account manager. No one else cares about where we are inside. Agreeing started / agreeing ended successfully / agreeing failed - that’s all the business (the process owner) is interested in.

I’ve seen process diagrams similar to the first one several times. For example, there was customer’s process of accepting goods to the warehouse with about twenty activities, all performed by a  storekeeper. It doesn’t make sence. You get cumbersome scheme prone to frequent changes yet the details do not add value to the process.

Target BPM on the overall process performance and cross-functional problems which are responsible for poor performance in most cases.

It is not easy - you must be ready to find yourself under crossfire if your studies affect existing borders between organization departments. But don’t go the path of least resistance and don’t use BPMS to document a sequence of activities performed by a single person.

Somehow I got involved into the long discussion with Rick Geneva about best practices of process modeling and execution. Now after we’ve spent a lot of bytes at his blog it’d be fair to allocate some of my provider’s. Besides, after initial verbal firefight Rick resorted to the heavy artillery of process diagrams. Being unable to retaliate in comments at his blog I had to retreat to my own turf.

I spoiled Rick’s intention to remain vendor-neutral and eventually we came to studying Intalio’s way of using pools and swimlanes which I believe is… well… not quite obvious and not widely used. Yet it’s great for me to know how things are done with Intalio because I know several specialists who seem to be happy with it but couldn’t get the idea.

Rick presented a diagram of a simplified sample process where a manager wants to hire someone:

Rick comments:

The diagram style is a hybrid, which shows both the swimlane approach and the single pool approach that shows only the BPM automation system. In this case the primary participant is the automation system. However I could easily reverse this to have any one of my people participants to be the primary.

Now this is how I’d model the same process:

It differs from Rick’s diagram in many aspects:

• Pools are used for external entities and/or processes. “Candidate” is such an entity: we don’t control his process, all we have is a messaging protocol.
• There is no such thing as “primary participant”. There is a single swimlane in “Hiring Process” pool but it’s only because we simplified it to a bare minimum. In reality there would be also “Functional Manager”, “Security Officer” and maybe others. From my point of view, the idea of primary participant is bad: the process is supposed to break the walls between department walls. More participants, more separated they are in terms of geography and organization structure - more value one may expect from process management and BPMS.
• There is no such thing as “executable system”. For me it’s like drawing “DBMS” on a database diagram. 

Rick argues:

Systems that only support one pool with many lanes are implying that they are human centric workflow, with some system capability.

Sorry Rick, I have enough of this: one party call the other “some legacy workflow vendors” and they in turn call the opponents “webservices orchestration vendors”. OK, I draw this diagram in a modeler that doesn’t have execution engine yet I can see how it can be executed in Tibco, Interstage or Oracle aka Fuego. The key is engine’s capability to process messages flow - workflow can handle only control flows.

Anyone wants more pools? No problem, I can draw internals of the Candidate’s process:

Is the diagram better now? I don’t think so. The beauty of process choreography is loose coupling: the changes in one process does not affect the other as long as messaging interface is followed. One cannot achieve this with process orchestration as I noted in my previous post and I believe this is Rick’s point too.

Previous diagram lacks loose coupling so I’d rather draw the candidate’s process in a separate diagram:

I share completely Rick’s concern:

I suggest the collaborative approach where IT and BA work together.

But sorry Rick - I didn’t meet a business analyst that would accept your style, it’s too technical. So we fall back to the traditional way: an analyst draws some vague diagram then throws it over the wall to developers and they convert it into executable code that the analyst is unable to read.

And I don’t believe it must be that technical like it is in Intalio.

How people activities look in various BPMN implementations? Let’s assume purely for illustration purposes that we have an “Inquiry to Order” process containing three activities: “Do This” (system), “Negotiate Contract” (human), “Do That” (system) and diagram it with several popular tools:

» read the rest

As mentioned at “End-to-End Process Orchestration anti-pattern“, business doesn’t work like “one-two-three”: shop floors don’t switch on/off by every client’s order even at produce-to-order scenario; material purchases are arranged for a production program rather than for a single finished product. Hence an end-to-end business process is modelled not by a sequence of activities within a single process (so called process orchestration) but by several processes executed independently and exchanging data and/or messages.

In BPMN each process is modelled by a separate pool. A process chain “Order to Cash” - “Manufacturing” - “Material Supply” for example may look as follows:

“Order to Cash” process is initiated by an incoming message - an order submitted by a client. “Manufacturing” is triggered by a timer, e.g. at the end of each working day, and handle queued orders within a cycle. If necessary materials are not available then “Manufacturing” waits for a signal from “Material Supply” that they are delivered. “Material Supply” is scheduled by a timer too and arranges purchasing for materials having the projected stock balance below the limit. (The internal orders chaing may have more links than in this example, e.g. a metallurgical plant may have inter-shop orders.)

Such asyncronous inter-process communication implies buffers between them. The buffers accumulate orders passed from process-requestor to process-provider. In the example above these are “Manufacturing Orders” and “Planned Stock Balance”. Technically such buffers may be implemented by a number of ways: message queue, database records, ERP objects at some specific state. Of course the buffer internals better to hide by wrapping them with services like “insert”, “traverse”, “extract”.

Modern BPM Suites are able to model and execute diagrams like above and this is the major BPMS advantage over traditional workflow systems. However such diagrams turned out to be difficult for analysts as Bruce Silver noted at “BPMN to Requester: Get Outta My Pool“. The major issue isn’t the notation but “asynchronous thinking”. One should develop an ability to extract separate asynchronous process from what business present to you as a single process. The answers to following questions should help: 1) what business object corresponds to a business process instance; 2) which events correspond to process instance start and stop.

For example, even in the relatively simple process of hiring an employee we can find a set of business objects: 1) a headcount item; 2) manager to HR request to fill the item; 3) a vacancy passed to a specific recruiting channel; 4) a candidate; 5) hired employee. They do not correspond to each other as 1:1 and their lifecycles aren’t synchronous. E.g. a candidate may submit his resume not caring about whether we have a vacancy for him - it’s HR task to assess which vacancy (or vacancies) it’s worth to consider for him. So a single process will hardly suffice; it depends on your business how much process there will be at the end of the day.

Worth to note that employee hiring is a classic business process example that BPM vendors love to use for their products demonstration. Yet they try to do it with a single process! Obviously such demos are made by developers, not consultants.

Final warning: please don’t consider the above as a call-up to breed too many asynchronous processes. In fact, the choice between synchronousness and asynchronousness is a non-trivial managerial decision but we’ll talk about it next time.

Definition:

• “Enterprise Process” (equivalent term “End-to-End Process”) is a business process connected to external customer at both ends and going through more than one top-level company’s departments.

Axiom:

• A BPM initiative would pay for itself only if you target end-to-end processes.

Otherwise you’ll be guilty of suboptimization sin according to Lean methodology or will “shoot sparrows with a cannon” according to Russian proverb. Yet it doesn’t mean you must begin with such a process - you may use something else for training but you won’t win a medal for training only.

Typical errors:

1. BPM vendors love to illustrate their products by supporting processes like “Employee Onboarding” or “Expenses Report” thus provoking prospects to do the same. It’s OK for training but there is simply not enough money beghind these processes to justify enterprise-wide project which what BPM project is by definition.
2. Many prospect wish to go for “Negotiating a Contract” process. There are money behind it but is this a process really? For me it’s rather a fragment of end-to-end process “Sales of Goods or Services”. Customer’s concern being the end result - i.e. performance of the process as a whole - it may happen that the bottleneck wouldn’t be this particular fragment. Narrowing a scope leads to suboptimization.
3. And here is the worst case: “We have a process here, well-studied and alredy automated, now let’s implement it in BPM for comparison”. In other words, let’s make a race between good and better. For your project to be acknowledged by the management you must not only implement some process in BPM but also significantly improve it. You’d hardly make it with a process that was well worked on already.

OK, let’s assume that we do everything right and consider “Order to Cash”, a classic example of end-to-end process. (Good questions would be: “Why exactly this process?” or “How to pick up the best one from many company’s processes?” These deserve detailed answers however so let’s leave them for the next time.)

“Order to Cash” process in “produce-to-order” business scenario may consist of prepayment, manufacturing, delivery and closure subprocesses:

What’s wrong with the above diagram? It assumes (since we have only one pool) that manufacturing works synchronously with sales: no order - manufacturing is idle, order arrives - manufacturing starts working on it. If we got deeper into manufacturing subprocess, we would probably found materials and/or parts purchasing subprocess, sychronized with manufacturing in turn.

But businesses just don’t work like “one-two-three”!

Manufacturing does not switch on/off by every client’s order even if we produce to order. Clients’ orders are accumulated and picked up on regular basis, say daily by production scheduling process. Similarly, purchasing is not triggered by a single order usually but rather runs regularily or is triggered by stock level going below a limit.

Technically such work is implemented in BPM not by a singe sychronous process like shown on the diagram above but by several asynchronously executed processes that communcate with each other by data and/or messages. Such scheme can be drawn with BPMN and executed by BPMS - which, from my point of view, is a big advantage of BPMS over workflow. But let’s consider the correct diagram next time - what I’m going to say here is that asychronous execution is a core feature not only of this particular but of every end-to-end process.

Definitions:

• “Process Orchestration” means tasks execution sequence and logic within a single process frame.
• “Process Choreography” means the logic of several processes asynchronous execution coordinated by data/message flows.

Theorem:

• End-to-end process should be modeled by the choreography, not by the orchestration.

It was confirmed by practice and could be proved by the following consideration: since end-to-end process by definition goes through several top-level departments, you’ll have to take into account the working rhytm of each one of them, which means - modeling asynchronous execution.

An attempt to model it by pure orchestration is nothing else but following an anti-pattern which I will call “End-to-End Process Archistration”.

Bruce Silver posted an article “BPMN’s Three Levels, Reconsidered” on his blog. (It’s a fllow-up to the earlier post on the matter: “Three Levels of Process Modeling with BPMN“.) From his two-years experience of giving BPMN classes Bruce noticed that many of students (he even says “most” couple of paragraphs forward) are simply trying to document, analyse and improve their processes and don’t bother about executable models. Bruce calls this Level 1 of BPMN usage. Level 2 also covers activity flow model suitable for direct execution inside BPMS that includes conditional logic, exceptions, events, messaging (process choreography assumed yet not mentioned).

But is it about BPMN really?

I like the statement Mark McGregor made on the cover page of his new book “Winning With Enterprise Process Management” (freely available at markmcgregor.com):

…process thinking takes many forms - Business Process Management, Continuous Process Improvement, Six Sigma, Lean Sigma, Business Process Reengineering and many others…

Mark is right: it’s about thinking. Process thinking. Different kinds of process thinking, to be precise.

Do you remember the times when object-oriented programming was just invented? It was noticed that it’s not about programming languages. One could write object-oriented software even with Fortran (and some people did when there were no decent C++ compilers) if he catched the idea. And of course you can (and many people actually do) write 100%-functional code on C++.

An interesting observation was made at that time: it’s much easier to teach C++ to a newbie than to experienced C programmers. The reason - it’s about installing a certain kind of mindset (object-oriented in that case) or changing it. The latter turns out to be much harder than the former.

Now I have absolutely the same experience with people trying to understand what BPMN (BPM, BPMS - you name it) is about. Those who deal with business processes for years and have strong background in BPR, ISO9000 etc. can’t grasp what’s so cool about executable process models. They always considered execution to be “implementation details”, something IT should care about. Some of them become irritated enough to say or write that BPM is nothing new, it’s pure marketing, it’s an “umbrella concept” etc.

By contrast, every student and/or junior consultant becomes excited about possibilities that this concept opens. When you just draw a process diagram you can make a dozen of them, all being valid and all being different. That’s no good. When execution is involved - even in it’s simplest form, with simple automatically generated screens and zero integration - you go from “diagrams” to “the diagram”. The analyst isn’t in position to draw an unconsistent diagram any more: if he does, the diagram returns back to him with developer’s note “sorry, can’t be executed this way, please correct”.

Getting back to BPMN Training - Bruce uses Process Modeler for Microsoft Visio by itp commerce for his classes. It could be a perfect choice: high level of BPMN compliance and strong simulation capabilities. But there is no execution. And you just can’t explain what is the execution by words and slides, without showing it.

When we realized this fact several years ago, we recorded and published a simple demo video showing BPMS modeling and execution. And it became a hit. Many people said “thank you” because it helped them understand the directly executable process model concept, regardless of BPMS used.

So Bruce, if you wish more people moved from Level 1 to Level 2, you must show them how a model can be executed in BPMS and how iterational modeling and execution is done. Don’t leave it implicit and don’t assume people know it already. Because as you say it yourself - most of them don’t and it’s the key assumption behind BPMN.