There is confusion in the classification of software used to manage business processes:

• Enterprise Architecture (EA) tools target the following aspects of enterprise architecture:
  1. Business architecture - business goals, organizational structure, functions, processes, roles, etc.
  2. Application Architecture - enterprise systems and interfaces
  3. Data architecture - logical and physical
  4. Technology infrastructure - hardware, software, networks
• Modeling abilities of Business Process Analysis (BPA) tools cover a subset of EA: business architecture. But in addition to modeling BPA may include specialized tools like process simulation or statistical analysis.
• Business Process Management Suite (BPMS) must include business processes modeling, execution (process engine) and monitoring/analysis. Optionally it may include simulation, business rules and much more.
• Some vendors define their product as “BPM Software”. Usually this means that the system doesn’t fit into BPMS category - for example there is no engine - but marketing wants it to be labeled with BPM.

BPMS enthusiasts (including myself) believe in the power of executable business processes, in “what you model is what you run” principle. The process engine is a must for them so they can’t live with EA tools only.

On the other hand, enterprise architects have no idea how to handle business processes apart from the organizational structure, data and applications architecture. While many BPMS allow modeling the organizational structure, roles and external systems, it is fragmented to the level of “project” - that is, a logical business process (there may be several physical processes in the project) and do not cover the entire organization. Therefore BPMS doesn’t fit architect’s needs.

Now what shall we do about it? Maybe it’s a temporary issue and BPMS will eventually grow up to EA or EA will include the BPMS functionality some day?

I’d be glad to be wrong but I won’t expect this. Let’s forget about processes for a moment and look at the data architecture. DBMS is much more mature technology compared to the BPMS yet the divide between EA and DBMS is not going to disappear. Enterprise architecture deal with databases but their internal structure is modeled by specialized tools.

Getting back to the processes: they can be modelled both in EA and BPMS. As for today BPMN may be used in both. Typical business analyst knows EA/BPA tools better so the natural way to go for him is:

1. Draw BPMN diagram within EA
2. Two possible options:
   • If BPMS directly executes BPMN, export-import via XPDL
   • If BPMS supports BPEL, translate BPMN->BPEL

For example, model in ARIS and export to WebMethods. Or model in Casewise and translate to Oracle BPEL.

This workflow is simple, logical, but… not working well. Both options work only if we consider a single pass: draw - handover - execute. Yet BPM is not a one-time process automation excercise but management of ever-changing business processes.

What does it mean in practice? Once the model is loaded into BPMS, the developer edits it to specify certain details required by the execution engine. At the same time the original process model in EA tool doesn’t remain unchanged because analyst improves it continually. (I strictly recommend a series of posts started with “Model Strategy: Preserving vs. Transforming” at Keith Swenson’s blog to anyone interested in the matter.)

Now we must either transfer the details of physical implementation back to EA and find a room in the repository for them (so-called “round-trip” problem) or merge changes made in both models (like branch merging in version control systems). In theory the problem can be solved but as a matter of fact it wasn’t for years. Shall we wait more?

I propose to accept that:

1. EA-BPMS divide exists and will remain for the foreseeable future.
2. There will be no satisfactory automatic artifacts passing between them in the foreseeable future.

So where should we draw this division? There is certain freedom because the tools overlap. It’s better indeed to minimize the volume of information transmitted.

Suggestion: let’s divide at the level of a single process -

• The value chain analysis and inter-process communication (via events and/or data) modeling is done within EA tool
• The internal process logic is modeled within BPMS

This way only the list of processes and their interfaces will be transferred from EA to BPMS. Automatic export-import is not necessary for this task, we may utilize “export-import via the printer”.

Every business analyst should obey the red line: never use EA tools to model business process internals if BPMS is present.

There is better application for BPMN capabilities of EA tools. BPMS mostly focus on modeling the internal logic of the processes (orchestration), it’s poorly suited for interprocess communication modeling (process choreography). But the orchestration alone can only solve the tasks at the “office automation” level; as soon as you target the end-to-end business processes - i.e. the processes of real value - you can’t do much without choreography. (See process anti-pattern “End-to-End Process Orchestration”.)

Interprocess communications can be modeled by EA tools using the so-called “black box BPMN diagram” - modeling technique presenting each process as a blank BPMN pool. Communications through messages is modeled by message flow, communications through the data is modeled by associations.

BPMN diagram of interprocess communication (black box diagram) produced by EA tool may look like this:

For each process of the above diagram a BPMN diagram is created in BPMS. It deals with the internal logic of the process (white box diagram):

Sorry, this entry is only available in Русский.

An example:

We can draw whatever we wish of course but as soon as we tried to make this diagram executable we’d find out that in the real world the payment doesn’t always arrive. As well as in the case of “one-way activity” antipattern, we communicate with an independent subject having a free will (the buyer) and we can’t assume that it will follow our prescriptions.

We should consider at least three outcomes in this example:

1. payment arrived
2. buyer informed about refusal to pay
3. neither payment nor refusal arrived in due time

There is a special BPMN construct exactly for such a case called “exclusive event gateway”:

The beauty of this construct is that the process executes in parallel after passing the gateway, expecting all three events simultaneously. But as soon as one event happens, the other branches stop executing automatically and immediately.

The big regret is that BPMS vendors tend to consider the event gatway as luxury. I know several systems being declared as supporting BPMN yet not supporting this construct.

This is mistake I’m afraid because alternative messages processing can’t be avoided! The only possible way to solve the task in the absence of the event gateway is parallel gateway. But here we hit the problem of stopping other routes as soon as one of possible events happens. It can only be resolved with ugly diagram constructs and/or program code. Of course the process visibility and standard compiance become lost on the way.

This antipattern is special because it comes not from process analysts but from BPMS developers. On the other hand, it’s BPMS vendors will to eliminate this antipattern - they only need to change their attitude towards the event gateway.

As long as particular BPMS does not support the exclusive event gateway, one cannot effectively utilize the message flow. The software supports orchestration but not choreography. An old good workflow to me, whether it has the BPMN label or not.

Sorry, this entry is only available in Русский.

This anitpattern is almost trivial. Yet watching how different people make the same mistake at different projects I come to conclusion that it’s quite popular

Example 1: Concluding a contract. The draft must must be signed on behalf of our company by CEO:

Please respect the boss by giving him the opportunity not to sign the contract - put the gateway immediately after “sign by the CEO” activity.

Example 2: while processing a customer’s order, an agent company places an order with a partner:

Recognize that the partner has a free will and take into account the the possibility that he will not accept our order - add the gateway to the process.

One-way modeling is OK at early stages of process discovery, it’s the so-called “happy path”. On the other hand, is there any activity with the predetermined outcome? Maybe it’s better to check the result after each and every activity?

Putting gateways everywhere will clutter the scheme. The lesson learned from this antipattern is this: as a minimum, put the checks after activities where the free will is clearly visible - for example, assigned to decision makers or organizations independent of our.

BPM conquers the new territory which is called in different ways: Dynamic Processes, Unstructured Processes, Knowledge Worker Processes, Barely Repeatable Processes, Case Management.

BPM now reached the maturity level where the management of repetitive and predictable processes has become a matter of technique. It grants reliable process execution by unreliable employees - low-paid, with low skills and low motivation. Such processes are common for state and semi-state organizations and also for businesses following an extensive path of development.

But only “very talented” individuals can believe that every process can be rigidly structured:

Leaving aside the extremes - fully structured and completely ad-hoc processes - there are two combinations: either a structured process launches the ad-hoc or vice versa.

1. “A Little Help From a Friend” pattern: a structured process launches ad-hoc subprocess. An example: system integrator’s “request-to-order” process. Let’s assume that sales rep’s meeting with a client was positive meaning that he found  ”pain points” - issues that the client will pay for if they were resolved by the integrator. The next step of the process is finding a solution. However, the client’s problems may vary at very wide range (note that integrator’s value is exactly his ability to solve a wide range of problems) starting from the simplest need of a boxed software to complex projects. In the latter case the process will follow a trajectory unknown in advance that may involve an architect, developer, sales manager, systems engineer, vendor’s tech support etc. Traditional BPMS can only represent this unpredictable subprocess as a single task (see “Process Antipattern: One Man Show“). This is a poor-man’s solution indeed because it isn’t visible who is addressing the issue at the moment, what progress has been made and what the current timeframe is.
2. The opposite situation, the “Process Toolset” pattern: unpredictable process at the top launching well-formalized subprocesses. An example: a law firm having client’s case as the top-level process. It’s absolutely impossible to predict in what direction the case will turn next day, e.g. a new document may be submitted by the opposite party that will change radically both  the case prospects and our plan for actions. Yet many of the actions initiated within the case are standard, making it possible to formalize each as a subprocess. They are mostly preparing applications or other documents for the court. Such subprocesses is executed by a dedicated specialist - a common resource not assigned to a specific case. From business perspective it’s interesting to monitor performance not only of the case as a whole but also of subprocesses and resources usage.

Going down from the top-level business processes (company’s value chain) to the lowest level (micromanagement) we must be prepared to encounter both structured and ad-hoc process. Support for the latter in today’s BPMS is far behind but this subject is widely discussed by researchers, analysts and vendors:

• The following estimate was made at Gartner’2009 conference: as much as 60% of an organization’s processes are unstructured – and probably also unmonitored, unmanaged, unknown and unruly. These 60% are like “average temperature by the clinic” but the “invisibility” of these processes can indeed be the major issue from business standpoint.
• “Tapping into Collective Knowledge Will Drive Unstructured Process Activity” - Jim Sinur predicts that organizations acceptance of collective knowledge, industry networks and even social networks will result in fundamental changes in BPM. His another post on the same subject: “White-collar and unstructured processes go together like cheese to wine“.
• The boom of social networks pushes the idea to borrow approaches evolved there for communication within dynamic processes - see the Workshop on BPM and social software at BPM conference in Ulm. For example, when confronted with a problem, I can publish the question on the corporate social network (”a help from a friend”). It’s visible to my “friends” including the project manager and team lead. The best answer gets a bonus.
• SAP shows how Google Wave collaboration technology can be used - not for process execution but for modeling. SAP Gravity is a BPMN modeler implemented within Google Wave. Taking into account that the ability to redesign a process on the fly is essential for dynamic process execution, SAP makes progress not only in process modeling and discovery but in the the execution too.
• Oracle talked about collaborative and dynamic BPM at Oracle OpenWorld’2009. They emphasized commitment to SCA that makes possible to combine different kinds of processes with business rules, analytics, etc. That’s no surprise taking into account that they have acquired about 50 companies in two years and hence face huge integration challanges.
• HandySoft, ActionBase and other vendors claim dynamic processes support in the latest versions of their products.
• The most authoritative industry experts gather to discuss dynamic processes in general and their support in BPMN.

So we can see a number of minds attacking the problem from different directions. Well the more alternative approaches, the better final solution should be.

Yet for BPMS vendors that may be a bloody battle. There is probably more BPM vendors than the market needs and now a new front  opens for the competition: dynamic processes. It’s quite wide front if all aspects of dynamics are taken into account so I’m afraid not all vendors will be able to fight it in the current economic situation. But at the end of the day those who provided full support to dynamic processes will posess a competitive advantage clearly visible for the users.

When downloading a BPMS trial or demo, keep in mind that when it comes to production, the resulting system will differ from out-of-the-box version in many essential aspects:

1. The user portal - web application that starts processes, displays the list of tasks assigned to the user, manage activity forms for these tasks, monitors and administers processes. It will have different design in production and most likely different functionality too. If you’re lucky you will be able to customize out-of-the-box portal but be prepared to rewrite it from scratch at some point. Or to get away from a standalone BPM portal completely and wire process functionality into corporate applications. The reason: users typically do not accept BPMS suppier’s opinion that BPM should be the center of user’s universe.
2. In particular, you should eventually get rid of “start process” button. From user’s perspective, he doesn’t “start a process” but do something real e.g. accepts the incoming order or submits a request for vacation. The system must start the appropriate process transparantely.
3. Be prepared that activity forms generated by BPMS in few mouse clicks will no longer meet the functionality, usability and design requirements at some point. So it’s better to have an idea how will you eventually develop these forms in terms of tools, labor force and costs. The importance of this issue can not be overestimated: what good is that the process scheme is depicted in two days if forms development for this process then takes say two months? (I do not play down the importance of rapid prototyping of screen interfaces - it’s the must for BPM, one won’t even come close to production without it.) By the way you probably would like to use the same tools to rewrite the BPM portal.
4. Similarly you will no longer be satisfied with out-of-the-box reporting and monitoring tools at some point.
5. Demo and pilot processes typically store all data in process attributes, process variables or operands (different systems use different terminology) but only relatively insufficient and/or temporary information will be stored this way in production. Most data will go into a traditional database and only the primary key of the corresponding record will be stored within the process. Considering the process of client purchase order negotiation as an example, the information about the client and the order items are likely to be stored in a database while customer and order identifiers will remain in process attributes together with the deadline date for the call to the client. The reason to act this way is obvious: data which may be of interest after the process instance has ended must be stored so that they could be accessed independently from the process instance. This also means a separate user interface to this data independent from process screen forms. As for the process screen forms, they should access both process attributes via BPMS API and database fields via DBMS API.
6. Building on the previous item, most likely the part of the long-term information (but usually not all) already have a room at your existing enterprise applications. Accordingly, the process attributes will store only the identifiers of the appropirate business objects and process screen forms will access the data stored within the application. (The latter isn’t an absolute requirement - the total integration is often very time-consuming so partial integration may be more justified.)
7. Similarly, while a demo or pilot most likely will store related documents (usually Word or Excel files) as attachments to a process instance, you’ll have to consider something more solid for production. The reason is the same: if the document may be of interest after the process instance has ended, then it must be kept independently from the process instances and user access to it must be provided independently from the user interface to the process. However you don’t need the full-blown ECM system: because BPMS takes care about the workflow you need only documents storage functionality with basic interfaces (user’s and programming) and services (search, archiving, security).
8. Users authentication and authorization in a demo or pilot is usually done via independent LDAP directory, database or even a static list stored in the XML file. It is obvious that production system should utilize your existing user directory. But a bad surprise may be the amount of effort it requires. To start with there are usually several such directories. A typical example: an Active Directory, a separate authorization system within the legacy accounting system and a database keeping the users of remote offices and partner companies. As the project evolves additional requirements may arise e.g. the planned absence and automatic rerouting of the tasks. It is known that for a company having about a hundred of users Active Directory implementation alone is a non-trivial project and now we are facing more difficult task. As a result as much as 50% of total BPM project costs are spent on authorization and authentication issues at some projects. Imagine for a moment that it happened in your project and you didn’t take it into account in project schedule: you are out of schedule and budget for as much as 100%!
9. For obvious reasons not the most complex business processes are taken for demos and pilot projects. That would be all right but worse than that, they are usually technically implemented as a single process thread. But in reality even the relatively simple employee onboarding process technically consists of several processes communicating with each other (it’s enough to notice that processing the incoming resumes is not directly related to the publication of vacancies). This is even more true for end-to-end processes that are of greatest interest in terms of business (see “End-to-end Process Orchestration” antipattern and “Internal Order” pattern). Accordingly, you will need more functionality from your BPMS pretty soon - not only the orchestration but also choreography. Modern BPMS are fine with that but if a rudimentary worflow and/or document management built into your accounting system is all what you have then you may be in trouble.
10. And finally, a production system differs from a pilot by reliability, performance, security … but these are standard requirements not specific to BPM.

But don’t be scared: these issues are typically resolved one by one in incremental fasion; all companies that are successfull in BPM (and any vendor can provide dozens of references) did the job. It’s just better to know in advance what’s awaiting you after successful BPM pilot and to address the appropriate questions to yourself and to your supplier.

Sorry, this entry is only available in Русский.

Sorry, this entry is only available in Русский.

Somehow I got involved into the long discussion with Rick Geneva about best practices of process modeling and execution. Now after we’ve spent a lot of bytes at his blog it’d be fair to allocate some of my provider’s. Besides, after initial verbal firefight Rick resorted to the heavy artillery of process diagrams. Being unable to retaliate in comments at his blog I had to retreat to my own turf.

I spoiled Rick’s intention to remain vendor-neutral and eventually we came to studying Intalio’s way of using pools and swimlanes which I believe is… well… not quite obvious and not widely used. Yet it’s great for me to know how things are done with Intalio because I know several specialists who seem to be happy with it but couldn’t get the idea.

Rick presented a diagram of a simplified sample process where a manager wants to hire someone:

Rick comments:

The diagram style is a hybrid, which shows both the swimlane approach and the single pool approach that shows only the BPM automation system. In this case the primary participant is the automation system. However I could easily reverse this to have any one of my people participants to be the primary.

Now this is how I’d model the same process:

It differs from Rick’s diagram in many aspects:

• Pools are used for external entities and/or processes. “Candidate” is such an entity: we don’t control his process, all we have is a messaging protocol.
• There is no such thing as “primary participant”. There is a single swimlane in “Hiring Process” pool but it’s only because we simplified it to a bare minimum. In reality there would be also “Functional Manager”, “Security Officer” and maybe others. From my point of view, the idea of primary participant is bad: the process is supposed to break the walls between department walls. More participants, more separated they are in terms of geography and organization structure - more value one may expect from process management and BPMS.
• There is no such thing as “executable system”. For me it’s like drawing “DBMS” on a database diagram. 

Rick argues:

Systems that only support one pool with many lanes are implying that they are human centric workflow, with some system capability.

Sorry Rick, I have enough of this: one party call the other “some legacy workflow vendors” and they in turn call the opponents “webservices orchestration vendors”. OK, I draw this diagram in a modeler that doesn’t have execution engine yet I can see how it can be executed in Tibco, Interstage or Oracle aka Fuego. The key is engine’s capability to process messages flow - workflow can handle only control flows.

Anyone wants more pools? No problem, I can draw internals of the Candidate’s process:

Is the diagram better now? I don’t think so. The beauty of process choreography is loose coupling: the changes in one process does not affect the other as long as messaging interface is followed. One cannot achieve this with process orchestration as I noted in my previous post and I believe this is Rick’s point too.

Previous diagram lacks loose coupling so I’d rather draw the candidate’s process in a separate diagram:

I share completely Rick’s concern:

I suggest the collaborative approach where IT and BA work together.

But sorry Rick - I didn’t meet a business analyst that would accept your style, it’s too technical. So we fall back to the traditional way: an analyst draws some vague diagram then throws it over the wall to developers and they convert it into executable code that the analyst is unable to read.

And I don’t believe it must be that technical like it is in Intalio.