It ain’t a classic blog post but rather a rolling-out story. No one knows how it will go neither who is the killer 
The reader of this blog Crisitan submitted a comment asking how to model “state machine”-like processes:
Here is my made-up story: we are trying to model and implement a computerized system for handling the lifecycle of some licence obeying the rules below.
The Ministry of Energy of some country issues licences for oil exploration and production to applying oil companies. Oil companies must get a licence before they may legally produce or explore for oil in that country. When companies get a licence from ministry, they are said to own the licence. In order to get the licence, a company must first apply for it via ministry. Any licence application gets reviewed by ministry staff and, if application is approved, it results in the issuance of the licence. A licence has an Issue Date and an Expiry Date. Every time Expiry Date is moved forward, it does so at most one year at a time, but it may be moved forward repeatedly thru the Renew Licence function of the system.
When applying for a licence, the applicant must specify if they want the licence to be issued Right Away or on the coming 1 Apr. If they specify Right Away, the licence will be issued on the day it’s approved (issue date) and it will be valid until next 1 Apr (expiry date). Usually, applications are assessed the day they arrive. If applicant specifies they want the licence to be issued on coming 1 Apr, if approved, it will be issued on the coming 1 Apr and it will be valid until the next 31 Mar coming right after 1 Apr when the licence was issued. A licence may have two statuses: Active or Inactive. If applicant applied for a Right Away licence, if approved, it will have its status set to Active. If the applicant specified they want licence issued on coming 1 Apr, if approved, licence will only be issued on coming 1 Apr, it will be valid until 31 Mar coming right after 1 Apr of issuance, and it will have status Active. The applications also have a status. When companies apply, their applications are stored in the system with status Pending. If application for licence is approved, its status becomes Approved. If application for licence is rejected, its status becomes Rejected. If an application was rejected, the applicant qualifies for a full refund of paid fees (50 dollars).
In order to keep a licence Active, the owner of licence must renew it using a Renew Licence function of system, which does not require staff assessment.Also, an owner may choose to Deactivate an active licence using the Deactivate Licence function of system. If the owner of a licence does not renew their licence before its Expiry Date, it will be deactivated automatically by the system. If a licence is deactivated, it may be reactivated through the Renew Licence function of the system. It doesn’t matter if a licence was inactive for one day or a number of years, it may still be renewed using Renew Licence function. Once a company owns or owned a licence, they may not apply again for a licence. When they need an active licence again, they need to use Renew Licence to bring it back to life. When applying for licence, there is a 50 US dollar fee. Same for renewal. If some company applies for licence, they need to pay the fee. Same for renewal. Also, if the licence owner company does some things which are against the law, their licence will be automatically deactivated if system can detect that. Also, if staff detects some illegal action of licence owner company, they may initiate Deactivate Licence function in the system and choose from a list of potential reasons why the deactivated the licence.
I’m going to model it in BPMN. Other readers are very welcome to participate in discussion and modeling.
Updated 11/16/2013:
1. State Transition Diagram
Let’s start with very simple diagram depicting the licence states and transitions:
I recommend to use the classic STD notation at the first step of process analysis when dealing with statemachine-like processes. We’ll translate it into BPMN later.
There are more than two states:
- Pending is the initial state
- Not Approved is the terminal state
- Granted means licences requested from 1 Apr, approved and waiting for 1 Apr to come
It looks a little bit suspicious that there is no terminal states except “Not approved” - this is because licence “cases” last forever. I’d suggest to close the case - move the licence into something like “Closed” state. But it’s just my guess - Cristian?
Updated 11/17/2013:
2. State Transition in BPMN
Let’s translate the state diagram into BPMN:
Subprocesses:
 |
 |
| Pending | Granted |
 |
 |
| Active | Inactive |
Actionable processes:
 |
| Application |
 |
 |
| Deactivation | Renewal |
Updated 11/17/2013:
3. State Transition in BPMN Data Store
The way a state diagram is implemented above is very powerful. It’s main advantage is that it watches not only events that happend but also ones expected yet didn’t happen, e.g. missed payment due date.
It also lets you implement sophisticated models and versatile escalation flows. In fact you may go beyond the pure state transition model which doesn’t allow parallel execution. The implementation proposed above lets you model e.g. payments in one branch and delivery in the other and watch statuses of both.
But isn’t it an overkill for the simple process under consideration? Sure it is so here is the alternative:
Here the state machine is implemented as a database record with “State” column, possible values are “Granted”, “Active”, “Inactive”. The diagram probably speaks for itself. Couple of notes:
- Incoming data flows from Licence to “Identify” and “Select” activities are not shown for clarity.
- There is one watchdog serving both expired and granted licences. You may opt for two watchdogs; in more complicated scenarios there may be plenty of them, launched daily or hourly rather then annually.
This model actually has more serious advantage over STD implemented as a process than just simplicity: it’s also more robust.
The state machine or lifecycle-like diagrams let you implement long-lasting or even endless processes. The common issue of such processes is this: let’s assume that certain process lasts for e.g. two years. What are the chances that we won’t need to change the process scheme during this period? Almost zero. Now take into account that most BPMS don’t allow to change the scheme of the running process instance on the fly - it’ll follow the scheme it inherited from the process template when it was instantiated. We’ve got the conflict here.
The STD implemented as a process lets you avoid this trap by keeping the lifecycle process as simple as possible and hence not vulnerable to changes. The surrounding workflows are short-living and they may be changed, duplicated etc. with no harm.
Yet there still are the chances that we’ll need to change the lifecycle process. STD implemented as a data store eliminates this threat completely: you may always add another state or another watchdog.
Crisitian
Your description misses the key point: who is the “user” of the process? I mean, shall we model it from the Ministry point of view or from the Oil company’s one? These two would be quite different.
Modeling both sides ain’t a good idea - the two participating organizations are autonomous and shouldn’t have neither need nor rights to dictate other party’s internal operations. They should only agree the protocol or “contract”: “if I send you this then you should respond by that in 10 days, otherwise I’ll treat the case as … and do the following …”.
Sorry, Anatoly.
It should be modeled from ministry’s point of view.
OK, great.
Now how shall the customers (oil companies - is it OK to call them customers? licensors? anything else?) interact with “our” organization (the ministry)? Would it be:
- traditional paper-based and/or email-based communication (no big difference between these), or
- self-service external portal serving as a gateway to the process?
Here is the link to the image of my diagram, but it follows a slightly different definition of the problem.
https://www.dropbox.com/s/vjucojy2gb6kcs3/licenceLifecycleImg.png
Well, the oil companies called clients in the staff jargon may either interact directly with a web application or they may ask staff to interact with the system on their behalf. Either way, they will have to provide some papers proving they are a registered company in the country.
An applicant may also withdraw their pending application and there must be a function called Generate Licence Certificate in the system used to generate and print only active licence certificates.
OK, got it.
Advice: try to avoid talking about systems and functions, think about events, processes and tasks.
Ok, I will. I have the reflexes of a non-BPM-er, but I understand your advice to shift to BPM paradigm of thinking.
Than you, Anatoly. Even though the business rules of handling licences do not say anything about having Closed or Demised licences, it makes sense to say that if licence holder company goes bankrupt or merges into another company, its status as a client (remember the concept of client?) in the system(?) becomes inactive. As a result of holder becoming inactive, its licence may become Closed/Demised, meaning that nothing may bring it back to life. However, a client may have been inactivated by mistake, so probably even a Closed/Demised licence might be brought back to life in the case of an inappropriate client inactivation, once the client is activated again.
Another thing I want to share is that the true life of licence starts only when it is created as Active. There is a state machine to the application and a state machine to the licence itself. You included the Pending and Granted states to the licence state machine, but I think that even though it works like this as well, in fact Granted state of licence corresponds to Approved state of application. I guess you want to consider a continuum between application approval and actual licence issuance when not Right Away. However, when application is approved Right Away, it would quickly produce state Granted on licence and then quickly Active state on licence. Both ways work in my understanding, so it matters less which way you model it. I guess having Granted state as well allows one to query on licence state machine for granted licences as well, so it’s more fine grained and state-rich.
Cristian
1) If we model the process having in mind that any activity could be performed by mistake then it’d become a total nightmare. It’s a valid concern indeed but it should be addressed on other level e.g. by four-eyes principle: important decisions shall be double-checked. So let’s introduce the terminal state for training purposes at least.
2) Generally, an endless business process is a bad idea. What if we’ve rejected client’s application - is there a chance they would make the necessary adjustments and come back for effectively the same licence again? I guess yes. Should we keep the licence in some waiting state? No - it should go to terminal state. When the client come again the first thing we should do is figuring out whether it’s primary or repeating application? If it’s repeating then it’d make sence to keep a reference to the original application in the new one. This way we make the process finite and yet have all the historical information at hand.
3) Do clients miss the renewal date often? I seriously doubt this, it’s rather a business exception.
More importantly, if the licence is inactive, shouldn’t the site (oil field) be available to licencing for other clients? But your model doesn’t allow this! The Inactive licence still “belong” to the intital client in a sence because the client may renew the licence even years after it has become inactive. I really can’t believe it goes this way.
So I’d suggest to make Inactive a terminal state. If the client missed the expiration date then this licence is over and all clients may claim the same site and obtain a new licence. The application will be primary if it’s a new client and repeating if it was submitted by the initial client.
4) As for the application vs. licence states, I believe the application doesn’t need a state machine model. It’s a plain orchestration: “do this - do that - finally do something else - done”. By contrast, licence is driven by external events (initial application submission, renewal request) hence the state machine.
It may seem unconvinent to treat Pending as a “pre-life” status of the licence but this is how it goes quite often. Consider e.g. a client’s purchase order before it is accepted or advertising campaign at assesment phase before the approval - it’s the same pattern.
Also look at it from monitoring perspective: would we like to know how many licences were requested, how long they lasted etc.? We’d get all slices and dices from one source - the licence process - in my model.
Cristian?
One more point:
5) Granted vs. Active. I need Granted to handle the Granted -> Active transition e.g. by setting a timer.
More importantly, Granted and Active are very different statuses from business perspective: if I got it right, the client shall not proceed to exploration and/or production immediately if they opted for “from the 1 Apr” until this date came.
I think I may have misled you with the permit lifecycle concept just before switching to licence lifecycle, because of some confidentiality issues. A licence is required in order to allow companies some operations in the system, such as applying to get a permit for exploration. The licence itself is per client and it is by itself simply a prerequisite for other possible activities of the client. Even though a client would usually not miss renewing the licence, they may choose to temporarily suspend activity for a period of time and then they may want to come back and reactivate licence so that they continue activities in the system. It is like suspending your phone subscription for 6 months because you will be away for so long. When you return home, you reactivate your phone subscription. But for 6 months you had no subscription.
So a licence never ever goes from one client to another, but it may be deactivated for periods of time.
Cristian
Please forget about operations with the system and say it in terms of business operations. I’m totally confused at the moment.
I will address your questions one by one…
1) nothing to add, I find it reasonable.
2) This originates from the fact that you chose to make application lifecycle part of licence lifecycle. If you separate them, it may be cleaner. The licence lifecycle only starts once an application is approved. This means, there could be some rejected applications which don’t initiate any licence lifecycle because there was never a licence. Or, if there was a licence and client applies again, that application would simply revive the inactive licence within the existing licence lifecycle.
3) sorry, I just re-read this and I realize you chose to keep licence and its application in one state machine.
Allow me to post this and re-post based on your choice of modelling the application and licence together.
Sorry, Anatoly. I will try to avoid mentioning the term system. Old habits …
A licence by itself is not something allowing companies to explore for oil, it is only a prerequisite in the same way running for US president requires the candidate to be an american citizen (plus others). There is another concept of permits, which may only be acquired if company trying to get the permit has an active licence. A permit would allow the company to explore for oil, but the licence would not. But because talking of permits would require me to provide some confidential info, I chose to talk about licence lifecycle which is simpler and not so critical with respect to confidentiality. Any lifecycle works fine as a teaching example of the state-machine pattern. I am sorry for not being clear enough.
I understand now why it makes more sense to combine in one lifecycle what I assumed would be two separate ones. I am aligned with your plan now.
I guess states Not Approved, Pending, Granted, Active, Inactive and Closed would be the ones describing licence state.
Unless we don`t consider Closed anymore
This is such a nice pattern! So this way the STD diagram is mirrored into the BPMN diagram…
You mentioned two ways of modelling these cases. Would you mind sharing the other pattern as well, please?
I only assume one would label Pending subprocess something like Handle Pending Licence, Active something like Handle Active Licence etc. I guess you chose short names for practical reasons in this example.
Unless those labels outside subprocess box are not intended to be the subprocess names. Then it makes sense to label them outside the box with state names and omit subprocess label inside the box.
Also, I assume the start event of licence application pool would be a message start event if we showed the client pool as well. That is a part of interest to me also because clients may apply themselves or go thru staff. Probably that would be modelled thru a multiple start events, one message start event when client does it directly, and one none start event when staff does it from Staff process lane. What do you think?
The labels outside subprocess boxes are the names in Bizagi Modeler.
It solely author’s discretion how to name activities. The common recommendation is “verb-noun” style, e.g. “Approve application”, and I agree with it. But the state machine is a special case: you can see that it’s totally task-less. All the “real” work is done in surrounding workflows (Application, Deactivation, Renewal) and the lifecycle process only watches the events generated from these workflows. Hence the subprocesses are named as states - they are states, in essence.
To be precice, the lifecycle process may contain tasks - as part of escalation or business exception. There is one such task in the process: “Reset expiration date”.
As for the message from the client pool,- “People Don’t Talk To Robots”, http://mainthing.ru/item/602/. The way it’s modeled above covers two scenarios:
- client talk with internal user and the latter starts the “Licence Application” process
- client has access to our system and is able to start the same process himself
The message start should be reserved for the scenarios where client don’t have access to our process system but there is some external application (e.g. a clients web portal) that interacts with client on the front end and with the process at the back end. If this scenario and scenarios above should be served then multiple start is valid approach, you are right.
There may be other styles indeed but this is what I’ve developed and follow.
I understand. So state machine modelling would work better by naming subprocesses by State name rather than something like verb-noun. I will look at item 602.
The case we have is: both clients and staff would gain the same kind of access. Just that some processes are only accessible to staff (such as assessing an application). while others are accessible to both (client may apply directly or they may apply thru staff as their proxy).
Is there another pattern for state machine? You mentioned two ways of doing this.
I also dare to ask one more thing. Let’s say you have some permit for exploration for oil. A permit is something allowing a company to explore for oil and a permit is only granted to companies that already have an active licence. No other thing is required in order for company to start exploring. Licence acts as the prerequisite for getting a permit (and for getting other things as well, but this question is related to permits). The permit lifecycle may be modeled the way you showed here, I can’t provide too many details because of confidentiality. For example, let’s say some permit accumulates some credits in its lifecycle thru some mechanism which I know how to model. Those credits are normally only usable by the permit itself for its benefit. But permits may become grouped if client so chooses (business rule) and if multiple permits are grouped thru a grouping event, their credits become accessible to every member of the group. How would this be modeled? Would there be a Group pool iterating thru permits and sending a message to each pool instance representing the individual permits to be grouped?
Cristian, I suggest treating credits just as plain data.
Yes, that is how I want to treat them. But grouping event causes their availability within group members. And grouping operates on multiple process instances, one for each permit being grouped. If permit is not grouped, it may not access credits from other permits and it may not give access to its credits to other permits either. I assume every event in the process must be modeled to provide the complete process definition. I am trying to show more than necessary on the process model?
Sorry Cristian, can’t help you with this. Need to have the whole picture and that’s impossible.
Ok, Anatoly. Thank you so much for all the super-generous help. I will try to figure out on my own.
I guess this is the second approach you were mentioning some time ago on item 692 comments
Anatoly,
Could you elaborate more on the statement “In fact you may go beyond the pure state transition model which doesn’t allow parallel execution.”, please?
I mean, I am not clear what you mean by parallel execution in this case
Well… processes may have parallel gateways and parallel flows. State machine is single-threaded.
Let’s consider a sale as an example. It may be treated as a state machine: spec is agreed, contract paid, goods shipped etc. But it’d only work until you’ll perform activities strictly sequentially. E.g. if shipment and payment may go asynchronously (which is quite common) then there won’t be states “paid”, “shipped”.
BPMN can handle such scenarios while classic STD can’t.
Thank you, Anatoly. I know what parallel flows means in general. I was wondering in our case where is that lack of parallellism if we model as state machine. But your example with shipment and payment made it clear for me in the case of licence as well. Because a state machine only has one active state, right?
Exactly.
Hi Anatoly,
If instead of having separate pools in ‘3 State Transition in Bpmn Data Store’ we used one single pool for all pools and then right after start event we had an event gateway waiting for intermediate message events Licence application, Licence renewal, Licence deactivation or Licence watchdog and we used one subprocess for each gate and at the end looped back to event gateway, would there be any inconvenience other than the fact that all these subprocesses are now coupled in one main process? So change to one of the subprocesses would imply change in main process. The advantage to this approach is one can see the big picture in one process diagram, I think.
Doesn’t make sence to me. Exactly what event starts the process? Does it starts exactly at the moment e.g. a Licence activation request has come or…?
What you call “the big picture” is actually a big drawback and not an advantage at all. The beauty of the diagram 3 and at some lesser extent of diagram 2 is loose coupling. It means that one can replace a given implementation of a workflow constituing the process by modified just this one workflow without affecting anything else. You switched to tight coupling which makes all construct more fragile. It’s like exe vs. ddl linkage, dude.
Yes, it starts when an application arrives. I was just asking for a point of view, Anatoly. And I was aware of this coupling if you read my comment again. So you are saying that the licence lifecyle is made of multiple pools, in your prefered modeling way…but then what about the suggestion to model as orchestration as much as possible? I am still a learner, please keep in mind. This was not a critic of your approach, I am just trying to learn here and I thank you for reacting to receiving ‘Question from Cristian arrived’ event… But message events are automatic, I know
Cristian
If you have a none start event followed by intermediate event gatway then your process won’t start when something arrived, it’d start when someone started it by hands. You may use starting event gateway though.
There is no contradiction in my words. “As long as possible” means just this - as possible yet no longer. When one task follows another, it’s orchestration. When an externally-triggered event happend, one have to respond to it, there is no way to command such events, they are beyond our control by definition. Switching to collaboration without a sound reason is equally bad to keeping an orchestration for too long.
As an alternative, a Message Start event “Application for Licence Arrived” could be used instead of none start event, followed by a sub-process for handling application which is followed on approval by an event gateway and on rejection by a process end event “Application Refused”.
Okay, I understand that “as much as possible” leaves room for disobeying this rule, but my question is: how to decide when to follow it and when to break it? I am not focusing on proving you wrong, I am focusing on the things which are unclear to me, I am not here to ‘attack’ someone that gives me their time and energy. I am sorry if I give this impression at all.
I think one can either have one pool called Licence Life-cycle and use the event gateway approach and I understand that the goal of this process if pretty vague, so from this point of view, besides the coupling you mentioned, I feel it’s something that could make way for a better approach. The better approach could be the way you modeled as separate pools. That way we have less coupling and more goal-oriented processes. Right or wrong?
I think my ignorance and learning curve are also slight contributors to your blog, they activate not only your teaching skills to the readers’ benefits, but also your exercise of generosity (which makes me feel constantly indebted to you).
Cristian
Your questions are ok, sorry if I provided an agressive impression.
Formally there are plenty of modeling opportunities.
It’s simply a mindset: an event starts a process instance unless there is an active instance already waiting for the event to happen. Different events = different workflows. Pretty straightforward for me.
Sorry if I couldn’t induce it to you. Probably have to find better arguments. But not at the moment, definitely.
Thank you, Anatoly. I am a like a vacuum these days, I am trying to absorb BPM(N) knowledge from anywhere I can, but this is my favorite place, by far.
Blog you later,
Cristian
Hi Anatoly,
Is there a link to an index of all your blogs/articles/others so I can read everything you posted on your site?
Thank you.
Christian
Maybe the tag BPMN http://mainthing.ru/tag/bpmn/ is what you are looking for?
There is also RSS feed http://mainthing.ru/feed/
Thank you, Anatoly.
Anatoly,
Regarding comment 10 - section 1) …
Would you ignore modeling of errors done by the performers of tasks in the bpmn diagram? Or at least, would you ignore some of them? If so, let’s say you execute the process in a process engine and an error ignored in the diagram model happens. How do you handle that case when it happens? And how do you reenter the normal flow?
Cristian - there are plenty of possible errors, which ones do you have in mind? E.g. there may be a server crash, communication failure or a human performer may enter wrong value in a screen field.
Entry error or missing some check that should have been done…I know you will point me to four-eye principle, but I find it so cumbersome to double activities over and over…
Cristian, either your application (i.e. a web form) is smart enough to check for such errors and in this case the task just wont be completed until the user entered it all right. Nothing to depict on the diagram except an annotation if analyst is eager to pass the validation requirements to a developer.
Or the application can’t do such check - in that case the error introduced by a user will go down the process flow unless not double-checked by someone else.
We don’t consider magic wands, orbs or similar instrument, do we?
I was rather thinking of errors which may not be picked up by validation, that case is clear. Let’s say validation accepts integers, but user types in 2 instead of 4. They won’t be caught by any validation. So the only way left is double-checking…But even in that case there may be some handwriting interpreted by double-checker the same way as the other person. In which case…some compensating action (not a compensation event!) would be required, I guess…
Thanks.
I re-read this post today and couldn’t resist telling “thank you”, again!