Tuesday, November 18, 2014

camunda BPM engine: use custom VariableType to resist the urge to flush

Introduction

I hope all of you are aware of the fact that you can provide a ProcessEnginewith your own VariableTypes. If not, I'll give you a short introduction. Please note that my descriptions are based on camunda-engine 7.1.0. There will be some changes in versoin 7.2.0 and I am not sure if my observations will still be true.

VariableType

 

VariableTypes help the ProcessEngine store your process variables in the table ACT_RU_VARIABLE. I would call them a mediator between the possible variables and the database schema. There are VariableType implementations for
  • Boolean
  • Serizable
  • Date
  • Double
  • Integer
  • JPA Entities
  • Long
  • Null
  • Short
  • String
  • and CustomObjects (about which I'll talk later)

If you try to add an object as process variable, which doesn't belong to one of those types, you'll see this exception:

org.camunda.bpm.engine.ProcessEngineException: couldn't find a variable type that is able to serialize \<object\>
    at org.camunda.bpm.engine.impl.variable.DefaultVariableTypes.findVariableType(DefaultVariableTypes.java:62)
    at org.camunda.bpm.engine.impl.persistence.entity.VariableScopeImpl.getNewVariableType(VariableScopeImpl.java:315)
    at org.camunda.bpm.engine.impl.persistence.entity.VariableScopeImpl.createVariableInstance(VariableScopeImpl.java:395)
    at org.camunda.bpm.engine.impl.persistence.entity.VariableScopeImpl.createVariableLocal(VariableScopeImpl.java:332)
    at org.camunda.bpm.engine.impl.persistence.entity.VariableScopeImpl.setVariable(VariableScopeImpl.java:259)
    at org.camunda.bpm.engine.impl.persistence.entity.VariableScopeImpl.setVariable(VariableScopeImpl.java:242)
    at de.blogspot.wrongtracks.StoreDataDelegate.execute(StoreDataDelegate.java:9)
    at org.camunda.bpm.engine.impl.delegate.JavaDelegateInvocation.invoke(JavaDelegateInvocation.java:34)
    at org.camunda.bpm.engine.impl.delegate.DelegateInvocation.proceed(DelegateInvocation.java:39)
    at org.camunda.bpm.engine.impl.delegate.DefaultDelegateInterceptor.handleInvocation(DefaultDelegateInterceptor.java:42)
    at org.camunda.bpm.engine.impl.bpmn.behavior.ServiceTaskJavaDelegateActivityBehavior.execute(ServiceTaskJavaDelegateActivityBehavior.java:49)


Provide your variable type

 

Every ProcessEngineConfiguration should have the methods setCustomPostVariableTypes(List<VariableType>) and setCustomPreVariableTypes(List<VariableType>) so you can add your variable types when configuring the engine.
But wait, why are there two methods, pre and post?
When searching which VariableType can handle the object you want to store as process variable the engine iterates over the list of VariableTypes and the first one, which can handle the object, wins. Maybe you want your own types to have precedence over the default types.

Flushing

 

Now that you know about VariableTypes I want to present to you my use case.

The case

 

Imagine a process that's supposed to run synchronously (i.e. without a wait state) within a JTA transaction and every task needs a result from the preceding one. Additionally, the results are JPA Entities. By default the JPAEntityVariableType would take care of the entity.
The implementation shows that every time setValue() is called the JPAEntityVariableType calls flush() on the EntityManager. Since the process runs synchronously within a transaction the flush results in unnecessary queries on my database during process execution.

The solution

 

Here comes the CustomObjectType class. The CustomObjectType only needs a name and a class to work. The class is used to determine if it can handle a certain object. The CustomObjectType stores all objects in the cache of the ValueField. To get rid of the flush I instantiated a CustomObjectType with the class of my result and passed it to the configuration. Now, every time I put an entity inside the process variables the CustomObjectType places them inside the cache and no flush is called.

 

 The downside

 

Well, nothing comes without a price: If I should ever need a wait state my solution won't work and I'll have to find another solution or live with the flush.

 

Alternatives

 

I am not sure if my solution is the best way to solve my problem. If anyone knows a better way please let me know.

 

Small example

 

I also created a small example to show the use of the CustomObjectType here on GitHub

Tuesday, November 11, 2014

camunda BPM engine: How many tasks can you execute without wait state

Today I got quite curious today about this topic and I don't know if anyone ever wondered/tried.
At work I introduced the camunda BPM engine a few months ago. One requirement was not to reach any wait state during the execution. That ways we want to make sure the process ends synchronously and we don't show stale data.

As you can image the stacktraces got pretty big when an exception occurred during the end of the process (>1000 lines). So I wondered how many tasks could be executed before the java stack is full (or anything else unexpected happens).
To try this I wrote a simple Java class:

public class Main {
private static final int NUMBER_TASKS = 100;

public static void main(String[] args) throws InterruptedException {

    ProcessEngineConfiguration configuration = ProcessEngineConfiguration
            .createStandaloneProcessEngineConfiguration();
    configuration.setJdbcUrl("jdbc:h2:mem:test;DB_CLOSE_DELAY=-1");
    configuration
            .setDatabaseSchemaUpdate(ProcessEngineConfiguration.DB_SCHEMA_UPDATE_CREATE_DROP);
    configuration.setJdbcUsername("sa");
    configuration.setJdbcPassword("");
    configuration.setHistory(ProcessEngineConfiguration.HISTORY_NONE);

    ProcessEngine engine = configuration.buildProcessEngine();
    BpmnModelInstance bpmn = erzeugeBpmn();
    engine.getRepositoryService().createDeployment()
            .addModelInstance("manyTasks.bpmn", bpmn).deploy();
    RuntimeService runtimeService = engine.getRuntimeService();
    ProcessInstance processInstance = runtimeService
            .startProcessInstanceByKey("manyTasks");
    System.out.println("Done");

}

private static BpmnModelInstance erzeugeBpmn() {
    AbstractFlowNodeBuilder<?, ?> builder = Bpmn.createProcess().id("manyTasks").executable().startEvent();
    for(int i = 0; i < NUMBER_TASKS; i++){
    builder = builder.serviceTask().camundaClass(EmptyDelegate.class.getName());
    }
    return builder.endEvent().done();
}

As you can see, nothing fancy (and I am still happy that there is a Java API for generating BPMN).
My computer has two Intel Core i7 with 2.9GHz and 16GB of RAM and runs Java 1.7.0_72 64bit. To run this I used the Eclipse defaults (Kepler SR2 x64):

--launcher.XXMaxPermSize
256M
--launcher.XXMaxPermSize
256m
-Xms40m
-Xmx512m


Ten and 100 tasks are no problem.
To get an overview of the size of the stacktraces I'll add a JavaDelegate, which throws an exception at the end.
So the method looks like this:

private static BpmnModelInstance erzeugeBpmn() {
   AbstractFlowNodeBuilder<?, ?> builder = Bpmn.createProcess().id("manyTasks").executable().startEvent();
      for(int i = 0; i < ANZAHL_TASKS; i++){
        builder = builder.serviceTask().camundaClass(EmptyDelegate.class.getName());
      }
      return builder.serviceTask().camundaClass(ExceptionDelegate.class.getName()).endEvent().done();
   }

Also, I'd like to see how log output grows, so here are the numbers. The additional task is always the exception task so the numbers are 11, 101, 1001...
  • 11 tasks: 392 lines log
  • 101 tasks: 1025 lines log
  • 1001 tasks: SOF
Yay, I reached the limit ;-)

Exception in thread "main" java.lang.StackOverflowError
    at java.lang.ThreadLocal$ThreadLocalMap.getEntry(ThreadLocal.java:376)
    at java.lang.ThreadLocal$ThreadLocalMap.access$000(ThreadLocal.java:261)
    at java.lang.ThreadLocal.get(ThreadLocal.java:146)
    at org.camunda.bpm.engine.impl.context.Context.getStack(Context.java:95)
    at org.camunda.bpm.engine.impl.context.Context.getCommandContext(Context.java:46)
    at org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.performOperationSync(ExecutionEntity.java:728)
    at org.camunda.bpm.engine.impl.persistence.entity.ExecutionEntity.performOperation(ExecutionEntity.java:719)
...

It seems like I gotta take some smaller steps:
  • 501 tasks: SOF
  • 401 tasks: SOF
  • 301 tasks: SOF
  • 201 tasks: 1025 lines log
  • 151 tasks: 1025 lines log
Wait, what?
Yes, strangely Eclipse always shows me the same amount of lines for the exception after reaching a certain threshold. And no, I didn't limit the console output in Eclipse. If anyone knows why this limit exists, please let me know.

The task limit I reached was 268 tasks (267 "normal" ones and one exception task).
I am not sure about the practical implications of my "research" but as I said, I was just curious.
Maybe we can agree that processes of a certain size should reach a wait state due to organizational and technical reasons ;-)

EDIT: Please note that when executing a task as multi instance every loop counts as one (I learned that the hard way ;-) )

 

Copyright @ 2013 Wrong tracks of a developer.

Designed by Templateiy