Intelligent Workflow - State of the Art in Workflow


This summary of our literature search in the area of workflow is intended as an introduction to the basic concepts and a statement about exisiting achievements and challenges within the field.

The first section gives the overview of the problems addressed by the workflow community and the basic concepts behind the workflow solution. The summary then divides the technical state of the art discussions into 2 broad fields:

  1. Process modeling techniques and tools including process specification languages
  2. Workflow management systems often referred to as workflow engines

Overview of Workflow Domain

Workflow is a domain that has evolved from several fields, business management, modeling, simulation and, to a lesser extent, planning and control. It has primarily evolved from the need to understand, to organise and, often ultimately, to automate the processes on which a business is based. However, it can also be characterized by the analysis and automation of processes of any kind involving combinations of humans and machine-based activities, particularly information intensive activities. Workflow, in general, is defined by the Workflow Management Coalition (WfMC) as
"The automation of a business process, in whole or part, during which documents, information or tasks are passed from one participant to another for action, according to a set of procedural rules." (WfMC '96)
We have emphasised the work business in the above definition because we see workflow as being applicable to many types of process not only business processes and therefore the techniques of workflow management as applicable to a wide variety of coordination and control problems. However, this summary will concentrate on the evolution of workflow within the business management domain as it dominates the literature.

In a final section we will explore the implications of policy and technologies being developed to allow effective statement of policy and integration with the emerging workflow tools and techniques. Figure 1 below is adapted from one by the WfMC in its Reference Model (WfMC '94)

WfMC-Architecture

Figure 1: Architecture for Current Workflow Management System Technology

Process Modeling Tools and Techniques

Abstract or graphical representations of an organization's processes are called "process model". Process models are an excellent tool for both the analysis and reengineering. However, they also form the basis of most automated workflow management systems. The purpose of process modeling is to ``produce an abstraction of a process (model) that serves as a basis for the workflow specification'' [Cichocki etal. 98]. Process models can be used for improved human understanding of operations within a domain, or to assist with analysis or re-engineering. Additionally, they can serve as the basis for automated workflow control. Modeling as a discipline had been around for some time and there are three basic approaches to modeling a process: Communication-based methodologies are based on "The Winograd/Flores Conversation for Action Model" (Winograd and Flores '87). An action in a workflow is represented by the communication between customer and performer. In artifact-based methodologies objects, or artifacts, are created, modified and used during the process and thus the model is based on work products and their paths through a series of workflow activities. An increasingly comkmonly used methodology in the workflow management domain is activity-based modeling. Here the overall process is decomposed into tasks that are ordered based on the dependencies among them. This model can most easily be transfered into a workflow specification. However, many technicians use a combination of these methodologies producing a set of models for an organizational process based on an information flow model (communication-based) a capabilities model (artifact-based) and a process-model (activity-based).

A process specification basically an activity-based model of a process. The WfMC has constructed a glossary of terms, (WfMC '96), from which Figure 2 the following diagram indicates the relationship between terminology.

WfMC glossary

Figure 2: WfMC Glossary of Workflow Terms
There have been several specification languages used to construct process models. Here is a list of a few: Other examples of representations or process modeling schema include CIMOSA(Kosanke'93), Petri Nets(Murata '89) and Finite State Automata(Peluso et al.'94). These like IDEF3 are based on the input-process-output paradigm in contrast to speech-act or communication paradigm.

There have also been significant development in recent years in the availablity of tools to aid in the creation and definition of "business" processes. These tools allow the user to describe a process usually using a mixture of graphical and textual building blocks. Thee resulting representation can then be easily translated into a process specification language. Examples of such systems include :

Workflow Management Systems (Workflow Engines)

The role of a workflow management system is to provide the services required for automating workflow processes. WFM systems can be characterized by the following functional components:

At the heart of a WFM system lies a library of templates that encode explicit process models for the relevant problem domain. Templates can be represented as explicit sets of tasks or activities to be undertaken, or more abstractly as collections of constraints on allowed activity. Activities can encompass a broad array of operations, including transmission of data, tasking of software agents, or communication with human operators. As reflected in Figure 1, process modeling and representation are typically limited to build time within current WFM technology. However, process creation and adaptation will inevitably migrate from build time to runtime as application domains demand flexible adaptation to environment dynamics.

Templates are selected for activation based on current tasking and environmental conditions. Selected templates can be tailored to a range of situations through appropriate instantiation of template variables. Typically, activation results in the addition of the constituent activities of the instantiated template to an activity list, which contains information about all current and pending tasks. As part of activation, resource allocation is performed for new activities based on current and projected resource availability. These activities are then scheduled in accordance with temporal sequencing constraints specified within the process template

The term enactment is used generally to refer to the execution and ongoing management of activated processes. For many domains, process execution will occur within highly unpredictable and dynamic operating environments. For this reason, enacted processes must evolve and adapt over time in response to a number of factors, including changes in the environment, the addition of new tasks, and partial execution results. Monitoring plays a critical role during enactment, to detect key events that may necessitate adaptations to current processes, or enactment of new or different processes. Similarly, process templates should evolve over time, to reflect improved models of the domain obtained by analyzing information gathered from previous enactment episodes.

The Workflow Engine

A workflow engine provides the runtime environment for activating, managing, and executing workflow processes. Different models for workflow engines can be employed. The WfMC focuses on a paradigm in which the engine instantiates a workflow specification, decomposes it into taskable activities, and then allocates activities to processing entities for execution. This approach distinguishes between the process definition, which describes the processes which may be executed, and the process instance, which is the actual enactment (execution) of the process. This paradigm is referred to as the scheduler-based paradigm [Cichocki etal. 98].

The implementation and deployment of the scheduler-based approach to workflow can be described in terms of a state transition machine. Individual process or activity instances change state in response to workflow engine decisions or external events, such as the completion of an activity. Thus, to give a few example states, a process instance may be initiated once selected for enactment, active after at least one of its activities has been started, suspended (perhaps waiting for some event), or completed. Similarly, an activity may be inactive, active, suspended, or completed. It is the role of the workflow engine to manage this state transition machine, selecting processes to be instantiated, initiating activities by scheduling them to processing components, and controlling and monitoring the resulting state transitions. The workflow engine must also implement the rules that govern the transitions between tasks, updating the processes as tasks complete or fail, and taking appropriate actions in response.

The scheduler-based paradigm is the most common, and as such will be the focus within this report. However, two alternative paradigms for enactment are worth mentioning, namely data-flow and information pull [Cichocki etal. 98].

Process Selection

One key responsibility of the workflow engine is to manage the selection and instantiation of process templates. The engine will respond to some stimulus (i.e., a triggering event) by selecting a suitable process from the library of templates. Examples of possible triggering events include arrival of a new user request, generation of a product by an already active process, or even the passage of time. The workflow engine manages the instantiation of the relevant process. There may be alternative applicable processes that must be compared to the triggering conditions and selected appropriately. In many existing WFM systems this task is trivial, as there is no or little choice among processes given the predefined stimulus for enactment. But there are domains where there may be many, or even no, directly applicable and valid processes for a given stimulus, thus requiring process selection, adaptation, or even dynamic process creation.

Task Allocation

Once a process is instantiated, the workflow manager forwards activities to an activity list manager to control the tasking of processing entities for the activities. An activity is assigned to a processing entity according to the capability and availability of the entity and temporal and sequencing constraints of the activity. This allocation task can be viewed as a scheduling problem. Thus, the workflow engine takes a centralized role in coordinating the operation of processing entities. The activity list manager can be implemented as a simple in-tray of work items or a complex set of agenda-based load balancing and interrelated work assignments with complex supervisory roles.

Scheduling techniques within workflow management systems have employed straightforward enumerative or heuristic-based algorithms to date. As the complexity of WFM domains increases, more sophisticated approaches that provide robust reactive scheduling will be critical to accommodate processing entities that require some autonomy, can be tasked by other workflow managers, may break down, or have variable capabilities. For example, there has been some work on exploiting the dependencies between activities within the workflow task allocation problem [Attie et al.'96].

Enactment Control, Execution Monitoring, and Failure Recovery

The workflow engine must maintain all the knowledge and internal control data to identify the states of each of the individually instantiated activities, transition conditions, connections among processes (for example, parent/child relationships), and performance metrics. The WfMC defines two types of data relevant to the control and monitoring of workflow processes:

Workflow Control Data
encompasses state information about processes, activities, and possibly performance criteria. It is internal information managed directly by the workflow engine.

Workflow Relevant Data
is external information used by the WFM system to determine when to enact new processes and when to transition among states within enacted processes.

There has been a significant amount of work on control within workflow enactment systems, based primarily on state transition mechanisms (as described above) and transaction processing. However, current-generation WFM systems have limited abilities to recover from failures, with many simply assuming that operations will complete successfully. Workflow failures may result from a processing entity not completing a task correctly or on time, or by an unexpected event in the environment. WFM systems should provide a variety of recovery mechanisms, from the clean termination of a process to the adaptation of the process to the failure.

Work in this area to date has focused on the use of transactional methods to ensure data integrity, especially when concurrent activities must share data. For example, most WFM systems offer only basic locking mechanisms to protect data, although there has been some work on longer-lived transactions [Garcia-Molina et al `90], [Reuter '89], [Mohan et al `95]. These approaches focus on returning the state of execution to a previously known safe state. Two popular techniques are checkpointing and roll-back.

These approaches, while suitable for certain forms of data-driven workflow, are inadequate for more general domains that involve highly dynamic environments in which the world changes continuously and unpredictably, and in which actions may not be undoable. More powerful and flexible methods for failure recovery are required in order for workflow technology to be effective in a broader range of application domains. Specific requirements include the ability to adapt processes at runtime in response to failures or changes in situation, and to accommodate open-ended activities.

Representative Workflow Management Systems

A diverse set of commercial and academic WFM products is currently available. Here, we describe several key systems that span a range of approaches and capabilities.

MENTOR [Wodtke et al. `96], [Wodtke and Weikum '97]
is a WFM system aimed at large-scale applications. It adopts a typical WfMC approach with workflow being defined in a specification formalism and transformed into distributed runtime executable workflow. An interesting contribution is that the transformation involves a semantic partitioning of a state chart to produce components that can be executed on different processing entities. This system has demonstrated that state and activity charts are suitable as a workflow specification formalism. As yet, MENTOR cannot support dynamic modification of workflow specifications.

WIDE [Ceri et al.`97]
is a distributed architecture for workflow management. WIDE uses enhanced database technology as the basis for its workflow management system and focuses on extended transaction management. It provides reactivity through the use of a decoupled rule execution model. Rule execution is divided into scheduling and interpretation of rules. Triggers inserted into the database management system cause events to be noticed. The scheduler inspects the event database and matches the event to the rules, which are then recorded in a to-execute list and interpreted for execution. WIDE is distinguished by its exploration of distributed WFM, and its sophisticated transactional capabilities. It adopts a two-layered transactional model: the upper layer provides global transactions with loose semantics and the lower provides strict, more conventional, transactional semantics. The two layers are orthogonal, providing one possible approach to long-lived transactions.

MIGRATING WORKFLOWS [Cichocki etal. 98]

As mentioned earlier, some researchers have been exploring a data-flow paradigm as an alternative to scheduler-based workflow. Influenced by INCAs [Barbera et al '96], the Migrating Workflows system has been implemented based on

This effort is distinguished by its emphasis on reactivity and robust handling of failure.

METEOR [Miller et al '98], [Sheth '97], [Sheth and Kochut '97]
METEOR (Managing End To End OpeRations) provides a range of products, including the METEOR Designer, the WEBWORK enactment system, and ORBWork, all grounded in the activity-based paradigm. It addresses the problem of compound transactions through a two-phase commitment coordinator, and provides a platform for experimentation with algorithms related to workflow, particularly tasking algorithms.

The Role of Policy

References

Bibliography

Other Relevant Links

Back to SWIM Project Page
Back to AI Center Home Page
Back to SRI International Home Page

Pauline M. Berry berry@ai.sri.com
Last modified: 2/10/98 Show Tools Section Show WFM Engine Section