Change-Resilient Scientific Workflow Systems with Automated Optimization

OR: Bringing together formal methods of databases and programming languages
  to build change-resilient distributed XML stream processing systems whose
  execution can be automatically parallelised and optimized by the execution
  environment

Daniel Zinn, UC Davis

I will present recent work in that area and outline future research potential.
The following abstract gives more concrete information about my talk and our
current work.

We propose a new stream-processing framework based on a virtual assembly line
model. We instantiate the "Virtual Assembly Line" framework obtaining
\Delta-xml, an approach for designing and optimizing distributed XML
processing pipelines. \Delta-xml greatly simplifies the design of
change-resilient dataflow pipelines: XML processors (called actors) can be
inserted, deleted, and their ``scope of work'' (the parts of the stream they
can read from and write to) changed freely, without compromising the overall
process pipeline design.  Unlike conventional approaches that rely on
adapters to ``glue'' together processing components, our actors employ
flexible configurations that select only relevant portions of the input
stream.  \Delta-xml pipelines are not only more flexible and change resilient
than current approaches, but can also be optimized by compiling them into
dataflow process networks that minimize shipping cost in distributed
settings: Using a static type inference approach based on regular expression
types for XML, we show how to perform a dataflow analysis to determine XML
stream fragments that are relevant to an actor, allowing irrelevant fragments
to be bypassed (``shipped'') to downstream actors.  We also show that our
approach is optimal for distributed XML pipelines, given the type information
available in \Delat-xml.