Category Cross-Omics>Workflow Knowledge Bases/Systems/Tools

Abstract Meandre is a semantic enabled web-driven, dataflow execution environment.

It provides the machinery for assembling and executing data flows - software applications consisting of software components that process data (such as by accessing a data store, transforming the data from that store and analyzing or visualizing the transformed results).

Within Meandre, each flow is represented as a graph that shows executable components (i.e., basic computational units, or building blocks) as icons linked through their input and output connections. Based on the inputs and properties of an executable component, a unique output is generated upon execution.

Meandre also provides publishing capabilities for flows and components, enabling users to assemble a repository of components for reuse and sharing.

This allows users to leverage other research and development efforts by querying and integrating component descriptions that have been published previously at other shareable repository locations.

Meandre Architecture –

Meandre builds on three main concepts:

1) Dataflow-driven execution;

2) Semantic-web metadata manipulation; and

3) Metadata publishing.

1) Dataflow Execution Engines –

Conventional programs perform their computational tasks by executing a sequence of instructions. One after another, each code instruction gets fetched and executed. Any data manipulation is performed by these basic units of execution.

However, data-driven execution (or dataflow execution) revolves around the idea of applying transformational operations to a flow or stream of data. In a data-driven model, data availability determines in what sequence code instructions are executed.

The analog of the dataflow execution model is the black box operand approach.

All dataflow execution engines provide a scheduler that determines the firing (execution) sequence of components. Meandre uses a decentralized scheduling policy designed to maximize the use of multi- core architectures. It also allows a custom threading priorization system to be built on top of the Java threading system.

2) Semantic Web 2.2 Concepts –

The ‘Semantic Web’ provides a common framework that allows data to be shared and reused across application, enterprise, and community boundaries. It is a collaborative effort led by W3C. In W3C’s words:

“The Semantic Web is about two (2) things. It is about common formats for integration and combination of data drawn from diverse sources, where on the original Web is mainly concentrated on the interchange of documents.

It is also about language for recording how the data relates to real world objects. That allows a person, or a machine, to start off in one database, and then move through an unending set of databases which are connected merely by the network but by being connected semantically.”

The semantic web effort relies on the Resource Description Framework (RDF). RDF is a simple notation for expressing graph relations. It relies on XML to provide a set of conventions and exchange information. It also provides a mechanism to express metadata and to determine how that metadata is arranged.

3) Publishing Schemes –

Meandre relies heavily on publishing schemes to create a distributed repository of shareable components. Each piece of the repository is published on some reachable, web-based location. RDF standardizes the publishing process.

For instance, this standard may be expressed as a simple RDF file containing the description of the components for a SPARQL call to a metadata store (that call is usually embedded on the options of the URL following the SPARQL protocol).

In Meandre, each component description is self-contained, in terms of having all the required information for its retrieval, regeneration, and execution.

Meandre’s publishing scheme allows dynamic inspection of published repositories. That is, Meandre can inspect locations (local files, remote web objects, or metadata stores using the SPARQL protocol) to discover a new location where components are published.

This way, the discovered components can be retrieved to form a custom- made repository, which can also be published for others to use.

Hence, different component views and flavors are easy to create. Moreover, this scheme simplifies upgrading components and fixing bugs.

Meandre Components –

A Meandre component serves as the basic building block of any computational task that is run on the Meandre semantic, data-driven workflow.

There are two (2) kinds of Meandre components:

1) executable and

2) flow.

Regardless of type, all Meandre components are described using metadata.

Executable components also require an executable implementation form that can be understood by the Meandre execution engine.

Meandre Workbench –

Meandre Workbench is a visual programming environment that allows users to easily connect software components together in a unique data flow environment.

This application relies on the Google Web Toolkit (GWT) and is accessed via your Internet browser. You can use this interface to develop diagrams of data operations relevant to your research.

Each operation is represented by an icon, and the icons are linked together in a flow representing the movement of data through each operation. Each of these icons represents a component. These software components are reusable components that facilitate collaboration among developers.

These components can be written in Java, Python, or Lisp. A set of components can be loaded into your Workbench for creating a flow (application).

Flows are essentially applications composed of components connected together. Flow complexity is limited only by the needs of your project. Pre-built flows can be loaded into the Workbench and modified as needed.

A number of flows for use in a variety of data mining problems and domains have been developed and can be added to your Workbench.

Components and flows have tags and additional metadata associated with them that can be used to assist in searching and sorting.

ZigZag –

Meandre’s success relies on its ability to easily assemble data- intensive flows.

Flows may be easily assembled by using the Meandre Workbench (MW) (see above...), an icon based programming environment for assembling data-intensive flows.

ZigZag targets rapid application development by speeding up the flow construction cycle.

ZigZag can accelerate the data-intensive flow development cycle. It allows you to easily describe a data-intensive flow using the ZigZag language, which can then be compiled into a self-contained flow task for later execution.

System Requirements

Contact manufacturer.


Manufacturer Web Site Meandre

Price Contact manufacturer.

G6G Abstract Number 20522

G6G Manufacturer Number 104139