Enabling Grids for E-sciencE
Astronomical data
processing workflows
on a service-oriented
Grid architecture
Valeria Manna INAF - SI
The 17th Global Grid Forum - GGF17
Tokyo, Japan
May 10-12, 2006
www.eu-egee.org
Motivation and Objectives
Enabling Grids for E-sciencE
Working Group:
Claudio Vuerli, Giuliano Taffoni, Valeria Manna,
Andrea Barisani, Fabio Pasian
•
•
Problem analysis related to the integration of
astronomical applications in a Grid environment
Description of astronomical tasks sequences
with workflow languages to process complex
compositions of astronomical applications
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006
2
Astrophysics and distributed computing
Enabling Grids for E-sciencE
Computational problems
•
•
•
•
Theoretical Codes
Computation and analysis of many TBs of images
Database of catalogues of the detected elements
data-mining tools
- Data source and organization interaction
- Geographically distributed resources (database, telescope, storage, etc)
Request of distributed computing to provide:
•User virtualisation
•Resource virtualisation
GRID
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006
3
Astrophysics Application
Enabling Grids for E-sciencE
Tasks:
•Database Access
•Data Reduction
•Pure Computational
•Instrument Monitoring
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006
4
Workload Management
Enabling Grids for E-sciencE
The task sequence of the astronomical
application is managed by a
Workload Management System that
makes a task- job mapping
User :
– Submit a job
– Create some script for the monitoring
of job status
– Create script to create some task
connecting link
– Recover I/O data
Need to a Workflow
Management System
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006
5
Workflow Management
Enabling Grids for E-sciencE
A workflow language allows
to describe the execution
flow of pipeline tasks :
–Query astronomical catalogues
–Data transfer
–Synchronize job
–Analyse the resulting data
Mapping task → service
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006
6
Web Services Composition
Enabling Grids for E-sciencE
Web Services Composition
The composition of WS consists of providing logic around a set of
interactions between the composition and the WS that participate in it.
These interactions are simply invocations to the operations offered by
the services in play. One approach to providing the control and data
logic is the use of workflow specifying:
–
the execution order of operations from a collection of WS
–
the data shared among composed WS
–
partner involved
–
how the partner are involved in the flow process
Service orchestration
web services can interact with each other at the message level,
including the flow logic and execution order of the interactions. For
orchestration, the process is always controlled by the perspective of
one of the parties.
Service choreography
typically associated with the public message exchanges that occur
between multiple web services, rather than a specific business process
that is executed by a single party. Each part involved in the process
describes the part it plays in the interaction.
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006
7
A workflow language: Bpel
Enabling Grids for E-sciencE
Bpel (Business Process Execution Language)
is a workflow-based composition language for Web Services
• Standard
• Open Source
• Support the Service Orchestration
Feature:
– Interaction: define WSDL port types for each interface that is
used in the workflow definition
– Basic Activity: to allow for interaction with the applications
being composed invoke, reply and receive
– Structured Activity : conditional execution and iteration /
recursion of a sequence of activities to manage process flow
Likely to be adopted as EGEE standard
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006
8
Bpel: process flow
Enabling Grids for E-sciencE
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006
9
Hypothetical Scenario using Bpel
Enabling Grids for E-sciencE
CREAM
EGEE
SERVICE
WF
WEB
SERVICES
GRID/WEB
SERVICES
BPEL ENGINE
ASTROGRID
SERVICES
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006
10
WSIF
Enabling Grids for E-sciencE
Problem
Currently the LCG middleware is not Web Service oriented and in
GLite (middleware of EGEE project) the WM and the other
services are expected to offer a Web Service interface.
In this scenario the use of BPEL is not feasible
In a BPEL workflow it is very useful to provide a set of services that
are always available.
Solution
To use BPEL partner abstraction that are services that workflow
needs to use and typically are mapped to Web (or Grid) services. It
is possible to map some BPEL partners to locally implemented
services as long as a WSDL port type for a partner is provided.
WSIF (Web Service Invocation Framework)
is a simple Java API for invoking Web services, enables to interact
with abstract representations of Web services through their WSDL
descriptions
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006
11
Current Scenario with WSIF
Enabling Grids for E-sciencE
LCG
WSIF
WF
WEB
SERVICES
W
S
I
F
GLOBUS 2x
BPEL ENGINE
WEB
SERVICES
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006
12
Conclusions
Enabling Grids for E-sciencE
• Astronomical applications described using workflow
language
• A solution to integrate astronomical task sequences in
gLite middleware
• Use of Service Oriented technology
• Use of Grid Service composition
Integrated solution in the current middleware and
in its future development
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006
13
The End
Enabling Grids for E-sciencE
Thank you for your attention
The 17th Global Grid Forum - GGF17 Tokyo, May 10-12, 2006
14
Descargar

GridIce Notification Service