Distributed Systems:
Coordination
models
October 2005
and
languages
Distributed systems: coordination
models and languages
1
Overview of chapters
• Introduction
• Coordination models and languages
– Ch 4: Interprocess communication
– Ch 5: Distributed objects and remote invocation
– Ch 20: Corba Case study
• General services
• Distributed algorithms
• Shared data
• Building distributed services
October 2005
Distributed systems: coordination
models and languages
2
Introduction
• Communication
– data representation
• Synchronization
– how express cooperation?
Data transfer
Synchronization
October 2005
Distributed systems: coordination
models and languages
3
This chapter: overview
• Data representation
• Message passing
• Remote procedure calls
• Object request brokers
• Generative communication
• Event Systems
October 2005
Distributed systems: coordination
models and languages
4
This chapter: overview
Applications, services
RMI and RPC
request-reply protocol
Middleware
layers
marshalling and external data representation
UDP and TCP
October 2005
Distributed systems: coordination
models and languages
5
Data representation
• Problem
int
int
System A
System B
int = 2-complement, 32 bits
int = 1-complement, 40 bits
on the wire: int = ??
October 2005
Distributed systems: coordination
models and languages
6
Data representation (cont.)
• Problem:
– program data: typed, structured, object
– message data: bit/byte stream
– different representations of data in
heterogeneous systems
• mapping to data items in messages:
– flattened before transmission
– rebuilt on arrival
October 2005
Distributed systems: coordination
models and languages
7
Data representation (cont.)
• Conversion: different approaches
– agreed form for transmission
• e.g. int = 2-complement, 32 bits
• both partners have full knowledge of transmitted data
• e.g.: Sun XDR, Corba CDR
– full data description transmitted
• type, length, value coding on the wire
• interpretation at receiving site possible
• e.g. ASN + BER, Java serialized form
– Conversion to ASCII text
• XML
October 2005
Distributed systems: coordination
models and languages
8
Data representation (cont.)
• Sun XDR
– Fixed representation of primitive & constructed types
– Message: ‘Smith’, ‘London’, 1934
5
“Smit”
“h___”
6
“Lond”
“on__”
1934
Length of string
‘Smith’
Length of string
‘London’
Cardinal
4 bytes
October 2005
Distributed systems: coordination
models and languages
9
Data representation (cont.)
• CORBA CDR
– Fixed representation of primitive & constructed types
• int: little- or big-endian: representation of sender
• Value of size n is placed at relative position that is a multiple
of n
Type
sequence
string
array
struct
enumerated
union
October 2005
Representation
length (unsigned long) followed by elements in order
length (unsigned long) followed by characters in order (can also
can have wide characters)
array elements in order (no length specified because it is fixed)
in the order of declaration of the components
unsigned long (the values are specified by the order declared)
type tag followed by the selected member
Distributed systems: coordination
models and languages
10
Data representation (cont.)
• CORBA CDR
index in
sequence of bytes
0–3
4–7
8–11
12–15
16–19
20-23
24–27
Struct Person{
string name;
string place;
unsigned long year;
};
4 bytes
5
"Smit"
"h___"
6
"Lond"
"on__"
1934
notes
on representation
length of string
‘Smith’
length of string
‘London’
unsigned long
The flattened form represents a Person struct with value: {‘Smith’, ‘London’, 1934}
October 2005
Distributed systems: coordination
models and languages
11
Data representation (cont.)
• ASN.1: abstract syntax notation
– application syntax
• type identifiers ( 4 classes)
• standard types
• constructors: sequence, sequenceof, set, setof
– encoding: TLV
• Type: defines ASN.1 type
• Length
• Value
October 2005
Distributed systems: coordination
models and languages
12
Data representation (cont.)
• Java: supports transfer of objects and primitive data
values
Public class Person implements Serializable{
private String name;
private String place;
private int year;
public Person(String aName, String aPlace, int aYear){
…
}
…
}
October 2005
Distributed systems: coordination
models and languages
13
Data representation (cont.)
• Java serialized form
– Handles: references to objects
– Primitive types: portable binary format
Person p = new Person(“Smith”, “London”, 1934);
Explanation
Serialized values
Person
8-byte version number
h0
class name, version number
3
int year
java.lang.String java.lang.String number, type and name of
name:
place:
instance variables
1934
5 Smith
6 London
h1
values of instance variables
The true serialized form contains additional type markers; h0 and h1 are handles
October 2005
Distributed systems: coordination
models and languages
14
Data representation (cont.)
• XML
– Markup language defined by World Wide Web
consortium
– Data items tagged with ‘markup’ strings
– Users can define their own tags
<person id=‘123456789”>
<name>Smith</name>
<place>London</place>
<year>1934</year>
<…>
</person>
October 2005
Distributed systems: coordination
models and languages
15
Data representation (cont.)
• Definitions:
– marshalling = assembling a collection of data
items into a form suitable for transmission
– unmarshalling = disassembling a message on
arrival to produce the equivalent collection of
data items
• operations can be generated from
specification
October 2005
Distributed systems: coordination
models and languages
16
This chapter: overview
• Data representation
• Message passing
• Remote procedure calls
• Object request brokers
• Generative communication
• Event Systems
October 2005
Distributed systems: coordination
models and languages
17
Message passing
• Basic functionality
– Procedure Send
(p: PortId; m: Message);
– Procedure Receive
(p: PortId; VAR m: Message);
October 2005
Distributed systems: coordination
models and languages
18
Message passing (cont.)
• Semantics:
synchronous <> asynchronous communication
– synchronous = blocking
• send: wait for corresponding receive
• receive: wait for message arrival
– asynchronous = no waiting for completion
• send: no wait for message arrival
• receive: announce willingness to accept
or
check for message arrival
October 2005
Distributed systems: coordination
models and languages
19
Message passing (cont.)
• Semantics: synchronous <> asynchronous
ty p e
B lo ck in g
S en d
B lo ck in g
R eceiv e
L an g u ag e
S y stem
Syn
Y es
Y es
o ccam
Syn
Y es
No
-
A sy n
No
Y es
M ach
C h o ru s
A sy n
No
No
C h arlo tte
October 2005
Distributed systems: coordination
models and languages
20
Message passing (cont.)
• Semantics: synchronous <> asynchronous
Example: Occam style
Sender:
…..
Send(p, m);
{message is accepted!!!}
…..
Receiver:
….
Receive(p, b);
{sender after send }
….
Communication  Synchronisation point
b := m
October 2005
Distributed systems: coordination
models and languages
21
Message passing (cont.)
• Semantics: synchronous <> asynchronous
Send (p, m)
Receive(p, b)
m 
b
October 2005
Distributed systems: coordination
models and languages
?

22
Message passing (cont.)
• Semantics: synchronous <> asynchronous
Example: Mach style
Sender:
…..
send(….);
{message is in buffer!!!
arrival??}
Receiver:
….
receive(….);
{message available }
….
Communication  NO synchronisation point
October 2005
Distributed systems: coordination
models and languages
23
Message passing (cont.)
• Semantics: synchronous <> asynchronous
Send (p, m)
Port p
October 2005
Distributed systems: coordination
models and languages
24
Message passing (cont.)
• Semantics: synchronous <> asynchronous
Send(p, m)
receive(p,b)
Port p
October 2005
Distributed systems: coordination
models and languages
25
Message passing (cont.)
• Semantics: message destinations
– message destination = communication identifier
• preference for location independent identifiers
– types of message destination:
• process
• port
• mailbox
October 2005
Distributed systems: coordination
models and languages
26
Message passing (cont.)
• Semantics: message destinations
process
Send (p, m)
Process p
October 2005
Distributed systems: coordination
models and languages
27
Message passing (cont.)
• Semantics: message destinations
port
port q
Send (p, m)
port p
October 2005
Distributed systems: coordination
models and languages
28
Message passing (cont.)
• Semantics: message destinations
mailbox
mailbox q
Send (p, m)
mailbox p
October 2005
Distributed systems: coordination
models and languages
29
Message passing (cont.)
• Semantics: message destinations
– types of message destination:
• process:
– single entry point per process for all messages
• port
– one receiver, many senders
– may have a message queue
– many ports per process
• mailbox
– may have many receivers
– message queue
October 2005
Distributed systems: coordination
models and languages
30
Message passing (cont.)
• Semantics: reliability
– possible failures
• Corrupted messages
Communication failure
• Duplicate messages
• Omission: loss of messages
• Messages out of order
• Receiver process failure
– Reliable communication
• Delivered uncorrupted, in order, without duplicates
• Despite a reasonable number of packets dropped or lost
• Perfectly reliable communication can not often be guaranteed
October 2005
Distributed systems: coordination
models and languages
31
Message passing (cont.)
• How to implement reliable communication:
– Avoiding corruption
• Include checksum in message
– Avoids order mistakes and duplicates
• Include a message number which identifies the message
– Avoiding omission
• Sender stores message in buffer, sends it and sets a time-out
• Receiver replies with acknowledgement
• Sender retransmits messages after timeout
October 2005
Distributed systems: coordination
models and languages
32
Message passing (cont.)
• Semantics: reliability
– sender of message gets no reply?
no distinction between
• process failure
• communication failure
October 2005
Distributed systems: coordination
models and languages
33
Message passing (cont.)
• Case study: UDP/TCP
– Sockets <> ports
• Socket bound to (TCP) port + Internet address
any port
socket
agreed port
socket
message
client
server
other ports
Internet address = 138.37.94.248
October 2005
Internet address = 138.37.88.249
Distributed systems: coordination
models and languages
34
Message passing (cont.)
• Case study UDP
– Messages:
• Restricted packet size: < 216 , < 213 (8Kbytes), truncation
• No conversion: bit transmission
– Blocking:
• Non-blocking send
• Blocking receive
– Timeouts: user can set timeout on receive operation
– Receive from any: receive returns port + Internet address
• But sockets can be bound to remote (IP address+port)
– Unreliable message service
• lost, out of order, duplicates
• no message corruption: checksum
October 2005
Distributed systems: coordination
models and languages
35
Message passing (cont.)
• Case study UDP: Java API
– Class InetAddress
• getByName(string) (underlying name server!)
– Class DatagramPacket
• (bufferlength, buffer, InetAddress, PortID)
– Class DatagramSocket
• Send(DatagramPacket)
• Receive(DatagramPacket)
• Close()
October 2005
Distributed systems: coordination
models and languages
36
Message passing (cont.)
• Case study UDP: example in Java
– UDP client sends a message to the server and
gets a reply (Figure 4.3)
– UDP server repeatedly receives a request and
sends it back to the client (Figure 4.4)
October 2005
Distributed systems: coordination
models and languages
37
4.3 UDP client sends a message to the server and gets a reply
import java.net.*;
import java.io.*;
public class UDPClient{
public static void main(String args[]){
// args give message contents and server hostname
DatagramSocket aSocket = null;
try {
aSocket = new DatagramSocket();
byte [] m = args[0].getBytes();
InetAddress aHost = InetAddress.getByName(args[1]);
int serverPort = 6789;
DatagramPacket request = new DatagramPacket(m, args[0].length(), aHost, serverPort
aSocket.send(request);
byte[] buffer = new byte[1000];
DatagramPacket reply = new DatagramPacket(buffer, buffer.length);
aSocket.receive(reply);
System.out.println("Reply: " + new String(reply.getData()));
}catch (SocketException e){System.out.println("Socket: " + e.getMessage());
}catch (IOException e){System.out.println("IO: " + e.getMessage());}
}finally {if(aSocket != null) aSocket.close();}
}
}
October 2005
Distributed systems: coordination
models and languages
38
4.4 UDP server repeatedly receives a request and sends it back to the
client
import java.net.*;
import java.io.*;
public class UDPServer{
public static void main(String args[]){
DatagramSocket aSocket = null;
try{
aSocket = new DatagramSocket(6789);
byte[] buffer = new byte[1000];
while(true){
DatagramPacket request = new DatagramPacket(buffer, buffer.length);
aSocket.receive(request);
DatagramPacket reply = new DatagramPacket(request.getData(),
request.getLength(), request.getAddress(), request.getPort());
aSocket.send(reply);
}
}catch (SocketException e){System.out.println("Socket: " + e.getMessage());
}catch (IOException e) {System.out.println("IO: " + e.getMessage());}
}finally {if(aSocket != null) aSocket.close();}
}
}
October 2005
Distributed systems: coordination
models and languages
39
Message passing (cont.)
• Case study TCP
– Stream communication: (<> message passing?)
• Connect: create a communication channel through
communicating sockets
• Communication: read and write through channel
• Close
– Implementation:
• TCP handles all communication
• Uses buffers at sender and receiver side
• No conversion: bit transmission
– Synchronization semantics:
• Non-blocking send, except for flow control
(when buffers of sender or receiver are full)
• Blocking receive
October 2005
Distributed systems: coordination
models and languages
40
Message passing (cont.)
• Case study TCP
– Setting up a client server connection:
• Client sends request for communication to Server port
• Server accepts client request
• Typically, server creates new thread which handles
communication with client
– Reliable message service
• Except broken connections
– Overhead compared to UDP:
• Buffering
• Creating a connection: 2(?) extra messages
– Sending a message, returning an acknowledgement
– May create unacceptable overhead if goal is to send a single
message.
October 2005
Distributed systems: coordination
models and languages
41
Message passing (cont.)
• Case study TCP: Java API
– Class ServerSocket
• Socket Accept()
– Class Socket
• getInputStream()
• getOutputStream()
– Class DataInputStream
• readUTF, …
(Universal Transfer Format)
– Class DataOutputStream
• writeUTF, …
October 2005
Distributed systems: coordination
models and languages
42
Message passing (cont.)
• Case study TCP: example
– TCP client makes connection to server, sends
request and receives reply
(Figure 4.5)
– TCP server makes a connection for each client
and then echoes the client’s request
(Figure 4.6)
October 2005
Distributed systems: coordination
models and languages
43
4.5 TCP client makes connection, sends request and receives reply
import java.net.*;
import java.io.*;
public class TCPClient {
public static void main (String args[]) {
// arguments supply message and hostname of destination
Socket s = null;
try{
int serverPort = 7896;
s = new Socket(args[1], serverPort);
DataInputStream in = new DataInputStream( s.getInputStream());
DataOutputStream out =
new DataOutputStream( s.getOutputStream());
out.writeUTF(args[0]);
// UTF is a string encoding see Sn 4.3
String data = in.readUTF();
System.out.println("Received: "+ data) ;
}catch (UnknownHostException e){
System.out.println("Sock:"+e.getMessage());
}catch (EOFException e){System.out.println("EOF:"+e.getMessage());
}catch (IOException e){System.out.println("IO:"+e.getMessage());}
}finally {if(s!=null) try {s.close();}catch (IOException e){System.out.println("close:"+e.getMessage());}}
}
}
October 2005
Distributed systems: coordination
models and languages
44
4.6 TCP server makes a connection for each client and then echoes the
request of the client
import java.net.*;
import java.io.*;
public class TCPServer {
public static void main (String args[]) {
try{
int serverPort = 7896;
ServerSocket listenSocket = new ServerSocket(serverPort);
while(true) {
Socket clientSocket = listenSocket.accept();
Connection c = new Connection(clientSocket);
}
} catch(IOException e) {System.out.println("Listen :"+e.getMessage());}
}
}
// this figure continues on the next slide
October 2005
Distributed systems: coordination
models and languages
45
4.6 TCP server …
class Connection extends Thread {
DataInputStream in;
DataOutputStream out;
Socket clientSocket;
public Connection (Socket aClientSocket) {
try {
clientSocket = aClientSocket;
in = new DataInputStream( clientSocket.getInputStream());
out =new DataOutputStream( clientSocket.getOutputStream());
this.start();
} catch(IOException e) {System.out.println("Connection:"+e.getMessage());}
}
public void run(){
try {
// an echo server
String data = in.readUTF();
out.writeUTF(data);
} catch(EOFException e) {System.out.println("EOF:"+e.getMessage());
} catch(IOException e) {System.out.println("IO:"+e.getMessage());}
} finally{ try {clientSocket.close();}catch (IOException e){/*close failed*/}}
}
}
October 2005
Distributed systems: coordination
models and languages
46
Message passing (cont.)
• Conclusion: UDP, TCP
– general purpose communication protocols
– primitive, low level operations:
• Setting up a communication
• No transfer of structured data
– Difficult to use
– efficient implementation
– building blocks used for more complex
interactions
October 2005
Distributed systems: coordination
models and languages
47
Message passing (cont.)
• Conclusion:
–
–
–
–
message passing
primitive, low level operations
difficult, hard to use
efficient implementation
building blocks used for more complex interactions
• From message passing
– to Client-server
– to RPC, RMI
October 2005
Distributed systems: coordination
models and languages
48
Message passing (cont.)
• Client-server communication
Client
doOperation
Server
Request
message
(wait)
Reply
message
getRequest
select object
execute
method
sendReply
(continuation)
October 2005
Distributed systems: coordination
models and languages
49
Message passing (cont.)
• Client-server messages/operations
– Designed to support roles and message
exchanges in typical client-server interactions
• Acks redundant (replies are used)
• Connections not necessary
• No flow control
– Basic operations:
• Client: doOperation: sends request and returns answer to
application program
• Server: getRequest, sendReply
October 2005
Distributed systems: coordination
models and languages
50
Message passing (cont.)
– Reliability measures of TCP are an overkill!!
• Acknowledgement of receiver is redundant :
the reply message is an acknowledgement!
• Limited size of data packet transfer
• One time communication
=> Making a connection is overhead
=> No stream needed, no flow control
Use of TCP is (often) an overkill and may cause
efficiency problems!
UDP can be used for building more efficient client
server communication.
• What about reliability??
October 2005
Distributed systems: coordination
models and languages
51
Message passing (cont.)
• Case: Client-server communication: HTTP
– Interactions:
• Open connection
• Client sends request
• Server sends reply
• Connection closed
 New in HTTP1.1: Persistent connections using TCP
(for multiple requests at same server)
– HTTP methods:
• Get(URL) : request for the resource referred to by URL
• Post(URL,data): replace or create resource at URL
• ..
October 2005
Distributed systems: coordination
models and languages
52
Message passing (cont.)
• Case: Client-server communication: HTTP
• Request message
method
URL or pathname
GET
//www.dcs.qmw.ac.uk/index.html
HTTP version
headers message body
HTTP/ 1.1
• Reply message
HTTP version
HTTP/1.1
October 2005
status code reason headers message body
200
OK
Distributed systems: coordination
models and languages
resource data
53
Message Passing: Conclusion
• So far rather low level implementations:
–
–
–
–
–
Setting up a connection
No transfer of structured data!!!
Concerns for synchronisation and failure model
No encryption of data
…
• Higher level Message Passing systems exist!!
– MPI, Mach, ..
October 2005
Distributed systems: coordination
models and languages
54
This chapter: overview
• Data representation
• Message passing
• Remote procedure calls
• Object request brokers
• Generative communication
• Event Systems
October 2005
Distributed systems: coordination
models and languages
55
Remote procedure calls
Overview
• Basic principles
• Design issues
• Implementation aspects
• Case studies
• Asynchronous RPC
October 2005
Distributed systems: coordination
models and languages
56
Remote procedure calls
Basic principles
• View on traditional application:
– no OO yet!
– application =
main program
+ procedures (functions)
 familiar paradigm: procedure call
October 2005
Distributed systems: coordination
models and languages
57
Remote procedure calls
Basic principles
• View on traditional application:
Main program
…
….
Procedure A
….
…
….
Procedure B
...
end
October 2005
Distributed systems: coordination
models and languages
58
Remote procedure calls
Basic principles
• View on traditional application:
– main program and procedures (functions)
– familiar paradigm: procedure call
• approach in distributed systems:
– group procedures into servers
– main programs become clients
– operations on server look like conventional
procedure calls
October 2005
Distributed systems: coordination
models and languages
59
Remote procedure calls
Basic principles
• Basic view on client-server model
client
server
Main program
…
Procedure A
….
….
…
….
RPC middleware
Procedure B
...
end
October 2005
Distributed systems: coordination
models and languages
60
Remote procedure calls
Basic principles
• RPC technology: client side
Main program
…
….
Client stub
procedure A
….
…
….
Client stub
procedure B
...
end
October 2005
Distributed systems: coordination
models and languages
61
Remote procedure calls
Basic principles
• RPC technology: server side
Server stub
…
….
Procedure A
….
…
….
Procedure B
...
end
October 2005
Distributed systems: coordination
models and languages
62
Remote procedure calls
Basic principles
• RPC technology
Main program
Server stub
…
…
….
….
…
….
...
end
October 2005
Client stub
procedure A
….
Procedure A
….
Message subsystem …
Client stub
….
procedure B
...
Procedure B
end
Distributed systems: coordination
models and languages
63
Remote procedure calls
Basic principles
• RPC technology
client process
server process
Request
Reply
client stub
procedure
client
program
October 2005
Communication
module
server stub
procedure
Communication
dispatcher
module
Distributed systems: coordination
models and languages
service
procedure
64
Remote procedure calls
• Application program calls client stub procedure
• Client stub procedure marshalls parameters of call and
gives it to communication module in client
• Communication module in client transmits a message with
the marshalled RPC
• Communication module in server receives message and
gives it to dispatcher
• Dispatcher determines which procedure is called and calls
correct servers stub procedure with marshalled data
• Server Stub procedure unmarshalls data and calls the
server procedure
October 2005
Distributed systems: coordination
models and languages
65
Remote procedure calls
• Server procedure returns an answer to Server Stub
procedure
• Server Stub procedure marshalls the answer and gives it to
the communication module at server side
• Communication module at server side transmits the reply
in a message
• Communication module at client side gives data to client
stub procedure
• Who unmarshalls data and returns the answer to calling
program
October 2005
Distributed systems: coordination
models and languages
66
Remote procedure calls
• Primary characteristics:
– code in client and server independent of
communication system
– familiar paradigm: procedure call
• no message preparation
• synchronous interaction
• semantics?
– IDL: Interface definition language
• independent of language used for client or server
• base for generation of stubs
October 2005
Distributed systems: coordination
models and languages
67
Remote procedure calls
Design issues
• Classes of RPC systems
• Interface Definition language
• Exception handling
• Semantics of RPC
• Transparency
October 2005
Distributed systems: coordination
models and languages
68
Remote procedure calls
Design issues
• Classes of RPC systems
– RPC integrated within a particular
programming language
• e.g. Argus (with CLU), Arjuna (C++)
– RPC based on a special IDL
• e.g. Sun RPC, ANSA RPC, OSF/DCE
October 2005
Distributed systems: coordination
models and languages
69
Remote procedure calls
Design issues
• Interface Definition language
– describes operation signatures
– interface compilers for
• generating client and server stubs
• in different languages (e.g. C, Pascal,…)
October 2005
Distributed systems: coordination
models and languages
70
Remote procedure calls
Design issues
• Exception handling
– failures cannot be hidden!
• Network <> server failure?
• Client cannot distinguish
– approaches to support failures:
• Language specific
• using return codes of functions
• extension provided by IDL
October 2005
Distributed systems: coordination
models and languages
71
Remote procedure calls
Design issues
• Semantics of RPC
– Maybe
– At-least-once (e.g. Sun RPC)
– At-most-once (e.g. ANSA RPC)
– Exactly-once:
• difficult or impossible given failures
October 2005
Distributed systems: coordination
models and languages
72
Remote procedure calls
D elivery gu aran tees
R P C sem an tics
retry request duplicate
filtering
re-exec
retrans reply
no
not applic.
not.
M aybe
yes
no
re-exec
A t-least-once
yes
yes
retrans reply A t-m ost-on ce
October 2005
Distributed systems: coordination
models and languages
73
Remote procedure calls
• Transparency
Design issues
– RPC more vulnerable to failure
• possibility of failure should not be hidden
– calling instructions different
– no shared memory between caller and callee
programming convenience
vs.
true transparency
October 2005
Distributed systems: coordination
models and languages
74
Remote procedure calls
Implementation aspects
• Tasks for interface compiler: generate
– client stub procedure
– server stub procedure
– marshalling and unmarshalling operations for
each argument type
– header for server procedure
October 2005
Distributed systems: coordination
models and languages
75
Remote procedure calls
Implementation aspects
• Binding:
linking client to server at execution time
• Binder interface
Procedure Register (serviceName: String;
serverPort: Port; version: integer);
Procedure Withdraw (…);
Procedure Lookup(serviceName: String;
version:…): Port;
October 2005
Distributed systems: coordination
models and languages
76
Remote procedure calls
Implementation aspects
• Binding:
– server will register service at binder
– client will lookup the service
• Locating the binder?
– well known host address
– responsibility of OS
– broadcast message by client
October 2005
Distributed systems: coordination
models and languages
77
Remote procedure calls
Case studies: Sun RPC
• Sun RPC
– designed by Sun for client-server
communication in the Sun NFS network file
system
– interface definition language: XDR
– at-least-once semantics
– local binding service per system: port mapper
October 2005
Distributed systems: coordination
models and languages
78
Remote procedure calls
Sun RPC: Interface in XDR
Const MAX = 1000;
struct writeargs{
typedef int FileIdentifier;
FileIdentifier f;
typedef int FilePointer;
FilePointer position;
typedef int Length;
Data data;
struct DATA {
};
int length;
struct readargs{
char buffer[Max];
FileIdentifier f;
};
FilePointer position;
Length length;
};
October 2005
Distributed systems: coordination
models and languages
79
Remote procedure calls
Sun RPC: Interface in XDR
Program FILEREADWRITE {
version VERSION {
void WRITE ( writeargs) = 1;
Data READ (readargs) = 2;
} = 2;
} = 9999;
October 2005
Distributed systems: coordination
models and languages
80
Remote procedure calls
Sun RPC: IDL
• Interface definition language
– extension of XDR, originally for external data
representation
– specifying an interface:
• program and version number
• procedure definitions
– procedures are numbered
– input parameters = single (C-)structure
– output parameters = a (C-)structure
• type definitions
October 2005
Distributed systems: coordination
models and languages
81
Remote procedure calls
Sun RPC: IDL
• Interface compiler generates:
– client stub procedures
– server main procedure, despatcher and stub
procedures
– XDR marshalling and unmarshalling
– header file for client and server
October 2005
Distributed systems: coordination
models and languages
82
Remote procedure calls
Sun RPC: Client in C
main(…) {
CLIENT *clientHandle;
char *serverName = “coffee”;
readargs a;
Data *data;
clientHandle = clnt_create( serverName,
FILEREADWRITE, VERSION, “udp”);
if(clientHandle == NULL) {
};
a.f = 10; a.position = 100; a.length = 1000;
data = read_2( &a, clientHandle);
…
clnt_destroy ( clientHandle);
}
October 2005
Distributed systems: coordination
models and languages
83
Remote procedure calls
Sun RPC: Client program
– Binding:
• via local port mapper, so hostname must be
specified
• use clnt_create(…)
– calling stub procedure
• stub procedure name: IDLname + underscore +
version number
• arguments: input parameters + client handle
– stub procedure calls clnt_call, at-least-once
semantics
October 2005
Distributed systems: coordination
models and languages
84
Remote procedure calls
Sun RPC: Server in C
Void *write_2( writeargs *a) {
/* do writing to file */
}
Generated headers
Data *read_2(readargs *a){
static Data result;
result.buffer = …
/* do the reading */
result.length = …
return &result;
}
October 2005
Distributed systems: coordination
models and languages
85
Remote procedure calls
Sun RPC: server program
• Main
– create socket
– export service by informing port mapper
• dispatcher
– check program and version number, unmarshal
– call server procedure
– marshal results and transmit reply message
October 2005
Distributed systems: coordination
models and languages
86
Remote procedure calls
Sun RPC
• Lower level facilities:
–
–
–
–
–
testing tools, such as null RPC
memory management for marshalling
broadcast RPC
batching client calls that require no reply
Authentication
• None
• Unix style
• Kerberos
October 2005
Distributed systems: coordination
models and languages
87
Remote procedure calls
Asynchronous RPC
• Problem: throughput RPC?
Process args client
marshal
send
server
Receive
unmarshal
execute request
marshal
send
Receive
unmarshal
process results
….
October 2005
Idle time!
time
Distributed systems: coordination
models and languages
88
Remote procedure calls
Asynchronous RPC
• Timing:
Synchronous RPC
October 2005
Asynchronous RPC
Distributed systems: coordination
models and languages
89
Remote procedure calls
Asynchronous RPC
• When to use?
– Many request, small amount of information,
limited processing: e.g. windows system
– parallel requests to several servers
• Additional optimisations:
– buffer request at client until …
– proceed without waiting when no reply is
required
October 2005
Distributed systems: coordination
models and languages
90
Remote procedure calls
Asynchronous RPC
• Extensions:
– call streams:
•
•
•
•
October 2005
mix of synchronous and asynchronous calls
message ordering preserved
connection oriented
connection breaks when semantics cannot be
guaranteed
Distributed systems: coordination
models and languages
91
Remote procedure calls
Asynchronous RPC
• Extensions:
– Promises:
• allow clients to continue with (other work) and
retrieve result of call later
• created at the time of call
• store results of call (object with same type)
• operations:
– get, await result
– result available?
• Alternative names: futures, tickets, continuations
October 2005
Distributed systems: coordination
models and languages
92
Remote procedure calls
• Conclusion:
– familiar paradigm
– has been basic primitive for distributed
programming
– limitations
• failures to be handled by clients
• no transaction support
• one-to-one communication
October 2005
Distributed systems: coordination
models and languages
93
This chapter: overview
• Data representation
• Message passing
• Remote procedure calls
• Object request brokers
• Generative communication
• Event Systems
October 2005
Distributed systems: coordination
models and languages
94
Object request brokers
• Basics:
Object request broker =
Objects +
RPC
+ ...
October 2005
Distributed systems: coordination
models and languages
95
Object request brokers
• Overview
– distributed object systems
– case study: Java RMI
– case study: OMG Corba
October 2005
Distributed systems: coordination
models and languages
96
ORB: Distributed object systems
• Why objects?
–…
• Why distributed objects?
– Maps naturally to distributed services
– Services are modeled as objects
– Functionality emerges by cooperation between
services and clients
October 2005
Distributed systems: coordination
models and languages
97
ORB: Distributed object model
• Local <> remote invocation
local
remote
invocation
A
October 2005
B
C
local E
invocation
invocation
local
invocation
D
Distributed systems: coordination
models and languages
remote
invocation
F
98
ORB: Distributed object model
• requirements
– synchronous invocation semantics
• location transparency: location transparent object
references
• access transparency: local & remote object
invocations identical
– inheritance
– polymorphism
October 2005
Distributed systems: coordination
models and languages
99
ORB: Distributed objects
• a distributed object consists of
– an implementation independent service
definition specified by an interface
• method signature
• multiple inheritance
– a concrete service implementation
• implements one or more interfaces
• provides the concrete implementation
October 2005
Distributed systems: coordination
models and languages
100
ORB: Distributed object model
• Remote object & its remote interface
remoteobject
remote
interface
{
October 2005
Data
m1
m2
m3
implementation
of methods
Distributed systems: coordination
models and languages
m4
m5
m6
101
ORB: Distributed object model
• Distributed objects act like normal objects
– can be used as parameters and return values
– can be invoked (across address space
boundaries)
– can be subclassed (inheritance)
– are invoked synchronously
October 2005
Distributed systems: coordination
models and languages
102
ORB: Distributed object model
• But there are exceptions
– Invocation semantics
• Java RMI: at most once
• Corba: at most once or maybe (if no result)
– Parameter passing
• distributed objects are passed by reference
• normal objects are passed by value
– additional exceptions due to distribution
• distribution is not transparent
October 2005
Distributed systems: coordination
models and languages
103
ORB: Object invocations
server
client
object A proxy for B
Request
skeleton
& dispatcher
for B’s class
remote
object B
Reply
Communication
Remote
reference module
module
October 2005
servant
Communication Remote reference
module
module
Distributed systems: coordination
models and languages
104
ORB: The object bus
foo()
Client
Server
Object Bus
Client - the invoking application object
Server - the invoked application object
October 2005
Distributed systems: coordination
models and languages
105
ORB: The object bus
foo()
Application
Client
Server
Stub/Proxy
Skeleton
Object Bus
Broker
October 2005
Distributed systems: coordination
models and languages
106
ORB: The object bus
•
•
•
•
Client - the invoking application object
Server - the invoked application object
Stub - reifies the invocation
Skeleton - dispatches the invocation to the
actual object implementation
• Broker - invocation distributor
October 2005
Distributed systems: coordination
models and languages
107
ORB: The object bus
Conceptual invocation
foo()
Client
Server
Stub
Skeleton
Broker
October 2005
Distributed systems: coordination
models and languages
108
ORB: The object bus
foo()
Client
Server
Stub
Skeleton
Method call
Broker
October 2005
Distributed systems: coordination
models and languages
109
ORB: The object bus
foo()
Reified
invocation
October 2005
Client
Server
Stub
Skeleton
Broker
Distributed systems: coordination
models and languages
110
ORB: The object bus
foo()
Reified
invocation
October 2005
Client
Server
Stub
Skeleton
Broker
Distributed systems: coordination
models and languages
111
ORB: The object bus
foo()
Client
Server
Stub
Skeleton
Method call
Broker
October 2005
Distributed systems: coordination
models and languages
112
ORB: The object bus
foo()
Client
Server
Return value
Stub
Skeleton
Broker
October 2005
Distributed systems: coordination
models and languages
113
ORB: The object bus
foo()
Client
Server
Stub
Skeleton
Reified return
value
Broker
October 2005
Distributed systems: coordination
models and languages
114
ORB: The object bus
foo()
Client
Server
Stub
Skeleton
Reified return
value
Broker
October 2005
Distributed systems: coordination
models and languages
115
ORB: The object bus
foo()
Client
Server
Return value
Stub
Skeleton
Broker
October 2005
Distributed systems: coordination
models and languages
116
ORB: advanced topics
• Active objects;
– Kept in running process
– Ready to execute methods
– Waste of resources if no requests
• Passive objects
– Not currently active; can be become active
– 2 parts:
• Implementation of methods
• State in marshalled form
October 2005
Distributed systems: coordination
models and languages
117
ORB: advanced topics
• Persistent object
– Guaranteed to live between activations of
processes
• as passive object
• in persistent store
– Strategy for: passive  active
• Object migration
– Find location using object reference
October 2005
Distributed systems: coordination
models and languages
118
Object request brokers
• Overview
– distributed object systems
– case study: Java RMI
– case study: OMG Corba
October 2005
Distributed systems: coordination
models and languages
119
Java RMI
• JAVA
– Driven by SUN Micro systems
– Platform independent
• JAVA + RMI  distribution:
– Remote Method Invocation
Distributed Java applications
October 2005
Distributed systems: coordination
models and languages
120
Java RMI: Architecture
implements
java.rmi.remote
extends
java.rmi.UnicastRemoteObject
foo()
Client
Server
generated
by rmic
Stub
Skeleton
Java Runtime Environment
October 2005
Distributed systems: coordination
models and languages
121
Java RMI: Services
• rmiregistry
– simple naming service at the server-side
– allows remote clients to get a reference to a
server object
• java.rmi.Naming
– Naming class
– provides methods for storing and obtaining
references to remote objects in the remote
object registry
October 2005
Distributed systems: coordination
models and languages
122
Java RMI: Naming Service
void rebind (String name, Remote obj)
This method is used by a server to register the identifier of a remote
object by name, as shown in Figure 5.13, line 3.
void bind (String name, Remote obj)
This method can alternatively be used by a server to register a remote
object by name, but if the name is already bound to a remote
object reference an exception is thrown.
void unbind (String name, Remote obj)
This method removes a binding.
Remote lookup(String name)
This method is used by clients to look up a remote object by name,
as shown in Figure 5.15 line 1. A remote object reference is
returned.
String [] list()
This method returns an array of Strings containing the names bound
in the registry.
October 2005
Distributed systems: coordination
models and languages
123
Java RMI
• Tools:
– rmic - Java RMI Stub Compiler
– rmiregistry - Java Remote Object Registry
– rmid - Java RMI Activation System Daemon
October 2005
Distributed systems: coordination
models and languages
124
Java RMI: a distributed application
• Step by step
– define the server interface
• a remote interface extends the interface
java.rmi.Remote
• each method of the interface declares
java.rmi.RemoteException in its throws clause, in
addition to any application-specific exceptions
– create the object implementation
• implement one or more server interfaces
October 2005
Distributed systems: coordination
models and languages
125
Java RMI: a distributed application
• Step by step (cont.)
– compile the Server source
• run javac on the java source file
– generate stubs and skeletons
• run the rmic compiler on the fully qualified class
name of the server object implementation
– create and compile the Client source
• this can be done at any time after the server
interface is defined
October 2005
Distributed systems: coordination
models and languages
126
Java RMI: Remote interface
package HelloAppRMI;
interface Hello extends java.rmi.Remote {
String sayHello() throws java.rmi.RemoteException;
}
October 2005
Distributed systems: coordination
models and languages
127
Java RMI: Server
package HelloAppRMI;
public class HelloServer {
public static void main(String args[]) {
try {
HelloServant obj = new HelloServant();
java.rmi.Naming.rebind("//ryslinge/HelloServer",obj;
System.out.println("HelloServer bound in registry");
} catch (Exception e) {
System.err.println("Hello err: " + e.getMessage());
}
}
}
October 2005
Distributed systems: coordination
models and languages
128
Java RMI: Server (cont.)
class HelloServant extends
java.rmi.server.UnicastRemoteObject implements
Hello
{
public HelloServant() throws
java.rmi.RemoteException {
super();
}
public String sayHello() {
return "\nHello world!!\n";
}
}
October 2005
Distributed systems: coordination
models and languages
129
RMI: Client
package HelloAppRMI;
public class HelloClient {
public static void main(String args[]) {
try {
Hello obj = (Hello)java.rmi.Naming.lookup
("//" + "ryslinge" + "/HelloServer");
String hello = obj.sayHello();
System.out.println(hello);
} catch (Exception e) {
System.err.println("Hello exception: "
+ e.getMessage());
}
}
}
October 2005
Distributed systems: coordination
models and languages
130
Object request brokers
• Overview
– distributed object systems
– case study: Java RMI
– case study: OMG Corba
October 2005
Distributed systems: coordination
models and languages
131
OMG CORBA
• OMG
– Object Management Group
– non-profit organization
– sponsored by over 500 organizations
• computer manufacturers, software companies,
telecommunications companies, end users
– overall objective:
promote the theory and practice of object
technology for the development of distributed
computing systems
October 2005
Distributed systems: coordination
models and languages
132
OMG CORBA
• OMG (cont.)
– approach to standardization
•
•
•
•
define requirements
issue Requests for Proposals
evaluate  revised proposals
selection
– major result: CORBA
October 2005
Distributed systems: coordination
models and languages
133
OMG CORBA
• Common Object Request Broker
Architecture
– Language independent
• Java, C++, C, Smalltalk, Ada, COBOL.
– Platform dependent
• requires an ORB implementation for each new
platform
A homogenous view on a heterogeneous world
October 2005
Distributed systems: coordination
models and languages
134
OMG CORBA
• OMA = Object management architecture
= Object model
+ reference model
• Object model
–
–
–
–
encapsulated entities
distinct immutable identity
accessed through well-defined interfaces
implementation and location hidden from clients
• Reference model (see next slide)
October 2005
Distributed systems: coordination
models and languages
135
OMA: Reference model
Application objects
Common facilities
Object Request broker
Object services
October 2005
Distributed systems: coordination
models and languages
136
OMA: Reference model
• Object request broker: CORBA
– Keep the ORB as simple as
possible
– push as much functionality as
possible to other OMA
components
October 2005
Distributed systems: coordination
models and languages
Application
objects
Common
facilities
Object Request broker
Object services
137
OMA: Reference model
• Object services
– domain-independent services
– used by many distributed
programs
Application
objects
Common
facilities
Object Request broker
Object services
– for discovery of other services
• naming service
• trading service
– for lifecycle management,
security, transactions, ...
October 2005
Distributed systems: coordination
models and languages
138
OMA: Reference model
• Common facilities
– horizontal orientation
– towards end-user applications
– e.g. DDCF Distributed document
Application
objects
Common
facilities
Object Request broker
Object services
Component Facility
• Application objects
– not standardised
October 2005
Distributed systems: coordination
models and languages
139
CORBA
• Main features:
–
–
–
–
–
–
–
–
ORB core
OMG Interface definition language
Language Mappings
Interface Repository
Stubs and Skeletons
Dynamic Invocation and Dispatch
Object adapters
Inter-ORB protocols
October 2005
Distributed systems: coordination
models and languages
140
CORBA: Architecture
simplified view
foo()
Client
Server
Stub
Skeleton
CORBA ORB
October 2005
Distributed systems: coordination
models and languages
141
CORBA: architecture
Client
Dyn.
Invoc.
IDL
Stub
Object Implementation
ORB
Intf
IDL
Skel
DSI
Object
adaptor
ORB Core
Same for all ORBs
Interface-specific
stubs and skeletons
October 2005
Multiple adaptors possible
ORB-private interface
Distributed systems: coordination
models and languages
142
CORBA: architecture
client
client proxy
program for A
implementation
repository
ORB
core
Request
Reply
server
interface
repository
object skeleton
adapter
ORB
core
or dynamic invocation
October 2005
Servant
A
or dynamic skeleton
Distributed systems: coordination
models and languages
143
CORBA: ORB Core
• Task:
– deliver requests to objects
– return responses
• transparency: ORB hides
–
–
–
–
object location
object implementation
object execution state
underlying communication mechanisms
October 2005
Distributed systems: coordination
models and languages
144
CORBA: IDL
• Interface Definition Language
– language-neutral
•
•
•
•
•
•
interfaces
operations
built-in types: short, long, float, boolean, enum,…
constructed types: struct, union
template types: string, sequence
object reference types
– each language has its own IDL mapping
October 2005
Distributed systems: coordination
models and languages
145
CORBA: IDL
• The Hello.idl file
module HelloAppCORBA
{
interface Hello
{
string sayHello();
};
};
October 2005
Distributed systems: coordination
models and languages
146
CORBA: IDL to Java mapping
• Mapping Hello.idl from IDL to Java
/* Hello.java as generated by idltojava
*/
package HelloAppCORBA;
public interface Hello
extends org.omg.CORBA.Object
{
String sayHello();
}
October 2005
Distributed systems: coordination
models and languages
147
CORBA: stub/skeleton
Static invocation!!
foo()
Client
Server
Stub
Skeleton
CORBA ORB
October 2005
Distributed systems: coordination
models and languages
148
CORBA: Dynamic Invocation
• Principle:
– Invoke request on any object without having
compile-time knowledge of the object’s
interfaces
• Interface repository
– gives access to IDL type system
– applications can
• can traverse hierarchy of IDL information
• use get_interface on any object
October 2005
Distributed systems: coordination
models and languages
149
CORBA: Dynamic Invocation
• Dynamic invocation
– create Request pseudo-object during execution
– invocation
• synchronous
• asynchronous
– Callback: server calls callback object with result
– Polling: client polls later the valueobject returned by server
• oneway (no result)
• Dynamic skeleton interface
– allows for servers without skeletons being compiled
statically into the program
October 2005
Distributed systems: coordination
models and languages
150
CORBA: Inter-ORB Protocols
• Interoperability between different
commercial products?
System B
System A
foo()
Client
Server
Stub
Skeleton
CORBA ORB
October 2005
Distributed systems: coordination
models and languages
151
CORBA: Inter-ORB Protocols
• General Inter-ORB Protocol
System A
System B
Client
Server
Stub
Skeleton
CORBA ORB
October 2005
GIOP - IIOP
Distributed systems: coordination
models and languages
CORBA ORB
152
CORBA: Inter-ORB Protocols
• GIOP: General Inter-ORB Protocol
– interoperability between different ORBs
– defines architecture
• standard set of message formats
• transfer syntax
• uses connection-oriented protocol
• IIOP¨: Internet Inter-ORB Protocol
– uses TCP/IP as transport protocol
October 2005
Distributed systems: coordination
models and languages
153
Object request brokers
• Conclusion:
– extends RPC
– allows for distribution of objects
– Supports heterogeneity
October 2005
Distributed systems: coordination
models and languages
154
Chapter overview
• Data representation
• Message passing
• Remote procedure calls
• Object request brokers
• Generative communication
• Event Systems
October 2005
Distributed systems: coordination
models and languages
155
Generative communication (GC)
• Overview
– goals / motivation
– how?
– the paradigm of GC
– Linda
– typical extensions
October 2005
Distributed systems: coordination
models and languages
156
GC: Goals and Motivation
• basic goals / motivation:
– communication paradigm for concurrently
active entities (e.g. processes, ...)
– heterogeneous environment
• programming language
• hardware / operating system
• communication media
– support for flexible "open systems"
• systems in which entities join and leave
dynamically
October 2005
Distributed systems: coordination
models and languages
157
GC: How ?
• separation
– computational aspects
– coordination aspects
calculations
communication
• uncoupling of concurrent entities
 no direct communication
October 2005
Distributed systems: coordination
models and languages
158
GC: the GC paradigm
• a concurrent system
– active computational entities
("processes")
– shared data space
– coordination through data exchange
• processes drop data into shared spaces
• processes retrieve data from shared spaces
– through associative "matching"
October 2005
Distributed systems: coordination
models and languages
159
GC: the GC paradigm
• example "configuration"
WordProcessor
October 2005
LinePrinter
PostscriptPrinter
Distributed systems: coordination
models and languages
160
GC: the GC paradigm
• discussion: how does it "solve" ...
– ... concurrent entities
– ... heterogeneity
– ... open systems
October 2005
processes
common data format
anonymous communication
(no inter-process
references)
Distributed systems: coordination
models and languages
161
GC: the GC paradigm
• flexible configurations: new entities join ...
WordProcessor
LinePrinter
PostscriptPrinter
PostscriptPrinter
October 2005
Distributed systems: coordination
models and languages
162
GC: the GC paradigm
• flexible configurations: entities leave ...
WordProcessor
LinePrinter
PostscriptPrinter
PostscriptPrinter
October 2005
Distributed systems: coordination
models and languages
163
GC: the GC paradigm
• characteristics
– non-determinism
• many "retrieve" requests for one data item
 any one process may get it
• many "dropped" data items, and one requests
 requesting process receives any one dropped data items
– atomic operations
October 2005
Distributed systems: coordination
models and languages
164
GC: the GC paradigm
• characteristics
– "space uncoupling"
• processes may be in different address spaces / on
different (distributed) computers
– "time uncoupling"
• a sender does not know when some process will
receive the sent items
• the sender may even no longer exist at that time
October 2005
Distributed systems: coordination
models and languages
165
GC: Linda
• Linda
– first GC-language
– binding with several computational languages
• C, C++, COBOL, Fortran, ...
– data items
– shared data space
October 2005


tuples
tuple space
Distributed systems: coordination
models and languages
166
GC: Linda
• Linda operations
– out ( … , … , … )
“generate a tuple in tuplespace”
out (“my name”, 5.2, 6);
– in / rd ( … , … , … )
“retrieve a tuple from tuplespace”
associative matching rules
float f ; int i;
rd (“my name”, ?f, ?i);
in (“string”, ?f, 6);
October 2005
Distributed systems: coordination
models and languages
167
GC: Linda
– eval ( … , … , … )
eval ( foo (a, b, c) ,
“asynchronously evaluate a tuple”
6);
– create new process, in which "foo" is performed
– when done, generate tuple:
( < result from foo(a,b,c) > , 6 )
October 2005
Distributed systems: coordination
models and languages
168
GC: Linda
• matching rules (in / rd)
– “pattern matching”
• actual / formal
• formal / actual
• actual / actual
October 2005
Distributed systems: coordination
models and languages
169
GC: Linda: a parallel compiler
while (1) {
in ( ?filename );
compile (filename);
}
...
out ("my_main.c");
...
...
out ("your_main.c");
...
while (1) {
in ( ?filename );
compile (filename);
}
("filename.c")
October 2005
...
Distributed systems: coordination
models and languages
shared tuple space
170
CG: typical extensions to Linda
• operations that simulate the use of futures
– e.g. Bonita
– dispatch
– arrived
– obtain
= in
+ continue
= check if a matching item is found
= wait until matching item is found
• operations that simulate RPC
– e.g. Laura
– tuples
October 2005
=
("procedure name", <parameters>)
Distributed systems: coordination
models and languages
171
CG: typical extensions to Linda
• multiple tuple spaces
• object-oriented versions
• data items

• shared data spaces 
objects
object spaces
– JavaSpaces
– Objective Linda / CO3PS
– matching of objects through
a "match" object operation
October 2005
class A {
…
bool match (A a) { … };
…
}
Distributed systems: coordination
models and languages
172
Chapter overview
• Data representation
• Message passing
• Remote procedure calls
• Object request brokers
• Generative communication
• Event Systems
October 2005
Distributed systems: coordination
models and languages
173
Event Systems
• Pure event-driven
– events are the basic objects of communication
– entities can
• throw particular events
• subscribe to other entities for particular events
• handle particular events
– Notification = object representing an event
– e.g. :
• Jini distributed event specification
• Corba Event service
October 2005
Distributed systems: coordination
models and languages
174
Event Systems
• Architecture
Event service
subscriber
object of interest
1.
notification
object of interest
2.
object of interest
observer
notification
subscriber
notification
observer
3.
subscriber
notification
October 2005
Distributed systems: coordination
models and languages
175
Event Systems
• Main characteristics
– Heterogeneous
– Asynchronous
– Delivery semantics ( multicast communication)
October 2005
Distributed systems: coordination
models and languages
176
Distributed Systems:
Coordination
models
October 2005
and
languages
Distributed systems: coordination
models and languages
177
Descargar

No Slide Title