Acceptor-Connector an Object Creational Pattern for Connecting

Acceptor-Connector

An Object Creational Pattern for Connecting and Initializing Communication Services Douglas C. Schmidt [email protected] Department of Computer Science Washington University St. Louis, Louis, MO 63130, USA An earlier version of this paper appeared in a chapter in the book  Pattern Languages of Program Design 3 , edited by Robert Martin, Frank Buschmann, and Dirke Riehle published by Addison-Wesley Addison-Wesley,, 1997.

1

TRACKING 

SATELLITES

STATION  PEERS 

Intent STATUS INFO 

The Acceptor-Connector design pattern decouples connection establishment and service initialization in a distributed system from the processing performed once a service is initializ tialized. ed. This This decoup decouplin ling g is achie achieved ved with three compocomponents: acceptors, connectors, and service handlers . A connector actively establishes a connection with a remote acceptor component and initializes a service handler to process data exchange exchanged d on the connection. connection. Likewise Likewise,, an acceptor passively waits for connection requests from remote connectors, establishing a connection upon arrival of such a request, and initializing a service handler to process data exchanged on the connection. The initialized service handlers then perform application-specific processing and communicate via the connection established by the connector and acceptor components.

2

WIDE AREA NETWORK  COMMANDS 

BULK DATA TRANSFER 

GATEWAY

LOCAL AREA NETWORK 

GROUND  STATION  PEERS 

Figu Figure re 1: The The Phys Physic ical al Arch Archit itec ectu ture re of a Conn Connec ecti tion on-oriented Application-level Application-level Gateway

The Gateway transmits data between its Peers using the connection-or connection-oriente iented d TCP/IP protocol [1]. In our example network configuration, each service is bound to a connection nection endpoint endpoint designated designated by an IP host host addres addresss and a TCP TCP port number number. The port number uniquely uniquely identifies identifies the type of  service. service. Maintainin Maintaining g separate separate connections connections for each type of  service/port increases the flexibility of routing strategies and provides more robust error handling if network connections shutdown unexpectedly. unexpectedly. In our distrib distributed uted applicati application, on, Gateway and Peers must be able to change their connection roles to support different use-cases. In particular, either may initiate a connection actively tively or may wait passively passively for connection connection requests. requests. For example, in one configuration, the Gateway may actively initiate connections to remote Peers in order to route data to them. In another configurati configuration, on, the Gateway may passively receive connection requests from Peers which then route data through the Gateway to another Peer. Like Like-wise, Peers may be active connection initiators in one usecase case and then then be passi passive ve connec connectio tion n accept acceptors ors in anothe anotherr useusecase.

Example

To illustrate illustrate the AcceptorAcceptor-Conne Connector ctor pattern, pattern, consider consider the multi-service, application-level application-level Gateway shown shown in FigFigure 1. In general general,, a Gateway decouples cooperating components in a distributed system and allows them to interact without having direct dependencies on each other. The particular Gateway in Figure 1 routes data between different service endpoints running on remote Peers used to monitor and control control a satellit satellitee constellation constellation.. Each service service in the Peers sends and receives several types of data via the Gateway, such as status information, bulk data, and commands. In general, Peers can be distributed throughout local area networks (LANs) and wide-area networks (WANs). The Gateway is a router that coordinates the communi˙ cation among its PeersFrom the Gateway’s perspective, perspective, the Peer services whose data it routes differ solely in terms of their application-level communication protocols, which may use different framing formats and payload types.

1

Due to the nature of our distri distribut buted ed applic applicati ation, on, conconventional ventional designs designs that designate designate connection connection establish establishment ment and service initialization roles a priori and hard-code them into the Gateway and Peer components are too inflexible. Such a design design overly couples the connection connection establish establish-ment, service initialization, and service processing components. This tight coupling make it hard to change connection roles independently of the communication roles.

3

may be necessary to extend the Gateway to interoperate with a directory service that runs over the IPX/SPX communication protocol, rather than TCP/IP.. It should be possible to decouple (1) the connection roles, i.e., which process process initiates initiates a connection connection vs. accepts the connection, connection, from (2) the communication roles, i.e., which service endpoint is the client or the server. server. In general, the distinction between “client” and “server” refer to communication roles, which may be orthogonal to connection roles. For instance, clients often play the active role when initiating connections with a passive server. However, these connection roles can be reversed. For example, a client that plays an active communication role may wait passively for another process to connect to it. The example example in Section 2 illustrates illustrates this latter use-case.

Context

A client/server application in a distributed system that utilizes connection connection-orie -oriented nted protocols protocols to communicat communicatee between between service endpoints.

4

Problem

It should be possible to write communication software that is portable to many OS platforms in order to maximize imize availab availabilit ility y and market share. Many low-lev low-level el network network programmin programming g APIs have semantics semantics that are only superficially different. different. Therefore, it hard to write portable portable applicati application on using low-lev low-level el APIs, such as sockets and TLI, due to syntactic incompatibilities.

Distribu Distributed ted applicatio applications ns often contain contain complex complex code that performs connection establishment and service initialization. In general, the processing of data exchanged between service endpoints in a distributed application is largely independent of configuration issues such as (1) which endpoint initiated the connection, i.e., the connection role vs. the communcation role and (2) the connec connectio tion n manage managemen mentt protoc protocol ol vs. the network programming API. These issues are outlined below:

It should be possible to shield programmers from the lack of typesafety in low-level network programming APIs like sockets or TLI. For example, example, connection connection establishment code should be completely decoupled from subsequent data transport code to ensure that endpoints are used correctly. Without this this strong decoupling, for instance, services may mistakenly read or write data on passive-mode transport endpoint factories that should only be used to accept connections.

Connection role vs. communication role: role: Connection establish establishment ment roles are inherently inherently asymmetrical, i.e., the the paspassive service endpoint waits and the active service endpoint initiates the connection. Once the connection is established, however, the communication role can be orthogonal to the connection role. Thus, data can be transferred between service endpoints in any manner that obeys the service’s communication protocol. protocol. Common communication protocols include peer-to-peer, peer-to-peer, request-response, and oneway streaming.

It should be possible to reduce connection latency by using OS features features like asynchronou asynchronouss connection establishme tablishment. nt. For instance, instance, application applicationss with a large number of peers may need to asynchronously establish many connections concurrently. concurrently. Efficient and scalable connection establishment is particularly important for applications that run over long-latency WANs. WANs.

The connection management protocol protocol vs. the network programming API: Different network programming interfaces, such as sockets or TLI, provide different APIs to establish connections using various connection management protocols. Regardless of the protocol used to establish a connection, however, data can be transferred between endpoints recv using uniform uniform message message passing passing operations, operations, e.g., send / recv calls.

It should be possible to reuse as much general-purpose connection connection establish establishment ment and service service initializ initialization ation software as possible in order to leverage prior development effort.

In general, the strategies for connection establishment and service initialization change much less frequently than application service implementations and communication protocols. Thus, decoupling these aspects so that they can vary independently is essential for developing and maintaining distributed applications. The following forces impact the solution to the problem of separating the connection and initialization protocols from the communication protocol:

5

Sol Solution tion

For each service offered by a distributed application, use the  Acceptor-Connector  pattern to decouple connection establishment and service initialization from subsequent processing performed by the two endpoints of a service once they are connected and initialized. Introduce two factories that produce connected and initialized service handlers, which implement the application services. services. The first factory factory,, called called acceptor, acceptor, creates creates and ini-

It should be easy to add new types of services, new service implementations, and new communication protocols without affecting affecting the existing existing connection connection establis establishhment and service initialization software. For instance, it

2

Service Handler

Connector connect(sh, addr) complete() 1

Service Handler

Service Handler

ACTIVATES 

n

Acceptor

ACTIVATES 

 peer_stream_  open()

n

1

NOTIFY WHEN 

NOTIFY WHEN 

CONNECTION COMPLETES 

CONNECTION ARRIVES 

 peer_acceptor_  accept() open()

Dispatcher Figure 2: Structure of Participants in the Acceptor-Connector Acceptor-Connector Pattern tializes a transport endpoint that passively listens at a particular address for connection requests from remote connectors. The second second factory factory,, a connector connector,, actively actively initiates initiates a connection to a remote acceptor. Acceptor and connectors both initialize the corresponding service handlers that process process the data exchanged exchanged on the connectio connection. n. Once the service service handlers handlers are connected and initialized they perform the applicationspecific processing, and generally do not interact with the acceptor and connector any further.

6

Handler and uses its transport endpoint factory to accept a new connection into the Service Handler.

Connector: A Connector is a factory that implements the strategy for actively establishing a connection and initializing its associated Service Handler. It provid provides es a method that initiates a connection to a remote Acceptor. Likewise, it provides another method that finishes activating Service Handlers Handlers whose connections connections were initiate initiated d either synchronously or asynchronously. asynchronously. The Connector uses two separate methods to support asynchronous connection establishment transparently. transparently.

Stru St ruct ctur uree

Both Both the the Acceptor and Connector activ activate ate a Service Handler by calling its activation hook method when when a conn connec ecti tion on is estab establi lish shed ed.. Once Once a Service Handler is completel completely y initializ initialized ed by an Acceptor or Connector factory it typically does not interact with these components any further.

The structure of the participants in the Acceptor-Connector pattern is illustrated by the Booch class diagram [2] in Figure 2. Service Service Handler: Handler: A Service Handler implemen implements ts an application service, typically playing the client role, server role, or both roles. It provides provides a hook method method that is called called byan Acceptor or Connector to activate activate the applicatio application n service service when the connection connection is establish established. ed. In addition, the Service Handler offers offers a data-mod data-modee transport transport endpoint endpoint that encapsulates an I/O handle, such as a socket. Once connected and initialized, initialized, this endpoint is used by the Service Handler to exchange data with its connected peer.

Dispatcher: For the Acceptor, the Dispatcher demultiplexes connection requests received on one or more transport transport endpoints endpoints to the appropriat appropriatee Acceptor. The Dispatcher allows allows multiple multiple Acceptors to regis register ter with with it in order to listen for connections from different peers on different ports simultaneously. simultaneously. For the Connector, the Dispatcher handles handles the comp comple leti tion on of conne connect ctio ions ns that that were were init initia iate ted d asyn asyn-chronously chronously.. In this case, case, the Dispatcher calls back to the Acceptor when an asynchronou asynchronouss connection connection is establi tablishe shed. d. The Dispatcher allows multiple Service Handlers to have have their their connection connectionss initiated initiated and completed pleted asynchronou asynchronously sly by a Connector. Note Note tha thatt the the Dispatcher is not necessary for synchronous connection establishment since the thread of control that initiates the connection also completes the service handler activation. A Dispatcher is typically implemented using an event demult demultipl iplexi exing ng patter pattern, n, such such those those provid provided ed by the Reactor actor [3] or Proact Proactor or [4], [4], which which handle handle synchrono synchronous us and asynchrono asynchronous us demultiplexin demultiplexing, g, respecti respectively vely.. Likewise, Likewise, the Dispatcher can be implemented as a separate thread or process using the Active Object pattern [5].

factory that implement implementss Acceptor: An Acceptor is a factory the strat strategy egy for passively establis establishing hing a connection connection and initializing initializing its associated Service Handler. In addi addi-tion, the Acceptor contains a passive-mode transport endpoint factory that creates new data-mode endpoints used by Service Handler’s to transmit data between connected peers. peers. The Acceptor’s open method initializes its transport port endpoi endpoint nt factor factory y once once by bindin binding g it to a networ network k addres address, s, such as the TCP port number the Acceptor is listening on. Once initialized, the passive-mode transport endpoint factory listens for connection requests from peers. When a connection request arrives, the Acceptor creates a Service In this diagram diagram dashed clouds clouds indicat indicatee classes; classes; dashed dashed boxes boxes in the clouds indicate template parameters; a solid undirected edge with a hollow circle at one end indicates a uses relation between two classes.

3

7

Dynam ynamic icss

threads, threads, opening opening log files, files, and/or registeri registering ng the Service Handler with a Dispatcher.

The follo followin wing g sectio section n descri describes bes the collab collabora orati tions ons perperformed by the Acceptor and Connector components in the AcceptorAcceptor-Conne Connector ctor pattern. pattern. We examine examine the three canoni canonical cal scenar scenarios ios:: for the Acceptor, asynchronous asynchronous Connector, and synchronous Connector.

7.1

3. Service Service proces processing sing phase: phase: After the connection has been established established passively passively and the Service Handler has been initialized, initialized, the service processing processing phase begins. In this phase, an application-level communication protocol, such as HTTP or IIOP, is used to exchange data between the local Service Handler and its connected remote Peer via its peer stream endpoint. endpoint. When this this exchange exchange is complete the connection and Service Handlers can be shut down and resources released.

Acceptor Acceptor Component Component Collaborat Collaborations ions

Figu Figure re

3

illu illust stra rate tess

the the

coll collab abor orat atio ion n

betw betwee een n

the the

Acceptor and Service Handler participan participants. ts. These

Server

   N INITIALIZE PASSIVE    O    T   I ENDPOINT    T    N    E    I    A   S    Z   A    O   I INSERT IN DISPATCHER     P   L    H    D   A    P    N   I START EVENT LOOP    T    E   I    N    I

acc : Acceptor

open()

sh: Service Handler

   G    E   N    I    E    C    S    I    S   S    V   E   A    R   C   H    E   O   P    S   R    P

CONNECTION EVENT CREATE, ACCEPT, AND ACTIVATE OBJECT

7.2

Connector Connector Component Component Collaborat Collaborations ions

The Connector component component can initialize initialize its Service Handler using two general general schemes: schemes: synchronous and Synchronouss service initializa initialization tion is useful useful asynchronous. Synchronou for the following situations:

register_handler(acc) handle_events() select()

FOREACH EVENT DO

   N    O    I    E   T    E    C    I    A    Z   S    V   I    A    R   L   H    E   A   P    S   I    T    I    N    I

: disp Dispatcher

accept()

If the latency for establishing a connection is very low, e.g., establish establishing ing a connect connection ion with a serve serverr on the same host via the loopback device; or

sh = new Service_Handler   peer_acceptor.accept (sh->peer_stream_) sh->open()

REGISTER HANDLER  FOR CLIENT I/O

DATA EVENT PROCESS MSG

If multiple threads of control are available and it is efficien cientt to use use a diff differ eren entt thre thread ad to conn connec ectt each each Service Handler synchronously; or

register_handler(sh)

handle_event()

If the services must be initialized in a fixed order and the client cannot perform useful work until connections are established.

service()

Figure 3: Collaborations Among Acceptor Participants Likewise, asynchronous service initialization is useful in opposite situations:

collaborations are divided into three phases: 1. Endpoint initialization initialization phase: phase: To initialize a connection passively, an application calls the open method of the Acceptor. This This method method create createss a passi passiveve-mod modee transtransport endpoint and binds it to a network address, e.g. the local local host’s host’s IP name name and a TCP port number, number, and then listens listens for connection connection requests requests from peer Connectors. Next, Next, the open method method register registerss the Acceptor object with a Dispatcher so that the dispatcher can call back  to the Acceptor when connection events events arrive. Finally, Finally, the applicati application on initiate initiatess the Dispatcher’s event event loop, loop, which which waits waits for connec connectio tion n reques requests ts to arriv arrivee from from peer peer Connectors.

If conn connec ecti tionlate onlatenc ncy y is high high and and ther theree are are many many peer peerss to connect with, e.g., establis establishing hing a large large number of connections over a high-latency WAN; or If only a single thread of control is available, e.g., if the OS platform does not provide application-level application-level threads; or If the order in which services are initialized is not important and if the client application must perform additional work, such as refreshing a GUI, while the connection is being established.

2. Ser Servic vicee initi initiali alizat zation ion phas phase: e: When When a connec connectio tion n reques requestt arriv arrives, es, the Dispatcher call callss back back to the the Acceptor’s accept meth method od.. The The accept method assemb assembles les the resour resources ces necessar necessary y to (1) create create a new new Service Handler, (2) use its passi passiveve-mod modee transp transport ort endpoi endpoint nt facto factory ry to accept accept the connec connectio tion n into into the datadatamode transport endpoint of this handler, and (3) activate the Service Handler by calling its open hook. The open hook hook of the the Service Handler can perform perform serviceservicespecific specific initiali initializati zation, on, such as allocati allocating ng locks, locks, spawning spawning

The collaborat collaborations ions among the participan participants ts in the synchronous Connector scenario can be divided into the following three phases: 1. Connection Connection initi initiation ation phase: phase: To synchronously initiate tiate a connec connectio tion n betwee between n a Service Handler and its remote remote Peer, an applic applicati ation on calls calls the Connector’s connect method. method. This method method actively actively establi establishes shes the connection by blocking the thread of control of the calling thread until the connection completes synchronously.

4

   /    N   N    O   O    I    T   I    T    A    I   A    Z    T    I   I    L   E    N    I   A   S    I   A    N   I    T    O   N   H    I    P    T   I    C   E    E   C    I    N   V    N   E    O   S    C    G    N    E    I   E    C    I   S    S   S    V    E   A    R    H    C    E    P    O    S   R    P

Client

FOREACH CONNECTION

sh: con : Connector Service Handler

: disp Dispatcher

connect(sh, addr)

INITIATE CONNECTION ACTIVATE OBJECT INSERT IN DISPATHER 

START EVENT LOOP FOREACH EVENT DO DATA ARRIVES PROCESS DATA

3. Service processing processing phase: This phase is similar to the other service processing phases described described earlier. earlier. Once the Service Handler is activated it performs applicationspecific specific service service processing processing using the data exchanged exchanged with the remote Service Handler it is connected to.

open()

Figure 5 illustrates these three phases of collaboration using asynchronous asynchronous connecti connection on establishme establishment. nt. In the asyn-

register_handler(sh)

handle_events()

Client Client

select()

   N    N    O    I    O    E FOREACH CONNECTION    T   I    T   S    C INITIATE CONNECTION    A    E   A    I   H    N INSERT IN DISPATCHER     T    P    I    N    N    O    I    C

handle_event() service()

con : Connector

sh: Service_Handler

connect(sh, addr) register_handler(con) handle_events()

START EVENT LOOP

   N    O FOREACH EVENT DO    I    E   T    E    C    I   A CONNECTION COMPLETE    Z   S    I    V    A    L   H    R    E   A   P ACTIVATE OBJECT    S   I    T    I    N INSERT IN REACTOR     I

Figure 4: Collaborations Among Connector Participants for Synchronous Synchronous Connection Connectionss

connection 2. Ser Servic vicee initiali initializat zation ion phase phase:: After the connection completes, the Connector’s connect method calls the complete method to activates the Service Handler. The complete method performs the activation by invoking the Service Handler’s open hook method, which performs service-specific initialization. initialization.

   G    E   N    I   E    C    I   S    S   S    V    E   A    R    C   H    E   O    P    S   R    P

3. Service processing phase: This phase is similar to the service processing phase a Service Handler performs once it is created by an Acceptor. In particu particular lar,, once a Service Handler is activated it performs applicationspecific service processing using data exchanged with the remote Service Handler it is connected to.

: disp Dispatcher Dispatcher

DATA ARRIVES PROCESS DATA

select() complete() open() register_handler(sh)

handle_event() service()

Figure 5: Collaborations Among Connector Participants for Asynchronous Asynchronous Connection Connectionss chronous scenario, note how the connection initiation phase is separated temporally from the service initialization phase. This decoupling enables multiple connection initiations (via connect) and completions (via complete) to proceed in parallel within each thread of control.

The collabora collaboration tion for synchronou synchronouss service service initializa initialization tion is shown shown in Figur Figuree 4. In this this sche scheme me,, the the Connector combines the connection initiation and service initialization phases into into a single single blocking operation operation.. In this scenario, scenario, only one connection is established for every invocation of  connect in each thread of control. The collaborations among the participants in the asynchronous Connector can be divided divided into the following following three phases:

8

Impl Implem emen enta tati tion on

This section explains the steps involved in building communication nication software software applicatio applications ns using using the AcceptorAcceptor-Connec Connector tor pattern. pattern. The implementat implementation ion in this section section is based on the reusable components and applications in the ACE OO network programmin programming g toolkit toolkit [6]. ACE provides provides a rich set of  reusable C++ wrappers and framework framework components that perform common communication software tasks across a range of OS platforms. The participants in the Acceptor-Connector Acceptor-Connector pattern are divided into the Reactive, Connection, and Application layers, as shown in Figure 6. The Reactive Reactive and Connection Connection layers perform perform generic, generic, applicatio application-ind n-independ ependent ent strategi strategies es for dispatchi dispatching ng events events and initializing services, respectively. The Application layer instantia instantiates tes these these generic generic strategie strategiess by providing providing concrete concrete

1. Connection Connection initiati initiation on phase: To asynchronously initiate tiate a connec connectio tion n betwee between n a Service Handler and its remote remote Peer, an applic applicati ation on calls calls the Connector’s connect method. method. As with the synchronous synchronous scenario, scenario, the Connector actively actively establi establishes shes the connect connection. ion. However, it does not block the thread of control of the caller while the connection connection completes completes asynchronousl asynchronously y. Instead, Instead, it registers the Service Handler’s transport endpoint, which we call peer stream in this this exampl example, e, with with the Dispatcher and returns control to its caller. 2. Service Service initializ initialization ation phase: phase: After After the connec connectio tion n comcompletes pletes asynchronou asynchronously sly,, the Dispatcher calls calls back back the Connector’s complete method. This method activates activates the Service Handler by calling its open hook. hook. This This open hook performs service-specific initialization.

This diagram illustrates additional additional Booch notation: directed edges indicate inheritance relationships between classes; a dashed directed edge indicates template instantiation; instantiation; and a solid circle illustrates a composition composition relationship relationship between two classes.

5

CONNECTOR  ROLE      N      O      I      T     R      E      A      Y      C      I     A      L     L      P      P      A

1

SERVICE_HANDLER  PEER_CONNECTOR 

Connector connect_service_handler() activate_service_handler() complete() connect(sh, addr)

1:

1

Concrete_Service_Handler SOCK_Acceptor

Concrete Acceptor

A

SERVICE_HANDLER  PEER_ACCEPTOR 

Acceptor

ACTIVATES 

PEER_STREAM

Service Handler open()

ACTIVATES 

A

connect_service_handler (sh, addr); addr);

make_service_handler() accept_service_handler() activate_service_handler() open() accept() sh = make_service_handler(); accept_service_handler (sh); activate_service_handler (sh);

2: activate_service_handler (sh);      E      V      I     R      T     E      Y      C      A      A      E     L      R

ACCEPTOR  ROLE

SOCK Stream Stream n Concrete Service Handler

Concrete_Service_Handler SOCK_Connector

Concrete Connector

A

     N      O      I      T     R      E      C      E     Y      A      N      L      N      O      C

SERVICE HANDLER  ROLE

Event Handler n

1

Initiation Dispatcher Dispatcher

Figure 6: Layering and Partitioning of Participants in the Acceptor-Connector Pattern Implementation Defines an interf interface ace for regis regis-Initiation Dispatcher: Defines Event Handler Handlers. tering tering,, removi removing, ng, and dispat dispatchi ching ng Event The Synchronous Synchronous Event Demultiplexer Demultiplexer, such as select [8] [8] or WaitForMultipleObjects [9], [9], ininforms the Initiation Dispatcher when to call back  applicati application-sp on-specific ecific event handlers handlers in response response to certain certain types of events. events. Common Common events events include connectio connection n acceptance events, data input and output events, and timeout events.

classes that establish connections and perform service processing. This separation of concerns increases increases the reusability, portability portability,, and extensibility extensibility in the implementation implementation of the Acceptor-Connector pattern. The following discussion of the Acceptor-Connector pattern implementation starts at the bottom with the Reactive layer and works upwards upwards through the Connection Connection layer and Application layer. layer.

8.1

Note that the Initiation Dispatcher is an implementat mentation ion of the Dispatcher participa participant nt described described in Section 6. In general, general, the Acceptor-C Acceptor-Connec onnector tor Dispatcher participant can be reactive, reactive, proactive, proactive, or multi-threaded. The Initiation Dispatcher Dispatcher in this implemen particular Initiation implementatation uses the reactiv reactivee model to demultipl demultiplex ex and dispatch dispatch concrete event event handlers handlers in a single thread of control. In our example the Initiation Initiation Dispatcher Dispatcher is a Singleton [10] since we only need one instance of it for the entire process.

React Reactive ive Layer Layer

The Reactive layer handles events that occur on transport endpoints represented by I/O handles, such as socket endpoints. The two participants in this layer, the Initiation Dispatcher and Event Event Handler Handler, are defined by the Reactor Reactor pattern [3]. This pattern pattern enables efficient efficient demuldemultiplexing of multiple types of events from multiple sources within a single thread of control. The two main roles in the Reactive layer are:

8.2 Event Handler: Specifies an interface consisting of hook  method methodss [7] that that abstra abstractl ctly y repres representthe entthe event event proces processi sing ng operations that can be provided by an application. application. For instance, instance, these hook methods signify events such as a new connection request, request, a completion completion of a connection connection request request started started asynchronously, asynchronously, or the arrival of data from a connected peer. The Acceptor and Connector components components are concrete event handlers that derive from Event Event Handler Handler.

Connec Connectio tion n Layer Layer

The Connection layer 1. Creates Creates Service Handlers; 2. Passive Passively ly or actively actively connects connects Service Handlers to their remote peers; and Handlers once they are con3. Activat Activates es Service Handlers nected.

6

All behavior behavior in this layer is completely completely generic. generic. In particular, note how the classes in the implementation implementation described below delegate to concrete IPC mechanisms and Concrete Service Handlers, which are instantiated by the Application layer described in Section 8.3. The manner manner in which which the Applic Applicati ation on layer layer delega delegates tes to the Connection layer is similar to how the Connection layer layer delegate delegatess to the Reacti Reactive ve layer. layer. For instanc instance, e, the Initiation Initiation Dispatcher Dispatcher in the Reacti Reactive ve layer layer proprocesses initialization-related events, such as establishing connections asynchronously, on behalf of the Connection layer. There are three primary roles in the Connection layer: Service Service Handler Handler, Acceptor, and Connector.

// Initialization Initialization method. Connector (void); // Actively Actively connecting connecting and activate activate a service. service. int connect connect (SERVICE (SERVICE_HAN _HANDLER DLER *sh, const PEER_CONNECTOR::PEE PEER_CONNECTOR::PEER_ADDR R_ADDR &addr, Connect_Mode Connect_Mode mode); protected: // Defines Defines the active active connecti connection on strategy strategy. . virtual int connect_service_hand connect_service_handler ler (SERVICE_HANDLER (SERVICE_HANDLER *sh, const PEER_CONNECTOR::PEE PEER_CONNECTOR::PEER_ADDR R_ADDR &addr, Connect_Mode Connect_Mode mode); // Regist Register er the SERVIC SERVICE_H E_HAND ANDLE LER R so that that it can can // be activate activated d when the connecti connection on complete completes. s. int register_handler register_handler (SERVICE_HANDLER *sh, Connect_Mode Connect_Mode mode);

This abst abstra ract ct clas classs inhe inheri rits ts from from Service Service Handler: Handler: This Event Handler and provides a generic interface for processing services provided by clients, servers, or components that perform both roles. Applicatio Applications ns must customize customize this class via inheritance to perform a particular type of service. The Service Handler interface is shown below:

// Defines Defines the handler’ handler’s s concurre concurrency ncy strategy strategy. . virtual int activate_service_han activate_service_handler dler (SERVICE_HANDLER (SERVICE_HANDLER *sh); // Activate Activate a SERVICE_ SERVICE_HAND HANDLER LER whose whose // non-blocking non-blocking connection completed. virtual virtual int complete complete (HANDLE (HANDLE handle); handle);

// PEER_ PEER_STR STREA EAM M is the type of the // Concrete Concrete IPC mechanis mechanism. m. template PEER_STREAM> class class Service_ Service_Hand Handler ler : public public Event_Ha Event_Handle ndler r { public: // Pure Pure virtua virtual l metho method d (defin (defined ed by a subcl subclas ass). s). virtua virtual l int open (void) (void) = 0;

private: // IPC mechanis mechanism m that establishes establishes // connecti connections ons actively actively. . PEER_CONNECTOR PEER_CONNECTOR connector_; connector_; // Collecti Collection on that maps HANDLEs HANDLEs // to SERVICE_ SERVICE_HAND HANDLER LER *s. Map_Manager handler_map_;

// Acces Accessor sor method method used used by Accep Accepto tor r and // Connecto Connector r to obtain obtain the underlyi underlying ng stream. stream. PEER_STR PEER_STREAM EAM &peer &peer (void) (void) { return peer_stream_; peer_stream_; }

// Inheri Inherited ted from from the Event Event_Ha _Handl ndler er -- will will be // called called back back by Eactor Eactor when when event events s comple complete te // asynchronously. asynchronously. virtual virtual int handle_e handle_event vent (HANDLE, (HANDLE, EVENT_TY EVENT_TYPE); PE); };

// Retur Return n the addre address ss that that we’re we’re connec connected ted to. PEER_STREAM::PEER_A PEER_STREAM::PEER_ADDR DDR &remote_addr (void) { return peer_stream_.remot peer_stream_.remote_addr e_addr (); }

// Useful Useful "short-h "short-hand" and" macros used below. #define SH SERVICE_HANDLER SERVICE_HANDLER #define #define PC PEER_CON PEER_CONNECT NECTOR OR

protected: // Concrete Concrete IPC mechanis mechanism m instance instance. . PEER_STREAM PEER_STREAM peer_stream_; peer_stream_; };

The Connector is parameterized by a particular type of  PEER CONNECTOR and SERVICE HANDLER. The PEER CONNECTOR provides the transport mechanism used by the Connector to actively actively establish establish the the connectio connection, n, either either synchronously chronously or asynchronousl asynchronously. y. The SERVICE HANDLER provides the service that processes data exchanged with its connected peer. C++ parameterized types are used to decouple (1) the connection establishment strategy from (2) the type of service handler, network programming programming interface, and transport layer connection protocol. Parameterized types are an implementation decision that help help improv improvee portabil portability ity.. For instanc instance, e, they they allow allow the wholes wholesale ale replac replaceme ement nt of the IPC mechan mechanism ismss used used by the Connector. This This makes makes the the Connector’s connection establishment code portable across platforms that contain different network programming interfaces, e.g., sockets ets but but not not TLI, or vice vice versa versa.. For For exam exampl ple, e, the the PEER CONNECTOR template argument can be instantiated with either ther a SOCK Connecto Connector r or a TLI Connect Connector or, depend depend-ing on whether the platform supports sockets or TLI [11]. Another motivation for using parameterized types is to im-

Once the Acceptor or Connector establishes a connectio nection, n, they they call call the open hookof hookof a Service Handler. This pure virtual method must be defined by a Concrete Service Handler subclass, subclass, which performs performs servicespecific initializations and subsequent processing. implementss the generic generic Connector: This abstract class implement strategy for actively actively establishing connections and initializing Service Handlers. The interfa interface ce of the Connector is shown below: // The The SERVIC SERVICE_H E_HAN ANDLE DLER R is the the type type of servic service. e. // The The PEER_C PEER_CONN ONNEC ECTOR TOR is the type type of concre concrete te // IPC active active connecti connection on mechanis mechanism. m. template PEER_CONNECTOR> class class Connecto Connector r : public public Event_Ha Event_Handle ndler r { public: enum Connect_ Connect_Mode Mode { SYNC, SYNC, // Initiate Initiate connecti connection on synchron synchronousl ously. y. ASYNC ASYNC // Initiate Initiate connecti connection on asynchro asynchronous nously. ly. };

7

prove run-time efficiency since template instantiation occurs at compile-time. An even even more more dynami dynamicc type type of decoup decoupli ling ng could could be achieved achieved via inheritance inheritance and polymorphis polymorphism m by using the Factory Method and Strategy patterns described in [10]. For instance, a Connector could store a pointer to a PEER CONNECTOR base base cla class ss.. The The connect method method of this this PEER CONNECTOR CONNECTOR could be dynamically bound at runtime in accordanc accordancee with the subclass subclass of  PEER PEER CONNECT CONNECTOR OR returned returned from a Factory. Factory. In general, general, the tradeoff tradeoff between parameterized types and dynamic binding is that parameterized types can incur additional compile/link-time overhead, whereas whereas dynamic dynamic binding binding can incur additional additional run-time run-time overoverhead. The connect method is the entry point an application uses to initiate a connection via a Connector. It’s implementation is shown below.

If the value of the Connect Mode parameter is SYNC the SERVICE HANDLER will be activated once the connection completes completes synchronou synchronously sly,, as illustrat illustrated ed in Figure Figure 7. This figure is similar to Figure 4, but provides additional implementation details, such as the use of the get handle and handle event hook methods.

   &    N   N    O    O   I    I    T   T    A    A    I   I    Z    T   L    I    E    N    I   A    I   S    T   A    N   I    H    O   N    I    I   P    T   E    C   C    E   I    N   V    N   E    O   S    C

Client FOREACH CONNECTION INITIATE CONNECTION SYNC CONNECT

con : : SOCK  Connector Connector

: Initiation Dispatcher Dispatcher

connect(sh, addr, SYNC) connect_service_handler(sh, connect_service_handler (sh, addr) connect() activate_service_handler(sh)

ACTIVATE OBJECT

open() register_handler(sh)

INSERT IN REACTOR 

get_handle()

EXTRACT HANDLE

handle_events()

   G START EVENT LOOP    N    E    I   E FOREACH EVENT DO    C    I   S    S   S    V    A    E    R    H DATA ARRIVES    C    E    P    O    S   R PROCESS DATA    P

templ template ate PC> int Connector::connect PC>::connect (SERVICE_HANDLER (SERVICE_HANDLER *service_handler, *service_handler, const PEER_CONNECTOR::PEE PEER_CONNECTOR::PEER_ADDR R_ADDR &addr, Connect_Mode Connect_Mode mode) { connect_service_han connect_service_handler dler (service_handler, (service_handler, addr, addr, mode); mode); }

sh: Service Handler

select() handle_event() service()

Figure 7: Collaborations Among the Connector Participants for Synchronous Synchronous Connection Connectionss To conn connec ectt with with mult multip iple le Peers effici efficient ently ly,, the Connector also may need to actively establish connections asynchronously, asynchronously, i.e., without without blocking blocking the caller caller.. Asynchronous behavior is specified by passing the ASYNC connection mode to Connector::connect, as illustrated in Figure Figure 8. This figure is similar similar to Figure 5, but it also pro-

This method uses the Bridge pattern [10] to allow Concrete Connector Connectors to transparent transparently ly modify modify the connect connection ion strategy strategy,, without without changing the compone component nt interf interface ace.. Therefo Therefore, re, the connect method delegates to the Connector’s connection strategy, strategy, connect service handler, which initiates a connection as shown below: templ template ate PC> int Connector::connect_servic PC>::connect_service_handler e_handler (SERVICE_HANDLER (SERVICE_HANDLER *service_handler, *service_handler, const PEER_CONNECTOR::PEE PEER_CONNECTOR::PEER_ADDR R_ADDR &remote_addr, &remote_addr, Connect_Mode Connect_Mode mode) { // Delegate Delegate to concrete concrete PEER_CON PEER_CONNECT NECTOR OR // to establis establish h the connecti connection. on.

Client

   N    N    O    I    O    E    T   I    T   S    C    A    E   A    I   H    N    T   P    I    N    N    O    I    C

FOREACH CONNECTION INITIATE CONNECTION ASYNC CONNECT INSERT IN DISPATCHER 

con : : SOCK  Connector Connector

if (connector_.connect (connector_.connect (*service_handler, remote_addr, mode mode) ) == -1) -1) { if (mode (mode == ASYNC ASYNC && errno errno == EWOUL EWOULDB DBLOC LOCK) K) { // If connecti connection on doesn’t doesn’t complete complete immediat immediately ely // and we are using non-bloc non-blocking king semantics semantics // then regist register er this this object object with with the // Initiati Initiation_D on_Dispa ispatche tcher r Singleto Singleton n so it will // callback callback when the connecti connection on is complete complete. . Initiation_Dispatcher::instance ()->register_handl ()->register_handler er (this, WRITE_MASK); WRITE_MASK);

: Initiation Dispatcher

connect(sh, addr, ASYNC) connect_service_handler(sh, addr) connect() register_handler(con) handle_events()

START EVENT LOOP

   N    O FOREACH EVENT DO    I    E   T    E    C    A    I   Z   S CONNECTION COMPLETE    I   A    V    L    R    H    E   A   P ACTIVATE OBJECT    S   I    T    I    N INSERT IN REACTOR     I

sh: Service Handler

select() handle_event() activate_service_handler(sh) open() register_handler(sh) get_handle()

   G    E   N    I   E    C    S    I   S   S    V    E   A    R    C   H    E   O    P    S   R    P

// Store Store the SERVIC SERVICE_H E_HAND ANDLER LER in the map of // pending pending connecti connections. ons. handler_map_.bind (connector_.get_ha (connector_.get_handle ndle (), service_handler); service_handler);

EXTRACT HANDLE

DATA ARRIVES PROCESS DATA

handle_event() service()

Figure 8: Collaborations Among the Connector Participants for Asynchronous Asynchronous Connections Connections

} } else else if (mode (mode == SYNC) SYNC) // Activate Activate if we connect connect synchron synchronousl ously. y. activate_service_ha activate_service_handler ndler (service_handler); (service_handler);

vides additional details that correspond to the current implementation. CONNECTOR class Once instantia instantiated, ted, the PEER CONNECTOR class provides the concrete IPC mechanism to initiate connections

}

To save space, most of the error handling in this paper has been omitted.

8

synchr synchrono onousl usly y or asynch asynchron ronous ously ly.. The implem implement entati ation on of the Connec Connector tor patter pattern n shown shown here here uses uses asynch asynchron ronous ous connect connection ion mechan mechanism ismss provid provided ed by the OS and comcommuni munica cati tion on protoc protocol ol stack stack.. For For inst instan ance ce,, on UNIX or Win32 the Connector can set sockets into non-blocking mode mode and use an event event demult demultipl iplex exer er like like select or WaitForMultipleObjects to determ determine ine when when the conconnection completes. To handle asynchronous connections that are still pending completion, the Connector maintains a map of  Service Handlers. Since Since the the Connector inherits from Event Handler, the Initiation Dispatcher can automatically ically call back back to the Connector’s Connector’s handle event method when a connection completes. The handle event method method is an Adapte Adapterr [10] [10] that that Initiation Initiat ion Dispatcher Dispatch er transforms the ’s event handlin dling g inte interf rfac acee to a call call to the the Conn Connec ecto torr patt patter ern’ n’ss complete method. The Connector’s handle event method is shown below:

Connector::activate_servic PC>::activate_service_handler e_handler (SERVICE_HANDLER (SERVICE_HANDLER *service_handler) *service_handler) { service_handler->o service_handler->open pen (); }

The Service Handler’s open hook is called when a connec connectio tion n is establi establishe shed d succes successfu sfully lly.. Note Note that that it is called called regardles regardlesss of whether whether (1) connections connections are initiated initiated synchronously or asynchronously or (2) they are connected actively actively or passively passively.. This uniformity uniformity makes it possible possible to write Service Handlers whose processing can be completely decoupled from how they are connected and initialized. This abstra abstract ct class class implem implement entss the generi genericc stratstratAcceptor: This egy for passively establishing connections and initializing Service Handlers. The interface of the Acceptor is shown below. // The SERVIC SERVICE_H E_HAND ANDLER LER is the type type of servic service. e. // The PEER_A PEER_ACCE CCEPTO PTOR R is the type type of concre concrete te // IPC passive passive connecti connection on mechanis mechanism. m. template PEER_ACCEPTOR> class class Acceptor Acceptor : public public Event_Ha Event_Handle ndler r { public: // Initiali Initialize ze local_ad local_addr dr transpor transport t endpoint endpoint factory factory // and register register with Initiation_ Initiation_Disp Dispatch atcher er Singleton. Singleton. virtual virtual int open (const PEER_ACCEPTOR::PEER PEER_ACCEPTOR::PEER_ADDR _ADDR &local_addr); &local_addr);

templ template ate PC> int Connector::handle_event PC>::handle_event (HANDLE handle, EVENT_TYPE type) { // Adapt Adapt the Initiati Initiation_D on_Dispa ispatche tcher’s r’s event event // handl handling ing API to the the Connec Connector tor’s ’s API. API. complete (handle); }

// Factory Factory Method that creates, creates, connects connects, , and // activates SERVICE_HANDLER’s. SERVICE_HANDLER’s. virtual virtual int accept accept (void); (void);

The complete method activates the SERVICE HANDLER whose non-blocking non-blocking connection just completed completed successfully, successfully, as follows:

protected: // Defines Defines the handler’ handler’s s creation creation strategy strategy. . virtual SERVICE_HANDLER * make_service_handl make_service_handler er (void);

templ template ate PC> int Connector::complete PC>::complete (HANDLE handle) { SERVICE_HANDLER SERVICE_HANDLER *service_handler *service_handler = 0;

// Defines Defines the handler’ handler’s s connecti connection on strategy strategy. . virtual int accept_service_handl accept_service_handler er (SERVICE_HANDLER (SERVICE_HANDLER *);

// Locate Locate the SERVICE_ SERVICE_HAND HANDLER LER correspo correspondin nding g // to the the HANDLE HANDLE. . handler_map_.find handler_map_.find (handle, service_handler); service_handler);

// Defines Defines the handler’ handler’s s concurre concurrency ncy strategy strategy. . virtual int activate_service_han activate_service_handler dler (SERVICE_HANDLER (SERVICE_HANDLER *);

// Transfer Transfer I/O handle handle to SERVICE_ SERVICE_HAND HANDLER LER *. service_handler->se service_handler->set_handle t_handle (handle); // Remove Remove handle handle from Initiati Initiation_D on_Dispa ispatche tcher. r. Initiation_Dispatcher::instance ()->remove_handler ()->remove_handler (handle, WRITE_MASK); WRITE_MASK);

// Demultip Demultiplexi lexing ng hooks hooks inherite inherited d from Event_Ha Event_Handle ndler, r, // which which is used by Initiati Initiation_D on_Dispa ispatche tcher r for // callbacks. callbacks. virtual virtual HANDLE HANDLE get_hand get_handle le (void) (void) const; const; virtual virtual int handle_c handle_close lose (void); (void);

// Remov Remove e handle handle from the map. handler_map_.unbind handler_map_.unbind (handle);

// Invoked Invoked when connecti connection on requests requests arrive. virtual virtual int handle_e handle_event vent (HANDLE, (HANDLE, EVENT_TY EVENT_TYPE); PE);

// Connecti Connection on is complete complete, , so activate activate handler. handler. activate_service_ha activate_service_handler ndler (service_handler); (service_handler);

private: // IPC mechanis mechanism m that establishes establishes // connections connections passively. passively. PEER_ACCEPTOR PEER_ACCEPTOR peer_acceptor_; peer_acceptor_; };

}

The complete method finds and removes the connected SERVICE HANDLER from from its its inte intern rnal al map map and and tran transsfers fers the the I/O I/O HANDLE to the the SERVICE HANDLER. FiFinally nally,, it initia initializ lizes es the SERVICE HANDLER by callin calling g activate service handler. This method method delega delegates tes to the concurrency concurrency strategy strategy designated designated by the SERVICE HANDLER’s open hook, as follows:

// Useful Useful "short-h "short-hand" and" macros used below. #define SH SERVICE_HANDLER SERVICE_HANDLER #define #define PA PEER_ACC PEER_ACCEPTO EPTOR R

The Acceptor is parameterized by a particular type of  PEER ACCEPTOR and SERVICE HANDLER. The PEER ACCEPTOR provides the transport mechanism used by the Acceptor to passiv passively ely establ establish ish the connect connection ion.. The

templ template ate PC> int

9

SERVICE HANDLER provides provides the service service that processes data data exchan exchanged ged with with its remot remotee peer peer. Note Note that that the SERVICE HANDLER is a concrete service handler that is provided by the application layer. Parameterized types decouple the Acceptor’s connection establishment strategy from the type of service handler, network programming interface, and transport transport layer connection initiation initiation protocol protocol.. As with the Connector, the use of parameterized types helps improve portability by allowing the wholesale replacement of the mechanisms used by the Acceptor. This makes the connection establishment code code portable across platforms that contain different network programming interfaces, such as sockets but not TLI, or vice versa. versa. For example example,, the PEER ACCEPTOR template argument can be instantiated with either a SOCK Acceptor or a TLI Acceptor, depending on whether the platform supports sockets or TLI more efficiently. The implementation of the Acceptor’s methods is presented below. Applications initiatize an Acceptor by calling its open method, as follows:

Demultiplexer, such as select, is then used to detect and demultipl demultiplex ex incoming incoming connection connection requests requests from client clients. s. Since Since the Acceptor class inherits from Event Handler, the Initiation Initiation Dispatcher Dispatcher can autoautomatically call back to the Acceptor’s handle event meth method od when a conn connec ecti tion on arri arrives ves from from a peer peer.. This This method method is an Adapte Adapterr that that transf transform ormss the Initiation Dispatcher’s event event handling handling interfa interface ce to a call call to the Acceptor’s accept method, as follows: templa template te int int Acceptor::handle_event PA>::handle_event (HANDLE, EVENT_TYPE) { // Adapt Adapt the Initiati Initiation_D on_Dispa ispatche tcher’s r’s event event handling handling // API to the the Accept Acceptor’ or’s s API. API. accept accept (); }

As shown shown below below,, the accept method method is a Templat emplatee Method [10] that implements the Acceptor-Connector pattern’ tern’ss passi passive ve initia initializ lizati ation on strate strategy gy for creati creating ng a new SERVICE HANDLER, accepting a connection into it, and activating activating the service:

templ template ate PA> int Acceptor::open PA>::open (const PEER_ACCEPTOR::PEER PEER_ACCEPTOR::PEER_ADDR _ADDR &local_addr) &local_addr) { // Forward Forward initiali initializati zation on to the PEER_ACC PEER_ACCEPTO EPTOR. R. peer_acceptor_.open peer_acceptor_.open (local_addr); (local_addr);

templa template te int int Acceptor::accept PA>::accept (void) { // Create Create a new SERVICE_ SERVICE_HAND HANDLER. LER. SH *service *service_han _handler dler = make_ser make_service vice_han _handler dler ();

// Register Register with Initiati Initiation_D on_Dispa ispatche tcher, r, which which // ‘‘double-dispatches ‘‘double-dispatches’’ ’’ without get_handle() // metho method d to extrac extract t the HANDLE HANDLE. . Initiation_Dispatcher::instance ()->register_handle ()->register_handler r (this, READ_MASK); READ_MASK);

// Accept Accept connecti connection on from client. accept_service_han accept_service_handler dler (service_handler); (service_handler); // Activate Activate SERVICE_ SERVICE_HAND HANDLER LER by calling calling // its open( open() ) hook. hook. activate_service_h activate_service_handler andler (service_handler); (service_handler);

} }

The open method is passed a local addr. This parameter contains a network address, e.g., the local host’s IP name and TCP port number, number, used to listen listen for connecti connections ons.. It forwards this address to the passive connection acceptance ACCEPTOR. This mechamechanism mechanism defined by the PEER ACCEPTOR nism initializes the transport endpoint factory, which advertises its address to clients who are interested in connecting with the Acceptor. The behavior of the transport endpoint factory is determined by the type of  PEER PEER ACCEPTO ACCEPTOR R instantiated by a user. For instance, it can be a C++ wrapper [12] for sockets [13], TLI [14], STREAM pipes [15], Win32 Named Pipes, etc. After the transport endpoint factory has been initialized, the open method method register registerss itself itself with the Initiation Dispatcher. The Initiation Initiation Dispatcher Dispatcher perform formss a “dou “doubl blee disp dispat atch ch”” back back to the the Acceptor’s get handle method method to obtain obtain the underlying underlying transport transport endpoint factory HANDLE, as follows:

This method is very concise since it factors all low-level details into the concrete SERVICE HANDLER and PEER ACCEPTOR, which are instantiated via parameterized types and can be customized by subclasses of the Acceptor. In particular, since the accept is a Template Method, subclasses can extend any or all of the Acceptor’s connection establishment and initialization strategies. This flexibility makes it possible to write Service Handlers whose behavior is decoupled from the manner in which they are passively connected and initialized. The make service handler factory method defines the default strategy an Acceptor uses to create SERVICE HANDLERs, as follows: templa template te SH * Acceptor::make_service_h PA>::make_service_handler andler (void) { return return new new SH; }

template template HANDLE Acceptor::get_handle PA>::get_handle (void) { return peer_acceptor_.get_ peer_acceptor_.get_handle handle (); }

The default behavior uses a “demand strategy,” which creates a new SERVICE HANDLER for every new connection. However, subclasses of  Acceptor can override this strategy to create SERVICE HANDLERs HANDLERs using other strategies, such as creating an individual Singleton [10] or dynamically linking the SERVICE HANDLER from a shared library.

Initiation Dispatcher Dispatcher stores The Initiation stores this HANDLE internally in a table. A Synchron Synchronous ous Event Event

10

The SERVICE HANDLER connection acceptance strategy egy used used by the the Acceptor is defin defined ed belo below w by the the accept service handler method:

SOCK Stream Stream classes used in Section 9 are part of the ACE ACE C++ socket socket wrapper wrapper library library [11]. [11]. These These wrappers wrappers encapsulat encapsulatee the stream-or stream-oriente iented d semantics semantics of connection connection-oriented oriented protocols protocols like like TCP and SPX with efficient, efficient, portable, portable, and type-safe C++ wrappers. The three three main main roles roles in the Applic Applicati ation on layer layer are descri described bed below.

templ template ate PA> int Acceptor::accept_servic PA>::accept_service_handler e_handler (SH *handler *handler) ) { peer_acceptor_->acc peer_acceptor_->accept ept (handler->peer (handler->peer ()); }

Concrete Service Handler: This class defines the concret cretee appl applic icat atio ion n serv servic icee acti activa vate ted d by a Concrete Acceptor or a Concrete Connector. A Concrete Service Handler is instantiated with a specific type of  C++ IPC wrapper that exchanges data with its connected peer.

The default behavior delegates to the accept method provide vided d by the the PEER ACCEPTOR ACCEPTOR. Subcla Subclass sses es can overri override de the accept service handler method method to perform perform more sophisti phisticat cated ed behav behavior ior such such as authen authenti ticat cating ing the identi identity ty of the client to determine whether to accept or reject the connection. Acceptor’s SERVICE HANDLER concurrency The Acceptor’s strategy is defined by the activate service handler method:

Concrete Connector: This class instantiates the generic Connector factory factory with concrete concrete parameter parameterized ized type arguCONNECTOR R. ments for SERVICE HANDLER and PEER CONNECTO Concrete Acceptor: This class instantia instantiates tes the generic generic Acceptor factory with concrete parameterized type arguACCEPTOR. ments for SERVICE HANDLER and PEER ACCEPTOR Concrete Concrete Service Handlers can also define a service’ vice’ss concur concurren rency cy strate strategy gy.. For example example,, a Service Handler may inherit from the Event Handler and employ the Reactor [3] pattern to process data from peers in a single-thr single-thread ead of control. Convers Conversely ely,, a Service Handler might use the Active Object pattern [5] to process incoming data in a different thread of control than the one used by the Acceptor to connect it. Below, we implement implement Concrete Service Handlers Handlers for our Gateway example, illustrates how several different concurrency strategies can be configured flexibly without affecting the structure or behavior of the Acceptor-Connector pattern.

templ template ate PA> int Acceptor::activate_serv PA>::activate_service_handle ice_handler r (SH *handler *handler) ) { handler->open handler->open (); }

The defa defaul ultt beha behavi vior or of this this meth method od is to acti activa vate te the the SERVICE HANDLER by callin calling g its open hook hook.. This This allow allowss the SERVICE SERVICE HANDLER HANDLER to choos hoosee its own conc concur urre renc ncy y strat strateg egy y. For For inst instan ance ce,, if the SERVICE HANDLER inherits inherits from Event Event Handler Handler it can regis regis-Initiation Dispatcher Dispatcher. This ter ter with with the Initiation This allo allows ws the Initiation Initiation Dispatcher Dispatcher to dispatch the SERVICE HANDLER’s HANDLER’s handle event event method method when when event eventss occur cur on its its PEER PEER STREAM STREAM endpoint endpoint of communicat communication. ion. Concrete Concrete Acceptor Acceptors can override this strategy to do more sophisticat sophisticated ed concurrenc concurrency y activati activations. ons. For instance, instance, a subclass could make the SERVICE HANDLER an Active Object Object [5] that that proces processes ses data data using using multimulti-thr thread eadingor ingor multimultiprocessing. When When an Acceptor term termin inat ates es,, eith either er due due to ererrors rors or due due to the the enti entire re appl applic icat atio ion n shut shutti ting ng down down,, Initiation Dispatcher Dispatcher calls the Initiation calls the Acceptor’s handle close method, which can release any dynamically acquired acquired resources. resources. In this case, case, the handle close method simply forwards the close request to the PEER ACCEPTOR’s transport endpoint factory, as follows:

In the sample code in Section 9, SOCK Connector Connector and SOCK Acceptor are the IPC mechanisms used to establish connections actively and passively, passively, respectively. respectively. Likewise, a SOCK Stream is used as the data transport delivery mechanism. However, However, parameterizing the Connector Acceptor and with different mechanisms (such as a TLI Connector or Named Named Pipe Acceptor Acceptor) is straightforward since the IPC mechanisms mechanisms are encapsulat encapsulated ed in C++ wrapper wrapper classes. classes. Likewise Likewise,, it is easy to vary the data transtransfer mechan mechanism ism by parame parameter terizi izing ng the Concrete Service Handler with a different PEER STREAM STREAM, such as an SVR4 TLI Stream UNIX or a Win32 Named Pipe Stream. Secti Section on 9 illust illustrat rates es how to instan instantia tiate te a Concrete Service Service Handler Handler, Concrete Concrete Connecto Connector r, and Concrete Acceptor that implement implement the Peers and Gateway described described in Section 2. This particular particular example example of the Application layer customizes the generic initialization tion strat strategi egies es provid provided ed by the Connector and Acceptor components in the Connection layer.

templ template ate PA> int Acceptor::handle_close PA>::handle_close (void) { peer_acceptor_.clos peer_acceptor_.close e (); }

8.3

Appli Applicat cation ion Layer Layer

The Application Layer supplies concrete interprocess communication munication (IPC) mechanism mechanismss and a concrete concrete Service Handler. IPC IPC mech mechan anis isms ms are encap encapsu sula late ted d in C++ classes to simplify programming, enhance reuse, and to enable wholesale wholesale replaceme replacement nt of IPC mechanism mechanisms. s. For example, ample, the SOCK Acceptor Acceptor, SOCK Connecto Connector r, and

9

Exam Exampl plee Reso Resolv lved ed

The code below illustrates how the Peers and Gateway described in Section 2 can use the Acceptor-Connector pat-

11

terns to simplify connection establishment and service initializat tialization. ion. Section Section 9.1 illustra illustrates tes how the Peers play a passive role and Section 9.2 illustrates how the Gateway plays an active role in establishing establishing connections with the passive Peers.

9.1

Handler class processes status data sent to and received from a Gateway: class class Status_H Status_Handl andler er : public public PEER_HAN PEER_HANDLER DLER { public: // Performs Performs handler handler activati activation. on. virtua virtual l int open (void (void) ) { // Make Make handl handler er run in separ separate ate thread thread (note (note // that that Thread Thread::s ::spaw pawn n requi requires res a pointe pointer r to // a stati static c method method as the threa thread d entry entry point) point). .

Concre Concrete te Component Componentss for Peers Peers

Concrete Acceptor and Figure 9 illustrates how the Concrete Concrete Service Handler components components are strucstructured in a Peer. The Acceptor components in this figure

     N      O      I      T     R      E      A      Y      C      I     A      L     L      P      P      A

1 ACTIVATES 

SOCK_Stream

Bulk Data n Handler

ACTIVATES 

SOCK_Stream

PEER_STREAM

Service Handler

// Static Static entry entry point point into into thread thread, , which which blocks blocks // on the handle_ handle_eve event nt () call call in its own thread thread. . static static void *service *service_run _run (Status_ (Status_Hand Handler ler *this_) *this_) { // This This metho method d can block block since since it // runs runs in its own thread thread. . while while (this_-> (this_->hand handle_e le_event vent () != -1) continue; }

Command Acceptor

Bulk_Data_Handler SOCK_Acceptor

Status n Handler ACTIVATES       R      O      R      T      P     E      E     Y      A      C      L      C      A

}

Command_Handler SOCK_Acceptor

SOCK_Stream

Command n Handler

Thread::spawn Thread::spawn (&Status_Handler::s (&Status_Handler::service_run ervice_run, , this);

1

Bulk Data Acceptor

// Receiv Receive e and proce process ss status status data data from from Gatew Gateway. ay. virtual virtual int handle_e handle_event vent (void) (void) { char buf[MAX_STATUS_DAT buf[MAX_STATUS_DATA]; A]; stream_. stream_.recv recv (buf, (buf, sizeof sizeof buf); buf); // ... ... }

Status_Handler SOCK_Acceptor

1

Status Acceptor

// ... ... };

SERVICE_HANDLER  PEER_ACCEPTOR 

Acceptor

HANDLER also can be subclassed The PEER HANDLER subclassed to produce concrete service handlers that process bulk data and commands. Bulk Data Handler Handler class processes For instance, instance, the Bulk bulk data sent to and received from the Gateway.

Figure 9: Structure of Acceptor Participants for Peers complement the Connector components in Figure 11.

class class Bulk_Dat Bulk_Data_Ha a_Handle ndler r : public public PEER_HAN PEER_HANDLER DLER { public: // Performs Performs handler handler activati activation. on. virtua virtual l int open (void (void) ) { // Handler Handler runs in separate separate process. process. if (for (fork k () == 0) // In chil child d proc proces ess. s. // This This metho method d can block block since since it // runs runs in its its own process process. . while while (handle_ (handle_even event t () != -1) continue; // ... ... }

Service Handlers for Communicating with a Gateway: Status Handler Handler, Bulk The classe classess shown shown below below,, Status Data Data Handler Handler, and Command Handler, process routing ing mess messag ages es sent sent to and and rece receiv ived ed from from a Gateway. Since these Concrete Concrete Service Service Handler Handler classes inherit from Service Handler they are capable of being initialized passively by an Acceptor. To illustrate the flexibility of the Acceptor-Connector pattern, each open routine in the Service Handlers can implement implement a different different concurrency concurrency strategy strategy.. In particular particular,, Status Handler Handler is activated it runs in a sepwhen the Status arate thread; thread; the Bulk Data Data Handler Handler runs as a separate arate proces process; s; and the Command Handler runs runs in the the same thread as the Initiation Initiation Dispatcher Dispatcher that demultiplexes connection requests for the Acceptor factories. ries. Note how changes to these concurrency concurrency strate strategies gies do not affect the implementation of the Acceptor, which is generic and thus highly flexible and reusable. We start start by definin defining g a Service Handler that uses SOCK SOCK Stream Stream for socket-based data transfer:

// Receiv Receive e and proce process ss bulk bulk data data from from Gatew Gateway ay. . virtual virtual int handle_e handle_event vent (void) (void) { char buf[MAX_BULK_DATA] buf[MAX_BULK_DATA]; ; stream_. stream_.recv recv (buf, (buf, sizeof sizeof buf); buf); // ... ... } // ... ... };

The Command Handler class processes bulk data sent to and received from a Gateway: class class Command_ Command_Hand Handler ler : public public PEER_HAN PEER_HANDLER DLER { public: // Performs Performs handler handler activati activation. on. virtua virtual l int open (void (void) ) { // Handle Handler r runs runs in same same thread thread as main main // Initiation_Dispatc Initiation_Dispatcher her singleton. Initiation_Dispatcher::instance ()->register_handle ()->register_handler r (this, READ_MASK); READ_MASK);

typedef typedef Service_Ha Service_Handler ndler am> PEER_HANDLER;

HANDLER typedef forms the basis for all the The PEER HANDLER subseq subsequen uentt service service handles. handles. For instanc instance, e, the Status

12

Gateway. When When connecti connections ons arri arrive, ve, the Initiation Dispatcher calls calls back back to the approp appropria riate te Acceptor, which creates the appropriate PEER HANDLER to perform

} // Receive Receive and process process command command data from Gateway. Gateway. virtual virtual int handle_e handle_event vent (void) { char buf[MAX_COMMAND_DAT buf[MAX_COMMAND_DATA]; A]; // This This method method cannot cannot block block since since it borro borrows ws // the thread thread of contr control ol from from the the // Initiation_Dispatch Initiation_Dispatcher. er. stream_. stream_.recv recv (buf, (buf, sizeof sizeof buf); buf); // ... ... }

the service, accepts the connection into the handler, and activates tivates the handler. Figure 10 illustrates the relationship between Concrete Acceptor components in the Peer after four connections have have been been establ establis ished hed with with the Gateway show shown n in Figu Figure re 12 and four Service Handler have been created and activated. tivated. While While the Concrete Service Handlers ex-

//... };

Accept Acceptors ors for cre creati ating ng Peer Peer Ser Servic vicee Handle Handlers: rs: The

: Command

s acceptor, bd acceptor, and c acceptor objects shown below are Concrete Acceptor factory instances that create and activate Status Handlers Handlers, Bulk Data Handlers, and Command Handlers, respectively.

: Acceptor

: Status : Status : Service Handler

: Bulk Data

: Service Handler

: Service Handler

: Acceptor // Accept Accept connecti connection on requests requests from Gateway Gateway and // activate Status_Handler. Status_Handler. Acceptor SOCK_Acceptor> s_acceptor; s_acceptor;

: Status

ACTIVE  PASSIVE  PASSIVE 

: Acceptor

// Accept Accept connecti connection on requests requests from Gateway Gateway and // activate Bulk_Data_Handler. Bulk_Data_Handler. Acceptor SOCK_Acceptor> bd_acceptor; bd_acceptor; // Accept Accept connecti connection on requests requests from Gateway Gateway and // activate Command_Handler. Command_Handler. Acceptor SOCK_Acceptor> c_acceptor; c_acceptor;

Note how the use of templates and dynamic binding permits specific specific details details to change change flexibly. flexibly. In particular particular,, no Acceptor component component changes changes when the concurrency concurrency stratstrategy is modified throughout this section. The reason for this flexibility is that the concurrency strategies have been factored out into the Service Handlers, rather than coupled coupled with the Acceptors.

: Command

HANDLERS 

: Bulk Data

LISTENERS 

: Initiation Dispatcher

: Service Handler

Figure 10: Object Diagram for Acceptor Components in the Peer change data with with the Gateway, the three Acceptors continue to listen for new connections in the main thread.

9.2

Concrete Concrete Components Components for the Gateway Gateway

Figure 11 illustrates how the Concrete Connector and Concrete Concrete Service Handler components components are strucstructured in a hypothetical configuration of the Gateway. The Connector compon component entss in this this figure figure comple complemen mentt the Acceptor components in Figure 9.

The Peer Peer main function: function: The main main progra program m initia initializ lizes es the concrete Acceptor factories by calling their open hooks with with the TCP ports ports for for each each servi service ce.. Each Each Acceptor factory factory automatic automatically ally registers registers itself itself with an instance instance of  Initiation Dispatcher Dispatcher in its open method, as the Initiation shown in Section 8.2.

classes Servicee Handle Servic Handlers rs for for Gatewa Gateway y routin routing: g: The classes shown below, Status Status Router Router, Bulk Bulk Data Data Rout Router er, Command Router Router, rout and Command routee data data they they rece receiv ivee from from a sour source ce Peer to one one or more more dest destin inat atio ion n Peers. Concrete e Service Service Handler Handler classes inSince these Concret herit from Service Handler they can be actively connected and initialized by a Connector. To illustrate the flexibility of the Acceptor-Connector pattern, each open routine in a Service Handler implements a different concurrency strategy. strategy. In particular, when the Status Status Router Router is activ activate ated d it runs runs in a separa separate te Bulk Data Data Route Router r runs as a separate prothread; the Bulk cess; and the Command Command Router Router runs in the same thread thread as Iniitation Dispatcher Dispatcher that demultiplexes conthe Iniitation nection completion events for the Connector factory. factory. As with the Acceptor, note how changes to these concurrency rency strate strategie giess do not affec affectt the implem implement entati ation on of the

// Main Main progr program am for for the Peer. Peer. int main main (void) (void) { // Initiali Initialize ze acceptor acceptors s with their their // well-kno well-known wn ports. ports. s_acceptor.open s_acceptor.open (INET_Addr (STATUS_PORT)); (STATUS_PORT)); bd_acceptor.open bd_acceptor.open (INET_Addr (INET_Addr (BULK_DATA_PORT)); (BULK_DATA_PORT)); c_acceptor.open c_acceptor.open (INET_Addr (COMMAND_PORT)); (COMMAND_PORT)); // Event Event loop that handles handles connecti connection on request request // event events s and proce processe sses s data data from from the Gatewa Gateway. y. for (;;) (;;) Initiation_Dispatcher::instance ()->handle_events ()->handle_events (); }

Once Once the Acceptors are are init initia iali lize zed, d, the the main main proprogram gram ente enters rs an even eventt loop loop that that uses uses the the Initiation Dispatcher to dete detect ct conn connec ecti tion on requ reques ests ts from from the the

13

Command n Router      N      O      I      T     R      E      A      Y      C      I     A      L     L      P      P      A

1 ACTIVATES 

Bulk Data n Router

ACTIVATES 

n

Status_Router SOCK_Connector

1

ACTIVATES 

Status Connector

PEER_STREAM

SERVICE_HANDLER  PEER_CONNECTOR 

Service Handler

Figure Figure 11:

class class Bulk_Dat Bulk_Data_Ro a_Router uter : public public PEER_ROU PEER_ROUTER TER { public: // Activate Activates s router router in separate separate process. process. virtua virtual l int open (void (void) ) { if (for (fork k () == 0) // In chil child d proc proces ess. s. // This This metho method d can block block since since it // runs runs in its its own process process. . while while (handle_ (handle_even event t () != -1) continue; // ... ... }

Bulk Data Connector

1

SOCK_Stream

     E      V      I     R      T     E      Y      C      A      A      E     L      R

Command Connector Bulk_Data_Router SOCK_Connector

SOCK_Stream

Status Router      N      O      I      T     R      E      C      E     Y      A      N      L      N      O      C

The PEER ROUTER can be subclassed to produce concrete service service handlers for routing routing bulk data and commands. commands. For Bulk Data Data Router Router routes bulk data to or instance, the Bulk from Peers:

Command_Router SOCK_Connector

SOCK_Stream

Connector

Event Handler n

1

// Receiv Receive e and route route bulk bulk data data from/t from/to o Peers. Peers. virtual virtual int handle_e handle_event vent (void) (void) { char buf[MAX_BULK_DATA] buf[MAX_BULK_DATA]; ; peer_stream_.recv peer_stream_.recv (buf, sizeof buf); // Routing Routing takes takes place place here... here... }

Initiation Dispatcher

};

The Command Router class routes Command data to or from Peers:

Struct Structure ure of Connec Connector tor Partici Participan pants ts for the

Gateway

class class Command_ Command_Rout Router er : public public PEER_ROU PEER_ROUTER TER { public: // Activa Activates tes router router in same same threa thread d as Conne Connect ctor. or. virtua virtual l int open (void (void) ) { Initiation_Dispatcher::instance ()->register_handle ()->register_handler r (this, READ_MASK); READ_MASK); }

Connector, which is generic and thus highly highly flexible and

reusable. We start by defining a Service Handler that is specialized for socket-based data transfer: typedef typedef Service_Ha Service_Handler ndler am> PEER_ROUTER;

// Receive Receive and route route command command data from/to from/to Peers. virtual virtual int handle_e handle_event vent (void) (void) { char buf[MAX_COMMAND_DA buf[MAX_COMMAND_DATA]; TA]; // This This metho method d canno cannot t block block since since it borrow borrows s the // thread thread of control control from the Initiati Initiation_D on_Dispa ispatche tcher. r. peer_stream_.recv peer_stream_.recv (buf, sizeof buf); // Routing Routing takes takes place place here... here... } };

This class forms the basis for all the subsequent routing serStatus Router Router class routes stavices. For instance, the Status tus data to or from Peers: class class Status_R Status_Route outer r : public public PEER_ROU PEER_ROUTER TER { public: // Activate Activate router in separate separate thread. virtua virtual l int open open (void) (void) { // Thread Thread::s ::spa pawn wn requi requires res a pointe pointer r to a // stati static c method method as the threa thread d entry point) point). . Thread::spawn Thread::spawn (&Status_Router::ser (&Status_Router::service_run, vice_run, this); }

A Connector for creating Peer Service Handlers: The following typedef defines a Connector factory specialized for PEER ROUTERS ROUTERS: typedef Connector SOCK_Connector> PEER_CONNECTOR;

// Stati Static c entry entry point point into into threa thread, d, which which blocks blocks // on the the handle handle_ev _event ent() () call call in its own own thread thread. . static static void *service *service_run _run (Status_ (Status_Rout Router er *this_) *this_) { // This This method method can block block since since it // runs runs in its its own own thre thread ad. . while while (this_-> (this_->hand handle_e le_event vent () != -1) continue; }

Unlike the Concrete Acceptor components, we only require a single Concrete Concrete Connector Connector. The reaso reason n for this is that each Concrete Acceptor is used as a factory tory to crea create te a spec specifi ificc type type of  Concrete Service Handler, such as a Bulk Bulk Data Data Handl Handler er or a Command Handler. Therefore, Therefore, the complete complete type must be known a  priori, which necessitates multiple Concrete Acceptor types. types. In contras contrast, t, the Concrete Service Handlers passed to the Connector’s connect method are initialized externally. Therefore, they can be treated uniformly as PEER ROUTERS.

// Recei Receive ve and and route route status status data data from/t from/to o Peers. Peers. virtual virtual int handle_e handle_event vent (void) { char buf[MAX_STATUS_DATA buf[MAX_STATUS_DATA]; ]; peer_str peer_stream_ eam_.rec .recv v (buf, (buf, sizeof sizeof buf); buf); // Routing Routing takes place place here... here... } // ... ... };

14

The Gateway main function:

The main program for the

Gateway is shown shown below. below. The get peer addrs funcBulk Data Data, and tion tion create createss the Status, Bulk and Command Routers that route messages through the Gateway. This

: Bulk Data : Command

function (whose implementation is not shown) reads a list of  Peer addresses from a configuration file or naming service. Each Peer addres addresss consi consists sts of an IP addre address ss and a port port numnumber. ber. Once Once the the Routers are initialized, the Connector factorie factoriess defined above initiate all the connection connectionss asynchronously chronously by passing passing the ASYNC flag to the connect method.

: Service Handler

: Status

: Service Handler : Status

: Status : Service

: Status

Handler

: Service Handler

: Service Handler

: Service Handler

: Command : Service Handler

P EN EN DI NG  CONNECTIONS  ACTIVE  HANDLERS 

// Main Main progr program am for for the Gatewa Gateway. y.

: Connector

// Obtain Obtain an STL vector of Status_R Status_Route outers, rs, // Bulk_Data_Routers, Bulk_Data_Routers, and Command_Routers Command_Routers // from from a config config file. file.

: Initiation Dispatcher

: Bulk Data : Service Handler

void get_peer_addrs get_peer_addrs (vector S> &peers);

Figure Figure 12: Object Object Diagra Diagram m for Connec Connector tor Compon Component entss in the Gateway

int main main (void) (void) { // Connecti Connection on factory factory for PEER_ROU PEER_ROUTERS TERS. . PEER_CONNECTOR PEER_CONNECTOR peer_connector; peer_connector;

established, the Gateway will route and forward messages sent to it by Peers.

// A vector vector of PEER_R PEER_ROUT OUTERs ERs that that perfor perform m // the Gateway’ Gateway’s s routing routing services services. . vector vector peers; // Get vector vector of Peers Peers to connec connect t with. with. get_peer_addrs get_peer_addrs (peers);

10

// Itera Iterate te throu through gh all all the Route Routers rs and // initiate connections asynchronously. asynchronously. for (vector< (vector TER>::it ::iterat erator or i = peers.be peers.begin gin

Know Kn own n Uses Uses

The Acceptor-Connector pattern has been used in a wide ();range of frameworks, toolkits, and systems:

i != peers. peers.end end (); i++) i++) { PEER_ROU PEER_ROUTER TER &peer &peer = *i; peer_connector.conn peer_connector.connect ect (peer, peer.remote_addr peer.remote_addr (), PEER_CONNECTOR::ASYNC);

such as inetd [13], UNIX network network superserv superservers: ers: such listen [14], Configurator dae[14], and the Service Configurator mon mon from from the the ASX framewo framework rk [6]. These superserv superservers ers utilize utilize a master Acceptor process that listens for connections on a set of communication communication ports. ports. Each port is associate associated d with a communicat communication-r ion-relat elated ed service service (such as the standard standard Internet Internet services ftp, telnet, daytime, and echo). The AccepAcceptor process decouples the functionality of the inetd superserver into two separate parts: one for establishing connections and another for receiving and processing requests from peers. peers. When a service service request arrives arrives on a monitored monitored port, the Acceptor process accepts the request and dispatches an appropriate pre-registered handler to perform the service.

} // Loop forever forever handling handling connecti connection on completi completion on // event events s and routi routing ng data data from from Peers Peers. . for (;;) (;;) Initiation_Dispatcher::instance ()->handle_events ()->handle_events (); /* NOTREACH NOTREACHED ED */ }

All connections connections are invoked invoked asynchronousl asynchronously. y. They complet pletee conc concur urre rent ntly ly via via the the Connector’s complete meth method od,, whic which h is call called ed back back with within in the the even eventt loop loop of the the Initiation Initiation Dispatcher Dispatcher. Thi This even eventt loo loop p also also demu demult ltip iple lexe xess and and disp dispat atch ches es rout routin ing g even events ts for for Command Router obje object cts, s, whic which h run run in the the Initiation Dispatcher’s thread of control. The Status Routers and Bulk Data Routers execute in separate threads and processes, respectively. respectively. Figure Figure 12 illustrat illustrates es the relations relationship hip between between compocomponents nents in the Gateway after four connections connections have been establis established hed with the Peer shown shown in Figure Figure 10 and four Concrete Service Handlers have been created and activate activated. d. This diagram illustra illustrates tes four connections connections to another Peer that have not yet completed are “owned” by the Connector. When When all all Peer connections are completely

CORBA ORBs: The ORB Core layer in many implementations of CORBA [16] use the Acceptor-Connector to passively initialize server object implementations when clients request request ORB services. services. [17] describes describes how how the AcceptorAcceptorConnec Connector tor pattern pattern is used used to implem implement ent The ACE ACE ORB (TA (TAO) [18], which is a real-tim real-timee implement implementation ation of CORBA. CORBA. WWW Browser Browsers: s: The HTML parsin parsing g compon component entss in WWW browsers like Netscape and Internet Explorer use the asynchronous version of the connector component to establish connections with servers associated with images embedded in HTML pages. This behavior is particularly important tant so that that multip multiple le HTTP HTTP connec connectio tions ns can be initia initiated ted asynasynchronously to avoid blocking the browsers main event loop.

15

Ericsson Ericsson EOS Call Center Management Management System: this system system uses uses the Accept Acceptor or-Co -Conne nnecto ctorr patter pattern n to allow allow application-level Call Center Manager Event Servers [19] to actively establish connections with passive Supervisors in a distributed center management system.

Efficiently utilize the inherent parallelism in the network shown and hosts: By using the asynchronous mechanisms shown in Figure 8, the Connector pattern can actively establish connections with a large number of peers efficiently over longlatency latency WANs. WANs. This is an important important property property since a large distributed system may have several hundred Peers connected to a single Gateway. One way to connect connect all these Peers to the Gateway is to use the synchronous mechanisms shown shown in Figure 7. However, the round trip delay for a 3-way TCP connection handshake over a long-latency WAN (such as a geosynchronous satellite or trans-atlantic trans-atlantic fiber cable) may take several several seconds per handshake. handshake. In this case, synchronous connection mechanisms cause unnecessary delays since the inherent parallelism of the network and computers is underutilized.

Project Spectrum: The high-speed medical image transfer subsystem of project Spectrum [20] uses the AcceptorConnector pattern to passively passively establish connections and initialize application services for storing large medical images. Once connections connections are establis established, hed, applicati applications ons then send send and receive multi-megabyte medical images to and from these image stores. ACE Framewor Framework: k: Implem Implement entati ations ons of the Service Handler, Connector, and Acceptor classes described in this this paper paper are provid provided ed as reusab reusable le compon component entss in the ACE ACE object-oriented network programming framework [6].

11 11.1 11.1

11.2 11.2

Draw Drawba back ckss

The Acceptor-C Acceptor-Connec onnector tor pattern has the followin following g drawbacks:

Cons Conseq eque uenc nces es

Additional indirection: The Acceptor-Connector pattern can incur additional indirection compared with using the underlying derlying network network programmi programming ng interfaces interfaces directly directly.. However However,, languages that support parameterized types (such as C++, Ada, or Eiffel), can implement these patterns with no significant overhead since compilers can inline the method calls used to implement these patterns.

Bene Be nefit fitss

The Acceptor-Conn Acceptor-Connector ector pattern pattern provides the following following benefits: Enhances the reusability, portability, and extensibility of  connection-oriented software by decoupling mechanisms for initializi initializing ng services services from subsequent subsequent service service processprocessing. For instance, the application-independent application-independent mechanisms mechanisms in the Acceptor and Connector are reusable components that know how to (1) establish connections passively and actively actively,, respecti respectively vely and (2) initiali initialize ze the associate associated d Service Handler once the connection is established. established. In contrast, the Service Handler knows how to perform application-specific application-specific service processing. This separation separation of concerns concerns is achieved achieved by decoupling decoupling the initialization strategy from the service handling strategy. Thus, each strategy can evolve independently. The strategy for active initialization can be written once, placed into a class library or framework, and reused via inheritance, ob  ject composition, or template template instantiation. Thus, the same passive passive initializa initialization tion code need not be rewritten rewritten for each applicatio application. n. Services, Services, in contrast, contrast, may vary according according to differen differentt applicatio application n requiremen requirements. ts. By parameteri parameterizing zing the Acceptor and Connector with with a Service Handler, the impact of this variation is localized to a small number of  components in the software.

unnecesAdditional complexity: This pattern may add unnecessary complexity for simple client applications that connect with a single server and perform a single service using a single network programming interface.

12

See Also

The Acceptor-Connector pattern use the Template Method and and Fact Factor ory y Meth Method od patte pattern rnss [10]. [10]. The The Acceptor’s accept and the Connector’s connect and complete functions are Template Methods that implements a generic service strategy for connecting to remote peers and initializing a Service Handler when the connection is established. The use of the Template Method pattern allows subclasses to modify the specific details of creating, connecting, and activating Concrete Service Handlers. The Factory Method pattern is used to decouple the creation of a Service Service Handler Handler from its subsequent use. The AcceptorAcceptor-Conne Connector ctor pattern pattern has an intent intent similar similar to the Client-Di Client-Dispatc spatcherher-Serv Server er pattern pattern described described in [21]. [21]. They both are concerned concerned with separating separating active active connecconnection establis establishment hment from the subsequent subsequent service. service. The primary difference is that the Acceptor-Connector pattern addresses passive and active service initialization for both synchronous chronous and asynchronou asynchronouss connections connections,, whereas whereas the ClientClientDispatcher-Server pattern focuses on synchronous connection establishment.

Application on robustrobustImproves Improves applicatio application n robustne robustness: ss: Applicati ness ness is improv improved ed by stron strongly gly decoup decouplin ling g the Service Handler from from the Acceptor. This This deco decoup upli ling ng enensures sures that that the passi passiveve-mod modee transp transport ort endpoi endpoint nt factor factory y peer acceptor cannot accidentally be used to read or write data. data. This eliminat eliminates es a common class class of errors that can arise when programming with weakly typed network programming interfaces such as sockets or TLI [11].

16

Acknowledgements

[18] D. C. Schm Schmidt, idt, D. L. Levine Levine,, and and S. Mungee Mungee,, “The “The Design Design and Performance of Real-Time Object Request Brokers,” Com puter Communications, vol. 21, pp. 294–324, Apr. 1998.

Thanks to Frank Buschmann and Hans Rohnert for their helpful comments on this paper.

[19] D. C. Schmidt and T. Suda, “An Object-Oriented Object-Oriented Framework  Framework  for Dynamically Configuring Extensible Distributed Communication Systems,” IEE/BCS Distributed Systems Engineering  Journal  Journal (Special Issue on Configurable Configurable Distributed Systems) , vol. 2, pp. 280–293, December 1994.

References [1] W. R. Stevens, Stevens, TCP/IP Illustrated, Volume 1 . Reading, Reading, Massachusetts: Addison Wesley, 1993.

[20] G. Blaine, M. Boyd, Boyd, and S. Crider, “Project Spectrum: Spectrum: Scalable Bandwidth for the BJC Health System,” HIMSS, Health Care Communications , pp. 71–81, 1994.

Object Oriente Oriented d Analysi Analysiss and Design with Ap[2] G. Booch, Booch, Object   plications plications ( Edition) . Redwoo Redwood d City, City, California California:: BenBen  jamin/Cummings, 1993.

[21] F. Buschmann, Buschmann, R. Meunier, H. Rohnert, P. Sommerlad, and M. Stal, Pattern-Oriented Software Architecture - A System of  Patterns . Wiley and Sons, 1996.

[3] D. C. Schmidt, Schmidt, “Reactor: “Reactor: An Object Object Behavioral Behavioral Pattern Pattern for Concurrent Concurrent Event Demultiplexing Demultiplexing and Event Handler Handler DisProgram Design (J. O. patching, patching,”” in Pattern Languages of Program Coplien and D. C. Schmidt, eds.), pp. 529 529–545, –545, Reading, Reading, MA: Addison-Wesley, Addison-Wesley, 1995. [4] T. Harrison, Harrison, I. Pyarali, D. C. Schmidt, and T. Jordan, Jordan, “Proactor – An Object Object Behavio Behavioral ral Pattern Pattern for Dispatch Dispatching ing AsynAsynchronous Event Handlers,” in The Pattern Languages of  Programmin Programming g Conference Conference (Washingt (Washington on University University technical technical report #WUCS-97-34) , September September 1997. [5] R. G. Lavender Lavender and D. C. Schmidt, “Active “Active Object: an Object Behavioral Pattern for Concurrent Programming,” in Pattern  Languages  Languages of Program Program Design (J. O. Coplien, J. Vlissides, and N. Kerth, eds.), Reading, MA: Addison-Wesley, 1996. [6] D. C. Schmidt Schmidt,, “ACE: “ACE: an ObjectObject-Orie Oriented nted Framewor Framework k for Developing Distributed Applications,” in Proceedings of the USENIX C++ Technical Conference , (Cambridge, Massachusetts), USENIX Association, April 1994. [7] W. Pree, Design Patterns for Object-Oriented Software Development . Reading, MA: Addison-Wesley, 1994. [8] W. R. Stevens, Stevens, UNIX Network Programming, Second Edition . Englewood Cliffs, NJ: Prentice Hall, 1997. [9] H. Custer Custer,, Inside Windows NT . Redmond, Redmond, Washing Washington: ton: Microsoft Press, 1993. [10] E. Gamma, Gamma, R. Helm, Helm, R. Johnso Johnson, n, and and J. Vlisside Vlissides, s, Design PatPatterns: Elements of Reusable Object-Oriented Software . Reading, MA: Addison-Wesley, 1995. [11] D. C. Schmidt, Schmidt, T. H. Harriso Harrison, n, and E. Al-Shae Al-Shaer, r, “Object“ObjectOriente Oriented d Compon Components ents for High-sp High-speed eed Networ Network k ProgramProgramming,” ming,” in Procee Proceedin dings gs of the Conference Conference on ObjectOriented Oriented Technol echnologi ogies es and Systems Systems, (Mon (Monte tere rey y, CA), CA), USENIX, June 1995. [12] D. C. Schmidt, Schmidt, “IPC SAP: An Object-Ori Object-Oriente ented d Interfac Interfacee to Interprocess Interprocess Communication Communication Services,” Services,” C++ Report , vol. 4, November/December 1992. [13] W. R. Stevens, UNIX Network Programming, First Edition . Englewood Cliffs, NJ: Prentice Hall, 1990. [14] S. Rago, Rago, UNIX System V Network Programming . Readin Reading, g, MA: Addison-Wesley, 1993. [15] D. L. Presotto and D. M. Ritchie, Ritchie, “Interprocess “Interprocess Communication in the Ninth Edition UNIX System,” UNIX Research System Papers, Tenth Edition , vol. 2, no. 8, pp. 523–530, 1990. Common Object Object Request  Request  [16] Object Management Management Group, The Common  Broker: Architecture and Specification, 2.0 ed., July 1995.

[17] D. C. Schmidt Schmidt and C. Cleeland, Cleeland, “Applyin Applying g Patterns Patterns to Develop Extensible ORB Middleware,” Submitted to the IEEE  Communications Magazine , 1998.

17