Demystifying Pipes, JxtaSockets, JxtaMulticastSocket, and JxtaBiDiPipes

Lately there has been several inquiries about JXTA's PipeService, and companion utilities (JxtaSocket, JxtaMulticastSocket, and JxtaBiDiPipe) on JXTA's discussion lists, hence this blog to shed more light on the PipeService and utilities provided, and their inherit features.
The initial goal of the PipeService was to provide primitive (unidirectional, and unreliable) message based communication channels upon which complex and sophisticated communication channels can be built. The PipeService provides three types of pipes described as follows:
  • Unidirectional
  • Unreliable
  • Many to one
  • Limited to 64KB messages
  • Unidirectional
  • Limited reliability (no direct notification of failures other than channel failure)
  • Many to one
  • Limited to 64KB messages
  • Multidirectional (many to many)
  • Unreliable
  • Limited to 64KB messages in infrastructure mode, and limited by physical transport limitation where applicable
JXTA JSE platform software stack JXTA JSE platform software stack
Unicast pipes (clear, and secure) are bound by the PipeResolver, which provides two modes of bindings, a static (where a set of peers is specified), and a dynamic (no peers specified) binding. Static binding is beneficial in constraining communication to a limited set of peers, while dynamic is beneficial in implementing unconstrained mobile communication channels. Both modes can be utilized in implementing fault tolerant communication channels.
Propagated pipes employ a mechanism similar to that of IGMP, where a creation of an input pipe results in propagated pipe channel join message (a pipe SRDI message to a the peer's rendezvous). In infrastructure mode propagated pipe messages are sent to the rendezvous for propagation to the subscribers of the propagated pipe channel, while in ad-hoc mode messages are propagated over the endpoint (currently only the TCP transport supports such function over multicast). A pipe closure results in a channel resign message (a pipe SRDI message to the rendezvous expiring the previous join). It's worth noting that the PipeResolver plays no part in binding of such pipe type
Pipe Binding An InputPipe creation results in the creation of an endpoint incoming messenger, and an SRDI message with the pipe ID with an infinite expiration time. Once the InputPipe is closed another SRDI message is emitted with the pipe ID, and expiration time of 0, invalidating any reference for the pipe and peer. note: This behavior applies to all pipe types. An output pipe creation results in a PipeResolver query message emitted querying for instances of the pipe. A match results in a PipeResolver response to the querying peer, where an endpoint messenger to responding peer is created. Note that such messenger may not be fully resolved until the EndpointRouter has determined a valid route. During such event few messages maybe queued (3 to be exact) until the messenger has been resolved. In addition, as part of the pipe resolver protocol the pipe resolver periodically reattempts pipe resolutions in the background to validate resolved pipe endpoints. It is through this mechanisms pipes can migrate from one peer to another. note: this behavior is limited to unicast pipe types.
SRDI message path within the JXTA network SRDI message path within the JXTA network
Pipe Endpoint A pipe endpoint is composed of a peer, group, and pipe ID, through this definition there are two inherit modes of pipe endpoint migration:
  1. Peer endpoint migration, an inherit feature of the Endpoint-Router, where a peer may change physical, or virtual addresses.
  2. Pipe endpoint migration, where a pipe endpoint ceases to exists on a peer, where the PipeResolver, attempts to rebind to the pipe (which I always refer to as a virtual channel)
The first mode is transparent the application, while the second may result in an IOException. Reliability A standalone message and stream based reliability layer over non reliable messengers, is provided in the platform to facilitate reliable communication channels. Reliability Library
  • Ensures message sequencing
  • Interfaces to pipes or messengers
  • Ensures delivery
  • Exposes message, and stream interfaces
  • Spawns a single thread to deal with retransmission
  • Does not ensure message integrity
Bidirectional communication channels JxtaServerSocket, and JxtaServerPipe expose a input pipe to process connection requests and negotiate communication parameters, whereby a JxtaSocket, or JxtaBiDipipe bind to respectively to establish private dedicated pipes independent of the connection request pipe. JxtaSocket, and JxtaBiDiPipe utilize a messenger instead of an output pipe for the back channel to reduce connection latency (avoids 4 messages), therefore only peer endpoint migration is inherited, and one should always expect an IOException if a PipeEndpoint migrates, or ceases to exist.
JxtaSocket, JxtaServerSocket
  • Subclass, and respectively
  • Built on top of pipes, endpoint messengers, and the reliability library
  • Provides bidirectional and reliable communication channels
  • Exposes stream based interface a la Socket
  • Provides configurable internal buffering, and message chunking
  • Does not implement the Nagels algorithm, therefore streams must be flushed as needed (a common oversight when wrapping object and data streams over the socket stream)
JxtaBiDiPipe, JxtaServerPipe
  • Built on top of pipes, endpoint messengers, and the reliability library
  • Provides bidirectional and reliable communication channels
  • Exposes message based interface
  • Requires no heart beat to maintain an open connection
  • Provides no message chunking
Multidirectional communication utility JxtaMulticastSocket extends and provides mutlicast functionality over JXTA propagated pipes. JxtaMulticastSocket
  • Subclass
  • Built on top of propagated pipes
  • Exposes datagram based interface a la MulticastSocket
  • Provides unicasting through datagram addressing
Message size and limiting factors
  1. The JXTA platform imposes a soft MTU of 64KB (which becomes a hard limit under load), messages exceeding MTU are simply dropped. While nodes maybe able to transmit and receive messages larger than 64KB, it is strongly advised this limit is not exceeded in reliable mode, as it will result in failed or broken channels.
  2. When utilizing relay peers, such limit must not be exceeded in either mode (reliable or otherwise) as the relay nodes are likely to be loaded and/or the fairness message queue size quotas are enforced, thus leading to message loss and consequently failed or broken channels.
Common pitfalls
  • Pipe advertisement mismatch. Typically occurs when dynamically creating and discovering pipe advertisements using non unique identifiers, or deterministic method. Normally overcome through hardcoded pipe advertisements (or pipe ID's), encoded pipe ID's.
  • Un-synchronized pipe binding. Typically caused by network islands, which are joined at a later point in time. Easily overcome through rendezvous event listener prior to any binding attempt.
  • Failed Secure pipe connections. Typically due to failure of establishing NetPeerGroup credentials (TLS resides in the NetPeerGroup, and inherited by all sub-groups), or missing a trusted certificate or certificate chain in PSE. This is overcome by stablishing NetPeerGroup credential, and ensuring the perspective trusted certificate or certificate chain is stored into PSE (See the shell command pse.importcert).

Post a Comment:
Comments are closed for this entry.



« September 2016