Wednesday May 23, 2007

Recent optimization in propagated pipes

There has been many performance enhancements introduced in the upcoming release of JXTA for JSE 2.5, which I will try to cover in a series of blogs detailing the enhancements and the expected benefits. Starting with the most recent, the propagated pipe.  To give a little background on how propagated pipes work, a propagated pipe utilizes a pipe service named WirePipe which deals with duplicate message filtering and TTL on the network.  The WirePipe service then relies on the rendezvous service for network topology and message distribution.   A less frequently publicized feature of propagated pipes, is the ability to specify a set of recipients, which eventually end up as individual messengers (1-1 connections) to recipients.  The performance enhancement was introduced as a new class, BlockingWireOuputPipe, in 2.5, which is automatically constructed when the specified set of nodes contains a singleton. BlockingWireOuputPipe then binds directly the destinations propagated pipe endpoint bypassing two layers of indirection (WirePipe and the RendezvousService), thus eliminating two message elements (wire, and propagation header),  resulting in smaller message overhead, and faster delivery, also since it is bound directly to the endpoint, it provides reliable feedback on message delivery.

It is highly recommended that such object is cached and reused, for an example, see the PropagatedPipeServer tutorial. 

 

 


Saturday Jun 24, 2006

NetworkConfigurator. A programmatic interface to JXTA configuration

If you are familiar with JXTA configuration, you will surely appreciate the new bug fixes, and features added to the latest JXTA platform release (2.4).

The latest release included significant fixes and features to deal with node configuration:
  1. ext:config included several bug fixes, and improved stability, it also included additional configuration profiles.
  2. A programmatic interface to node configuration (see NetworkConfigurator), by design, NetworkConfigurator is a JavaBean, allowing for declarative configuration deployment (see jxta-spring).

    In addition NetworkConfigurator exposes API's (see load(uri), getPlatformConfig(), etc.), which would facilitate secure network based deployment services, as well as automate certificate, ID, and rendezvous/relay seed provisioning.

    Special thanks to the community members who contributed and helped drive these new API's (especially vwilliams, ae6rt, bondolo, fdimglio, ibirrer)
Network Based Configuration Provisioning Network Based Configuration Provisioning
 
   NetworkConfigurator config = new NetworkConfigurator();
if (!config.exists()) {
// Create a new configuration with a new name, principal, and pass
config.setName("New Name");
config.setPrincipal("username");
config.setPassword("password");
try {
//persist it
config.save();
} catch (IOException io) {
// deal with the io error
}
} else {
// Load the pre-existing configuration
File pc = new File(config.getHome(), "PlatformConfig");
try {
config.load(pc.toURI());
// make changes if so desired
..
..
// store the PlatformConfig under the default home
config.save();
} catch (CertificateException ce) {
// In case the root cert is invalid, this creates a new one
try {
//principal
config.setPrincipal("principal");
//password to encrypt private key with
config.setPassword("password");
config.save();
} catch (Exception e) {
e.printStackTrace();
}
}

Thursday May 25, 2006

JxtaSocket vs. Java Socket

If you have wondered about JxtaSocket performance vs. Java Socket then this maybe of interest. I have always wondered about how JxtaSockets stack up to Java Sockets (there has some research done on this topic, but it's not the same unless you can see for yourself). So, I have decided to write a benchmark comparing the two. The benchmark included two machines connected through a 100Mbit switch, and JSE5.0. Each benchmark run included a data transfer of 65MB, which was repeated 3 times. The chart shows the lowest and highest rates measured. You should also note, that the transfer unit size, and the VM min heap size had an effect on both benchmarks.JxtaSocket vs. Java Socket Benchmarks.

P1050862

 

Wednesday Jan 04, 2006

JXTA's newest member of the socket family "JxtaDatagramSocket"

In the spirit of providing JXTA interfaces a la java.net.Socket, I would like to introduce you to the newest member to the net.jxta.socket family "JxtaDataDramSocket", which can be found under http://platform.jxta.org/issues/show_bug.cgi?id=1470 Please provide comments and feedback under the issue described above.

Thursday Oct 20, 2005

Are you interested in a UDP transport for JXTA?

If you are interested in a JXTA UDP transport. an early copy of the transport is available under http://platform.jxta.org/issues/show_bug.cgi?id=1448 It would be interesting to benchmark JXTA's tcp transport against the new UDP transport.

Thursday Sep 15, 2005

JXTA for J2ME JXME 2.1 (Tantà) Screen Shots

I captured the following images during the JXME 2.1 functional test against the JXTA public network.
        JXME 2.1 HelloWorld JXME 2.1 HelloWorld

        JXME 2.1 group chat session within MyJXTA's sub-group JXME 2.1 group chat session within MyJXTA's sub-group

Wednesday Sep 14, 2005

JXME 2.1 (Tantà) on an iPAQ 3600

I had promised to provide detailed step-by-step instructions on installing and running JXME on an iPAQ. So here goes. Hardware
  • iPAQ 3600
    • 64MB RAM
    • 32MB flash
    • Orinoco 802.11B
build and install JXME 2.1
  • set optimize on, and debug off
  • build against log4j_stub
  • copy jxme-cdc.jar, and log4j_stub.jar to the device
Installing handhelds.org familiar
  • Download and install handhelds familiar opie release v0.8.2 image for your device
  • Follow the excellent installation instructions on handhelds.org
  • Once booted
  • Install libfloat
  • ipkg install libfloat_1.0_arm.ipkg
  • cd /usr/lib
  • ln -s /home/QtPalmtop/lib/libfoat.so .
  • install cvm 1.1
    • cp -rf cvm /tmp (or any directory with available space)
Running the demos
    GroupDemo
    • cvm -classpath \\ jxme-cdc.jar:log4j_stub.jar GroupDemo
    ChatDemo
    • cvm -classpath \\ jxme-cdc.jar:log4j_stub.jar ChatDemo

Tuesday Aug 30, 2005

JXTASink. Peer emulation, and load generation

JxtaSink and companion utilities are some of the least publicized tools available underneath the JXTA platform hood, however don't underestimate their power. JxtaSink emulates the JXTA endpoint protocol, and provides API for message callback, or it can act as a message drain. Such features are beneficial when trying simulate a network of 1000's nodes on a single machine, and in determining breaking, or tuning points. Using JxtaSink requires knowledge of the JXTA protocols and behavior. One test developed which makes use of these utilities is the RendezvousLoad. RendezvousLoad emulates the rendezvous connection protocol and can scale to 1000's peers (on certain platforms). This load test was beneficial in exposing unforeseen OS limitation, as well as bottlenecks. Similar load tests can be written using the forementioned utilities, provided good understanding of protocol of interest. JxtaSink resides in the platform project under platform/binding/java/test/src/net/jxta/test/util

Wednesday Aug 24, 2005

Demystifying Pipes, JxtaSockets, JxtaMulticastSocket, and JxtaBiDiPipes

Lately there has been several inquiries about JXTA's PipeService, and companion utilities (JxtaSocket, JxtaMulticastSocket, and JxtaBiDiPipe) on JXTA's discussion lists, hence this blog to shed more light on the PipeService and utilities provided, and their inherit features.
The initial goal of the PipeService was to provide primitive (unidirectional, and unreliable) message based communication channels upon which complex and sophisticated communication channels can be built. The PipeService provides three types of pipes described as follows:
JxtaUnicast
  • Unidirectional
  • Unreliable
  • Many to one
  • Limited to 64KB messages
JxtaUnicastSecure
  • Unidirectional
  • Limited reliability (no direct notification of failures other than channel failure)
  • Many to one
  • Limited to 64KB messages
JxtaPropagate
  • Multidirectional (many to many)
  • Unreliable
  • Limited to 64KB messages in infrastructure mode, and limited by physical transport limitation where applicable
JXTA JSE platform software stack JXTA JSE platform software stack
Unicast pipes (clear, and secure) are bound by the PipeResolver, which provides two modes of bindings, a static (where a set of peers is specified), and a dynamic (no peers specified) binding. Static binding is beneficial in constraining communication to a limited set of peers, while dynamic is beneficial in implementing unconstrained mobile communication channels. Both modes can be utilized in implementing fault tolerant communication channels.
Propagated pipes employ a mechanism similar to that of IGMP, where a creation of an input pipe results in propagated pipe channel join message (a pipe SRDI message to a the peer's rendezvous). In infrastructure mode propagated pipe messages are sent to the rendezvous for propagation to the subscribers of the propagated pipe channel, while in ad-hoc mode messages are propagated over the endpoint (currently only the TCP transport supports such function over multicast). A pipe closure results in a channel resign message (a pipe SRDI message to the rendezvous expiring the previous join). It's worth noting that the PipeResolver plays no part in binding of such pipe type
Pipe Binding An InputPipe creation results in the creation of an endpoint incoming messenger, and an SRDI message with the pipe ID with an infinite expiration time. Once the InputPipe is closed another SRDI message is emitted with the pipe ID, and expiration time of 0, invalidating any reference for the pipe and peer. note: This behavior applies to all pipe types. An output pipe creation results in a PipeResolver query message emitted querying for instances of the pipe. A match results in a PipeResolver response to the querying peer, where an endpoint messenger to responding peer is created. Note that such messenger may not be fully resolved until the EndpointRouter has determined a valid route. During such event few messages maybe queued (3 to be exact) until the messenger has been resolved. In addition, as part of the pipe resolver protocol the pipe resolver periodically reattempts pipe resolutions in the background to validate resolved pipe endpoints. It is through this mechanisms pipes can migrate from one peer to another. note: this behavior is limited to unicast pipe types.
SRDI message path within the JXTA network SRDI message path within the JXTA network
Pipe Endpoint A pipe endpoint is composed of a peer, group, and pipe ID, through this definition there are two inherit modes of pipe endpoint migration:
  1. Peer endpoint migration, an inherit feature of the Endpoint-Router, where a peer may change physical, or virtual addresses.
  2. Pipe endpoint migration, where a pipe endpoint ceases to exists on a peer, where the PipeResolver, attempts to rebind to the pipe (which I always refer to as a virtual channel)
The first mode is transparent the application, while the second may result in an IOException. Reliability A standalone message and stream based reliability layer over non reliable messengers, is provided in the platform to facilitate reliable communication channels. Reliability Library
  • Ensures message sequencing
  • Interfaces to pipes or messengers
  • Ensures delivery
  • Exposes message, and stream interfaces
  • Spawns a single thread to deal with retransmission
  • Does not ensure message integrity
Bidirectional communication channels JxtaServerSocket, and JxtaServerPipe expose a input pipe to process connection requests and negotiate communication parameters, whereby a JxtaSocket, or JxtaBiDipipe bind to respectively to establish private dedicated pipes independent of the connection request pipe. JxtaSocket, and JxtaBiDiPipe utilize a messenger instead of an output pipe for the back channel to reduce connection latency (avoids 4 messages), therefore only peer endpoint migration is inherited, and one should always expect an IOException if a PipeEndpoint migrates, or ceases to exist.
JxtaSocket, JxtaServerSocket
  • Subclass java.net.Socket, and java.net.ServerSocket respectively
  • Built on top of pipes, endpoint messengers, and the reliability library
  • Provides bidirectional and reliable communication channels
  • Exposes stream based interface a la Socket
  • Provides configurable internal buffering, and message chunking
  • Does not implement the Nagels algorithm, therefore streams must be flushed as needed (a common oversight when wrapping object and data streams over the socket stream)
JxtaBiDiPipe, JxtaServerPipe
  • Built on top of pipes, endpoint messengers, and the reliability library
  • Provides bidirectional and reliable communication channels
  • Exposes message based interface
  • Requires no heart beat to maintain an open connection
  • Provides no message chunking
Multidirectional communication utility JxtaMulticastSocket extends java.net.MutlicastSocket and provides mutlicast functionality over JXTA propagated pipes. JxtaMulticastSocket
  • Subclass java.net.MulticastSocket
  • Built on top of propagated pipes
  • Exposes datagram based interface a la MulticastSocket
  • Provides unicasting through datagram addressing
Message size and limiting factors
  1. The JXTA platform imposes a soft MTU of 64KB (which becomes a hard limit under load), messages exceeding MTU are simply dropped. While nodes maybe able to transmit and receive messages larger than 64KB, it is strongly advised this limit is not exceeded in reliable mode, as it will result in failed or broken channels.
  2. When utilizing relay peers, such limit must not be exceeded in either mode (reliable or otherwise) as the relay nodes are likely to be loaded and/or the fairness message queue size quotas are enforced, thus leading to message loss and consequently failed or broken channels.
Common pitfalls
  • Pipe advertisement mismatch. Typically occurs when dynamically creating and discovering pipe advertisements using non unique identifiers, or deterministic method. Normally overcome through hardcoded pipe advertisements (or pipe ID's), encoded pipe ID's.
  • Un-synchronized pipe binding. Typically caused by network islands, which are joined at a later point in time. Easily overcome through rendezvous event listener prior to any binding attempt.
  • Failed Secure pipe connections. Typically due to failure of establishing NetPeerGroup credentials (TLS resides in the NetPeerGroup, and inherited by all sub-groups), or missing a trusted certificate or certificate chain in PSE. This is overcome by stablishing NetPeerGroup credential, and ensuring the perspective trusted certificate or certificate chain is stored into PSE (See the shell command pse.importcert).

Saturday Jul 30, 2005

JxtaNetMap gets a new face-lift

JxtaNetMap is a pretty useful network debugging tool, however you would probably agree that the color choices and arrow representation were kind of dated (reminiscent of the 70's). Since I use the tool often I decided to give it a little face-lift. Some of the changes were simple, such as choosing a color scheme, and node text spacing, others were a bit more involved and entailed changing the arrow drawing algorithm. I hope you like the new look. New look 

 

Tuesday Jun 14, 2005

JXME/JSR218 2.0 (Arrakis) stable announced

I am pleased to announce the availability of JXME 2.0 (Arrakis) stable release. This release contains the new binding of JXME proxy-less platform on CDC1.1 (JSR218). A stable version of JXME 2.0 platform is now available. The platform provides edge only functionality with TCP/IP only support, the platform also provides relay transports for peers thar are behind a firewall or NAT. You can read more details about the release at :
http://jxme.jxta.org/release_note_2_0.html

Monday Jun 13, 2005

JXME/JSR218, 1 more day before FCS

Today, I have addressed a bug in the in memory cache used by JXME's discovery service. The bug manifested itself through lack of proper advertisement indexing, rendering the cache unusable. Services which rely on discovery for advertisement caching, had to rely on the network instead to resolve resources, which caused unnecessary network traffic, and slowed down operations.
I can't wait to see how the JXME platform performs with a in memory LRU cache in comparison to a persistent cache. Once the JXME 2.0 platform passes a functional test which includes it's cousins (J2SE platform, and JXTA-C) it can be declared stable.

Wednesday Jun 08, 2005

مرحبا بكم, 欢迎, Welcome, Bienvenue, Bien Venido to JXME/JSR218

Below are snapshots of JXME's ChatDemo on an Ipaq running familiar Linux with Opie 1.2 and CDC 1.1, interacting with a JXTA Shell over a propagate pipe. And yes I will share (sometime soon) the step-by-step details on how to get started.

JXTA Shell window exchanging messages over a propagate pipe JXTA Shell window exchanging messages over a propagate pipe
ChatDemo app in a console window on a 3600 exchanging messages with the above shell over a propagate pipe ChatDemo app in a console window on a 3600 exchanging messages with the above shell over a propagate pipe

Tuesday Jun 07, 2005

Did someone say Alpha?


If you have been tracking progress of the JXME/JSR218 effort, then you should be pleased to know that the JXME platform is close to declaring an alpha milestone. Here are the details of the current state of the platform:
RendezVousService, now fully functional in both AD-HOC, and infrastructure mode. It supports connections to rendezvous peers directly, through a seed list, and local sub-net multicast. This service is key in the operation of most of platform service.
RelayTransport, fully functional. A JXME peer is able to establish relay leases, and operate in a firewalled, and/or NAT'd configuration. ResolverService, verified fully functional. PipeService, fully functional. The PipeService supports unicast and propagated pipes ConfigurationFactory, produces a simple and functional PlatformConfig. Features planned are support for specifying seed address, an interface address, and whatever people scream for. DisocveryService, not yet fully functional. There still remains work to be done, with an in memory CacheManager, advertisement caching is a bit different from that of a persistent store, as some advertisements do not define an ID and those must be dealt with properly, in addition it is simply new code and as expected bugs lurk. ChatDemo, verified fully functional. Interoperable with JXTA shell, instantp2p, and MyJxta2 at the NetPeerGroup level. The app displays messages received on the pipe, and responds with a greeting and a message contains the "jxme" substring. GroupDemo, Same as the ChatDemo, however it creates a MyJxta subgroup and joins it, and operates the those pipes within the new group. So when is alpha you ask. As soon as the DiscoveryService is verified to be fully functional, an alpha will be declared.
How can you help?
-Start developing or verifying your application on the JXME platform. Keep in mind application will just compile, given they are not using any deprecated API's. -Report issues under the JXME project homepage -Post questions, feature and enhancements requests on discuss@jxme.jxta.org
And finally, if you are interested in JxtaBiDiPipe, and JxtaSocket stay tuned for upcoming updates

Wednesday May 25, 2005

JXME/JSR218, Configuration and unit tests

Continuing yesterday's thread, the JXME two additional components :
A PlatformConfigurationFactory. A lean an simple configuration factory (~140 lines of code w/o JavaDoc ~362/w). The factory provides a standard default edge configuration, which the application can augment prior to calling the factory, e.g. :

ConfigurationFactory.setHome(homeDir); Advertisement config = ConfigurationFactory.newPlatformConfig(); ConfigurationFactory.save(config);

The above code snippet overrides the default $JXTA_HOME from $CWD/.jxta to homeDir, a default edge configuration is created, and a call to save stores the PlatformConfig in homeDir.
Unit tests (a must for any project) were also added to validate the newly added components, ans as expected were quite useful in shedding light on lurking bugs. Currently they provided minimum coverage and expected to provide coverage with time.
Now onto platform bring up :)
About

hamada

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today