In a recent article, economist Tim Harford summarized some of the key ideas in a paper by Paul A. David, "The Dynamo and the Computer: An Historical Perspective on the Modern Productivity Paradox". While I'll further distill it, I really encourage you to read one or both of the pieces.

I want to start with the idea that new technologies, though they may have the potential to massively and positively transform institutions, and although it may even be envisaged how they may do so, nevertheless take far longer to do so than their proponents expect, even in the face of years of high levels of investment in the new technologies. Professor David's paper was written in 1990 as an investigation into the question of why, as the Nobel Laureate economist Robert Solow remarked, "we see computers everywhere but in the productivity statistics". (This problem, the "productivity paradox", persists today; in the last handful of years we have seen some increases in productivity that are thought by some to be attributable to technology investments of the preceding decades.)

David gives us a history of the electric dynamo and the electrification of factories (you really should read the paper; at least read section II) as an analogy to help us think about the possible future of computers. While he talks about some of the specific changes that electrification wrought on factories, he doesn't go into specifics of what computer technology may do and how it might do it. Well, he's an economist writing 17 years ago, and I'm a (hmm...let's go with...) technologist and visionary with a blog today, so I'll be your huckleberry.

Electrification did little to change factories and work practices for decades, and there were many reasons why. The one that that is most analogously relevant is this: The major benefits of electricity didn't come from replacing steam with electricity per se; they came because electricity allowed "unit drive", which could let you run a factory in a cheaper, more efficient, higher-quality way than was possible with "group drive", which was steam engines limited you to. Electricity allowed you to go from a steam Factory 1.0 to an electric Factory 2.0. (Two examples of the differences: 2.0 buildings could be smaller, lighter, single-story structures because shafts and belts could be replaced with wiring, which was both lighter and allowed more flexible placement of powered machinery; and 2.0 factories could be more efficient because machines could be laid out according to flow of materials and tasks, and furthermore easily rearranged according to process changes.)

But even though the first central electric plant in the country had been operating since 1881, it wasn't until the 1920s, four decades later, that factory productivity growth started to improve. What happened? Well, there were all these existing buildings and machines and training and instructions and processes and institutions and relationships and habits that were all based on the steam-powered Factory 1.0. Even when they electrified an old steam factory, and even when they built new electric factories, they would simply replace the steam engine with an electric dynamo, and would end up with an electric Factory 1.0, not a Factory 2.0.

We've done similar with computer technology. Yes, we have email, and email has substituted as a better and cheaper memo and mail and bulletin board and meeting and runner and pneumatic tube. And yes, we have web sites, and they are a lot better than bulletin boards and forms and catalogs and brochures and manuals and calling up a rep to ask a question. But they do the same thing. Yes, better and faster and cheaper and with much greater reach, but it's the same thing. Going further back, pre-networking, computer reporting and printing was more efficient than accountants with (paper) spreadsheets and ledgers and typing and stenography and typesetting, but it was again the same thing. (Now I'm not discounting that at some point, speed and efficiency are quantitatively so much greater that in itself induces qualitative change--email has arguably reached that point--but that's an emergent rather than a contemplated change.)

I say then that Web 2.0 consists of those applications that not only use the underlying technologies of computers and networks and HTTP and HTTP-stuff; but furthermore and more saliently do so in ways that are particularly well-matched to the routines and practice of Org 2.0 organizations. (I know people sometime use "Enterprise 2.0", but I actually want to include organizations that may not be enterprises.) What's Org 2.0? Well, let's talk first about Org 1.0. Org 1.0 has a "traditional" corporate form: driven by standardized operating processes and organized around departments dedicated to performing those processes. Many organizations (or parts of organizations) at least aspire to this form, analyzing and optimizing business processes, doing TQM and ISO9001 and Six Sigma, installing whatever software brings them closer to the ideal. The 1.0 organization has persistent, determined outcomes, expected inputs, and reasonably well-known tools and practices; the main challenge in Org 1.0 is coordinating the inputs and tools and practices to achieve those goals effectively and efficiently. Configuring and ordering a PC on Dell's website, assembling a car, getting the right amount of beer and iPhones to the retail stores, checking out at the supermarket; those are the kinds of things that Org 1.0 is good at doing.

The key is that Org 1.0 is dedicated to repeating its outcomes, perhaps better each time, and with parameterized variations, but repeating. In contrast, Org 2.0 might be doing something new all the time; that something may be similar to what it may have done before, but not in any easily-formalizable way. I made a chart.

Table 1: I need to fancy this post up for you people with short attention spans.

Now, it's often assumed that what I've called Org 1.0 is actually just how you have to run a large organization, to be able to scale up without screwing up (after all, "repeating" is pretty much how a company scales up); and what I'm calling Org 2.0 is characteristic of small organizations, who just don't have the scale or the institutional knowledge to implement structured repeatable processes. Small firms that grow become more structured and more process-oriented: because they can, as they learn from what they do and standardize that into repeatable processes; and because they must, as informal mechanisms of communications break down when there are lots of people, and formal processes can ensure that individuals get the information they need to do their jobs.

That's true, and Org 1.0 represents the pinnacle of industrial achievement. The only way the world economy can work is with organizations dedicated to scaling up, in predictable and efficient and repeatable ways. Much of what Org 2.0 does is as part of or in direct service to a bigger Org 1.0.

But the thing is, in gaining all the scale advantages of Org 1.0, an organization loses the great advantages of Org 2.0: flexibility and adaptability, resulting from the high level of communication and overall situational awareness of each member of the organization. Yes, Org 1.0 realizes economies of scale, but at the finer grain, Org 1.0 is in fact much less efficient than Org 2.0, and the economies of scale have to offset the diseconomies. (That's something a lot of small and mid-sized companies often forget, btw. In the drive to formalize to be able to grow, they smother their own advantages.) That's why a lot of companies that work in Org 2.0 ways (architecture firms, law firms, creative agencies, research labs, medical specialists) are either small, or decentralized collections of small groups. That's why large organizations often break off smaller, less formal groups for innovation.

Ah, and here's where Web 2.0 comes in. Web 2.0 applications are those applications that are meant to help Org 2.0 to scale up, take advantage of specialization, and stop being so small. (Here I'll add something to Gerald's post and say that Web 2.0 applications can also help the already large ones i.e. Org 1.0 reap the benefits of Org 2.0). They help with Org 2.0-specific activities and share those characteristics: informal communications and collaboration, ad hoc inclusion of people and information, structured in ways that are easy for humans to understand (say, by faces, or by event, or by conversation, or by relationship) and synthesize. They do this so that the efficient working habits of a small group can be used by larger groups of people who are farther apart, so that we can get economies of scale, without the heavy diseconomies. They try to do this by improving communications and the sharing of information that works in small localized teams, so that it can work across larger dispersed groups of individuals. The point is to take advantage of what computer and networking technologies do to remake how and organization can operate, rather than to continue to operate it the same way, just with more speed or more cheaply, just as electrification and unit drive remade how a factory worked.

And so that's what we're trying to do. Just as animals and steam and electricity augmented human muscle, and earlier computer software and processes augmented human logical thinking and computation, Web 2.0 applications are meant to augment those unique human abilities that can't be automated. We're trying to produce applications that increase the power of human judgement, cognition, social understanding, inspiration, and adaptability.

After Weblogic 9.x, I couldn't find the script and basically had to resort to command-line startup of the server for a while. Now, I generally don't like to have command-line startups cluttering my taskbar, especially when I'm using Windows and doing demoes.

So, with the help of my colleagues Ian and Devon, we wrote a script that would install a managed server as a Windows service and which you can quietly start in the background.

Here it is.Replace your paths as appropriate. If you use the script below, make sure that you also include the username and password in your boot.properties file (the username and password are required at startup of the server). Hang on, did I just say write a username and password in a file that can then be read by anyone? Nope. When you first start Weblogic server, the username and password values in boot.properties would be encrypted at the first run.

Naturally, all the usual disclaimers apply: i.e. I'm not responsible for anything etc etc.

================================================================
set DOMAIN_NAME=mydomain
set USERDOMAIN_HOME=D:\bea\user_projects\domains\mydomain
set SERVER_NAME=myserver
set PRODUCTION_MODE=true

set JAVA_OPTIONS=-Dweblogic.Stdout="D:\bea\user_projects\domains\mydomain\stdout.txt" -Dweblogic.Stderr="D:\bea\user_projects\domains\mydomain\stderr.txt" -Dweblogic.management.server="http://localhost:7001" -Dweblogic.management.discover=false

set MEM_ARGS=-Xms512m -Xmx1024m

call "D:\bea\wlserver_10.0\server\bin\installSvc.cmd"
================================================

Enjoy!

In this post, I'll instead talk about WebCenter Interaction or more precisely, how WCI makes use of network protocols. This post was originally posted by the great GeraldK on dev2dev, so I'm publishing it here to share some of his wisdom with you. Here we go.

This post gets pretty low-level, but I think this is important. As much as everything in our business relies on us abstracting away things at lower levels and specializing, abstraction only works up to a point. In some ideal, interfaces are clean and well-specified and focused; the application doesn't worry about how the OS works, and the OS doesn't worry about how the hardware works, and layer 4 of the OSI model doesn't worry about how layer 3 works. All you are supposed to do as a specialist at one level is assume the levels below you just work. But does that really work? If you're writing an application, it's better if you really know how the OS works and you do have to worry about how the database stores characters, and a good OS must worry about how the hardware works, and you're on crack if you think that TCP didn't worry about how IP was implemented.

So here, even though we're supposed to be about web applications and services that run up above OSI layer 7, we're going to go down n look at layer 4 (and even a bit lower) and see what's happening. We'll discuss what the differences are between TCP and UDP, what multicast is and how and why is works and doesn't work. Trust me, it'll be useful, and it all leads to some info about how PTSpy works in G6 and some ideas on how you can use it better.

So, let's start with HTTP. That's the protocol that matters directly for most of what we do here. HTTP, along with most of the rest of our application networking (SQL*NET, WCI Search) run at network layer 7, the application layer, on top of TCP which operates at layer 4. So what's TCP?

TCP (Transmission Control Protocol), along with UDP is one of the main underlying network protocols of the internet. Both of those are built on top of yet another one, IP (Internet Protocol). IP works one layer below TCP and UDP (layer 3). So to understand TCP, let's look at IP. We'll then work our way back up and talk about what TCP does on top of that.

IP: Internet Protocol
IP provides the basic ability to route a packet of data from one place to another. It does this by providing each "place" or "device on the network" with a specified address (the IP address) and specifying how such devices should move packets around based on the address. Now, the difference in functionality between IP and the protocol layer below it (layer 2, the data link layer, typically represented by Ethernet or WiFi) is that at layer 2, devices always know exactly how to send to every device on the network. At layer 2, either you know exactly how to get data to your destination (btw, usually represented by a "MAC address") or it's not possible to get it there. (For example, Ethernet and WiFi simply broadcast the entire packet onto the whole network, and the destination is expected to be listening for its MAC address and pick up the packet. If it's not there or not listening, Ethernet can't get it there. [And btw, network "sniffers" works by taking advantage of this broadcasting.] PPP, used in modem dial-up connections, can send everything to one destination only: the machine you dialed in to.)

What IP does is provide a way to connect separate networks so that devices on one network can send data to devices on another network without knowing where they are or how to get there exactly. Hence the appellation inter-net, "between networks". It does this by specifying routing rules that define what a network device does with a data packet with any destination address. Basically the rules are: if the destination is local (i.e. you know where it is because it's on the same network) send it there; otherwise, send it to one of a list of routers that vary according to the address. A router follows the exact same rules, except that it's assumed that it belongs to two or more different networks at the same time; and therefore has more local destinations, and also usually has a longer list of other routers for unknown destinations.

Very specifically now, IP does no more than give you the ability to send a single packet of data to a destination specified by a single IP address, and that's it. It can get that packet to any place on any network (unlike the lower level protocol), but that's about it. Significantly:

• It does not provide notification of delivery or receipt or failure


• It does not provide any notion of "port numbers" or other ways of segregating packets at the destination IP addresses


• It does not provide two-way communications


• It does not order or otherwise group multiple packets in any particular way

The simplest analogy is to think of IP like the postal service. You drop a postcard into the mail with an address on it, and somehow it makes it to the address you wrote on it. If it makes it, or if it doesn't, you don't know. When it gets to the house, you don't know if the roommate reads it. And if you want a reply, your recipient can't just write on the same card and hand it back to the mailman; they'd have to write out their own note, stamp and address it, and send it out again themselves.

TCP: Transmission Control Protocol
While IP does not provide any of those features, TCP does. If you looked at the list of the features that IP doesn't provide, and then you looked at the analogy of the postal service and said, "Huh? Of course you can have two-way communications. People used to write letters back and forth and have conversations all the time," or "Well, you could just request that either the recipient or the postal service send you back a return receipt," or "Come on, dummy, you can sequence your postcards by just writing numbers on them and telling the recipient to read them in order and let you know if any are missing," well, then, you're right. That's exactly what TCP does. It takes the basic IP (or postal service) and specifies how to use it and what to put in your message to get all those extra features.

So what TCP really winds up doing is implementing the concept of multiple reliable connections between IP devices. What does that mean? It means that you can send a sequence of messages (packets) about a specific conversation (port, or connection) and that the packets will be received in the same order they were sent, and that there won't be missing packets in between sent packets. (Also, there won't be duplicate packets.) It does this by attaching to every packet both a port number, to separate and identify multiple connections or conversations; and a sequence number, to let the recipient identify when data might have been lost in transmission. In addition, it specifies the recipient has to acknowledge every data byte that was received (not necessarily explicitly: it might simply say, "I've received everything up till byte 13456"; and it can also acknowledge selectively, "I've received everything between 845 and 13433") so that the sender can know if missing data needs to be resent. And finally, every connection is implicitly two-way, not just for acknowledgements, but also so that the recipient can just send data "back" rather than having to explicitly re-address (sort of the equivalent of sending a self-addressed reply envelope with every packet).

As you can see, if packets are getting lost or arrive out of order, there is actually quite a lot of work that TCP has to do. If we're continuing the postal service analogy, TCP is a bit like a personal assistant who collects your mail, groups it, sorts it in order, brings and reads it to you, takes your replies and organizes and sends them back. If the postal service is super-reliable, his job is easy, he's just a middleman passing paper around. If the postal service loses a lot of stuff, and/or you have a great volume of mail, he may have to do a lot of work, request that things be re-sent, and keep track of and store a lot of messages.

UDP: User Datagram Protocol
UDP on the other hand is a lot simpler. It does what IP does, but it adds the concept of a port, so that you can send messages to a specific recipient at one IP address. It doesn't have order, or connections, or two-way communication, or acknowledgements.

You might think that UDP is unreliable, because, you know, TCP is supposed to be the reliable one of the siblings. But in fact, over the same network segment, or over LANs with good quality gear and not excessive traffic, UDP is in practice very reliable. If there's no packet loss and packets arrive in order (which is almost always the case on a short LAN link), there's no need for any retransmissions of packets, so all the acknowledgements and waiting around of TCP is just a bunch of wasted overhead, creating latency. For applications that can tolerate packet losses (say, real-time audio and video), it's often a good choice even over not-so-good networks. It's also used for small messages and notifications. For example DHCP and DNS both use UDP. And notably, the Unix Network File System (NFS) used UDP over LANs. While you'd think a file system should have used a reliable TCP connection, the implementers of NFS felt they could get better performance using UDP and building their own mechanisms for ensuring reliability that were tuned to the application.

Btw, it's called "User Datagram Protocol" because it's was designed by operating system guys. "Datagram" is just another word for "packet". "User" in this case doesn't mean, like, you; it means a program on a computer that is not part of the operating system. The idea was that IP is low-level and it was used by people who write parts of the OS, while UDP provides mostly the same functionality (the datagram), but is intended for "user" (non-OS) programs.

Multicasting
I simplified the TCP/IP/UDP discussion up there a little when I implied that IP (and hence UDP and TCP) would route packets from one device on a network to another device. More accurately, IP routes packets from one IP address to another IP address. The trick with multicasting, or sending the same packets of data at the same time to many devices is that certain IP addresses can be designated as multicast address and assigned to many devices at the same time.

The first thing to know about IP multicasting is that there only exists UDP multicasting. There's no such thing as TCP multicast. Why not? See, the point of multicast is to be efficient with the network and send the same data packet to as many different, possibly unknown computers. But each TCP connection can potentially require retransmission of different lost packets, or different delays or ordered arrival and assembly of packets. So managing and sending all that for each possible device would be resource-intensive even if it were possible, which would defeat much of the point of using multicast. (And it's not possible anyway because multicast can't know where the outbound packets actually wind up.)

Oh, and via back-formation, regular non-multicast UDP (and TCP) messages are called unicast.

The next thing to know is that multicast will often not be sent over a router to another network. There are a few reasons why:

• A low TTL for most multicast packets: All IP packets have a time-to-live, or TTL. Unlike with DNS records, TTL on IP packets refers to the maximum number of network hops that a packet can make to get to its destination. Unicast packets typically are allowed to cross about 30 networks (i.e., get routed by, or "hop", 29 routers). Getting across the internet rarely takes more than 15 hops, so the limit is really just to kill packets if they get stuck in loops in badly configured networks. But most applications that send multicast set the TTL to a low number, often zero (where the message doesn't even leave the originating device),
  


• one (so it will go only to computers on the local net), or two (where it will cross a single router). It's quite rare to have an application that is intended to multicast to unknown machines over an entire campus network, much less the entire Internet.
  


• High TTL thresholds set on most routers: Many network routers, particularly WAN routers and internet gateway routers, have a higher TTL threshold set, such that they won't send out multicast packets whose TTL is below (say) 15. This is intended to prevent accidental leakage of multicast off a local network.
  


• Routers are simply not configured to route multicast at all, or only configured for specific addresses, or otherwise blocking multicast packets.
  

UDP multicast might seem a bit exotic, but it's actually more common that you think. It's not used for web video sites like YouTube, since those need to send a video at a time each user demands, rather than at the same time for all users, and it's not used for VoIP traffic. However, it is used for a lot for discovery and automatic confguration in apps like Skype, iTunes, and uPnP. It's also used in a few places in the WCI portal.

Spy Logging
Since G6 was released, Logging Spy (formerly PTSpy) and the Plumtree Logging Framework (AL-something Logging Toolkit or whatever) have worked using UDP. The application that has messages to log just sends it out via UDP. Now, recall that UDP doesn't worry about things like actual delivery and order and error correction and flow control. The message can be sent out as "fire-and-forget", or "drop it in the outbox and forget about it forever". There's no record-keeping or waiting, so it's very cheap and quick for the program to just put send the message.

In fact, it costs pretty much the same for a program to just send out the log message via UDP as it would for it to check to see if the message should be filtered. So, the G6 portal (and collab, and the api and automation servers, etc.) doesn't bother checking; it just sends out the UDP log message unconditionally.

In contrast, 5.x and earlier portal versions logged positively: every message had to be written to a file (or the screen) before the program could proceed to the next operation. This impacted performance severely enough that you could only enable tracing when you were actively debugging, and even then had to try to limit what you logged. Throwing out the UDP packet causes so little delay that G6 traces a lot more, and it sends the trace messages the whole time the program is running.

Now, we also have a set of programs that read these UDP packets from the network. The most familiar one is Spy itself, which captures the packets, picks out the ones you told it you wanted to see, and displays only those. There is also the Logger Service, which works similarly, but reformats the messages and writes them to a text file.

You might not even actually know that the Logging Framework is using UDP to send log messages, because normally the applications are set to "local logging only" which sets the packet TTL to zero, which doesn't let it leave the originating computer (but which can be picked up by a destination on the same computer). At the same time, Spy is usually installed along with the application. When you start Spying, it just picks up the packets from the local application, and it looks like the old 5.x PTSpy connecting locally to the locally-installed application. But in fact, the application and the viewer for the Spy messages are now very loosely coupled over the network, and run independently of each other.

While UDP does offer great performance advantages, so much so that we now can have much more detailed logging available when necessary, there is the possibility that some log messages can be lost, just because of the unreliable nature of UDP. This isn't really a disaster, but it could be annoying if you're debugging and you don't know it may be happening. It may happen when there is a lot of activity on the portal (and hence a lot of messages being sent), the network is very busy so packets are lost, or the capture program is running slowly. In practice, the most common case in which messages are lost is when the Spy program is running slowly, maybe because the machine it's on is busy, or because of some other resource issues. We've found in particular that the X-Windows Spy, when writing to the screen, can miss a lot of messages. I usually recommend using the command-line logger on Unix anyway.

One more thing to note is that Spy is actually sent out via UDP multicast, not unicast. This means, first of all, it's possible to have several programs picking up and reading and recording log messages from a single application. For instance, while you are looking interactively at Spy, the Logging Service can also be recording the same or a different subset of messages to a file. Second, you can also monitor several applications remotely using a single (or multiple) programs on a remote machine. You simply need to make sure that the application "local logging only" option is set to false, and that your network and routers will route the packets from the WCI applications to the logging listeners. Then, just open up Spy, select View->Set Filters, then Edit->Add Message Sender. Spy will look for applications that are configured to log via UDP multicast over the network (and which can reach Spy) and allow you to choose to capture and record the messages.

Collab Clustering
The other place that we encounter UDP is when configuring Collaboration Server clustering. Collab clustering is used for cache invalidation among the Collab servers. When an object is updated on one Collab server, a message needs to be sent to all of them identifying the object. That's a small, simple message.

There are several configurations for setting up Collab to send the message. The two most common should be the "lan-cluster" configuration and the "lan-multicast-cluster" configuration. Both of them use UDP to send the messages. The unicast configuration requires you to list out every other server in the cluster in each server's config, but will work on any network where the servers are actually reachable to each other. The multicast setting requires less configuration and config maintenance, since you all you do is set the multicast listen address on each one to the same value, but you run the risk that the network may not support or route multicast among your servers. It's also hard to know if it's working without running network traces to look for the Collab cache invalidation messages, as for the most part Collab will run fine and no one single user is likely to notice inconsistencies in caches across different servers.

Well, I hope that was informative and interesting. And useful.

When I first joined Plumtree 5+ years ago, there was so much to learn: Portal, search, collaboration, publisher, content, architecting the whole set etc. I was thrust deep into an implementation primarily because the deployment was on UNIX and I knew UNIX (at least I knew enough to fumble my way around).

As you can guess, I ran into installation problems (hey that was the old 4.5WS on UNIX after all) and given the baffling nature of these problems, quite a few people stepped up to help me. But one guy stood out by the sheer quality and detailed suggestions to resolve the problems. And his solutions worked. Oh, he was also helping others who had hairy issues too. His name is Gerald Kanapathy.

A couple of months later, we had our first kick-off in Vegas and the PS team had its own breakout sessions. When Gerald was introduced as the speaker, someone behind me muttered with a mixture of both awe and reverence: “Yeah, that’s the Plumtree God”. Since then, Gerald’s reputation was cemented in my mind. In our 2004 kick-off, he was nominated (and won) for Best Employee Award or something similar and one the nominators wrote: “He’s the guy we folks in engineering go to when we are stuck.”

For those of you ALUI (now WebCenter Interaction) afficionados, Gerald used to run a blog on dev2dev@bea where he shared a lot of his invaluable knowledge and experience on technology, the WebCenter Interaction product and Web 2.0 with us lesser mortals. He has graciously shared the posts with me and now I can in turn share it with you. In the coming weeks, I’ll be republishing them here.

I hope they will be useful to you as they have been to me.

Cheers,
Ali