Right tools for the job? - Interesting discoveries when processing large volume of data

Just now I had an occasion to work with an integration solution intended to process lots of records. By lots I mean over 1 million smallish records. My customary platform to experiment on is Windows XP. Lots of reasons for that, most of them historical – I have tools I know and like and so on. While trying to work with such a volume of data I noticed a number of “interesting” things, which I thought I should share. These things are related to both the platforms (Windows vs. Linux), the tools and the architectural decisions.

I needed lots off data to test the solution I was contemplating, which involved XML processing, to see how constructing and parsing XML affects solution performance. To make it easier to compare timing differences I though I should use lots of records.

The discoveries are discussed in Right tools for the job.pdf.
Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

In this Blog I post abstracts of articles / writeups / notes on various aspects of Java CAPS and SOA Suite including solutions, discussions and screencasts. The links to the referenced material are included in the bodies of the abstracts.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today