This rather long posting details the steps required to
grouping of fixed length records using Format Builder.
If it’s 10 pm and you’re feeling beat you might want to leave this until tomorrow.
But if it’s 10 pm and you need to get a Format Builder Complex template done, read on…
The goal is to process individual orders from a file using the 11g File Adapter and Format Builder
001Square Widget 0245.98
102Triagular Widget 1120.00
403Circular Widget 0099.45
301Hexagon Widget 1150.98
The records are fixed length records representing a number of logical Order records.
Each order record consists of a number of item records starting with a 3 digit number,
followed by a single Summary Record which starts with the constant ORD.
How can this file be processed so that the first poll returns the first order?
102Triagular Widget 1120.00
403Circular Widget 0099.45
And the second poll returns the second order?
Note: if you need more than one order per poll, that’s also
see the “Multiple Messages” field in the “File Adapter Step 6 of 9” snapshot further down.
To follow along with this example you will need
- Studio Edition Version 184.108.40.206.0 with
- SOA Extension for JDeveloper 220.127.116.11.0 installed
Both can be downloaded from here: http://www.oracle.com/technetwork/middleware/soasuite/downloads/index.html
You will not need a running WebLogic Server domain to complete the steps and Format Builder tests in this article.
Start with a SOA Composite containing a File Adapter
The Format Builder is part of the File Adapter so start by
a new SOA Project and Composite.
Here is a quick summary for those not familiar with these steps
- Start JDeveloper
- From the Main Menu choose File->New
- In the New Gallery window that opens Expand the “General” category and Select the Applications node.
Then choose SOA Application from the Items section on the right. Finally press the OK button.
- In Step 1 of the “Create SOA Application wizard” that appears enter an Application Name and an Directory of your
choice, then press the Next button.
- In Step 2 of the “Create SOA Application wizard”, press the Next button leaving all entries as defaulted.
- In Step 3 of the “Create SOA Application wizard”, Enter a composite name of your choice and Press the Finish
These steps result in a new Application and SOA Project.
The SOA Project contains a composite.xml file which is opened and shown below.
For our example we have not defined a Mediator or a BPEL
minimize the steps,
but one or the other would eventually be needed to use the File Adapter we are about to create.
Drag and drop the File Adapter icon from the Component
either the LEFT side of the diagram
under “Exposed Services” or the right side under “External References”. (See the Green Circle in the image below).
Placing the adapter on the left side would indicate the file being processed is inbound to the composite,
if the adapter is placed on the right side then the data is outbound to a file.
Note that the same Format Builder definition can be used in
For example we could use the format with a File Adapter on the left side of the composite to parse fixed data into XML,
modify the data in our Composite or BPEL process and then use the same Format Builder definition with a File adapter
on the right side of the composite to write the data back out in the same fixed data format
When the File Adapter is dropped on the Composite the File Adapter Wizard Appears.
Skip Past the first page, Step 1 of 9 by pressing the Next button.
In Step 2 enter a service name of your choice as shown below, then press Next
When the Native Format Builder appears, skip the welcome page by pressing next.
Also press the Next button to accept the settings on Step 3 of 9
On Step 4, select Read File and press the Next button as shown below.
On Step 5 enter a directory that will contain a file with the input data, then Press the Next button as shown below.
In step 6, enter *.txt or another file format to select input
from the input directory mentioned in step 5.
ALSO check the “Files contain Multiple Messages” checkbox and set the “Publish Messages in Batches of” field to 1.
The value can be set higher to increase the number of logical order group records returned on each poll of the file adapter.
In other words, it determines the number of Orders that will be sent to each instance of a Mediator
or Composite processing using the File Adapter.
Skip Step 7 by pressing the Next button
In Step 8 press the Gear Icon on the right side to load the Native Format Builder.
Native Format Builder appears
Before diving into the format, here is an overview of the process.
Approach - Bottom up
Assuming an Order is a grouping of item records and a summary record….
- Define a separate Complex Type for each Record Type
(One for itemRecord and one for summaryRecord)
- Define a Complex Type to contain the Group of Record types defined above (LogicalOrderRecord)
- Define a top level element to represent an order.
The order element will be of type LogicalOrderRecord
Defining the Format
In Step 1 select “Create new” and “Complex Type” and “Next”
In Step two browse to and select a file containing the test
shown at the start of this article.
A link is provided at the end of this article to download a file containing the test data.
Press the Next button
In Step 3 Complex types must be define for each type of input record.
This creates a new empty complex type definition shown below.
The fastest way to create the definition is to highlight the first line of the Sample File data and drag the line
onto the <new_complex_type>
Format Builder introspects the data and provides a grid to
Change the “Complex Type Name” to “itemRecord”
Then click on the ruler to indicate the position of fixed
Drag the red triangle icons to the exact columns if necessary.
Double click on an existing red triangle to remove an unwanted entry.
In the case below fields are define in columns 0-3, 4-28, 29-eol
When the field definitions are correct, press the “Generate Fields” button.
Click on the field names and rename them from C1->itemNum,
C2->itemDesc and C3->itemCost
When all the fields are correctly defined press OK to save the complex type.
Then highlight and drag the Summary Record from the sample data onto the <new_complex_type>
Change the complex type name to “summaryRecord”
Mark the fixed fields for Header, Order Number and Order
Press the Generate Fields button and rename C1, C2 and C3 to Header, itemNum and orderDate respectively.
The last complex type to be defined is a type to hold the
items and the summary record.
Select the Root-Element in the schema tree and click the new complex type icon
Select the “<new_complex_type>” entry and click the pencil icon
On the Complex Type Details page change the name and type of each input field.
Change line 1 to be named item and set the Type to “itemRecord”
Change line 2 to be named summary and set the Type to “summaryRecord”
We also need to indicate that itemRecords repeat in the input file.
Click the pencil icon at the right side of the item line.
On the Edit Details page change the “Max Occurs” entry from 1 to UNBOUNDED.
We also need to indicate how to identify an itemRecord.
Since each item record has “.” in column 32 we can use this fact to differentiate an item record from a summary record.
Change the “Look Ahead” field to value 32 and enter a period in the “Look For” field
Press the OK button to save entry.
Finally, its time to create a top level element to represent an order.
Click on the <new_element> and press the pencil icon.
Set the Element Name to “order” and change the Data Type to “logicalOrderRecord”
Press the OK button to save the element definition.
The final definition should match the screenshot below.
Press the Next Button to view the definition source.
Press the Test Button to test the definition
Press the Green Triangle Icon to run the test.
And we are presented with an unwelcome error.
The error states that the processor ran out of data while
through the definition.
The processor was unable to differentiate between itemRecords and summaryRecords and therefore treated the entire file
as a list of itemRecords. At end of file, the “summary” portion of the logicalOrderRecord remained unprocessed but mandatory.
This root cause of this error is the loss of our “lookAhead” definition used to identify itemRecords.
This appears to be a bug in the Native Format Builder
Luckily, a simple workaround exists.
Press the Cancel button and return to the “Step 4 of 4” Window.
Manually add nxsd:lookAhead="32"
nxsd:lookFor="." attributes after the maxOccurs
of the item element.
as shown in the highlighted text below.
When the lookAhead and lookFor attributes have been added Press the Test button
and on the Test page press the Green Triangle.
The test is now successful, the first order in the file is returned by the File Adapter.
Below is a complete listing of the Result XML from the right column of the screen above
Try running it
The downloaded input test file and completed schema file can
for testing without
following all the Native Format Builder steps in this example.
Use the following link to download a file containing the sample data.
This is the best approach rather than cutting and pasting the
data at the top of the article.
Since the data is fixed length it’s very important to watch out for trailing spaces in the data and to
ensure an eol character at the end of every line. The download file is correctly formatted.
The final schema definition can be downloaded at the following link
- Save the inputData.txt file to a known location like the xsd folder in your project.
- Save the Completed Schema Definition inputData_6.xsd file
xsd folder in your project.
Edit inputData_6.xsd and modify the last two comment lines to point to the inputData.txt location on your system.
Without this modification the format will be declared invalid.
- At step 1 in the Native Format Builder wizard (as
above) check the “Edit existing” radio button,
then browse and select the inputData_6.xsd file
- At step 2 of the Format Builder configuration Wizard (as
above) supply the path and filename for
the inputData.txt file.
- You can then proceed to the test page and run a test.
- Remember the wizard bug will drop the lookAhead and lookFor
attributes, you will need to manually add
nxsd:lookAhead="32" nxsd:lookFor="." after the maxOccurs attribute of the item element in the
LogicalOrderRecord Complex Type. (as shown above)
The complete example is available on GitHub here
Good Luck with your Format Project