Wednesday Jan 22, 2014

New CAM XML Editor v3.1 and CAMV validation release now available

The focus for this release is enhancements, bug fixes and performance improvements for both the CAM editor and CAMV validation engine.

The new CAM Editor V3.1 provides the following improved functionality:

  • Enhanced XSD schema importing especially for HL7, GML, OAGi and NIEM complexities
  • New UTF-8 handling to improve international support in elements and annotations
  • Editor entry of annotations improved and enhanced
  • Better XML example generating details (choice items, negative numbers and repeat limits)
  • Improved xsd:annotations handling during import processing (faster + suppress duplicates)
  • Improved bi-directional data processing using Open-XDX for open data query and update
  • Feature and bug fixes for the CAMV rules engine
  • CAMV now allows mixed content for validation

Available from http://www.cameditor.org

Monday Nov 04, 2013

Creating, using and managing XML component dictionaries quick tutorials

XML Component Dictionary capabilities are provided in conjunction with the CAM Editor toolset.  These dictionaries accelerate the development of consistent XML information exchanges using standard sets of dictionary components.

The quick tutorials are aimed at showing the 'how to' of the basic capabilities to jump start use of XML dictionaries with the CAM Editor.

The collection of dictionary tutorials videos run for a total of approximately 20 minutes.  Each video can be reviewed individually also.

Learn how to use the dictionary functions to create dictionaries by harvesting data model components from existing XSD schema, SQL database table schema, or simple Excel / Open Office spreadsheets with tables of components listed.

Also included are tips and functions relating to use of NIEM exchange development, IEPD and EIEM techniques.

These videos should be viewed in conjunction with reviewing the overall concepts and techniques described in the companion video on the CAM Editor and Dictionaries overview.  The approach is aligned with OASIS and Core Components Technical Specification (CCTS) standards specifications for XML components and dictionaries.

Dictionary collections can be stored locally on the file system, or local network, or collaboratively on the web or cloud deployment, or can be shared and managed securely using the Oracle Enterprise Repository (OER) tool.

Also included are techniques relating to the use of the NIEM approach for developing XML exchange schema and IEPD packages.  This includes generating reuse scores, wantlist, and cross reference spreadsheets.

Included in the latest release of the CAM Editor is the ability to use the analyse dictionary tool to determine duplicate components, conflicting component definitions, missing component descriptions and so on.  This ensures high quality dictionary component specifications.  Using the CAM Editor you can also create MindMap models and UML physical models of your dictionary components sets.

For a complete guide to using the CAM Editor see the main YouTube video tutorials website and the CAM Editor website.


Monday Oct 21, 2013

Oracle BPM and Open Data integration development

Rapidly developing Oracle BPM application solutions with data source integration previously required significant Java and JDeveloper skills. Now using open source tools for open data development significantly reduces the coding needed.  Key tasks can be performed with visual drag and drop designing combined with menu selections entry and automatic form generation directly from XSD schema definitions.

The architecture used is extremely lightweight, portable, open platform and scalable allowing integration with a variety of Oracle and non-Oracle data sources and systems.

Two videos available on YouTube walk through the process at both an introductory conceptual level and then a deep dive into the programming needed using JDeveloper, Oracle BPM composer and Oracle WLS (WebLogic Server) along with the CAM editor and Open-XDX open source tools.

Also available are coding samples and resources from the GitHub project page, along with working online demonstration resources on the VerifyXML site.

Combining Oracle BPM with these open source tools provides a comprehensive simple and elegant solution set. Development times are slashed and rapid prototyping is enabled. Also existing data sources can be integrated using open data formats with either XML or JSON along with CRUD accessing via the Open-XDX Java component. The Open-XDX tool is a code-free approach where data mapping is configured as templates using visual drag and drop in the CAM Editor open source tool.  XML or JSON is then automatically generated or processed (output or input) and appropriate SQL statements created to support the data accessing.  

Also included is the ability to integrate with fillable PDF forms via the XML templates and the Java PDF form filling library.  Again minimal Java coding is needed to associate the XML source content with the PDF named fields. 

The Oracle BPM forms can be automatically generated from XSD schema definitions that are built from the data mapping templates.  This dramatically simplifies development work as all the integration artifacts needed are created by the open source editor toolset.

The developer level video is designed as a tutorial with segments, hands-on demonstrations and reviews.  This allows developers to learn the techniques and approaches used in incremental steps. The intended audience ranges from data analysts to developers and assumes only entry level Java skills and knowledge.  Most actions are menu driven while Java coding is limited to simply configuring values and parameters along with performing builds and deployments from JDeveloper and Oracle WLS.  

Additional existing Oracle online training resources can be referenced on Oracle BPM and WLS that cover other normal delivery aspects such as user management and application deployment.

Monday Aug 05, 2013

New CAM v3.0 ships with JSON support and significant performance enhancements

Today we released the new and significantly improved CAM editor toolset along with 3 new companion 'How to' quick videos (see here to view).

The main focus is integrating JSON handling alongside the existing XML capabilities to provide developers with the ability to use either or both from the single set of infrastructure.

This provide JSON developers with the ability to quickly build visual data models, use robust XML content validation services and generate XSD schema and JAXB bindings without having to do all those tasks by hand or know the nuances of complex XSD schema or XML handling.

For XML developers it provides a rapid ability to use JSON as an option in their information exchanges and web service integration for supporting mobile and web-based application needs.

In addition to these new JSON capabilities the existing functionality has significantly improved performance and capability.  The CAMV validation engine now runs up to 20 times faster for large XML validation input and with templates containing setChoice() rules.  For comparison a 500+ rules validation template with large 15MB sample COBie CAD/CAM smart building XML export now runs in 19 seconds instead of previously taking over 9 minutes.

Then the drag and drop dictionary components handling has similarly been significantly improved.  Large sets of components now are inserted in real time with low memory overhead thus dramatically improving the user experience and ability to quickly build information exchanges from XML dictionaries of predefined domain components.  The video shows using the Education domain to rapidly build a StudentDetails report with grades, achievements and student data.

For the Open-XDX open data API toolset we have added bi-directional support.  This means using the same CAM template and the SQL drag and drop interface you can design Update/Insert SQL database web services along with the query services.  Again the focus is on providing simple and rapid application development support.  Example code and resources can be found at our GitHub site while on line demonstrations are available from the VerifyXML.org site.

Further enhancements include a new Dictionary Evaluation report.  This tool analyzes the XML components in a dictionary and highlights design issues, omissions, duplicates and more that would be extremely tedious to detect by hand.  This allows a development team to collaboratively improve the quality of their core components and their reuse across a project implementation.

Last but not least we have improved the XSD schema importing and exporting resolving a range of complexity nuances not previously handled allow improved accuracy and compatibility with XSD schema.

In summary the new release provides:

o All new JSON capabilities and template type
o Bi-directional data processing using Open-XDX for open data query and update
o Dramatically improved Dictionary components drag and drop
o New report for Dictionary evaluation and analysis
o Significant CAMV rules engine performance improvements
o Better XSD schema importing and exporting

We look forward to seeing the enhanced solutions this helps people deliver to their customers.



Friday May 10, 2013

White House announces Open Data policy - dawn of a new age of information sharing

The White House today released an Executive Order -- Making Open and Machine Readable the New Default for Government Information.

In addition there is now a new open source tools project and resources on GitHub in support of this initiative.

The potential here to change how a whole range of services are delivered to citizens is significant and also for new services and commercial opportunities to emerge that utilize these data services.

To see the types of potential here - see sample Open Data API show case work on the related VerifyXML.org site also.


Saturday Mar 02, 2013

Migrate Excel text CSV / SQL Server CSV -> SQL TABLE CREATE / INSERT data tool

Loading CSV text data into a MySQL database table is an art form. Obviously the text data can trip up for a variety of reasons from non-unique keys to missing data columns to invalid number or date formats. I recently needed to load over 20 such tables from a SQL Server CSV text file dump.  The same techniques would work for data from an Excel csv text file too.

To automate the process as much as possible I wrote a quick XSLT utility (called converter-txt-2-sql.xsl) that reads in the CSV text file, examines the first line that contains the table column field names, then analyses the samples by scanning the entire input data lines, and generates a valid CREATE TABLE {name} ( {column(s)}); SQL statement.  Then in a second pass it builds the INSERT VALUES ({data}) statement for all the following data lines in the CSV.

It does a pretty good job, about 98% of what you need.  You still need to do some manual editing of the CREATE TABLE SQL generated.  Essentially it can only guess at the lengths for each column - so you may want to manually adjust those, along with setting the key field column name (it assumes the first one for that), and then if your data is null or unique and so on.  But those are quick edits once it has all the basics there for you.

It assumes that each line is one data record in the CSV text file input; so you cannot have multiple linefeeds inside your data lines, only one at the end of each line; a fair assumption most of the time.

Having got it working I was able to load up the twenty tables in less than an hour.  There is still room to improve the XSLT logic to handle various edge conditions better, but for the time I invested in writing the XSLT its a fair level of maturity, awaiting the next project to see if it needs more refinement.  Plus it is a nifty example of using XSLT to read in a text source file and output text (in this case SQL statements).  Note: depending on your XSLT processor (I used Saxon) you may need to feed a dummy xml file in e.g. <dummy/> just to satisfy the processing engine.

Anyway - you can find the XSLT here: http://sourceforge.net/projects/camprocessor/files/CAM%20Editor/Misc/  and download the converter-txt-2-sql.xsl and try it for your own text data loading needs. It is provided "as is" so use at your own discretion and you should probably have run XSLT before using a tool like OxygenXML or similar that supports the Saxon processor.

I found it worked well to migrate over SQL Server tables quickly into MySQL just working from the raw csv text export files from SQL Server that had been sent to me.  It's not completely perfect but it should suffice for proof of concept purposes and quick demonstrations. And you have to know what you are doing, to be able to resolve syntax and data integrity errors.  However I was able to load over 50,000 data records well enough.

Of course if you can get a live connection from MySQL to SQL Server then you can use the built-in migration tools MySQL has.  This little XSLT utility is useful when you do not have that option.  Or if people are using Excel spreadsheets with data tables and you want to convert those over to SQL tables.


Thursday Oct 25, 2012

SQL to XML open data and NIEM training video posted

Learn how to build a working XML query/response system with SQL database accessing and XML components from example NIEM schema and dictionary.

Software development practitioners, business analysts and managers will find the materials accessible and valuable in showing the decision making processes that go into constructing a working XML exchange.

The 22 minute video available online shows how to build a fully working ULEXS-SR exchange using a Vehicle license search example.  Also included are aspects of NIEM training for assembling an IEPD schema with data models.

Materials are focused on practical implementers, after viewing the instruction material you can use the open source tools and apply to your own SQL to XML use cases and information exchange projects.

All the SQL and XML code, editor tools, dictionary and instructions that accompany the tutorial video are also available for download so you can try everything yourself. 

See http://www.youtube.com/user/TheCameditor to run the video.

And the open source project web site (sponsored by Oracle) contains all the resources, downloads and supplemental materials.

Enjoy.

Sunday Oct 07, 2012

New CAM Editor v2.3 with Open-XDX for Open Data APIs

Creating actual working XML exchanges, loading data from data stores, generating XML, testing, integrating with web services and then deployment delivery takes a lot of coding and effort. Then writing the documentation, models, schema and doing naming and design rule (NDR) checks and packaging all this together (such as for NIEM IEPD use).

What if there was a tool that helped you do all that easily and simply?

Welcome to the new Open-XDX and the CAM Editor!

Open-XDX uses code-free techniques in combination with CAM templates and visual drag and drop to rapidly design your XML exchange. Then Open-XDX will automatically generate all the SQL for you, read the database data, generate and populate the valid output XML, and filter with parameters. To complete the processing solution Open-XDX works with web services and JDBC database connections as a callable module that can be deployed plug and play with your middleware stack, all with just a few lines of Java code (about 5 actually).

You can build either Query/Response or Publish/Subscribe services from existing data stores to XML literally in minutes. To see a demonstration of using Open-XDX, a MySQL data store and integrating with Oracle Web Logic server please see this short few minutes video - http://youtube.com/user/TheCameditor

There is also a Quick Guide available that provides more technical insights along with a sample pack download of templates and SQL that you can try for yourself.

 To view online demonstrations of using Open-XDX see the VerifyXML.org site and GitHub resources.

Head on over to our project resource site to learn more, download the latest CAM Editor and see links to all the resources and materials.

We look forward to seeing how the developer community is able to jump start information sharing initiatives using this new innovative approach.

About

Not all XML is created equal. XML Orb looks at the challenges of creating information exchanges with XML and NIEM and how this can be made simpler, comprehensible, consistent and reliable.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today