Monday Aug 05, 2013

New CAM v3.0 ships with JSON support and significant performance enhancements

Today we released the new and significantly improved CAM editor toolset along with 3 new companion 'How to' quick videos (see here to view).

The main focus is integrating JSON handling alongside the existing XML capabilities to provide developers with the ability to use either or both from the single set of infrastructure.

This provide JSON developers with the ability to quickly build visual data models, use robust XML content validation services and generate XSD schema and JAXB bindings without having to do all those tasks by hand or know the nuances of complex XSD schema or XML handling.

For XML developers it provides a rapid ability to use JSON as an option in their information exchanges and web service integration for supporting mobile and web-based application needs.

In addition to these new JSON capabilities the existing functionality has significantly improved performance and capability.  The CAMV validation engine now runs up to 20 times faster for large XML validation input and with templates containing setChoice() rules.  For comparison a 500+ rules validation template with large 15MB sample COBie CAD/CAM smart building XML export now runs in 19 seconds instead of previously taking over 9 minutes.

Then the drag and drop dictionary components handling has similarly been significantly improved.  Large sets of components now are inserted in real time with low memory overhead thus dramatically improving the user experience and ability to quickly build information exchanges from XML dictionaries of predefined domain components.  The video shows using the Education domain to rapidly build a StudentDetails report with grades, achievements and student data.

For the Open-XDX open data API toolset we have added bi-directional support.  This means using the same CAM template and the SQL drag and drop interface you can design Update/Insert SQL database web services along with the query services.  Again the focus is on providing simple and rapid application development support.  Example code and resources can be found at our GitHub site while on line demonstrations are available from the VerifyXML.org site.

Further enhancements include a new Dictionary Evaluation report.  This tool analyzes the XML components in a dictionary and highlights design issues, omissions, duplicates and more that would be extremely tedious to detect by hand.  This allows a development team to collaboratively improve the quality of their core components and their reuse across a project implementation.

Last but not least we have improved the XSD schema importing and exporting resolving a range of complexity nuances not previously handled allow improved accuracy and compatibility with XSD schema.

In summary the new release provides:

o All new JSON capabilities and template type
o Bi-directional data processing using Open-XDX for open data query and update
o Dramatically improved Dictionary components drag and drop
o New report for Dictionary evaluation and analysis
o Significant CAMV rules engine performance improvements
o Better XSD schema importing and exporting

We look forward to seeing the enhanced solutions this helps people deliver to their customers.



Friday Apr 26, 2013

Analysis of JSON use cases compared to XML

Background

Before there was either XML or JSON there was EDI. JSON is very reminiscent of EDI, both syntactically and conceptually, and that the claims made back then as to why EDI would be sustained over XML. EDI was lightweight, human readable, fast to process, compact and worked well with existing systems exchanges and interfaces and had a dedicated following of advocates. But EDI has significant flaws, it is brittle, difficult to extend and has weak data typing, content validation and rendering support. Also semantics in EDI are very limited and rely on external referenced specifications with local human knowledge that is notoriously difficult to align across implementations. Particularly code lists value sets and cross field content validation rules were especially problematic for EDI.

Moving past these limitations standards setting organizations have adopted XML technologies as the primary toolset in defining information exchange specifications. Further more there is an extensive family of XML technologies that support the complete ecosystem of semantics and particularly the need for interoperability, security and common meaning and rules. The diagram here illustrates that.

Figure 1 – Information Exchange Conceptual Components

Referencing this diagram, JSON is restricted to the Structure and Content capabilities. XML on the other hand provides the ability to handle rich exchanges where all the aspects shown can be managed. In today's challenging commercial and government information sharing world you must have the complete set of robust capabilities available.

The JSON primary use case

JSON is designed for web client interfaces to web services on the internet. Essentially it is serialized Javascript objects which makes it a strong fit for web browser native client side scripting that all the major web browsers provide.

While XML does not fit as well for that scenario there are many equivalent solutions using different interfacing in the browser such as Adobe Flash, Microsoft InfoPath, Oracle ADF, or open source solutions such as Netbeans forms that are using XML. One advantage of these is the “write once” approach and deploy anywhere to tablet, smart phone, or web browser.

XML and JSON Performance Analysis

The presumptions of how slow and resource-demanding "Fat” XML is compared to JSON’s lightweight payload do not hold up to a test. An experiment with 33 different documents and almost 1200 tests on the most commonly used browsers and operating systems found the performance for the total user experience, (transfer, parsing and querying of a document), to be nearly identical for both XML and JSON formats.

Clearly this shows that you should perform experiments, test your own data and code with your own users and devices to determine real results. What "seems obvious" is not always true.

Selection of useful links of peoples opinions and thoughts

We present here a selection of “what does the internet think?” resources to show the context to the use of JSON and insights into processing and handling content in a web browser delivery context.

Broad Discussions

http://blog.technologyofcontent.com/2010/01/json-vs-xml/

http://broadcast.oreilly.com/2011/06/the-good-the-bad-the-ugly-of-rest-apis.html

http://digitalbazaar.com/2010/11/22/json-vs-xml/

Landscape and Performance Comparisons

http://java.dzone.com/articles/streaming-apis-json-vs-xml-and

"We are conducting an experiment in XML and JSON performance in browsers and would appreciate anyone with a couple minutes to spare to visit this website and push one button.
http://speedtest.xmlsh.org (the results will be analysed and published - at this coming Balisage 2013)

Pro-XML

http://metajack.im/2010/02/01/json-versus-xml-not-as-simple-as-you-think/

http://www.ajaxonomy.com/2008/xml/why-xml-is-far-superior-to-json

http://stackoverflow.com/questions/3536893/what-are-the-pros-and-cons-of-xml-and-json

Pro-JSON

http://www.advertserve.com/blog/2012/01/api-json-vs-xml/

http://myarch.com/json-pros-and-cons

http://www.scriptol.com/ajax/json-xml.php

http://www.slideshare.net/AnandRaj5/json-13725923

http://bitworking.org/news/JSON_isnt_XML

http://www.json.org/xml.html

http://blog.appfog.com/why-json-will-continue-to-push-xml-out-of-the-picture/

JSON and Security

http://stackoverflow.com/questions/395592/json-security-best-practices

Summary and Conclusions

Number one thing to notice here is you are reading this document and it is being delivered and rendered to your computer screen using XML, RSS and xhtml and not JSON.

Back in the day when XML was brand new, Bill Gates held a press conference to announce that Microsoft would be adopting XML wholesale for use across its products and Windows operating system. This tells us that XML is ubiquitous and extensible (that is in its name). There is now a huge number of XML based standards in a family of solutions that support all aspects of the needs of information exchange. In today's challenging world you cannot just discount those as unnecessary.

When you look at information exchanges the diagram provided in the introduction section above it shows the complete ecosystem of components that you need for effective consistent, trusted, predictable, reusable and extensible information flows. Also we can see that JSON is missing key delivery control and semantic pieces and thus JSON has a very limited mission profile. Within that profile when fit-to-purpose it can be effective, but as a general solution it does not meet all the extended requirements.

Clearly JSON has its niche following and will continue to serve that for its primary use case of web based point client-server information exchanges. That is not necessarily a bad thing. Having lightweight alternative solutions is perfectly acceptable for a lot of content delivery circumstances.

People should not confuse business operational convenience with overall applicability - e.g. Twitter and FourSquare dropping XML and relying solely on JSON. Both of these services use simplistic formats totally under their sole edict that are unlikely to change in the future. Also there are competitive reasons, JSON actually can make it harder with its limited semantics for competitive sites to harvest then analyze and reuse and republish their content.

As a technology XML continues to improve and its use is being better optimized and refined, with tooling support that is narrowing the gap in areas where JSON claims to have the technical edge today. Specifically we can point to Oracle's work on Open Data APIs using Open-XDX that supports both XML and JSON outputs. And the CAM templates approach with NIEM that comes with that and enables content providers to rapidly build working web services and user form interfaces from SQL data stores.

In short we can expect to see both XML and JSON to continue to fulfill information delivery needs going forward but the differentiations are likely to blur. Neither one is going to displace the other in core areas of use. Providing the capability to use and support both is not a significant burden and thus meets personal preferences and local project nuances.

To get a sense for all this as a brief real time interactive example you can try these two live demonstration service points.

This one is using XML when you click here. And this one is doing the same thing (its actually the same Open-XDX service component) but returns JSON instead when you click here.

And if you visit http://www.verifyxml.org you can find more Open-XDX examples and details.

Addendum

Table produced by JSON advocates to support JSON adoption

Capability

XML

JSON

Comment

Simplicity

XML is simpler than SGML

JSON is much simpler than XML. JSON has a much smaller grammar and maps more directly onto the data structures.

Simplicity is deceptive. XML can easily be used as simply as JSON syntactically. But simplicity comes at the price of ignoring many common more robust information sharing needs in an extended network – rather than just point-to-point.

The mapping referenced here is for objects within a JavaScript environment only. Outside of that context this is not so. All major programming environments have robust XML support.

Extensibility

XML is extensible because it is a mark-up language

JSON is not extensible because it does not need to be. JSON is not a document markup language, so it is not necessary to define new tags or attributes to represent data in it.

This is a naïve view. Things change constantly with new information sharing needs. Particularly as more participants are added to exchanges and standards evolve. Only in limited cases such as Twitter can we see set formats.

Interoperability

XML is an interoperability standard.

JSON has the same interoperability potential as XML.

JSON clearly has significant limitations and gaps with regard to information semantics and reuse.

Openness

XML is an open standard

JSON is at least as open as XML, perhaps more so because it is not in the center of corporate/political standardization struggles.

This is a highly subjective statement. XML has proven to be universally adopted and implemented not just in software but firmware devices and communications systems. Notice the JSON work is not immune from manipulation as anything else as happened with JavaScript itself.

Human Readable

XML is human readable

JSON is much easier for human to read than XML. It is easier to write, too. It is also easier for machines to read and write.

Again this is an entirely subjective statement. Markup is markup there is no “easier” here. Machines have no notion of “easier”. The notion of “easier to read” and presumably comprehend the meaning of is notoriously hard to define.

Exchange Formats

XML can be used as an exchange format to enable users to move their data between similar applications

The same is true for JSON

Agreed.

However XML also has security and other capabilities that are absent from JSON.

Structure

XML provides a structure to data so that it is richer in information

The same is true for JSON.

However XML can provide deeper structuring than JSON supports. It can also handle more extended content types.

Processed

XML is easily processed because the structure of the data is simple and standard.

JSON is processed more easily because its structure is simpler.

Again this is entirely subjective. See the link provided in the links section on machine timings testing.

Code Re-invention

There is a wide range of reusable software available to programmers to handle XML so they don't have to re-invent code

JSON, being a simpler notation, needs much less specialized software

JSON is mainly available in JavaScript and not in a wide range of programming environments. Further it is not the simplicity of the syntax, it is the drastically reduced capabilities. Hence JSON only provides very limited functionality.

XML separates the presentation of data from the structure of that data.

XML requires translating the structure of the data into a document structure.

JSON structures are based on arrays and records.

This is only in the context of the data within a web browser memory. Whereas XML is the native format that underpins the spreadsheets, databases and array stores that JSON content must ultimately be persisted to and from!

A common exchange format

XML is a better document exchange format. Use the right tool for the right job.

JSON is a better data exchange format.

Again this is entirely subjective and no metrics are being given here. What defines “better”? Clearly JSON is significantly less capable and restricted in it use cases. Therefore “your mileage may vary in actual use” would be an appropriate caution here when trying to measure what is “better” where and how.

Data Views

XML displays many views of one data

JSON does not provide any display capabilities because it is not a document markup language.

XML has broader applicability. Therefore you can write once, use everywhere. While JSON can expect to be changed into XML for such extended uses.

Self-Describing Data

This is a key XML design objective.

XML and JSON have this in common.

However XML has richer semantics available than JSON.

Complete integration of all traditional databases and formats

(Statements about XML are sometimes given to a bit of hyperbole.) XML documents can contain any imaginable data type - from classical data like text and numbers, or multimedia objects such as sounds, to active formats like Java applets or ActiveX components.

JSON does not have a <[CDATA[]]> feature, so it is not well suited to act as a carrier of sounds or images or other large binary payloads. JSON is optimized for data.

Visual content is data! Ask the FBI analyzing the recent Boston attacks. One could also say that JSON is limited to only simple basic data content and lacks extended validation such as code values, date and number formatting.

Internationalization

XML and JSON both use Unicode.

XML and JSON both use Unicode.

However JSON has limitations in its use of encoding and exchanges.

Open and extensible

XML’s one-of-a-kind open structure allows you to add other state-of-the-art elements when needed. They can always adapt your system to embrace industry-specific vocabulary.

Those vocabularies can be automatically converted to JSON, making migration from XML to JSON very straightforward.

Exactly, if you have XML it is trivial to generate JSON. The reverse is not the case however.

Readability

XML is easily readable by both humans and machines

JSON is easier to read for both humans and machines

This is an entirely subjective statement. The better answer is that well written XML and JSON are equivalent for human and machine handling.

Object-Oriented

XML is document-oriented.

JSON is data-oriented. JSON can be mapped more easily to object-oriented systems.

The reverse is an issue however, objects do not necessarily map easily to documents. Plus not all content is objects; it actually constrains the use model. XML on the other hand is well equipped for use as object-oriented content as well as documents.

Adoptation

XML is being widely adopted by the computer industry

JSON is just beginning to become known. Its simplicity and the ease of converting XML to JSON make JSON ultimately more adoptable.

The use of JSON is limited to web client-server scenarios. Within that domain it is popular. Outside of that domain XML completely dominates.




Monday Nov 05, 2012

XML Rules Engine and Validation Tutorial with NIEM

.

On the technical XML side the video introduces XPath validation rules and illustrates and the concepts of XML content and structure validation. CAM validation templates allow contextual parameter driven dynamic validation services to be implemented compared to using a static and brittle XSD schema approach.

The SQL table lookup and code list validation are discussed and examples presented.

Features are highlighted along with a demonstration of the interactive generation of actual live XML data from a SQL data store and then validation processing complete with errors and warnings detection.

The presentation provides a primer for developing web service XML validation and integration into a SOA approach along with examples and resources. Also alignment with the NIEM IEPD process for interoperable information exchanges is discussed along with NIEM rules services.

The CAMV engine is a high performance scalable Java component for rapidly implementing code-free validation services and methods. CAMV is a next generation WYSIWYG approach that builds from older Schematron coding based interpretative runtime tools and provides a simpler declarative metaphor for rules definition.

See: http://www.NIEMtrainingvideos.org

Thursday Oct 25, 2012

SQL to XML open data and NIEM training video posted

Learn how to build a working XML query/response system with SQL database accessing and XML components from example NIEM schema and dictionary.

Software development practitioners, business analysts and managers will find the materials accessible and valuable in showing the decision making processes that go into constructing a working XML exchange.

The 22 minute video available online shows how to build a fully working ULEXS-SR exchange using a Vehicle license search example.  Also included are aspects of NIEM training for assembling an IEPD schema with data models.

Materials are focused on practical implementers, after viewing the instruction material you can use the open source tools and apply to your own SQL to XML use cases and information exchange projects.

All the SQL and XML code, editor tools, dictionary and instructions that accompany the tutorial video are also available for download so you can try everything yourself. 

See http://www.youtube.com/user/TheCameditor to run the video.

And the open source project web site (sponsored by Oracle) contains all the resources, downloads and supplemental materials.

Enjoy.

Friday Jun 29, 2012

CAM XML Editor version 2.2.1 now available.

CAM Editor v2.2.1 release is now available. Lots of nice enhancements, CAMV performance boost and important bug fixes for DoD, NIEM and LEXS schema.

Download is available from the CAM XML Editor Resource Site


The CAM editor is the leading open source XML Editor/Validation/Schema designer for rapidly building and deploying complete XML information exchanges. Provides a visual WYSIWYG structure with rule entry wizards and drag and drop dictionary components. Will import, analyze and refactor existing XML Schema.

Oracle is a proud sponsor of the project and its use on the NIEM.gov initiative.

Creates XSD schema + JAXB bindings, Mindmap or UML models (XMI), XML test suite examples, HTML documentation + spreadsheets (NIEM IEPDs). XSD schema export in default, flatten, NIEM, and OASIS modes. Generates canonical component dictionaries from schema sets, ERwin models, or spreadsheets.

Friday Jun 08, 2012

CAMツールキットは、XMLビジネス情報交換を開発し、管理するプロセスを劇的に簡素化するための "WYSIWYG"ヴィジュアル構造化エディターを使用して直感的なアプローチを提供する。

CAMエディターは、XML交換を組み立て、配置するための先端的で、オープンな基本ツールキットである。CAMツールキットは、XMLビジネス情報交換を開発し、管理するプロセスを劇的に簡素化するための "WYSIWYG" ヴィジュアル構造化エディターを使用して直感的なアプローチを提供する。このことは、開発者に、安定し、相互運用性があり、信頼性のある交換のために必要な管理、洞察そして分析を提供する。

主要な特徴は以下の通りである:

  • 構成要素集辞書からXML交換構造をヴィジュアル・ドラッグ&ドロップによる作成
  • XSDスキーマとしてデータモデルをインポート、及び WSDLを生成し又はXML交換を強化
  • NIEM, OASIS, WSDLのようなプロファイルを用いて、 モデル準拠のXMLスキーマの生成
  • スキーマ設計品質評価の評価報告
  • 正規モデル辞書の管理と生成
  • 業界標準 XML枠組みの支援
  • UML/XMIによる交換のモデル化
  • ランタイム実証エンジン “CAMV Java モジュール”
  • コードリストとSQLテーブル検索 評価ツールとランタイム CAMV
  • XPath 表現テンプレートを用いて、業務論理ルールの評価
  • 内容の暗示により、XMLインスタンス例を生成
  • CAMVの “ANT スクリプティング” を用いた XMLテストスーツ
  • XMLエディター及び統合開発環境
  • メニューと書式に関する各種言語の支援

CAMエディターは、現在のXMLスキーマ交換をより良い互換性を保ってインポート、分析そして反応できる、かつ、ミドルウェアで使用できる。完全なXML交換構造、規則及び文書化の詳細は、OASIS CAM標準XMLテンプレート書式で保存されている。 OASIS CAM は、公開のオープンな標準である。

CAMツールキットはまた、業務の文書化、相互参照表、モデル、XMLスキーマやテストXMLインスタンス等を支援する加工品を生成する作業を自動化する。このことは、 NIEM, OASIS他のための拡張プロファイルによる情報交換統合のためのNIEM アプローチと互換性がある。

CAMアプローチは、正規辞書群から視覚的なドラッグ&ドロップ設計やコアコンポーネントメッセージアッセンブリを可能とする。また、供給されたものは、現在のXMLスキーマライブラリ又は企業データモデル化ツールから正規辞書を採取、生成するためのツールのセットである。

独立CAMV評価エンジンは、Javaで書かれた、基礎としてOASIS CAM仕様を使用して、XML評価枠組みを実施する。また、サービス指向アーキテクチャー (SOA) と共に統合化のために設計された、CAMV XML評価枠組み支援は、 企業アプリケーション統合 (EAI), LEXS (論理的エンティティ交換システム) や ebXML メッセージングシステムの様な他のメッセージを基礎とする統合化パターンとの使用を支援するものである。

Download 

Monday May 21, 2012

CAM v2.2 release with multi-lingual support taking NIEM international

New release now available with French, Spanish, Russian, Norwegian, Chinese and Japanese support from http://www.cameditor.org


Complete highlights include the following:
o International localization language support for menus and screens
o JAXB binding and ConTessa support for XSD schema export
o Better XSD schema importing and exporting
o Dictionary collection manager wizard
o Better dictionary namespace handling
o Better UML physical model export to XMI 2.1.2 (now works with popular UML tools)
o CAMV validation engine support for W3C DOM and extended API options

Notice the Dictionary Collection Manager tool uses enhanced namespace techniques. Please download the latest NIEM dictionary pack from Sourceforge files to take advantage of these.

For each language we are establishing a resource page and translation:

o French XML editor details

o Spanish XML editor details

o Russian XML editor details

o Norwegian XML editor details

o Chinese XML editor details

o Japanese XML editor details

Thursday Jul 28, 2011

CAM editor tutorial video launches

To accompany the new CAM editor v2.0 release we now have an 8 minute online video. The video provides a quick introduction to the features and capabilities in the latest CAM editor release made available for community use courtesy of Oracle's media folks.

 The demonstration steps through a scenario of enhancing an existing OASIS HAVE schema to add new components and then generate the resulting updated exchange schema. The tutorial inserts new NIEM core components to the existing OASIS schema via drag and drop from the dictionary - showing you how you can tailor or build a new NIEM exchange from existing library components - and/or add your own components and rules.

To access the tutorial - see this link and just enter in an email address to launch the video page - http://launch.oracle.com/?NIEM

Oracle also has a new NIEM focus page with more resources and information: http://www.oracle.com/goto/niem

The main CAM editor download page is http://www.cameditor.org

Thursday Jun 23, 2011

CAM v2.0 ships – all new foundation version

The latest release of the CAM editor toolset is now available on Sourceforge.net – search NIEM. In this all new version the support from Oracle has enabled a transformation of the editor underpinning Java framework and results in 3x performance improvement and 50% better memory utilization. The result of nearly six months of improvements are catalogued in the release notes.[Read More]
About

Not all XML is created equal. XML Orb looks at the challenges of creating information exchanges with XML and NIEM and how this can be made simpler, comprehensible, consistent and reliable.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today