Dancing with xslt: wadm scripting for Sun Java System Web Server 7.0 (part IX)

Though generally wadm returns data as a tcl list that can be directly used, there are
multiple instances where more interesting data are made available as Xml.

Eg:

|get-stats-xml --config=test --node=agneyam
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE stats SYSTEM "sun-web-server-stats_7_0_0.dtd">
<stats versionMajor="1" versionMinor="2" enabled="1">
    <server id="https-test" versionServer="Sun Java System Web Server 7.0-Technology"  .... >
        <connection-queue id="cq1"/>
        <thread-pool id="thread-pool-0" name="NativePool"/>
        <thread-pool id="thread-pool-1" name="TestPool"/>
        <profile id="profile-0" name="all-requests" description="All requests"/>
        <profile id="profile-1" name="default-bucket" description="Default bucket"/>
        <profile id="profile-2" name="cache-bucket" description="Cached responses"/>
......
</stats>


More over the main configuraion file is also in xml.
|get-config-file --config=test server.xml
<?xml version="1.0" encoding="UTF-8"?>
<server>
...
</server>

There may be times when the normal manipulation of configuration using wadm just dont
cut it and you may wish to manipulate the server.xml directly.

Asking TCL to help

Looking at the tcl syntax and the xml syntax again, we see that the xml looks pretty similar
infact An xml is nothing but a hierarchy of lists. (This is nothing new, and has been beaten to
death by the lisp folks -- the other language that uses lists as the mainstay.)

We can represent an xml like this
            <thread mode="idle" timeStarted="1161917015" connection-queue="cq1">
                <request-bucket countRequests="0" countBytesReceived="0" countBytesTransmitted="0"/>
                <profile-bucket profile="profile-0" countCalls="0" />
                <profile-bucket profile="profile-1" countCalls="0" />
                ABCD
            </thread>
in tcl syntax as a list as below
         thread { :mode "idle" :timestarted "
1161917015" :connection-queue "cq1"
                  request-bucket { :countRequests "0" :countBytesReceived "0" :countBytesTransmitted "0"
                                           profile-bucket { :profile "profile-0" :countCalls "0"
                                          }
                                           profile-bucket {:profile "profile-1" :countCalls "0"
                                          }
                  }
                  % {ABCD}
         }


There are multiple packages available in tcl that can parse the xml (this being one of
them), but it is not necessary for us to take that approach. When the language is xml,
it is needlessly complicated to parse the xml on our own, instead, It would be nice
if there is some tool that would allow us to just specify what we want and where,
and convert the xml to that format. In short a macro expander.

There is a language that fits the bill. XSLT

XSLT

Usint XSLT we can transform the xml into tcl source. It allows all the necessary primitives
    1) fetch the values of elements and attributes
    2) recursively iterate over xml

    3) Compute results and process conditionaly when necessary.

First try.

Let us try and match all the elements.
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output method="text" encoding="UTF-8"/>

    <xsl:template match="\*">
        <xsl:value-of select="name()"/>
        {
        <xsl:apply-templates select="\*"/>
        }
    </xsl:template>
</xsl:stylesheet>


The infrastructure.

======================transform.tcl=====================
namespace eval Transform {
    java::import javax.xml.parsers.SAXParserFactory
    java::import javax.xml.transform.Source
    java::import javax.xml.transform.TransformerFactory
    java::import javax.xml.transform.Transformer
    java::import javax.xml.transform.stream.StreamSource
    java::import javax.xml.transform.stream.StreamResult
    java::import javax.xml.transform.sax.SAXSource
    java::import org.xml.sax.XMLReader
    java::import java.io.File
    java::import java.io.ByteArrayOutputStream
    java::import java.io.ByteArrayInputStream

    set xsl {<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="text" encoding="UTF-8"/>
    <xsl:template match="\*">
        <xsl:value-of select="name()"/>
        {
        <xsl:apply-templates select="\*"/>
        }
    </xsl:template>
</xsl:stylesheet>
}

    proc get_source {reader arg} {
        set stream [java::new ByteArrayInputStream [[java::new String $arg] getBytes]]
        set source [java::new StreamSource $stream]
        return [java::new SAXSource $reader [java::call SAXSource sourceToInputSource $source]]
    }

    proc xslt {xml} {
        set  pfactory [java::call SAXParserFactory newInstance]
        $pfactory setValidating 0
        set reader [[$pfactory newSAXParser] getXMLReader]
        set trans [java::call TransformerFactory newInstance]
        set xsl_trans [$trans newTransformer [get_source $reader $Transform::xsl]]
        set bo [java::new ByteArrayOutputStream]
        $xsl_trans transform [get_source $reader $xml] [java::new StreamResult $bo]
        return [$bo toString]
    }
}
===================================================================
|source xslt.tcl                                             
|Transform::xslt [get-stats-xml --config test --node agneyam]
ERROR:  'Content is not allowed in prolog.'
ERROR:  'com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: Content is not allowed in prolog.'
javax.xml.transform.TransformerException: javax.xml.transform.TransformerException: com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: Content is not allowed in prolog.


Now, this means that we wont be able to use the xml directly, We need to strip out the starting and
ending blank lines, the DocType tag (to avoid validation) etc.
Cleaning up the xml file,
===================================================================
    proc clean_xml in {
        set out {}
        regsub {<!DOCTYPE [\^>]+>} $in {} _out
        foreach {line} [split $_out "\\n"] {
            switch -regexp -- $line {
                {\^$} {} ;#strip blanks.
                default { append out $line "\\n" }
            }
        }
        #some command return xml with in a list
        #ugly but we dont have an option.
        if {[llength $out] == 1} {
            set out [lindex $out 0]
        }
        return $out
    }

===================================================================
|source xslt.tcl                                            
|Transform::xslt [get-stats-xml --config test --node agneyam]
....
    cpu-info
        {
        }
    cpu-info
        {
        }
        }
        }
Seems to be working..
Checking if it is the syntax we are interested in,
|array set stats_root [Transform::xslt [get-stats-xml --config test --node agneyam]]
|array names stats_root                                                            
stats
|array set stats $stats_root(stats)
|array names stats
server
|array set server $stats(server)
|array names server
auth-db audit-accesses connection-queue profile cpu-info http-listener access-log temp-path thread-pool log acl-file jvm default-auth-db-name process http cluster virtual-server mime-file user

The tcl list still needs more things to make it completely representative of the xml it parsed.
adding those to the xsl,

<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="text" encoding="UTF-8"/>

    <xsl:template match="@\*">
        :<xsl:value-of select="name()"/> "<xsl:value-of select="."/>"
    </xsl:template>
    <xsl:template match="\*">
        <xsl:value-of select="name()"/>
        {
        <xsl:apply-templates select="@\*"/>
        <xsl:apply-templates select="\*"/>
        <xsl:choose>
            <xsl:when test='string-length(translate(translate(text(),"&#10;","")," ","")) &gt; 1'>
            % {<xsl:value-of select="translate(text(),'&#10;','')"/>}
            </xsl:when>
        </xsl:choose>
        }
    </xsl:template>
</xsl:stylesheet>

|array set stats_root [Transform::xslt [get-stats-xml --config test --node agneyam]]
|array names stats_root                                                            
stats
|array set stats $stats_root(stats)                                                
|array names stats                                                                 
:enabled :versionMajor server :versionMinor
|puts $stats(:enabled)
1
|array set names $stats(server)
|array names names
:secondsRunning connection-queue :load5MinuteAverage profile cpu-info :maxThreads
:maxProcs thread-pool :rateBytesReceived :timeStarted :load15MinuteAverage
process :load1MinuteAverage :id :rateBytesTransmitted :versionServer virtual-server
:ticksPerSecond :flagProfilingEnabled

|puts $names(profile)
        :id "profile-2"
        :name "cache-bucket"
        :description "Cached responses"

While this seems to be fine, Accessing it through setting and accesssing arrays
does not seem to be the friendliest or the most powerful approach,we should
provide an API to access the values.

Creating an API.

What we need is an xpath like expression that allow arbitrary tcl statements to
be used for filtering. It should be some thing like below,
-- {stats server {profile {~ \*1\* $id} }}
where stats/server/profile is the node of the xml we are looking for, and
~ \*1\* $id is the filter which will do a pattern match on the value of attribute
with name id and return only those nodes that match what we provide.

===================================================================
     #Filter returns true if either of following is satisfied.
    #1) the length of filter list is 0 (no filter)
    #the filter evaluates to true after setting the attributes
    #as variables
    proc filter {filter val} {
        if {![string length $filter]} {
            return 1
        } else {
            #transform ->:id "mexico"<- into ->set id "mexico"<-
            regsub -all {:([a-zA-Z0-9_-]+) } $val {set \\1 } val
            foreach {v} [split $val "\\n"] {
                eval $v
            }
            return [eval $filter]
        }
    }

    #collect all elements of name where filter evaluates to true
    proc fetch_alist {xml name {filter {}}} {
        set res {}
        foreach {n v} $xml {
            if [string match $n $name] {
                if [filter $filter $v] { lappend res $v }
            }
        }
        return $res
    }

    proc helper {lst res} {
        set rest [lrange $lst 1 end]
        if {![llength $rest]} {
            return $res
        } else {
            set out {}
            foreach {r} $res {
                set out [concat $out [get $r $rest]]
            }
            return $out
        }
    }

    proc get {xml lst} {
        set res $xml
        set element [lindex $lst 0]
        switch -exact -- [llength $element] {
            1 {
                #name alone. no filter
                #collect the corresponding lists from $xml
                set res [fetch_alist $res $element]
                return [helper $lst $res]
            }
            default {
                #use filter too.
                set res [fetch_alist $res [lindex $element 0] [lindex $element 1]]
                return [helper $lst $res]
            }
        }
    }
===================================================================

Using It,

(The regexp command is aliased to ~ and string match is aliased to ? -- see full implementation.)
|set xml [Transform::xslt [get-stats-xml --config test --node agneyam]]
|Transform::get $xml {stats server {profile {~ 1 $id} }}
{
        :id "profile-1"
        :name "default-bucket"
        :description "Default bucket"
        }
|Transform::get $xml {stats server {profile {? \*1\* $id} }}
{
        :id "profile-1"
        :name "default-bucket"
        :description "Default bucket"
        }

|Transform::get $xml {stats server {profile {? \* $id} }} 
{
        :id "profile-0"
        :name "all-requests"
        :description "All requests"
        } {
        :id "profile-1"
        :name "default-bucket"
        :description "Default bucket"
        } {
        :id "profile-2"
        :name "cache-bucket"
        :description "Cached responses"
        }

The finished transform.tcl is available here
I will cover using Xpath to access the same data in a later post.

Comments:

Hi I having a problem processing XSLT on unix at the moment, it works fine on windows but when I try process the same file on Unix I keep getting the same error. I'm wondering does anyone know is there a difference between the two operating systems that maay cause this problem....if so have you any solutions Most current operating systems have an XSLT processor installed. For example, Windows XP comes with the MSXML3 library, which includes an XSLT processor. Earlier versions may be upgraded and there are many alternatives, see the External Links section. Seems like JMeter is generating some dodgy XML, but not sure why that wouldn't fail on Windows also JMeter uses an XSLT file, jmeter-detail-report.xsl to transform the JMeter JTL output into a nicely formatted html table of results. The version of the file which I run with ant-jmeter (as a task in build.xml) runs well on Windows but fails on Unix with the following exception: [xslt] Processing /export/home/sfuser/sola/JMeter/JMeterResults.jtl to /export/home/sfuser/sola/JMeter/JMeterResults_detail.html [xslt] Loading stylesheet /export/home/sfuser/sola/JMeter/jmeter-results-detail-report.xsl [xslt] : Error! Content is not allowed in prolog. [xslt] : Error! com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: Content is not allowed in prolog. [xslt] Failed to process /export/home/sfuser/sola/JMeter/JMeterResults.jtl BUILD FAILED /export/home/sfuser/sfv4-client/staging/sfv4client_jmeter.xml:132: javax.xml.transform.TransformerException: javax.xml.transform.TransformerException: com.sun.org.apache.xml.internal.utils.WrappedRuntimeException: Content is not allowed in prolog. I tried removing spaces, changed the xml prolog in the xsl file, etc but it still doesn’t work. Could you suggest what I can do to fix the problem? cheers flannan

Posted by fbrody on December 14, 2006 at 10:22 PM IST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

blue

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today