Monday Dec 07, 2009

Working for Sun, Oh, the places I have been

Working for Sun, Oh, the places I have been

There was fun to be done,
There was Java versus .Net to be won.
There was butt to kick!
There was .com and startups making $$ really quick

I joined the high fliers
who soared to high heights and saw some great sights

How much can you win?
We were the best of the best, on top of all the rest.
then Bang-ups and Hang-ups did happen
How much can you lose?
We got all hung up in a prickle-ly perch.
and then all of a sudden we were left in a Lurch.

The EU took us
toward a most useless place.

The Waiting Place
for people just waiting.
waiting around for a Yes or No
Simple it's not, I'm afraid you will find,
for a mind-maker-upper to make up his mind.

I made MY decision,
I decided not to stay
and I found another job
so, I will be on my way!

And the Tech Days gang will fly on.
But I'm not crying because it's over. I'm smiling because it happened.

Places I have been with Sun (some more than once):
Amsterdam, London, Paris, Nice, Antwerp, Frankfurt, Berlin, Cologne, Athens, Budapest,  Dublin, Warsaw, Madrid, Barcelona, Istanbul, Israel, Johannesburg, Sydney, Beijing, Shanghai, St.Petersburg, Hyderabad, Tokyo, Manila, Singapore, Taipei, Mexico city, Sao Paulo, Brasilia, Austin, San Francisco, New York,  DC,  Atlanta, Miami, Orlando, Manchester, Boston, Philadelphia, Raleigh, Charlotte, Memphis, Dallas,  Seattle and probably some I forgot. 

It's been really great working for Sun on and off for 8 yrs.
In my new job I will be working for http://availity.com/  Availity connects health care professionals with a smarter, online approach to electronic claims and other daily health plan transactions.

Wednesday Dec 02, 2009

Wicket, JPA, GlassFish and Java Derby or MySQL

This Pet Catalog app explains a web application that uses Wicket, JPA, GlassFish and MySQL. I took this example JSF 2.0, JPA, GlassFish and MySQL and modified it to use Wicket instead of JSF. 

Explanation of the usage of Wicket, Java Persistence APIs, Glassfish and MySQL in a sample Store Catalog Application

The image below shows the Catalog Listing page, which allows a user to page through a list of items in a store.






The Wicket Framework



The Wicket Web Framework provides a component-oriented programmatic manipulation of HTML markup. The Diagram below shows how the Wicket Framework fits into the MVC design pattern.




A Wicket component controller receives user input. The component uses the user input to interact with the model, to handle page navigation and events. The Model provide components with an interface to domain data. The Component view renders the User Interface HTML elements. The controller and view parts are combined into the Component  in Wicket.


Wicket Triad of HTML Templates, Components, and Model


The Wicket component, the model, and HTML template work together. The HTML template defines the static parts of the pages, components fill in the dynamic parts, and models are used by components to get the domain data for the dynamic parts.  Wicket matches wicket:id tags with components, Models are a reference to the data for the Java components.

Wicket  HTML Templates


Wicket templates are written in plain HTML, consumable by standard HTML editors. Wicket uses HTML namespace standards to extend HTML with placeholders,  where Wicket components are hooked in, so that there is no custom syntax at all. Wicket matches wicket: id attributes and attaches Java components to the tags in which these attributes are defined.

Wicket Web pages are all Java classes that map to an HTML template, with the same name as the Java class.




The ListPage.html file holds the markup for the ListPage.java.  Wicket matches the Java page instance and the HTML template file with the same name in the same Java package.

Code Sample from:  ListPage.html

<table>
  <tr>
   <th> Name </th>
   <th> Photo</th>
   <th> Price </th>
  </tr>
  <tr wicket:id="rows" >
     <td>
        <a wicket:id="details" href="#">
         <span wicket:id="name">name</span>
        </a>
     </td>
     <td> <img wicket:id="photo"/></td>
     <td><span wicket:id="price"></span></td>

   </tr>
</table>
<span wicket:id="pager">navigation controls here</span>



Pages are special top-level components that hold the root for component trees. Below is the component tree for the ListPage above:



Wicket Components


Wicket matches the wicket:id="rows" attribute in the ListPage.html with the corresponing Java component in ListPage.java,  which is a DataViewComponent which has a DataProvider model as shown in the diagram below.


The corresponding ListPage.java code is shown below:

Code Sample from: ListPage.java



public class ListPage extends WebPage {

// create the Model DataProvider
        IDataProvider itemDataProvider = new IDataProvider<Item>() {

            public Iterator iterator(int first, int count) {
                return itemController.findItemEntities(count, first).iterator();
            }
            public int size() {
                return itemController.getItemCount();
            }
            public IModel model(final Item object) {
                return new LoadableDetachableModel() {
                    @Override
                    protected Item load() {
                        return (Item) object;
                    }
                };
            }
            public void detach() {
            }
        };

// create the DataView component for the wicketid "rows" attribute in ListPage.html
        DataView dataView = new DataView<Item>("rows", itemDataProvider, ROWS_PER_PAGE) {

            @Override
            protected void populateItem(org.apache.wicket.markup.repeater.Item<Item> repItem) {
                Item item = (Item) repItem.getModelObject();
                repItem.setModel(new CompoundPropertyModel<Item>(item));
                repItem.add(ItemDetails.link("details", item));
                repItem.add(new Image("photo", new ResourceReference(this.getClass(),  item.getImagethumburl()))); 
                repItem.add(new Label("price"));

            }
        };
// add the DataView component to the page
        add(dataView);
// create the PagingNavigator component for the "pager" attribute in ListPage.html
        PagingNavigator pager = new PagingNavigator("pager", dataView);
        add(pager);



DataView is a component which makes it simple to populate a Wicket RepeatingView from a database by utilizing IDataProvider to act as an interface between the database and the dataview.  A Wicket RepeatingView renders the components added to it from data from a collection of objects.

The  DataView instantiation passes to the constructor  the wicket id "rows" , the itemDataProvider, and the number of rows for paging, and
anonymously overrides the populateItem() method, which will be called for each Item object provided by itemDataProvider.  The  populateItem method adds the child components ( Link, Image, Label)  with the Item  Model to the Dataview.

The itemDataProvider IDataProvider provides the Pet Catalog  Item data to the DataView.  The IDataProvider iterator(int first, int count) method gets an iterator for the subset of total data.  The itemDataProvider iterator method calls the  itemController.findItemEntities , which uses JPA to query the database and return a list of Item entities.  The itemDataProvider model method  is a Callback used by the consumer of this data provider to wrap objects retrieved from iterator(int, int) with a model (usually a detachable one).




Using the Java Persistence API (JPA) with Wicket

The ItemJpaController findItemEntities method is defined as shown below:

Code Sample from: ItemJpaController.java

public class ItemJpaController {

    public ItemJpaController() {
        emf = Persistence.createEntityManagerFactory("wicketCatalogPU");
    }
    private EntityManagerFactory emf = null;

    public EntityManager getEntityManager() {
        return emf.createEntityManager();
    }
    private List<Item> findItemEntities(boolean all, int maxResults, int firstResult) {
        EntityManager em = getEntityManager();
        try {
            Query q = em.createNamedQuery("Item.findAll");
            if (!all) {
                q.setMaxResults(maxResults);
                q.setFirstResult(firstResult);
            }
            return q.getResultList();
        } finally {
            em.close();
        }
    }





The ItemJpaController uses the Java Persistence API EntityManager Query object to return a list of items. The ItemJpaController calls Persistence.createEntityManagerFactory which gets an EntityManagerFactory  when it is instatiated. 

The Java Persistence Query APIs are used to create and execute queries that can return a list of results.  The JPA Query interface provides support for pagination via the setFirstResult() and setMaxResults() methods: q.setMaxResults(int maxResult) sets the maximum number of results to retrieve. q.setFirstResult(int startPosition) sets the position of the first result to retrieve.

In the code below, we show the Item entity class which maps to the  ITEM table that stores the item instances. This is a typical Java Persistence entity object. There are two requirements for an entity:
  1. annotating the class with an @Entity annotation.
  2. annotating   the primary key identifier with @Id
Because the fields name, description.... are basic mappings from the object fields to columns of the same name in the database table, they don't have to be annotated. 

Code Sample from: Item.java

@Entity
public class Item implements java.io.Serializable {

@Id
    private Integer id;

    private String name;   
    private String description;   
    private String imageurl;   
    private String imagethumburl; 
    private BigDecimal price;


    public Item() { }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }


    ...
}   


A Wicket PagingNavigator component is used to provide navigation links to the next, previous, first and last page of catalog Items.  The PagingNavigator component maintains a complete page navigator, meant to be easily added to any PageableListView.

Code Sample from: ListPage.html

<span wicket:id="pager">navigation controls here</span>


Code below for adding creating a PagingNavigator for the dataView and adding it to the ListPage:

Code Sample from: ListPage.java


public class ListPage extends WebPage {


PagingNavigator pager = new PagingNavigator("pager", dataView);
add(pager);




A Wicket Link component is used to provide a link to click on to navigate to a page with the selected item details.

Code Sample from: ListPage.html

<a wicket:id="details" href="#">
        <span wicket:id="name" >name</span>
  </a>



Code below show the call to  the ItemDetails.link method to create and add the link:

Code Sample from: ListPage.java


public class ListPage extends WebPage {

// create the DataView component corresponding to the wicketid "rows" attribute in ListPage.html
        DataView dataView = new DataView<Item>("rows", itemDataProvider, ROWS_PER_PAGE) {
            @Override
            protected void populateItem(org.apache.wicket.markup.repeater.Item<Item> repItem) {
           '    ...
// call ItemDetails to create the link component
                repItem.add(ItemDetails.link("details", item));
...

            }
        };
// add the DataView component to the page
        add(dataView);



Code below for creating a BookmarkablePageLink for adding to the dataView, clicking on this link will Navigate to the ItemDetails page, passing the selected Itemid as a parameter.

Code Sample from: ItemDetails.java

public class ItemDetails extends BasePage {

        public static BookmarkablePageLink<Void> link(final String name, final Item item) {

        final BookmarkablePageLink<Void> link = new BookmarkablePageLink<Void>(name, ItemDetails.class);

        if (item != null) {
            link.setParameter("itemid", item.getItemid());
            link.add(new Label("name", new Model<Item>(item)));
        }

        return link;
    }


The ItemDetails page, shown below, displays details about the selected Catalog Item:

Code Sample from: ItemDetails.html

<html xmlns:wicket="http://wicket.apache.org/">
    <head>
        <title></title>
        <link wicket:id='stylesheet'/>
    </head>
    <body>
        <span wicket:id='mainNavigation'/>
        <table>
            <tr>
                <td align="right">Name:</td>
                <td>
                    <span wicket:id="name">name </span>
                </td>
            </tr>
            <tr>
                <td align="right">Description:</td>
                <td> <span wicket:id = "description">
                  description
                    </span>
                </td>
            </tr>
            <tr>
                <td align="right">Photo:</td>
                <td> <img wicket:id="imagethumburl"/>   </td>
            </tr>
        </table>
    </body>
</html>




The ItemDetails constructor gets the item data, and adds Labels and a image, for the name, description and photo to the ItemDetails page.

Code Sample from: ItemDetails.java

public class ItemDetails extends BasePage {

    public ItemDetails(PageParameters params) {
        Item item = itemController.findItem(params.getString("itemid"));
        add(new Label("name", item.getName()));
        add(new Label("description", item.getDescription()));
        add(new Image("imagethumburl", new ResourceReference(this.getClass(),item.getImageurl())));
    }






Hot Deployment with Wicket and Glassfish

  • Incremental compile of all Wicket  artifacts when you save.
  • Auto-deploy of all web artifacts

Conclusion
This concludes the sample application which demonstrates a pet catalog web application which uses  Wicket, JPA, GlassFish and MySQL.

Running the Sample Application

  1. If you haven't already done so, download and install NetBeans IDE , GlassFish , and MySQL Community Server . You can download and install GlassFish with NetBeans as a single bundle.
  2. Follow these instructions to setup Netbeans with the Wicket plugin.
  3. Download the sample code.

Create the Pet Catalog database

In order to run the sample code you first have to create the Pet Catalog database and fill in  the Item table.

  1. Start NetBeans IDE
  2. Ensure that GlassFish is registered in the NetBeans IDE, as follows:
    • Click the Services tab in the NetBeans IDE.
    • Expand the Servers node. You should see GlassFish v2 in the list of servers. If not, register GlassFish v2 as follows:
      • Right-click the Servers node and select Add Server. This opens an Add Server Instance wizard.
      • Select GlassFish v2 in the server list of the wizard and click the Next button.
      • Enter the location information for the server and click the Next button.
      • Enter the admin name and password and click the Finish button.
  3. Start the MySQL or Java DB database as follows:
    • Click the Services tab in the NetBeans IDE.
    • Expand the databases node. You should see the Java DB database in the list of databases. If you have installed the MySQL server database, you should also see the MySQL database in the list of databases.. Note:  Java DB  comes bundled with Netbeans, you can  download MySQL separately.

    • Right-mouse click on the Java DB or MySQL server database and select Start.
  4. If you installed MySQL, set the properties of the MySQL server database as follows:
    • Right-click on the MySQL server database and select Properties. This opens the MySQL Server Properties dialog box, as shown in Figure 8.

      MySQL Server Basic Properties
      Figure 8. MySQL Server Basic Properties

    • In the Basic Properties tab, enter the server host name and port number. The IDE specifies localhost as the default server host name and 3306 as the default server port number.
    • Enter the administrator user name, if not displayed, and the administrator password -- the default administrator password is blank.
    • Click the Admin Properties tab.
    • Enter an appropriate path in the Path/URL to admin tool field. You can find the path by browsing to the location of a MySQL Administration application such as the MySQL Admin Tool.
    • Enter an appropriate path in the Path to start command. You can find the path by browsing to the location of the MySQL start command. To find the start command, look for mysqld in the bin folder of the MySQL installation directory.
    • Enter an appropriate path in the Path to stop command field. You can find the path by browsing to the location of the MySQL stop command. This is usually the path to mysqladmin in the bin folder of the MySQL installation directory. If the command is mysqladmin, in the Arguments field, type -u root stop to grant root permissions for stopping the server. The Admin Properties tab should look similar to Figure 9.

      MySQL Server Administration Properties
      Figure 9. MySQL Server Administration Properties

    • Click the OK button.

  5. Right-click on the MySQL server or Java DB database and select Start.
  6. Create the petcatalog database as follows:
    • Right-mouse click on the Java DB or MySQL server database and select Create Database. This will open a create Database window.
    • Enter the database name catalog for Java DB or petcatalog for MySQL.


      For Java DB enter userid password app app as shown below:


       Click O.K. to accept the displayed settings.
  7. Create the tables in the catalog database as follows:
    • Underneath Databases you should see a database connection for the petcatalog database. For example MySQL:

      or Java DB:

    • Right-mouse click on the petcatalog connection and select Connect.
    • Right-mouse click on the petcatalog connection and select Execute Command. This will open up a SQL command window.
    • Copy the contents of the catalog.sql file in the riapetcatalog\\exercises\\exercise0 directory and paste the contents into the SQL command window, as shown in below:

      Creating Tables in the Database
    • Click the Run SQL icon Run SQL icon (Ctrl+Shift+E) above the SQL command window.
    • Note: It is ok to see this: "Error code -1, SQL state 42Y55: 'DROP TABLE' cannot be performed on 'ITEM' because it does not exist. Line 2, column 1" . This just means you are deleting a table that does not exist.  If you need to delete and recreate the tables you will not  see this message the second time.
  8. View the data in the Pet Catalog database Item table as follows:
    • Underneath Databases you should see a database connection for the petcatalog database. For example MySQL:

      or Java DB:

    • If the database connection is broken like in the following diagram:

      • Right-mouse click on the petcatalog connection and select Connect. as shown below:

      • if prompted for a password, for MySQL leave it blank, for JavaDB enter user app password app.
    • Expand the Tables node below the petcatalog database in the Services window. You should see the item table under the Tables node. You can expand the item table node to see the table columns, indexes, and any foreign keys, as shown in below :
      An Expanded Table Node
      Figure 12. An Expanded Table Node

      You can view the contents of a table or column by right-clicking the table or column and selecting View Data as shown  below:

      Viewing the Contents of a Table
      Figure 13. Viewing the Contents of a Table


  9. Follow these instructions to Create a JDBC Connection pool and JDBC resource.Name the  pool mysql_petcatalog_rootPool and the jndi resource jdbc/petcatalog. Note: you do not have to create a JDBC connection pool and resource if you use the Netbeans wizard to generate JPA entities from database tables as described in this article GlassFish and MySQL, Part 2: Building a CRUD Web Application With Data Persistence.

Running the Sample solution:

If you want to run the sample solution, you have to create the catalog database tables first as described above.

  1. If you haven't already download the sample code and start the NetBeans IDE. Unzip the catalog.zip file which you downloaded, this will create a catalog directory with the project code.
  2. Open the catalog/setup/sun-resources.xml file and verify that the property values it specifies match those of the petcatalog database and jdbc resources you created. Edit the property values as necessary.

  3. Open the catalog project as follows:
    • In NetBeans IDE, click Open Project in the File menu. This opens the Open Project dialog.
    • Navigate in the Open Project dialog to the catalog  directory and click the Open Project button.

    In response, the IDE opens the catalog project.  You can view the logical structure of the project in the Projects window (Ctrl-1).
  4. Run the catalog by right-clicking on the catalog project in the Projects window and selecting Run Project. The NetBeans IDE compiles the application, deploys it on Glassfish, and brings up the default page in your browser.  (at http://localhost:8080/catalog/).

For more information see the following resources:


Thursday Nov 12, 2009

Java Garbage Collection, Monitoring and Tuning

Java garbage collection, monitoring and tuning Yesterday I gave a talk at a the Jacksonville JUG about Java garbage collection, monitoring and tuning, which included a demo of Finding Memory Leaks Using the NetBeans Profiler and a demo of the  VisualGC plugin for  VisualVM

beach.jpg

You can view or download the presentation here

Java garbage collection, monitoring and tuning

References and More Information:



Friday Nov 06, 2009

Web Site Performance Tips and Tools

The Top 10 Web Application security vulnerabilities

Web Site Performance Tips

  • Make Fewer HTTP Requests
    • reduce the number of components in a page by combining CSS into single stylesheet,  JS single script 
  • Use A Content Delivery Network
    • such as Akamai Technologies, Mirror Image Internet, or Limelight Networks. 
  • Add An Expires Header
    • for static components set a far future expires header
    • for dynamic us a cache-control header 
  • GZIP Components
    • reduces response times by reducing the size of the HTTP response.
  • Put Stylesheets At The Top
    • allows the page to display whatever content it has as soon as possible. 
  • Put Scripts At The Bottom
    • scripts block parallel downloads, which slows rendering of page
  • Avoid CSS Expressions For Dynamic Properties
    • they may be evaluated thousands of times and could affect the performance of your page.
  • Make JS and CSS External
    • if multiple pages re-use the same scripts and stylesheets, then making JS and CSS external files produces faster pages because the files are cached by the browser.
  • Reduce DNS Lookups
    • Reducing the number of unique hostnames used in the page's URL, images, script files, stylesheets.. reduces the number of DNS lookups but also reduces the amount of parallel downloading.  Split these components across at least 2 but no more than 4 hostnames.
  • Minify JavaScript
    • remove unnecessary characters from code to reduce its size  : remove whitespace, comments... Two popular tools for minifying JavaScript code are  JSMin and YUI Compressor. 
  • Avoid Redirects
    • redirects are slow, using Alias and mod_rewrite is faster
  • Remove Duplicate Scripts
  • Configure ETAGS
    • ETag is a string that uniquely identifies a specific version of a component. Used in conditional GET requests, browser uses the If-None-Match header. When ETags match, a 304 status code is returned reducing the response 
  • Optimize Images
  • Reduce Cookie Size
    • Information about cookies is exchanged in the HTTP headers between web servers and browsers.  Keep the size of cookies as low as possible to minimize the impact on the user's response time.
  • Avoid 404 errors
  • JavaScript:
    • make the AJAX responses cacheable, add an Expires or a Cache-Control Header. 
    • write efficient JS
    • reduce DOM elements
    • Make the messages between server and browser as small as possible
    • Don't rewrite the server application in javascript
  • Only Generate Content when Things Change
    • Break the system into components so that you can isolate the costs
      of things that change rapidly from those that change infrequently.

Web Site Performance Tools

Tools to find out What to Fix

  • HttpWatch
    • HttpWatch is an HTTP viewer and debugger that integrates with IE and Firefox to provide seamless HTTP and HTTPS monitoring
  • Firebug
    • Firebug integrates with Firefox . With Firebug you can edit, debug, and monitor CSS, HTML, and JavaScript live in any web page.
  • Fiddler
    • Fiddler is a Web Debugging Proxy which logs all HTTP(S) traffic between your computer and the Internet.
  • Network Protocol Analyzers:

Tools to find out How to Fix it

  • YSlow
    • YSlow analyzes web pages and suggests ways to improve their performance based on a set of rules for high performance web pages. YSlow is a Firefox add-on integrated with the Firebug web development tool. YSlow grades web page based on one of three predefined ruleset or a user-defined ruleset. It offers suggestions for improving the page's performance, summarizes the page's components, displays statistics about the page, and provides tools for performance analysis, including Smush.it™ and JSLint.
  • Page Speed  http://code.google.com/speed/page-speed/
    • Page Speed is an open-source Firefox/Firebug Add-on. Webmasters and web developers can use Page Speed to evaluate the performance of their web pages and to get suggestions on how to improve them.
  • Pagetest
    • Pagetest allows you to provide the URL of a webpage to be tested. The test will be conducted from the location specified and you will be provided a waterfall of your page load performance as well as a comparison against an optimization checklist.
  • Visual Round Trip Analyzer
    • Microsoft Web page performance visualizer and analyzer
  • neXpert Performance Tool
    • neXpert is an add-on to Fiddler Web Debugger which aids in performance testing web applications.  neXpert was created to reduce the time it takes to look for performance issues with Fiddler and to create a deliverable that can be used to educate development teams.
  • MSFast
    • MSFast is a browser plugin that help developers to improve their code performance by capturing and measuring possible bottlenecks on their web pages.

Other Tools 

  • CSS Sprite Generator  http://spritegen.website-performance.org/
    • CSS Sprites are the preferred method for reducing the number of image requests. Combine your background images into a single image and use the CSS background-image and background-position properties to display the desired image segment.
  • SpriteMe  http://spriteme.org/ (in progress)
  • Hammerhead  http://stevesouders.com/hammerhead/
    • Hammerhead adds a tab to Firebug for measuring the load time of web pages
  • Cuzillion  http://cuzillion.com/
    • Cuzillion is a tool for quickly constructing web pages to see how components interact.
  • Smush.it  http://smush.it/
    • Smush.it uses optimization techniques specific to image format to remove unnecessary bytes from image files
  • http://code.google.com/p/modjsmin/
    • Once installed, all JS files are minified on demand, leaving commented source intact
  • JSLint
    • JavaScript Code Quality Tool

References and More Information:


Thursday Oct 15, 2009

The Top 10 Web Application security vulnerabilities


Yesterday I gave a talk at a the Jacksonville JUG about the  Top 10 most critical web application security vulnerabilities identified by the Open Web Application Security Project (OWASP).
beach.jpg

You can view or download the presentation here

Top 10 Web Security Vulnerabilities




References and More Information:



You can use OWASP's WebGoat to learn more about the OWASP Top Ten security vulnerabilties. WebGoat is an example web application, which has lessons showing "what not to do code", how to exploit the code, and corrected code for each vulnerability.




You can use the OWASP Enterprise Security API Toolkit to protect against the OWASP Top Ten security vulnerabilties.



The ESAPI Swingset is a web application which demonstrates the many uses of the Enterprise Security API.




Thursday Oct 08, 2009

OWASP Top 10 number 3: Malicious File Execution

Number 3 in the Top 10 most critical web application security vulnerabilities identified by the Open Web Application Security Project (OWASP) is Malicious File Execution, which occurs when attacker's files are executed or processed by the web server. This can happen when an input filename is compromised or an uploaded file is improperly trusted.

Examples

  • file is accepted from the user without validating content
  • filename is accepted from the user
In the example below a file name is accepted from the user and appended to the server's filesystem path.
// get the absolute file path on the server's filesystem 
String dir = servlet.getServletContext().getRealPath("/ebanking")
// get input file name
String file = request.getParameter(“file”); 
//  Create a new File instance from pathname string   
File f = new File((dir + "\\\\" + file).replaceAll("\\\\\\\\", "/")); 



If the filename was compromised to  ../../web.xml , it might allow access to web server properties

Malicious File Execution can result in:

  • files loaded from another server and executed within the context of the web server
  • modifying paths to gain access to directories on the web server
  • malicious scripts put into a directory with inadequate access controls

Protecting against Malicious File Execution

  • the Java EE Security Manager should be properly configured to not allow access to files outside the web root.
  • do not allow user input to influence the path name for server resources
    • Inspect code containing a file open, include, create, delete...
  • firewall rules should prevent new outbound connections to external web sites or internally back to any other server. Or isolate the web server in a private subnet
  • Upload files to a destination outside of the web application directory.
    • Enable virus scan on the destination directory.

Java specific Protecting against Malicious File Exection

Use the OWASP ESAPI  HTTPUtilities interface:

  • The ESAPI HTTPUtilities interface is a collection of methods that provide additional security related to HTTP requests, responses, sessions, cookies, headers, and logging.

    The HTTPUtilities getSafeFileUploads method uses the Apache Commons FileUploader to parse the multipart HTTP request and extract any files therein
    public class HTTPUtilities 
    
        public void getSafeFileUploads(java.io.File tempDir,
                                   java.io.File finalDir)
                            throws ValidationException
    


References and More Information:




Friday Oct 02, 2009

Top 10 web security vulnerabilities number 2: Injection Flaws

OWASP Top 10 number 2: Injection Flaws

Number 2 in the Top 10 most critical web application security vulnerabilities identified by the Open Web Application Security Project (OWASP) is Injection Flaws. Injection happens whenever an attacker's data is able to modify a query or command sent to a database, LDAP server, operating system or other Interpreter. Types of injections are SQL, LDAP, XPath, XSLT, HTML, XML, OS command... SQL injection and Cross-Site Scripting account for more than 80% of the vulnerabilities being discovered against Web applications (SANS Top Cyber Security Risks).

SQL Injection Example

Use of string concatenation to build query: SQL Injection can happen with dynamic database queries concatenated with user supplied input, for example with the following query:
 "select \* from MYTABLE where name=" + parameter
if the user supplies "name' OR 'a'='a' " as the parameter it results in the following:
"select \* from MYTABLE where name= 'name' OR 'a'='a'; 
the OR 'a'='a' causes the where clause to always be true which is the equivalent of the following:
"select \* from MYTABLE; 
if the user supplies "name' OR 'a'='a' ; delete from MYTABLE" as the parameter it results in the following:
"select \* from MYTABLE where name= 'name' OR 'a'='a'; delete from MYTABLE;
the OR 'a'='a' causes the where clause to always be true which is the equivalent of the following:
"select \* from MYTABLE; delete from MYTABLE;
some database servers, allow multiple SQL statements separated by semicolons to be executed at once.

SQL Injection can be used to:
  • create , read , update, or delete database data

Protecting against SQL Injection

  • Don't concatenate user input data to a query or command!
    • Use Query Parameter binding with typed parameters, this ensures the input data can only be interpreted as the value for the intended parameter so the attacker can not change the intent of a query.
  • Validate all input data to the application using white list (what is allowed) for type, format, length, range, reject if invalid. (see previous blog entry)
  • don't provide too much information in error messages (like SQL Exception Information, table names..) to the user.

Java specific Protecting against SQL Injection

Don't concatenate user input data to a query or command:

  • Don't do this with JDBC:
    String empId= req.getParameter("empId") // input parameter
    String query = "SELECT \* FROM Employee WHERE 
                         id = '" + empId +"'";  
    
    
  • Don't do this with JPA:
    q = entityManager.createQuery(“select e from Employee e WHERE ”
    		+ “e.id = '” + empId + “'”);

Use Query Parameter binding with typed parameters

  • With JDBC you should use a PreparedStatement and set values by calling one of the setXXX methods on the PreparedStatement object, For example:
    String selectStatement = "SELECT \* FROM Employee WHERE id = ? ";
    PreparedStatement pStmt = con.prepareStatement(selectStatement);
    pStmt.setString(1, empId);
    This sets the first question mark placeholder to the value of the input parameter empId in the SQL command. Any dangerous characters - such as semicolons, quotes, etc.. should be automatically escaped by the JDBC driver.

  • With JPA or Hibernate you should use Named Parameters. Named parameters are parameters in a query that are prefixed with a colon (:). Named parameters in a query are bound to an argument by the javax.persistence.Query.setParameter(String name, Object value) method. For example:
    q = entityManager.createQuery(“select e from Employee e WHERE ”
                 + “e.id = ':
    id'”);
    q.setParameter(“id”,
    empId);
    This sets the id to the empId in the SQL command, again any dangerous characters should be automatically escaped by the JDBC driver.

  • With JPA 2.0 or Hibernate you can use the Criteria API. The JPA 2.0 criteria API providies a typesafe object-based Query API based on a metamodel of the Entity classes, rather than a string-based Query API. This allows you to develop queries that a Java compiler can verify for correctness at compile time. Below is an example using the Criteria API for the same query as before :

    QueryBuilder qb = em.getQueryBuilder();
    CriteriaQuery<
    Employee> q = qb.createQuery(Employee.class);
    Root<
    Employee> e = q.from(Employee.class);
    ParameterExpression<String>
    id = cb.parameter(String.class);

    TypedQuery<
    Employee> query = em.createQuery(
    q.select(e).where(cb.equal(e.get(Employee_.id), id) );
    query.setParameter(
    id, empId);

References and More Information:




Tuesday Sep 29, 2009

The Top 10 Web Application security vulnerabilities starting with XSS

This and the next series of blog entries will highlight the Top 10 most critical web application security vulnerabilities identified by the Open Web Application Security Project (OWASP).

You can use OWASP's WebGoat to learn more about the OWASP Top Ten security vulnerabilties. WebGoat is an example web application, which has lessons showing "what not to do code", how to exploit the code, and corrected code for each vulnerability.




You can use the OWASP Enterprise Security API Toolkit to protect against the OWASP Top Ten security vulnerabilities.



The ESAPI Swingset is a web application which demonstrates the many uses of the Enterprise Security API.



OWASP Top 10 number 1: XSS = Cross Site Scripting

Cross Site Scripting (XSS) is one of the most common security problems in today's web applications. According to the SANS Top Cyber Security Risks, 60% of the total attack attempts observed on the Internet are against Web applications and SQL injection and Cross-Site Scripting account for more than 80% of the vulnerabilities being discovered. You are at risk of an XSS attack any time you put content that could contain scripts from someone un-trusted into your web pages.
There are 3 types of cross site scripting:
  • Reflected XSS: is when an HTML page reflects user input data, e.g. from HTTP query parameters or a HTML form, back to the browser, without properly sanitizing the response. Below is an example of this in a servlet:
     out.writeln(“You searched for: “+request.getParameter(“query”);
                                        
  • Stored XSS: is when an Attacker’s input script is stored on the server (e.g. a database) and later displayed in the web server HTML pages, without proper HTML filtering. Examples of this are in blogs, or forums where users can input data that will be displayed to others. Below is an example of this in a servlet, where data is retrieved from the database and returned in the HTML page without any validation:
    out.writeln("<tr><td>" + guest.name + "<td>" + guest.comment); 
    
  • DOM XSS: is when JavaScript uses input data or data from the server to write dynamic HTML (DOM) elements, again without HTML sanitizing/escaping/filtering.

XSS can be used to:
  • deface web pages
  • hijack user sessions
  • conduct phishing attacks
  • execute malicious code in the context of the user's session
  • spread malware

Protecting against XSS

To protect against XSS all the parameters in the application should be validated and/or encoded before being output in HTML pages.
  • Always validate on the server side for data integrity and security:
    • Validate all input data to the application for type, format, length, range, and context before storing or displaying.
    • Use white-listing (what is allowed), reject if invalid, instead of filtering out black-list (what is not allowed).
  • Output encoding:
    • Explicitly set character encoding for all web pages (ISO-8859-1 or UTF 8):
      <%@ page contentType="text/html;charset=ISO-8859-1" language="java" %>
    • all user supplied data should be HTML or XML entity encoded before rendering.

Java specific Protecting against XSS

Validating Input with Java

  • You can use Java regular expressions to validate input, this example from WebGoat allows whitespace, a-zA-Z_0-9, and the characters - and ,
     
    String regex = "[\\\\s\\\\w-,]\*";
    Pattern pattern = Pattern.compile(regex);
    validate(stringToValidate, pattern);
    
    
  • Use Framework (Struts, JSF, Spring...) validators. With Java EE 6 you can use the Bean Validation Framework to centrally define validation constraints on model objects and with JSF 2.0 to extend model validation to the UI. For example here is a JSF 2.0 input field:
    <h:inputText id="creditCard" value="#{booking.creditCardNumber}"/>
    
    Here is the JSF 2.0 booking Managed Bean using the Bean Validation Framework :
    @ManagedBean
    public class Booking { 
     ... 
     @NotNull(message = "Credit card number is required") 
     @Size(min = 16, max = 16, 
     message = "Credit card number must 16 digits long") 
     @Pattern(regexp = "\^\\\\d\*$", 
     message = "Credit card number must be numeric") 
     public String getCreditCardNumber() { 
     return creditCardNumber; 
     } 
    }
    
    In addition there are new JSF 2.0 Validators:
    • <f:validateBean> is a validator that delegates the validation of the local value to the Bean Validation API.
    • <f:validateRequired> provides required field validation.
    • <f:validateRegexp> provides regular expression-based validation


  • Use the OWASP Enterprise Security API Java Toolkit's Validator interface:
    ESAPI.validator().getValidInput(String context,String input,String type,int maxLength,
       boolean allowNull,ValidationErrorList errorList)
    ESAPI.validator().getValidInput() returns canonicalized and validated input as a String. Invalid input will generate a descriptive ValidationErrorList, and input that is clearly an attack will generate a descriptive IntrusionException.

Output Encoding with Java

  • You can use Struts output mechanisms such as <bean:write… >, or use the default JSTL escapeXML="true" attribute in <c:out … > 
  • JSF output components filter output and escape dangerous characters as XHTML entities.
    <h:outputText value="#{param.name}"/>

  • You can use the OWASP Enterprise Security API Java Toolkit's ESAPI Encoder.encodeForHTML() method to encode data for use in HTML content. The encodeForHTML() method uses a "whitelist" HTML entity encoding algorithm to ensure that encoded data can not be interpreted as script. This call should be used to wrap any user input being rendered in HTML element content. For example:
    <p>Hello, <%=ESAPI.encoder().encodeForHTML(name)%></p>

References and More Information:


Thursday Sep 17, 2009

Some Concurrency Tips

Here is a review of some concurrency tips from Joshua Bloch, Brian Goetz and others.

Prefer immutable objects/data

Immutable objects do not change after construction. Immutable objects are simpler, safer, require no locks, and are thread safe. To make an object immutable don't provide setters/mutator methods, make fields private final, and prevent subclassing. If immutability is not an option, limit mutable state, less mutable state means less coordination.  Declare fields final wherever practical, final fields are simpler than mutable fields.

When threads share mutable data, each thread that reads or writes must coordinate access to the data. Failing to synchronize shared mutable data can lead to atomicity failures, race conditions, inconsistent state, and other forms of non-determinism. These erratic problems are among the most difficult to debug.

Limit concurrent interactions to well defined points, limit shared data, consider copying instead of sharing.

Threading risks for Web applications

A Servlet get, post, service method can be called for multiple clients at the same time. Multi-threaded Servlet Instance and Static variables are shared and therefore if mutable, access must be coordinated. Servlets are typically long-lived objects with a high thread load, if you over-synchronize performance suffers, try to either share immutable (final) data, or don’t share at all, request arguments and local variables are safer.

Hold Locks for as short a time as possible


Do as little work as possible inside synchronized regions.  Move code that doesn't require the lock out of synchronized block, especially if  time-consuming(!).


The Lock interface provides more extensive locking operations than using a synchronized block, one advantage is the ability to not block if a lock is not available.  You should obtain the lock, read or write shared data only as necessary, and unlock within a finally clause to ensure that the lock is released. Below is an example using a ReentrantReadWriteLock:



A way to reduce the time that a lock is held is lock splitting or lock striping, which uses different locks for state variables instead of a single lock. This reduces the lock granularity, allowing greater scalability but you must take locks in a disciplined order or risk deadlock.

Prefer executors and tasks to threads


Instead of working directly with threads, use the Java Concurrency Utilities Executor Framework. The Executor service decouples task submission from execution policy. Think in terms of runnable tasks and let an executor service execute them for you.



Executors can be created either directly or by using the factory methods in the Executors class:





Here is an example that uses the Executor, Executors and ExecutorService classes:

The example is a web service class that handles multiple incoming connections simultaneously with a fixed pool of threads. A fixed thread pool is initialized with the newFixedThreadPool method of the Executors class which returns an ExecutorService object. Incoming connections are handled by calling execute on the ExecutorService pool object, passing it a Runnable object. The Runnable object's run method processes the connection. When the run method completes the thread will automatically be returned to the thread pool. If a connection comes in and all threads are in use, then the main loop will block until a thread is freed.

Prefer Concurrency utilities to wait and notify


Whenever you are about to use wait and notify check and see if there is a class in java.util.concurrent that does what you need.  The concurrent collections provide high-performance concurrent implementations of standard collection interfaces such as List, Queue, and Map.



BlockingQueues are concurrent queues extended with blocking methods, which wait (or block) until an element becomes available for retrieving or space becomes available for storing.


Producer Consumer Pattern


Blocking queues are useful for the Producer Consumer Pattern where producer threads enqueue work items and consumer threads dequeue and process work items. Below is an example of a Consumer Pattern for a logger used by multiple threads. The Logger constructor takes a BlockingQueue as an input argument. In the run method messages are retrieved from the queue and logged. When the queue is empty the logging thread will block until an message becomes available for retrieving.


Below is an example of a Producer that uses the logger. A new ArrayBlockingQueue is instantiated for passing to the logger constructor. In the run method messages are put into the queue for logging. If the queue is full, the put will block until the logger has removed messages.


Synchronizers


Synchronizers are objects that help with coordinating access between threads. The most used synchronizers are CountDownLatch and Semaphore.  The use of synchronizers can eliminate most uses of wait or notify.


Below is an example of the use of a semaphore to control access to pool of resources. Multiple threads can request the use of a resource and return it when they have finished with it.

In the creator we create a new semaphore with the same size as the pool of resources we're creating.

In the getResource() method the semaphore aquire method is called to try to aquire a permit to use a resource. If there are resources available this will return and a resource will be returned from the pool. If all the resources are in use the call to aquire will block until another thread calls release on the semaphore.  When a thread finishes with a resource the resource is returned to the pool and the release method is called. Both aquire and release can be considered atomic operations.



Multithreaded Lazy Initialization is tricky

When threads share a lazily initialized field, access to the field must be synchronized, or non-determinism type bugs can result.



Prefer Normal initialization

Don't use lazy initialization unless an object or field is costly to initialize and not used often. Normally normal initialization is best ;) Below is a thread safe example of eager initialization for a singleton, the private final instance field and the private constructor make it immutable.




If you need to use lazy initialization for performance on a static field, use the initialize-on-demand holder pattern. This pattern takes advantage of the guarantee that a class will not be initialized until it is used :

References and More Information:

Effective Java, Second Edition by Joshua Bloch
Java Concurrency in Practice by Brian Goetz
Robust and Scalable Concurrent Programming: Lessons from the Trenches
Concurrency: Past and Present






Friday Aug 28, 2009

JPA Performance, Don't Ignore the Database

jpaconcurrency


Database Schema

Good Database schema design is important for performance. One of the most basic optimizations is to design your tables to take as little space on the disk as possible , this makes disk reads faster and uses less memory for query processing.

Data Types

You should use the smallest data types possible, especially for indexed fields. The smaller your data types, the more indexes (and data) can fit into a block of memory, the faster your queries will be.

Normalization

Database Normalization eliminates redundant data, which usually makes updates faster since there is less data to change. However a Normalized schema causes joins for queries, which makes queries slower, denormalization speeds retrieval. More normalized schemas are better for applications involving many transactions, less normalized are better for reporting types of applications.  You should normalize your schema first, then de-normalize later.  Applications often need to mix the approaches, for example use a partially normalized schema, and duplicate, or cache, selected columns from one table in another table. With JPA O/R mapping you can use the @Embedded annotation for denormalized columns to specify a persistent field whose @Embeddable type can be stored as an intrinsic part of the owning entity and share the identity of the entity.



Database Normalization and Mapping Inheritance Hiearchies

The Class Inheritance hierarchy shown below will be used as an example of JPA O/R mapping.


In the Single table per class mapping shown below, all classes in the hierarchy are mapped to a single table in the database. This table has a discriminator column (mapped by @DiscriminatorColumn), which identifies the subclass.  Advantages: This is fast for querying, no joins are required. Disadvantages:  wastage of space since all inherited fields are in every row, a deep inheritance hierarchy will result in wide tables with many, some empty columns.



In the Joined Subclass mapping shown below, the root of the class hierarchy is represented by a single table, and each subclass has a separate table that only contains those fields specific to that subclass. This is normalized (eliminates redundant data) which is better for storage and updates. However queries cause joins which makes queries slower especially for deep hierachies, polymorphic queries and relationships.


In the Table per Class mapping (in JPA 2.0, optional in JPA 1.0),  every concrete class is mapped to a table in the database and all the inherited state is repeated in that table. This is not normlalized, inherited data is repeated which wastes space.  Queries for Entities of the same type are fast, however  polymorphic queries cause unions which are slower.



Know what SQL is executed

You need to understand the SQL queries your application makes and evaluate their performance. Its a good idea to enable SQL logging, then go through a use case scenario to check the executed SQL.  Logging is not part of the JPA specification, With EclipseLink you can enable logging of SQL by setting the following property in the persistence.xml file:

<properties>
    <property name="eclipselink.logging.level" value="FINE"/>
</properties>


With Hibernate you set the following property in the persistence.xml file:

<properties>
    <property name="hibernate.show_sql" value="true" />
</properties>


Basically you want to make your queries access less data, is your application retrieving more data than it needs, are queries accessing too many rows or columns? Is the database query analyzing more rows than it needs? Watch out for the following:
  • queries which execute too often to retrieve needed data
  • retrieving more data than needed
  • queries which are too slow
    • you can use EXPLAIN to see where you should add indexes

With MySQL you can use the slow query log to see which queries are executing slowly, or you can use the MySQL query analyzer to see slow queries, query execution counts, and results of EXPLAIN statements.

Understanding EXPLAIN

For slow queries, you can precede a SELECT statement with the keyword EXPLAIN  to get information about the query execution plan, which explains how it would process the SELECT,  including information about how tables are joined and in which order. This helps find missing indexes early in the development process.




You should index columns that are frequently used in Query WHERE, GROUP BY clauses, and columns frequently used in joins, but be aware that indexes can slow down inserts and updates.

Lazy Loading and JPA

With JPA many-to-one and many-to-many relationships lazy load by default, meaning they will be loaded when the entity in the relationship is accessed. Lazy loading is usually good, but if you need to access all of the "many" objects in a relationship, it will cause n+1 selects where n is the number of "many" objects.




You can change the relationship to be loaded eagerly as follows :




However you should be careful with eager loading which could cause SELECT statements that fetch too much data. It can cause a Cartesian product if you eagerly load entities with several related collections.


If you want to override the LAZY fetch type for specific use cases, you can use Fetch Join. For example this query would eagerly load the employee addresses:

In General you should lazily load relationships, test your use case scenarios, check the SQL log, and use @NameQueries with JOIN FETCH to eagerly load when needed.

Partitioning

the main goal of partitioning is to reduce the amount of data read for particular SQL operations so that the overall response time is reduced

Vertical Partitioning  splits tables with many columns into multiple tables with fewer columns, so that only certain columns are included in a particular dataset, with each partition including all rows.

Horizontal Partitioning segments table rows so that distinct groups of physical row-based datasets are formed. All columns defined to a table are found in each set of partitions. An example of horizontal partitioning might be a table that contains historical data being partitioned by date.

Vertical Partitioning


In the example of vertical partitioning below a table that contains a number of very wide text or BLOB columns that aren't referenced often is split into two tables with the most referenced columns in one table and the seldom-referenced text or BLOB columns in another.

By removing the large data columns from the table, you get a faster query response time for the more frequently accessed Customer data. Wide tables can slow down queries, so you should always ensure that all columns defined to a table are actually needed.

The example below shows the JPA mapping for the tables above. The Customer data table with the more frequently accessed and smaller data types  is mapped to the Customer Entity, the CustomerInfo table with the less frequently accessed and larger data types is mapped to the CustomerInfo Entity with a lazily loaded one to one relationship to the Customer.



Horizontal Partitioning

The major forms of horizontal partitioning are by Range, Hash, Hash Key, List, and Composite.

Horizontal partitioning can make queries faster because the query optimizer knows what partitions contain the data that will satisfy a particular query and will access only those necessary partitions during query execution. Horizontal Partitioning works best for large database Applications that contain a lot of query activity that targets specific ranges of database tables.


Hibernate Shards

Partitioning data horizontally into "Shards" is used by google, linkedin, and others to give extreme scalability for very large amounts of data. eBay "shards" data horizontally along its primary access path.

Hibernate Shards is a framework that is designed to encapsulate support for horizontal partitioning into the Hibernate Core.


Caching

JPA Level 2 caching avoids database access for already loaded entities, this make reading reading frequently accessed unmodified entities faster, however it can give bad scalability for frequent or concurrently updated entities.

You should configure L2 caching for entities that are:
  • read often
  • modified infrequently
  • Not critical if stale
You should also configure L2 (vendor specific) caching for maxElements, time to expire, refresh...

References and More Information:

JPA Best Practices presentation
MySQL for Developers Article
MySQL for developers presentation
MySQL for developers screencast
Keeping a Relational Perspective for Optimizing Java Persistence
Java Persistence with Hibernate
Pro EJB 3: Java Persistence API
Java Persistence API 2.0: What's New ?
High Performance MySQL book
Pro MySQL, Chapter 6: Benchmarking and Profiling
EJB 3 in Action
sharding the hibernate way
JPA Caching
Best Practices for Large-Scale Web Sites: Lessons from eBay




Friday Aug 21, 2009

JPA Caching


JPA Level 1 caching

JPA has 2 levels of caching. The first level of caching is the persistence context.

The JPA Entity Manager maintains a set of Managed Entities in the Persistence Context.

The Entity Manager guarantees that within a single Persistence Context, for any particular database row, there will be only one object instance. However the same entity could be managed in another User's transaction, so you should use either optimistic or pessimistic locking  as explained in JPA 2.0 Concurrency and locking

The code below shows that a find on a managed entity with the same id and class as another in the same persistence context , will return the same instance.

@Stateless public ShoppingCartBean implements ShoppingCart {

 @PersistenceContext EntityManager entityManager;

 public OrderLine createOrderLine(Product product,Order order) {
        OrderLine orderLine = new OrderLine(order, product);
        entityManager.persist(orderLine);   //Managed
        OrderLine orderLine2 =entityManager.find(OrderLine,
orderLine.getId()));
     (orderLine == orderLine2  // TRUE
        return (orderLine);
    }

}

The diagram below shows the life cycle of an Entity in relation to the Persistent Context.

The code below illustrates the life cycle of an Entity. A reference to a container managed EntityManager is injected using the persistence context annotation. A new order entity is created and the entity has the state of new. Persist is called, making this a managed entity. because it is a stateless session bean it is by default using container managed transactions , when this transaction commits , the order is made persistent in the database. When the orderline entity is returned at the end of the transaction it is a detached entity.

The Persistence Context can be either Transaction Scoped-- the Persistence Context 'lives' for the length of the transaction, or Extended-- the Persistence Context spans multiple transactions. With a Transaction scoped Persistence Context, Entities are "Detached" at the end of a transaction.

As shown below, to persist the changes on a detached entity, you call the EntityManager's merge() operation, which returns an updated managed entity, the entity updates will be persisted to the database at the end of the transaction.

An Extended Persistence Context spans multiple transactions, and the set of Entities in the Persistence Context stay Managed. This can be useful in a work flow scenario where a "conversation" with a user spans multiple requests.

The code below shows an example of a Stateful Session EJB with an Extended Persistence Context in a use case scenario to add line Items to an Order. After the Order is persisted in the createOrder method, it remains managed until the EJB remove method is called. In the addLineItem method , the Order Entity can be updated because it is managed, and the updates will be persisted at the end of the transaction.


The example below contrasts updating the Order using a transaction scoped Persistence Context verses an extended Persistence context. With the transaction scoped persistence context, an Entity Manager find must be done to look up the Order, this returns a Managed Entity which can be updated. With the Extended Persistence Context the find is not necessary. The performance advantage of not doing a database read to look up the Entity, must be weighed against the disadvantages of memory consumption for caching, and the risk of cached entities being updated by another transaction.  Depending on the application and the risk of contention among concurrent transactions this may or may not give better performance / scalability.

JPA second level (L2) caching

JPA second level (L2) caching shares entity state across various persistence contexts.


JPA 1.0 did not specify support of a second level cache, however, most of the persistence providers provided support for second level cache(s). JPA 2.0 specifies support for basic cache operations with the new Cache API, which is accessible from the EntityManagerFactory, shown below:


If L2 caching is enabled, entities not found in persistence context, will be loaded from L2 cache, if found.

The advantages of L2 caching are:
  • avoids database access for already loaded entities
  • faster for reading frequently accessed  unmodified entities
The disadvantages of L2 caching are:
  • memory consumption for large amount of objects
  • Stale data for updated objects
  • Concurrency for write (optimistic lock exception, or pessimistic lock)
  • Bad scalability for frequent or concurrently updated entities

You should configure L2 caching for entities that are:
  • read often
  • modified infrequently
  • Not critical if stale
You should protect any data that can be concurrently modified with a locking strategy:
  • Must handle optimistic lock failures on flush/commit
  • configure expiration, refresh policy to minimize lock failures
The Query cache is useful for queries that are run frequently with the same parameters, for not modified tables.

The EclipseLink JPA persistence provider caching Architecture

The  EclipseLink caching Architecture is shown below.


Support for second level cache in EclipseLink is turned on by default, entities read are L2 cached. You can disable the L2 cache. EclipseLink caches entities in L2, Hibernate caches entity id and state in L2. You can configure caching by Entity type or Persistence Unit with the following configuration parameters:
  • Cache isolation, type, size, expiration, coordination, invalidation,refreshing
  • Coordination (cluster-messaging)
  • Messaging: JMS, RMI, RMI-IIOP, …
  • Mode: SYNC, SYNC+NEW, INVALIDATE, NONE
The example below shows configuring the L2 cache for an entity using the @Cache annotation

The Hibernate JPA persistence provider caching Architecture

The Hibernate JPA persistence provider caching architecture is different than EclipseLink: it is not configured by default, it does not cache enities just id and state, and you can plug in different L2 caches. The diagram below shows the different L2 cache types that you can plug into Hibernate.

The configuration of the cache depends on the type of caching plugged in. The example below shows configuring the hibernate L2 cache for an entity using the @Cache annotation

For More Information:

Introducing EclipseLink
EclipseLink JPA User Guide
Hibernate Second Level Cache
Speed Up Your Hibernate Applications with Second-Level Caching
Hibernate caching
Java Persistence API 2.0: What's New ?
Beginning Java™ EE 6 Platform with GlassFish™ 3
Pro EJB 3: Java Persistence API (JPA 1.0)





Tuesday Aug 11, 2009

Java EE 6 Pet Catalog with GlassFish v3 preview and MySQL


Java EE 6 Pet Catalog with GlassFish v3 preview and MySQL


This Pet Catalog app explains a web application that uses JSF 2.0, Java EE 6, GlassFish and MySQL. I took this example  GlassFish and MySQL, Part 2: Building a CRUD Web Application With Data Persistence and modified it to use some of the new features of JSF 2.0 and Java EE 6.
http://www.netbeans.org/kb/samples/index.html 

Explanation of the usage of JSF 2.0 and Java EE 6 in a sample Store Catalog Application

The image below shows the Catalog Listing page, which allows a user to page through a list of items in a store.



JSF 2.0 Facelets XHTML instead of JSP

For JSF 2.0, Facelets XHTML is the preferred way to declare JSF Web Pages. JSP is supported for backwards compatibility, but not all JSF 2.0 features will be available for views using JSP as their page declaration language.  JSF 2.0 Facelets has some nice features like templating (similar in functionality to Tiles) and composite components, which I'm not going to discuss here but you can read about that in this article: http://www.ibm.com/developerworks/java/library/j-jsf2fu2/index.html and in this Tech Tip Composite UI Components in JSF 2.0.


The Catalog application's resources


JSF 2.0 standardizes how to define web resources. Resources are any artifacts that a component may need in order to be rendered properly -- images, CSS, or JavaScript files.  With JSF 2.0 you put resources in a resources directory or a subdirectory.


In your Facelets pages, you can access css files with the  <h:outputStylesheet>,  javascript files with the <h:outputScript> , and images with the <h:graphicImage> JSF tags. The list.xhtml uses the  <h:outputStylesheet tag to load the styles.css stylesheet , and the <h:graphicImage tag to display images from the resources as shown below:

Code Sample from:  list.xhtml

<h:outputStylesheet name="css/styles.css" target="body"/>

<h:graphicImage library="images" name="banner_logo.gif"  />   




The Catalog application uses a resource bundle to contain the static text and error messages used by the Facelets pages. Putting messages in a resource bundle makes it easier to modify and internationalize your Application text.  The messages are in a properties file in a java package directory.

Code Sample from:  messages.properties 

Title=Pet Catalog
Next=Next
Previous=Prev
Name=Name



The resource bundle is configured in the faces-config.xml File (you don't need any other configuration in the faces-config.xml for JSF 2.0, as explained later you no longer have to configure managed beans and navigation with XML).


Code Sample from:  faces-config.xml

<application>
    <resource-bundle>
        <base-name>web.WebMessages</base-name>
        <var>msgs</var>
    </resource-bundle>
</application>




The List.xhtml facelets page uses a JSF dataTable component to display a list of catalog items in an html table.  The dataTable component is useful when you want to show a set of results in a table. In a JavaServer Faces application, the UIData component (the superclass of dataTable)  supports binding to a collection of data objects. It does the work of iterating over each record in the data source. The HTML dataTable renderer displays the data as an HTML table.

In the list.xhtml web page the dataTable is defined as shown below:  (Note: Red colors are for Java EE tags, annotations code,  and Green is for my code or variables)

Code Sample from:  list.xhtml

<h:dataTable value='#{catalog.items}' var='row' border="1"
      cellpadding="2" cellspacing="0">



The value attribute of a dataTable tag references the data to be included in the table. The var attribute specifies a name that is used by the components within the dataTable tag as an alias to the data referenced in the value attribute of dataTable.  In the dataTable tag from the List.jsp page, the value attribute points to a list of catalog items. The var attribute points to a single item in that list. As the dataTable component iterates through the list, each reference to dataTableItem points to the current item in the list.

JSF 2.0 Annotations instead of XML configuration

The dataTable's value is bound to the items property of the catalog managed bean. With JSF 2.0 managed beans do not have to be configured in the faces-config.xml file, you annotate the managed beans instead as shown below:

Code Sample from: Catalog.java


@ManagedBean
@SessionScoped
public class Catalog implements Serializable {




By convention, the name of a managed bean is the same as the class name, with the first letter of the class name in lowercase. To specify a managed bean name you can use the name attribute of the ManagedBean annotation, like this: @ManagedBean(name = "Catalog").


This Catalog ManagedBean items property is defined as shown below:

Code Sample from: Catalog.java

    private List<Item> items = null;

    public List<Item> getItems() {
        if (items == null) {
            getPagingInfo();
            items = getNextItems(pagingInfo.getBatchSize(), pagingInfo.getFirstItem());
        }
        return items;
    }





The getItems() method returns a List of item objects. The JSF dataTable, supports data binding to a collection of data objects.  The dataTable object is modeled as a collection of row objects that can be accessed by a row index.  The APIs provide mechanisms to position to a specified row index, and to retrieve an object that represents the data that corresponds to the current row index.   

The Item properties name, imagethumburl, and price are displayed with the column component:

Code Sample from: list.xhtml

<h:dataTable var="row" value="#{catalog.items}">
  <h:column>
      <f:facet name="header">
          <h:outputText value="#{msgs.Name}"/>
      </f:facet>
      <h:commandLink action="#{catalog.showDetail(row)}" value="#{row.name}" />
  </h:column>

  <h:column>
      <f:facet name="header">
          <h:outputText value="#{msgs.Photo}"/>
      </f:facet>
<h:graphicImage library="images" name="#{row.imagethumburl}"/>
  </h:column>

  <h:column>
      <f:facet name="header">
          <h:outputText value="#{msgs.Price}"/>
      </f:facet>
      <h:outputText value="#{row.price}"/>
  </h:column>
</h:dataTable>


The column tags represent columns of data in a dataTable component. While the dataTable component is iterating over the rows of data, it processes the UIColumn component associated with each column tag for each row in the table.

The dataTable component  iterates through the list of items (catalog.items)  and displays the item (var="row") attribute value. Each time UIData iterates through the list of items, it renders one cell in each column.

The dataTable and column tags use facet to represent parts of the table that are not repeated or updated. These include headers, footers, and captions.

Java EE 6: JSF 2.0,  EJB 3.1, and Java Persistence API (JPA)  2.0

The Catalog ManagedBean annotates the field private ItemFacade itemFacade;  with @EJB , which causes an itemFacade EJB to be injected when the managed bean is instatiated. The Catalog getNextItems method calls the ItemFacade Stateless EJB which uses the Java Persistence API EntityManager Query object to return a list of items.


Code Sample from: Catalog.java

@ManagedBean
@SessionScoped
public class Catalog implements Serializable {


@EJB
    private ItemFacade itemFacade;

    public List<Item>  getNextItems(int maxResults, int firstResult) { 
       return itemFacade.findRange(maxResults, firstResult);    
   }


EJB 3.1 No-interface local client View

With EJB 3.1, local EJBs  do not  have to a implement separate interface, that is, all public methods of the bean class are automatically exposed to the caller.

Simplified Packaging

With Java EE 6, EJBs can be directly packaged in a WAR file just like web components.

The ItemFacade EJB uses the Java Persistence API EntityManager Query object to return a list of items. The ItemFacade EJB annotates the field private EntityManager em;  with @PersistenceContext , which causes an entity manager to be injected when it is instatiated.

Code Sample from: ItemFacade.java

@Stateless
public class ItemFacade {


@PersistenceContext(unitName = "catalogPU")
    private EntityManager em;

    public List<Item> findRange(int maxResults, int firstResult) {
Query q = em.createQuery("select object(o) from Item as o");
        q.setMaxResults(maxResults);
        q.setFirstResult(firstResult);
        return q.getResultList();
    }


The Java Persistence Query APIs are used to create and execute queries that can return a list of results.  The JPA Query interface provides support for pagination via the setFirstResult() and setMaxResults() methods: q.setMaxResults(int maxResult) sets the maximum number of results to retrieve. q.setFirstResult(int startPosition) sets the position of the first result to retrieve.

In the code below, we show the Item entity class which maps to the  ITEM table that stores the item instances. This is a typical Java Persistence entity object. There are two requirements for an entity:
  1. annotating the class with an @Entity annotation.
  2. annotating   the primary key identifier with @Id
Because the fields name, description.... are basic mappings from the object fields to columns of the same name in the database table, they don't have to be annotated.  The O/R  relationships with Address and Product are also annotated. For more information on defining JPA entities see Pro EJB 3: Java Persistence API book.

Code Sample from: Item.java

@Entity
public class Item implements java.io.Serializable {

@Id
    private Integer id;

    private String name;   
    private String description;   
    private String imageurl;   
    private String imagethumburl; 
    private BigDecimal price;
@ManyToOne
    private Address address;
@ManyToOne
    private Product product;


    public Item() { }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }


    ...
}   




The Catalog ManagedBean pages through the list of Items by maintaining the PagingInfo.firstItem and PagingInfo.batchSize attributes and passing these as parameters to the  getNextItems(firstItem, batchSize) method. The catalog's scope  is defined with the annotation @SessionScoped, a JSF Managedbean with session scope will be stored in the session meaning that the bean's properties will stay alive for the life of the Http Session.


A JSF commandButton is  used to provide a button to click on to display the next page of items.  The commandButton tag is used to submit an action event to the application. 

Code Sample from: list.xhtml

 <h:commandButton action="#{catalog.next}" value="#{msgs.Next}" />   


This commandButton action attribute references the catalog Managed bean next() method which calculates the next page's first row number  and returns a logical outcome String, which causes the list.xhtml page to display the next page's list . The catalog next method is defined as shown below:

Code Sample from: catalog.java

   public String next() {
       if (firstItem + batchSize < itemCount()) {
           firstItem += batchSize;
       }
       return "list";
   }


JSF 2.0 Simplified Navigation


The JavaServer Faces 2.0  NavigationHandler convention adds .xhtml to the logical outcome of the action method (in this example list) and loads that file, in this case, it loads the list.xhtml page after this method returns. If the action doesn't begin with a forward slash (/), JSF assumes that it's a relative path.  You can specify an absolute path by adding the slash like this "/items/list".


A JSF commandLink is  used to provide a link to click on to display a page with the item details. This commandLink action attribute  references The catalog showDetail() method:

Code Sample from: list.xhtml

   <h:column>
       <f:facet name="header">
          <h:outputText value="Name"/>
       </f:facet>
       <h:commandLink action="#{catalog.showDetail(row)}" value="#{row.name}"/>   
   </h:column>


With JSF 2.0 you can now specify parameters in method expressions. The dataTable row object associated with the selected link is passed as a parameter in the  "#{catalog.showDetail(row)}" method expression.
The Catalog showDetail() method  gets the item data from the input parameter, and returns a string which causes the detail.xhtml page to display the item details :

Code Sample from: Catalog.java

    public String showDetail(Item item) {
        this.item = item;
        return "detail";
    }


The JavaServer Faces NavigationHandler adds .xhtml to the logical outcome of the action, detail and loads that file. In this case, the JavaServer Faces implementation loads the detail.xhtml page after this method returns.

The detail.xhtml uses the outputText component to display the catalog ManagedBean's item properties:

Code Sample from: detail.xhtml

    <h:outputText value="#{catalog.item.name}" title="Name" />
    <h:outputText value="#{catalog.item.description}" title="Description"/>
    <h:graphicImage library="images" name="#{catalog.item.imageurl}" title="Imageurl" />

    <h:outputText value="#{
catalog.item.price}" title="Price" />
    <h:outputText value="#{
catalog.item.address.city}" title="Address" />
    <h:outputText value="#{
catalog.item.contactinfo.email}" title="Address"/>  





GlassFish v3 is a lightweight server

  • OSGi-based; Embedded API; RESTful admin API; Lightweight and fast startup;
  • iterative development cycle "edit-save-refresh browser":
    • Incremental compile of all JSF 2.0 artifacts when you save.
    • Auto-deploy of all web or Java EE 6 artifacts
  • Session retention: maintain sessions across re-deployments


Conclusion
This concludes the sample application which demonstrates a pet catalog web application which uses Java EE 6, GlassFish v3 and MySQL.

Running the Sample Application

  1. Download and install NetBeans IDE 6.8 M1 with GlassFish v3 b57 (Glassfish v3 preview is Java EE 6 Preview) , and MySQL Community Server .
  2. Follow these instructions to set up a  jdbc-driver for MySQL. (Normally this is already setup with Glassfish, but I got an errror message with Glassfish v3 b57 that it was missing)
  3. Download the sample code. Unzip the catalog.zip file which you downloaded, this will create a catalog directory with the project code.

Create the Pet Catalog database

In order to run the sample code you first have to create the Pet Catalog database and fill in  the Item table.

  1. Start NetBeans IDE
  2. Ensure that GlassFish is registered in the NetBeans IDE, as follows:
    • Click the Services tab in the NetBeans IDE.
    • Expand the Servers node. You should see GlassFish v2 in the list of servers. If not, register GlassFish v2 as follows:
      • Right-click the Servers node and select Add Server. This opens an Add Server Instance wizard.
      • Select GlassFish v2 in the server list of the wizard and click the Next button.
      • Enter the location information for the server and click the Next button.
      • Enter the admin name and password and click the Finish button.
  3. Start the MySQL or Java DB database as follows:
    • Click the Services tab in the NetBeans IDE.
    • Expand the databases node. You should see the Java DB database in the list of databases. If you have installed the MySQL server database, you should also see the MySQL database in the list of databases.. Note:  Java DB  comes bundled with Netbeans, you can  download MySQL separately.

    • Right-mouse click on the Java DB or MySQL server database and select Start.
  4. If you installed MySQL, set the properties of the MySQL server database as follows:
    • Right-click on the MySQL server database and select Properties. This opens the MySQL Server Properties dialog box, as shown in Figure 8.

      MySQL Server Basic Properties
      Figure 8. MySQL Server Basic Properties

    • In the Basic Properties tab, enter the server host name and port number. The IDE specifies localhost as the default server host name and 3306 as the default server port number.
    • Enter the administrator user name, if not displayed, and the administrator password -- the default administrator password is blank.
    • Click the Admin Properties tab.
    • Enter an appropriate path in the Path/URL to admin tool field. You can find the path by browsing to the location of a MySQL Administration application such as the MySQL Admin Tool.
    • Enter an appropriate path in the Path to start command. You can find the path by browsing to the location of the MySQL start command. To find the start command, look for mysqld in the bin folder of the MySQL installation directory.
    • Enter an appropriate path in the Path to stop command field. You can find the path by browsing to the location of the MySQL stop command. This is usually the path to mysqladmin in the bin folder of the MySQL installation directory. If the command is mysqladmin, in the Arguments field, type -u root stop to grant root permissions for stopping the server. The Admin Properties tab should look similar to Figure 9.

      MySQL Server Administration Properties
      Figure 9. MySQL Server Administration Properties

    • Click the OK button.

  5. Right-click on the MySQL server or Java DB database and select Start.
  6. Create the petcatalog database as follows:
    • Right-mouse click on the Java DB or MySQL server database and select Create Database. This will open a create Database window.
    • Enter the database name catalog for Java DB or petcatalog for MySQL.


      For Java DB enter userid password app app as shown below:


       Click O.K. to accept the displayed settings.
  7. Create the tables in the catalog database as follows:
    • Underneath Databases you should see a database connection for the petcatalog database. For example MySQL:

      or Java DB:

    • Right-mouse click on the petcatalog connection and select Connect.
    • Right-mouse click on the petcatalog connection and select Execute Command. This will open up a SQL command window.
    • Copy the contents of the catalog.sql file in the catalog directory and paste the contents into the SQL command window, as shown in below:

      Creating Tables in the Database
    • Click the Run SQL icon Run SQL icon (Ctrl+Shift+E) above the SQL command window.
    • Note: It is ok to see this: "Error code -1, SQL state 42Y55: 'DROP TABLE' cannot be performed on 'ITEM' because it does not exist. Line 2, column 1" . This just means you are deleting a table that does not exist.  If you need to delete and recreate the tables you will not  see this message the second time.
  8. View the data in the Pet Catalog database Item table as follows:
    • Underneath Databases you should see a database connection for the petcatalog database. For example MySQL:

      or Java DB:

    • If the database connection is broken like in the following diagram:

      • Right-mouse click on the petcatalog connection and select Connect. as shown below:

      • if prompted for a password, for MySQL leave it blank, for JavaDB enter user app password app.
    • Expand the Tables node below the petcatalog database in the Services window. You should see the item table under the Tables node. You can expand the item table node to see the table columns, indexes, and any foreign keys, as shown in below :
      An Expanded Table Node
      Figure 12. An Expanded Table Node

      You can view the contents of a table or column by right-clicking the table or column and selecting View Data as shown  below:

      Viewing the Contents of a Table
      Figure 13. Viewing the Contents of a Table


  9. Follow these instructions to Create a JDBC Connection pool and JDBC resource. Name the  pool mysql_petcatalog_rootPool and the jndi resource jdbc/petcatalog. Note: you do not have to create a JDBC connection pool and resource if you use the Netbeans wizard to generate JPA entities from database tables as described in this article GlassFish and MySQL, Part 2: Building a CRUD Web Application With Data Persistence.
  10. Open the catalog/setup/sun-resources.xml file and verify that the property values it specifies match those of the petcatalog database and jdbc resources you created. Edit the property values as necessary.

Running the Sample solution:

If you want to run the sample solution, you have to create the catalog database tables first as described above.

  1. Open the catalog project as follows:
    • In NetBeans IDE, click Open Project in the File menu. This opens the Open Project dialog.
    • Navigate in the Open Project dialog to the catalog  directory and click the Open Project button.

    In response, the IDE opens the catalog project.  You can view the logical structure of the project in the Projects window (Ctrl-1).
  2. Run the catalog by right-clicking on the catalog project in the Projects window and selecting Run Project. The NetBeans IDE compiles the application, deploys it on Glassfish, and brings up the default page in your browser.  (at http://localhost:8080/catalog/).

For more information see the following resources:



Thursday Jul 30, 2009

JPA 2.0 Concurrency and locking

Optimistic Concurrency


Optimistic locking lets concurrent transactions process simultaneously, but detects and prevent collisions, this works best for applications where most concurrent transactions do not conflict. JPA Optimistic locking allows anyone to read and update an entity, however a version check is made upon commit and an exception is thrown if the version was updated in the database since the entity was read.  In JPA for Optimistic locking you annotate an attribute with @Version as shown below:

public class Employee {
    @ID int id;
@Version int version;

The Version attribute will be incremented with a successful commit. The Version attribute can be an int, short, long, or timestamp.  This results in SQL like the following:

“UPDATE Employee SET ..., version = version + 1
     WHERE id = ? AND version = readVersion

The advantages of optimistic locking are that no database locks are held which can give better scalability. The disadvantages are that the user or application must refresh and retry failed updates.

Optimistic Locking Example


In the optimistic locking example below, 2 concurrent transactions are updating employee e1. The transaction on the left commits first causing the e1 version attribute to be incremented with the update. The transaction on the right throws an OptimisticLockException because the e1 version attribute is higher than when e1 was read, causing the transaction to roll back.
img60.jpg

Additional Locking with JPA Entity Locking APIs


With JPA it is possible to lock an entity, this allows you to control when, where and which kind of locking to use. JPA 1.0 only supported Optimistic read or Optimistic write locking.  JPA 2.0 supports Optimistic and Pessimistic locking, this is layered on top of @Version checking described above.

JPA 2.0 LockMode values :
  • OPTIMISTIC (JPA 1.0 READ):
    • perform a version check on locked Entity before commit, throw an OptimisticLockException if Entity version mismatch.
  • OPTIMISTIC_FORCE_INCREMENT (JPA 1.0 WRITE)
    • perform a version check on locked Entity before commit, throw an OptimisticLockException if Entity version mismatch, force an increment to the version at the end of the transaction, even if the entity is not modified.
  • PESSIMISTIC:
    • lock the database row when reading
  • PESSIMISTIC_FORCE_INCREMENT
    • lock the database row when reading, force an increment to the version at the end of the transaction, even if the entity is not modified.
There are multiple APIs to specify locking an Entity:
  • EntityManager methods: lock, find, refresh
  • Query methods: setLockMode 
  • NamedQuery annotation: lockMode element

OPTIMISTIC (READ) LockMode Example


In the optimistic locking example below,  transaction1 on the left updates the department name for dep , which causes dep's version attribute to be incremented. Transaction2 on the right gives an employee a raise if he's in the "Eng" department. Version checking on the employee attribute would not throw an exception in this example since it was the dep Version attribute that was updated in transaction1. In this example the employee change should not commit if the department was changed after reading, so an OPTIMISTIC lock is used : em.lock(dep, OPTIMISTIC).  This will cause a version check on the  dep Entity before committing transaction2  which will throw an OptimisticLockException because the dep version attribute is higher than when dep was read, causing the transaction to roll back.
img62.jpg

OPTIMISTIC_FORCE_INCREMENT (write) LockMode Example


In the OPTIMISTIC_FORCE_INCREMENT locking example below,  transaction2 on the right wants to be sure that the dep name does not change during the transaction, so transaction2 locks the dep Entity em.lock(dep, OPTIMISTIC_FORCE_INCREMENT) and then calls em.flush() which causes dep's version attribute to be incremented in the database. This will cause any parallel updates to dep  to throw an OptimisticLockException and roll back. In transaction1 on the left at commit time when the dep version attribute is checked and found to be stale, an OptimisticLockException is thrown
img63.jpg

Pessimistic Concurrency

Pessimistic concurrency locks the database row when data is read, this is the equivalent of a (SELECT . . . FOR UPDATE [NOWAIT]) .  Pessimistic locking ensures that transactions do not update the same entity at the same time, which can simplify application code, but it limits concurrent access to the data which can cause bad scalability and may cause deadlocks. Pessimistic locking is better for applications with a higher risk of contention among concurrent transactions.
The examples below show:
  1. reading an entity and then locking it later
  2. reading an entity with a lock
  3. reading an entity, then later refreshing it with a lock

The Trade-offs are the longer you hold the lock the greater the risks of bad scalability and deadlocks. The later you lock the greater the risk of stale data, which can then cause an optimistic lock exception, if the entity was updated after reading but before locking.
img66.jpg
img672.jpg
The right locking approach depends on your application:
  • what is the risk of risk of contention among concurrent transactions?
  • What are the requirements for scalability?
  • What are the requirements for user re-trying on failure?

References and More Information:

Preventing Non-Repeatable Reads in JPA Using EclipseLink
Java Persistence API 2.0: What's New ?
What's New and Exciting in JPA 2.0
Beginning Java™ EE 6 Platform with GlassFish™ 3
Pro EJB 3: Java Persistence API (JPA 1.0)

Java Persistence API: Best Practices and Tips





Friday Jul 24, 2009

JSF 2.0, JPA, GlassFish and MySQL

JSF 2.0, JPA, GlassFish and MySQL

JSF 2.0, JPA, GlassFish and MySQL


This Pet Catalog app explains a web application that uses JSF 2.0, JPA, GlassFish and MySQL. I took this example  GlassFish and MySQL, Part 2: Building a CRUD Web Application With Data Persistence and modified it to use some of the new features JSF 2.0. 

Explanation of the usage of JSF 2.0, Java Persistence APIs, Glassfish and MySQL in a sample Store Catalog Application

The image below shows the Catalog Listing page, which allows a user to page through a list of items in a store.



JSF 2.0 Facelets XHTML instead of JSP

For JSF 2.0, Facelets XHTML is the preferred way to declare JSF Web Pages. JSP is supported for backwards compatibility, but not all JSF 2.0 features will be available for views using JSP as their page declaration language.  JSF 2.0 Facelets has some nice features like templating (similar in functionality to Tiles) and composite components, which I'm not going to discuss here but you can read about that in this article: http://www.ibm.com/developerworks/java/library/j-jsf2fu2/index.html and in this Tech Tip Composite UI Components in JSF 2.0.


The Catalog application's resources


JSF 2.0 standardizes how to define web resources. Resources are any artifacts that a component may need in order to be rendered properly -- images, CSS, or JavaScript files.  With JSF 2.0 you put resources in a resources directory or a subdirectory.


In your Facelets pages, you can access css files with the  <h:outputStylesheet>,  javascript files with the <h:outputScript> , and images with the <h:graphicImage> JSF tags. The list.xhtml uses the  <h:outputStylesheet tag to load the styles.css stylesheet , and the <h:graphicImage tag to display images from the resources as shown below:

Code Sample from:  list.xhtml

<h:outputStylesheet name="css/styles.css" target="body"/>

<h:graphicImage library="images" name="banner_logo.gif"  />   




The Catalog application uses a resource bundle to contain the static text and error messages used by the Facelets pages. Putting messages in a resource bundle makes it easier to modify and internationalize your Application text.  The messages are in a properties file in a java package directory.

Code Sample from:  messages.properties 

Title=Pet Catalog
Next=Next
Previous=Prev
Name=Name



The resource bundle is configured in the faces-config.xml File (you don't need any other configuration in the faces-config.xml for JSF 2.0, as explained later you no longer have to configure managed beans and navigation with XML).


Code Sample from:  faces-config.xml

<application>
    <resource-bundle>
        <base-name>web.WebMessages</base-name>
        <var>msgs</var>
    </resource-bundle>
</application>




The List.xhtml facelets page uses a JSF dataTable component to display a list of catalog items in an html table.  The dataTable component is useful when you want to show a set of results in a table. In a JavaServer Faces application, the UIData component (the superclass of dataTable)  supports binding to a collection of data objects. It does the work of iterating over each record in the data source. The HTML dataTable renderer displays the data as an HTML table.

In the list.xhtml web page the dataTable is defined as shown below:  (Note: Red colors are for Java EE tags, annotations code,  and Green is for my code or variables)

Code Sample from:  list.xhtml

<h:dataTable value='#{catalog.items}' var='row' border="1"
      cellpadding="2" cellspacing="0">



The value attribute of a dataTable tag references the data to be included in the table. The var attribute specifies a name that is used by the components within the dataTable tag as an alias to the data referenced in the value attribute of dataTable.  In the dataTable tag from the List.jsp page, the value attribute points to a list of catalog items. The var attribute points to a single item in that list. As the dataTable component iterates through the list, each reference to dataTableItem points to the current item in the list.

JSF 2.0 Annotations instead of XML configuration

The dataTable's value is bound to the items property of the catalog managed bean. With JSF 2.0 managed beans do not have to be configured in the faces-config.xml file, you annotate the managed beans instead as shown below:

Code Sample from: Catalog.java


@ManagedBean
@SessionScoped
public class Catalog implements Serializable {




By convention, the name of a managed bean is the same as the class name, with the first letter of the class name in lowercase. To specify a managed bean name you can use the name attribute of the ManagedBean annotation, like this: @ManagedBean(name = "Catalog").


This Catalog ManagedBean items property is defined as shown below:

Code Sample from: Catalog.java

    private DataModel items = null;

    public DataModel getItems() {
        if (items == null) {
            getPagingInfo();
            items = new ListDataModel(getNextItems(pagingInfo.getBatchSize(), pagingInfo.getFirstItem()));
        }
        return items;
    }





The getItems() method wraps a List of item objects in a DataModel. UIData, the superclass of dataTable, supports data binding to a collection of data objects represented by a DataModel instance.  The data collection underlying a DataModel instance is modeled as a collection of row objects that can be accessed by a row index.  The APIs provide mechanisms to position to a specified row index, and to retrieve an object that represents the data that corresponds to the current row index.   

The Item properties Name, Photo, and price are displayed with the column component:

Code Sample from: list.xhtml

<h:dataTable var="row" value="#{catalog.items}">
  <h:column>
      <f:facet name="header">
          <h:outputText value="#{msgs.Name}"/>
      </f:facet>
      <h:outputText value="#{row.name}"/>
  </h:column>

  <h:column>
      <f:facet name="header">
          <h:outputText value="#{msgs.Photo}"/>
      </f:facet>
<h:graphicImage library="images" name="#{row.imagethumburl}"/>
  </h:column>

  <h:column>
      <f:facet name="header">
          <h:outputText value="#{msgs.Price}"/>
      </f:facet>
      <h:outputText value="#{row.price}"/>
  </h:column>
</h:dataTable>


The column tags represent columns of data in a UIData component. While the UIData component is iterating over the rows of data, it processes the UIColumn component associated with each column tag for each row in the table.

The UIData component  iterates through the list of items (catalog.items)  and displays the row.price. Each time UIData iterates through the list of items, it renders one cell in each column.

The dataTable and column tags use facet to represent parts of the table that are not repeated or updated. These include headers, footers, and captions.

Using the Java Persistence API (JPA) with JSF

The Catalog ManagedBean uses the Java Persistence API EntityManager Query object to return a list of items. The Catalog ManagedBean annotates the field private EntityManager em;  with @PersistenceContext , which causes an entity manager to be injected when the managed bean is instatiated.


Code Sample from: Catalog.java

@ManagedBean
@SessionScoped
public class Catalog implements Serializable {


@PersistenceUnit(unitName = "catalogPU")
    private EntityManagerFactory emf;

    private EntityManager getEntityManager() {
        return emf.createEntityManager();
    }

    public List<Item>  getNextItems(int maxResults, int firstResult) { 
       EntityManager em = getEntityManager();
       try {    
Query q = em.createQuery("select object(o) from Item as o");
         q.setMaxResults(maxResults);
         q.setFirstResult(firstResult);
         return q.getResultList();
        } finally {
            em.close();
        }      
   }


The Java Persistence Query APIs are used to create and execute queries that can return a list of results.  The JPA Query interface provides support for pagination via the setFirstResult() and setMaxResults() methods: q.setMaxResults(int maxResult) sets the maximum number of results to retrieve. q.setFirstResult(int startPosition) sets the position of the first result to retrieve.

In the code below, we show the Item entity class which maps to the  ITEM table that stores the item instances. This is a typical Java Persistence entity object. There are two requirements for an entity:
  1. annotating the class with an @Entity annotation.
  2. annotating   the primary key identifier with @Id
Because the fields name, description.... are basic mappings from the object fields to columns of the same name in the database table, they don't have to be annotated.  The O/R  relationships with Address and Product are also annotated. For more information on defining JPA entities see Pro EJB 3: Java Persistence API book.

Code Sample from: Item.java

@Entity
public class Item implements java.io.Serializable {

@Id
    private Integer id;

    private String name;   
    private String description;   
    private String imageurl;   
    private String imagethumburl; 
    private BigDecimal price;
@ManyToOne
    private Address address;
@ManyToOne
    private Product product;


    public Item() { }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }


    ...
}   




The Catalog ManagedBean pages through the list of Items by maintaining the PagingInfo.firstItem and PagingInfo.batchSize attributes and passing these as parameters to the  getNextItems(firstItem, batchSize) method. The catalog's scope  is defined with the annotation @SessionScoped, a JSF Managedbean with session scope will be stored in the session meaning that the bean's properties will stay alive for the life of the Http Session.


A JSF commandButton is  used to provide a button to click on to display the next page of items.  The commandButton tag is used to submit an action event to the application. 

Code Sample from: list.xhtml

 <h:commandButton action="#{catalog.next}" value="#{msgs.Next}" />   


This commandButton action attribute references the catalog Managed bean next() method which calculates the next page's first row number  and returns a logical outcome String, which causes the list.xhtml page to display the next page's list . The catalog next method is defined as shown below:

Code Sample from: catalog.java

   public String next() {
       if (firstItem + batchSize < itemCount()) {
           firstItem += batchSize;
       }
       return "list";
   }


JSF 2.0 Simplified Navigation


The JavaServer Faces 2.0  NavigationHandler convention adds .xhtml to the logical outcome of the action method (in this example list) and loads that file, in this case, it loads the list.xhtml page after this method returns. If the action doesn't begin with a forward slash (/), JSF assumes that it's a relative path.  You can specify an absolute path by adding the slash like this "/items/list".


A JSF commandLink is  used to provide a link to click on to display a page with the item details. This commandLink action attribute  references The catalog getDetail() method:

Code Sample from: list.xhtml

   <h:column>
       <f:facet name="header">
          <h:outputText value="Name"/>
       </f:facet>
       <h:commandLink action="#{catalog.getDetail}" value="#{row.name}"/>   
   </h:column>


The catalog getDetail() method  gets the item data from the current row of the dataModel, and returns a string which causes the detail.xhtml page to display the item details :

Code Sample from: Catalog.java

    public String getDetail() {
        item = (Item) model.getRowData();
        return "detail";
    }


The JavaServer Faces NavigationHandler adds .xhtml to the logical outcome of the action, detail and loads that file. In this case, the JavaServer Faces implementation loads the detail.xhtml page after this method returns.

The detail.xhtml uses the outputText component to display the catalog ManagedBean's item properties:

Code Sample from: detail.xhtml

    <h:outputText value="#{catalog.item.name}" title="Name" />
    <h:outputText value="#{catalog.item.description}" title="Description"/>
    <h:graphicImage library="images" name="#{catalog.item.imageurl}" title="Imageurl" />

    <h:outputText value="#{
catalog.item.price}" title="Price" />
    <h:outputText value="#{
catalog.item.address.city}" title="Address" />
    <h:outputText value="#{
catalog.item.contactinfo.email}" title="Address"/>  





Hot Deployment and Session Retention with JSF 2.0 and Glassfish

  • Incremental compile of all JSF 2.0  artifacts when you save.
  • Auto-deploy of all web or Java EE 6 artifacts
  • Session retention: maintain stateful sessions across re-deployments


Conclusion
This concludes the sample application which demonstrates a pet catalog web application which uses JSF 2.0, JPA, GlassFish and MySQL.

Running the Sample Application

  1. If you haven't already done so, download and install NetBeans IDE , GlassFish , and MySQL Community Server . You can download and install GlassFish with NetBeans as a single bundle.
  2. Follow these instructions to install JSF Mojarra 2.0.0 Beta2 on GlassFish v2 or Glassfish v3
  3. Download the sample code.

Create the Pet Catalog database

In order to run the sample code you first have to create the Pet Catalog database and fill in  the Item table.

  1. Start NetBeans IDE
  2. Ensure that GlassFish is registered in the NetBeans IDE, as follows:
    • Click the Services tab in the NetBeans IDE.
    • Expand the Servers node. You should see GlassFish v2 in the list of servers. If not, register GlassFish v2 as follows:
      • Right-click the Servers node and select Add Server. This opens an Add Server Instance wizard.
      • Select GlassFish v2 in the server list of the wizard and click the Next button.
      • Enter the location information for the server and click the Next button.
      • Enter the admin name and password and click the Finish button.

  3. Start the MySQL or Java DB database as follows:
    • Click the Services tab in the NetBeans IDE.
    • Expand the databases node. You should see the Java DB database in the list of databases. If you have installed the MySQL server database, you should also see the MySQL database in the list of databases.. Note:  Java DB  comes bundled with Netbeans, you can  download MySQL separately.

    • Right-mouse click on the Java DB or MySQL server database and select Start.
  4. If you installed MySQL, set the properties of the MySQL server database as follows:
    • Right-click on the MySQL server database and select Properties. This opens the MySQL Server Properties dialog box, as shown in Figure 8.

      MySQL Server Basic Properties
      Figure 8. MySQL Server Basic Properties

    • In the Basic Properties tab, enter the server host name and port number. The IDE specifies localhost as the default server host name and 3306 as the default server port number.
    • Enter the administrator user name, if not displayed, and the administrator password -- the default administrator password is blank.
    • Click the Admin Properties tab.
    • Enter an appropriate path in the Path/URL to admin tool field. You can find the path by browsing to the location of a MySQL Administration application such as the MySQL Admin Tool.
    • Enter an appropriate path in the Path to start command. You can find the path by browsing to the location of the MySQL start command. To find the start command, look for mysqld in the bin folder of the MySQL installation directory.
    • Enter an appropriate path in the Path to stop command field. You can find the path by browsing to the location of the MySQL stop command. This is usually the path to mysqladmin in the bin folder of the MySQL installation directory. If the command is mysqladmin, in the Arguments field, type -u root stop to grant root permissions for stopping the server. The Admin Properties tab should look similar to Figure 9.

      MySQL Server Administration Properties
      Figure 9. MySQL Server Administration Properties

    • Click the OK button.

  5. Right-click on the MySQL server or Java DB database and select Start.
  6. Create the petcatalog database as follows:
    • Right-mouse click on the Java DB or MySQL server database and select Create Database. This will open a create Database window.
    • Enter the database name catalog for Java DB or petcatalog for MySQL.


      For Java DB enter userid password app app as shown below:


       Click O.K. to accept the displayed settings.
  7. Create the tables in the catalog database as follows:
    • Underneath Databases you should see a database connection for the petcatalog database. For example MySQL:

      or Java DB:

    • Right-mouse click on the petcatalog connection and select Connect.
    • Right-mouse click on the petcatalog connection and select Execute Command. This will open up a SQL command window.
    • Copy the contents of the catalog.sql file in the riapetcatalog\\exercises\\exercise0 directory and paste the contents into the SQL command window, as shown in below:

      Creating Tables in the Database
    • Click the Run SQL icon Run SQL icon (Ctrl+Shift+E) above the SQL command window.
    • Note: It is ok to see this: "Error code -1, SQL state 42Y55: 'DROP TABLE' cannot be performed on 'ITEM' because it does not exist. Line 2, column 1" . This just means you are deleting a table that does not exist.  If you need to delete and recreate the tables you will not  see this message the second time.
  8. View the data in the Pet Catalog database Item table as follows:
    • Underneath Databases you should see a database connection for the petcatalog database. For example MySQL:

      or Java DB:

    • If the database connection is broken like in the following diagram:

      • Right-mouse click on the petcatalog connection and select Connect. as shown below:

      • if prompted for a password, for MySQL leave it blank, for JavaDB enter user app password app.
    • Expand the Tables node below the petcatalog database in the Services window. You should see the item table under the Tables node. You can expand the item table node to see the table columns, indexes, and any foreign keys, as shown in below :
      An Expanded Table Node
      Figure 12. An Expanded Table Node

      You can view the contents of a table or column by right-clicking the table or column and selecting View Data as shown  below:

      Viewing the Contents of a Table
      Figure 13. Viewing the Contents of a Table


  9. Follow these instructions to Create a JDBC Connection pool and JDBC resource.Name the  pool mysql_petcatalog_rootPool and the jndi resource jdbc/petcatalog. Note: you do not have to create a JDBC connection pool and resource if you use the Netbeans wizard to generate JPA entities from database tables as described in this article GlassFish and MySQL, Part 2: Building a CRUD Web Application With Data Persistence.

Running the Sample solution:

If you want to run the sample solution, you have to create the catalog database tables first as described above.

  1. If you haven't already download the sample code and start the NetBeans IDE. Unzip the catalog.zip file which you downloaded, this will create a catalog directory with the project code.
  2. Open the catalog/setup/sun-resources.xml file and verify that the property values it specifies match those of the petcatalog database and jdbc resources you created. Edit the property values as necessary.

  3. Open the catalog project as follows:
    • In NetBeans IDE, click Open Project in the File menu. This opens the Open Project dialog.
    • Navigate in the Open Project dialog to the catalog  directory and click the Open Project button.

    In response, the IDE opens the catalog project.  You can view the logical structure of the project in the Projects window (Ctrl-1).
  4. Run the catalog by right-clicking on the catalog project in the Projects window and selecting Run Project. The NetBeans IDE compiles the application, deploys it on Glassfish, and brings up the default page in your browser.  (at http://localhost:8080/catalog/).

For more information see the following resources:



Wednesday Jul 01, 2009

2 JavaOne Hands On Labs , Sun Technology Exchange, Java Technology Day Israel, and Java Day Turkey

jax

June: 2 JavaOne Hands On Labs , Sun Technology Exchange, Java Technology Day Israel, and Java Day Turkey  

I had a very busy June, I gave two Hands on Labs at JavaOne, two sessions at the Sun Technology Exchange, three sessions at Java Technology Day in Tel Aviv Israel, and one session at Java Day in Istanbul Turkey.

JavaOne Hands On Labs:

iguana.jpg iguana.jpg

I co-developed and delivered 2 Hands On Labs for JavaOne this year:

You can download these 2 HOLs documentation and code below:


Sun Technology Exchange:

iguana.jpg iguana.jpg
In Fort Lauderdale as part of the Sun Technology Exchange I gave two educational sessions to learn how:
  • JavaFX can help you build rich internet applications (RIAs) and includes the tools and platform SDK for developers, web developers, and designers to create dynamic applications.
  • GlassFish, an enterprise-quality Java EE 5 application server, offers advanced clustering, centralized administration, and best-in-class performance.
  • download the slides


Java Technology Day Israel

iguana.jpg iguana.jpgiguana.jpg
At the Java Technology Day in Israel I gave the following sessions:
  • WSIT Reliability Security and Transactions in Web Services
    • Metro is a high-performance, extensible, easy-to-use web service stack. You can use it for every type of web service, from simple to reliable, secured, and transacted web services that interoperate with .NET services. Metro bundles stable versions of the JAX-WS (Java API for XML Web Services) reference implementation and WSIT (Web Services Interoperability Technology). JAX-WS is a fundamental technology for developing SOAP-based and RESTful Java technology-based web services. WSIT enables secure, reliable interoperability between Java technology-based web services and Microsoft's Windows Communication Foundation.
    • you can download and try out WSIT in this JavaOne HOL: Metro: Try Out Simple and Interoperable Web Services and with these lab instructions.
    • You can read more about some of the example code for this session at
      GlassFish and MySQL, Part 3: Creating a Pet Catalog Web Service
  • MySQL for Developers
    • If you are a developer using MySQL, you should learn enough to take advantage of its strengths, because having an understanding of the database can help you develop better-performing applications. This session talks about MySQL database design and SQL tuning for developers.
    • download or view a screencast of this presentation
  • OpenESB and Connecting Enterprises
    • This session  explains and demonstrates several concrete technologies that make SOA architecture possible - BPEL (Business Process Execution Language), JBI (Java Business Integration) and OpenESB. The part of of BPEL starts with an explanation of the requirements of standardized business process language. The BPEL language is then described using an example. The relationship between BPEL and WSDL is also explained. Finally, BPEL designer and runtime that comes with NetBeans IDE is demonstrated using Travel reservation sample BPEL project. It also explains the motivation of the JBI and OpenESB as a standardized application integration framework in the same way J2EE architecture standardized how enterprise applications are built and deployed. Finally Sun's solution in SOA and application integration space is discussed. Whenever possible, concrete steps of building, deploying and testing SOA applications will be demonstrated step by step.
  • download the slides for all 3

Java Day Turkey

iguana.jpg iguana.jpg iguana.jpg
At the Java Day in Istanbul Turkey I gave the MySQL for Developers session again, see above for more information.




About

caroljmcdonald

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today