Monday Sep 24, 2012

Design for complex ATG applications

Overview

Needless to say, some ATG applications are more complex than others.  Some ATG applications support a single site, single language, single catalog, single currency, have a single development staff, single business team, and a relatively simple business model.  The real complex applications have to support multiple sites, multiple languages, multiple catalogs, multiple currencies, a couple different development teams, multiple business teams, and a highly complex business model (and processes to go along with it).  While it's still important to implement a proper design for simple applications, it's absolutely critical to do this for the complex applications.  Why?  It's all about time and money.  If you are unable to manage your complex applications in an efficient manner, the cost of managing it will increase dramatically as will the time to get things done (time to market).  On the positive side, your competition is most likely in the same situation, so you just need to be more efficient than they are.

This article is intended to discuss a number of key areas to think about when designing complex applications on ATG.  Some of this can get fairly technical, so it may help to get some background first.  You can get enough of the required background information from this post.  After reading that, come back here and follow along.


Application Design

Of all the various types of ATG applications out there, the most complex tend to be the ones in the telecommunications industry - especially the ones which operate in multiple countries.  To get started, let's assume that we are talking about an application like that.  One that has these properties:
  • Operates in multiple countries - must support multiple sites, catalogs, languages, and currencies
  • The organization is fairly loosely-coupled - single brand, but different businesses across different countries
  • There is some common functionality across all sites in all countries
  • There is some common functionality across different sites within the same country
  • Sites within a single country may have some unique functionality - relative to other sites in the same country
  • Complex product catalog (mostly in terms of bundles, eligibility, and compatibility)
At this point, I'll assume you have read through the required reading and have a decent understanding of how ATG modules work...

Code / configuration - assemble into modules

When it comes to defining your modules for a complex application, there are a number of goals:
  • Divide functionality between the modules in a way that maps to your business
  • Group common functionality 'further down in the stack of modules'
  • Provide a good balance between shared resources and autonomy for countries / sites
Now I'll describe a high level approach to how you could accomplish those goals...  Let's start from the bottom and work our way up.  At the very bottom, you have the modules that ship with ATG - the 'out of the box' stuff.  You want to make sure that you are leveraging all the modules that make sense in order to get the most value from ATG as possible - and less stuff you'll have to write yourself.  On top of the ATG modules, you should create what we'll refer to as the Corporate Foundation Module described as follows:

  • Sits directly on top of ATG modules
  • Used by all applications across all countries and sites - this is the foundation for everyone
  • Contains everything that is common across all countries / all sites
  • Once established and settled, will change less frequently than other 'higher' modules
  • Encapsulates as many enterprise-wide integrations as possible
  • Will provide means of code sharing therefore less development / testing - faster time to market
  • Contains a 'reference' web application (described below)
The next layer up could be multiple modules for each country (you could replace this with region if that makes more sense).  We'll define those modules as follows:

  • Sits on top of the corporate foundation module
  • Contains what is unique to all sites in a given country
  • Responsible for managing any resource bundles for this country (to handle multiple languages)
  • Overrides / replaces corporate integration points with any country-specific ones
Finally, we will define what should be a fairly 'thin' (in terms of functionality) set of modules for each site as follows:

  • Sits on top of the country it resides in module
  • Contains what is unique for a given site within a given country
  • Will mostly contain configuration, but could also define some unique functionality as well
  • Contains one or more web applications

The graphic below should help to indicate how these modules fit together:



Web applications

As described in the previous section, there are many opportunities for sharing (minimizing costs) as it relates to the code and configuration aspects of ATG modules.  Web applications are also contained within ATG modules, however, sharing web applications can be a bit more difficult because this is what the end customer actually sees, and since each site may have some degree of unique look & feel, sharing becomes more challenging.  One approach that can help is to define a 'reference' web application at the corporate foundation layer to act as a solid starting point for each site.  Here's a description of the 'reference' web application:

  • Contains minimal / sample reference styling as this will mostly be addressed at the site level web app
  • Focus on functionality - ensure that core functionality is revealed via this web application
  • Each individual site can use this as a starting point
  • There may be multiple types of web apps (i.e. B2C, B2B, etc)
There are some techniques to share web application assets - i.e. multiple web applications, defined in the web.xml, and it's worth investigating, but is out of scope here.


Reference infrastructure

In this complex environment, it is assumed that there is not a single infrastructure for all countries and all sites.  It's more likely that different countries (or regions) could have their own solution for infrastructure.  In this case, it will be advantageous to define a reference infrastructure which contains all the hardware and software that make up the core environment.  Specifications and diagrams should be created to outline what this reference infrastructure looks like, as well as it's baseline cost and the incremental cost to scale up with volume.  Having some consistency in terms of infrastructure will save time and money as new countries / sites come online.  Here are some properties of the reference infrastructure:

  • Standardized approach to setup of hardware
    • Type and number of servers
    • Defines application server, operating system, database, etc... - including vendor and specific versions
  • Consistent naming conventions
    • Provides a consistent base of terminology and understanding across environments
  • Defines which ATG services run on which servers
    • Production
    • Staging
    • BCC / Preview
  • Each site can change as required to meet scale requirements


Governance / organization

It should be no surprise that the complex application we're talking about is backed by an equally complex organization.  One of the more challenging aspects of efficiently managing a series of complex applications is to ensure the proper level of governance and organization.  Here are some ideas and goals to work towards:

  • Establish a committee to make enterprise-wide decisions that affect all sites
    • Representation should be evenly distributed
    • Should have a clear communication procedure
    • Focus on high level business goals
    • Evaluation of feature / function gaps and how that relates to ATG release schedule / roadmap
    • Determine when to upgrade & ensure value will be realized
  • Determine how to manage various levels of modules
    • Who is responsible for maintaining corporate / country / site layers
    • Determine a procedure for controlling what goes in the corporate foundation module
  • Standardize on source code control, database, hardware, OS versions, J2EE app servers, development procedures, etc
    • only use tested / proven versions - this is something that should be centralized so that every country / site does not have to worry about compatibility between versions
  • Create a innovation team
    • Quickly develop new features, perform proof of concepts
    • All teams can benefit from their findings

Summary

At this point, it should be clear why the topics above (design, governance, organization, etc) are critical to being able to efficiently manage a complex application.  To summarize, it's all about competitive advantage...  You will need to reduce costs and improve time to market with the goal of providing a better experience for your end customers.  You can reduce cost by reducing development time, time allocated to testing (don't have to test the corporate foundation module over and over again - do it once), and optimizing operations.  With an efficient design, you can improve your time to market and your business will be more flexible  and agile.  Over time, you'll find that you're becoming more focused on offering functionality that is new to the market (creativity) and this will be rewarded - you're now a leader.

In addition to the above, you'll realize soft benefits as well.  Your staff will be operating in a culture based on sharing.  You'll want to reward efforts to improve and enhance the foundation as this will benefit everyone.  This culture will inspire innovation, which can only lend itself to your competitive advantage.

Key ATG architecture principles

Overview

The purpose of this article is to describe some of the important foundational concepts of ATG.  This is not intended to cover all areas of the ATG platform, just the most important subset - the ones that allow ATG to be extremely flexible, configurable, high performance, etc.  For more information on these topics, please see the online product manuals.


Modules
The first concept is called the 'ATG Module'.  Simply put, you can think of modules as the building blocks for ATG applications.  The ATG development team builds the out of the box product using modules (these are the 'out of the box' modules).  Then, when a customer is implementing their site, they build their own modules that sit 'on top' of the out of the box ATG modules.  Modules can be very simple - containing minimal definition, and perhaps a small amount of configuration.  Alternatively, a module can be rather complex - containing custom logic, database schema definitions, configuration, one or more web applications, etc.  Modules generally will have dependencies on other modules (the modules beneath it).  For example, the Commerce Reference Store module (CRS) requires the DCS (out of the box commerce) module.

Modules have a ton of value because they provide a way to decouple a customers implementation from the out of the box ATG modules.  This allows for a much easier job when it comes time to upgrade the ATG platform.  Modules are also a very useful way to group functionality into a single package which can be leveraged across multiple ATG applications.

One very important thing to understand about modules, or more accurately, ATG as a whole, is that when you start ATG, you tell it what module(s) you want to start.  One of the first things ATG does is to look through all the modules you specified, and for each one, determine a list of modules that are also required to start (based on each modules dependencies).  Once this final, ordered list is determined, ATG continues to boot up.  One of the outputs from the ordered list of modules is that each module can contain it's own classes and configuration.  During boot, the ordered list of modules drives the unified classpath and configpath.  This is what determines which classes override others, and which configuration overrides other configuration.  Think of it as a layered approach.

The structure of a module is well defined.  It simply looks like a folder in a filesystem that has certain other folders and files within it.  Here is a list of items that can appear in a module:

MyModule:
  • META-INF - this is required, along with a file called MANIFEST.MF which describes certain properties of the module.  One important property is what other modules this module depends on.
  • config - this is typically present in most modules.  It defines a tree structure (folders containing properties files, XML, etc) that maps to ATG components (these are described below).
  • lib - this contains the classes (typically in jarred format) for any code defined in this module
  • j2ee - this is where any web-apps would be stored.
  • src - in case you want to include the source code for this module, it's standard practice to put it here
  • sql - if your module requires any additions to the database schema, you should place that schema here

Here's a screenshots of a module:

Modules can also contain sub-modules.  A dot-notation is used when referring to these sub-modules (i.e. MyModule.Versioned, where Versioned is a sub-module of MyModule).


Finally, it is important to completely understand how modules work if you are going to be able to leverage them effectively.  There are many different ways to design modules you want to create, some approaches are better than others, especially if you plan to share functionality between multiple different ATG applications.


Components

A component in ATG can be thought of as a single item that performs a certain set of related tasks.  An example could be a ProductViews component - used to store information about what products the current customer has viewed.  Components have properties (also called attributes).  The ProductViews component could have properties like lastProductViewed (stores the ID of the last product viewed) or productViewList (stores the ID's of products viewed in order of their being viewed).  The previous examples of component properties would typically also offer get and set methods used to retrieve and store the property values.  Components typically will also offer other types of useful methods aside from get and set.  In the ProductViewed component, we might want to offer a hasViewed method which will tell you if the customer has viewed a certain product or not.

Components are organized in a tree like hierarchy called 'nucleus'.  Nucleus is used to locate and instantiate ATG Components.  So, when you create a new ATG component, it will be able to be found 'within' nucleus.  Nucleus allows ATG components to reference one another - this is how components are strung together to perform meaningful work.  It's also a mechanism to prevent redundant configuration - define it once and refer to it from everywhere.

Here is a screenshot of a component in nucleus: 



Components can be extremely simple (i.e. a single property with a get method), or can be rather complex offering many properties and methods.  To be an ATG component, a few things are required:

  • a class - you can reference an existing out of the box class or you could write your own
  • a properties file - this is used to define your component
  • the above items must be located 'within' nucleus by placing them in the correct spot in your module's config folder
Within the properties file, you will need to point to the class you want to use:

$class=com.mycompany.myclass

You may also want to define the scope of the class (request, session, or global):

$scope=session

In summary, ATG Components live in nucleus, generally have links to other components, and provide some meaningful type of work.  You can configure components as well as extend their functionality by writing code.


Repositories

Repositories (a.k.a. Data Anywhere Architecture) is the mechanism that ATG uses to access data primarily stored in relational databases, but also LDAP or other backend systems.  ATG applications are required to be very high performance, and data access is critical in that if not handled properly, it could create a bottleneck.  ATG's repository functionality has been around for a long time - it's proven to be extremely scalable.  Developers new to ATG need to understand how repositories work as this is a critical aspect of the ATG architecture.  

Repositories essentially map relational tables to objects in ATG, as well as handle caching.  ATG defines many repositories out of the box (i.e. user profile, catalog, orders, etc), and this is comprised of both the underlying database schema along with the associated repository definition files (XML).  It is fully expected that implementations will extend / change the out of the box repository definitions, so there is a prescribed approach to doing this.  The first thing to be sure of is to encapsulate your repository definition additions / changes within your own module (as described above).  The other important best practice is to never modify the out of the box schema - in other words, don't add columns to existing ATG tables, just create your own new tables.  These will help ensure you can easily upgrade your application at a later date.

xml-combination

As mentioned earlier, when you start ATG, the order of the modules will determine the final configpath.  Files within this configpath are 'layered' such that modules on top can override configuration of modules below it.  This is the same concept for repository definition files.  If you want to add a few properties to the out of the box user profile, you simply need to create an XML file containing only your additions, and place it in the correct location in your module.  At boot time, your definition will be combined (hence the term xml-combination) with the lower, out of the box modules, with the result being a user profile that contains everything (out of the box, plus your additions).  Aside from just adding properties, there are also ways to remove and change properties.

types of properties

Aside from the normal 'database backed' properties, there are a few other interesting types:
  • transient properties - these are properties that are in memory, but not backed by any database column.  These are useful for temporary storage.
  • java-backed properties - by nature, these are transient, but in addition, when you access this property (by called the get method) instead of looking up a piece of data, it performs some logic and returns the results.  'Age' is a good example - if you're storing a birth date on the profile, but your business rules are defined in terms of someones age, you could create a simple java-backed property to look at the birth date and compare it to the current date, and return the persons age.
  • derived properties - this is what allows for inheritance within the repository structure.  You could define a property at the category level, and have the product inherit it's value as well as override it.  This is useful for setting defaults, with the ability to override.
caching

There are a number of different caching modes which are useful at different times depending on the nature of the data being cached.  For example, the simple cache mode is useful for things like user profiles.  This is because the user profile will typically only be used on a single instance of ATG at one time.  Simple cache mode is also useful for read-only types of data such as the product catalog.  Locked cache mode is useful when you need to ensure that only one ATG instance writes to a particular item at a time - an example would be a customers order.  There are many options in terms of configuring caching which are outside the scope of this article - please refer to the product manuals for more details.


Other important concepts - out of scope for this article

There are a whole host of concepts that are very important pieces to the ATG platform, but are out of scope for this article.  Here's a brief description of some of them:

  • formhandlers - these are ATG components that handle form submissions by users.
  • pipelines - these are configurable chains of logic that are used for things like handling a request (request pipeline) or checking out an order.
  • special kinds of repositories (versioned, files, secure, ...) - there are a couple different types of repositories that are used in various situations.  See the manuals for more information.
  • web development - JSP/ DSP tag library - ATG provides a traditional approach to developing web applications by providing a tag library called the DSP library.  This library is used throughout your JSP pages to interact with all the ATG components.
  • messaging - a message sub-system used as another way for components to interact.
  • personalization - ability for business users to define a personalized user experience for customers.  See the other blog posts related to personalization.

About

Welcome to the Oracle Commerce Product Strategy Community. We are dedicated to driving Best Business Practices across Oracle ATG products, Oracle ATG and Endeca Solutions and overall eCommerce Strategy. We welcome all collaborative discussions on blog posts. If you have a question for one of our experts, feel free to ask it here:
ATG Business Forum

Search

Archives
« September 2012 »
SunMonTueWedThuFriSat
      
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
25
26
27
28
29
30
      
Today