Changes to File Store Provider in UCM PS3

In the recent PS3 release of UCM (11.1.1.4.0) there are some significant changes to the File Store Provider (FSP) configuration. For new PS3 installs (not upgrades from PS2) the FSP default storage rule includes a dispersion rule that will change the web-layout and vault paths by adding dispersion directories to the paths to limit the number of files in the vault and web-layout directories. What that means is that if you install a new PS3 UCM instance and migrate content in from a previous version of UCM the web URL will change. That is a critical problem for web sites and just general document management.

Read the rest of this post where I describe the issue in detaill and provide instructions for how to modify a PS3 instance to use the old format for the web-layout path.In PS2 (11.1.1.3.0) the FSP configuration was the same as in 10gR3. Before you could use the storage rule feature of FSP you had to upgrade the FSP by clicking the Upgrade button on the FSP provider page. To see the FSP page go to the Providers page (Administration -> Providers) and click on the Info link for the DefaultFileStore Provider.

ps3fsp1small.jpg

In PS2 you will see that the FSP has an Upgrade button.

ps2fsp1small.jpg

In PS3 the FSP is automatically upgraded as part of the base install and you can use the Edit button to view the FSP configuration.

ps3fsp2small.jpg

After clicking on the Edit button you will see the Edit File Store Provider page and you can click the Edit Rule button to see the default storage rule.

ps3fsp3small.jpg

The default storage rule is displayed and you can see that it now include a dispersion rule.

ps3fsp4small.jpg

The default dispersion rule will result in vault and web-layout paths that have extra directory levels to spread the files across more directories. The impact on the Vault path is not important since that is not visible to the users, but the changes to the web-viewable path affects the web URL. Here is an example URL for a content item with the dispersion rules in place.

http://host:port/cs/groups/public/documents/document/mdaw/mdaw/~edisp/ucm_000010.pdf

The storage rule page does not allow you to edit the dispersion rule, web-viewable path or web URL file path. Accordiing to product management this is going to be addressed in a patch. Until then if you want to remove the dispersion rule from the vault and web-layout paths you will need to manually edit the FSP provider.hda file.

If you have already imported content into your PS3 instance follow these steps to remove the displersion rule from the vault and web-layout paths. if you do this before importing any content then you can save yourself the effort of the export and import.

  1. Create an archive and export all the content. Delete all the existing content as part of the export.
  2. Stop the UCM managed server.
  3. Edit the following file:

    domain_directory/ucm/cs/data/providers/defaultfilestore/provider.hda

  4. To change the vault path, replace

    vault
    $#env.VaultDir$$dDocType$/$dDocAccount$/$dispersion$/$dID$$ExtensionSeparator$$dExtension$

    with

    vault
    $#env.VaultDir$$dDocType$/$dDocAccount$/$dID$$ExtensionSeparator$$dExtension$

  5. To change the web-layout path and web URL, replace

    web
    $FsWeblayoutDir$groups/$dSecurityGroup$/$dDocAccount$/documents/$dDocType$/$dispersion$/~edisp/$dDocName$$RenditionSpecifier$$RevisionLabel$$ExtensionSeparator$$dWebExtension$

    with

    web
    $FsWeblayoutDir$groups/$dSecurityGroup$/$dDocAccount$/documents/$dDocType$/$dDocName$$RenditionSpecifier$$RevisionLabel$$ExtensionSeparator$$dWebExtension$

    and replace

    weburl.file
    $FsHttpWebRoot$groups/$dSecurityGroup$/$dDocAccount$/documents/$dDocType$/$dispersion$/~edisp/$dDocName$$RenditionSpecifier$$RevisionLabel$$ExtensionSeparator$$dWebExtension$

    with

    weburl.file
    $FsHttpWebRoot$groups/$dSecurityGroup$/$dDocAccount$/documents/$dDocType$/$dDocName$$RenditionSpecifier$$RevisionLabel$$ExtensionSeparator$$dWebExtension$

  6. Restart the UCM managed server.
  7. Import all the content from the Archive created in step #1.
  8. Validate the web-viewable URL no longer contains the dispersion directories.
  9. Check in a new content item and validate the web-viewable URL no longer contains the dispersion directories.

Comments:

What is mdaw, and why are there two of them?

Posted by guest on May 18, 2011 at 06:10 AM CDT #

mdaw comes from the Dispersion rule $dRevClassID[-9:-6:0:b]/$dRevClassID[-6:-3:0:b] This uses certain digits of the dRevClassID and :b encodes it. Once you get a lot of content checked in your will see something like this in your vault or weblayout using the new storage rule. .../ucm/cs/weblayout/groups/public/documents/digitalmedia/mdaw> ll total 32 drwxrwxr-x 3 oracle dba 4096 Mar 30 16:53 mda0 drwxrwx--- 3 oracle dba 4096 Mar 31 14:11 mda1 drwxrwx--- 3 oracle dba 4096 Apr 5 13:57 mda2 drwxrwx--- 3 oracle dba 4096 Apr 9 15:35 mda5 drwxrwxr-x 3 oracle dba 4096 Mar 30 10:17 mdaz drwxrwx--- 3 oracle dba 4096 Apr 27 08:38 mde0 drwxrwx--- 3 oracle dba 4096 Apr 27 12:49 mde1 drwxrwx--- 3 oracle dba 4096 Apr 26 16:11 mdez

Posted by guest on May 18, 2011 at 06:24 AM CDT #

Which document describes the usage of the brackets, e.g. [-9:-6:0:b] in the dispersion rule. I wonder how to get the first character of the a string, [-3:0:0] won't work? Thanks.

Posted by guest on May 19, 2011 at 08:54 PM CDT #

There is a description of the dispersion rule syntax in the System Admin Guide. See section 4.3.5.5.3 Limiting the Number Files in a Directory. http://download.oracle.com/docs/cd/E21764_01/doc.1111/e10792/c02_settings.htm#BEICFCEI

Posted by Kevin on May 20, 2011 at 12:00 AM CDT #

How do I proceed using UCM10g? Any dispersion rules available? We have millions of content items and more than 100,000 items per folder. Thanks in advance

Posted by guest on July 12, 2011 at 03:45 AM CDT #

In UCM 10g you can create your own dispersion rule using portions of the dID to generate directoryt levels in the vault and weblayout paths.

Update the FSP paths as follows:

Vault:

$#env.VaultDir$$dDocType$/$dDocAccount$/$dID[-12:-10:0]/$dID[-10:-8:0]$/$dID[-8:-4:0]$/$dID$$ExtensionSeparator$$dExtension$

WebLayout:

$#env.WeblayoutDir$groups/$dSecurityGroup$/$dDocAccount$/sg/$dDocType$/$dID[-12:-10:0]/$dID[-10:-8:0]$/$dID[-8:-4:0]$/d/$dDocName$$RenditionSpecifier$$RevisionLabel$$ExtensionSeparator$$dWebExtension$

Web URL:

$HttpWebRoot$groups/$dSecurityGroup$/$dDocAccount$/sg/$dDocType$/$dID[-12:-10:0]/$dID[-10:-8:0]$/$dID[-8:-4:0]$/d/$dDocName$$RenditionSpecifier$$RevisionLabel$$ExtensionSeparator$$dWebExtension$

The syntax for the usage of the dID variable, e.g., $dID[-12:-10:0], is explained in the 10g FSP manual. The example above will create a layout with at most 10,000 files or sub-directories per directory.

Posted by Kevin on July 18, 2011 at 08:07 AM CDT #

Ok, but where should I change the value of the variable env.VaultDir?

Posted by guest on August 02, 2011 at 07:41 AM CDT #

I am trying to do complete the steps you listed... but I'm getting an error on import

"Import error for archive 'all_content' in collection 'ucmdev': Unable to execute service INSERT_NATIVE and function docRefinery.
The path metadata for rule 'default' and storage class 'web' is missing."

Not sure where to go from here. I know all my dev content is gone... and that worries me a bit.

Posted by Bryan on October 25, 2011 at 02:05 AM CDT #

We will be upgrading to 11g PS5 from 10g. Is it possible to import existing content from 10g without the dispersion rule so url for existing content does not change, but apply the dispersion for any new content?
Thanks.

Posted by guest on April 18, 2013 at 12:58 PM CDT #

Yes, I don't know if it exists in PS5, but in PS6 there is a storage rule called "Web Asset" that does not have a dispersion rule. If this exists in PS5 you could use it when importing your content. If it does not exist in PS5 then you can add a rule of your own that does not have a dispersion rule and will keep the same path as the 10g UCM system.

Either way you will need to use an import map in Archiver to set the storage rule field for the content you are migrating into 11g. If you don't do this it will get 11g default storage rule which has a dispersion rule.

BTW, is later version of 11g the default value for the xStorageRule field is no longer "default" but is now DispByContentId.

Posted by Kevin Smith on April 23, 2013 at 07:44 AM CDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Kevin Smith is a Technical Director in Oracle Consulting's WebCenter practice. He has been working with content management products since 2004 when he joined Stellent.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today