Sunday May 29, 2016

Autosuggest generates huge amounts of DB transactions and severely impedes Indexer performance

I came upon an interesting issue last week. I had a 11.1.1.8.0 content server configured with a ADF UI domain with ~850,000 documents. Users were reporting that any new documents that were being checked-in using the interface or Desktop Integration Suite (DIS) - Windows Explorer Integration (WEI) were not immediately available. In fact they were taking far too long, at times up to 15 minutes, to become available. 

Monitoring the indexer, it became clear that the indexer thread, which is expected to start at a checkin/update or every 5 minutes by default, was not firing. I was able to manually start the Automatic Update Cycle though and it would pick up the documents in queue and index them immediately. It became pretty evident that something was holding back the indexer thread. There was also a general slowness with search query performance.

The content server systemdatabase traces were reporting massive amounts of database transactions to the CACHESTORE table, especially for Auto-Suggest index update.documents.doriginalname. 

>systemdatabase/7 05.25 12:21:45.396 Auto-Suggest index update.documents.doriginalname (start) SELECT dCacheValue FROM CacheStore WHERE dRegionName='autosuggestindexprimary' AND dCacheKey='autosuggestindexprimary.documents.doriginalname:OccurrenceStorage.2E85C9E57D12A7E9182954E57BC0707C'
>systemdatabase/6 05.25 12:21:45.398 Auto-Suggest index update.documents.doriginalname 2.51 ms. SELECT dCacheValue FROM CacheStore WHERE dRegionName='autosuggestindexprimary' AND dCacheKey='autosuggestindexprimary.documents.doriginalname:OccurrenceStorage.2E85C9E57D12A7E9182954E57BC0707C'[Executed. Returned row(s): true]
>systemdatabase/7 05.25 12:21:45.399 Auto-Suggest index update.documents.doriginalname (start) Executing PreparedStatement (UPDATE CacheStore SET dCacheValue=?, dCreateOrUpdateTime=?, dEntryStatus=?, dAutoExpiryTime=? WHERE dRegionName=? AND dCacheKey=?)
>systemdatabase/6 05.25 12:21:45.403 Auto-Suggest index update.documents.doriginalname 3.72 ms. Executing PreparedStatement (UPDATE CacheStore SET dCacheValue=?, dCreateOrUpdateTime=?, dEntryStatus=?, dAutoExpiryTime=? WHERE dRegionName=? AND dCacheKey=?)[Executed. 1 row(s) affected.]

The content server system logs were also reporting the following message frequently: 

!csSubjectMonitorStop!csUnableToLoadSubject,idccacheevent-ucm-persistent.autosuggestindexprimary,intradoc.server.cache.IdcCacheEventSubjectCallback!csIdcCacheNotificationError!syJavaExceptionWrapper,java.io.IOException: ORA-01555: snapshot too old: rollback segment number with name "" too small
ORA-22924: snapshot too old
!syJavaExceptionWrapper,java.sql.SQLException: ORA-01555: snapshot too old: rollback segment number with name "" too small
ORA-22924: snapshot too old

The database was reporting huge amounts (in GB) of REDO logs being generated within hours. The following SQL statements were being executed millions of times: 

UPDATE CacheStore SET dCacheValue=:1 , dCreateOrUpdateTime=:2 , dEntryStatus=:3 , dAutoExpiryTime=:4 WHERE dRegionName=:5 AND dCacheKey=:6
SELECT dCacheValue FROM CacheStore WHERE dRegionName=:"SYS_B_0" AND dCacheKey=:"SYS_B_1"

The AutoSuggest feature is required for the ADF UI domain to work and so the AutoSuggestConfig component on the content server cannot be disabled. However, you can disable the auto suggest activity by adding the following configuration entry into your config.cfg file and restarting the content server. 

EnableAutoSuggest=0

Sure enough, after making the changes, the system became much more responsive and the indexer thread was firing off as expected. This, however, did disable all auto suggest type-ahead fields, like the User ACL fields in the ADF UI. There is another configuration variable that can be used to disable auto suggest indexing of certain fields. As the issue is with the update.documents.doriginalname field, adding the following configuration entry into your config.cfg file and restarting the content server will also help without disabling auto suggest. 

# EnableAutoSuggest=0
DisabledAutoSuggestFields=table=Documents:fields=dOriginalName

I would highly recommend that you read and implement guidelines in this wonderful note by my colleague from Oracle Support, Cordell Melgaard, if you plan to use the ADF UI and auto-suggest: What is AutoSuggest and How To Tune It? (Doc ID 1938996.1)

Wednesday Apr 29, 2015

JDeveloper REST Connection for Oracle Documents Cloud

If you have read my earlier post about creating a Data Control for the RESTful services exposed by the Oracle Documents Cloud Service, you may have wondered why I did not use the out of the box Web Service Data Control (SOAP/REST).
Web Service Data Control (SOAP/REST) in Gallery

The reason is that the Web Service Data Control (SOAP/REST) feature that comes bundled with JDeveloper uses a URLConnection to connect to a RESTful Web Service. You will notice that when you try to test a URL Connection to the Documents cloud, it fails with a Authentication is required error message. URLConnection does not work well with the Oracle Documents Cloud yet.
Create URL Connection Dialog

The Mobile Application Framework update to JDeveloper introduces a new REST Connection resource. To install the Mobile Application Framework (MAF), go to Help -> Check for Updates. Click Next and select Mobile Application Framework and then follow the on screen instructions to install it. After JDeveloper restarts, you may see a warning message that MAF requires JDK 1.8 or higher. You can ignore this message for now.

Among other features, MAF installs a REST Connection resource. You can go to File -> New -> From Gallery -> General -> Connections to create a REST Connection.
New REST Connection in Gallery

You will also notice that the URL Connection dialog in the Create Web Service Data Control (REST) has been replaced with the new REST Connection dialog.
Create Web Service Data Control - Create REST Connection

Now when you try to connect to the Oracle Documents Cloud Service using a REST Connection, it connects just fine.
Create REST Connection Dialog

NOTE: You will need an Oracle Documents Cloud Services subscription and account for this example. Please note that these instructions should be used for development and testing purposes only, as such they are unsupported and should not be used on production environments.

Monday Apr 27, 2015

Being RESTful with Oracle Documents Cloud and Oracle ADF

Oracle Documents Cloud Service is the next-gen content collaboration solution built for the enterprise. It allows for anytime access to files from your Apple or Android device and automatically syncs files with your Mac or Windows PC. It provides application integration through multiple ways including embedding its Web UI and a powerful REST API.

In this post, I will demonstrate how to use the REST API to create a Web Application with Oracle ADF. The Web Application will list the files and folders in a base folder specified in a resource bundle file. It will allow a user to navigate inside and out of folder but does not allow file uploads and downloads yet. I will cover these topics in future posts.

This blog post shows you how to create a simple databound application. To see the complete application you will create, download the application here.

STEP 1: Create a new application and project

  1. Open JDeveloper (12.1.3) and from the main menu, select File -> New -> Application.
  2. In the Items list in the New Gallery, select ADF Fusion Web Application and click Ok.
  3. Enter a Application Name, say OracleDocumentsCloudDataControlPart1. Click Next in following screens to select the default values and finally select Finish to create a new application with the Model and ViewController projects.
STEP 2: Add the necessary libraries to the Model project
  1. In the Projects panel within the Applications window, select the Model project, right-click and select Project Properties.
  2. On the Libraries and Classpath page, use the Add Library... button to add the following libraries.
    1. Apache Http Components
    2. JAX-RS Jersey (Bundled)
    3. JAX-RS Jersey 2.x Client
  3. Create a resource bundle file, Model.properties and add the following entries. [Model.properties]
           ServiceURL=https://oradocs-corp.documents.us2.oraclecloud.com/documents/api/1.1
           UserName=abc.def-AT-ghi-DOT-com
           Password=mypassword#1
           BaseFolderID=self
STEP 3: Create Java classes to interact with Oracle Documents Cloud
  1. Create entity classes to represent items in Documents Cloud. [Type.java Item.java]
  2. Create a service class to submit the RESTful calls. This is the class we will use to create a Data Control. [ItemsDC.java]
STEP 4: Create a Data Control
  1. In the Applications window, right-click ItemsDC.java and select Create Data Control.
  2. In the Create Bean Data Control dialog, click Next, then Finish to accept the default values.
STEP 5: Create a JSF page and bind an ADF Faces table to the service bean
  1. In the Projects panel within the Applications window, select the ViewController project, right-click and select New > Page. [index1.jspx]
  2. In the Create JSF Page dialog, enter index1.jspx as the file name. On the Page Layout, select Copy Quick Start Layout and then choose the one column one row layout and click Ok. You may also select a layout of your choice.
  3. In the Data Controls panel, expand ItemsDC, expand listItemsInFolder(String) and then expand Parameters.
  4. Click and drag folderid to the center facet on the page in the visual editor. From the Create context menu, select Create -> Text -> ADF Input Text. Change the Visible property of the control to false.
  5. In the Data Controls panel, expand ItemsDC, expand listItemsInFolder(String) and then click and drag Item to the center facet below the ADF Input Text control (invisible). Select Create -> Table/List View -> ADF Table.
  6. In the Create Table dialog, change the Component To Use for all columns to ADF Output Text and click Ok.
  7. Open the Source tab for the index1.jspx file and locate the "#{row.name}" column. Comment the existing code for the ADF Output Text and replace it with a link control. To do this, just expand ItemsDC in the Data Controls panel, click and drag listItemsInFolder(String) to just after the code you just commented. Select Create -> Method -> ADF Link. Drag and drop a setPropertyListener component inside the ADF Link control. Set the attributes as shown below.
  8. Scroll down the Source window and remove the last two columns for file and folder.
STEP 6: Save and run the project
  1. Save all the files.
  2. Set the -Dweblogic.security.SSL.ignoreHostnameVerification=true JVM Argument.
  3. In the Projects panel within the Applications window, select the ViewController project, expand Web Content, right-click index1.jspx and click Run.
NOTE: You will need an Oracle Documents Cloud Services subscription and account for this example. Please note that these code snippets should be used for development and testing purposes only, as such it is unsupported and should not be used on production environments.

Wednesday May 21, 2014

WebCenter Portal .8: Extending portal functionality with ADF Taskflows

This post describes how to extend your portal application functionality using ADF Taskflow.[Read More]
About

Welcome to my blog. I use this site to share my experience as well as tips and tricks on Oracle Fusion Middleware products.

Contributors

Search

Archives
« July 2016
SunMonTueWedThuFriSat
     
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
      
Today