Tuesday Mar 01, 2016

Differences between Automatic Statistics Gathering job and GATHER_SCHEMA_STATS

Recently a customer raised a question whether there are differences between the Automatic Statistics Gathering job and a manual creation of stats via the GATHER_SCHEMA_STATS procedure.

The results in performance were quite interesting. Performance after an upgrade from Oracle Database 11.2.0.3 to Oracle Database 11.2.0.4 was not good when the automatic stats job got used. But performance changed significantly to the better when schema stats were created with the downside of taking more resources during the gathering.

Is the Automatic Stats Gathering job enabled?

That question can be answered quite easily. There's a very good MOS Note:1233203.1 - FAQ: Automatic Statistics Collection displaying this query:

SELECT CLIENT_NAME, STATUS FROM DBA_AUTOTASK_CLIENT WHERE CLIENT_NAME='auto optimizer stats collection';

The MOS Note has also the code to enable (or disable) the job.
.

Which parameters/settings are used?

That question is a bit more tricky as the Note says: "The automatic statistics-gathering job uses the default parameter values for the DBMS_STATS procedures". But how do I display them?

The following script will display the parameters being used during the Automatic Statistics Gathering:

SET ECHO OFF
SET TERMOUT ON
SET SERVEROUTPUT ON
SET TIMING OFF
DECLARE
   v1  varchar2(100);
   v2  varchar2(100);
   v3  varchar2(100);
   v4  varchar2(100);
   v5  varchar2(100);
   v6  varchar2(100);
   v7  varchar2(100);
   v8  varchar2(100);
   v9  varchar2(100);
   v10 varchar2(100);        
BEGIN
   dbms_output.put_line('Automatic Stats Gathering Job - Parameters');
   dbms_output.put_line('==========================================');
   v1 := dbms_stats.get_prefs('AUTOSTATS_TARGET');
   dbms_output.put_line(' AUTOSTATS_TARGET:  ' || v1);
   v2 := dbms_stats.get_prefs('CASCADE');
   dbms_output.put_line(' CASCADE:           ' || v2);
   v3 := dbms_stats.get_prefs('DEGREE');
   dbms_output.put_line(' DEGREE:            ' || v3);
   v4 := dbms_stats.get_prefs('ESTIMATE_PERCENT');
   dbms_output.put_line(' ESTIMATE_PERCENT:  ' || v4);
   v5 := dbms_stats.get_prefs('METHOD_OPT');
   dbms_output.put_line(' METHOD_OPT:        ' || v5);
   v6 := dbms_stats.get_prefs('NO_INVALIDATE');
   dbms_output.put_line(' NO_INVALIDATE:     ' || v6);
   v7 := dbms_stats.get_prefs('GRANULARITY');
   dbms_output.put_line(' GRANULARITY:       ' || v7);
   v8 := dbms_stats.get_prefs('PUBLISH');
   dbms_output.put_line(' PUBLISH:           ' || v8);
   v9 := dbms_stats.get_prefs('INCREMENTAL');
   dbms_output.put_line(' INCREMENTAL:       ' || v9);
   v10:= dbms_stats.get_prefs('STALE_PERCENT');
   dbms_output.put_line(' STALE_PERCENT:     ' || v10);
END;
/

The settings of the DBMS_STATS.GATHER_SCHEMA_STATS procedure are documented:
https://docs.oracle.com/database/121/ARPLS/d_stats.htm#ARPLS68577 

When you compare the two you'll see that the settings/defaults are identical. 
.

But what is the difference between these two?

Both activities use the same parameters. So the stats will look the same - IF they get created. The real difference between the Automatic Statistics Gathering job and a manual invocation of GATHER_SCHEMA_STATS is that the latter will refresh ALL statistics whereas the Automatic Statistics Gathering job will refresh only statistics on objects where statistics are missing or marked as STALE.

The same behavior appears when you compare the recommendation to gather dictionary statistics before the upgrade by using DBMS_STATS.GATHER_DICTIONARY_STATS versus a DBMS_STATS.GATHER_SCHMEA_STATS('SYS')call. The latter will refresh all statistics whereas the first one will take less resources but refresh only STALE and missing statistics.
.

A simple example

This script is kept as simple as possible.

  • It creates a test user
  • It creates two tables within this user - tablespace USERS
  • It inserts and updates information in the two tables
  • It flushes the monitoring information (how many DMLs got run?) out
  • It gathers stats on only one table to verify that STALE is working as intended
  • It kicks off the automatic stats gathering job
  • It kicks off the schema stats gathering call
  • It compares results before/after in the stats history table 

set timing on
set serverout on
set echo on
set termout on
column table_name Format a5
column owner      Format a6
column stale_stats Format a4
column last_analyzed Format a15
column sample_size format 9999999
drop user test1 cascade;
create user test1 identified by test1;
grant connect, resource, dba to test1;
alter user test1 default tablespace USERS;
create table TEST1.TAB1 as select * from dba_objects where rownum<50001;
exec dbms_stats.gather_table_stats('TEST1','TAB1');
create table TEST1.TAB2 as select * from dba_objects where rownum<50001;
exec dbms_stats.gather_table_stats('TEST1','TAB2');
insert into TEST1.TAB1 select * from dba_objects where rownum<50001;
commit;
insert into TEST1.TAB2 select * from dba_objects where rownum<50001;
commit;
insert into TEST1.TAB2 select * from dba_objects where rownum<50001;
commit;
update TEST1.TAB1 set object_id=object_id+0;
commit;
update TEST1.TAB2 set object_id=object_id+1;
commit;
exec DBMS_STATS.FLUSH_DATABASE_MONITORING_INFO;
select table_name,owner,stale_stats,to_char(last_analyzed,'DD-MON HH24:MI:SS') LAST_ANALYZED,SAMPLE_SIZE from dba_tab_statistics where table_name in ('TAB1','TAB2');
exec DBMS_STATS.GATHER_TABLE_STATS('TEST1','TAB1');
select table_name,owner,stale_stats,to_char(last_analyzed,'DD-MON HH24:MI:SS') LAST_ANALYZED,SAMPLE_SIZE from dba_tab_statistics where table_name in ('TAB1','TAB2');
exec DBMS_AUTO_TASK_IMMEDIATE.GATHER_OPTIMIZER_STATS;
pause Wait a bit - then press return ...
select table_name,owner,stale_stats,to_char(last_analyzed,'DD-MON HH24:MI:SS') LAST_ANALYZED,SAMPLE_SIZE from dba_tab_statistics where table_name in ('TAB1','TAB2');
exec dbms_stats.gather_schema_stats('TEST1');
select table_name,owner,stale_stats,to_char(last_analyzed,'DD-MON HH24:MI:SS') LAST_ANALYZED,SAMPLE_SIZE from dba_tab_statistics where table_name in ('TAB1','TAB2');
prompt End ...

.

The results

exec
DBMS_STATS.
FLUSH_DATABASE_MONITORING_INFO;
TABLE OWNER  STAL LAST_ANALYZED   SAMPLE_SIZE
----- ------ ---- --------------- -----------
TAB1  TEST1  YES  29-FEB 22:37:07       50000
TAB2  TEST1  YES  29-FEB 22:37:07       50000

exec
DBMS_STATS.
GATHER_TABLE_STATS('TEST1','TAB1');
TABLE OWNER  STAL LAST_ANALYZED   SAMPLE_SIZE
----- ------ ---- --------------- -----------
TAB1  TEST1  NO   29-FEB 22:37:12      100000
TAB2  TEST1  YES  29-FEB 22:37:07       50000

exec
DBMS_AUTO_TASK_IMMEDIATE.
GATHER_OPTIMIZER_STATS;

TABLE OWNER  STAL LAST_ANALYZED   SAMPLE_SIZE
----- ------ ---- --------------- -----------
TAB1  TEST1  NO   29-FEB 22:37:12      100000
TAB2  TEST1  NO   29-FEB 22:37:13      150000

exec
dbms_stats.
gather_schema_stats('TEST1');

TABLE OWNER  STAL LAST_ANALYZED   SAMPLE_SIZE
----- ------ ---- --------------- -----------
TAB1  TEST1  NO   29-FEB 22:37:43      100000
TAB2  TEST1  NO   29-FEB 22:37:43      150000

The results can be interpreted this way:

  • The sample size of 50k is based on the first activity during the CTAS
  • Once table TAB1 gets analyzed the sample size is now correct - and the time stamp got updated - statistics on TAB2 are still marked STALE of course as the underlying table has changed by more than 10%
  • The Automatic Statistics Gathering job will refresh only stats for objects where stats are missing or marked STALE - in this example here TAB2. Table TAB1's statistics remain unchanged.
  • When the GATHER_SCHEMA_STATS job gets invoked it will refresh all statistics - regardless if they were STALE or not. 

This is the behavior the customer who raised the question about differences in these two ways to create statistics may have seen. The GATHER_SCHEMA_STATS job took longer and consumed more resources as it will refresh all statistics regardless of the STALE attribute.

And it's hard to figure out why the refresh of statistics created in a previous release may have led to suboptimal performance, especially as we talk about a patch set upgrade - and not a full release upgrade. Thanks to Wissem El Khlifi who twittered the following annotations I forgot to mention:

  • The Automatic Statistics Gathering job prioritizes objects with NO statistics over objects with STALE statistics
  • The Automatic Statistics Gathering job may get interrupted or skip objects leaving them with NO statistics gathered. You can force this by locking statistics - so the Auto job will skip those completely

You'll find more information about the Automatic Statistics Gathering job here:

And another strange finding ...

When I played with this example in 12c I encountered the strange behavior of the GATHER_OPTIMIZER_STATS call taking exactly 10 minutes unti it returns to the command prompt.

First I thought this is a Multitenant only issue. But I realized quickly: this happens in non-CDB databases in Oracle 12c as well. And when searching the bug database I came across the following unpublished bug:

  • Bug 14840737
    DBMS_AUTO_TASK_IMMEDIATE.GATHER_OPTIMIZER_STATS RETURNS INCORRECTLY

which got logged in Oct 2012 and describes this exact behavior. I kick off the job - it will update the stats pretty soon after - but still take 10 minutes to return control to the command prompt. It is supposed to be fixed in a future release of Oracle Database ... 

 

--Mike 

Monday May 30, 2011

Will gathering fixed object stats reduce recompilation time post upgrade?

Interesting question, isn't it?
Will the time to recompile invalid objects post upgrade decreased once fixed object stats have been gathered?

First of all fixed object stats on X$-tables won't be gathered by default. X$ structures are undocumented. V$ views are build on top of them and should only be used even though it might be sometimes useful to access X$ tables such as X$BH (buffer headers - contains information describing the current contents of a piece of the buffer cache) sometimes directly. 

Anyway, back to the upgrade topic. Post upgrade you'll have a good bunch of invalid objects in your database. You would start now the recompilation with either utlrp.sql or, to decrease CPU load with utlrprp <number> to recompile with a distinct <number> of parallel threads. Since Oracle 10g we are using the package UTL_RECOMP for recompilation taking the object dependencies into consideration. Therefore jobs will be created to run the recompilation tasks in parallel and leverage from the CPU power of multiple cores. In both cases you'll get 4 queries to monitor progress and jobs as script output displayed.

In larger sized databases with many objects and components our recommendation is always to gather first fixed object stats prior to start the recompilation. Some time ago I've learned from to very different customer database projects that these stats will speed up the efficient job creation for recompilation. And last week I've got this feedback from an EBS 9.2.0.8 to 11.2.0.2 upgrade project:

  • Approx 120,000 objects invalid post database upgrade
  • Recompilation without fixed object stats: 14:44 hrs (yes, not minutes ... hours!)
  • Recompilation with fixed object stats: 12:09 hrs (!!!)
    Time it took to gather fixed object stats: 00:07 hrs
  • Benefit: 7 minutes to gather fixed object stats decreased the recompilation time by 2:35hrs (or by 18%)

  • How to gather fixed object stats in Oracle 11g:

    SQL> exec DBMS_STATS.GATHER_FIXED_OBJECTS_STATS;

It's always worth a try - you won't see a dramatic effect on a small ORCL type database with some thousand tables, indexes and views. But in a Siebel, Peoplesoft, EBS or any other huge application nvironment the effect might be remarkable. Still I agree in this EBS case: 12 hours recompilation time is a lot. The only way to decrease this time will be the (temporary) addition of CPU power to the system to satisfy more parallel recompilation threads.

About

Mike Dietrich - Oracle Mike Dietrich
Master Product Manager - Database Upgrade & Migrations - Oracle

Based in Germany. Interlink between customers/partners and the Upgrade Development. Running workshops between Arctic and Antartica. Assisting customers in their reference projects onsite and remotely. Connect via:

- -

Search

Archives
« May 2016
SunMonTueWedThuFriSat
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
    
       
Today
Slides Download Center
Visitors since 17-OCT-2011
White Paper and Docs
Workshops
Viewlets and Videos
Workshop Map
x Oracle related Tech Blogs
This week on my Rega & Pono
Upgrade Reference Papers