Thursday Dec 03, 2015

Upgrade to Oracle Database 12c and Avoid Query Regression

Those of you that made it to the demo grounds at Oracle Open World this year (BTW - it’s still 2015 - just) will have had the chance to chat with the Oracle developers and throw lots of difficult questions at them! For everyone in the Optimizer development team it was a great opportunity to get lots of interaction and feedback, so thanks to all of you that took the time to come along and talk to us. We’re not all lucky enough to get a trip out to San Francisco, so I’ve been collating the main topics that came up to steer the subject matter of the next few blog posts. In this way I hope that we can all benefit from these interactions.

I can tell you right away that the number one demo ground question at OOW 2015 was …drum roll… “How can I reduce the risk of query regression when I upgrade to Oracle Database 12c?”. Well, maybe not those words exactly, but words to that effect. There is quite a lot of information out there on this topic, but people seem to struggle to find it… so we put our heads together and we realized that we should publish a 20,000ft view of this topic with pointers down into the detail. That’s the purpose of this post and, for the sake of brevity, I’m going to make the assumption that you are upgrading your Enterprise Edition database from Oracle Database 11g Release 2 to Oracle Database 12c.

The bottom line is this: if you want to mitigate the risk of query regression when you upgrade to Oracle Database 12c, then use SQL Plan Management (SPM). This is the recommended approach for the vast majority of systems out there, where the most critical SQL statements are reusable or, in other words, they are executed more than once.

Here are a couple of common scenarios:

Scenario#1 You want to use all Oracle Database 12c Optimizer features right away, but you need to “repair” any regressed queries quickly and with minimum effort.
Scenario#2 You’re upgrading and want to keep your “tried-and-tested”, Oracle Database 11g execution plans. Nevertheless, you do not want your application to be frozen in time: you want to evolve and use improved execution plans that are available in the new release, and you need to do this in a proven and controlled way.

Scenario 1

This is something you’ll want to think about before your production system goes live, particularly if you have not been able to test a realistic workload on all of your production data. It’s also very relevant if you are running a benchmark or proof of concept, where the time that’s available to resolve problem queries can be pretty limited (I’m using some understatement there!).

Ideally you will have captured SQL plan baselines before you’ve upgraded, because then you’ll have a set of “good” execution plans at-the-ready. It isn’t absolutely necessity to do this, though. As long as you can reproduce or find an example of a good plan, then this can be used to create a SQL plan baseline on-demand. For example, you may find a better plan:

  • By running the problem query in a pre-upgrade environment (remembering that you can export and import SQL plan baselines to copy them between databases).
  • Inside an existing SQL Tuning Set (STS).
  • By reproducing the good plan in the post-upgrade environment using (for example) “alter session set optimizer_features_enabled = 11…”, adjusting other Optimizer database parameters or by using hints. Yes, setting this parameter to an older version will give you the plan of the older version; that’s the whole purpose of it (and if it doesn’t work for you then it usually means that there’s a bug).

The next step is the particularly clever part, but I get the impression that a lot of Oracle experts don’t know that it’s even possible. When you’ve found a good plan and captured the details in a SQL plan baseline, you can use SPM to associate it with a regressed SQL statement without having to change the existing query or the existing application code. For details, take a look in the section, “Creating an accepted plan by modifying the SQL text” in an earlier Optimizer blog entry and also page 24 of SQL Plan Management with Oracle Database 12c. In both cases, an improved SQL execution plan is found using a hint. This plan is associated with a regressed SQL statement so that, in future, the better plan is used.

Scenario 2

You should capture SQL Plan Baselines in your Oracle Database 11g environment and export them so that they can be imported into the upgraded database. If you are upgrading in-place, then existing SQL plan baselines will be available without the need to export and import them. If you neglected to capture baselines in the pre-upgrade environment, then you still have the option to capture 11.2 execution plans in an Oracle Database 12c environment by executing your queries in a session where the database parameter optimizer_features_enabled is set to “11.2.0.4” (or whatever version you like).

Once SQL plan baselines are established in the upgraded database, you will enjoy plan stability while you get to know your system in the critical early days after the upgrade. Once you are happy with your shiny new database, you can evolve SQL plan baselines either automatically or at your own pace. Then you will gain advantage of all the new Optimizer features available in Oracle Database 12c.

Licensing

SQL Plan Management is an Oracle Database Enterprise Edition (EE) feature (you can see this here). You don’t need to buy an additional license to use SPM on EE. Optionally, you might choose to use SPM with SQL Tuning Sets (STS). If you do, then you will need to have purchased the Oracle Tuning Pack for Oracle Database (PDF) in adition to EE because STS requires this pack.

Top Tip

Whenever you plan to upgrade, check out Oracle’s Database Upgrade Blog. It’s full of really great information and it will hook you up with the latest advice at the finest level of detail. For example, here are some useful specifics on SPM to get you started.

So now is the time to upgrade, unless you’re a retailer like Amazon who’s heating up its systems for the big Christmas push, or perhaps you’re heading into your end-of-year financial reporting period. Nevertheless, even for you, the “now” is pretty close…



Monday Sep 07, 2015

Tips on SQL Plan Management and Oracle Database In-Memory – Part 3

In Part 1 of this series of tips on SQL Plan Management (SPM) and Oracle Database In-Memory I covered an example where a full table scan query made use of the In-Memory column store immediately without changing the SQL execution plan. In Part 2 I presented an example where the In-Memory column store made an alternative SQL execution plan viable, and where there was a corresponding SQL plan baseline already in place so that this plan could be used immediately. 

In this post I will consider a slightly different scenario:
  • SQL plan management is used to stabilize the SQL execution plans for our critical application (that until now has not been using the In-Memory column store).
  • The In-Memory column store is subsequently enabled and populated with application tables. 
  • The Optimizer identifies new SQL execution plans.
  • Most of the new execution plans have never been chosen by the Optimizer before.
Let me say right away; SPM behaves in a business-as-usual manner: 
  • New plans for existing baselines are captured by SPM but they will not be used until they are accepted.
  • Existing SQL plan baselines are used, so queries continue to use "approved" SQL execution plans.
  • The database administrator chooses how and when to evolve the SQL plan baselines to take full advantage of In-Memory SQL execution plans.
This is probably one of the most common scenarios you’ll encounter if you use SPM and you start to use Oracle Database In-Memory. As Andy Rivenes pointed out in his blog post, SPM is a very good way to avoid query regressions by controlling how and when queries are affected as you populate the In-Memory column store with more and more tables. I'll use the following example to show you how SPM behaves:
  • There is a query called Q3.
  • Q3 queries a table called MYSALES. 
  • MYSALES is not yet populated into the In-Memory column store.
  • Q3 filters rows in MYSALES using a column called SALE_TYPE.
  • SALE_TYPE has relatively low cardinality, but an index is still useful.
  • There is a SQL plan baseline for Q3 to ensure that it will uses an index range scan and not a full table scan.
This is the plan before the In-Memory column store is populated with MYSALES:

PLAN_TABLE_OUTPUT
-----------------
SQL_ID  8xkx5abshb4rz, child number 2
-------------------------------------
select /* SPM */ count(*),sum(val) from mysales where sale_type in (2,3)

Plan hash value: 719460714

-------------------------------------------------------------------------------------------------
|     | Operation                             | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                      |         |       |       |   246 (100)|          |
|   1 |  SORT AGGREGATE                       |         |     1 |     7 |            |          |
|   2 |   INLIST ITERATOR                     |         |       |       |            |          |
|   3 |    TABLE ACCESS BY INDEX ROWID BATCHED| MYSALES | 20000 |   136K|   246   (0)| 00:00:01 |
|*  4 |     INDEX RANGE SCAN                  | SI      | 20000 |       |    40   (0)| 00:00:01 |
-------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   4 - access(("SALE_TYPE"=2 OR "SALE_TYPE"=3))

Note
-----
   - SQL plan baseline SQL_PLAN_bk42daz2f53zwb9fe04b5 used for this statement

It's an index range scan, and the “note” section (above) shows us that a SQL plan baseline is being used (it's name ending in "4b5" ). Let's take a look at our baselines- there's just one: 

SELECT plan_name,sql_handle,sql_text,enabled, accepted
FROM   dba_sql_plan_baselines
WHERE  sql_text LIKE '%SPM%';

PLAN_NAME                       SQL_HANDLE           SQL_TEXT                            ENA ACC
------------------------------- -------------------- ----------------------------------- --- ---
SQL_PLAN_bk42daz2f53zwb9fe04b5  SQL_b9104d57c4e28ffc select /* SPM */ count(*),sum(val)  YES YES
                                                     from mysales
                                                     where sale_type in (2,3)

Now, populate the In-Memory column store:  

-- Mark MYSALES with the In-Memory attribute
ALTER TABLE mysales INMEMORY;

-- Access MYSALES to trigger population into In-Memory column store
SELECT count(*) FROM mysales;

If we re-run Q3, we still get an index range scan rather than the INMEMORY FULL scan we might have anticipated (because an In-Memory scan can be more efficient than an index range scan in some cases): 

-------------------------------------------------------------------------------------------------
|     | Operation                             | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                      |         |       |       |   246 (100)|          |
|   1 |  SORT AGGREGATE                       |         |     1 |     7 |            |          |
|   2 |   INLIST ITERATOR                     |         |       |       |            |          |
|   3 |    TABLE ACCESS BY INDEX ROWID BATCHED| MYSALES | 20000 |   136K|   246   (0)| 00:00:01 |
|*  4 |     INDEX RANGE SCAN                  | SI      | 20000 |       |    40   (0)| 00:00:01 |
-------------------------------------------------------------------------------------------------

Has the Optimizer decided that an index range scan is still the best option? We can answer that if we take another look at the SQL plan baselines: 

SELECT plan_name,sql_handle,sql_text,enabled, accepted
FROM   dba_sql_plan_baselines
WHERE  sql_text LIKE '%SPM%';

PLAN_NAME                       SQL_HANDLE           SQL_TEXT                            ENA ACC
------------------------------- -------------------- ----------------------------------- --- ---
SQL_PLAN_bk42daz2f53zwc69cec1f  SQL_b9104d57c4e28ffc select /* SPM */ count(*),sum(val)  YES NO
                                                     from mysales
                                                     where sale_type in (2,3)
SQL_PLAN_bk42daz2f53zwb9fe04b5  SQL_b9104d57c4e28ffc select /* SPM */ count(*),sum(val)  YES YES
                                                     from mysales
                                                     where sale_type in (2,3)

OK. There’s a new baseline, but it isn’t accepted (the value “NO” appears in the accepted column). This is exactly what SPM is supposed to do: we continue to use accepted and "approved" plans until we have verified or chosen to use alternatives. What is the new baseline plan in this case?

SELECT PLAN_TABLE_OUTPUT
FROM   V$SQL s, DBA_SQL_PLAN_BASELINES b,
        TABLE(
          DBMS_XPLAN.DISPLAY_SQL_PLAN_BASELINE(b.sql_handle,b.plan_name,'basic')
        ) t
WHERE  s.EXACT_MATCHING_SIGNATURE=b.SIGNATURE
AND    b.PLAN_NAME='SQL_PLAN_bk42daz2f53zwc69cec1f';

PLAN_TABLE_OUTPUT
-----------------
--------------------------------------------------------------------------------
SQL handle: SQL_b9104d57c4e28ffc
SQL text: select /* SPM */ count(*),sum(val) from mysales where sale_type in
         (2,3)
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Plan name: SQL_PLAN_bk42daz2f53zwc69cec1f         Plan id: 3332172831
Enabled: YES     Fixed: NO      Accepted: NO      Origin: AUTO-CAPTURE
Plan rows: From dictionary
--------------------------------------------------------------------------------

Plan hash value: 3292460164

-----------------------------------------------
| Id  | Operation                   | Name    |
-----------------------------------------------
|   0 | SELECT STATEMENT            |         |
|   1 |  SORT AGGREGATE             |         |
|   2 |   TABLE ACCESS INMEMORY FULL| MYSALES |
-----------------------------------------------

There it is! The Optimizer has established that the In-Memory full table scan is a good choice, but it will not be used until the new SQL plan baseline has been accepted. Let's go ahead and accept it, but take note that in this example there’s a good chance that the difference in performance will be very small because, after all, it’s only a simple query on a small dataset. If the performance difference is small then automatic plan evolution won’t deem the performance improvement to be sufficient to trigger automatic acceptance of the new baseline. It's worth remembering this if you find that you have a bunch of new baselines that are not accepted automatically. I'll use “verify=>’NO’” to force acceptance: 

cVal := dbms_spm.evolve_sql_plan_baseline(sql_handle=>' SQL_b9104d57c4e28ffc',verify=>'NO');

SELECT plan_name,sql_handle,sql_text,enabled, accepted
FROM   dba_sql_plan_baselines
WHERE  sql_text LIKE '%SPM%';

PLAN_NAME                       SQL_HANDLE           SQL_TEXT                            ENA ACC
------------------------------- -------------------- ----------------------------------- --- ---
SQL_PLAN_bk42daz2f53zwc69cec1f  SQL_b9104d57c4e28ffc select /* SPM */ count(*),sum(val)  YES YES
                                                     from mysales
                                                     where sale_type in (2,3)
SQL_PLAN_bk42daz2f53zwb9fe04b5  SQL_b9104d57c4e28ffc select /* SPM */ count(*),sum(val)  YES YES
                                                     from mysales
                                                     where sale_type in (2,3)

Now, re-run the query:

PLAN_TABLE_OUTPUT
-----------------
SQL_ID  8xkx5abshb4rz, child number 0
-------------------------------------
select /* SPM */ count(*),sum(val) from mysales where sale_type in (2,3)

Plan hash value: 3292460164

---------------------------------------------------------------------------------------
|     | Operation                   | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |         |       |       |    24 (100)|          |
|   1 |  SORT AGGREGATE             |         |     1 |     7 |            |          |
|*  2 |   TABLE ACCESS INMEMORY FULL| MYSALES | 20000 |   136K|    24   (9)| 00:00:01 |
---------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

   2 - inmemory(("SALE_TYPE"=2 OR "SALE_TYPE"=3))
       filter(("SALE_TYPE"=2 OR "SALE_TYPE"=3))

Note
-----
   - SQL plan baseline SQL_PLAN_bk42daz2f53zwc69cec1f used for this statement

That’s more like it! We have accepted the new In-Memory execution plan and we’ve done it in a controlled manner. We are using the new SQL plan baseline (the name ends in "c1f" ).

In reality, you might have thousands of SQL plan baselines to evolve, but you can use the SPM evolve advisor task to automate the process of verification and acceptance. If you use this feature, then any SQL statements in your baseline that don’t benefit from the In-Memory column store significantly will continue to use their existing SQL execution plans.  

In this series of posts I don’t pretend to have covered every possible scenario, but I hope that this has given some idea of how SPM will behave if you choose to use Oracle Database In-Memory. I'm still not absolutely sure that "Part 3" will be the last part, so this might end up being a trilogy in four or five parts (to steal a quote from a famous author).You can take and develop the scripts I wrote to try out scenarios of your own. They are available on GitHub. So go ahead and check them out, and post any questions you have in the comments section below.


Tuesday Aug 25, 2015

Tips on SQL Plan Management and Oracle Database In-Memory - Part 2

In Part 1 of this series of tips on SQL Plan Management (SPM) and Oracle Database In-Memory, I covered what would happen if we have a SQL plan baseline for a full table scan query when the table was populating the In-Memory column store. 

In this part I’m going to cover a scenario where a query has more than one SQL plan baseline: 

  • There is a query (called Q2, for short).
  • Q2 queries a table called MYSALES, which is not yet populating the In-Memory column store.
  • Q2 filters rows in MYSALES using a predicate on the SALE_TYPE column.
  • Data in SALE_TYPE is skewed, so there’s an index and a histogram on this column.
  • Because there is data skew, Q2 has two accepted SQL plan baselines; one with a full table scan and one with an index range scan.

You’ve probably come across this situation many times: the Oracle Optimizer must choose between a full table scan or an index range scan depending on predicate selectivity. The ability to change the execution plan based on the value of bind variables is called adaptive cursor sharing. If you’ve not come across that, then you’ll find it useful to check out the section on this topic in the Database SQL Tuning Guide.

What’s great about SPM is that it allows you to have multiple SQL plan baselines for individual queries, so you're not forced to pick one plan in preference to another. This capability is most relevant in environments where SQL statements use bind variables and there is a good deal of data skew. Queries like this are likely to have their plans affected by Oracle In-Memory Database because in-memory full table scans will have a lower cost than storage-resident table scans. Clearly, the In-Memory column store will affect the point of inflection where a full table scan will become more efficient than an index range scan. How is this going to work with SPM? 

Take a look at the following example. Q2 executes and matches 2 million rows because I picked the value of bind variable “:val” to do just that. The Optimizer chooses a full table scan: 

PLAN_TABLE_OUTPUT
-----------------
SQL_ID  d3u63rk540w0r, child number 1
-------------------------------------
select /* SPM */ count(*),sum(val) from mysales where sale_type = :val

Plan hash value: 3292460164

------------------------------------------------------------------------------
  Id  | Operation          | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |         |       |       |  2475 (100)|          |
|   1 |  SORT AGGREGATE    |         |     1 |    17 |            |          |
|*  2 |   TABLE ACCESS FULL| MYSALES |  2000K|    32M|  2475   (1)| 00:00:01 |
------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------

  2 - filter("SALE_TYPE"=:VAL)

Note
-----
  - SQL plan baseline SQL_PLAN_93ct9zmnvtbuhc69cec1f used for this statement

For the second execution, the value “:val” is set so that it would match only 20,001 rows. This time the Optimizer chooses an index range scan: 

PLAN_TABLE_OUTPUT
-----------------
SQL_ID  d3u63rk540w0r, child number 2
-------------------------------------
select /* SPM */ count(*),sum(val) from mysales where sale_type = :val

Plan hash value: 1266559460

------------------------------------------------------------------------------------------------
  Id  | Operation                            | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
------------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                     |         |       |       |   133 (100)|          |
|   1 |  SORT AGGREGATE                      |         |     1 |    17 |            |          |
|   2 |   TABLE ACCESS BY INDEX ROWID BATCHED| MYSALES | 20001 |   332K|   133   (0)| 00:00:01 |
|*  3 |    INDEX RANGE SCAN                  | SI      | 20001 |       |    44   (0)| 00:00:01 |
------------------------------------------------------------------------------------------------

Predicate Information (identified by operation id):a
---------------------------------------------------
  3 - access("SALE_TYPE"=:VAL)

Note
-----
  - SQL plan baseline SQL_PLAN_93ct9zmnvtbuh5d8bf80c used for this statement

As you will have figured out, the Optimizer has calculated that the index is less efficient than a full table scan when Q2 matches a large number of rows (2 million in this case) so we have two viable SQL execution plans for this query. Before I ran the queries above, I accepted two SQL plan baselines for Q2. You can see in the “note” sections above that two different baselines are used (one ending in “80c” and one ending in “c1f”). They can be seen in the dba_sql_plan_baselines view: 

SELECT plan_name,sql_text,enabled, accepted 
FROM   dba_sql_plan_baselines
WHERE  sql_text LIKE '%SPM%';

PLAN_NAME                           SQL_TEXT                                ENA ACC
----------------------------------- ----------------------------------      --- ---
SQL_PLAN_93ct9zmnvtbuhc69cec1f      select /* SPM */ count(*),sum(val)      YES YES
                                    from mysales where sale_type = :val           
SQL_PLAN_93ct9zmnvtbuh5d8bf80c      select /* SPM */ count(*),sum(val)      YES YES
                                    from mysales where sale_type = :val

We’re good shape here. The Optimizer is adapting the query execution plan to take into account bind variable values and data skew. What’s more, SPM is working with us and not against us because it is not forcing Q2 to use a single SQL execution plan.

What happens if we populate MYSALES into the In-Memory column store? 

-- Mark MYSALES with the In-Memory attribute
ALTER TABLE mysales INMEMORY;

-- Access MYSALES to trigger population into In-Memory column store
SELECT count(*) FROM mysales;

If we execute Q2 to match 2 million rows, the Optimizer continues to choose a full table scan: 

PLAN_TABLE_OUTPUT
-----------------
SQL_ID  d3u63rk540w0r, child number 1
-------------------------------------
select /* SPM */ count(*),sum(val) from mysales where sale_type = :val

Plan hash value: 3292460164

---------------------------------------------------------------------------------------
  Id  | Operation                   | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |         |       |       |   115 (100)|          |
|   1 |  SORT AGGREGATE             |         |     1 |    17 |            |          |
|*  2 |   TABLE ACCESS INMEMORY FULL| MYSALES |  2000K|    32M|   115  (20)| 00:00:01 |
---------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
  2 - inmemory("SALE_TYPE"=:VAL)
      filter("SALE_TYPE"=:VAL)

Note
-----
  - SQL plan baseline SQL_PLAN_93ct9zmnvtbuhc69cec1f used for this statement

The full table scan is now annotated with INMEMORY, so we know that some or all of the data for MYSALES is scanned via the In-Memory column store. The “note” section reports that the same baseline is being used as before (ending in “c1f”). This is good news, and it’s the scenario that was covered in Part 1 of this series.  

What if we executed the query to match 20,001 rows? You can probably guess what’s coming; the Optimizer judges that the In-Memory scan is more efficient than the index range scan: 

PLAN_TABLE_OUTPUT
-----------------
SQL_ID  d3u63rk540w0r, child number 2
-------------------------------------
select /* SPM */ count(*),sum(val) from mysales where sale_type = :val

Plan hash value: 3292460164

---------------------------------------------------------------------------------------
  Id  | Operation                   | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |         |       |       |   115 (100)|          |
|   1 |  SORT AGGREGATE             |         |     1 |    17 |            |          |
|*  2 |   TABLE ACCESS INMEMORY FULL| MYSALES | 20001 |   332K|   115  (20)| 00:00:01 |
---------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
  2 - inmemory("SALE_TYPE"=:VAL)
      filter("SALE_TYPE"=:VAL)

Note
-----
  - SQL plan baseline SQL_PLAN_93ct9zmnvtbuhc69cec1f used for this statement

Since there is a SQL plan baseline that allows a full table scan to be used, Q2 can use this access method straight away and we get immediate benefit from scanning the In-Memory column store!

Hold on a minute! Wasn’t that just a little bit too convenient? I arranged it so that there was a handy full-table-scan SQL plan baseline ready and waiting for when I "flipped the switch" and started using the In-Memory column store. This example might seem a little contrived, but it is a real-world example and I chose it to illustrate how SPM works together with both Oracle In-Memory Database and adaptive cursor sharing (and if you want more, there's an earlier blog on how adaptive cursor sharing interacts with SPM).

If, instead, I had started out with a single baseline that specified an index range scan, then this is the plan that would have been used even after MYSALES populated the In-Memory column store (and we would not have had an INMEMORY FULL scan). That’s not a bad thing; it is exactly what plan stability means and it is how SPM is meant to work. In the example above I made use of a couple of SQL execution plans that were validated and accepted before I initiated the In-Memory column store. In the more general case, where the Optimizer identifies a brand new execution plan for use with the In-Memory column store, we might want to validate it before we allow the database to use it in our critical application. How can we do that? Happily, it's what SPM evolution was built for, and it goes all the way back to the initial scenario I mentioned in Part 1. I'll cover the details in Part 3 (coming soon). 

If you want to try out this example for yourself, the scripts are in GitHub.

Wednesday Aug 12, 2015

Tips on SQL Plan Management and Oracle Database In-Memory Part 1

If you follow Oracle’s In-Memory blog then you probably came across a post mentioning how you should use SQL Plan Management when you’re upgrading to Oracle Database In-Memory. Whether you have read that post or not, you might be wondering what will happen if you have some SQL plan baselines and you begin to populate the In-Memory column store with a bunch of tables as used by those baselines. That’s what this post is about. Well, in fact, I’m going to break the topic up into a few posts because (as ever!) there is a little bit of subtlety to cover. Luckily, this will make your life easier rather than more difficult because you can get immediate benefit from In-Memory even if you don’t evolve SQL plan baselines on day one.  

When I started to think about this post I thought that I would start with the first scenario that probably comes to mind if you’re familiar with SQL Plan Management (SPM): 
  • The Optimizer comes up with a new execution plan for a SQL statement because something has changed, and Oracle Database In-Memory would be a very good example of that! 
  • If there’s a SQL plan baseline for the statement, the database will use the baseline execution plan and capture the new plan.
  • Where appropriate, the new plan will be validated and accepted using SQL plan evolution. 

I will get to that, but first it’s better to start with a couple of more subtle points. With this information in our back pocket it will be easier to understand (and explain) the more traditional aspects of SQL plan evolution in the context of Oracle Database In-Memory. 

Here, I will cover the following example:
  • There is a table called MYSALES that’s not yet populated into the In-Memory column store. 
  • A query (called “Q1”) includes a full table scan of MYSALES. There is no index on the table that’s useful to Q1. 
  • Q1 has an active SQL plan baseline.
  • MYSALES is subsequently populated into the In-Memory column store.

Let’s take a look at Q1 and its SQL execution plan before populating MYSALES into the In-Memory column store (and I'll explain the significance of the highlighted text further down)...

SQL_ID  4ss4zbb813250, child number 0
-------------------------------------
SELECT /* SPM */ COUNT(*) FROM   mysales WHERE  val = 'X'

Plan hash value: 3292460164

------------------------------------------------------------------------------
  Id  | Operation          | Name    | Rows  | Bytes | Cost  %CPU | Time     |
------------------------------------------------------------------------------
|   0 | SELECT STATEMENT   |         |       |       |    69 (100)|          |
|   1 |  SORT AGGREGATE    |         |     1 |     2 |            |          |
|*  2 |   TABLE ACCESS FULL| MYSALES | 99991 |   195K|    69   (2)| 00:00:01 |
------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
  2 - filter("VAL"='X')

Note
-----
  - SQL plan baseline SQL_PLAN_7469nmnn7nsu3c69cec1f used for this statement

Q1 performs a full table scan of MYSALES. The "note" section makes it clear that a SQL plan baseline is used. This is what that looks like:

SELECT PLAN_TABLE_OUTPUT
FROM   V$SQL s, DBA_SQL_PLAN_BASELINES b,
      TABLE(
        DBMS_XPLAN.DISPLAY_SQL_PLAN_BASELINE(b.sql_handle,b.plan_name,'basic')
      ) t
WHERE  s.EXACT_MATCHING_SIGNATURE=b.SIGNATURE
AND    b.PLAN_NAME=s.SQL_PLAN_BASELINE
AND    s.SQL_ID='4ss4zbb813250';

--------------------------------------------------------------------------------
SQL handle: SQL_7219349d287a6343
SQL text: SELECT /* SPM */ COUNT(*) FROM   mysales WHERE  val = 'X'
--------------------------------------------------------------------------------

--------------------------------------------------------------------------------
Plan name: SQL_PLAN_7469nmnn7nsu3c69cec1f         Plan id: 3332172831
Enabled: YES     Fixed: NO      Accepted: YES     Origin: AUTO-CAPTURE
Plan rows: From dictionary
--------------------------------------------------------------------------------

Plan hash value: 3292460164

--------------------------------------
  Id  | Operation          | Name    |
--------------------------------------
|   0 | SELECT STATEMENT   |         |
|   1 |  SORT AGGREGATE    |         |
|   2 |   TABLE ACCESS FULL| MYSALES |
--------------------------------------

What happens if MYSALES is now populated into the In-Memory column store? 

-- Mark MYSALES with the In-Memory attribute
ALTER TABLE mysales INMEMORY;

-- Access MYSALES to trigger population into In-Memory column store
SELECT count(*) FROM mysales;

Let’s rerun our query and examine the execution plan:

SQL_ID  4ss4zbb813250, child number 1
-------------------------------------
SELECT /* SPM */ COUNT(*) FROM   mysales WHERE  val = 'X'

Plan hash value: 3292460164

---------------------------------------------------------------------------------------
  Id  | Operation                   | Name    | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |         |       |       |     3 (100)|          |
|   1 |  SORT AGGREGATE             |         |     1 |     2 |            |          |
|*  2 |   TABLE ACCESS INMEMORY FULL| MYSALES |   100K|   195K|     3   (0)| 00:00:01 |
---------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   2 - inmemory("VAL"='X')
       filter("VAL"='X')

Note
-----
   - SQL plan baseline SQL_PLAN_7469nmnn7nsu3c69cec1f used for this statement

There is still a full table scan, but this time the query will read data from MYSALES via the In-Memory column store rather than the storage-resident table and, even better, the same SQL plan baseline is used. That was pretty easy! The Optimizer chose a full table scan in both cases, so the same SQL plan baseline was used both cases. The INMEMORY annotation for the full table scan is “for your information only”; it tells you that the query scanned some or all of the data for your table via the In-Memory column store but as far as the Optimizer is concerned it is “just” a full table scan, as the keyword INMEMORY does not affect the plan hash value, so it will match the existing the SQL plan baseline (above, you can see that the plan hash value is always "3292460164" ).

Why do I say the INMEMORY keyword indicates some or all of the data for your table is scanned via the In-Memory column store? Remember until all of the data belonging to MYSALES has been populated into the In-Memory column store, Oracle will automatically pick up the rest of the data from wherever it resides. That could be from memory (e.g. the buffer cache) or from flash or from disk.

It should be pretty obvious by now that if we decide to remove MYSALES from the In-Memory column store, the query will revert to scanning the storage-resident table and the plan will display “TABLE ACCESS FULL”. 

This example is very simple, but the principle applies to queries that have the same execution plan for In-Memory versus non-In-Memory. What happens if there are execution plan changes and, in particular, if indexes are involved? Start by looking at Part 2.

If you want to try out this example for yourself, the scripts are in GitHub.


Wednesday Jun 24, 2015

What you need to know about SQL Plan Management and Auto Capture

SQL Plan Management (SPM) is an Oracle database feature that allows you to establish a set of SQL execution plans that will be used even if the database is subject to changes that would otherwise cause execution plan changes to occur. For example, you might have an end-of-day batch run that operates in a business context where there are extreme peaks and troughs in daily volume, or perhaps you are upgrading a database and want to be sure that plans are carried over (at least initially). You do not have to fix execution plans in stone with SPM, you can use plan evolution to automate the process of finding improved plans, improving performance in a controlled way and at your own pace. If you’re not familiar with SPM, a very good place to start is to take a look at Maria Colgan’s four-part blog post on the subject. It gives you all the tools you need to get started.

If you are using SPM in Oracle Database 11gR2 or 12c, or if you are considering whether you should use it, then this blog post is for you. I decided to publish this post because I recently encountered a couple of environments that ran into, let’s say, “difficulties” with SPM when capturing SQL plan baselines automatically and continuously over a very long period of time (more than a year in fact). I’d like to give you a few pointers to avoid running into the same problems and why automatic SQL baseline capture was never intended to be used in that way.

[Read More]

Tuesday May 26, 2015

Space Management and Oracle Direct Path Load

Most of you will be familiar with the concept of direct path load and how it’s an efficient way to load large volumes of data into an Oracle database as well as being a great technique to use when moving and transforming data inside the database. It’s easy to use in conjunction with parallel execution too, so you can scale out and use multiple CPUs and database servers if you need to process more data in less time.

Probably less well known is how the Oracle database manages space during direct path load operations. This is understandable because the Oracle database uses a variety of approaches and it has not always been very obvious which one it has chosen. The good news is that Oracle Database 12c from version 12.1.0.2 onwards makes this information visible in the SQL execution plan and it is also possible to understand what’s happening inside earlier releases with a bit of help from this post and some scripts I’ve linked to at the end.

[Read More]

Friday May 08, 2015

Controlling Access To The In-Memory Column Store Via Hints

It’s been almost a year since I’ve had an opportunity to write a new blog post here due to my crazy schedule now that I'm the In-Memory Maven (thanks to Doug Burns & Graham Wood for the new and improved title). But this morning while I was writing a post for the In-Memory blog about controlling the use of the In-Memory column store (IM column store), I realized that the content on Optimizer hints really belonged over here.

So, I decided to split the blog post in two. I’ve put the information on the initialization parameter that control the use of the IM column store on the In-Memory blog and the information about the Optimizer hint that control the use of the IM column store here.

[Read More]

Monday Jul 07, 2014

The Optimizer & the new Oracle Database In-Memory option

I know its been forever since I've posted a new blog but I am no longer the Optimizer lady. In case you haven't heard already,  I have a new role now as the In-Memory lady (definitely need to work on a better title).

But for this weeks In-Memory blog post I got a chance to combine both my old and my new roles when I wrote about how the In-Memory option and the Optimizer interact. And I thought the readers of this blog might be interested in this topic too.

So, if you are interested in seeing how these these two fascinating pieces of the Oracle technology work together, check out the latest post on the new In-Memory blog.

You should also mark your calendars because of the head of the Optimizer development team, Mohamed Zait, will also discuss this topic at Oracle Open World later this year!

 Hope to catch up with a lot of you then.

Friday Sep 13, 2013

What's new in 12c: Adaptive joins part 2

In our earlier post on adaptive joins we explained how this new 12c functionality works and said we would follow up this post with a real-world demo.

[Read More]

Sunday Aug 25, 2013

What's new in 12c: Adaptive joins

As we promised in our previous post, we are starting a blog series describing all of new Optimizer and statistics related functionality on Oracle Database 12c. We begin the series with an in-depth look at adaptive plans, one of the key features in the new adaptive query optimization framework.[Read More]

Tuesday Jun 25, 2013

Oracle Database 12c is here!

Oracle Database 12c was officially release today and is now available for download. Along with the software release comes a whole new set of collateral that explains in detail all of the new features and functionality you will find in this release.

The Optimizer page on Oracle.com has all the juicy details about what you can expect from the Optimizer in Oracle Database 12c. Direct links are below.

[Read More]
About

The Oracle Optimizer blog is written by members of the Optimizer development team. The goal of this blog is to provide an insight into the workings of the Optimizer and the statistics it relies on. The views expressed on this blog are our own and do not necessarily reflect the views of Oracle and its affiliates. The views and opinions expressed by visitors on this blog are theirs solely and may not reflect ours.

Search

Archives
« February 2016
SunMonTueWedThuFriSat
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
     
       
Today