Index statistics

There are so many situations in the Oracle database world that I would have probably never encountered if there hadn’t been so many different real life applications with different problems that needed solving. A couple of days ago, a critical query of one of such applications, experienced severe performance deterioration. When I started looking at this query I’ve first decided to print out the statistics of its historical executions shown below:

SQL> @HistStats
Enter value for sql_id: gqj0qgtduh1k7
Enter value for from_date: 01122017

SNAP_BEGIN                PLAN_HASH_VALUE      EXECS  AVG_ETIME    AVG_PIO    AVG_LIO   AVG_ROWS
------------------------- --------------- ---------- ---------- ---------- ---------- ----------
01/12/17 12:00:44.897 AM        179157465        178          4        357     244223        543
01/12/17 01:00:50.192 AM        179157465        40           6       3976     246934        400
01/12/17 04:00:02.121 AM        179157465         2           2        171     246203        400
../..
03/12/17 05:00:22.064 AM        179157465        10           18      4231     245802        400
../..
04/12/17 09:00:19.716 AM        179157465        13           1        4       253433        482

The most important characteristic of this query is that it consumes 250K logical I/O per execution and that it can sometime last up to 18 seconds. This is a query generated by Hibernate. Unfortunately, due to the Hibernate obsession of changing table aliases, this query has got a new sql_id when it has been deployed in production via the December release. This is why we lost the track of the query pre-release statistics that would have allowed us to get the previous acceptable execution plan.

Starting from the end
Let’s, exceptionally, start this article by showing the statistics of this query after I have fixed it

SQL> @HistStats
Enter value for sql_id: gqj0qgtduh1k7
Enter value for from_date: 01122017

SNAP_BEGIN                PLAN_HASH_VALUE      EXECS  AVG_ETIME    AVG_PIO    AVG_LIO   AVG_ROWS
------------------------- --------------- ---------- ---------- ---------- ---------- ----------
01/12/17 12:00:44.897 AM        179157465        178          4        357     244223        543
01/12/17 01:00:50.192 AM        179157465        40           6       3976     246934        400
01/12/17 04:00:02.121 AM        179157465         2           2        171     246203        400
../..
03/12/17 05:00:22.064 AM        179157465        10           18      4231     245802        400
../..
04/12/17 09:00:19.716 AM        179157465        13           1        4       253433        482
../..
11/12/17 02:00:04.453 PM        1584349102        4           0        14       17825        482 –- fixed
12/12/17 12:00:38.622 AM        1584349102       67           1        661      13840        522 
12/12/17 01:00:42.740 AM        1584349102       40           0        103       7605        852 
13/12/17 05:00:27.270 AM        1584349102       10           1        845      16832        400 

Notice how, thanks to the fix I will explain in the next section, I have got a remarkable drop of the number of Logical I/O consumption from 244K to 17K per execution on average.

How this has been achieved?
The real time sql monitoring report of the degraded query confirms the bad response time of 12 seconds:

Global Information
------------------------------
 Status              :  DONE (ALL ROWS)                 
 Instance ID         :  2                    
 Session             :  xxxx (324:25737) 
 SQL ID              :  gqj0qgtduh1k7        
 SQL Execution ID    :  16777216             
 Execution Started   :  12/05/2017 06:01:20  
 First Refresh Time  :  12/05/2017 06:01:29  
 Last Refresh Time   :  12/05/2017 06:01:32  
 Duration            :  12s                  
 Module/Action       :  JDBC Thin Client/-           
 Service             :  xxxxx            
 Program             :  JDBC Thin Client/
 Fetch Calls         :  10

This report presents an oddity in the following part of its 54 execution plan operations:

---------------------------------------------------------------------------------------
| Id  | Operation                           | Name         | Starts | E-Rows | A-Rows |
---------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                    |              |      1 |        |    398 |
|* 39 |  TABLE ACCESS BY INDEX ROWID BATCHED| T1           |      1 |     26 |    398 |
|* 40 |   INDEX RANGE SCAN                  | IDX_PRD_TYPE |      1 |     2M |     3M |
---------------------------------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
  39 - filter(("PRODUCT_ID"=:1 OR "PRODUCT_ID"=:2 OR "PRODUCT_ID"=:3 OR 
        "PRODUCT_ID"=:4 ../..  OR "PRODUCT_ID"=:398))
  40- access("PRODUCT_TYPE"='M'

This oddity resides in the fact that we went from 3M of rowids supplied by the index at operation 40 to only 398 rows at the table level. Such a situation can have one of the following explanations:

• The CBO has made a very bad choice preferring the IDX_PRD_TYPE index over another existing index with a much better index effective selectivity.
• IDX_PRD_TYPE is the sole index with which Oracle can cover the PRODUCT_ID and PRODUCT_TYPE predicates. And, in this case, it represents an imprecise index that should be re-designed if we want to efficiently cover the above query

The answer is straightforward If you remember that a column having an _ID suffix or prefix in its name refers generally to a primary key. The T1 table has indeed a primary key index defined as follows:

 T1_PK (PRODUCT_ID) 

In addition the 398 rows returned at the table level are nothing else than the 398 elements constituting the in-list of the corresponding query reproduced here below:

SELECT 
  *
FROM
   t1
WHERE 
   PRODUCT_ID 
       in (:1,:2,:3,:4,:5,....,:398)
AND PRODUCT_TYPE = 'M';

Table altered.

While the bad selected index has the following definition:

 IDX_PRD_TYPE(PRODUCT_TYPE,COLX)

And the initial question turned to : why the primary key index has not been used in this critical part of the executio plan?

At this stage of the investigations it becomes crystal clear for me that using the primary key index is, to a very large extent, better that using the index on the PRODUCT_TYPE column. This is simply because, if used by the CBO, the primary key index will supply its parent table operation by at most one rowid per element in the in-list predicate. That is for an in-list of 398 elements Oracle will only filter 398 rows at the table level using the PRODUCT_TYPE = ‘M’ filter. That’s largely better than filtering 3M of rows at the table level using the 3M of index rowids supplied by the IDX_PRD_TYPE(PRODUCT_TYPE,COLX) index.

Bear in mind as well that all parameters participating in the index desirability are at their default values. And that the PRODUCT_TYPE column has a remarkable frequency histogram showing perfectly the not evenly distribution of its 13 distinct values.

select 
   column_name
  ,histogram 
from 
  user_tab_col_statistics 
where 
  table_name = 'T1'
and
  column_name = 'PRODUCT_TYPE';

COLUMN_NAME  HISTOGRAM
------------ ---------------
PRODUCT_TYPE FREQUENCY

The explain plan for command applied when the primary key index is forced and when not used shows that the combined table/index access cost is cheaper when the IDX_PRD_TYPE index is used as the following proves:

-------------------------------------------------------------------
| Id  | Operation                    | Name  | Rows  | Cost (%CPU)|
-------------------------------------------------------------------
|   0 | SELECT STATEMENT             |       |    27 |   519   (1)|
|   1 |  INLIST ITERATOR             |       |       |            |
|*  2 |   TABLE ACCESS BY INDEX ROWID| T1    |    27 |   519   (1)|
|*  3 |    INDEX UNIQUE SCAN         | T1_PK |   398 |   399   (0)|
-------------------------------------------------------------------

--------------------------------------------------------------------------------
| Id  | Operation                           | Name         | Rows  |Cost (%CPU)|
--------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                    |              |    27 |  58   (0)|
|*  1 |  TABLE ACCESS BY INDEX ROWID BATCHED| T1           |    27 |  58   (0)|
|*  2 |   INDEX RANGE SCAN                  | IDX_PRD_TYPE |  2183K|   3   (0)|
--------------------------------------------------------------------------------

And the initial question turned to : why the cost of the inapropriate index is cheaper?

After few minutes of reflection I decided to get the statistics of these two indexes:

select
   index_name
  ,distinct_keys
  ,leaf_blocks
from
   user_indexes
where
  table_name = 'T1';
 
INDEX_NAME       DISTINCT_KEYS LEAF_BLOCKS
---------------- ------------- -----------
T1_PK                 32633730    111089
IDX_PRD_TYPE

And finally the initial question turned to be: why the IDX_PRD_TYPE index has no statistics?

The answer is:

• The statistics of T1 table have been voluntarily locked since several months ago
• The IDX_PRD_TYPE index has been very recently created and implemented in Production

The combination of the above two points resulted in a newly created index without any statistics making this index appearing very cheap in the CBO eyes and consequently leading to a very bad choice and to a noticeable performance issue.

Obviously gathering the statistics of this index makes it less desirable in favour of the primary key index as the following proves:

exec dbms_stats.unlock_table_stats(user, 't1');
begin
 dbms_stats.gather_index_stats(user, 'IDX_PRD_TYPE', no_invalidate =>false);
end;
/
exec dbms_stats.unlock_table_stats(user, 't1');

-------------------------------------------------------------------
| Id  | Operation                    | Name  | Rows  | Cost (%CPU)|
-------------------------------------------------------------------
|   0 | SELECT STATEMENT             |       |    27 |   519   (1)|
|   1 |  INLIST ITERATOR             |       |       |            |
|*  2 |   TABLE ACCESS BY INDEX ROWID| T1    |    27 |   519   (1)|
|*  3 |    INDEX UNIQUE SCAN         | T1_PK |   398 |   399   (0)|
-------------------------------------------------------------------

--------------------------------------------------------------------------------
| Id  | Operation                           | Name         | Rows  |Cost (%CPU)|
--------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                    |              |    27 |  137K   (2|
|*  1 |  TABLE ACCESS BY INDEX ROWID BATCHED| T1           |    27 |  137K   (2)|
|*  2 |   INDEX RANGE SCAN                  | IDX_PRD_TYPE |  2183K| 5678    (1)|
--------------------------------------------------------------------------------

Bottom Line

All things being equal when you decide to lock the statistics of one your tables then bear in mind that any newly created index on this table will not have any statistics. This index will the be, naturally, much more desirable than any other index created before the lock of the table statistics resulting, fairlly likely, into wrong index choice and bad execution plans. If you are in a similar situation you had then better to manually gather statistics of your new indexes as long as the statistics of your table are kept locked.

Null-Accepting Semi-Join

Introduction

Null-Accepting semi-join is a new enhancement brought to the CBO by the 12cR1 release. It extends the semi-join algorithm to qualify rows from the table in the left hand side that have a null value in the join column.It kicks in for queries like the following one:

SELECT  
   count(1)
FROM t1
   WHERE(t1.n1 is null
         OR exists (SELECT null FROM t2 where t2.n1 = t1.n1)
		);

It is recognisable in execution plans via its acronym NA (Null-Accepting)

----------------------------------------------------
| Id  | Operation           | Name | Rows  | Bytes |
----------------------------------------------------
|   0 | SELECT STATEMENT    |      |       |       |
|   1 |  SORT AGGREGATE     |      |     1 |     6 |
|*  2 |   HASH JOIN SEMI NA |      |     7 |    42 |
|   3 |    TABLE ACCESS FULL| T1   |    10 |    30 |
|   4 |    TABLE ACCESS FULL| T2   |    10 |    30 |
----------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   2 - access("T2"."N1"="T1"."N1")

It appears in the CBO 10053 trace file as (for its HASH JOIN version):

 Best:: JoinMethod: HashNullAcceptingSemi

Don’t get confused by the NA acronym that appears in the ANTI-JOIN operation where it refers to Null-Aware rather than to Null-Accepting as shown in the following execution plan and 10053 trace file respectively:

SELECT  
   count(1)
FROM t1
   WHERE t1.n1 NOT IN (select n1 from t2);

----------------------------------------------------
| Id  | Operation           | Name | Rows  | Bytes |
----------------------------------------------------
|   0 | SELECT STATEMENT    |      |       |       |
|   1 |  SORT AGGREGATE     |      |     1 |     6 |
|*  2 |   HASH JOIN ANTI NA |      |     1 |     6 |
|   3 |    TABLE ACCESS FULL| T1   |    10 |    30 |
|   4 |    TABLE ACCESS FULL| T2   |    10 |    30 |
----------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   2 – access("T1"."N1"="N1")

 Best:: JoinMethod: HashNullAwareAnti

If you want to clear this confusion then remember that the Null-Accepting transformation occurs for rows that (SEMI) join while Null-Aware transformation is for rows that would not join (ANTI).

The semi-join Null-Accepting logical operation can also be serviced by the NESTED LOOP physical operation as the following demonstrates:

SELECT  
   count(1)
FROM t1
   WHERE(t1.n1 is null
         OR exists (SELECT /*+ NL_SJ */ null FROM t2 where t2.n1 = t1.n1)
		); 

------------------------------------------------------
| Id  | Operation             | Name | Rows  | Bytes |
------------------------------------------------------
|   0 | SELECT STATEMENT      |      |       |       |
|   1 |  SORT AGGREGATE       |      |     1 |     6 |
|   2 |   NESTED LOOPS SEMI NA|      |     7 |    42 |
|   3 |    TABLE ACCESS FULL  | T1   |    10 |    30 |
|*  4 |    TABLE ACCESS FULL  | T2   |     7 |    21 |
------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   4 – filter("T2"."N1"="T1"."N1")

Best:: JoinMethod: NestedLoopNullAcceptingSemi

It is the ability, acquired by the CBO as from 12cR1, to unnest the above kind of disjunctive subquery that makes the Null-Accepting transformation possible as shown in the corresponding10053 trace file:

*****************************
Cost-Based Subquery Unnesting
*****************************
SU: Unnesting query blocks in query block SEL$1 (#1) that are valid to unnest.
Subquery Unnesting on query block SEL$1 (#1)
SU: Performing unnesting that does not require costing.
SU: Considering subquery unnest on query block SEL$1 (#1).
SU: Checking validity of unnesting subquery SEL$2 (#2)
SU: Transforming EXISTS subquery to a join.
Registered qb: SEL$5DA710D3 0x1d225e60 (SUBQUERY UNNEST SEL$1; SEL$2)

Prior to 12cR1 it was not possible to automatically unnest the above subquery to join it with its parent block leading to the below execution plan where the inner table T2 is scanned mutliple times:

SELECT /*+ gather_plan_statistics */ 
   count(1)
FROM t1
   WHERE(t1.n1 is null
         OR exists (SELECT /*+ no_unnest */ null FROM t2 where t2.n1 = t1.n1)
        );  
---------------------------------------------------------------
| Id  | Operation           | Name | Starts | E-Rows | A-Rows |
---------------------------------------------------------------
|   0 | SELECT STATEMENT    |      |      1 |        |      1 |
|   1 |  SORT AGGREGATE     |      |      1 |      1 |      1 |
|*  2 |   FILTER            |      |      1 |        |     10 |
|   3 |    TABLE ACCESS FULL| T1   |      1 |     10 |     10 |
|*  4 |    TABLE ACCESS FULL| T2   |      7 |      1 |      7 |
---------------------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   2 - filter(("T1"."N1" IS NULL OR  IS NOT NULL))
   4 - filter("T2"."N1"=:B1)

The Bug

The Null-Accepting semi-join transformation comes, unfortunately, with a bug already identified in MyOracle Support via number 21201446. Here’s below how it can be reproduced:

SQL> select banner from v$version where rownum = 1;

BANNER
--------------------------------------------------------------------------------
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production

create table t1 as select rownum n1, trunc((rownum -1/5)) n2 from dual connect by level <= 10; 

create table t2 as select rownum n1, trunc((rownum -1/3)) n2 from dual connect by level <= 10;

update t1 set n1 = null where n1 in (5,6,7);

exec dbms_stats.gather_table_stats(user, 't1');

exec dbms_stats.gather_table_stats(user, 't2');

SQL> SELECT  
         count(1)
    FROM t1
      WHERE(t1.n1 is null
         OR exists 
           (SELECT null FROM t2 where nvl(t2.n1,42) = nvl(t1.n1,42))
	    );

  COUNT(1)
----------
         7

----------------------------------------------------
| Id  | Operation           | Name | Rows  | Bytes |
----------------------------------------------------
|   0 | SELECT STATEMENT    |      |       |       |
|   1 |  SORT AGGREGATE     |      |     1 |     6 |
|*  2 |   HASH JOIN SEMI NA |      |     7 |    42 |
|   3 |    TABLE ACCESS FULL| T1   |    10 |    30 |
|   4 |    TABLE ACCESS FULL| T2   |    10 |    30 |
----------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
 2 - filter(NVL("T2"."N1",42)=NVL("T1"."N1",42))

Using the HASH JOIN physical operation the query returns 7 rows. But it returns 10 rows when it uses the NESTED LOOP operation a shown below:

SELECT  
   count(1)
FROM t1
   WHERE(t1.n1 is null
          OR exists 
           (SELECT /*+ NL_SJ */ null FROM t2 where nvl(t2.n1,42) = nvl(t1.n1,42))
	   );

  COUNT(1)
----------
        10
		
------------------------------------------------------
| Id  | Operation             | Name | Rows  | Bytes |
------------------------------------------------------
|   0 | SELECT STATEMENT      |      |       |       |
|   1 |  SORT AGGREGATE       |      |     1 |     6 |
|   2 |   NESTED LOOPS SEMI NA|      |     7 |    42 |
|   3 |    TABLE ACCESS FULL  | T1   |    10 |    30 |
|*  4 |    TABLE ACCESS FULL  | T2   |     7 |    21 |
------------------------------------------------------
 
Predicate Information (identified by operation id):
---------------------------------------------------
   4 – filter(NVL("T2"."N1",42)=NVL("T1"."N1",42))	

The Null-Accepting semi-join transformation is driven by the following hidden parameter which , if cancelled, will workarround this bug as shown below:

SQL> alter session set "_optimizer_null_accepting_semijoin"=false;

SELECT  
   count(1)
FROM t1
   WHERE(t1.n1 is null
          OR exists 
           (SELECT null FROM t2 where nvl(t2.n1,42) = nvl(t1.n1,42))
	   );

  COUNT(1)
----------
        10

----------------------------------------------------
| Id  | Operation           | Name | Rows  | Bytes |
----------------------------------------------------
|   0 | SELECT STATEMENT    |      |       |       |
|   1 |  SORT AGGREGATE     |      |     1 |     3 |
|*  2 |   FILTER            |      |       |       |
|   3 |    TABLE ACCESS FULL| T1   |    10 |    30 |
|*  4 |    TABLE ACCESS FULL| T2   |     1 |     3 |
----------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   2 - filter(("T1"."N1" IS NULL OR  IS NOT NULL))
   4 - filter(NVL("T2"."N1",42)=NVL(:B1,42))

How this has been fixed in 12cR2?

Because of the bug n° 21201446 It seems that Oracle has completely cancelled the Null-Accepting semi-join transformation in 12cR2 for both NESTED LOOP and HASH JOIN physical operations when the NVL function is applied on the join column. Here’s below why I am thinking so:

SQL> select banner from v$version where rownum=1;

BANNER
-----------------------------------------------------------------------------
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SELECT  
   count(1)
FROM t1
   WHERE(t1.n1 is null
          OR exists 
           (SELECT null FROM t2 where nvl(t2.n1,42) = nvl(t1.n1,42))
	   );

  COUNT(1)
----------
        10

----------------------------------------------------
| Id  | Operation           | Name | Rows  | Bytes |
----------------------------------------------------
|   0 | SELECT STATEMENT    |      |       |       |
|   1 |  SORT AGGREGATE     |      |     1 |     3 |
|*  2 |   FILTER            |      |       |       |
|   3 |    TABLE ACCESS FULL| T1   |    10 |    30 |
|*  4 |    TABLE ACCESS FULL| T2   |     1 |     3 |
----------------------------------------------------

Predicate Information (identified by operation id):
---------------------------------------------------
   2 - filter(("T1"."N1" IS NULL OR  IS NOT NULL))
   4 – filter(NVL("T2"."N1",42)=NVL(:B1,42))

In 12cR1, as we’ve shown above, the NESTED LOOP was not concerned by the bug. As from 12cR2 the NESTED LOOP SEMI JOIN is not anymore allowed to occur if a NVL function is applied on the join column as the following proves:

SELECT  
   count(1)
FROM t1
   WHERE(t1.n1 is null
          OR exists 
           (SELECT /*+ NL_SJ */ null FROM t2 where nvl(t2.n1,42) = nvl(t1.n1,42))
	   );

  COUNT(1)
----------
        10

----------------------------------------------------
| Id  | Operation           | Name | Rows  | Bytes |
----------------------------------------------------
|   0 | SELECT STATEMENT    |      |       |       |
|   1 |  SORT AGGREGATE     |      |     1 |     3 |
|*  2 |   FILTER            |      |       |       |
|   3 |    TABLE ACCESS FULL| T1   |    10 |    30 |
|*  4 |    TABLE ACCESS FULL| T2   |     1 |     3 |
----------------------------------------------------

Query Block Name / Object Alias (identified by operation id):
-------------------------------------------------------------
   1 - SEL$1
   3 - SEL$1 / T1@SEL$1
   4 - SEL$2 / T2@SEL$2

Outline Data
-------------
  /*+
      BEGIN_OUTLINE_DATA
      IGNORE_OPTIM_EMBEDDED_HINTS
      OPTIMIZER_FEATURES_ENABLE('12.1.0.1')
      DB_VERSION('12.2.0.1')
      OPT_PARAM('_optimizer_null_accepting_semijoin' 'false')
      ALL_ROWS
      OUTLINE_LEAF(@"SEL$2")
      OUTLINE_LEAF(@"SEL$1")
      FULL(@"SEL$1" "T1"@"SEL$1")
      PQ_FILTER(@"SEL$1" SERIAL)
      FULL(@"SEL$2" "T2"@"SEL$2")
      END_OUTLINE_DATA
  */

Predicate Information (identified by operation id):
---------------------------------------------------
   2 - filter(("T1"."N1" IS NULL OR  IS NOT NULL))
   4 - filter(NVL("T2"."N1",42)=NVL(:B1,42))

Compress enable novalidate

We knew here that we can’t DDL optimise a BASIC compressed table. So I uncompressed my table in a 12.2.0.1.0 and DDL optimise it as in the following:

SQL> select 
        pct_free
	,compression
	,compress_for
    from 
	 user_tables
    where 
	   table_name = 'T1';

  PCT_FREE COMPRESS COMPRESS_FOR
---------- -------- -------------
         0 DISABLED

SQL> alter table t1 add c_ddl number default 42 not null;

Table altered.

But I changed my mind and finally decided to revert back and drop this newly added column:

SQL> alter table t1 drop column c_ddl;
alter table t1 drop column c_ddl
                           *
ERROR at line 1:
ORA-39726: unsupported add/drop column operation on compressed tables

Damned database!!!

What’s going on here? My table is not compressed. Is it?

In fact I didn’t show you how I have uncompressed the t1 table:

SQL> alter table t1 nocompress;

Table altered.

Compressing or un-compressing a table without moving it will do nothing to its data. When the COMPRESSION column of a table is DISABLED this doesn’t necessarily mean that this table doesn’t contain compressed data . And this is probably the reason why the above bizarre error pops up out from nowhere.
In fact in order to have a significant meaning, the COMPRESSION column should normally always be considered with the PCT_FREE column (notice in passing that this is why I printed above the pct_free value):

SQL> select
         table_name
        ,compression
        ,pct_free
        ,compress_for
     from 
     user_tables
    where table_name = 'T1';

TABLE_NAME COMPRESS   PCT_FREE COMPRESS_FOR
---------- -------- ---------- --------------
T1         DISABLED          0

When PCT_FREE equals zero this is generally a hint that Oracle is considering BASIC compression since it thinks that the table is not considered for updates anymore. The definition given by Oracle to the COMPRESSION column goes in the same direction as well:

COMPRESSION	 	Indicates whether table compression 
                    is enabled (ENABLED) or not (DISABLED)

This is crystal clear: COMPRESSION referes to the status of the current compression. It doesn’t tell anything about the compression history of the table if any.

Compressing/uncompressing a table without moving it is nothing else than implementing or desimplementing compression but only for newly inserted rows. This is why I think that Oracle could have avoided the confusion by using the same command as when dealing with table constraints. Instead of this:

SQL> alter table t1 nocompress;

We would have this :

SQL> alter table t1 nocompress enable novalidate;

And of course the COMPRESSION column should have had a status indicating whether it is validated or not.

In the 10046 file of a compressed table with a move of its data and without a move we can see this respectively:

alter table t1 move compress basic

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.01       0.02          0          0          1           0
Execute      1      1.43       1.44          3       1865       2548     1000000
Fetch        0      0.00       0.00          0          0          0           0
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        2      1.45       1.46          3       1865       2549     1000000

Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 106  
Number of plan statistics captured: 1

Rows (1st) Rows (avg) Rows (max)  Row Source Operation
---------- ---------- ----------  ---------------------------------------------------
         0          0          0  LOAD AS SELECT  T1 (cr=1905 pr=1 pw=1382 time=1436345 us
   1000000    1000000    1000000   TABLE ACCESS FULL T1 (cr=1797 pr=0 pw=0 time=104378 

Elapsed times include waiting on following events:
  Event waited on                             Times   Max. Wait  Total Waited
  ----------------------------------------   Waited  ----------  ------------
  PGA memory operation                         1140        0.00          0.01
  Disk file operations I/O                        1        0.00          0.00
  db file sequential read                         3        0.00          0.00
  direct path write                               1        0.00          0.00
  reliable message                                4        0.00          0.00
  enq: RO - fast object reuse                     2        0.00          0.00
  enq: CR - block range reuse ckpt                2        0.00          0.00
  SQL*Net message to client                       1        0.00          0.00
  SQL*Net message from client                     1        5.93          5.93
********************************************************************************

alter table t1 nocompress


call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          2           0
Fetch        0      0.00       0.00          0          0          0           0
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total        2      0.00       0.01          0          0          2           0

Misses in library cache during parse: 1
Optimizer mode: ALL_ROWS
Parsing user id: 106  

Elapsed times include waiting on following events:
  Event waited on                             Times   Max. Wait  Total Waited
  ----------------------------------------   Waited  ----------  ------------
  Compression analysis                            8        0.00          0.00
  SQL*Net message to client                       1        0.00          0.00
  SQL*Net message from client                     1        6.65          6.65
********************************************************************************

Spot that whileOracle managed to rework a million rows of in the first case it didn’t touch any in the second case proving that nocompress/compress without a move doesn’t concern existing data.

Since it concerns only new data then an inserted row should be compressed/non-compressed. Let’s see:

SQL> create table t1 as select
      rownum n1
     ,mod(rownum,10) n2
     from dual
     connect by level <=1e6;

Table created.

SQL> @sizeBysegNameMB
Enter value for segment_name: t1
Enter value for owner: c##mhouri


SEGMENT_TYPE  TABLESPACE_NAME  SEGMENT_NAME PARTITION_NAME          MB
------------- ---------------- ------------ --------------- ----------
TABLE         USERS            T1                                   15
                                                            ----------
Total Segment Size                                                  15

Let’s compress this table without moving its data and get its new size:

SQL> alter table t1 compress;

Table altered.

SQL> @sizeBysegNameMB
Enter value for segment_name: t1
Enter value for owner: c##mhouri


SEGMENT_TYPE  TABLESPACE_NAME  SEGMENT_NAME PARTITION_NAME          MB
------------- ---------------- ------------ --------------- ----------
TABLE         USERS            T1                                   15
                                                            ----------
Total Segment Size                                                  15

As you can see the size of the table remains intact indicating again that compressing without moving does nothing to the existing data.

Let’s now direct path load a single row, dump the corresponding block and analyse the dump to see if compression has been activated or not:

SQL> select rowid, n1,n2 from t1 where n1=1;

ROWID                      N1         N2
------------------ ---------- ----------
AAATYNAAHAAAdpTAAA          1          1

SQL> insert /*+ append */ into t1 select 1,42 from dual;

1 row created.

SQL> select rowid, n1,n2 from t1 where n1=1;
select rowid, n1,n2 from t1 where n1=1
                         *
ERROR at line 1:
ORA-12838: cannot read/modify an object after modifying it in parallel


SQL> commit;

Commit complete.

SQL> select rowid, n1,n2 from t1 where n1=1;

ROWID                      N1         N2
------------------ ---------- ----------
AAATYNAAHAAAdpTAAA          1          1
AAATYNAAHAADOgcAAA          1         42

SQL> select
  2    dbms_rowid.rowid_relative_fno('AAATYNAAHAADOgcAAA') file_no
  3  , dbms_rowid.rowid_block_number('AAATYNAAHAADOgcAAA') block_no
  4  from dual;

   FILE_NO   BLOCK_NO
---------- ----------
         7     845852


SQL> alter session set tracefile_identifier='DumpingDirectLoadRowInsert';

Session altered.

SQL> alter system dump datafile 7 block 845852;

System altered.

Unfortunately I couldn’t spot anything interesting in the dump file which indicates that the inserted row has been compressed. I will probably come back to this article in a near future.

Optimizer statistics gathering

As from Oralce 12cR1, the Optimizer can automatically collect statistics on empty tables provided they are, notably, direct path loaded. This is recognisable in execution plans through the new row source operation named OPTIMIZER STATISTICS GATHERING

-----------------------------------------------------------
| Id  | Operation                        | Name   | Rows  |
-----------------------------------------------------------
|   0 | INSERT STATEMENT                 |        |  1000K|
|   1 |  LOAD AS SELECT                  | T1_TAR |       |
|   2 |   OPTIMIZER STATISTICS GATHERING |        |  1000K|
|   3 |    TABLE ACCESS FULL             | T1     |  1000K|
-----------------------------------------------------------

This is, in principle, a good initiative. But look at what I have been asked to trouble shoot a couple of days ago:

Global Information
------------------------------
 Status              :  EXECUTING                 
 Instance ID         :  1                    
 Session             :  xxxx (15:32901) 
 SQL ID              :  am6923m45s203        
 SQL Execution ID    :  16777216             
 Execution Started   :  12/02/2017 08:01:17  
 First Refresh Time  :  12/02/2017 08:01:28  
 Last Refresh Time   :  12/05/2017 11:55:51  
 Duration            :  272276s                  
 Module/Action       :  SQL*Plus/-           
 Service             :  xxxxx            
 Program             :  sqlplus@xxxx

===================================================================================================
| Id |             Operation               |  Name  | Activity | Activity Detail                  |
|    |                                     |        |   (%)    |   (# samples)                    |
===================================================================================================
| -> 0 | INSERT STATEMENT                  |        |   0.00   | Data file init write             |   
| -> 1 |   LOAD AS SELECT                  | T1_TAR |   0.06   | Cpu(28)                          |
|                                          |        |          | direct path write(2)             |
| -> 2 |    OPTIMIZER STATISTICS GATHERING |        |  99.93   | Cpu(52956)                       |
| -> 3 |     TABLE ACCESS FULL             | T1     |          |  SQL*Net more data from dblink(1)|               
==================================================================================================    

You have to believe me when I say that this insert had been running for 3 days when the above SQL report was taken. As you can see 99% of the direct path load execution time was spent gathering statistics at operation in line n°2. This is how a new feature designed to help starts disturbing you.

The insert/select statement selects a CLOB from a remote database. I was curious to see whether getting rid of this CLOB from the insert operation would make things better:

SQL>set timing on
 
    INSERT
   /*+ append */
 INTO T1_TAR
    (col1
   -- ,clob_col2
	,col3
	,col4
	,col5
	,col6
	,col7
	,col8
	)
 SELECT
    t1.col1 
   --,t1.clob_col2
   ,t1.col3
   ,t1.col4
   ,t1.col5
   ,t1.col6
   ,t1.col7
   ,t1.col8
  FROM
    t1@dblink t1;

338481182 rows created.

Elapsed: 00:11:38.85

------------------------------------------------------------
| Id  | Operation          | Name   | Rows  | Bytes |IN-OUT|
-----------------------------------------------------------
|   0 | INSERT STATEMENT   |        |  318M|    28GB|      |   
|   1 |  LOAD AS SELECT    | T1_TAR |      |        |      |
|   2 |   REMOTE           | T1     |  318M|    28GB|R->S  |
------------------------------------------------------------

That’s really interesting.

Why would the presence or the absence of a CLOB column in the insert/select statement allow or prevent the CBO from gathering statistics online?

In fact the CLOB column has nothing to do with the online statistics gathering decision. Should I have commented any other column I would have had the same behaviour as the following prooves:

INSERT
   /*+ append */
 INTO T1_TAR
    (col1
    ,clob_col2
	,col3
	,col4
	,col5
	,col6
	--,col7
	,col8
	)
 SELECT
    t1.col1 
   ,t1.clob_col2
   ,t1.col3
   ,t1.col4
   ,t1.col5
   ,t1.col6
   --,t1.col7
   ,t1.col8
  FROM
    t1@dblink t1;
------------------------------------------------------------
| Id  | Operation          | Name   | Rows  | Bytes |IN-OUT|
-----------------------------------------------------------
|   0 | INSERT STATEMENT   |        |  318M|    28GB|      |   
|   1 |  LOAD AS SELECT    | T1_TAR |      |        |      |
|   2 |   REMOTE           | T1     |  318M|    28GB|R->S  |
------------------------------------------------------------

The OPTIMIZER STATISTICS GATHERING operation requires the presence of the totality of the table columns in order to kick in during direct path load operations. This is probably because Oracle doesn’t want to collect statistics for a bunch of columns and keep others without statistics. This being said, this is another non-really documented restriction that comes with this new 12c feature.

Since I was not going to challenge my client getting rid of one column from the insert/select I finally opted for locally cancelling the online statistics gathering using the corresponding hint as shown below:

INSERT
   /*+ append 
       no_gather_optimizer_statistics
   */
 INTO T1_TAR
    (col1
    ,clob_col2
    ,col3
    ,col4
    ,col5
    ,col6
    ,col7
    ,col8
    )
 SELECT
    t1.col1 
   ,t1.clob_col2
   ,t1.col3
   ,t1.col4
   ,t1.col5
   ,t1.col6
   ,t1.col7
   ,t1.col8
  FROM
    t1@dblink t1;

Global Information
------------------------------
 Status              :  DONE                 
 Instance ID         :  1                    
 Session             :  xxxx (132:63271) 
 SQL ID              :  Od221zf0srs6q        
 SQL Execution ID    :  16777216             
 Execution Started   :  12/02/2017 15:10:16  
 First Refresh Time  :  12/02/2017 15:10:22  
 Last Refresh Time   :  12/05/2017 15:32:56  
 Duration            :  1360s                  
 Module/Action       :  SQL*Plus/-           
 Service             :  xxxxx            
 Program             :  sqlplus@xxxx

SQL Plan Monitoring Details (Plan Hash Value=2098243032)
================================================================
| Id |             Operation   |  Name  |   Time    |   Rows   |
|    |                         |        | Active(s) | (Actual) |
================================================================
|  0 | INSERT STATEMENT        |        |      1355 |        2 |
|  1 |   LOAD AS SELECT        | T1_TAR |      1355 |        2 |
|  2 |     REMOTE              | T1     |      1355 |     338M |
================================================================    

That’s how we fixed this issue.

Watch out your direct path load over empty tables selecting from either local or distant databases; the new 12c online table and columns statics gathering might considerably delay your treatement.