Mohamed Houri’s Oracle Notes

January 30, 2016

RAC : Uncached TEMP SPACE

Filed under: Oracle — hourim @ 8:00 am

We parallelized a very big index (171GB) creation using this:

SQL> create UNIQUE index PK_TABLE_XXXX ON TABLE_XXXX
          (COL1
          ,COL2
          ,COL3
          ,COL4
          ,COL5)
     LOCAL
     TABLESPACE TBS_IDX_XXXX
     parallel 8
     NOLOGGING;

SQL> alter index PK_TABLE_XXXX noparallel;

Unfortunately we went in TEMP tablespace shortage after a couple of minutes of run

Error: ORA-12801
------------------------------
ORA-12801: erreur signalée dans le serveur de requête parallèle P001, instance xxxxxx(1)
ORA-01652: impossible d'étendre le segment temporaire de 128 dans le tablespace TEMP

SQL Plan Monitoring Details (Plan Hash Value=762135660)
=============================================================================
| Id |            Operation            |       Name       |  Rows   | Temp  |
|    |                                 |                  | (Estim) | (Max) |
=============================================================================
|  0 | CREATE INDEX STATEMENT          |                  |         |       |
|  1 |   PX COORDINATOR                |                  |         |       |
|  2 |    PX SEND QC (RANDOM)          | :TQ10000         |      3G |       |
|  3 |     PX PARTITION HASH ALL       |                  |      3G |       |
|  4 |      INDEX BUILD UNIQUE (LOCAL) | PK_TABLE_XXXX    |         |       |
|  5 |       SORT CREATE INDEX         |                  |      3G |   32G | -->
|  6 |        TABLE ACCESS FULL        | TABLE_XXXX       |      3G |       |
=============================================================================

This 32GB of maximum TEMP space looks very odd. I have already been working with this data base and I was practically sure that it allows more than this limit. So I looked at the v$sort_segment view:

SQL> compute sum Label 'Total Temp Used' of "Space(GB)" on report
SQL> break on report
SQL> select
       *
    from
    (select
            tablespace_name
           ,inst_id
           ,round((total_blocks*8192)/(1024*1024*1024),2) "Space(GB)"
      from
          gv$sort_segment order by 1,2);

TABLESPACE_NAME                    INST_ID  Space(GB)
------------------------------- ---------- ----------
TEMP                                     1      31.25
TEMP                                     2     656.22
                                           ----------
Total Temp Used                                687.47

Notice this 31.25GB of TEMP Space in instance 1. It looks to be closely related to the limit we’ve hit. And, indeed, the index creating script was started from instance 1 as shown in the corresponding Real Time SQL Monitoring report:

Global Information
------------------------------
 Status              :  DONE (ERROR)
 Instance ID         :  1                    --> spot this
 Session             :  xxxxxx(908:33137)
 SQL ID              :  1h8puyf4b3bw7
 SQL Execution ID    :  16777216
 Execution Started   :  01/25/2016 18:02:40
 First Refresh Time  :  01/25/2016 18:02:40
 Last Refresh Time   :  01/25/2016 18:04:07
 Duration            :  87s
 Module/Action       :  SQL Developer/-
 Service             :  xxxxxx_INST1
 Program             :  SQL Developer

This database is a RAC (11.2.0.3.0) with 2 nodes. It possesses a TEMP tablespace composed with 22 temporary files each of which has 32GB of size. This makes a total available TEMP space of 22*32 = 704GB. Having no clue about the reason for which Oracle has not been able to use the remaining TEMP space from instance 2, i asked to run the same script from instance 2 of the same database:

Global Information
------------------------------
 Status              :  DONE
 Instance ID         :  2   --> second instance
 Session             :  xxxxxx(401:717)
 SQL ID              :  1h8puyf4b3bw7
 SQL Execution ID    :  33554432
 Execution Started   :  01/26/2016 12:02:59
 First Refresh Time  :  01/26/2016 12:03:00
 Last Refresh Time   :  01/26/2016 12:30:07
 Duration            :  1628s
 Module/Action       :  SQL Developer/-
 Service             :  xxxxxx_INST2
 Program             :  SQL Developer       

SQL Plan Monitoring Details (Plan Hash Value=762135660)
=============================================================================
| Id |            Operation            |       Name       |  Rows   | Temp  |
|    |                                 |                  | (Estim) | (Max) |
=============================================================================
|  0 | CREATE INDEX STATEMENT          |                  |         |       |
|  1 |   PX COORDINATOR                |                  |         |       |
|  2 |    PX SEND QC (RANDOM)          | :TQ10000         |      3G |       |
|  3 |     PX PARTITION HASH ALL       |                  |      3G |       |
|  4 |      INDEX BUILD UNIQUE (LOCAL) | PK_TABLE_XXXX    |         |       |
|  5 |       SORT CREATE INDEX         |                  |      3G |   99G |
|  6 |        TABLE ACCESS FULL        | TABLE_XXXX       |      3G |       |
=============================================================================

Notice how the index creation, this time, completes without error, in about 27 minutes and consumes 99GB of TEMP space.
Here’s below the situation of the cached extends in gv$temp_extent_pool view immediately after the index successful creation :

SQL> compute sum Label 'Total Temp Used' of extents_cached on report
SQL> break on report
SQL> select inst_id
              , file_id
              , extents_cached
              , extents_used
        from gv$temp_extent_pool
        order by 1,2;

   INST_ID    FILE_ID EXTENTS_CACHED EXTENTS_USED
---------- ---------- -------------- ------------
         1          1          31994            8
         1          2              4            3
         1          3              0            0
         1          4              0            0
         1          5              3            0
         1          6              0            0
         1          7              1            0
         1          8              0            0
         1          9              0            0
         1         10              0            0
         1         11              0            0
         1         12              0            0
         1         13              0            0
         1         14              0            0
         1         15              0            0
         1         16              0            0
         1         17              0            0
         1         18              0            0
         1         19              0            0
         1         20              0            0
         1         21              0            0
         1         22              0            0
         2          1              0            0
         2          2          31995            0
         2          3          31999            0
         2          4          31999            0
         2          5          31996            0
         2          6          31999            0
         2          7          31998            0
         2          8          31999            0
         2          9          31999            0
         2         10          31999            0
         2         11          31999            0
         2         12          31999            0
         2         13          31999            0
         2         14          31999            0
         2         15          31999            0
         2         16          31999            0
         2         17          31999            0
         2         18          31999            1
         2         19          31999            0
         2         20          31999            0
         2         21          31999            0
         2         22          31999            0
                      --------------
Total Temp                    703973
SQL> select
           inst_id,
           tablespace_name,
           total_blocks,
           used_blocks,
           free_blocks
    from gv$sort_segment;

   INST_ID TABLESPACE_NAME                 TOTAL_BLOCKS USED_BLOCKS FREE_BLOCKS
---------- ------------------------------- ------------ ----------- -----------
         1 TEMP                                 4096256        1536     4094720
         2 TEMP                                86012288         128    86012160

We have a TEMP tablespace of 22 temporary files in a RAC configuration with 2 nodes. We can point out that, in instance 1, only the first temporary file that has been used. While in instance 2 we see that many extents have been uniformly allocated during the index creation.

SQL> select inst_id, trunc(bytes_cached/1024/1024/1024,2) Gbytes_cached from gv$temp_extent_pool;

   INST_ID GBYTES_CACHED
---------- -------------
         1         31.24
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         1             0
         2             0
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24
         2         31.24

44 rows selected.

Clearly for a reason I am not aware of, Instance 2 is refusing to uncache its temp extents to instance 1. Thanks to Riyaj Shamsudeen I knew that this situation could be due to the bug n° 14383007 which necessitates a patch to be solved.

1 Comment »

  1. Good to know this bug, but I wonder in January 2016, why use 11.2.0.3.0?

    Comment by Shripad Godbole — January 31, 2016 @ 1:43 pm | Reply


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Create a free website or blog at WordPress.com.

Tony's Oracle Tips

Tony Hasler's light hearted approach to learning about Oracle

Richard Foote's Oracle Blog

Focusing Specifically On Oracle Indexes, Database Administration and Some Great Music

Hatem Mahmoud Oracle's blog

Just another Oracle blog : Database topics and techniques

Mohamed Houri’s Oracle Notes

Qui se conçoit bien s’énonce clairement

Oracle Diagnostician

Performance troubleshooting as exact science

Raheel's Blog

Things I have learnt as Oracle DBA

Coskan's Approach to Oracle

What I learned about Oracle

So Many Oracle Manuals, So Little Time

“Books to the ceiling, Books to the sky, My pile of books is a mile high. How I love them! How I need them! I'll have a long beard by the time I read them”—Lobel, Arnold. Whiskers and Rhymes. William Morrow & Co, 1988.

EU Careers info

Your career in the European Union

Carlos Sierra's Tools and Tips

Tools and Tips for Oracle Performance and SQL Tuning

Oracle Scratchpad

Just another Oracle weblog

OraStory

Dominic Brooks on Oracle Performance, Tuning, Data Quality & Sensible Design ... (Now with added Sets Appeal)

%d bloggers like this: