Sunday, May 29, 2011

Histograms on Varchar2(32) - 11GR2 Optimizer Improvement

This will be the prolongation of  previous post. Oracle 11GR2 Optimizer got improved and able to provide very accurate cardinality beyond 33 byte string.

On Varchar2(33) bytes of String datas - Estimated Cardinality matches with Actual Cardinality.

rajesh@ORA11GR2> drop table t purge;

Table dropped.

Elapsed: 00:00:00.07
rajesh@ORA11GR2>
rajesh@ORA11GR2> create table t
  2  nologging
  3  as
  4  select rownum as x,
  5        rpad('*',28,'*')||to_char(rownum,'fm00000') as y,
  6        sysdate as z,
  7        a.*
  8  from all_objects a;

Table created.

Elapsed: 00:00:05.70
rajesh@ORA11GR2>
rajesh@ORA11GR2>
rajesh@ORA11GR2> create index t_ind on t(y) nologging;

Index created.

Elapsed: 00:00:00.43
rajesh@ORA11GR2>
rajesh@ORA11GR2> begin
  2     dbms_stats.gather_table_stats(
  3     ownname =>user,
  4     tabname =>'T',
  5     estimate_percent=> dbms_stats.auto_sample_size,
  6     method_opt=>'for all indexed columns size 254' );
  7  end;
  8  /

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.79
rajesh@ORA11GR2>
rajesh@ORA11GR2> select column_name,count(*)
  2  from user_tab_histograms
  3  where table_name ='T'
  4  group by column_name;

COLUMN_NAME          COUNT(*)
-------------------- --------
Y                         255

Elapsed: 00:00:00.00
rajesh@ORA11GR2>
rajesh@ORA11GR2> variable y varchar2(40);
rajesh@ORA11GR2> exec :y := rpad('*',28,'*')||to_char(55,'fm00000');

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.01
rajesh@ORA11GR2>
rajesh@ORA11GR2> set autotrace traceonly explain statistics;
rajesh@ORA11GR2>
rajesh@ORA11GR2> select x,y,z
  2  from t
  3  where y = :y;

Elapsed: 00:00:00.00

Execution Plan
----------------------------------------------------------
Plan hash value: 1376202287
-------------------------------------------------------------------------------------
| Id  | Operation                   | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |       |     1 |    56 |     4   (0)| 00:00:01 |
|   1 |  TABLE ACCESS BY INDEX ROWID| T     |     1 |    56 |     4   (0)| 00:00:01 |
|*  2 |   INDEX RANGE SCAN          | T_IND |     1 |       |     3   (0)| 00:00:01 |
-------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
   2 - access("Y"=:Y)

Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
          5  consistent gets
          0  physical reads
          0  redo size
        567  bytes sent via SQL*Net to client
        415  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

rajesh@ORA11GR2>
rajesh@ORA11GR2> set autotrace off;
rajesh@ORA11GR2>
rajesh@ORA11GR2>

On Varchar2(34) bytes of String datas - Estimated Cardinality matches with Actual Cardinality.

rajesh@ORA11GR2> drop table t purge;

Table dropped.

Elapsed: 00:00:00.07
rajesh@ORA11GR2>
rajesh@ORA11GR2> create table t
  2  nologging
  3  as
  4  select rownum as x,
  5        rpad('*',29,'*')||to_char(rownum,'fm00000') as y,
  6        sysdate as z,
  7        a.*
  8  from all_objects a;

Table created.

Elapsed: 00:00:05.68
rajesh@ORA11GR2>
rajesh@ORA11GR2>
rajesh@ORA11GR2> create index t_ind on t(y) nologging;

Index created.

Elapsed: 00:00:00.42
rajesh@ORA11GR2>
rajesh@ORA11GR2> begin
  2     dbms_stats.gather_table_stats(
  3     ownname =>user,
  4     tabname =>'T',
  5     estimate_percent=> dbms_stats.auto_sample_size,
  6     method_opt=>'for all indexed columns size 254' );
  7  end;
  8  /

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.78
rajesh@ORA11GR2>
rajesh@ORA11GR2> select column_name,count(*)
  2  from user_tab_histograms
  3  where table_name ='T'
  4  group by column_name;

COLUMN_NAME          COUNT(*)
-------------------- --------
Y                         255

Elapsed: 00:00:00.00
rajesh@ORA11GR2>
rajesh@ORA11GR2> variable y varchar2(40);
rajesh@ORA11GR2> exec :y := rpad('*',29,'*')||to_char(55,'fm00000');

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.00
rajesh@ORA11GR2>
rajesh@ORA11GR2> set autotrace traceonly explain statistics;
rajesh@ORA11GR2>
rajesh@ORA11GR2> select x,y,z
  2  from t
  3  where y = :y;

Elapsed: 00:00:00.00

Execution Plan
----------------------------------------------------------
Plan hash value: 1376202287
-------------------------------------------------------------------------------------
| Id  | Operation                   | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |       |     1 |    57 |     4   (0)| 00:00:01 |
|   1 |  TABLE ACCESS BY INDEX ROWID| T     |     1 |    57 |     4   (0)| 00:00:01 |
|*  2 |   INDEX RANGE SCAN          | T_IND |     1 |       |     3   (0)| 00:00:01 |
-------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
   2 - access("Y"=:Y)

Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
          5  consistent gets
          0  physical reads
          0  redo size
        568  bytes sent via SQL*Net to client
        415  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

rajesh@ORA11GR2>
rajesh@ORA11GR2> set autotrace off;
rajesh@ORA11GR2>

On Varchar2(35) bytes of String datas - Estimated Cardinality starts variying with Actual Cardinality.
rajesh@ORA11GR2> drop table t purge;

Table dropped.

Elapsed: 00:00:00.07
rajesh@ORA11GR2>
rajesh@ORA11GR2> create table t
  2  nologging
  3  as
  4  select rownum as x,
  5        rpad('*',30,'*')||to_char(rownum,'fm00000') as y,
  6        sysdate as z,
  7        a.*
  8  from all_objects a;

Table created.

Elapsed: 00:00:05.60
rajesh@ORA11GR2>
rajesh@ORA11GR2>
rajesh@ORA11GR2> create index t_ind on t(y) nologging;

Index created.

Elapsed: 00:00:00.45
rajesh@ORA11GR2>
rajesh@ORA11GR2> begin
  2     dbms_stats.gather_table_stats(
  3     ownname =>user,
  4     tabname =>'T',
  5     estimate_percent=> dbms_stats.auto_sample_size,
  6     method_opt=>'for all indexed columns size 254' );
  7  end;
  8  /

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.76
rajesh@ORA11GR2>
rajesh@ORA11GR2> select column_name,count(*)
  2  from user_tab_histograms
  3  where table_name ='T'
  4  group by column_name;

COLUMN_NAME          COUNT(*)
-------------------- --------
Y                          72

Elapsed: 00:00:00.00
rajesh@ORA11GR2>
rajesh@ORA11GR2> variable y varchar2(40);
rajesh@ORA11GR2> exec :y := rpad('*',30,'*')||to_char(55,'fm00000');

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.00
rajesh@ORA11GR2>
rajesh@ORA11GR2> set autotrace traceonly explain statistics;
rajesh@ORA11GR2>
rajesh@ORA11GR2> select x,y,z
  2  from t
  3  where y = :y;

Elapsed: 00:00:00.00

Execution Plan
----------------------------------------------------------
Plan hash value: 1376202287
-------------------------------------------------------------------------------------
| Id  | Operation                   | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
-------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT            |       |   351 | 20358 |    13   (0)| 00:00:01 |
|   1 |  TABLE ACCESS BY INDEX ROWID| T     |   351 | 20358 |    13   (0)| 00:00:01 |
|*  2 |   INDEX RANGE SCAN          | T_IND |   351 |       |     5   (0)| 00:00:01 |
-------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
   2 - access("Y"=:Y)

Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
          5  consistent gets
          0  physical reads
          0  redo size
        569  bytes sent via SQL*Net to client
        415  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

rajesh@ORA11GR2>
rajesh@ORA11GR2> set autotrace off;
rajesh@ORA11GR2>


On Varchar2(37) bytes of String datas - Estimated Cardinality varies with Actual Cardinality and produced wrong plan.

rajesh@ORA11GR2> drop table t purge;

Table dropped.

Elapsed: 00:00:00.07
rajesh@ORA11GR2>
rajesh@ORA11GR2> create table t
  2  nologging
  3  as
  4  select rownum as x,
  5        rpad('*',32,'*')||to_char(rownum,'fm00000') as y,
  6        sysdate as z,
  7        a.*
  8  from all_objects a;

Table created.

Elapsed: 00:00:05.70
rajesh@ORA11GR2>
rajesh@ORA11GR2>
rajesh@ORA11GR2> create index t_ind on t(y) nologging;

Index created.

Elapsed: 00:00:00.48
rajesh@ORA11GR2>
rajesh@ORA11GR2> begin
  2     dbms_stats.gather_table_stats(
  3     ownname =>user,
  4     tabname =>'T',
  5     estimate_percent=> dbms_stats.auto_sample_size,
  6     method_opt=>'for all indexed columns size 254' );
  7  end;
  8  /

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.76
rajesh@ORA11GR2>
rajesh@ORA11GR2> select column_name,count(*)
  2  from user_tab_histograms
  3  where table_name ='T'
  4  group by column_name;

COLUMN_NAME          COUNT(*)
-------------------- --------
Y                           1

Elapsed: 00:00:00.00
rajesh@ORA11GR2>
rajesh@ORA11GR2> variable y varchar2(40);
rajesh@ORA11GR2> exec :y := rpad('*',32,'*')||to_char(55,'fm00000');

PL/SQL procedure successfully completed.

Elapsed: 00:00:00.00
rajesh@ORA11GR2>
rajesh@ORA11GR2> set autotrace traceonly explain statistics;
rajesh@ORA11GR2>
rajesh@ORA11GR2> select x,y,z
  2  from t
  3  where y = :y;

Elapsed: 00:00:00.00

Execution Plan
----------------------------------------------------------
Plan hash value: 1601196873
--------------------------------------------------------------------------
| Id  | Operation         | Name | Rows  | Bytes | Cost (%CPU)| Time     |
--------------------------------------------------------------------------
|   0 | SELECT STATEMENT  |      | 35867 |  2101K|   428   (1)| 00:00:06 |
|*  1 |  TABLE ACCESS FULL| T    | 35867 |  2101K|   428   (1)| 00:00:06 |
--------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
   1 - filter("Y"=:Y)

Statistics
----------------------------------------------------------
          1  recursive calls
          0  db block gets
       1540  consistent gets
          0  physical reads
          0  redo size
        571  bytes sent via SQL*Net to client
        415  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed

rajesh@ORA11GR2>
rajesh@ORA11GR2> set autotrace off;
rajesh@ORA11GR2>


This shows that Histograms in 11GR2 Optimizer are good to handle with 34 bytes of character data's where as 10G Optimizer is limited to 32 Bytes of character datas.

7 comments:

  1. This was done on 11gR2:

    http://www.oracle.com/technetwork/issue-archive/2011/11-jan/o11asktom-195084.html

    it is still 32 bytes.


    If you drop your index on Y, you'll find the estimates are off in all cases. It was the presence of the index - NOT the histograms - that provided the information used by the optimizer.

    ReplyDelete
  2. Tom:

    As per your suggession Dropped the Index and still estimated cardinality matches with Actual cardinality for 34 bytes.

    On Varchar2(33) bytes of String datas - Estimated Cardinality matches with Actual Cardinality.

    rajesh@ORA11GR2> drop table t purge;

    Table dropped.

    Elapsed: 00:00:00.18
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> create table t
    2 nologging
    3 as
    4 select rownum as x,
    5 rpad('*',28,'*')||to_char(rownum,'fm00000') as y,
    6 sysdate as z,
    7 a.*
    8 from all_objects a;

    Table created.

    Elapsed: 00:00:05.68
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> begin
    2 dbms_stats.gather_table_stats(
    3 ownname =>user,
    4 tabname =>'T',
    5 estimate_percent=>dbms_stats.auto_sample_size,
    6 method_opt=>'for all columns size 254');
    7 end;
    8 /

    PL/SQL procedure successfully completed.

    Elapsed: 00:00:01.42
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> select column_name,count(*)
    2 from user_tab_histograms
    3 where table_name ='T'
    4 and column_name ='Y'
    5 group by column_name;

    COLUMN_NAME COUNT(*)
    -------------------- --------
    Y 255

    Elapsed: 00:00:00.00
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> variable y varchar2(40);
    rajesh@ORA11GR2> exec :y := rpad('*',28,'*')||to_char(55,'fm00000');

    PL/SQL procedure successfully completed.

    Elapsed: 00:00:00.00
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> set autotrace traceonly explain statistics;
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> select x,y,z
    2 from t
    3 where y = :y;

    Elapsed: 00:00:00.01

    Execution Plan
    ----------------------------------------------------------
    Plan hash value: 1601196873

    --------------------------------------------------------------------------
    | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
    --------------------------------------------------------------------------
    | 0 | SELECT STATEMENT | | 1 | 47 | 416 (1)| 00:00:05 |
    |* 1 | TABLE ACCESS FULL| T | 1 | 47 | 416 (1)| 00:00:05 |
    --------------------------------------------------------------------------

    Predicate Information (identified by operation id):
    ---------------------------------------------------

    1 - filter("Y"=:Y)


    Statistics
    ----------------------------------------------------------
    1 recursive calls
    0 db block gets
    1498 consistent gets
    0 physical reads
    0 redo size
    567 bytes sent via SQL*Net to client
    415 bytes received via SQL*Net from client
    2 SQL*Net roundtrips to/from client
    0 sorts (memory)
    0 sorts (disk)
    1 rows processed

    rajesh@ORA11GR2>
    rajesh@ORA11GR2> set autotrace off;
    rajesh@ORA11GR2>
    rajesh@ORA11GR2>

    ReplyDelete
  3. On Varchar2(34) bytes of String datas - Estimated Cardinality matches with Actual Cardinality.

    rajesh@ORA11GR2> drop table t purge;

    Table dropped.

    Elapsed: 00:00:00.18
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> create table t
    2 nologging
    3 as
    4 select rownum as x,
    5 rpad('*',29,'*')||to_char(rownum,'fm00000') as y,
    6 sysdate as z,
    7 a.*
    8 from all_objects a;

    Table created.

    Elapsed: 00:00:05.56
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> begin
    2 dbms_stats.gather_table_stats(
    3 ownname =>user,
    4 tabname =>'T',
    5 estimate_percent=>dbms_stats.auto_sample_size,
    6 method_opt=>'for all columns size 254');
    7 end;
    8 /

    PL/SQL procedure successfully completed.

    Elapsed: 00:00:01.21
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> select column_name,count(*)
    2 from user_tab_histograms
    3 where table_name ='T'
    4 and column_name ='Y'
    5 group by column_name;

    COLUMN_NAME COUNT(*)
    -------------------- --------
    Y 255

    Elapsed: 00:00:00.00
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> variable y varchar2(40);
    rajesh@ORA11GR2> exec :y := rpad('*',29,'*')||to_char(55,'fm00000');

    PL/SQL procedure successfully completed.

    Elapsed: 00:00:00.00
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> set autotrace traceonly explain statistics;
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> select x,y,z
    2 from t
    3 where y = :y;

    Elapsed: 00:00:00.01

    Execution Plan
    ----------------------------------------------------------
    Plan hash value: 1601196873

    --------------------------------------------------------------------------
    | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
    --------------------------------------------------------------------------
    | 0 | SELECT STATEMENT | | 1 | 48 | 418 (1)| 00:00:06 |
    |* 1 | TABLE ACCESS FULL| T | 1 | 48 | 418 (1)| 00:00:06 |
    --------------------------------------------------------------------------

    Predicate Information (identified by operation id):
    ---------------------------------------------------

    1 - filter("Y"=:Y)


    Statistics
    ----------------------------------------------------------
    1 recursive calls
    0 db block gets
    1508 consistent gets
    0 physical reads
    0 redo size
    568 bytes sent via SQL*Net to client
    415 bytes received via SQL*Net from client
    2 SQL*Net roundtrips to/from client
    0 sorts (memory)
    0 sorts (disk)
    1 rows processed

    rajesh@ORA11GR2>
    rajesh@ORA11GR2> set autotrace off;
    rajesh@ORA11GR2>
    rajesh@ORA11GR2>
    rajesh@ORA11GR2>

    ReplyDelete
  4. On Varchar2(35) bytes of String datas - Estimated Cardinality varies with Actual Cardinality.

    rajesh@ORA11GR2> drop table t purge;

    Table dropped.

    Elapsed: 00:00:00.18
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> create table t
    2 nologging
    3 as
    4 select rownum as x,
    5 rpad('*',30,'*')||to_char(rownum,'fm00000') as y,
    6 sysdate as z,
    7 a.*
    8 from all_objects a;

    Table created.

    Elapsed: 00:00:05.56
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> begin
    2 dbms_stats.gather_table_stats(
    3 ownname =>user,
    4 tabname =>'T',
    5 estimate_percent=>dbms_stats.auto_sample_size,
    6 method_opt=>'for all columns size 254');
    7 end;
    8 /

    PL/SQL procedure successfully completed.

    Elapsed: 00:00:01.12
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> select column_name,count(*)
    2 from user_tab_histograms
    3 where table_name ='T'
    4 and column_name ='Y'
    5 group by column_name;

    COLUMN_NAME COUNT(*)
    -------------------- --------
    Y 72

    Elapsed: 00:00:00.00
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> variable y varchar2(40);
    rajesh@ORA11GR2> exec :y := rpad('*',30,'*')||to_char(55,'fm00000');

    PL/SQL procedure successfully completed.

    Elapsed: 00:00:00.00
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> set autotrace traceonly explain statistics;
    rajesh@ORA11GR2>
    rajesh@ORA11GR2> select x,y,z
    2 from t
    3 where y = :y;

    Elapsed: 00:00:00.01

    Execution Plan
    ----------------------------------------------------------
    Plan hash value: 1601196873

    --------------------------------------------------------------------------
    | Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
    --------------------------------------------------------------------------
    | 0 | SELECT STATEMENT | | 315 | 15435 | 422 (1)| 00:00:06 |
    |* 1 | TABLE ACCESS FULL| T | 315 | 15435 | 422 (1)| 00:00:06 |
    --------------------------------------------------------------------------

    Predicate Information (identified by operation id):
    ---------------------------------------------------

    1 - filter("Y"=:Y)


    Statistics
    ----------------------------------------------------------
    1 recursive calls
    0 db block gets
    1519 consistent gets
    0 physical reads
    0 redo size
    569 bytes sent via SQL*Net to client
    415 bytes received via SQL*Net from client
    2 SQL*Net roundtrips to/from client
    0 sorts (memory)
    0 sorts (disk)
    1 rows processed

    rajesh@ORA11GR2>
    rajesh@ORA11GR2> set autotrace off;
    rajesh@ORA11GR2>
    rajesh@ORA11GR2>

    ReplyDelete
  5. ummm, please explain the estimated cardinality of 315 versus the other of 1 then??????????????????

    the column is unique. why is it estimated at 315 if the histogram was working?

    ReplyDelete
  6. @Thomas Kyte:

    I am saying that Histograms in 11GR2 Optimizer was not working beyond 34 Bytes.
    1) when we uses 35 Bytes string data Oracle Estimates 351 for 1 rows.
    2) But when we less than 35 Bytes string data Oracle Estimates 1 for 1 rows.

    ReplyDelete
  7. Thanks for correcting me again Tom!

    http://tinyurl.com/3erm5ng

    ReplyDelete