Wednesday, January 28, 2015

Attribute clustering


Attribute clustering is a feature that’s new to Oracle Database 12.1.0.2 (Enterprise Edition). It was designed to work with other features, such as compression, storage indexes, and especially with a new feature called zone maps (Note zone maps don’t work on non-Exadata storage)

So what is attribute clustering? It is simply a table property - just like compression - but it defines how rows should be ordered and clustered together in close physical proximity, based on one or more column values. For example, in a sales transaction table you could choose to cluster together rows that share common customer ID values. Perhaps the system frequently queries data related to particular customers.  The physical database design will probably incorporate an index on customer_id column, but you can gain further benefit if all rows for a particular customer are physically next to each other.



With attribute clustering, the matching sales table rows are near to one another, so it is likely that fewer database blocks will need to be read from storage (or database cache) than if the rows are scattered throughout the sales table. The reason for this is that database blocks will usually contain multiple rows, so it is beneficial if each block we read happens to contains multiple matching rows

Many DBAs have used a similar trick in the past by ordering rows as they are loaded into the database (using an explicit SQL “ORDER BY” clause). Attribute clustering has the advantage of being transparent and a property of the table itself; clustering behavior is inherited from the table definition and is implemented automatically. Just like compression, attribute clustering is a directive that transparently kicks in for certain operations, namely direct path insertion or data movement.

Here’s an example of using attribute clustering to speed up a query. We'll compare before and after; so start by creating a table that is not attribute clustered:

rajesh@PDB1> create table t
  2  as
  3  select a.* ,
  4     mod(rownum,10000)+1 as cust_id
  5  from all_objects a ,
  6     all_users
  7  where rownum <=1e6; 

Table created. 

rajesh@PDB1> create index t_idx on t(cust_id); 

Index created. 

rajesh@PDB1> set autotrace traceonly explain statistics
rajesh@PDB1> select sum(object_id)
  2  from t
  3  where cust_id = 55 ; 

1 row selected.
 

Execution Plan
----------------------------------------------------------
Plan hash value: 1339972470

----------------------------------------------------------------------------------------------
| Id  | Operation                            | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
----------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                     |       |     1 |     9 |   103   (0)| 00:00:01 |
|   1 |  SORT AGGREGATE                      |       |     1 |     9 |            |          |
|   2 |   TABLE ACCESS BY INDEX ROWID BATCHED| T     |   100 |   900 |   103   (0)| 00:00:01 |
|*  3 |    INDEX RANGE SCAN                  | T_IDX |   100 |       |     3   (0)| 00:00:01 |
---------------------------------------------------------------------------------------------- 

Predicate Information (identified by operation id):
--------------------------------------------------- 

   3 - access("CUST_ID"=55)
 

Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
        103  consistent gets
          0  physical reads
          0  redo size
        551  bytes sent via SQL*Net to client
        552  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed 

rajesh@PDB1> set autotrace off


Attribute clustering is a property of the table, so when it is added, existing rows are not re-ordered. The following command is very fast because it just makes a change to the data dictionary:


rajesh@PDB1> alter table t
  2  add clustering by linear order(cust_id)
  3  without materialized zonemap; 

Table altered. 

rajesh@PDB1>

Now we can physically cluster the table data by moving the table.

rajesh@PDB1> alter table t move; 

Table altered. 

rajesh@PDB1> alter index t_idx rebuild nologging; 

Index altered. 

rajesh@PDB1>


Now running the same query, we see this:

rajesh@PDB1> set autotrace traceonly explain statistics
rajesh@PDB1> select sum(object_id)
  2  from t
  3  where cust_id = 55 ; 

1 row selected.
 

Execution Plan
----------------------------------------------------------
Plan hash value: 1339972470 

----------------------------------------------------------------------------------------------
| Id  | Operation                            | Name  | Rows  | Bytes | Cost (%CPU)| Time     |
----------------------------------------------------------------------------------------------
|   0 | SELECT STATEMENT                     |       |     1 |     9 |     5   (0)| 00:00:01 |
|   1 |  SORT AGGREGATE                      |       |     1 |     9 |            |          |
|   2 |   TABLE ACCESS BY INDEX ROWID BATCHED| T     |   100 |   900 |     5   (0)| 00:00:01 |
|*  3 |    INDEX RANGE SCAN                  | T_IDX |   100 |       |     3   (0)| 00:00:01 |
----------------------------------------------------------------------------------------------
 

Predicate Information (identified by operation id):
--------------------------------------------------- 

   3 - access("CUST_ID"=55)
 

Statistics
----------------------------------------------------------
          0  recursive calls
          0  db block gets
          6  consistent gets
          0  physical reads
          0  redo size
        551  bytes sent via SQL*Net to client
        552  bytes received via SQL*Net from client
          2  SQL*Net roundtrips to/from client
          0  sorts (memory)
          0  sorts (disk)
          1  rows processed 

rajesh@PDB1> set autotrace off
rajesh@PDB1>

Tuesday, January 27, 2015

ORA-31969: ZONEMAP not supported for table stored in tablespace of this storage type

Attempting to create zone map on non-Exa storage failed – meaning that zone map are not supported on non-Exadata storage ( I am on 12.1.0.2 )


rajesh@PDB1> select default_tablespace
  2  from user_users;
 

DEFAULT_TABLESPACE
------------------------------
DATA_12C 

1 row selected. 

rajesh@PDB1>
rajesh@PDB1> select tablespace_name,extent_management,
  2      allocation_type,segment_space_management,
  3      predicate_evaluation
  4  from dba_tablespaces
  5  where tablespace_name ='DATA_12C' ;
 

TABLESPACE_NAME                EXTENT_MAN ALLOCATIO SEGMEN PREDICA
------------------------------ ---------- --------- ------ -------
DATA_12C                       LOCAL      SYSTEM    AUTO   HOST
 

1 row selected. 

rajesh@PDB1>
rajesh@PDB1>
rajesh@PDB1> create table mysales
  2  ( sales_id number,
  3    prod_id  number,
  4    cust_id  number,
  5    trans_amt number )
  6  clustering by linear order(cust_id)
  7  with materialized zonemap ;

create table mysales
*

ERROR at line 1:
ORA-31969: ZONEMAP not supported for table stored in tablespace of this storage type 

rajesh@PDB1>

Tuesday, January 20, 2015

How indexes store NULL values


Learnt something new in oracle it’s all about indexing NULL values.  How Oracle goes about indexing NULLs has prompted me to show how one could go about actually determining the answer. The basic question is are NULLs treated as just another column value and grouped accordingly or does Oracle have to somehow search through all the leaf blocks looking for all occurrences of these mysterious NULLs

The answer is that NULLs are basically considered to be potentially the largest value possible by Oracle and so are all grouped and sorted together at the “end” of the index structure (assuming the column is the leading column in the concatenated index, else they’ll be listed last for each distinct column that precedes it in the index).

The first obvious thing to check would be to create a little table and associated index with a few rows and a few NULL column values thrown in and see the results of a SELECT … ORDER BY.

The next thing to check would possibly be to use the DUMP function to again see what Oracle is likely to do with NULL values.

The best place to check of course is within the actual index itself. By determining the actual block that stores our example index, we can perform an index block dump and look at the resultant trace file that describes a representation of the index block to see precisely how Oracle deals with NULLs within indexes.


rajesh@ORA11G> create table t(x varchar2(1),y varchar2(1)); 

Table created. 

rajesh@ORA11G> insert into t values('A','A'); 

1 row created. 

rajesh@ORA11G> insert into t values('B','B'); 

1 row created. 

rajesh@ORA11G> insert into t values('C',NULL); 

1 row created. 

rajesh@ORA11G> insert into t values(NULL,'D'); 

1 row created. 

rajesh@ORA11G> insert into t values(NULL,NULL); 

1 row created. 

rajesh@ORA11G> commit; 

Commit complete. 

rajesh@ORA11G> select * from t order by x,y; 

X Y
- -
A A
B B
C
  D 

5 rows selected. 

rajesh@ORA11G>
### Note how the NULL values are logically ordered last in the above output. 
rajesh@ORA11G> column dump_x format a20
rajesh@ORA11G> column dump_y format a20
rajesh@ORA11G> select x, dump(x) dump_x,
  2     y, dump(y) dump_y
  3  from t
  4  order by x,y; 
X DUMP_X               Y DUMP_Y
- -------------------- - --------------------
A Typ=1 Len=1: 65      A Typ=1 Len=1: 65
B Typ=1 Len=1: 66      B Typ=1 Len=1: 66
C Typ=1 Len=1: 67        NULL
  NULL                 D Typ=1 Len=1: 68
  NULL                   NULL 
 
5 rows selected.
 
rajesh@ORA11G>
### Note the character “A” is represented by 65 and “B” by 66 etc
 
rajesh@ORA11G> create index t_idx on t(x,y); 
 
Index created.
rajesh@ORA11G>
rajesh@ORA11G> alter system flush buffer_cache; 
 
System altered. 
rajesh@ORA11G> column header_file new_val hf
rajesh@ORA11G> column header_block new_val hb
rajesh@ORA11G> select header_file, header_block+1 header_block
  2  from dba_segments
  3  where segment_name ='T_IDX' ; 
 
HEADER_FILE HEADER_BLOCK
----------- ------------
          6        19163 
 
1 row selected. 
 
rajesh@ORA11G> @tkfilename.sql
D:\APP\179818\diag\rdbms\ora11g\ora11g\trace\ora11g_ora_5528.trc 
 
rajesh@ORA11G> alter system dump datafile &hf block &hb;
old   1: alter system dump datafile &hf block &hb
new   1: alter system dump datafile          6 block      19163 
 
System altered. 
 
rajesh@ORA11G> exit 
### Block dump follows
Block header dump:  0x01804adb
 Object id on Block? Y
 seg/obj: 0x17c61  csc: 0x507.9b70310b  itc: 2  flg: E  typ: 2 - INDEX
     brn: 0  bdba: 0x1804ad8 ver: 0x01 opc: 0
     inc: 0  exflg: 0 
 Itl           Xid                  Uba         Flag  Lck        Scn/Fsc
0x01   0x0000.000.00000000  0x00000000.0000.00  ----    0  fsc 0x0000.00000000
0x02   0xffff.000.00000000  0x00000000.0000.00  C---    0  scn 0x0507.9b70310b
Leaf block dump
===============
header address 370311780=0x16128264
kdxcolev 0
KDXCOLEV Flags = - - -
kdxcolok 0
kdxcoopc 0x80: opcode=0: iot flags=--- is converted=Y
kdxconco 3
kdxcosdc 0
kdxconro 4
kdxcofbo 44=0x2c
kdxcofeo 7986=0x1f32
kdxcoavs 7942
kdxlespl 0
kdxlende 0
kdxlenxt 0=0x0
kdxleprv 0=0x0
kdxledsz 0
kdxlebksz 8036
row#0[8023] flag: ------, lock: 0, len=13       <<=== 1st index row
col 0; len 1; (1):  41     <<=== this is HEX ASCII value of column X has value of A
col 1; len 1; (1):  41     <<=== this is HEX ASCII value of column Y has value of A
col 2; len 6; (6):  01 80 07 cd 00 00  <<=== this represents the ROWID
row#1[8010] flag: ------, lock: 0, len=13       <<=== 2nd index row
col 0; len 1; (1):  42     <<=== this is HEX ASCII value of column X has value of B
col 1; len 1; (1):  42     <<=== this is HEX ASCII value of column Y has value of B
col 2; len 6; (6):  01 80 07 cd 00 01 <<=== this represents the ROWID
row#2[7998] flag: ------, lock: 0, len=12       <<=== 3rd index row
col 0; len 1; (1):  43     <<=== this is HEX ASCII value of column X has value of C
col 1; NULL                <<=== this is HEX ASCII value of column Y has value of NULL  
col 2; len 6; (6):  01 80 07 cd 00 02 <<=== this represents the ROWID
row#3[7986] flag: ------, lock: 0, len=12       <<=== 4th index row
col 0; NULL                <<=== this is HEX ASCII value of column X has value of NULL
col 1; len 1; (1):  44     <<=== this is HEX ASCII value of column Y has value of D
col 2; len 6; (6):  01 80 07 cd 00 03 <<=== this represents the ROWID
 
----- end of leaf block dump -----
End dump data blocks tsn: 7 file#: 6 minblk 19163 maxblk 19163
 
*** Note there are only 4 index entries for the 5 rows. Both rows with both X and Y set to NULL is NOT indexed as expected.

Monday, January 12, 2015

PL/SQL might run faster in SQL - 12c


As of Oracle database 12c, two kinds of plsql functions might run faster in SQL.
        PL/SQL functions declared and defined in the With clause of sql statement.
        PL/SQL functions that are defined with “UDF pragma”


rajesh@PDB1> create table t
  2  as
  3  select *
  4    from all_objects; 

Table created. 

rajesh@PDB1> create or replace function is_number(x varchar2)
  2  return varchar2
  3  as
  4     num_error exception;
  5     pragma exception_init(num_error,-6502);
  6  begin
  7     if to_number(x) is not null then
  8             return 'Y';
  9     else
 10             return 'N';
 11     end if;
 12  exception
 13     when num_error then
 14             return 'N';
 15  end;
 16  /

Function created. 

rajesh@PDB1>
rajesh@PDB1> create or replace function is_number2(x varchar2)
  2  return varchar2
  3  as
  4     num_error exception;
  5     pragma exception_init(num_error,-6502);
  6     PRAGMA UDF;
  7  begin
  8     if to_number(x) is not null then
  9             return 'Y';
 10     else
 11             return 'N';
 12     end if;
 13  exception
 14     when num_error then
 15             return 'N';
 16  end;
 17  / 

Function created.
 

rajesh@PDB1> /*
rajesh@PDB1> set termout off
rajesh@PDB1> select object_id, is_number(object_id) from t ;
rajesh@PDB1>
rajesh@PDB1> with function is_num(x varchar2)
rajesh@PDB1> return varchar2
rajesh@PDB1> as
rajesh@PDB1>    num_error exception;
rajesh@PDB1>    pragma exception_init(num_error,-6502);
rajesh@PDB1> begin
rajesh@PDB1>    if to_number(x) is not null then
rajesh@PDB1>            return 'Y';
rajesh@PDB1>    else
rajesh@PDB1>            return 'N';
rajesh@PDB1>    end if;
rajesh@PDB1> exception
rajesh@PDB1>    when num_error then
rajesh@PDB1>            return 'N';
rajesh@PDB1> end ;
rajesh@PDB1> select object_id, is_num(object_id)        from t ;
rajesh@PDB1> /
rajesh@PDB1>
rajesh@PDB1> select object_id, is_number2(object_id) from t ;
rajesh@PDB1> set termout on
rajesh@PDB1> */
rajesh@PDB1>
rajesh@PDB1>
rajesh@PDB1> 

select object_id, is_number(object_id)
from t 
 

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch      597      1.10       1.12       1502       2095          0       89341
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total      599      1.10       1.12       1502       2095          0       89341
 

with function is_num(x varchar2)
return varchar2
as
       num_error exception;
       pragma exception_ini(num_error,-6502);
begin
       if to_number(x) is not null then
              return 'Y';
       else
              return 'N';
       end if;
exception
       when num_error then
              return 'N';
end ;

select object_id, is_num(object_id)      from t
 

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.01       0.06          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch      597      0.29       0.45       1502       2095          0       89341
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total      599      0.31       0.52       1502       2095          0       89341
 
 

select object_id, is_number2(object_id)
from  
 

call     count       cpu    elapsed       disk      query    current        rows
------- ------  -------- ---------- ---------- ---------- ----------  ----------
Parse        1      0.00       0.00          0          0          0           0
Execute      1      0.00       0.00          0          0          0           0
Fetch      597      0.26       0.45       1502       2095          0       89341
------- ------  -------- ---------- ---------- ---------- ----------  ----------
total      599      0.26       0.46       1502       2095          0       89341