1 Hend 4 F 3 Hru 8 Dfu 504 Un
1 Hend 4 F 3 Hru 8 Dfu 504 Un
1 Hend 4 F 3 Hru 8 Dfu 504 Un
Tushar Gupta
Introduction:
“Database performance is one of the trickiest aspects of the operation
of your database. You can usually recognize bad performance easily, but
good performance is more likely the absence of bad performance”
This paper presents the actual results which I obtained during the
course of my study (which I believe will never end) on optimizers. The white
paper deals more of the practical implications rather than the theoretical
aspects of the Optimizers. As rightly believed that Oracle optimizer is very
complex and the experiences I had were more amusing than I had ever
expected. Although after deciding to write a white paper on optimizers it took
me quite a lot amount of time to understand as from where to start on
Optimizers. After reading bits about optimizer its now proving to be a great
fun to write conditional queries, to predict their results and to get overjoyed
when optimizer produces the predicted results, and when it doesn’t work as
predicted then start the think tank to find the answer to the big Why’s raised
by the Optimizers.
• Application design
• SQL
• Memory Usage
• Data Storage
• Data Manipulation
• Physical Storage
• Logical Storage
• Network traffic
Application design:
The application design has the greatest impact on the system performance.
Several factors like table design, distribution of CPU requirements should be
considered while designing an application.
Poor table design normally can overcast a good database design. This
is due to the fact that while fully relational table designs (said to be in the
Third Normal Form are logically desirable, they are physically undesirable).
The problem with such designs is that although they accurately reflect the
ways in which an application’ data is related to other data, they do not reflect
the normal access paths that users will employ to access that data. Once the
users access’s requirements are evaluated, the fully relation table design will
become unworkable for many large queries. Typically, the first problems will
occur with queries that return a large number of columns. These columns are
usually scatte5rred among several tables, forcing the tables to be joined
together during the query. If one of the joined tables is large, then
performance of the whole query may suffer.
In designing the table for an application, we should therefore consider
denormalizing data, i.e. creating small summary tables from large, static
tables. User-centered table design, rather than theory-centered table design,
will yield a system that better meet the user’s requirements.
Tuning Sql:
A well-designed application may still experience performance problems if the
SQL it uses is poorly constructed. In a relational database, the physical
location of data is not as important as it’s logical place within the application
design. However, the database has to find the data in order to return it to a
user performing a query. The key to tuning SQL is to minimize the search
path that the database uses to find the data.
Memory Usage:
Data Storage:
How the database actually stores data also has an effect on the performance
of queries. If the data is fragmented into multiple extents, then resolving a
query may cause the database to look in several physical locations for related
rows. Fragmentation may slow performance when storing new records.
Tuning data storage thus involves tuning both used space and free space.
Data Manipulation:
There are several data manipulation tasks that may be involve manipulation
of large quantities of data. There are several options when loading and
deleting large volume of data, like SQL*Loader Direct Path loading option
provides significant performance improvements over SQL*Loader
Conventional Path loader in loading data into Oracle tables by bypassing SQL
processing, buffer cache management, and unnecessary reads for the data
blocks. The Parallel Data Loading option of SQL*Loader allows multiple
loading processes to work on loading the same table, utilizing spare
resources on the system and there by reducing the overall elapsed time for
loading.
Physical Storage:
How the database actually stores data also has an effect on the performance
of queries. If the data is fragmented into multiple extents, then resolving a
query may cause the database to look in several physical locations for related
rows. Fragmentation may slow performance when storing new records. If
free space in a tablespace is fragmented, then database may have to
dynamically combine neighboring free extents to create a single extent that
is large enough to handle the new space requirements. Tuning data storage
thus involves tuning both used space and free space.
Logical Storage:
From a logical storage point, like objects should be stored together. Objects
should be grouped based on their space usage and user interaction
characteristics. Based on these groupings, tablespaces should be created that
cater to specific types of objects.
Network Traffic:
As the databases and the applications that use them become more
distributed, the network that supports the servers may become the cause of
the delays in delivering the data to the user. Since the same can’t be
avoided, it is important to use the database’s capabilities to reduce the
number of network packets that are required for the data to be delivered. For
example proper use of options like data replication and remote procedure
calls can have a significant impact in improving the database performance.
Overview of SQL Processing Architecture
• Parser
• Optimizer
• Row Source Generator
• SQL Execution Engine
User
Dictionary
Parser
RBO CBO
Rule Based Cost Based
Optimizer Optimizer Optimizer
Mode?
Sql Execution
Row
Source
Generator
• Row source generator: It takes the best plan from optimizer and
generates an execution plan for the sql statement. The execution
plan is a collection of row sources structured in the form of a tree.
Each row source returns a set of rows for that step.
Creating Objects:
Creating table
Create table emp (
empno number,
empname varchar2(100)
);
INSERTING RECORDS:
Insert into emp values(1,’TUS1’);
Insert into emp values(2,’TUS2’);
Insert into emp values(3,’TUS3’);
Insert into emp values(4,’TUS4’);
Formula used:
Remarks:
Since optimizer sees that only one row will be returned, the query is very
selective.
Calculating selectivity:
Selectivity= (100/100)= 1
Remarks:
Before running this query a duplicate record is inserted in the table.
Since the distribution of data is not know to optimizer, it gets the estimate
of the distribution using the data dictionary tables and uses these
estimates to compute selectivity. The optimizer uses following data
dictionary tables to calculate the distribution:
USER_TAB_COLUMNS.NUM_DIST : No of distinct rows from
USER_TABLES.NUM_ROWS :Total no of rows.
i.e :
Selectivity =
(USER_TAB_COLUMNS.NUM_DIST/USER_TABLES.NUM_ROWS)
Calculating selectivity:
The table emp has 5 records, of which 4 are distinct empname and the
rows fetched by the query are 2.
TABLE_NAME
COLUMN_NAME
NUM_DISTINCT
-------------------------------------------------------------------
EMP
EMPNAME
4
TABLE_NAME
NUM_ROWS
EMP
5
The optimizer will read these statistics form user tables and calculate
selectivity as:
Selectivity = (4/5)=0.8
This is crearly wrong selectivity as the distibution of rows is not (4/5) i.e
each distinct value has 0.8 rows for it as in case when empname=’TUS1’
the distinct values are 2. The difference is more visible if number of rows
in the table are very large.
Remarks:
The optimizer uses the boundary value of 3 and the low and high values
of empno in the table and calculates the distribution of values in the table.
Calculating selectivity:
LOW_VALUE HIGH_VALUE
---------------------------------------------- --------------------------
54555331 54555334
The above data is of raw type and thus not in readable format.
The low and high values actually should be 1 and 4 repectively.
Now the optimizer knows that there the low value is 1 and high value is 4
so the 5 values of empno are distributed between 1 and 4 as:
1 = 1.0
1 +1* (3)/4 = 1.75
1 +2*(3)/4 = 2.25
1 +3*(3)/4 = 3.25
1 +4*(3)/4 = 4.0
This shows that there are 3 rows which meet the query creiteria, but
actual are 4!!.
4. Select * from emp where empno < :e1 ; (using bind variable for
boundary value)
Remarks;
Since the optimizer does not know the values of e1 (which can vary)
so it cannot use method as described in example 3 above to determine
the selectivity. Optimizer guesses a small value for the selectivity, taking
into consideration the type of boundary value in the statement. This is
internal default
Remarks:
Remarks:
We will now interpret by means of sample queries and their execution results
how oracle manages the memory allocated to it in form of SGA.
Each set of results are termed as as Result Set in this section and is
referred to as RS from section.
Elapsed: 00:00:02.01
Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=1786 Card=1 Bytes=4)
1 0 TABLE ACCESS (FULL) OF 'EMP' (Cost=1786 Card=1 Bytes=4)
Statistics
----------------------------------------------------------
445 recursive calls
10 db block gets
11835 consistent gets
11775 physical reads
Explanation of outcomes:
Statistics
----------------------------------------------------------
0 recursive calls
9 db block gets
11765 consistent gets
8705 physical reads
Explanation of outcomes:
RS2- The rerunning of the query should have very well avail the data
blocks present in the databata buffer cache buffer. The consistent gets
will always be same as far as the query fetches same type of result,
this is because all the data blocks are first loaded in the data buffer
cache before the user process displays them on sql*plus. So we see
the consistent gets remain same always. The difference here is in the
physical gets which have been reduced drastically from 111775 to
8705. The physical gets have been reduced to some extent but ideally
it should be 0 !!. The reason for as I checked proved to be the lack of
adequate number of buffers in the data cache.
Result Set 3: Rerunning the query of RS1 after flusing the shared pool.
Statistics
----------------------------------------------------------
209 recursive calls
9 db block gets
11795 consistent gets
8709 physical reads
Explanation of outcomes:
RS3- The recursive calls get reduced because the data dictionary data
is flushed. The db block gets, consistent gets should remain same as
the data has come in the data buffer cache. Physical reads will still be
there due to paging. Seeing RS2 it implies that flushing of shared pool
flushes the library cache and the shared sql pool and data buffer cache
and redo buffer cache are not effected.
Result Set 4 : Runing the query as of RS1 after changing the column fetched from
empno to ename (Should have no effect here as this is a full table scan ).
ENAME
----------
Tushar
Elapsed: 00:00:02.09
Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=1786 Card=1 Bytes=10
)
Statistics
----------------------------------------------------------
466 recursive calls
10 db block gets
11838 consistent gets
11775 physical reads
Explanation of outcomes:
RS4- The table empno is not indexed, so changing the column name
which is fetched from empno to ename doesn’t make any difference as
full table scan been performed and it has fetched whole of table data
into the cache.
Result Set5 - Running the query as of RS1 by flushing the shared pool but doing a desc
of table emp before running the query.
Statistics
----------------------------------------------------------
61 recursive calls
9 db block gets
11778 consistent gets
8704 physical reads
Explanation of outcomes:
RS5- Flushing the shared pool should flush all the data dictionary
information from the library cache and the recursive calls should be
very high. On contrary the recursive calls are not much only 61 (445
as in original query). Since we assume that the library cached should
be totally cleared, we conclude that since recursive calls are both due
to space allocation and dictionary loading in the library cache, this
means in this query the space allocation is already defined and oracle
only reloads the data dictionary information in the library cache which
only takes few recursive calls (61).
Result Set6 - Running the query as of RS1 by restarting the database but doing a desc of
table emp before running the query.
Statistics
----------------------------------------------------------
300 recursive calls
10 db block gets
11818 consistent gets
11774 physical reads
Explanation of outcomes:
RS6- What I hardly believed by the result set of RS5 can be assumed
proved by this case. Since the table has been described before running
the select query, some data dictionary information( at least tables )
must have come in the data buffer cache and thus when query is run it
shows lesser recursive calls. I believe Oracle is making more recursive
call to allocate space than to load data dictionary information in my
case.
Effect on Index Scans on SGA
In this section we will look into implications of using index scan on
SGA.
Result Set7: Running query after creating index ind_empno on empno and restarting the
database.
EMPNO
----------
19238
Elapsed: 00:00:00.02
Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=1 Card=1 Bytes=4)
1 0 INDEX (RANGE SCAN) OF 'IND_EMPNO' (NON-UNIQUE) (Cost=1
Car
d=1 Bytes=4)
Statistics
----------------------------------------------------------
505 recursive calls
1 db block gets
86 consistent gets
14 physical reads
Explanation of outcomes:
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
4 consistent gets
0 physical reads
Explanation of outcomes:
Result Set 9 : Rerunning same query after flushing the shared pool
Statistics
----------------------------------------------------------
259 recursive calls
0 db block gets
46 consistent gets
0 physical reads
Explanation of outcomes:
RS9- The recursive calls are well understandable as we saw in RS3. The
increase in consistent (4 to 46) while using index will seem surprising
but actually the increase is due to the some more data block being
read from data buffer cache, Well I wonder why these extra blocks are
getting loaded as data dictionary information can be loaded in the
dictionary cache, I guess this is due to the loading of space
management information in the data cache which is increase the count
of consistent gets.
Result Set 10 : Rerunning same query after changing the column fetched from empno
(which is indexed) to empname (not indexed) after restarting the database.
ENAME
----------
Tushar
Elapsed: 00:00:00.03
Execution Plan
----------------------------------------------------------
0 SELECT STATEMENT Optimizer=CHOOSE (Cost=2 Card=1 Bytes=10)
1 0 TABLE ACCESS (BY INDEX ROWID) OF 'EMP' (Cost=2 Card=1 Byte
s=10)
Statistics
----------------------------------------------------------
526 recursive calls
1 db block gets
90 consistent gets
15 physical reads
Explanation of outcomes:
RS10- On comparing with RS7 the small increase in consistent gets and
physical reads can be explained as follows: The access to index in this
case is followed by a rowid level access of the table as the column
fetch is not indexed. Since the query performed here fetches all the
block as fetched in the RS7 plus block/blocks (very few) corresponding
to table level access, thus increase in consistent gets and the physical
gets is very small.
Now a question arises as why is physical read increased by only 1
while the consistent gets by 4, I guess from the increase in physical
reads that the row containing the empname is a single block and not
many blocks (as the same could have been in case of row-chaining)
and the consistent gets which should increase by 1 have increased by
4 due so some unknow reasons. But what can be predicted from this is
when we rerun this query again the consistent gets should be
increased by 1 (ie to 4 to 5: See RS8). Let’s check it!
Statistics
----------------------------------------------------------
0 recursive calls
0 db block gets
5 consistent gets
0 physical reads
Explanation of outcomes:
1. Access path.
2. Cardinality
3. CBO
4. Consistent gets
5. Cost
6. Db block gets
7. Execution plan
8. Explain plan
9. Optimizer
10.Parser
11.Physical reads
12.RBO
13.Recursive calls
14.Row Source Generator
15.Selectivity
16.SQL Execution Engine
Access Path:
Access path is way data is accessed from a base table. The access path
can be using one of the following ways:
• Full table scan
• Using Index
• Using Rowid
Cardinality:
Represents number of rows in a row set.
CBO:
Cost-based optimization considers statistical information about the
volume and distribution of data within tables and indexes before determining
the execution path for a statement. It then tries to chose the execution path
that has the least “cost.”
Consistent gets:
The number of blocks accessed in the buffer cache for queries without
the SELECT FOR UPDATE clause. The value for this statistic plus the value of
the “db blocks get” constitute what is referred to as a logical read.
Cost:
The cost used by the CBO represents an estimate of the number of
disk I/Os and amount of CPU and memory used in performing an operation
Db block gets:
The number of blocks in the buffer cache that were accessed for
INSERT, UPDATE, DELETE, and SELECT for UPDATE statements.
Execution plan:
Is a collection or row sources structured in the form of a tree. Each
row source returns a set of rows of for that step.
Explain plan:
The Explain page displays explain plans in a hierarchical form for the
SQL statement shown in the SQL Text window. The statement is shown in the
left column. The number of rows each sub-query is expected to yield is listed
in the right column.
Optimizer:
Optimizer uses internal set of Rules or Costing methods to determine
the most efficient plan to produce the results of the query.
Parse:
The main job of parsed is to do syntax and semantic analysis of the sql
statements.
Physical reads:
The number of blocks that were read from disks to satisfy a SELECT,
SELECT FOR UPDATE, INSERT, UPDATE, or DELETE statements.
RBO:
In Rule-based optimization, the Oracle optimizer executes SQL
statements based on a set of syntactical rules and the rankings of various
access paths. It does not consider statistical information relating to the
volume and distribution of data within tables and indexes.
Recursive calls:
The recursive calls are additional statements Oracle must issue, to execute a
query. For example, if you insert a row into a table tat does not have enough
space to hold that row, then Oracle makes recursive calls to allocate that
space dynamically. Recursive calls are also generated when data dictionary
information is not available in the data dictionary cache and must be
retrieved from the disk.
So 2 causes of recursive calls are:
1. To allocate space dynamically.
2. To retrieve data dictionary information to dictionary cache : ( This is
information is loaded into to data dictionary cache as this information
contains information about the object accessed in query, the user
querying has privileges to query this table, also other information are
datafile name, segment name, extent location, table description,
privileges.
Selectivity:
Represents fraction of rows from a row set.
Resources
Following resources were referred during the analyses
1. Oracle 9I Documentation
2. Site: http://www.Performance-Insight.com