Blogging about Software Development » Databases

Guru4Pro: Tom Kyte @ Logica

Roel — Fri, 10 Oct 2008 12:17:51 +0000

In the afternoon of November 3 Oracle guru Tom Kyte will talk about “11 Things about 11g” at the Logica office in Amstelveen (near Amsterdam). I have attended this presentation at ODTUG this year and I assure you it’s worthwhile! After his presentation Tom will also do an AskTom Live session. So a great opportunity to enhance your knowledge with everything you wanted to know about Oracle but you never dared to ask…
There is more info on the Logica site (in Dutch, but Tom will do his presentation in English ofcourse). You can also register (”aanmeldingsformulier”) there.

APEX application for maintaining personal contracts

Jasper — Fri, 19 Sep 2008 12:35:34 +0000

APEX is famous for it’s useful applications, so when the idea poped into my head that I would like to track my personal contracts, APEX was first in line to develop such an application.

The purpose of this app is to be able to track longrunning contracts with their renewalperiods. One month before you should end the contract (checked on contract enddate) you’ll receive an email. I used Oracle’s DBMS_SCHEDUAL to call the procedure that sends out the email. Also, it has an authentication scheme so that different users can use this online app. Although it’s dutch, everyone can apply now.

http://apex.oracle.com/pls/otn/f?p=23729

The use of dynamic vs aggregated functions

Jasper — Wed, 03 Oct 2007 11:00:41 +0000

An eyeopener for me when constructing sql queries in a complex environment was the use of analytic functions. As appose to aggregated functions, you can use more analytic functions on one table instead of constructing multiple inline views to accomplish “the same”.

For those of you who like to see examples:

select job_id , sum(salary) from employees group by job_id;

has the same result as:

select distinct job_id , sum(salary) over (partition by job_id) from employees;

But using the second query you could also add for example, the avarage salary per job:

select distinct job_id , sum(salary) over (partition by job_id) , avg(salary) over (partition by job_id) from employees;

If I would want this result using an aggregated function, I would have to use a inline views, like this:

select s.job_id, s.ssal, a.asal from ( select job_id, sum(salary) ssal from employees group by job_id ) s , ( select job_id, avg(salary) asal from employees group by job_id ) a where s.job_id = a.job_id order by s.job_id ;

So that makes the code much more readable. Advantage number one…

But also, something that can’t be accomplished using aggregated functions, is selecting other data besides the grouped data from the table. Such as:

select job_id , first_name , sum(salary) over (partition by job_id) ssal , avg(salary) over (partition by job_id) asal from employees order by job_id ;

Now I can make much better use of this data, and it’s less code!

Sort of automatically I figured that using analytic functions would also be quicker. Because after all, we would only have to do a full table scan once. But I found myself to be greatly mistaken when I checked the explain plans of two different statements, one using aggregated and one using analytic functions. Explain plan for the query above using aggregated function vs the analytic functions shows that the aggregated function has much better performance.

So by using analytic functions on large sets of data… causes some performance difficulties. So using aggregated functions would be better here?

I started searching the net for explanations. Discovering first of all, that analytic functions should not be used to aggregate. Because when you aggregate you return only the aggregated rows. By using a distinct in the query’s above using the analytic functions, you get the same result, but for every row that is selected, the analytic function is added. So logically this is costing us performance.

I discovered this by reading an article on asktom.com. So I guess I should always ask myself the question: do I want to analyse or aggregate?…

Other articles show that the use of analytic of aggregated functions is all up to you. What’s your gain? What’s your personal preference?

I think analytic functions are fantastic and I couldn’t stop using them even if I wanted to But I guess it’s up to you to deside…

I Struck Oil at Miracle Oracle Open World 2007

Yuri van Buren — Tue, 02 Oct 2007 13:09:02 +0000

This was my second time at the Miracle Conference at Lalandia in Denmark.
Again it was “power-pact”.

Mogens Norgaard explained that a lot of conferences have about 80% information and only 20% networking possibilities. This conference is different. Mogens managed to increase the 20% networking up to 80% while still remaining 80% of usefull information. The way he does this is by supplying Miracle ÃƒËœl. As a former geophycicist I now know how it is to struck Miracle ÃƒËœl in the search of indepth Oracle knowledge.

Next to networking with 170 Oracle Database Professionals you could pick out 7 presentations from three tracks all given by experts in the Oracle database field. My picks where Tapio Lahdenmaki, Michael MÃƒÂ¶ller, Cary Millsap (twice), Graham Wood (twice) and Jonathan Lewis.

Tapio’s : rethink your indexes, really gave some hands-on advice. Basically inadequate indexing comes from three things:
- Important Index Missing
- Important Columns Missing
- Wrong Column Order
You even learn new jargon : like Fat and Semi-Fat indexes.
Jonathan Lewis remarked that : “Sometimes the optimizer has to be persuaded that Tapio’s index design is right.”

Then there is the story of Micheal MÃƒÂ¶ller. On the opening evening of this event, Michael gave us a very well executed presentation of Murphy’s Law in practice. Although he had a quadrible back-up the presentation ended with all the lights out in the conference room. It was hilarious. I think Michael has followed a workshop at John Cleese’s. The day after he warned that his presentation on Standby database really contained a serious demo. As always something went wrong now it was a nasty Windows message saying going in Standby mode
The demo had to be postponed to the afternoon.

Cary Millsap showed that almost every body is a performance guy like him.
If you say faster = better, then you are a performance guy as well.
Performance analysis is the study of “how long something takes”. Essential is the concept of a profile: it is full accounting of the time you care about. His story about Bob, Bob’s boy, and the bicycle that became a Ferrari is classic. And it shows how you can explain performance management to people in the business.

Cary also made clear that diagnose-ability is a feature that every application should have. This cost resources just like any other application feature. Jonathan Lewis gave it two different names depending on the situation: “Overhead” or “Infrastructure”. We need pre-emptive logging in our code.

In Cary’s second session he explained why you can’t see your real performance problems. Undiagnosed skew is the common cause why performance projects fail. In many cases it does more harm to go even look at totals aggregated across tasks. He introduced a new Law called Bragan’s Law which says that: If the number you look at it is a total or average, then there is a list underneath. Look at the list is there skew? A nice example of Bragan’s law was given: Statistici will say that your comfortable if you stand in two buckets of water: One Very Hot and one Ice Cold Ã¯ÂÅ . Skew can be detected in raw 10046 trace files.

The session’s of Graham Wood where also interesting. I always like to here Oracle’s side of the story (And not from a Marketing Guy). Did you for instance knew that TK was a performance group in Oracle and PROF there profiler? Graham is the founding father of Statspack, the last years at Oracle he was the architect of DB Time, AWR, ADDM and ASH. Did you knew that dbms_monitor could trace a Module persistent? Every time it starts it is traced.
Did you know that TKPROF cursors must be closed to get plan data? This sometimes is troublesome due to the fact that Oracle has some features to keep cursors open for a while.
Graham showed me that ASH is a great feature in Oracle, it is always on. It samples every second all active session activity data (On CPU, Or Waiting on non-idle events). The design goal from Oracle was to hold one hour of activity in memory. You don’t loose data because every hour the ASH data is flushed to disk or when it is 66% full. With a MOD(10) only 1 of 10 samples is kept in dba_hist_active_sess_history. Although this data does not capture every read and wait it is statistically valid, and you can do all kinds of statistical analysis on it in several dimensions (Like Application, Program, Module, Action, Client-Id and Service.

During the gala dinner I had the chance to sit opposite of Graham Wood, and go into more detail on several of his favorite topics.

The second presentation of Graham focused on DB Time. It combines the best of the YAPP and Method-R methods. It has low intrusion and still has detailed data. You don’t need to scope your collections, you have no requirements to reproduce the problem (This is THE major, difference with Method R!), and you can investigate concurrency problems like locking. ASH tracks down the blocking SID and has details from this SID logged as well.

In short: DB Time is database load, DB time increases when more work is done in the database.
The fundamental performance metric is Average Active Sessions (AAS) = DB Time / Wall Clock Time. The recommendations in ADDM are all focused in reducing DB Time.

Last but not least: Jonathan Lewis gave some very nice tips in the understanding of Statspack data. Jonathan showed that you have to practice a lot with Statspack data. The easiest way is to practice generating the symptoms yourself. Just test and run a full table scan a 100 times and look how it is reported in Statspack. Often you expect to see high “db file” wait events but it turns out to be CPU-intensive (Short explanation is that you might have read from the SAN-cache). Jonathan also mentioned that Skewed data in Wait events is visible in the 10gR2 Statspack report. His final statement: “Knowing the answers, before people ask questions, make you look clever” was a remarkable statement.

To wrap it up. I really liked this conference. The presentations/presenters were inspiring.
The food and beverages where fine. And no people got hurt in the swimming-pool (remember last years pictures). Can’t wait for MOOW08.

Blogging about 11g - Part 10 - Enhanced Optimizer Statistics Maintenance

Yuri van Buren — Tue, 25 Sep 2007 09:34:46 +0000

In Oracle 11g you now can modify gather_database_stats default values with set_global_prefs, set_database_prefs, set_schema_prefs, set_table_prefs in the dbms_stats package.

An even greater new feature is that in Oracle Database 11g, you can separate statistic gathering from statistic publishing.

This allows you to control when new statistics are published. Everybody must have run in to this once, that queries ran differently because part of the statistics where just refreshed or being gathered.

TEST STEPS modifying preferences

About this subject Oracle wrote some Oracle By Example (OBE) scripts available at:

http://otnbeta.us.oracle.com/db11gr1/trng/obes/index.htm

This really speeded up the testing of this new feature.
Get the custemers_obe.dmp file and import it under the SH Schema.

imp sh/sh file= customers_obe.dmp log=imp.log full=y

Import: Release 11.1.0.3.0 - Beta on Thu Mar 29 13:04:10 2007

Connected to: Oracle Database 11g Enterprise Edition Release 11.1.0.3.0 - Beta
With the Partitioning, OLAP and Data Mining options

Export file created by EXPORT:V11.01.00 via conventional path
import done in US7ASCII character set and AL16UTF16 NCHAR character set
import server uses WE8MSWIN1252 character set (possible charset conversion)
. importing SH’s objects into SH
. importing SH’s objects into SH
. . importing table “CUSTOMERS_OBE” 630 rows imported
Import terminated successfully without warnings.

SQL> connect sh/sh

CHECK Default Settings of STALE_PERCENT. Note that STALE_PERCENT determines the percentage of rows
in a table that have to change before the statistics on that table are deemed stale and should be regathered

select dbms_stats.get_prefs(’STALE_PERCENT’, ‘SH’, ‘SALES’) stale_percent
from dual;

STALE_PERCENT
——————————————————————————–
10

To change the default setting for the database do:
execute dbms_stats.set_global_prefs(’STALE_PERCENT’, ‘13′);

PL/SQL procedure successfully completed.

And Check:
SQL> connect sh/sh
select dbms_stats.get_prefs(’STALE_PERCENT’, ‘SH’, ‘SALES’) stale_percent
from dual;

STALE_PERCENT
——————————————————————————–
13

11g allows to change this parameter even on a table by table basis.

For example set the STALE_PERCENT for the SALES table to 65%.
execute dbms_stats.set_table_prefs(’SH’, ‘SALES’, ‘STALE_PERCENT’, ‘65′);

PL/SQL procedure successfully completed.

select dbms_stats.get_prefs(’STALE_PERCENT’, ‘SH’, ‘SALES’) stale_percent
from dual;

STALE_PERCENT
——————————————————————————–
65

Note that another table the product table still uses 13%.

SQL> select dbms_stats.get_prefs(’STALE_PERCENT’, ‘SH’, ‘PRODUCTS’) stale_percent
from dual;

STALE_PERCENT
——————————————————————————–
13

If you want to reset everything back to the default. You first need to delete the table preference you set on SALES so that the table has the global default value. Execute the following SQL script:

execute dbms_stats.delete_table_prefs(’SH’, ‘SALES’, ‘STALE_PERCENT’);

select dbms_stats.get_prefs(’STALE_PERCENT’, ‘SH’, ‘SALES’) stale_percent
from dual;

STALE_PERCENT
——————————————————————————–
13

Now you have to reconnect as system or sysdba to reset the global defaults. You don’t need to remember what the default values are, you simple need to set the preference value to null and that restores the “factory” default value.

Execute the following SQL script:

SQL> execute dbms_stats.set_global_prefs(’STALE_PERCENT’, null);

PL/SQL procedure successfully completed.

SQL> conn sh/sh
Connected.
SQL> select dbms_stats.get_prefs(’STALE_PERCENT’, ‘SH’, ‘SALES’) stale_percent
from dual;

STALE_PERCENT
——————————————————————————–
10

###

TEST STEPS Gathering Private Statistics

You first want to alter the date format and reset the stats for the CUSTOMERS_OBE table.

Execute the following commands:
connect sh/sh
alter session set nls_date_format=’mm/dd hh24:mi:ss’;

– delete statistics
exec dbms_stats.delete_table_stats(’SH’, ‘CUSTOMERS_OBE’);

Now you can review the public statistics for the CUSTOMERS_OBE table.
With the following command:

select table_name, last_analyzed analyze_time, num_rows, blocks, avg_row_len
from user_tables
where table_name = ‘CUSTOMER_OBE’;

no rows selected

– indexes
select index_name, last_analyzed ANALYZE_TIME, num_rows,
leaf_blocks, distinct_keys
from user_indexes
where table_name = ‘CUSTOMER_OBE’
order by index_name;

no rows selected

– columns
select column_name, last_analyzed ANALYZE_TIME, num_distinct,
num_nulls, density
from user_tab_columns
where table_name = ‘CUSTOMER_OBE’
order by column_name;

COLUMN_NAME ANALYZE_TIME NUM_DISTINCT NUM_NULLS DENSITY
—————————— ————– ———— ———- ———-
COUNTRY_ID
CUST_CITY
CUST_CREDIT_LIMIT
CUST_EMAIL
CUST_FIRST_NAME
CUST_GENDER
CUST_ID
CUST_INCOME_LEVEL
CUST_LAST_NAME
CUST_MAIN_PHONE_NUMBER
CUST_MARITAL_STATUS
CUST_POSTAL_CODE
CUST_STATE_PROVINCE
CUST_STREET_ADDRESS
CUST_YEAR_OF_BIRTH

15 rows selected.

Now review the private statistics for the CUSTOMERS_OBE table.

select table_name, last_analyzed “analyze time”, num_rows, blocks, avg_row_len
from user_tab_private_stats
where table_name = ‘CUSTOMER_OBE’ and partition_name is null;

– indexes
select index_name, last_analyzed “analyze time”, num_rows,
leaf_blocks, distinct_keys
from user_ind_private_stats
where table_name = ‘CUSTOMER_OBE’ and partition_name is null
order by index_name;

select column_name, last_analyzed “analyze time”, num_distinct,
num_nulls, density
from user_col_private_stats
where table_name = ‘CUSTOMER_OBE’ and partition_name is null
order by column_name;

Now 3 times no rows selected.

Currently the CUSTOMERS_OBE table does not have any public or private statistics.

NOTE THAT the default behavior in Oracle Database 11g is the same as before where the statistic is published
as soon as the gather is complete.

You can check the preferences by executing the following script:

select dbms_stats.get_prefs(’PUBLISH’) publish from dual;

PUBLISH
——————————————————————————–
TRUE

You can also check or change the publish mode at a table level. You want to check the publish preference
value for the CUSTOMERS_OBE table. It should be the same as the global default.

Check this with:

select dbms_stats.get_prefs(’PUBLISH’, ‘SH’, ‘CUSTOMERS_OBE’) publish from dual;

PUBLISH
——————————————————————————–
TRUE

Set the CUSTOMERS_OBE tables publish value to false. This means that any statistics gathered from now on
will not be automatically published. Execute the following SQL script:

exec dbms_stats.set_table_prefs(’SH’, ‘CUSTOMERS_OBE’, ‘PUBLISH’, ‘false’);

PL/SQL procedure successfully completed.

select dbms_stats.get_prefs(’PUBLISH’, ‘SH’, ‘CUSTOMERS_OBE’) publish from dual;

PUBLISH
——————————————————————————–
FALSE

Now you can gather statistics on the CUSTOMERS_OBE table by executing the following:

execute dbms_stats.gather_table_stats(’SH’, ‘CUSTOMERS_OBE’);

PL/SQL procedure successfully completed.

Since the CUSTOMERS_OBE table has its publish preference set to false there should not be any
public statistics for this table after the gather statistics.

Check with:

– tables
select table_name, last_analyzed analyze_time, num_rows, blocks, avg_row_len
from user_tables
where table_name = ‘CUSTOMER_OBE’;

TABLE_NAME ANALYZE_TIME NUM_ROWS BLOCKS AVG_ROW_LEN
—————————— ————– ———- ———- ———–
CUSTOMERS_OBE

– indexes
select index_name, last_analyzed ANALYZE_TIME, num_rows,
leaf_blocks, distinct_keys
from user_indexes
where table_name = ‘CUSTOMER_OBE’
order by index_name;

INDEX_NAME ANALYZE_TIME NUM_ROWS LEAF_BLOCKS DISTINCT_KEYS
———————— ————– ———- ———– ————-
OBE_CUST_CRED_LMT_IND

– columns
select column_name, last_analyzed ANALYZE_TIME, num_distinct,
num_nulls, density
from user_tab_columns
where table_name = ‘CUSTOMER_OBE’
order by column_name;

15 rows selected.

But now check to see if you have private statistics.

– tables
select table_name, last_analyzed “analyze time”, num_rows, blocks, avg_row_len
from user_tab_private_stats
where table_name = ‘CUSTOMER_OBE’ and partition_name is null;

TABLE_NAME analyze time NUM_ROWS BLOCKS AVG_ROW_LEN
—————————— ————– ———- ———- ———–
CUSTOMERS_OBE 03/29 13:33:01 630 12 137.646032

INDEX_NAME analyze time NUM_ROWS LEAF_BLOCKS DISTINCT_KEYS
———————— ————– ———- ———– ————-
OBE_CUST_CRED_LMT_IND 03/29 13:33:02 630 2 8

– columns
select column_name, last_analyzed “analyze time”, num_distinct,
num_nulls, density
from user_col_private_stats
where table_name = ‘CUSTOMER_OBE’ and partition_name is null
order by column_name;

COLUMN_NAME analyze time NUM_DISTINCT NUM_NULLS DENSITY
—————————— ————– ———— ———- ———-
COUNTRY_ID 03/29 13:32:59 19 0 .052631579
CUST_CITY 03/29 13:32:59 300 0 .003333333
CUST_CREDIT_LIMIT 03/29 13:32:59 8 0 .125
CUST_EMAIL 03/29 13:32:59 400 0 .0025
CUST_FIRST_NAME 03/29 13:32:59 450 0 .002222222
CUST_GENDER 03/29 13:32:59 2 0 .5
CUST_ID 03/29 13:32:59 630 0 .001587302
CUST_INCOME_LEVEL 03/29 13:32:59 12 0 .083333333
CUST_LAST_NAME 03/29 13:32:59 400 0 .0025
CUST_MAIN_PHONE_NUMBER 03/29 13:32:59 630 0 .001587302
CUST_MARITAL_STATUS 03/29 13:32:59 2 234 .5
CUST_POSTAL_CODE 03/29 13:32:59 301 0 .003322259
CUST_STATE_PROVINCE 03/29 13:32:59 120 0 .008333333
CUST_STREET_ADDRESS 03/29 13:32:59 630 0 .001587302
CUST_YEAR_OF_BIRTH 03/29 13:32:59 66 0 .015151515

15 rows selected.

All of the statistics on the CUSTOMERS_OBE table are private. If you wanted to test the new statistics
you could export them from the private statistics table and import them into a test system.
If they were found to be unacceptable you could simply delete them from the private statistics tables
without effecting production.

To test a query without using the private statistics do:
alter session set optimizer_private_statistics = false;
alter session set optimizer_dynamic_sampling = 0;

Now you can get and display an explain plan for your query.

explain plan for
select * from customers_obe where CUST_CREDIT_LIMIT=1500;

Explained.

set linesize 140
select plan_table_output plan from table(dbms_xplan.display(’plan_table’,null,’serial’))
/

PLAN
——————————————————————————————————————————————–
Plan hash value: 278193547

———————————————————————————————
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
———————————————————————————————
| 0 | SELECT STATEMENT | | 1 | 208 | 1 (0)| 00:00:01 |
|* 1 | TABLE ACCESS BY INDEX ROWID| CUSTOMERS_OBE | 1 | 208 | 1 (0)| 00:00:01 |
|* 2 | INDEX RANGE SCAN | CUSTOMERS_OBE | 1 | | 1 (0)| 00:00:01 |
———————————————————————————————

Predicate Information (identified by operation id):
—————————————————

PLAN
——————————————————————————————————————————————–

2 - access(”CUST_CREDIT_LIMIT”=1500)

This is not a very optimal plan since it requeries an index look-up and then a single row access of the table
for every row that has a CUST_CREDIT_LIMIT=1500. Over 20% of the rows in the table have the CUST_CREDIT_LIMIT value of 1500.
As the OBE_CUSTOMERS table grows this plan will get slower and slower as the number of rows matching the query increases.

To see if the optimizer does better when it uses the statistics in the private statistic tables, you need to set
optimizer_private_statistics to true by running the following script:

alter session set optimizer_private_statistics = true;

Session altered.

explain plan for
select * from customers_obe where CUST_CREDIT_LIMIT=1500;

select plan_table_output plan from table(dbms_xplan.display(’plan_table’,null,’serial’));

Explained.

PLAN
——————————————————————————————————————————————–
Plan hash value: 520139036

———————————————————————————–
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
———————————————————————————–
| 0 | SELECT STATEMENT | | 79 | 10586 | 4 (0)| 00:00:01 |
|* 1 | TABLE ACCESS FULL| CUSTOMERS_OBE | 79 | 10586 | 4 (0)| 00:00:01 |
———————————————————————————–

Predicate Information (identified by operation id):
—————————————————

PLAN
——————————————————————————————————————————————–

1 - filter(”CUST_CREDIT_LIMIT”=1500)

13 rows selected.

The OBE Guide states that this plan with the private statistics is much better.
You now do a full table scan which retrieves all of the matching rows with a single pass of the table.
This is the plan you want to run in production. We can make them public by executing the following script:

exec dbms_stats.publish_private_stats(null, null);

PL/SQL procedure successfully completed.

Now if you check the public statistics for the CUSTOMERS_OBE table you should see all of the statistics you gathered earlier.

– tables
select table_name, last_analyzed analyze_time, num_rows, blocks, avg_row_len
from user_tables
where table_name = ‘CUSTOMERS_OBE’;

TABLE_NAME ANALYZE_TIME NUM_ROWS BLOCKS AVG_ROW_LEN
—————————— ————– ———- ———- ———–
CUSTOMERS_OBE 03/29 13:33:01 630 12 137

– indexes
select index_name, last_analyzed ANALYZE_TIME, num_rows,
leaf_blocks, distinct_keys
from user_indexes
where table_name = ‘CUSTOMERS_OBE’
order by index_name;

INDEX_NAME ANALYZE_TIME NUM_ROWS LEAF_BLOCKS DISTINCT_KEYS
—————————— ————– ———- ———– ————-
OBE_CUST_CRED_LMT_IND 03/29 13:33:02 630 2 8

– columns
select column_name, last_analyzed ANALYZE_TIME, num_distinct,
num_nulls, density
from user_tab_columns
where table_name = ‘CUSTOMERS_OBE’
order by column_name;

COLUMN_NAME ANALYZE_TIME NUM_DISTINCT NUM_NULLS DENSITY
—————————— ————– ———— ———- ———-
COUNTRY_ID 03/29 13:32:59 19 0 .052631579
CUST_CITY 03/29 13:32:59 300 0 .003333333
CUST_CREDIT_LIMIT 03/29 13:32:59 8 0 .125
CUST_EMAIL 03/29 13:32:59 400 0 .0025
CUST_FIRST_NAME 03/29 13:32:59 450 0 .002222222
CUST_GENDER 03/29 13:32:59 2 0 .5
CUST_ID 03/29 13:32:59 630 0 .001587302
CUST_INCOME_LEVEL 03/29 13:32:59 12 0 .083333333
CUST_LAST_NAME 03/29 13:32:59 400 0 .0025
CUST_MAIN_PHONE_NUMBER 03/29 13:32:59 630 0 .001587302
CUST_MARITAL_STATUS 03/29 13:32:59 2 234 .5
CUST_POSTAL_CODE 03/29 13:32:59 301 0 .003322259
CUST_STATE_PROVINCE 03/29 13:32:59 120 0 .008333333
CUST_STREET_ADDRESS 03/29 13:32:59 630 0 .001587302
CUST_YEAR_OF_BIRTH 03/29 13:32:59 66 0 .015151515

15 rows selected.

Then check the private statistics for the CUSTOMERS_OBE table. From your terminal window, execute the following script:
– tables
select table_name, last_analyzed “analyze time”, num_rows, blocks, avg_row_len
from user_tab_private_stats
where table_name = ‘CUSTOMERS_OBE’ and partition_name is null;

– indexes
select index_name, last_analyzed “analyze time”, num_rows,
leaf_blocks, distinct_keys
from user_ind_private_stats
where table_name = ‘CUSTOMERS_OBE’ and partition_name is null
order by index_name;

– columns
select column_name, last_analyzed “analyze time”, num_distinct,
num_nulls, density
from user_col_private_stats
where table_name = ‘CUSTOMERS_OBE’ and partition_name is null
order by column_name;

3 times no rows selected

Notice that your private statistics are now gone because they have been published.

#######
Conclusion: In 11g Enhanced Optimizer Statistics Maintenance is possible.
The default behaviour is the same as in 10g, publish automatically.

Beware to document / monitor private statistics areas. They could have a big impact on the explain plan and
on the performance of the queries on those objects.

Blogging about 11g - Part 9 - Password Complexity Checker

Yuri van Buren — Fri, 31 Aug 2007 12:10:17 +0000

In Oracle 11g you can enable the built-in Password Complexity Checker function.

Background info can be found in the Database Security Guide.

The Oracle Database provides a sample password verification function in the PL/SQL script UTLPWDMG.SQL
(located in ORACLE_BASE/ORACLE_HOME/RDBMS/ADMIN) that, when enabled, checks whether users are correctly creating or modifying their passwords. The UTLPWDMG.SQL script provides two password verification functions: one for previous releases of Oracle Database (which is commented out) and an updated version for Oracle Database Release 11g.

The UTLPWDMG.SQL script checks for the following when users create or modify passwords:
-The password contains no fewer than 8 characters.
-The password is not the same as the user name, nor is it the user name spelled backward or with digits appended.
-The password is not the same as the server name or the server name with the digits 1:100 appended.
-The password is not too simple, for example, welcome1, database1, account1, user1234, password1, oracle, oracle123, computer1, abcdefg1, or change_on_install.
-The password includes at least 1 digit and 1 alphabetic character.
-The password differs from the previous password by at least 3 letters.

Now Lets Test.

SQL> show user
USER is “SYS”
SQL> @?\rdbms\admin\utlpwdmg

Function created.

Profile changed.

Function created.

SQL> alter user scott identified by tiger1;
alter user scott identified by tiger1
*
ERROR at line 1:
ORA-28003: password verification for the specified password failed
ORA-20001: Password length less than 8

SQL> alter user scott identified by welcome1;
alter user scott identified by welcome1
*
ERROR at line 1:
ORA-28003: password verification for the specified password failed
ORA-20006: Password too simple

SQL> alter user scott identified by change_on_install;
alter user scott identified by change_on_install
*
ERROR at line 1:
ORA-28003: password verification for the specified password failed
ORA-20006: Password too simple

SQL> alter user scott identified by oracle23;
alter user scott identified by oracle23
*
ERROR at line 1:
ORA-28003: password verification for the specified password failed
ORA-20007: Password too simple

SQL> alter user scott identified by manager1;

User altered.

Well what is simple and what is not. The function works for what its worth …

#######
Conclusion: In 11g a built-in password complexity function can be used to rapidly enforce a stonger password.

I believe this verification function was already available in 8i or 9i.
So not very new feature, it is however very easy to enable. Run the utlpwdmg.sql as SYS and you are done!

Blogging about 11g Part 8 - Interval Partitioning

Yuri van Buren — Fri, 24 Aug 2007 14:14:42 +0000

DBA’s can finally use the Automatic Interval Partitioning option.

Test steps following the Oracle By Example (OBE) scripts available at
http://otnbeta.us.oracle.com/db11gr1/trng/obes/index.htm:

– create interval partitioned table

create table newsales
( prod_id number(6) not null
, cust_id number not null
, time_id date not null
, channel_id char(1) not null
, promo_id number(6) not null
, quantity_sold number(3) not null
, amount_sold number(10,2) not null
)
partition by range (time_id)
interval (numtodsinterval(1,’DAY’))
( partition p_before_1_jan_2005 values
less than (to_date(’01-01-2005′,’dd-mm-yyyy’)))
/

Table is created.

Show partitions and partition boundaries for the interval partitioned table

select partition_name, high_value
from user_tab_partitions
where table_name = ‘NEWSALES’
order by partition_position
/

Insert some data after January 1 2005 that forces the creation of a new partition (segment)

insert into newsales values (11160,17450,to_date(’01-jan-2005′,’dd-mon-yyyy’),’I',9999,19,798) ;
insert into newsales values (1340,33710,to_date(’02-jan-2005′,’dd-mon-yyyy’),’S',9999,16,1264) ;
insert into newsales values (25270,65880,to_date(’05-jan-2005′,’dd-mon-yyyy’),’I',9999,5,210) ;
insert into newsales values (1615,73480,to_date(’05-jan-2005′,’dd-mon-yyyy’),’I',9999,8,96) ;
insert into newsales values (1900,84910,to_date(’06-jan-2005′,’dd-mon-yyyy’),’I',9999,42,378) ;
insert into newsales values (8085,37900,to_date(’09-jan-2005′,’dd-mon-yyyy’),’S',9999,1,68) ;
insert into newsales values (755,26590,to_date(’09-jan-2005′,’dd-mon-yyyy’),’I',9999,11,132) ;
insert into newsales values (10,68060,to_date(’09-jan-2005′,’dd-mon-yyyy’),’P',9999,28,4900) ;
insert into newsales values (13425,109310,to_date(’10-jan-2005′,’dd-mon-yyyy’),’I',9999,1,68) ;
insert into newsales values (1955,65190,to_date(’10-jan-2005′,’dd-mon-yyyy’),’S',9999,28,1512) ;

commit
/

Check again:

select partition_name, high_value
from user_tab_partitions
where table_name = ‘NEWSALES’
order by partition_position
/

PARTITION_NAME HIGH_VALUE
————————- ————————————————————————————-
P_BEFORE_1_JAN_2005 TO_DATE(’ 2005-01-01 00:00:00′, ‘SYYYY-MM-DD HH24:MI:SS’, ‘NLS_CALENDAR=GREGORIAN’)
SYS_P41 TO_DATE(’ 2005-01-02 00:00:00′, ‘SYYYY-MM-DD HH24:MI:SS’, ‘NLS_CALENDAR=GREGORIAN’)
SYS_P42 TO_DATE(’ 2005-01-03 00:00:00′, ‘SYYYY-MM-DD HH24:MI:SS’, ‘NLS_CALENDAR=GREGORIAN’)
SYS_P43 TO_DATE(’ 2005-01-06 00:00:00′, ‘SYYYY-MM-DD HH24:MI:SS’, ‘NLS_CALENDAR=GREGORIAN’)
SYS_P44 TO_DATE(’ 2005-01-07 00:00:00′, ‘SYYYY-MM-DD HH24:MI:SS’, ‘NLS_CALENDAR=GREGORIAN’)
SYS_P45 TO_DATE(’ 2005-01-10 00:00:00′, ‘SYYYY-MM-DD HH24:MI:SS’, ‘NLS_CALENDAR=GREGORIAN’)
SYS_P46 TO_DATE(’ 2005-01-11 00:00:00′, ‘SYYYY-MM-DD HH24:MI:SS’, ‘NLS_CALENDAR=GREGORIAN’)

7 rows selected.

Wow ==> Automatic Interval Partitioning Works Fine.

But when people insert strange values you might end up with a lot of partitions you don’t really want to have.
So does this really help the lazydba?

#######
Conclusion: In 11g finally Automatic Interval Partitioning is available.
No more full overflow partitions, and DBA’s who forgot to create a new partition on time.

Blogging about 11g - Part 5 - SQL Plan Management

Yuri van Buren — Mon, 20 Aug 2007 10:46:21 +0000

The new 11g SQL plan management feature enables the optimizer to maintain a history of
execution plans for a SQL statement. Using the execution plan history the optimizer is
able to detect a new plan representing a plan change for a SQL statement. When the
optimizer detects a new plan, it stores the new plan and marks it for performance
evaluation and uses the old plan. The optimizer uses the new plan only after its performance
is verified to be better than that of the old plan. With this feature performance regressions
caused by execution plan changes can be prevented.

SQL plan management is a preventative mechanism that records and evaluates the execution plans of SQL statements over time, and builds SQL plan baselines composed of a set of existing plans known to be efficient. The SQL plan baselines are then used to preserve performance of corresponding SQL statements, regardless of changes occurring in the system.

TEST STEPS
1-Capturing SQL Plan Baselines
2-Selecting SQL Plan Baselines
3-Evolving SQL Plan Baselines

Step 1) You can use Automatic or Manual Plan Capture

1a) Automatic Plan Capture
To enable automatic plan capture, set the OPTIMIZER_CAPTURE_SQL_PLAN_BASELINES initialization parameter to TRUE.
By default, this parameter is set to FALSE.

When automatic plan capture is enabled, the system automatically creates and maintains the plan history for SQL statements using information provided by the optimizer. The plan history will include relevant information used by the optimizer to reproduce an execution plan, such as the SQL text, outline, bind variables, and compilation environment.

The initial plan generated for a SQL statement is marked as accepted for use by the optimizer, and represents both the plan history and the SQL plan baseline. All subsequent plans will be included in the plan history, and those plans that are verified not to cause performance regressions will be added to the SQL plan baseline during the SQL plan baseline evolution phase.

1b) Manual Plan Loading
Another way to create SQL plan baselines is by manually loading existing plans for a
set of SQL statements as SQL plan baselines. The manually loaded plans are not
verified for performance, but are added as accepted plans to existing or new SQL plan
baselines. Manual plan loading can be used in conjunction with or as an alternative to
automatic plan capture. Manual plan loading can be performed by:
-Loading Plans from SQL Tuning Sets and AWR Snapshots
-Loading Plans from the Cursor Cache

To load plans from a SQL Tuning Set, use the LOAD_PLANS_FROM_SQLSET function of the DBMS_SPM package:

I used my SQL Tuning Set named SQLSET2 (See blogging about 11g - Part 2 SQL Replay).

declare
xx PLS_INTEGER;
begin
xx :=DBMS_SPM.LOAD_PLANS_FROM_SQLSET(sqlset_name => ‘SQLSET2′);
end
;
/

PL/SQL procedure successfully completed.

To load plans from Automatic Workload Repository (AWR), load the plans stored in AWR snapshots into a SQL Tuning Set before using the LOAD_PLANS_FROM_SQLSET function. Loading Plans from the Cursor Cache To load plans from the cursor cache, use the LOAD_PLANS_FROM_CURSOR_CACHE function of the DBMS_SPM package.

To load plans from the cursor cache look for the correct sql_id and do:

declare
YY PLS_INTEGER;
BEGIN
YY :=DBMS_SPM.LOAD_PLANS_FROM_CURSOR_CACHE(sql_id=>’40b1ud5f95×9p’);
end;
/

PL/SQL procedure successfully completed.

Checking the sql_id’s can be done with:

select user_name, hash_value, sql_id, sql_text from v$open_cursor
where user_name =’SCOTT’ and hash_value like ‘1553134901′
/

USER_NAME HASH_VALUE SQL_ID
—————————— ———- ————-
SQL_TEXT
————————————————————
SCOTT 1553134901 40b1ud5f95×9p
select e.ename from scott.emp e where e.ename = ‘BLAKE’

As you can see If tested with the SCOTT user. It has ran the query “select e.ename from scott.emp e where e.ename = ‘BLAKE’ ”
a few times, so it is in the cursor cache.

To be able to see captured baselines I altered scott’s session with:

alter session set optimizer_capture_sql_plan_baselines = TRUE;

Re-executed the query a few times and then checked if there where any SQL Plan Baselines.

SQL> select sql_handle, plan_name, enabled, accepted, fixed from dba_sql_plan_baselines;

SQL_HANDLE PLAN_NAME ENA ACC FIX
—————————— —————————— — — —
SYS_SQL_02a86218930bbb20 SYS_SQL_PLAN_930bbb20b349b57a YES YES NO
SYS_SQL_0542ffe08bcc9712 SYS_SQL_PLAN_8bcc97123fa57a94 YES YES NO
SYS_SQL_07b3f7d6dc71eb16 SYS_SQL_PLAN_dc71eb1660a8367e YES YES NO
SYS_SQL_115024ccba5e158c SYS_SQL_PLAN_ba5e158c9d3031ef YES YES NO
SYS_SQL_1278d3e5ddb954cd SYS_SQL_PLAN_ddb954cd3b22d354 YES YES NO
SYS_SQL_294c437e331fa51f SYS_SQL_PLAN_331fa51f7f6c12a1 YES YES NO
SYS_SQL_4a0d0d05997d40be SYS_SQL_PLAN_997d40be68603fd4 YES YES NO
SYS_SQL_683ffb98878e87db SYS_SQL_PLAN_878e87db68603fd4 YES YES NO
SYS_SQL_a987298e43f76bc3 SYS_SQL_PLAN_43f76bc3ebce39a4 YES YES NO
SYS_SQL_ab81ddd2fd7e0a66 SYS_SQL_PLAN_fd7e0a6652c669e9 YES YES NO
SYS_SQL_b5a286368feca840 SYS_SQL_PLAN_8feca84068603fd4 YES YES NO
SYS_SQL_cf8bfba0667f84ff SYS_SQL_PLAN_667f84ff0377d7c1 YES YES NO

12 rows selected.

==> Our query from SCOTT is: SYS_SQL_a987298e43f76bc3 SYS_SQL_PLAN_43f76bc3ebce39a4 YES YES NO

I found the PLAN_NAME named SYS_SQL_PLAN_43f76bc3ebce39a4 by using: set autotrace on

SQL> select e.ename from scott.emp e where e.ename = ‘BLAKE’
2 ;

ENAME
———-
BLAKE

Execution Plan
———————————————————-
Plan hash value: 3956160932

————————————————————————–
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
————————————————————————–
| 0 | SELECT STATEMENT | | 1 | 6 | 3 (0)| 00:00:01 |
|* 1 | TABLE ACCESS FULL| EMP | 1 | 6 | 3 (0)| 00:00:01 |
————————————————————————–

Predicate Information (identified by operation id):
—————————————————

1 - filter(”E”.”ENAME”=’BLAKE’)

SQL> /

ENAME
———-
BLAKE

Execution Plan
———————————————————-
Plan hash value: 3956160932

Predicate Information (identified by operation id):
—————————————————

1 - filter(”E”.”ENAME”=’BLAKE’)

Note
—–
- SQL plan baseline “SYS_SQL_PLAN_43f76bc3ebce39a4″ used for this statement

HERE it is at the bottem.

Using the following command I can create a report for this plan.

select * from table(
dbms_xplan.display_sql_plan_baseline(
plan_name=>’SYS_SQL_PLAN_43f76bc3ebce39a4′,
format=>’basic’));

PLAN_TABLE_OUTPUT
——————————————————————————–

——————————————————————————–
SQL handle: SYS_SQL_a987298e43f76bc3
SQL text: select e.ename from scott.emp e where e.ename = ‘BLAKE’
——————————————————————————–

——————————————————————————–
Plan name: SYS_SQL_PLAN_43f76bc3ebce39a4
Enabled: YES Fixed: NO Accepted: YES Origin: AUTO-CAPTURE
——————————————————————————–

Plan hash value: 3956160932

———————————-
| Id | Operation | Name |
———————————-
| 0 | SELECT STATEMENT | |
| 1 | TABLE ACCESS FULL| EMP |
———————————-

19 rows selected.

YOU can see the Origin: AUTO-CAPTURE. I still have some issues doing a manual capture?!

I can also report on this query using the SQL_HANDLE that I have found in dba_sql_plan_baselines.

select * from table(
dbms_xplan.display_sql_plan_baseline(
sql_handle=>’SYS_SQL_a987298e43f76bc3′,
format=>’basic’));

PLAN_TABLE_OUTPUT
——————————————————————————–

Plan hash value: 3956160932

———————————-
| Id | Operation | Name |
———————————-
| 0 | SELECT STATEMENT | |
| 1 | TABLE ACCESS FULL| EMP |
———————————-

19 rows selected.

It shows the same result.

Note that the FORMAT string determines what information stored in the plan
displayed. One of three format values (’BASIC’, ‘TYPICAL’, ‘ALL’)
can be used, each representing a common use case.

Let report all, then we get:

select * from table(
dbms_xplan.display_sql_plan_baseline(
sql_handle=>’SYS_SQL_a987298e43f76bc3′,
format=>’all’));

PLAN_TABLE_OUTPUT
——————————————————————————–

Plan hash value: 3956160932

Query Block Name / Object Alias (identified by operation id):
————————————————————-

1 - SEL$1 / E@SEL$1

Predicate Information (identified by operation id):
—————————————————

1 - filter(”E”.”ENAME”=’BLAKE’)

Column Projection Information (identified by operation id):
———————————————————–

1 - “E”.”ENAME”[VARCHAR2,10]

34 rows selected.

AHA we get some more information. Especially the Column Projection Information is interesting.
It shows which column is in the where clause, and with an index this query can be speeded up.

Lets tune this “Full Table Scanned” query.

SQL> create index EMP_NAME on EMP(ename);

Index created.

SQL> select e.ename from scott.emp e where e.ename = ‘BLAKE’;

ENAME
———-
BLAKE

Execution Plan
———————————————————-
Plan hash value: 3956160932

Predicate Information (identified by operation id):
—————————————————

1 - filter(”E”.”ENAME”=’BLAKE’)

Note
—–
- SQL plan baseline “SYS_SQL_PLAN_43f76bc3ebce39a4″ used for this statement

No Index is used. SO THIS IS A FEATURE TO RECON WITH.

Step 2) Selecting SQL Plan Baselines
The DBA first has to checkout the available SQL PLans ..

SQL> select sql_handle, plan_name, enabled, accepted, fixed from dba_sql_plan_baselines;

SQL_HANDLE PLAN_NAME ENA ACC FIX
—————————— —————————— — — —
SYS_SQL_02a86218930bbb20 SYS_SQL_PLAN_930bbb20b349b57a YES YES NO
SYS_SQL_0542ffe08bcc9712 SYS_SQL_PLAN_8bcc97123fa57a94 YES YES NO
SYS_SQL_07b3f7d6dc71eb16 SYS_SQL_PLAN_dc71eb1660a8367e YES YES NO
SYS_SQL_115024ccba5e158c SYS_SQL_PLAN_ba5e158c9d3031ef YES YES NO
SYS_SQL_1278d3e5ddb954cd SYS_SQL_PLAN_ddb954cd3b22d354 YES YES NO
SYS_SQL_181e870c7b6b7300 SYS_SQL_PLAN_7b6b730052c669e9 YES YES NO
SYS_SQL_294c437e331fa51f SYS_SQL_PLAN_331fa51f7f6c12a1 YES YES NO
SYS_SQL_4a0d0d05997d40be SYS_SQL_PLAN_997d40be68603fd4 YES YES NO
SYS_SQL_683ffb98878e87db SYS_SQL_PLAN_878e87db68603fd4 YES YES NO
SYS_SQL_a987298e43f76bc3 SYS_SQL_PLAN_43f76bc37adb8510 YES NO NO
SYS_SQL_a987298e43f76bc3 SYS_SQL_PLAN_43f76bc3ebce39a4 YES YES NO
SYS_SQL_ab81ddd2fd7e0a66 SYS_SQL_PLAN_fd7e0a6652c669e9 YES YES NO
SYS_SQL_b5a286368feca840 SYS_SQL_PLAN_8feca84068603fd4 YES YES NO
SYS_SQL_cf8bfba0667f84ff SYS_SQL_PLAN_667f84ff0377d7c1 YES YES NO

14 rows selected.

==> Their is an extra PLAN_NAME for our sql_handle : SYS_SQL_a987298e43f76bc3.
It now has an extra NO(t) Accepted plan.

Lets report all again:

select * from table(
dbms_xplan.display_sql_plan_baseline(
sql_handle=>’SYS_SQL_a987298e43f76bc3′,
format=>’all’));

PLAN_TABLE_OUTPUT
——————————————————————————–

——————————————————————————–
Plan name: SYS_SQL_PLAN_43f76bc37adb8510
Enabled: YES Fixed: NO Accepted: NO Origin: AUTO-CAPTURE
——————————————————————————–

Plan hash value: 2061206800

—————————————————————————–
| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |
—————————————————————————–
| 0 | SELECT STATEMENT | | 1 | 6 | 1 (0)| 00:00:01 |
|* 1 | INDEX RANGE SCAN| EMP_NAME | 1 | 6 | 1 (0)| 00:00:01 |
—————————————————————————–

Query Block Name / Object Alias (identified by operation id):
————————————————————-

1 - SEL$1 / E@SEL$1

Predicate Information (identified by operation id):
—————————————————

1 - access(”E”.”ENAME”=’BLAKE’)

Column Projection Information (identified by operation id):
———————————————————–

1 - “E”.”ENAME”[VARCHAR2,10]

Plan hash value: 3956160932

Query Block Name / Object Alias (identified by operation id):
————————————————————-

1 - SEL$1 / E@SEL$1

Predicate Information (identified by operation id):
—————————————————

1 - filter(”E”.”ENAME”=’BLAKE’)

Column Projection Information (identified by operation id):
———————————————————–

1 - “E”.”ENAME”[VARCHAR2,10]

63 rows selected.

AHA An index scan gives cost of 1 in stead of 3. It is 3 times cheaper.

Now is the question how to Accept this better plan.

Step 3) Evolving SQL Plan Baselines

SQL> select sql_handle, plan_name, enabled, accepted, fixed from dba_sql_plan_baselines;

SQL_HANDLE PLAN_NAME ENA ACC FIX
—————————— —————————— — — —
SYS_SQL_a987298e43f76bc3 SYS_SQL_PLAN_43f76bc3ebce39a4 YES NO NO
SYS_SQL_a987298e43f76bc3 SYS_SQL_PLAN_43f76bc37adb8510 YES YES NO
…

The first method is to load plans manually from the cursor cache or a SQL tuning set,
so that the SQL plan baselines are evolved to contain loaded plans as additional accepted plans.

declare
YY PLS_INTEGER;
BEGIN
YY :=DBMS_SPM.LOAD_PLANS_FROM_CURSOR_CACHE(sql_id=>’40b1ud5f95×9p’);
end;
/

PL/SQL procedure successfully completed.

SQL> select sql_handle, plan_name, enabled, accepted, fixed from dba_sql_plan_baselines;

SQL_HANDLE PLAN_NAME ENA ACC FIX
—————————— —————————— — — —
SYS_SQL_a987298e43f76bc3 SYS_SQL_PLAN_43f76bc3ebce39a4 YES YES NO
SYS_SQL_a987298e43f76bc3 SYS_SQL_PLAN_43f76bc37adb8510 YES YES NO
…

==> The plan is now accepted!

Now remove the invalid PLAN_NAME (Which did a full Table scan).

The bad plan with the Full Table Scan is:
SYS_SQL_a987298e43f76bc3 SYS_SQL_PLAN_43f76bc3ebce39a4

PURGE_SQL_PLAN_BASELINE as user SYSTEM Works with:

declare
ZZ PLS_INTEGER;
begin
ZZ:= DBMS_SPM.PURGE_SQL_PLAN_BASELINE(sql_handle =>’SYS_SQL_a987298e43f76bc3′,plan_name =>’SYS_SQL_PLAN_43f76bc3ebce39a4′);
end;
/

PL/SQL procedure successfully completed.

SQL> select sql_handle, plan_name, enabled, accepted, fixed from dba_sql_plan_baselines;

SQL_HANDLE PLAN_NAME ENA ACC FIX
—————————— —————————— — — —
SYS_SQL_02a86218930bbb20 SYS_SQL_PLAN_930bbb20b349b57a YES YES NO
SYS_SQL_b5a286368feca840 SYS_SQL_PLAN_8feca84068603fd4 YES YES NO
SYS_SQL_07b3f7d6dc71eb16 SYS_SQL_PLAN_dc71eb1660a8367e YES YES NO
SYS_SQL_683ffb98878e87db SYS_SQL_PLAN_878e87db68603fd4 YES YES NO
SYS_SQL_4a0d0d05997d40be SYS_SQL_PLAN_997d40be68603fd4 YES YES NO
SYS_SQL_115024ccba5e158c SYS_SQL_PLAN_ba5e158c9d3031ef YES YES NO
SYS_SQL_294c437e331fa51f SYS_SQL_PLAN_331fa51f7f6c12a1 YES YES NO
SYS_SQL_0542ffe08bcc9712 SYS_SQL_PLAN_8bcc97123fa57a94 YES YES NO
SYS_SQL_cf8bfba0667f84ff SYS_SQL_PLAN_667f84ff0377d7c1 YES YES NO
SYS_SQL_1278d3e5ddb954cd SYS_SQL_PLAN_ddb954cd3b22d354 YES YES NO
SYS_SQL_ab81ddd2fd7e0a66 SYS_SQL_PLAN_fd7e0a6652c669e9 YES YES NO
SYS_SQL_181e870c7b6b7300 SYS_SQL_PLAN_7b6b730052c669e9 YES YES NO

12 rows selected.

==> The plan has been purged.

The second method to evolve plans is to use the SQL Tuning Advisor, which identifies tuned plans for a set of SQL statements stored in a SQL tuning set. After verifying the performance of a tuned plan, the SQL Tuning Advisor makes a recommendation to accept a SQL profile. Once the SQL profile is accepted, the tuned plan is added to the corresponding SQL plan baseline.

In Oracle Database 11g, an automatically configured task runs the SQL Tuning Advisor during a maintenance window. This automatic SQL tuning task targets high-load SQL statements as identified by the execution performance data collected in the Automatic Workload Repository (AWR) snapshots. The SQL profile recommendations made by the SQL tuning advisor are implemented by the automatic SQL tuning task. Tuned plans are thus automatically added to the SQL plan baselines of the identified high-load SQL statements.

#######
Conclusion: In 11g SQL Plan Management is possible.

The default behaviour is that automatic plan capture is not enabled
(optimizer_capture_sql_plan_baselines is FALSE).

Does work as designed. If you have the right documentation Associated bug 6010750, wrong documentation. SR Number 6239541.994

Notes:
-Default the purge job waits for 1 year, for UNUSED plans!).
-Disable SQL Plan usage with : optimizer_use_sql_plan_baselines = FALSE. Default is TRUE!
-Stored outlines are deprecated in Oracle Database 11g. Oracle highly recommends migrating existing stored outlines to SQL plan baselines.

Blogging about 11g - Part 4 - Invisible Indexes

Yuri van Buren — Fri, 27 Jul 2007 19:30:30 +0000

The 11g Oracle DBA can now apply an invisible performance patch (an Invisible Index)
on a non-performing query on a production system, without any impact on other queries.

Invisible indexes are not considered by the optimizer. These indexes are only used with a hint.

Here is a demo.

SQL> conn scott/tiger
Connected.

SQL> select * from cat;

TABLE_NAME TABLE_TYPE
—————————— ———–
DEPT TABLE
EMP TABLE
BONUS TABLE
SALGRADE TABLE

4 rows selected.

SQL> select index_name from user_indexes;

INDEX_NAME
——————————
PK_EMP
PK_DEPT

select index_name
, index_owner
, table_name
, column_name
, column_position cpos
, column_length clen
from all_ind_columns
where index_name like ‘PK%’
and index_owner = ‘SCOTT’
order by index_name, table_name, cpos;

INDEX_NAME INDEX_OWNER TABLE_NAME COLUMN_NAME CPOS CLEN
——————– ———– ————— ——————– —– —–
PK_DEPT SCOTT DEPT DEPTNO 1 22
PK_EMP SCOTT EMP EMPNO 1 22

SQL> desc emp
Name Null? Type
—————————————————————– ——– ——————————————–
EMPNO NOT NULL NUMBER(4)
ENAME VARCHAR2(10)
JOB VARCHAR2(9)
MGR NUMBER(4)
HIREDATE DATE
SAL NUMBER(7,2)
COMM NUMBER(7,2)
DEPTNO NUMBER(2)

NOW THE MAGIC TRICK
– Has anyone seen Hans Klok (Our Dutch top Magician) in Las Vegas yet?
– He hired Pamela Anderson as side-kick in his show.

create index EMP_NAME on EMP(ename) invisible;

Index created.

Now test if it is picked up.
SQL> set autotrace on
SQL> select e.ename from emp e where e.ename = ‘BLAKE’
SQL> /

ENAME
———-
BLAKE

Execution Plan
———————————————————-
Plan hash value: 3956160932

Predicate Information (identified by operation id):
—————————————————

1 - filter(”E”.”ENAME”=’BLAKE’)

Statistics
———————————————————-
0 recursive calls
0 db block gets
8 consistent gets
0 physical reads
0 redo size
423 bytes sent via SQL*Net to client
416 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1 rows processed

SQL> select /*+INDEX(e EMP_NAME) */ e.ename from emp e where e.ename = ‘BLAKE’;

ENAME
———-
BLAKE

Execution Plan
———————————————————-
Plan hash value: 2061206800

Predicate Information (identified by operation id):
—————————————————

1 - access(”E”.”ENAME”=’BLAKE’)

Statistics
———————————————————-
1 recursive calls
0 db block gets
2 consistent gets
0 physical reads
0 redo size
423 bytes sent via SQL*Net to client
416 bytes received via SQL*Net from client
2 SQL*Net roundtrips to/from client
0 sorts (memory)
0 sorts (disk)
1 rows processed

It really works! We get a reduction from 8 to 2 consistent gets!

Handy information from the data dictionary:

SQL> select index_name , visibility from user_indexes;

INDEX_NAME VISIBILITY
—————————— ———-
PK_EMP VISIBLE
EMP_NAME INVISIBLE
PK_DEPT VISIBLE

SQL> alter index EMP_NAME visible;

Index altered.

SQL> select index_name , visibility from user_indexes;

INDEX_NAME VISIBILITY
—————————— ———-
PK_EMP VISIBLE
EMP_NAME VISIBLE
PK_DEPT VISIBLE

#######
Conclusion: In 11g Invisible Indexes seems to work as designed.

You can test adding indexes without impact on current plans!
You can also test the removal of indexes on plans before actually dropping the index!
You can fine tune specific statements when adding one index changes other plans badly.

Blogging about 11g - Part 3 - Flashback Data Archive a.k.a. Total Recall

Yuri van Buren — Tue, 24 Jul 2007 06:13:24 +0000

Flashback Data Archive (track all changes to a record over time).

For flashback data archive you need a tablespace first.
In this demo I have created a tablespace named testarea1

Now I can create a Flashback Archive with:
CREATE FLASHBACK ARCHIVE DEFAULT fla1 TABLESPACE testarea1
QUOTA 10G RETENTION 1 YEAR;

Flashback archive created.

You can Remove the Flashback Archive as well with:
DROP FLASHBACK ARCHIVE fla1;

Flashback archive dropped.

Note that this leaves the tablespace intact.

After re-creating the Flashback Archive you can test with changing the retention time

SQL> alter flashback archive fla1 modify retention 7 days;
alter flashback archive fla1 modify retention 7 days
*
ERROR at line 1:
ORA-55603: Invalid Flashback Archive command

SQL> alter flashback archive fla1 modify retention 7 day;

Flashback archive altered.

As you can see you have to specify Day NOT Days Not very intuitive.

Now lets query the dictionary to check what we have created
SQL> select * from flashback_archive_tablespaces;

FLASHBACK_ARCHIVE_NAME
——————————————————————————————
FLASHBACK_ARCHIVE# TABLESPACE_NAME QUOTA_IN_MB
—————— —————————— —————————————-
FLA1
1 TESTAREA1 10240

1 row selected.

Now set a table in the flashback arhive mode.

select table_name from dba_tables where owner = ‘HR’
/

TABLE_NAME
——————————
EMPLOYEES
JOB_HISTORY
JOBS
REGIONS
LOCATIONS
DEPARTMENTS
COUNTRIES

7 rows selected.

alter table hr.employees flashback archive fla1;

Table altered.

Check which tables are in flachback archive mode.

select * from flashback_archived_tables;

TABLE_NAME OWNER_NAME FLASHBACK_ARCHIVE_NAME
—————————— —————————— ——————————
EMPLOYEES HR FLA1

1 row selected.

SQL> select count(*) from hr.employees;

COUNT(*)
———-
107

1 row selected.

Checked : 19-JUN-2007 20:22:39

SQL> desc hr.employees;
Name Null? Type
———————————————– ——– ——————————–
EMPLOYEE_ID NOT NULL NUMBER(6)
FIRST_NAME VARCHAR2(20)
LAST_NAME NOT NULL VARCHAR2(25)
EMAIL NOT NULL VARCHAR2(25)
PHONE_NUMBER VARCHAR2(20)
HIRE_DATE NOT NULL DATE
JOB_ID NOT NULL VARCHAR2(10)
SALARY NUMBER(8,2)
COMMISSION_PCT NUMBER(2,2)
MANAGER_ID NUMBER(6)
DEPARTMENT_ID NUMBER(4)

SQL> select employee_id, salary from hr.employees;

..
192 4000
193 3900
194 3200
195 2800
196 3100
197 3000
198 2600
199 2600
200 4400
201 13000
202 6000
203 6500
204 10000
205 12000
206 8300

107 rows selected.

SQL> update hr.employees set salary = 5000 where salary = 4000;

1 row updated.

–> 192 5000
193 3900
194 3200
195 2800
196 3100
197 3000
198 2600
199 2600
200 4400
201 13000
202 6000
203 6500
204 10000
205 12000
206 8300

107 rows selected.

Checked : 19-JUN-2007 20:29:52

Prompt now see If I can my data back!
SELECT employee_id, salary from hr.employees AS OF
TIMESTAMP TO_TIMESTAMP (’2007-06-19 20:28:00′, ‘YYYY-MM-DD HH24:MI:SS’);

190 2900
191 2500
==>Cool 192 4000
193 3900
194 3200
195 2800
196 3100
197 3000
198 2600
199 2600
200 4400
201 13000
202 6000
203 6500
204 10000
205 12000
206 8300

107 rows selected.

To really test if Oracle does not get this data from the UNDO tablespace you have to test during a longer period.
I wonder how Oracle has tested a retention time of 10 YEAR? I don’t know. (Did they change the clock on there Server?).

Basically Oracle keeps all the necessary UNDO data in the flashback area.
There is an asynchronous background process called FBDA which does this work.

Check the fla1 area in the dictionary.

select * from dba_extents where tablespace_name = ‘TESTAREA1′
and owner = ‘HR’
/

OWNER
——————————
SEGMENT_NAME
———————————————————————————
PARTITION_NAME SEGMENT_TYPE TABLESPACE_NAME
—————————— —————— ——————————
EXTENT_ID FILE_ID BLOCK_ID BYTES BLOCKS RELATIVE_FNO
———- ———- ———- ———- ———- ————
HR
SYS_FBA_TCRV_IDX_67448
INDEX TESTAREA1
0 6 233 65536 8 6

HR
SYS_FBA_TCRV_67448
HIGH_PART TABLE PARTITION TESTAREA1
0 6 225 65536 8 6

HR
SYS_FBA_HIST_67448
HIGH_PART TABLE PARTITION TESTAREA1
0 6 217 65536 8 6

HR
SYS_FBA_DDL_COLMAP_67448
HIGH_PART TABLE PARTITION TESTAREA1
0 6 209 65536 8 6

SQL> desc hr.SYS_FBA_DDL_COLMAP_67448
Name Null? Type
———————————————– ——– ——————————–
STARTSCN NUMBER
ENDSCN NUMBER
XID RAW(8)
OPERATION VARCHAR2(1)
COLUMN_NAME VARCHAR2(255)
TYPE VARCHAR2(255)
HISTORICAL_COLUMN_NAME VARCHAR2(255)

SQL> desc hr.SYS_FBA_TCRV_67448
Name Null? Type
———————————————– ——– ——————————–
RID ROWID
STARTSCN NUMBER
ENDSCN NUMBER
XID RAW(8)
OP VARCHAR2(1)

SQL> desc hr.SYS_FBA_HIST_67448
Name Null? Type
———————————————– ——– ——————————–
RID VARCHAR2(4000)
STARTSCN NUMBER
ENDSCN NUMBER
XID RAW(8)
OPERATION VARCHAR2(1)
EMPLOYEE_ID NUMBER(6)
FIRST_NAME VARCHAR2(20)
LAST_NAME VARCHAR2(25)
EMAIL VARCHAR2(25)
PHONE_NUMBER VARCHAR2(20)
HIRE_DATE DATE
JOB_ID VARCHAR2(10)
SALARY NUMBER(8,2)
COMMISSION_PCT NUMBER(2,2)
MANAGER_ID NUMBER(6)
DEPARTMENT_ID NUMBER(4)

To disable flashback archiving for the table employee do:
ALTER TABLE hr.employees NO FLASHBACK ARCHIVE;

BEWARE of some very STRICT constraints when using Flashback Data Archive.

DDL Statements are Not Allowed on Tables Enabled for Flashback Data Archive
Using any of the following DDL statements on a table enabled for Flashback Data
Archive causes error ORA-55610:
ALTER TABLE statement that does any of the following:
: Drops, renames, or modifies a column
: Performs partition or subpartition operations
: Converts a LONG column to a LOB column
: Includes an UPGRADE TABLE clause, with or without an INCLUDING DATA clause
DROP TABLE statement
RENAME TABLE statement
TRUNCATE TABLE statement

Lets test:
SQL> truncate table hr.employees;
truncate table hr.employees
*
ERROR at line 1:
ORA-55610: Invalid DDL statement on history-tracked table

and:

SQL> alter table hr.employees drop column commision_pct;
alter table hr.employees drop column commision_pct
*
ERROR at line 1:
ORA-55610: Invalid DDL statement on history-tracked table

Looks pretty protected. But is it really handy?

Especially the perform partition operation worries me.
Does this really mean your big compliancy regulated partitioned table cannot be flashback archived?

#######
Conclusion: In 11g Flashback Data Archive is possible.

It seems to work as designed. If you have the right documentation But investigate the pre-requisites carefully or you really have to call Arnold Schwarzenegger.