Liang's Blog

Happy New Year 2012

lianggang — Wed, 04 Jan 2012 06:40:42 +0000

Wrapping up the first working day in 2012, I want to say “Happy 2012″ to everyone. Looking back in 2011, I am quite happy with the projects completed, new territory explored, books read, and more exercise I made. In 2012, ever changing world would accelerate its pace. This blog would change too, I would blog more, technology, life, and thought.

Partitioning exchange between compressed and non-compressed tables with and without indexes

lianggang — Tue, 08 Nov 2011 09:00:40 +0000

It might come to the time that you want to compress a large partitioned table one partition a time. Without indexes, you can exchange compressed standalone table into a non-compressed table; and Vice Versa. With indexes, you’d be careful about indexes compression attribute. If any index on either partitioned table or stand-alone table is compressed, you’d also be sure that indexes on the partner is compressed too. Moreover, you’d be sure the prefix degree of compress should be the same on both tables.

Preparation:
create table liang_t1 ( cyear number, OWNER VARCHAR2(30), OBJECT_NAME VARCHAR2(128), SUBOBJECT_NAME VARCHAR2(30), OBJECT_ID NUMBER )
partition by range (cyear)
( partition p2010 values less than (2011),
partition p2011 values less than (2012));

insert into liang_t1 select ’2010′, owner, object_name, subobject_name, object_id from dba_objects;
insert into liang_t1 select ’2011′, owner, object_name, subobject_name, object_id from dba_objects;
Commit;

Case 1: without indexes, you can exchange compressed standalone table into a non-compressed table.

select table_name, partitioned, compression, compress_for from dba_tables where table_name = ‘LIANG_T1′;

TABLE_NAME PAR COMPRESS COMPRESS_FOR
—————————— — ——– ————
LIANG_T1 YES

select table_name, partition_name, compression, compress_for from dba_tab_partitions where table_name = ‘LIANG_T1′;

TABLE_NAME PARTITION_NAME COMPRESS COMPRESS_FOR
—————————— —————————— ——– ————
LIANG_T1 P2010 DISABLED
LIANG_T1 P2011 DISABLED

create table liang_t1_comp compress for oltp as select * from liang_t1 partition (p2010) order by owner, object_name;

select table_name, partitioned, compression, compress_for from dba_tables where table_name = ‘LIANG_T1_COMP’;

TABLE_NAME PAR COMPRESS COMPRESS_FOR
—————————— — ——– ————
LIANG_T1_COMP NO ENABLED OLTP

alter table liang_t1 exchange partition p2010 with table liang_t1_comp with validation;

Vola, you get a compressed partition into a non-compressed table.

select table_name, partitioned, compression, compress_for from dba_tables where table_name = ‘LIANG_T1_COMP’;

TABLE_NAME PAR COMPRESS COMPRESS_FOR
—————————— — ——– ————
LIANG_T1_COMP NO DISABLED

select table_name, partition_name, compression, compress_for from dba_tab_partitions where table_name = ‘LIANG_T1′;

TABLE_NAME PARTITION_NAME COMPRESS COMPRESS_FOR
—————————— —————————— ——– ————
LIANG_T1 P2010 ENABLED OLTP
LIANG_T1 P2011 DISABLED

You can exchange a compressed partition out with non-compressed table.

alter table liang_t1 exchange partition p2010 with table liang_t1_comp with validation;

Case 2: Partition exchange fails if one of exchange partner has compression indexes and the other does not.

create index idx_liang_t1_first3 on liang_t1 (cyear, owner, OBJECT_NAME) local;
select index_name, partition_name, compression from dba_ind_partitions where index_name = ‘IDX_LIANG_T1_FIRST3′;

INDEX_NAME PARTITION_NAME COMPRESS
—————————— —————————— ——–
IDX_LIANG_T1_FIRST3 P2010 DISABLED

IDX_LIANG_T1_FIRST3 P2011 DISABLED

create index idx_liang_t1_comp_first3 on liang_t1_comp (cyear, owner, OBJECT_NAME) compress 1;

select index_name, compression, prefix_length from dba_indexes where index_name = ‘IDX_LIANG_T1_COMP_FIRST3′;

INDEX_NAME COMPRESS PREFIX_LENGTH
—————————— ——– ————-
IDX_LIANG_T1_COMP_FIRST3 ENABLED 1

alter table liang_t1 exchange partition p2010 with table liang_t1_comp including indexes with validation;

alter table liang_t1 exchange partition p2010 with table liang_t1_comp including indexes with validation
*
ERROR at line 1:
ORA-28665: table and partition must have same compression attribute

Case 3: Partition exchange fails if one of exchange partner has compression indexes with different prefix degree.

drop index idx_liang_t1_first3;
create index idx_liang_t1_first3 on liang_t1 (cyear, owner, OBJECT_NAME) compress 1 local;

drop index idx_liang_t1_comp_first3;
create index idx_liang_t1_comp_first3 on liang_t1_comp (cyear, owner, OBJECT_NAME) compress 2;

alter table liang_t1 exchange partition p2010 with table liang_t1_comp including indexes with validation;
alter table liang_t1 exchange partition p2010 with table liang_t1_comp including indexes with validation
*
ERROR at line 1:
ORA-28665: table and partition must have same compression attribute

Case 4: Only both indexes are compressed, partition exchange “including indexes with validation” would succeed.

drop index idx_liang_t1_first3;
create index idx_liang_t1_first3 on liang_t1 (cyear, owner, OBJECT_NAME) compress 1 local;

select index_name, compression, prefix_length from dba_indexes where index_name = ‘IDX_LIANG_T1_FIRST3′;

INDEX_NAME COMPRESS PREFIX_LENGTH
—————————— ——– ————-
IDX_LIANG_T1_FIRST3 ENABLED 1

drop index idx_liang_t1_comp_first3;

create index idx_liang_t1_comp_first3 on liang_t1_comp (cyear, owner, OBJECT_NAME) compress 1;

alter table liang_t1 exchange partition p2010 with table liang_t1_comp including indexes with validation;

Table altered.

Tiny tip – “clear logfile group” to create redo logs on Standby

lianggang — Fri, 03 Dec 2010 22:41:55 +0000

If the redo log files are missing from Standby, e.g. creating a standby from backup, you can just do “clear logfile group”.

On standby DB, if you clear logfile group without log_file_name_convert being set, you will see
SQL> alter database clear logfile group 1;
alter database clear logfile group 1
*
ERROR at line 1:
ORA-19527: physical standby redo log must be renamed
ORA-00312: online log 1 thread 1:
‘/path/filename’

With the right log_file_name_convert setting, you will see

SQL> alter database clear logfile group 1;
Database altered.

Data files will be created on standby by Oracle.

Using bulk binds for fewer redo records

lianggang — Wed, 20 Oct 2010 16:47:14 +0000

Redo logs are made of redo records; each redo record is composed by redo change vectors, each of which describes a DB block level change.

Insert with bulk binds, aka bulk insert, can boost your program’s performance by processing many rows from a cursor in one call instead of one row at a time. It also generates fewer redo change vectors therefore less amount of redo logs records.

create table t1 (
a number,
b varchar2(10)
);

[Case 1] Insert one row at a time.

DECLARE
v_Counter BINARY_INTEGER := 1;
BEGIN
WHILE v_Counter <= 10 LOOP
INSERT INTO t1 (a, b) VALUES (v_Counter, v_counter);
v_Counter := v_Counter + 1;
END LOOP;
END;
/
commit;

I dump the redo logs about the changes, after finding redo log name and DBA information.
ALTER SYSTEM DUMP LOGFILE '/…/redolog-1554-1-1232324.arc' DBA MIN 1 39898 DBA MAX 1 39898;

What I see is 10 redo change records. 1 redo record per insert. I paste 1 of 10 here.

REDO RECORD – Thread:1 RBA: 0×000612.0000014d.0140 LEN: 0x016c VLD: 0×01
…
CHANGE #1 TYP:0 CLS:19 AFN:2 DBA:0×00800099 OBJ:4294967295 SCN:0×0000.00d3ccef SEQ: 1 OP:5.2 ENC:0
…
CHANGE #2 TYP:0 CLS:20 AFN:2 DBA:0x0081b92d OBJ:4294967295 SCN:0×0000.00d3ccee SEQ: 3 OP:5.1 ENC:0
…
Tablespace Undo: No
0×00000000 prev ctl uba: 0x0081b92d.03bb.07
..
CHANGE #3 TYP:0 CLS: 1 AFN:1 DBA:0x00409bda OBJ:21674 SCN:0×0000.00d3cd01 SEQ: 3 OP:11.2 ENC:0
KTB Redo
…
null: –
col 0: [ 2] c1 02
col 1: [ 1] 31

[Case 2] Bulk insert with 10 rows at a time. To get an individual redo log, I switch logfile first.

declare
type testnumberarray is table of number index by binary_integer;
type testchararray is table of varchar2(10) index by binary_integer;

v_a testnumberarray;
v_b testchararray;

cursor cu_cursor is select a,b from t1_copy; — (t1_copy has 10 records.)
begin
open cu_cursor;

loop
fetch cu_cursor bulk collect into v_a, v_b LIMIT 10;

forall i in 1 .. v_a.count
insert into t1
values
( v_a(i), v_b(i) );

exit when cu_cursor%notfound;
end loop;
close cu_cursor;
end;
/

Then I dump the new redo log for the change to blocks.

REDO RECORD – Thread:1 RBA: 0x00061b.00000043.0140 LEN: 0x01dc VLD: 0×01
…
CHANGE #1 TYP:0 CLS:23 AFN:2 DBA:0x008000b9 OBJ:4294967295 SCN:0×0000.00d41e1a SEQ: 1 OP:5.2 ENC:0
…
CHANGE #2 TYP:0 CLS:24 AFN:2 DBA:0x0080c891 OBJ:4294967295 SCN:0×0000.00d41e19 SEQ: 1 OP:5.1 ENC:0
…
KTB Redo
…
tabn: 0 lock: 0 nrow: 10
slot[0]: 0
slot[1]: 1
slot[2]: 2
slot[3]: 3
slot[4]: 4
slot[5]: 5
slot[6]: 6
slot[7]: 7
slot[8]: 8
slot[9]: 9
CHANGE #3 TYP:0 CLS: 1 AFN:1 DBA:0x00409bda OBJ:21691 SCN:0×0000.00d41e38 SEQ: 3 OP:11.11 ENC:0
KTB Redo
…
tabn: 0 lock: 1 nrow: 10
slot[0]: 0
tl: 8 fb: –H-FL– lb: 0×0 cc: 2
col 0: [ 2] c1 02
col 1: [ 1] 31
slot[1]: 1
tl: 8 fb: –H-FL– lb: 0×0 cc: 2
col 0: [ 2] c1 03
col 1: [ 1] 32
slot[2]: 2
tl: 8 fb: –H-FL– lb: 0×0 cc: 2
col 0: [ 2] c1 04
col 1: [ 1] 33
slot[3]: 3
tl: 8 fb: –H-FL– lb: 0×0 cc: 2
col 0: [ 2] c1 05
col 1: [ 1] 34
slot[4]: 4
tl: 8 fb: –H-FL– lb: 0×0 cc: 2
col 0: [ 2] c1 06
col 1: [ 1] 35
slot[5]: 5
tl: 8 fb: –H-FL– lb: 0×0 cc: 2
col 0: [ 2] c1 07
col 1: [ 1] 36
slot[6]: 6
tl: 8 fb: –H-FL– lb: 0×0 cc: 2
col 0: [ 2] c1 08
col 1: [ 1] 37
slot[7]: 7
tl: 8 fb: –H-FL– lb: 0×0 cc: 2
col 0: [ 2] c1 09
col 1: [ 1] 38
slot[8]: 8
tl: 8 fb: –H-FL– lb: 0×0 cc: 2
col 0: [ 2] c1 0a
col 1: [ 1] 39
slot[9]: 9
tl: 9 fb: –H-FL– lb: 0×0 cc: 2
col 0: [ 2] c1 0b
col 1: [ 2] 31 30
END OF REDO DUMP

With bulk binds, Oracle generates 1 redo change records for all 10 rows, since only 1 insert sql is fired. As an result, given the same data to insert, with bulk binds generates much less amount of redo logs.

My bloginality – INTP

lianggang — Fri, 15 Oct 2010 23:39:41 +0000

As an INTP, you are Introverted, iNtuitive, Thinking, Perceiving.
This makes your primary focus on Introverted Thinking with an Extraverted Intution.

This is defined as a NT personality, which is part of Carl Jung’s Rational (Knowledge Seeking) type, and more specifically the Architect or Thinker.

As a weblogger, you might not be as concerned about popularity, but more with the ideas and theories that you strive to understand. Because routines aren’t your strong point, you might be more likely to work on the concept of how to do a blog, but not be as excited to keep it up.

http://bloginality.love-productions.com/index2.php

So it is said. …

Netflix sharing “Availabile-Network Partition Tolerant” Storage practice

lianggang — Fri, 15 Oct 2010 07:04:46 +0000

In May, I read the news that Netflix had been using Amazon Web Service for a year. Now, in October, Sid Anand from Netflix revealed “Netflix’s transition to high-availability storage system” practice. The paper has 2 attractive sections to me.
1. eBay’s best practice to achieve “Availabile-Network Partition Tolerant” (AP) with Oracle.
2. Netflix’s Best practice of “leaving RDBMS behind” and on SimpleDB.

Good stuff.

Load hinted SQL execution plan into SQL Plan Baselines – 11g

lianggang — Wed, 13 Oct 2010 08:32:00 +0000

In 11g, I like 11g’s SQL plan management feature a lot. It comes handy when I need a good execution plan to correct CBO misbehavior. I can just load existing better SQL execution plans from cache, awr, SQL profile into SQL Plan Baseline if they exist, accept them, fix them if needed, instruct Oracle sessions to use them.

It is not always the case that an ideal execution plan already exists somewhere in database. During a production firefight, I like to tune the problematic query with SQL hints, get a good execution plan, and then use this good plan for the problematic query. It is tricky because original query and tuned query with hints are different SQL, having different SQL ID e.g. In 9i and 10g, we tweak stored outlines to achieve; in 11g, it is actually more straightforward using dbms_spm. This is how to do it.

1. Get the sql_handle for original SQL.
If no baseline exists before, you create one for the original SQL from cache.

var v_out number ;
exec :v_out:= dbms_spm.load_plans_from_cursor_cache(
sql_id => ‘&original_sql_id’,
plan_hash_value => ‘&original_plan_hash_value’ );

select sql_text, sql_handle, plan_name, enabled, accepted from dba_sql_plan_baselines;

2. Tune the original query using right hints. Execute hinted SQL. Find SQL_ID and PLAN_HASH_VALUE for the hinted SQL statements.
Validate the plan as an efficient plan.

3. Associate the hinted execution plan to the original sql_handle.

var v_out number
exec :v_out := dbms_spm.load_plans_from_cursor_cache(
sql_id => ‘&HINTED_sql_id’,
plan_hash_value => ‘&HINTED_plan_hash_value’,
sql_handle => ‘&sql_handle_for_ORIGINAL_sql’);

select sql_text, sql_handle, plan_name, enabled, accepted from dba_sql_plan_baselines;

4. Check now, you can see 2 execution plans for the same SQL. Drop the old, inefficient plans.

var v_out number
exec :v_out :=DBMS_SPM.DROP_SQL_PLAN_BASELINE (‘&original_sql_handle’,'&original_plan_name’);

Done. Please check out a demo in the comments.

As I know, people still tweak stored outlines with hints to get a sql execution plan needed. Here is a good post about tweaking stored outline from Paul Tabashov. In 11g release 2, you can keep playing with stored outline, at the same time, use sql plan baseline in action. You just need to load tweaked stored outlines into SQL baselines using dbms_spm.migrate_stored_outline.

http://download.oracle.com/docs/cd/E11882_01/appdev.112/e16760/d_spm.htm#ARPLS72437

Manual SQL execution plan loading into SQL baseline – part 2 – from cursor cache

lianggang — Fri, 21 May 2010 06:16:58 +0000

Manual SQL execution plan loading into SQL baseline from cursor cache

If an execution plan is in cursor cache, you can to load it into sql plan baseline using dbms_spm.load_plans_from_cursor_cache.

You can load a sql plan associated with sql id.
SQL>
set serveroutput on
declare
v_int pls_integer;
begin
v_int := dbms_spm.load_plans_from_cursor_cache (
sql_id => ‘&sql_id’,
plan_hash_value => ‘&sql_plan_hash_value’,
fixed => ‘NO’,
enabled => ‘YES’);
DBMS_OUTPUT.PUT_line(v_int);
end;
/
or if you prefer loading plans of matching SQL_TEXT,
SQL>
set serveroutput on
declare
v_int pls_integer;
begin
v_int := dbms_spm.load_plans_from_cursor_cache (
attribute_name => ‘SQL_TEXT’,
attribute_value => ‘SELECT%FROM ABC%’,
fixed => ‘NO’,
enabled => ‘YES’);
DBMS_OUTPUT.PUT_line(v_int);
end;
/

Next step is to check whether a SQL plan is in use.
SQL>
select sql_id, plan_hash_value, sql_plan_baseline,executions, sql_text
from v$sql
where sql_id = ‘&sql_id’;
or
SQL>
select sql_handle, plan_name, enabled, accepted, fixed, origin,
to_char(created, ‘YYYY/MM/DD HH24:MI:SS’) created
from dba_sql_plan_baselines sqlbase, v$sql sql
where sql.sql_id = ‘&sql_id’ and sqlbase.plan_name = sql.sql_plan_baseline;

If there are multiple plans in baseline as accepted, you can instruct Optimizer always use a plan by marking its attribute FIXED to “YES”. When Optimizer see a fixed plan, it will choose it and ignore other accepted plans even if they might have lower cost. If there are more than one fixed plan, optimizer will choose among them.

Here is how.
SQL>
declare
v_int binary_integer;
begin
v_int := dbms_spm.alter_sql_plan_baseline(
sql_handle =>’&sql_handle’,
plan_name=>’&plan_name’,
attribute_name=>’fixed’,
attribute_value=>’YES’);
DBMS_OUTPUT.PUT_line(v_int);
end;
/

Manual SQL execution plan loading into SQL baseline – part 1 – from AWR or SQL Tuning Set

lianggang — Thu, 20 May 2010 03:56:40 +0000

Manual SQL execution plan loading into SQL baseline from AWR or SQL Tuning Set

In 11g, manually loading existing SQL plans as SQL plan baseline can help stabilize sql query performance. Existing plan can be loaded from AWR/SQL tuning set or from Cursor cache. Here let us load plans from AWR and SQL tuning set; later, we visit how to load from cursor cache.

If a SQL tuning set with a good execution plan already exists, we can skip step 1 and load plans into SQL baseline using DBMS_SPM.LOAD_PLANS_FROM_SQLSET function. If we have to load a execution plan in AWR, we need first create a SQL tuning set for the query and then load it into SQL baseline.

Step 1: Create SQL tuning set (SQLSET) from AWR.
SQL> exec dbms_sqltune.create_sqlset(
sqlset_name => ‘sql_tuning_set_name’,
description => ‘sqlset descriptions’);

SQL>
declare
ref_cur DBMS_SQLTUNE.SQLSET_CURSOR;
begin
open ref_cur for
select VALUE(p) from table(
DBMS_SQLTUNE.SELECT_WORKLOAD_REPOSITORY( &begin_snap_id, &end_snap_id,
basic_filter=>’sql_id=”&sql_id”’,
attribute_list =>’ALL’)) p;
DBMS_SQLTUNE.LOAD_SQLSET(‘sql_tuning_set_name’, ref_cur);
end;
/

basic_filter is a SQL predicate to filter the SQL from the workload repository defined on attributes of the SQLSET_ROW.
We use sql_id and/or plan_hash_value here.

SQL>
declare
ref_cur DBMS_SQLTUNE.SQLSET_CURSOR;
begin
open ref_cur for
select VALUE(p) from table(
DBMS_SQLTUNE.SELECT_WORKLOAD_REPOSITORY( &begin_snap_id, &end_snap_id,
basic_filter=>’sql_id=”&sql_id” and plan_hash_value=”&sql_plan_hash_value”’,
attribute_list =>’ALL’)) p;
DBMS_SQLTUNE.LOAD_SQLSET(‘sql_tuning_set_name’, ref_cur);
end;
/

Check out in SQLSET
SQL>
select * from table(dbms_xplan.display_sqlset(‘sql_tuning_set_name’,'&sql_id’));

Step 2: Load SQL execution plans from SQL set into SQL baseline.

set serveroutput on
declare
v_int pls_integer;
begin
v_int := dbms_spm.load_plans_from_sqlset (
sqlset_name => ‘sql_tuning_set_name’,
sqlset_owner => ‘SYS’,
basic_filter => ‘sql_id=”&sql_id” and plan_hash_value =”&sql_plan_hash_value”’,
fixed => ‘NO’,
enabled => ‘YES’);
DBMS_OUTPUT.PUT_line(v_int);
end;
/

basic_filter is used here to filter out only qualifying plans. It can take the form of any WHERE clause predicate that can specified against the view DBA_SQLSET_STATEMENTS.

For details, please visit

http://download.oracle.com/docs/cd/B19306_01/appdev.102/b14258/d_sqltun.htm#CACJHDDC

http://download-west.oracle.com/docs/cd/B28359_01/appdev.111/b28419/d_spm.htm#CACBCAGD

Things I like or don’t like about iPad

lianggang — Fri, 14 May 2010 08:05:18 +0000

What I like about iPad most,
Kids friendly: Books with color, video, sound, actions, and more. Education games with feedbacks. These are available on PC, but iPad makes it easy for Kids to reach.
Simplicity of UI: Even my 1 year old can flip photos, play simple games on iPad as soon as touch it.
Apple style: Everyone has own take on what Apple style means. Apple wins me big time on consumer electronics, ipod, iphone, ipad, with its simplicity, functionality.

What I think iPad does not do well,
Weight: I’d love to use iPad as a e-reader. But it is too heavy to hold for meaningful long as we can do to a book or a good e-reader.
Openness: App store module is not that friendly. Can you imagine if you write a program on windows and you have to get Microsoft’s okay to sale. Adobe flash is just an example. Internet has been there long before iPad. Given time, flash could become less favorable due to this; but it is not Adobe alone but general flash content providers and consumers feel like not respected. That is a sizable population.
Productivity support: Yes, iPad is for fun, for consuming Internet/photo/movies/music. When it comes to produce documents/edit file, it is painful. Even Internet experience is wedged when use safari. Safari is far less popular to a website when comes to visitor volume currently. As a result, websites might have not even tested on Sarafi and their contents can not be displayed correctly.