Oracle ASM

Automatic Storage Management (ASM) Best Practices OUGN 27.05.2009 Marie-Louise Lindstrøm Service Delivery Engineer Oracle Support ACS

Agenda

• Concepts, Architecture • New Features in Database Release 11g • Tips and Techniques • Examples, Demo

ASM ASM Archit Architect ectur ure e - Overv Overvie iew w

1. Automatic Storage Management Management (ASM) instance

•

Oracle Oracle instance instance that manages manages the diskgroup diskgroup metadata metadata

2. ASM Disk Groups

•

Logical grouping of disks

•

Determines file mirroring options

3. ASM Disks

•

LUNs LUNs presen presented ted to to ASM ASM

4. ASM Files

•

Files that are stored in ASM disk groups are called ASM files, this includes database files

ASM ASM Archit Architect ectur ure e - Overv Overvie iew w

• ASM provides filesystem and volume manager capabilities capabilities built into the Oracle Database kernel • Provides a simple storage management interface interface across across all server server and storage platforms • Discover and associate (adding) disks under ASM management • A disk can be partition of a physical spindle, the entire spindle, spindle, or a LUN in a RAID group set • Logical volumes can be presented to ASM as disks, but not a good practice, since an extra layer is added to IO code path, and thus some level of overhead

Key Benefits of ASM

•

I/O is spread spread evenly evenly across across all available disk drives to prevent hot spots and maximize performance

• ASM eliminates the need for over over provisioning and maximizes maximizes storage resource utilization facilitating database consolidation •

Performs automatic online redistribution after the incremental addition or removal of storage capacity

•

Maintains redundant redundant copies copies of data to provide high high availability, or leverage 3rd party RAID functionality

•

Capable of leveraging leveraging 3rd party multi-pathing technologies

Key Benefits of ASM (cont.)

• An Oracle Database Database can contain ASM and non-ASM non-ASM files. Any new files can be created as ASM files whilst existing existing files can also be migrated to ASM •

RMAN commands enable non-ASM managed files to be relocated to an ASM disk group

•

Enterprise Manager can be used to manage ASM disk and file management activities

• ASM reduces Oracle Oracle Database cost and complexity without compromising performance or availability

Software Stack With and Without ASM

APPL1 APPL2 APPL3

APPL1 APPL2 APPL3 DB1

DB2 DB1

File System

DB2

…

Logical Volume Manager

ASM

OPERATING SYSTEM

OPERATING SYSTEM

Disk Group Dynamic Rebalancing

• Automatic online rebalancing whenever storage configuration changes •

Only move data proportional to storage added

•

No need for manual I/O tuning

•

Online migration to new storage

Mirror Mirrorin ing g And Strip Stripin ing g 

Mirroring : Redundant copies of data over multiple disks Primary data extents Secondary data extents



Striping : Spreading data over multiple disks

Unstriped Disks

Striped Disks

ASM Instance

•

Oracle instance to manage the metadata for disk groups

• All metadata modifications are done done by the ASM instance •

Database instances connect to ASM instance to create, delete, resize, open, or close files

•

Database instances read/write directly to the disks

•

One ASM instance is required per node

•

The ASM instance comprises shared shared memory structure (SGA) and background processes

•

The ASM instance contains no physical files other than the password file and the parameter file

•

Instance alert log file and trace trace files in the admin directories (release 10g), 10g), or diag diag dire direct ctor ory y (relea (release se 11g) 11g)

ASM Instance (cont.)

•

Works with the RAC architecture to support clustered storage environments

•

In RAC environments, there will be one ASM instance per clustered node, and the ASM instances communicate with each other on a peer-to-peer basis using the interconnect

•

Generally named +ASM, in a RAC environment +ASM1, +ASM2, etc

ASM Instance (cont.) • • • • • •

• •

instance_type=asm asm_diskstring asm_diskgroups asm_power_limit sga_target or memory_target ; used in ASM release 11.1 db_cache_size=64M ; The buffer cache area is used to cache metadata blocks shared_pool=12M ; Used for standard memory usage (control structures, etc.) to manage the instance large_pool=128M ; Used to store extent maps processes

•

Background processes

•

• • •

RBAL – This process process perform performs s global opens of all all the disks disks in the disk disk groups, groups, manages rebalancing operations ASMB – Periodic messages are exchanged exchanged to update statistics and provide a heartbeat mechanism O00x – A group of slave processes establish establish connections to the ASM instance

ASM Diskgroups

• •

Highest-level data structure in ASM Compar Comparabl able e to a LVM’s LVM’s volume volume group group

SQL> create diskgroup disk ‘’,’’; •

When the diskgroup has been created, metadata information, information, which includes creation creation date, date, diskgroup diskgroup name, and redundancy type, is stored in the SGA and on each disk (in disk header) within the diskgroup • A disk and a database database file can only be part of one diskgroup diskgroup • A database can store store its files in multiple diskgroups diskgroups • Recommended to have two diskgroups: di skgroups: • Database area • Flash recovery area

ASM Disks

•

Unit of persistent storage for a disk group

• Accessed through through normal OS interfaces interfaces •

Oracle must have read/write access

•

Must be accessible to all nodes of a cluster

•

May have different names on different nodes

•

Cannot Cannot be a local local filesys filesystem tem file

•

Must not include partition table

•

Unit that fails if using redundancy

•

Disks in a disk group are the same percent full



Best practice is to have disks of same capacity (size, throughput) belonging to the same diskgroup

• ASM must scale to support support at least 10,000 disks disks in a disk group

Allocation Unit

ASM Disk

1 MB

• ASM disks are divided divided into allocation units •

Unit of allocating storage to a file

•

1 MB in size ; release 10g

•

1, 2, 4, 8, 16, 32, 64 MB ; release 11g

•

Small enough to not be hot

•

Large enough for efficient sequential access

•

Many allocation units in a single file

Redundancy and Failure Groups

• • •

External redundancy • No mirroring, no failure groups Normal redundancy ; default • Files are two-way mirrored, requiring at least two failure groups High redundancy • Using three-way mirroring, requiring at least three failure groups

• Redundancy level set at creation time • A failure group, a subset subset of a diskgroup, is a collection collection of disks that can become unavailable due to a failure of one of its associated components; e.g., controllers or entire arrays • Disks in two two separate failure groups (for the same diskgroup) must must not share a common failure component • The primary extent and its mirror copy are always stored in separate failure groups

Redundancy Levels

External redundancy

•

data exists only once

Normal redundancy

•

two-way mirroring

•

one additional extent copy

•

at least two failure groups

High redundancy

•

three-way mirroring

•

two additional copies

•

at least three failure groups

ASM Files

•

Datafiles, control control files, spfiles, and redo log files, incl archived archived log files, can be stored in ASM diskgroups • Rman backups • ASM file names are in the format format of

+diskgroup_name/database_name/file_type/tag_name.file_n umber.incarnation • ASM Alias name format: instead instead of a file and incarnation number, number, a user-friendly string name is used • For best practices, implement the Oracle Managed File (OMF) feature •

Log files, trace files and executables are stored outside ASM diskgroups

ASM Rebalance and Redistribution

• ASM allocates space in units called allocation units (AU) (AU) • A database file is broken up into file extents (coarse (coarse and fine distribution) • Changing the storage configuration, configuration, for example adding a disk to a diskgroup, will trigger a rebalance operation • The objective of the rebalance operation is to provide an even distribution of file extents and space space usage across all disks in the diskgroup • Rebalancing involves physical movement of file extents  If

removing or adding several disks, disks, it is best best to add or or remove drives all at once, this will reduce the number rebalance operations that are needed for storage changes

ASM Views

• V$ASM_DISK • V$ASM_DISKGROUP • V$ASM_DISK_STAT • V$ASM_DISKGROUP_STAT • V$ASM_OPERATION • V$ASM_CLIENT • V$ASM_FILE • V$ASM_ALIAS • V$ASM_TEMPLATE

ASMCMD Utility •

Command line interface providing access to the files and directories within the ASM diskgroups

$ export ORACLE_SID=+ASM $ asmcmd ASMCMD> ASMCMD > lsdg State Type Reb Re bal Se Sec ctor Blo lock ck AU Usable_file_MB Offline_disks Name MOUNTED MOUNTE D NORMAL N 512 4096 1048576 111 0 DATA1/ MOUNTED MOUNTE D EXTERN N 512 4096 1048576 885 0 FRA1/ ASMCMD> ASMCMD > cd data1 ASMCMD> ASMCMD > ls X1020/ X1110/ ASMCMD> ASMCMD > cd X1110/ ASMCMD> ASMCMD > ls CONTROLFILE/ DATAFILE/ ONLINELOG/ PARAMETERFILE/ TEMPFILE/ spfileX1110.ora ASMCMD> ASMCMD > ls DATAFILE DATAFILE/ / SYSAUX.260.687886011 SYSTEM.261.687886193

Total al_ _MB

Free ee_ _MB

Req_ q_m mir_free_ e_M MB

8188

2269

2047

2047

885

0

Migrating to ASM

• ASM migration using Data Guard physical standby • Use this method if your requirement requirement is to minimize downtime during the migration. It is possible to reduce total downtime to just seconds by using the best practices described in the white paper • ASM migration using Rman • A simpler approach, but but one that can result in downtime downtime measured in minutes to hours, depending on the t he method used for migration • ASM migration using DBMS_FILE_TRANSFER

Copying Copying Files Files To and and From From ASM ASM Diskgro Diskgroups ups

•

Across ASM diskgroups diskgroups

•

Between the OS file system and ASM diskgroups

1.

Using Rman

2.

Using the DBMS_FILE_TRANSFER package

3. 4.

Through ftp to the XML DB Virtual Folder Using the ASMCMD command cp in release 11g

ASM New Features in Release 11g

• ASM Fast Mirror Resync Resync • ASM Preferred Mirror Read Read • ASM Rolling Upgrades and and Patches • ASM Scalability and Performance Performance Enhancements • Multiple Allocation Unit Size • SYSASM Privilege • ASMCMD Extensions Extensions  To

use the the new diskgroup features, the disk group compatibility attribute must be set to 11.1

Without ASM Fast Mirror Resync 1

ASM redundancy used

2

Disk access failure

Oracle Database 10g and 11g

4

Disk added back: Extents rebalanced

3

Disk automatically dropped: All dropped extents re-created

With ASM Fast Mirror Resync ASM redundancy used

1

2

Disk access failure

Oracle Database 11g

Disk again accessible: Auto resync modified extents 4

3

Failure time < DISK_REPAIR_TIME> Access other extents

ASM Fast Mirror Resync

• Define DISK_REPAIR_TIME ALTER DISKGROUP DISKGROUP dgroupA SET dgroupA SET ATTRIBUTE 'DISK_REPAIR_TIME'='3H'; 'DISK_REPAIR_TIME'='3H'; ALTER DISKGROUP DISKGROUP dgroupA dgroupA OFFLINE OFFLINE DISKS IN FAILGROUP contrl2 DROP AFTER 5H; 5H;

• Check time left before ASM drops offlined disk SELECT REPAIR_TIMER FROM V$ASM_DISK;

• Online disk after completing maintenance ALTER DISKGROUP DISKGROUP dgroupA dgroupA ONLINE ONLINE DISKS IN FAILGROUP contrl2 POWER 2 WAIT; WAIT;

ASM Preferred Mirror Read • Allow local mirror read operations operations • Eliminate network latencies in extended clusters Site A

Site B

Extended Cluster

S

P ASM_PREFERRED_READ_ FAILURE_GROUPS=DATA .SITEA

ASM_PREFERRED_READ_ FAILURE_GROUPS=DATA .SITEB

SELECT preferred_read FROM v$asm_disk; SELECT * FROM v$asm_disk_iostat;

ASM Rolling Upgrades and Patches

• Maximizes database availability in a cluster • How it works • Place cluste clusterr in ‘Rolling ‘Rolling Migration Migration’’ mode • Bring down ASM on a cluster node • Upgrade or patch software • Restart ASM • Stop ‘Rolling ‘Rolling Migration Migration’’ mode after after upgrading upgrading all nodes nodes

ASM Scalability and Performance Enhancements • Variable size extents • Grows automatically with file size • Benefits

• 63 disk groups • 10,000 ASM disk isks • 4 petabyt byte per ASM disk • 40 ex exabyte of st storage

• Increase ASM file size

• 1 mil millio lion n fil files es per per dis disk k gro group up

• Reduce memory utilization in SGA

• Max Maximu imum file file siz size:

• 100% automatic

• External redundancy: 140 PB • Normal redundancy: 42 PB • High redundancy: 15 PB

SYSASM Privilege • SYSASM role to manage ASM instances instances separates separates duty of of DBAs and storage administrators SQL> CONNECT / AS SYSASM SQL> CREATE USER ossysasmusername IDENTIFIED by passwd; SQL> GRANT SYSASM TO ossysasmusername ; SQL> CONNECT ossysasmusername / passwd AS AS SYSASM; SQL> DROP USER ossysasmusername ;

• SYSDBA will be deprecated for ASM management: • Oracle Database 11g Release 1 behaves as in 10g • In future releases SYSDBA privileges restricted in ASM instances

Multiple Allocation Unit Size

• Allocation Unit (AU) selected selected at disk group creation creation time and may be 1,2,4,8,16,32,64 MB • Set Oracle MAX_IO_SIZE = AU size • Default MAX_IO_SIZE is 1MB • MAX_IO_SIZE determines default AU size • Striping • coarse stripe size = 1 AU • fine stripe size = 128KB  Higher  Better

performance for large sequential I/O (DW)

leverage of Hardware RAID read-ahead

ASMCMD Extensions

User created directories Templates Disk group compatibility Disk group name Disk names and failure groups

repair

Repair Bad Blocks

md_backup md_restore

$ as asmcmd help

cp

Copy files lsdsk

List ASM disks

Tips Tips and and Techni Techniques ques – Best Practice Practices s •

Should ASM be installed in a separate Oracle_Home?

Yes, it is recommended to install and run ASM in a separate Oracle_Home. Use the latest RDBMS version, currently version 11.1.0.7. For release 10g, use version 10.2.0.4 plus patch for defect 6163771. •

Where do I run the database listener from, the ASM Oracle_Home or the Database Oracle_Home?

It is recommended to run the listener from the ASM Oracle_Home, or a separate Oracle_Home. This is particularly important for RAC installations since the listener is a node-level resource.

Tips Tips and Techni Techniques ques – Best Practice Practices s (cont.) (cont.) •

How to separate ASM administrators from DBAs?

Choose a different OSDBA group for the ASM installation (asmdba) than for the Database installation (dba). The Database instance connects to ASM using OSDBA group of the ASM instance, which means that the software owner for each Database instance connectin connecting g to ASM ASM must must be a member member of ASM's OSDBA group. group. In 11g, ASM administrator has to be member of a separate SYSASM group to separate ASM administrators and DBAs.


Can the Database instance and the ASM instance run different versions?

Yes, the ASM instance can be at a higher version or a lower version than its client databases. d atabases. There are two components of compatibility: 1. Software compatibility • The compatible instance parameter 2. Diskgroup Diskgroup compatibil compatibility ity attributes attributes:: • compatible.asm

•

compatible.rdbms


What instance parameters need to be configured for ASM instances?

The default parameter settings work perfectly for ASM release 11g. The only parameters needed for ASM release 11g:

•

•

instance_type

•

asm_diskstring

•

asm_diskgroups

Does the FILESYSTEMIO_OPTIONS parameter need to be configured in the Database instance when using ASM?

No, the Database instance does I/O directly to the raw disk devices, whereas the FILESYSTEMIO_OPTIONS parameter is only for file systems


How many diskgroups should I have?

The general recommendation is two diskgroups: 1. Database area 2. Flash recovery area •

What redundancy level should be used?

Use normal redundancy level, or high redundancy level, if the underlying storage system does not provide mirroring. You can rely on external redundancy level if the underlying storage system provides mirroring and you determine that failgroups are not needed.


What size would be a good allocation unit (AU) for disk groups?

In 10g, the allocation unit size is, by default, set to 1 MB. In 11g, it is recommended to use 4 MB. •

What is the best LUN size for ASM?

There is no generic best size, in most cases the storage team will dictate based on their standardized LUN size. Communicate the ASM Best Practices and application characteristics to the storage team:  Need equally equally sized / perfo performanc rmance e LUNs • Do not use use LUNs larger larger than 2 TB (minimum (minimum of 4 MB) • The capacity requirement • The workload characteristic (random (random r/w, sequential r/w) & any response time SLA


Would itit be better to use use BIGFILE tablespaces tablespaces,, or default default (SMALLFILE) (SMALLFILE) tablespace tablespaces s for ASM?

The use of BIGFILE BIGFILE tablespaces has no bearing on ASM, or vice versa. In fact, most database object related decisions are transparent to ASM. •

How to backup the ASM instance?

All the metadata the ASM instance instance needs to startup is on on-disk structures (in the disk headers). There are no datafiles or control files that need to be backed up. The only file you can consider to back up is the instance parameter file. In 11g, use the ASMCMD md_backup to take a copy of the metadata.


When should I use RMAN and when should I use ASMCMD copy?

RMAN is the recommended and most complete and flexible method to backup and transport database files in ASM. ASMCMD cp is good for copying single files •

Supports all Oracle file types

•

Can be used to instantiate a Data Guard environment

•

Does not update update the controlfile controlfile

•

Does not create OMF files


Is it possible to unplug an an ASM disk group group from one platform and plug into a server on another platform, for example from Solaris to Linux)?

No, cross-platform disk group migration is not supported. To move datafiles datafiles between between endian-nes endian-ness s platform platforms, s, you you need need to use XTTS, XTTS, Datap Datapump ump or Strea Streams. ms. •

How does does ASM work work with multi multipat pathin hing g softwar software? e?

Oracle does does not certify certify or qualify multi-pathing tools. ASM does, does, however, however, leverage leverage multi-pa multi-pathing thing tools. tools. Use multimulti-pathin pathing g if the underlying underlying storage storage suppor supports ts it. Multipathin Multipathing g software is at a layer lower than ASM, and thus transparent to ASM. You may need to adjust the ASM_DISKSTRING parameter to specify only only the path path to the multip multipath athing ing pseud pseudo o device devices. s.


AIX specific information: Do not assign a PVID to ASM disks

Assigning a Physical Physical Volume ID (PVID) to an existing existing ASM disk will destroy the ASM disk header rendering the ASM disk unusable. •

How to repair damaged disk headers?

If you encounter an error situation caused by corrupt disk headers leading to invalid ASM metadata, open a service request with Oracle Support to get technical assistance from the ASM team. Depending on the extent of the damage, it may be possible to manually repair the disk headers. However, if internal structures cannot be patched, the only solution will be to recreate the diskgroup and restore data from backup. Make sure you always have a good backup of your data. Also, store backup copies outside ASM diskgroups.


Should Should ASMLib ASMLib be used used on Linux Linux installations installations? ?

Yes, this is the the recomm recommendat endation. ion. The The ASMLib ASMLib interface interface is not required to run ASM, however, it provides easier management of disk discovery and provisioning. Set the ASM_DISKSTRING parameter to ”ORCL:”.

References • Oracle® Database Storage Storage Administrator's Administrator's Guide 11g Release 1 (11.1) B31107-05 http://download.oracle.com/docs/cd/B28359_01/server.111/b31 107.pdf

• ASM Collateral and Content http://www.oracle.com/technology/asm

• Best Practices for Minimal Downtime Migration to ASM Oracle 10g Release 2 http://www.oracle.com/technology/deploy/availability/pdf/ma a_wp_10gr2_asmmigrationwithdg.pdf

References (cont.) • ASM and Multipathing Generic Best Practices Practices and Information Matrix Matrix March 2008 http://www.oracle.com/technology/products/database/asm/pdf/ asm%20and%20multipathing%20best%20practices%20info%20matrix %203-5-08.pdf

• MyOracleS MyOracleSuppor upportt Note:2656 Note:265633.1 33.1 - ASM Tech Technical nical Best Practices Practices • MyOracleS MyOracleSuppor upportt Note:7513 Note:751300.1 00.1 - How to perform perform Rolling Rolling Upgrade/Downgrade Upgrade/Downgrade in 11g ASM

Oracle ASM

Recommend Documents