Thursday, December 30, 2010

Test storage performance using Orion (Come with 11gR2 Grid Infrastructure)


[grid@vmxdb03 grid]$ orion -help
ORION: ORacle IO Numbers -- Version 11.2.0.2.0

ORION runs IO performance tests that model Oracle RDBMS IO workloads.
It measures the performance of small (2-32K) IOs and large (128K+) IOs
at various load levels.

Each Orion data point is a test for a specific mix of small and large
IO loads sustained for a duration. An Orion test consists of multiple
data point tests. These data point tests can be represented as a
two-dimensional matrix. Each column in the matrix represents data
point tests with the same small IO load, but varying large IO loads.
Each row represents data point tests with the same large IO load, but
varying small IO loads. An Orion test can be for a single point, a
single row, a single column, or the whole matrix.

The 'run' parameter is the only mandatory parameter. Defaults
are indicated for all other parameters. For additional information on
the user interface, see the Orion User Guide.

is the prefix used for all input and output filenames. By
default, it is 'orion'. It can be specified with the 'testname'
parameter.

.lun should contain a carriage-return-separated list of LUNs.

The output files for a test run are prefixed by _ where
date is "yyyymmdd_hhmm".

The output files are:
__summary.txt - Summary of the input parameters, along with
the minimum small IO latency (in usecs), the maximum
MBPS, and the maximum IOPS observed.
__mbps.csv - Performance results of large IOs in MBPS.
__iops.csv - Performance results of small IOs in IOPS.
__lat.csv - Latency of small IOs in microseconds.
__hist.csv - Histogram of IO latencies.
__trace.txt - Extended, unprocessed output.

WARNING: IF YOU ARE PERFORMING WRITE TESTS, BE PREPARED TO LOSE ANY DATA STORED
ON THE LUNS.

Mandatory parameters:
run Type of workload to run (simple, normal, advanced, dss, oltp).
simple - Tests random small (8K) IOs at various loads,
then random large (1M) IOs at various loads.
normal - Tests combinations of random small (8K) IOs and
random large (1M) IOs.
advanced - Tests the workload specified by the user
using optional parameters.
dss - Tests with random large (1M) IOs at increasing
loads to determine the maximum throughput.
oltp - Tests with random small (8K) IOs at increasing
loads to determine the maximum IOPS.

Optional parameters:
testname Name of the test run.

num_disks Number of disks (physical spindles). This number is
used to gauge the range of loads that Orion should test
at. Increasing this parameter results in Orion using
heavier IO loads. Default is the number of LUNs in
.lun.

size_small Size of small IOs in KB. Default is 8.

size_large Size of large IOs in KB. Default is 1024.

type Type of large IOs (rand, seq). Default is rand.
rand - Randomly distributed large IOs.
seq - Sequential streams of large IOs.

num_streamIO Number of concurrent IOs per stream. This parameter is only
used if -type is seq. Default is 4.

simulate Orion tests on a virtual LUN formed by combining the
specified LUNs in one of these ways. This parameter is
typically only used if -type is seq. Default is concat.
concat - A serial concatenation of the LUNs. Each
sequential stream issues IOs to only one LUN.
raid0 - A RAID-0 mapping across the LUNs. Each
sequential stream issues IOs across all LUNs,
using RAID-0 striping.

write Percentage of IOs that are writes (SEE WARNING ABOVE).
Default is 0.

cache_size Size in MBs of the array's cache.
Unless this option is set to 0, Orion issues a number
of unmeasured, random IOs before each large sequential
data point. These IOs fill up the storage array's cache
(if any) with random data so that IOs from one
data point do not result in cache hits for the next
data point. Read tests are preceded with junk reads
and write tests are preceded with junk writes. If
specified, this 'cache warming' is performed until
cache_size MBs of IO have been read or written.
Default behavior is to issue 2 minutes of unmeasured random
IOs before each data point.

duration Duration of each data point in seconds. Default is 60.

num_small Number of outstanding small IOs. This parameter controls the
small IO load. Only used if -matrix is point, col, or max.
No default.

num_large This parameter controls the large IO load.
For -type rand, number of outstanding large IOs.
For -type seq, number of sequential IO streams. Only used
if -matrix is point, row, or max. No default.

matrix An Orion test consists of multiple data point tests. These data
point tests can be represented as a two-dimensional matrix.
Each column in the matrix represents data point tests with
the same small IO load, but varying large IO loads. Each
row represents data point tests with the same large IO load,
but varying small IO loads. An Orion test can be for a
single point, a single row, a single column, or the whole
matrix, depending on the matrix option setting below.
Default is basic.
basic - Test small IOs only, then large IOs only.
detailed - Test entire matrix.
point - Test with num_small small IOs, num_large large
IOs.
col - Test a varying large IO load with num_small
small IOs.
row - Test a varying small IO load with num_large
large IOs.
max - Test varying loads up to the num_small and
num_large limits.

verbose Prints tracing information to standard output if set.
Not set by default.

Examples:
For a preliminary set of data
-run simple
For a basic set of data
-run normal
To evaluate storage for an OLTP database
-run oltp
To evaluate storage for a data warehouse
-run dss
To generate combinations of 32KB and 1MB reads to random locations
-run advanced
-size_small 32 -size_large 1024 -type rand
-matrix detailed
To generate multiple sequential 1MB write streams, simulating RAID0 striping
-run advanced
-simulate RAID0 -write 100 -type seq
-matrix col -num_small 0




[grid@vmxdb03 grid]$ cat iscsi_io.lun
/dev/iscsi/asmdisk2p1
/dev/iscsi/asmdisk3p1
/dev/iscsi/asmdisk4p1
/dev/iscsi/asmdisk5p1


[grid@vmxdb03 grid]$ orion -run simple -testname iscsi_io -num_disks 4
ORION: ORacle IO Numbers -- Version 11.2.0.2.0
iscsi_io_20101218_1854
Calibration will take approximately 30 minutes.
Using a large value for -cache_size may take longer.

ORA-56727: orion must be invoked using its full, absolute path
orion_main: orion_spawn sml failed
Test aborted due to errors.
[grid@vmxdb03 grid]$ $ORACLE_HOME/bin/orion -run simple -testname iscsi_io -num_disks 4
ORION: ORacle IO Numbers -- Version 11.2.0.2.0
iscsi_io_20101218_1854
Calibration will take approximately 30 minutes.
Using a large value for -cache_size may take longer.

[root@vmxdb03 ~]# iostat -x 5 1000
Linux 2.6.18-194.el5 (vmxdb03.lab.dbaglobe.com) 12/18/2010

avg-cpu: %user %nice %system %iowait %steal %idle
1.12 0.00 9.77 45.07 0.00 44.05

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 64.60 0.00 22.00 0.00 694.40 31.56 0.39 17.67 0.91 2.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda2 0.00 64.60 0.00 22.00 0.00 694.40 31.56 0.39 17.67 0.91 2.00
dm-0 0.00 0.00 0.00 86.80 0.00 694.40 8.00 0.86 9.96 0.23 2.00
dm-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
hdc 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sdb 0.00 0.00 0.00 0.80 0.00 2.20 2.75 0.02 26.75 26.75 2.14
sdb1 0.00 0.00 0.00 0.80 0.00 2.20 2.75 0.02 26.75 26.75 2.14
sdc 0.00 0.00 29.80 0.00 476.80 0.00 16.00 0.55 18.41 13.70 40.82
sdc1 0.00 0.00 29.80 0.00 476.80 0.00 16.00 0.55 18.41 13.70 40.82
sdd 0.00 0.00 33.40 0.00 534.40 0.00 16.00 0.59 17.75 13.46 44.94
sdd1 0.00 0.00 33.40 0.00 534.40 0.00 16.00 0.59 17.75 13.46 44.94
sde 0.00 0.00 32.80 0.00 524.80 0.00 16.00 0.58 17.60 13.35 43.80
sde1 0.00 0.00 32.80 0.00 524.80 0.00 16.00 0.58 17.60 13.35 43.80
sdf 0.00 0.00 27.40 0.00 438.40 0.00 16.00 0.49 17.84 14.42 39.52
sdf1 0.00 0.00 27.40 0.00 438.40 0.00 16.00 0.49 17.84 14.42 39.52


avg-cpu: %user %nice %system %iowait %steal %idle
1.42 0.00 9.50 43.98 0.00 45.10

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s avgrq-sz avgqu-sz await svctm %util
sda 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sda2 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
hdc 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
sdb 0.00 0.00 0.00 0.80 0.00 2.20 2.75 0.10 121.75 121.75 9.74
sdb1 0.00 0.00 0.00 0.80 0.00 2.20 2.75 0.10 121.75 121.75 9.74
sdc 0.00 0.00 28.40 0.00 454.40 0.00 16.00 2.21 78.43 24.70 70.16
sdc1 0.00 0.00 28.40 0.00 454.40 0.00 16.00 2.21 78.43 24.70 70.16
sdd 0.00 0.00 31.00 0.00 496.00 0.00 16.00 2.15 68.19 23.50 72.84
sdd1 0.00 0.00 31.00 0.00 496.00 0.00 16.00 2.15 68.19 23.50 72.84
sde 0.00 0.00 39.20 0.00 627.20 0.00 16.00 2.49 63.71 18.80 73.70
sde1 0.00 0.00 39.20 0.00 627.20 0.00 16.00 2.49 63.71 18.80 73.70
sdf 0.00 0.00 36.20 0.00 579.20 0.00 16.00 2.31 63.98 21.02 76.08
sdf1 0.00 0.00 36.20 0.00 579.20 0.00 16.00 2.31 63.98 21.02 76.08


---- OpenFiler ----
[root@openfiler ~]# iostat -x 5 1000 |grep sda5

avg-cpu: %user %nice %system %iowait %steal %idle
0.00 0.00 1.07 0.00 0.00 98.93

Device: rrqm/s wrqm/s r/s w/s rsec/s wsec/s rkB/s wkB/s avgrq-sz avgqu-sz await svctm %util
sda5 0.33 0.02 68.91 1.93 863.40 492.88 431.70 246.44 19.15 0.58 8.24 1.43 10.10
sda5 0.00 0.40 133.87 1.20 2141.88 4.41 1070.94 2.20 15.89 7.97 59.10 7.41 100.12
sda5 0.00 0.00 130.34 1.60 2085.43 2.99 1042.71 1.50 15.83 7.98 60.25 7.57 99.88
sda5 0.00 0.00 135.20 1.40 2163.20 2.80 1081.60 1.40 15.86 7.99 58.62 7.32 100.00
sda5 0.00 0.20 131.80 1.60 2108.80 3.20 1054.40 1.60 15.83 7.96 59.64 7.50 100.00



[grid@vmxdb03 grid]$ $ORACLE_HOME/bin/orion -run simple -testname iscsi_io -num_disks 4
ORION: ORacle IO Numbers -- Version 11.2.0.2.0
iscsi_io_20101218_1854
Calibration will take approximately 30 minutes.
Using a large value for -cache_size may take longer.

Maximum Large MBPS=6.86 @ Small=0 and Large=4

Maximum Small IOPS=136 @ Small=16 and Large=0
Small Read Latency: avg=117588 us, min=29496 us, max=269881 us, std dev=25157 us @ Small=16 and Large=0

Minimum Small Latency=11782 usecs @ Small=1 and Large=0
Small Read Latency: avg=11782 us, min=2988 us, max=201139 us, std dev=3985 us @ Small=1 and Large=0
Small Read / Write Latency Histogram @ Small=16 and Large=0
Latency: # of IOs (read) # of IOs (write)
0 - 1 us: 0 0
2 - 4 us: 0 0
4 - 8 us: 0 0
8 - 16 us: 0 0
16 - 32 us: 0 0
32 - 64 us: 0 0
64 - 128 us: 0 0
128 - 256 us: 0 0
256 - 512 us: 0 0
512 - 1024 us: 0 0
1024 - 2048 us: 0 0
2048 - 4096 us: 6 0
4096 - 8192 us: 640 0
8192 - 16384 us: 4251 0
16384 - 32768 us: 189 0
32768 - 65536 us: 0 0
65536 - 131072 us: 0 0
131072 - 262144 us: 1 0
262144 - 524288 us: 0 0
524288 - 1048576 us: 0 0
1048576 - 2097152 us: 0 0
2097152 - 4194304 us: 0 0
4194304 - 8388608 us: 0 0
8388608 - 16777216 us: 0 0
16777216 - 33554432 us: 0 0
33554432 - 67108864 us: 0 0
67108864 - 134217728 us: 0 0
134217728 - 268435456 us: 0 0

-rw-r--r-- 1 grid oinstall 88 Dec 18 18:51 iscsi_io.lun
-rw-r--r-- 1 grid oinstall 23211 Dec 18 19:22 iscsi_io_20101218_1854_trace.txt
-rw-r--r-- 1 grid oinstall 1969 Dec 18 19:22 iscsi_io_20101218_1854_summary.txt
-rw-r--r-- 1 grid oinstall 738 Dec 18 19:22 iscsi_io_20101218_1854_mbps.csv
-rw-r--r-- 1 grid oinstall 903 Dec 18 19:22 iscsi_io_20101218_1854_lat.csv
-rw-r--r-- 1 grid oinstall 838 Dec 18 19:22 iscsi_io_20101218_1854_iops.csv
-rw-r--r-- 1 grid oinstall 26434 Dec 18 19:22 iscsi_io_20101218_1854_hist.txt


[grid@vmxdb03 grid]$ cat iscsi_io_20101218_1854_summary.txt
ORION VERSION 11.2.0.2.0

Command line:
-run simple -testname iscsi_io -num_disks 4

These options enable these settings:
Test: iscsi_io
Small IO size: 8 KB
Large IO size: 1024 KB
IO types: small random IOs, large random IOs
Sequential stream pattern: one LUN per stream
Writes: 0%
Cache size: not specified
Duration for each data point: 60 seconds
Small Columns:, 0
Large Columns:, 0, 1, 2, 3, 4, 5, 6, 7, 8
Total Data Points: 29

Name: /dev/iscsi/asmdisk2p1 Size: 2145957888
Name: /dev/iscsi/asmdisk3p1 Size: 2145957888
Name: /dev/iscsi/asmdisk4p1 Size: 2145957888
Name: /dev/iscsi/asmdisk5p1 Size: 2145957888
4 files found.

Maximum Large MBPS=6.86 @ Small=0 and Large=4

Maximum Small IOPS=136 @ Small=16 and Large=0
Small Read Latency: avg=117588 us, min=29496 us, max=269881 us, std dev=25157 us @ Small=16 and Large=0

Minimum Small Latency=11782.15 usecs @ Small=1 and Large=0
Small Read Latency: avg=11782 us, min=2988 us, max=201139 us, std dev=3985 us @ Small=1 and Large=0
Small Read / Write Latency Histogram @ Small=16 and Large=0
Latency: # of IOs (read) # of IOs (write)
0 - 1 us: 0 0
2 - 4 us: 0 0
4 - 8 us: 0 0
8 - 16 us: 0 0
16 - 32 us: 0 0
32 - 64 us: 0 0
64 - 128 us: 0 0
128 - 256 us: 0 0
256 - 512 us: 0 0
512 - 1024 us: 0 0
1024 - 2048 us: 0 0
2048 - 4096 us: 6 0
4096 - 8192 us: 640 0
8192 - 16384 us: 4251 0
16384 - 32768 us: 189 0
32768 - 65536 us: 0 0
65536 - 131072 us: 0 0
131072 - 262144 us: 1 0
262144 - 524288 us: 0 0
524288 - 1048576 us: 0 0
1048576 - 2097152 us: 0 0
2097152 - 4194304 us: 0 0
4194304 - 8388608 us: 0 0
8388608 - 16777216 us: 0 0
16777216 - 33554432 us: 0 0
33554432 - 67108864 us: 0 0
67108864 - 134217728 us: 0 0
134217728 - 268435456 us: 0 0



[grid@vmxdb03 grid]$ $ORACLE_HOME/bin/orion -run oltp -testname iscsi_io -num_disks 4 -duration 120
ORION: ORacle IO Numbers -- Version 11.2.0.2.0
iscsi_io_20101218_1947
Calibration will take approximately 49 minutes.
Using a large value for -cache_size may take longer.


Maximum Small IOPS=135 @ Small=48 and Large=0
Small Read Latency: avg=354599 us, min=46399 us, max=1336551 us, std dev=52605 us @ Small=48 and Large=0

Minimum Small Latency=32641 usecs @ Small=4 and Large=0
Small Read Latency: avg=32641 us, min=7475 us, max=733521 us, std dev=12886 us @ Small=4 and Large=0
Small Read / Write Latency Histogram @ Small=48 and Large=0
Latency: # of IOs (read) # of IOs (write)
0 - 1 us: 0 0
2 - 4 us: 0 0
4 - 8 us: 0 0
8 - 16 us: 0 0
16 - 32 us: 0 0
32 - 64 us: 0 0
64 - 128 us: 0 0
128 - 256 us: 0 0
256 - 512 us: 0 0
512 - 1024 us: 0 0
1024 - 2048 us: 0 0
2048 - 4096 us: 0 0
4096 - 8192 us: 3 0
8192 - 16384 us: 57 0
16384 - 32768 us: 7611 0
32768 - 65536 us: 7011 0
65536 - 131072 us: 5 0
131072 - 262144 us: 0 0
262144 - 524288 us: 0 0
524288 - 1048576 us: 4 0
1048576 - 2097152 us: 0 0
2097152 - 4194304 us: 0 0
4194304 - 8388608 us: 0 0
8388608 - 16777216 us: 0 0
16777216 - 33554432 us: 0 0
33554432 - 67108864 us: 0 0
67108864 - 134217728 us: 0 0
134217728 - 268435456 us: 0 0