Linux/CGP作業メモ

■さくらのVPS 8GのIOPSを調べてみた
　レスポンスが酷くなって死に体だった某SNSがさくらのVPS 8Gにサーバを移設したことでサクサク快適になったと聞いて、お試しを利用して比較的負荷の高いSNSサービスを快適に提供できるIOPSというのはどのくらいなのかを調べてみた。VPSのOSがCentOS6 x86_64なので調査に使ったのはfioとorion。fioでは8k,10MB,128MBの３つのサイズを、orionではディスクの数を指定するパラメーターで負荷を上げることができることで今回は１，４，８で試してみた。SNSサービスを前提にするならデータベースに近い測定方法のorionの結果を重視するのでいいのではないかと思う。

　非常に興味深かったのは負荷かけながらtopで様子を見ていたところ、さくらのVPSでは大きな負荷をかけてもiowaitが16%前後で頭打ちになり、他のVPSに大きな迷惑をかけないよううまく制御されているように見えたこと。見ている限りfioでもorionでもCPUは数％程度、iowaitも16%までと非常に余裕を持って動作していた。これならおそらく他のVPSからの影響も大きく受けることなく安定してサーバ運用ができると思われる。

■fioの結果
１）サイズ 8K IOPS 2000
[code]# fio random-read-test.fio
random-read: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio-2.0.9
Starting 1 process

random-read: (groupid=0, jobs=1): err= 0: pid=1659: Sat Nov 17 22:16:19 2012
read : io=8192 B, bw=8000.0KB/s, iops=2000 , runt= 1msec
clat (usec): min=355 , max=550 , avg=452.50, stdev=137.89
lat (usec): min=356 , max=550 , avg=453.00, stdev=137.18
clat percentiles (usec):
| 1.00th=[ 354], 5.00th=[ 354], 10.00th=[ 354], 20.00th=[ 354],
| 30.00th=[ 354], 40.00th=[ 354], 50.00th=[ 354], 60.00th=[ 548],
| 70.00th=[ 548], 80.00th=[ 548], 90.00th=[ 548], 95.00th=[ 548],
| 99.00th=[ 548], 99.50th=[ 548], 99.90th=[ 548], 99.95th=[ 548],
| 99.99th=[ 548]
lat (usec) : 500=50.00%, 750=50.00%
cpu : usr=0.00%, sys=0.00%, ctx=2, majf=0, minf=27
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=2/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=8KB, aggrb=8000KB/s, minb=8000KB/s, maxb=8000KB/s, mint=1msec, maxt=1msec

Disk stats (read/write):
vda: ios=0/0, merge=0/0, ticks=0/0, in_queue=0, util=0.00%[/code]

２）サイズ 10M IOPS 3932
[code]# fio random-read-test.fio
random-read: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio-2.0.9
Starting 1 process
random-read: Laying out IO file(s) (1 file(s) / 10MB)

random-read: (groupid=0, jobs=1): err= 0: pid=1647: Sat Nov 17 22:06:54 2012
read : io=10240KB, bw=15730KB/s, iops=3932 , runt= 651msec
clat (usec): min=174 , max=566 , avg=247.94, stdev=39.81
lat (usec): min=174 , max=567 , avg=248.42, stdev=39.81
clat percentiles (usec):
| 1.00th=[ 185], 5.00th=[ 199], 10.00th=[ 207], 20.00th=[ 219],
| 30.00th=[ 229], 40.00th=[ 235], 50.00th=[ 241], 60.00th=[ 247],
| 70.00th=[ 258], 80.00th=[ 274], 90.00th=[ 294], 95.00th=[ 330],
| 99.00th=[ 382], 99.50th=[ 410], 99.90th=[ 486], 99.95th=[ 490],
| 99.99th=[ 564]
bw (KB/s) : min=15560, max=15560, per=98.93%, avg=15560.00, stdev= 0.00
lat (usec) : 250=63.75%, 500=36.21%, 750=0.04%
cpu : usr=2.31%, sys=14.77%, ctx=2560, majf=0, minf=27
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=2560/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=10240KB, aggrb=15729KB/s, minb=15729KB/s, maxb=15729KB/s, mint=651msec, maxt=651msec

Disk stats (read/write):
vda: ios=2390/0, merge=0/0, ticks=430/0, in_queue=425, util=60.03%[/code]

３）サイズ 128MB IOPS 4300
[code]# fio random-read-test.fio
random-read: (g=0): rw=randread, bs=4K-4K/4K-4K, ioengine=sync, iodepth=1
fio-2.0.9
Starting 1 process
random-read: Laying out IO file(s) (1 file(s) / 128MB)
Jobs: 1 (f=1): [r] [100.0% done] [18308K/0K /s] [4577 /0 iops] [eta 00m:00s]
random-read: (groupid=0, jobs=1): err= 0: pid=1651: Sat Nov 17 22:09:38 2012
read : io=131072KB, bw=17201KB/s, iops=4300 , runt= 7620msec
clat (usec): min=149 , max=907 , avg=226.51, stdev=33.92
lat (usec): min=150 , max=907 , avg=227.01, stdev=33.93
clat percentiles (usec):
| 1.00th=[ 167], 5.00th=[ 175], 10.00th=[ 181], 20.00th=[ 197],
| 30.00th=[ 209], 40.00th=[ 219], 50.00th=[ 231], 60.00th=[ 239],
| 70.00th=[ 243], 80.00th=[ 249], 90.00th=[ 262], 95.00th=[ 278],
| 99.00th=[ 322], 99.50th=[ 350], 99.90th=[ 422], 99.95th=[ 454],
| 99.99th=[ 644]
bw (KB/s) : min=16136, max=19488, per=99.96%, avg=17194.67, stdev=1014.85
lat (usec) : 250=80.75%, 500=19.22%, 750=0.02%, 1000=0.01%
cpu : usr=2.61%, sys=16.14%, ctx=32769, majf=0, minf=27
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued : total=r=32768/w=0/d=0, short=r=0/w=0/d=0

Run status group 0 (all jobs):
READ: io=131072KB, aggrb=17201KB/s, minb=17201KB/s, maxb=17201KB/s, mint=7620msec, maxt=7620msec

Disk stats (read/write):
vda: ios=31738/3, merge=0/0, ticks=5512/0, in_queue=5490, util=73.44%[/code]

■orionの結果
１）DISK数1 IOPS 879
[code]# ./orion_linux_x86-64 -run oltp -testname vps -num_disks 1
# cat vps_20121117_xxxx_summary.txt
ORION VERSION 11.1.0.7.0

Commandline:
-run oltp -testname vps -num_disks 1

This maps to this test:
Test: vps
Small IO size: 8 KB
Large IO size: 1024 KB
IO Types: Small Random IOs, Large Random IOs
Simulated Array Type: CONCAT
Write: 0%
Cache Size: Not Entered
Duration for each Data Point: 60 seconds
Small Columns:, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20
Large Columns:, 0
Total Data Points: 21

Name: /dev/vda3 Size: 856582766080
1 FILEs found.

Maximum Small IOPS=879 @ Small=20 and Large=0
Minimum Small Latency=8.03 @ Small=1 and Large=0[/code]

２）DISK数4 IOPS 1210
[code]# ./orion_linux_x86-64 -run oltp -testname vps -num_disks 4
# cat vps_20121117_xxxx_summary.txt
ORION VERSION 11.1.0.7.0

Commandline:
-run oltp -testname vps -num_disks 4

This maps to this test:
Test: vps
Small IO size: 8 KB
Large IO size: 1024 KB
IO Types: Small Random IOs, Large Random IOs
Simulated Array Type: CONCAT
Write: 0%
Cache Size: Not Entered
Duration for each Data Point: 60 seconds
Small Columns:, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52, 56, 60, 64, 68, 72, 76, 80
Large Columns:, 0
Total Data Points: 24

Name: /dev/vda3 Size: 856582766080
1 FILEs found.

Maximum Small IOPS=1210 @ Small=76 and Large=0
Minimum Small Latency=10.66 @ Small=4 and Large=0[/code]

３）DISK数8 IOPS 1214
[code]# ./orion_linux_x86-64 -run oltp -testname vps -num_disks 8
# cat vps_20121117_xxxx_summary.txt
ORION VERSION 11.1.0.7.0

Commandline:
-run oltp -testname vps -num_disks 8

This maps to this test:
Test: vps
Small IO size: 8 KB
Large IO size: 1024 KB
IO Types: Small Random IOs, Large Random IOs
Simulated Array Type: CONCAT
Write: 0%
Cache Size: Not Entered
Duration for each Data Point: 60 seconds
Small Columns:, 8, 16, 24, 32, 40, 48, 56, 64, 72, 80, 88, 96, 104, 112, 120, 128, 136, 144, 152, 160
Large Columns:, 0
Total Data Points: 28

Name: /dev/vda3 Size: 856582766080
1 FILEs found.

Maximum Small IOPS=1214 @ Small=120 and Large=0
Minimum Small Latency=13.30 @ Small=8 and Large=0[/code]

■参考
１）ASUS K55DR + Samsung 830 SSD
[code]# ./orion_linux_x86-64 -run oltp -testname test -num_disks 4
# less test_20121117_xxxx_summary.txt
ORION VERSION 11.1.0.7.0

Commandline:
-run oltp -testname test -num_disks 4

This maps to this test:
Test: test
Small IO size: 8 KB
Large IO size: 1024 KB
IO Types: Small Random IOs, Large Random IOs
Simulated Array Type: CONCAT
Write: 0%
Cache Size: Not Entered
Duration for each Data Point: 60 seconds
Small Columns:, 4, 8, 12, 16, 20, 24, 28, 32, 36, 40, 44, 48, 52, 56, 60, 64, 68, 72, 76, 80
Large Columns:, 0
Total Data Points: 24

Name: /dev/mapper/vg_mx-lv_root Size: 121769033728
1 FILEs found.

Maximum Small IOPS=25169 @ Small=72 and Large=0
Minimum Small Latency=0.19 @ Small=4 and Large=0[/code]

２）DELL D520 + Hitachi HTS723232A7A364
[code]# ./orion_linux_x86-64 -run oltp -testname test -num_disks 4
# less test_20121117_xxxx_summary.txt
ORION VERSION 11.1.0.7.0

Commandline:
-run oltp -testname test -num_disks 1

This maps to this test:
Test: test
Small IO size: 8 KB
Large IO size: 1024 KB
IO Types: Small Random IOs, Large Random IOs
Simulated Array Type: CONCAT
Write: 0%
Cache Size: Not Entered
Duration for each Data Point: 60 seconds
Small Columns:, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20
Large Columns:, 0
Total Data Points: 21

Name: /dev/sda3 Size: 314460679680
1 FILEs found.

Maximum Small IOPS=94 @ Small=20 and Large=0
Minimum Small Latency=17.53 @ Small=1 and Large=0[/code]

■まとめ
　さくらVPS 8Gの環境はorionのテストにおいてSSDには遠く及ばないが、通常のHDDの約12倍のパフォーマンスを出していることが確認できた。VPSとしては十分な速度だと思われる。IOPSのテストツールとしてfioとorionを比較した場合、fioの方がIOPSを高めに出す傾向があるようだ。ファイルの読み出しという点ではキャッシュなども動作するため、IOPSが高くなる傾向があってもおかしくない。SNS等のデータベースの利用度の高いアプリケーションではorionを使ったIOPSで評価を行う方がより現実的なレスポンスに近い結果が得られるのではないかとテスト結果からも感じられた。