关于磁盘IO的那些事
本文最后更新于 2024-07-08，文章内容可能已经过时。
关于磁盘IO的那些事
运维小弟 2023-02-24 18:07 发表于江苏
一、背景
需要把生产上一份clickhouse的备份数据到测试环境上，心想也就59G 的大小，应该很快吧，殊不知压缩就搞了3个多小时，于是好奇看下磁盘IO的一些参数，分享给大家学习参考
在分析存储性能之前，需要先了解存储性能定义的三个核心指标：IOPS、Throughput、await
指标 | 说明 |
|---|---|
IOPS | 每秒进行的IO操作次数 |
Throughput[吞吐量] | 每秒的I/O请求大小 |
await[响应时间] | 指I/O请求从发出到收到响应的间隔时间 |
二、使用fio工具进行磁盘性能测试
1、FIO是一款开源的 I/O 压力测试工具，主要是用来测试磁盘的IO性能。它可以支持多种不同的I/O引擎
fio官方文档：https://fio.readthedocs.io/en/latest/index.html
2、安装
[root@sre ~]# yum provides fio
已加载插件：fastestmirror
Loading mirror speeds from cached hostfile
fio-3.7-2.el7.x86_64 : Multithreaded IO generation tool
源 ：base
[root@sre ~]# yum install fio-3.7-2.el7.x86_64 -y3、参数
name 要测试的文件
filename=/dev/emcpowerb 支持文件系统或者裸设备，-filename=/dev/sda2或-filename=/dev/sdb
direct=1 测试过程绕过机器自带的buffer，使测试结果更真实
thread 创建线程为POSIX线程
rw=randwread 测试随机读的I/O
rw=randwrite 测试随机写的I/O
rw=randrw I/O类型为测试随机混合写和读的I/O，默认写占30%，读占70%
rw=read 测试顺序读的I/O
rw=write 测试顺序写的I/O
rw=rw 测试顺序混合写和读的I/O
bs=4k 单次io块大小为4k
size 本次的测试文件大小为500M，以每次4k的io进行测试
iodepth=1 队列深度，这是一个可以影响IOPS的参数，增加队列深度，可以测试出磁盘的峰值
numjobs=6 总共开6个线程读
ioengine=psync io引擎使用pync方式，如果要使用libaio引擎，需要yum install libaio-devel包
runtime: 总共运行时间为10秒，如果不写则一直将500M文件分4k每次写完为止
group_reporting 关于显示结果的，汇总每个进程的信息
4、其他参数
lockmem=1g 只使用1g内存进行测试
zero_buffers 用0初始化系统buffer
nrfiles=8 每个进程生成文件的数量
5、磁盘读写常用测试点
Read=100% Sequence=100% rw=read （100%顺序读）
Write=100% Sequence=100% rw=write （100%顺序写）
Read=70% Sequence=100% rw=rw, rwmixread=70, rwmixwrite=30（70%顺序读，30%顺序写）
Read=100% Ramdon=100% rw=randread (100%随机读)
Write=100% Ramdon=100% rw=randwrite （100%随机写）
Read=70% Ramdon=100% rw=randrw, rwmixread=70, rwmixwrite=30 (70%随机读，30%随机写)
6、FIO 工具测试模式：
测试模式 | 场景 |
|---|---|
顺序读写 | 进行大容量文件读写时，读写过程中会遵循先后顺序，数值越高代表读写性能越强，主要针对大容量文件。 |
随机读写 | 不遵循文件的先后顺序，读写时可以任意跳到某个文件。性能主要针对是小文件相关读写任务 |
7、测试样例
使用FIO进行读写操作为高危操作，请务必提前查询和确认好用于FIO测试的卷挂载详情，并在后续测试过程中，严格使用查询到的测试卷设备名，避免出现对系统盘或者其他业务数据卷进行误操作
（1）、100%，100%顺序读 ，4K
[root@sre ~]# fio -name=sqe_100read_4k -filename=/dev/sda2 -direct=1 -iodepth=1 -thread -rw=read -ioengine=psync -bs=4k -size=1G -numjobs=10 -runtime=120 -group_reporting
sqe_100read_4k: (g=0): rw=read, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.7
Starting 10 threads
Jobs: 10 (f=10): [R(10)][100.0%][r=137MiB/s,w=0KiB/s][r=35.1k,w=0 IOPS][eta 00m:00s]
sqe_100read_4k: (groupid=0, jobs=10): err= 0: pid=30672: Fri Feb 24 13:02:53 2023
read: IOPS=9002, BW=35.2MiB/s (36.9MB/s)(4220MiB/120005msec)
clat (usec): min=11, max=835604, avg=1109.96, stdev=8600.91
lat (usec): min=11, max=835604, avg=1110.08, stdev=8600.92
clat percentiles (usec):
| 1.00th=[ 37], 5.00th=[ 64], 10.00th=[ 82], 20.00th=[ 105],
| 30.00th=[ 120], 40.00th=[ 133], 50.00th=[ 147], 60.00th=[ 161],
| 70.00th=[ 184], 80.00th=[ 217], 90.00th=[ 734], 95.00th=[ 5669],
| 99.00th=[ 13304], 99.50th=[ 24511], 99.90th=[123208], 99.95th=[168821],
| 99.99th=[308282]
bw ( KiB/s): min= 8, max=19456, per=10.02%, avg=3606.87, stdev=4579.11, samples=2392
iops : min= 2, max= 4864, avg=901.69, stdev=1144.79, samples=2392
lat (usec) : 20=0.01%, 50=2.58%, 100=14.34%, 250=67.72%, 500=3.93%
lat (usec) : 750=1.50%, 1000=1.08%
lat (msec) : 2=1.40%, 4=1.35%, 10=4.21%, 20=1.27%, 50=0.34%
lat (msec) : 100=0.15%, 250=0.10%, 500=0.02%, 750=0.01%, 1000=0.01%
cpu : usr=0.21%, sys=0.89%, ctx=1079963, majf=0, minf=21
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=1080317,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=35.2MiB/s (36.9MB/s), 35.2MiB/s-35.2MiB/s (36.9MB/s-36.9MB/s), io=4220MiB (4425MB), run=120005-120005msec
Disk stats (read/write):
sda: ios=1078390/8445, merge=0/31, ticks=1188172/367350, in_queue=1555389, util=100.00%（2）、100%，100%顺序写 ，4K
[root@sre ~]# fio -name=sqe_100write_4k -filename=/dev/sda2 -direct=1 -iodepth=1 -thread -rw=write -ioengine=psync -bs=4k -size=1G -numjobs=10 -runtime=120 -group_reporting
sqe_100write_4k: (g=0): rw=write, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.7
Starting 10 threads
Jobs: 10 (f=10): [W(10)][100.0%][r=0KiB/s,w=159MiB/s][r=0,w=40.8k IOPS][eta 00m:00s]
sqe_100write_4k: (groupid=0, jobs=10): err= 0: pid=32135: Fri Feb 24 13:07:20 2023
write: IOPS=34.5k, BW=135MiB/s (141MB/s)(10.0GiB/75895msec)
clat (usec): min=51, max=1688.4k, avg=288.56, stdev=6133.39
lat (usec): min=51, max=1688.4k, avg=288.74, stdev=6133.39
clat percentiles (usec):
| 1.00th=[ 78], 5.00th=[ 90], 10.00th=[ 98], 20.00th=[ 110],
| 30.00th=[ 119], 40.00th=[ 126], 50.00th=[ 135], 60.00th=[ 143],
| 70.00th=[ 153], 80.00th=[ 167], 90.00th=[ 204], 95.00th=[ 253],
| 99.00th=[ 3261], 99.50th=[ 5145], 99.90th=[ 12518], 99.95th=[ 34866],
| 99.99th=[202376]
bw ( KiB/s): min= 8, max=39896, per=10.48%, avg=14474.40, stdev=8454.32, samples=1440
iops : min= 2, max= 9974, avg=3618.58, stdev=2113.59, samples=1440
lat (usec) : 100=11.56%, 250=83.19%, 500=3.79%, 750=0.14%, 1000=0.05%
lat (msec) : 2=0.05%, 4=0.32%, 10=0.78%, 20=0.06%, 50=0.03%
lat (msec) : 100=0.01%, 250=0.02%, 500=0.01%, 750=0.01%, 1000=0.01%
cpu : usr=0.79%, sys=4.06%, ctx=2621452, majf=0, minf=9
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,2621440,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=135MiB/s (141MB/s), 135MiB/s-135MiB/s (141MB/s-141MB/s), io=10.0GiB (10.7GB), run=75895-75895msec
Disk stats (read/write):
sda: ios=109/2623562, merge=0/28, ticks=4282/910098, in_queue=914111, util=100.00%（3）、100%，70%顺序读，30%顺序写 4K
[root@sre ~]# fio -name=sqe_rw_70read_4k -filename=/dev/sda2 -direct=1 -iodepth=1 -thread -rw=rw -rwmixread=70 -ioengine=psync -bs=4k -size=1G -numjobs=10 -runtime=120 -group_reporting
sqe_rw_70read_4k: (g=0): rw=rw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.7
Starting 10 threads
Jobs: 10 (f=10): [M(10)][100.0%][r=24.1MiB/s,w=9.99MiB/s][r=6172,w=2556 IOPS][eta 00m:00s]
sqe_rw_70read_4k: (groupid=0, jobs=10): err= 0: pid=3512: Fri Feb 24 13:24:03 2023
read: IOPS=6946, BW=27.1MiB/s (28.5MB/s)(3258MiB/120049msec)
clat (usec): min=44, max=1405.9k, avg=1219.85, stdev=8850.12
lat (usec): min=44, max=1405.9k, avg=1219.98, stdev=8850.13
clat percentiles (usec):
| 1.00th=[ 62], 5.00th=[ 78], 10.00th=[ 89], 20.00th=[ 104],
| 30.00th=[ 117], 40.00th=[ 128], 50.00th=[ 141], 60.00th=[ 155],
| 70.00th=[ 178], 80.00th=[ 229], 90.00th=[ 1401], 95.00th=[ 6980],
| 99.00th=[ 14877], 99.50th=[ 25822], 99.90th=[ 98042], 99.95th=[133694],
| 99.99th=[265290]
bw ( KiB/s): min= 56, max=14016, per=10.12%, avg=2812.14, stdev=2999.01, samples=2371
iops : min= 14, max= 3504, avg=702.97, stdev=749.75, samples=2371
write: IOPS=2980, BW=11.6MiB/s (12.2MB/s)(1398MiB/120049msec)
clat (usec): min=52, max=491303, avg=507.44, stdev=5171.58
lat (usec): min=52, max=491304, avg=507.65, stdev=5171.58
clat percentiles (usec):
| 1.00th=[ 72], 5.00th=[ 86], 10.00th=[ 101], 20.00th=[ 126],
| 30.00th=[ 145], 40.00th=[ 161], 50.00th=[ 176], 60.00th=[ 192],
| 70.00th=[ 212], 80.00th=[ 237], 90.00th=[ 293], 95.00th=[ 490],
| 99.00th=[ 4817], 99.50th=[ 11994], 99.90th=[ 83362], 99.95th=[117965],
| 99.99th=[208667]
bw ( KiB/s): min= 15, max= 6128, per=10.12%, avg=1206.56, stdev=1287.84, samples=2371
iops : min= 3, max= 1532, avg=301.57, stdev=321.97, samples=2371
lat (usec) : 50=0.01%, 100=14.76%, 250=67.61%, 500=6.31%, 750=2.02%
lat (usec) : 1000=1.10%
lat (msec) : 2=1.28%, 4=1.17%, 10=3.97%, 20=1.19%, 50=0.35%
lat (msec) : 100=0.15%, 250=0.08%, 500=0.01%, 750=0.01%, 1000=0.01%
cpu : usr=0.24%, sys=0.99%, ctx=1191799, majf=0, minf=9
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=833936,357813,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=27.1MiB/s (28.5MB/s), 27.1MiB/s-27.1MiB/s (28.5MB/s-28.5MB/s), io=3258MiB (3416MB), run=120049-120049msec
WRITE: bw=11.6MiB/s (12.2MB/s), 11.6MiB/s-11.6MiB/s (12.2MB/s-12.2MB/s), io=1398MiB (1466MB), run=120049-120049msec
Disk stats (read/write):
sda: ios=831488/361338, merge=0/71, ticks=998352/328430, in_queue=1326637, util=100.00%
（4）、100%，100%随机读 ，4K
[root@sre ~]# fio -name=rand_100read_4k -filename=/dev/sda2 -direct=1 -iodepth=1 -thread -rw=randread -ioengine=psync -bs=4k -size=1G -numjobs=10 -runtime=120 -group_reporting
rand_100read_4k: (g=0): rw=randread, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.7
Starting 10 threads
Jobs: 10 (f=10): [r(10)][100.0%][r=521KiB/s,w=0KiB/s][r=130,w=0 IOPS][eta 00m:00s]
rand_100read_4k: (groupid=0, jobs=10): err= 0: pid=10961: Fri Feb 24 14:16:54 2023
read: IOPS=420, BW=1683KiB/s (1724kB/s)(197MiB/120022msec)
clat (usec): min=73, max=976990, avg=23750.11, stdev=41850.90
lat (usec): min=73, max=976991, avg=23750.48, stdev=41850.92
clat percentiles (usec):
| 1.00th=[ 412], 5.00th=[ 2180], 10.00th=[ 4752], 20.00th=[ 7177],
| 30.00th=[ 9110], 40.00th=[ 10945], 50.00th=[ 12911], 60.00th=[ 15533],
| 70.00th=[ 19268], 80.00th=[ 25822], 90.00th=[ 43254], 95.00th=[ 82314],
| 99.00th=[217056], 99.50th=[291505], 99.90th=[455082], 99.95th=[541066],
| 99.99th=[784335]
bw ( KiB/s): min= 7, max= 400, per=10.00%, avg=168.34, stdev=89.05, samples=2393
iops : min= 1, max= 100, avg=41.85, stdev=22.29, samples=2393
lat (usec) : 100=0.01%, 250=0.08%, 500=1.99%, 750=1.12%, 1000=0.75%
lat (msec) : 2=0.91%, 4=3.10%, 10=27.07%, 20=36.74%, 50=19.71%
lat (msec) : 100=4.53%, 250=3.24%, 500=0.69%, 750=0.05%, 1000=0.02%
cpu : usr=0.03%, sys=0.13%, ctx=50535, majf=0, minf=19
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=50512,0,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=1683KiB/s (1724kB/s), 1683KiB/s-1683KiB/s (1724kB/s-1724kB/s), io=197MiB (207MB), run=120022-120022msec
Disk stats (read/write):
sda: ios=51029/15975, merge=10/120, ticks=1228499/486616, in_queue=1715958, util=100.00%（5）、100%，100%随机写 ，4K
[root@sre ~]# fio -name=rand_100write_4k -filename=/dev/sda2 -direct=1 -iodepth=1 -thread -rw=randwrite -ioengine=psync -bs=4k -size=1G -numjobs=10 -runtime=120 -group_reporting
rand_100write_4k: (g=0): rw=randwrite, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.7
Starting 10 threads
Jobs: 10 (f=10): [w(10)][100.0%][r=0KiB/s,w=10.7MiB/s][r=0,w=2747 IOPS][eta 00m:00s]
rand_100write_4k: (groupid=0, jobs=10): err= 0: pid=11803: Fri Feb 24 13:29:35 2023
write: IOPS=1188, BW=4755KiB/s (4869kB/s)(558MiB/120134msec)
clat (usec): min=54, max=1692.6k, avg=8404.99, stdev=42966.47
lat (usec): min=54, max=1692.6k, avg=8405.21, stdev=42966.47
clat percentiles (usec):
| 1.00th=[ 103], 5.00th=[ 133], 10.00th=[ 157],
| 20.00th=[ 200], 30.00th=[ 255], 40.00th=[ 322],
| 50.00th=[ 404], 60.00th=[ 529], 70.00th=[ 734],
| 80.00th=[ 1352], 90.00th=[ 8455], 95.00th=[ 41157],
| 99.00th=[ 164627], 99.50th=[ 246416], 99.90th=[ 517997],
| 99.95th=[ 750781], 99.99th=[1417675]
bw ( KiB/s): min= 7, max= 4088, per=10.46%, avg=497.31, stdev=463.38, samples=2295
iops : min= 1, max= 1022, avg=124.17, stdev=115.89, samples=2295
lat (usec) : 100=0.75%, 250=28.39%, 500=29.08%, 750=12.21%, 1000=5.44%
lat (msec) : 2=8.18%, 4=3.49%, 10=3.13%, 20=2.38%, 50=2.45%
lat (msec) : 100=2.01%, 250=2.01%, 500=0.38%, 750=0.06%, 1000=0.03%
cpu : usr=0.04%, sys=0.15%, ctx=142837, majf=0, minf=8
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=0,142815,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
WRITE: bw=4755KiB/s (4869kB/s), 4755KiB/s-4755KiB/s (4869kB/s-4869kB/s), io=558MiB (585MB), run=120134-120134msec
Disk stats (read/write):
sda: ios=40/148476, merge=17/74, ticks=18004/1646246, in_queue=1668467, util=100.00%（6）、100%，70%随机读，30%随机写 4K
[root@sre ~]# fio -name=rand_rw_70read_4k -filename=/dev/sda2 -direct=1 -iodepth=1 -thread -rw=rw -rwmixread=70 -ioengine=psync -bs=4k -size=1G -numjobs=10 -runtime=120 -group_reporting
rand_rw_70read_4k: (g=0): rw=rw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=psync, iodepth=1
...
fio-3.7
Starting 10 threads
Jobs: 10 (f=10): [M(10)][100.0%][r=10.1MiB/s,w=4670KiB/s][r=2591,w=1167 IOPS][eta 00m:00s]
rand_rw_70read_4k: (groupid=0, jobs=10): err= 0: pid=26920: Fri Feb 24 13:34:23 2023
read: IOPS=6645, BW=25.0MiB/s (27.2MB/s)(3115MiB/120003msec)
clat (usec): min=38, max=610563, avg=1279.77, stdev=8021.32
lat (usec): min=38, max=610563, avg=1279.90, stdev=8021.33
clat percentiles (usec):
| 1.00th=[ 53], 5.00th=[ 63], 10.00th=[ 73], 20.00th=[ 88],
| 30.00th=[ 100], 40.00th=[ 113], 50.00th=[ 127], 60.00th=[ 145],
| 70.00th=[ 176], 80.00th=[ 330], 90.00th=[ 1418], 95.00th=[ 6915],
| 99.00th=[ 15795], 99.50th=[ 29754], 99.90th=[119014], 99.95th=[160433],
| 99.99th=[325059]
bw ( KiB/s): min= 8, max=16128, per=10.00%, avg=2657.88, stdev=3002.47, samples=2389
iops : min= 2, max= 4032, avg=664.31, stdev=750.64, samples=2389
write: IOPS=2849, BW=11.1MiB/s (11.7MB/s)(1336MiB/120003msec)
clat (usec): min=46, max=419565, avg=520.42, stdev=4599.74
lat (usec): min=46, max=419565, avg=520.62, stdev=4599.75
clat percentiles (usec):
| 1.00th=[ 63], 5.00th=[ 74], 10.00th=[ 84], 20.00th=[ 105],
| 30.00th=[ 125], 40.00th=[ 143], 50.00th=[ 161], 60.00th=[ 182],
| 70.00th=[ 210], 80.00th=[ 262], 90.00th=[ 562], 95.00th=[ 799],
| 99.00th=[ 6259], 99.50th=[ 11076], 99.90th=[ 56361], 99.95th=[ 90702],
| 99.99th=[229639]
bw ( KiB/s): min= 7, max= 6912, per=10.10%, avg=1151.11, stdev=1288.82, samples=2365
iops : min= 1, max= 1728, avg=287.61, stdev=322.25, samples=2365
lat (usec) : 50=0.22%, 100=26.02%, 250=51.97%, 500=6.21%, 750=5.09%
lat (usec) : 1000=1.68%
lat (msec) : 2=1.70%, 4=1.39%, 10=3.74%, 20=1.39%, 50=0.35%
lat (msec) : 100=0.15%, 250=0.09%, 500=0.01%, 750=0.01%
cpu : usr=0.19%, sys=0.74%, ctx=1139413, majf=0, minf=8
IO depths : 1=100.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=0.0%, >=64=0.0%
submit : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
complete : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
issued rwts: total=797444,341945,0,0 short=0,0,0,0 dropped=0,0,0,0
latency : target=0, window=0, percentile=100.00%, depth=1
Run status group 0 (all jobs):
READ: bw=25.0MiB/s (27.2MB/s), 25.0MiB/s-25.0MiB/s (27.2MB/s-27.2MB/s), io=3115MiB (3266MB), run=120003-120003msec
WRITE: bw=11.1MiB/s (11.7MB/s), 11.1MiB/s-11.1MiB/s (11.7MB/s-11.7MB/s), io=1336MiB (1401MB), run=120003-120003msec
Disk stats (read/write):
sda: ios=796912/350573, merge=0/72, ticks=914709/402509, in_queue=1317165, util=99.50%
测试结果：
1、磁盘的每秒读写次数iops，这个是随机读写考察的重点;
2、磁盘的吞吐量bw(MB/s)，这个是顺序读写考察的重点;
3、await响应时间[usec代表微秒]，avg平均响应时间）值越小,性能越好
测试模式(4K) | iops | bw(MB/s) | await(ms) |
|---|---|---|---|
顺序读 | 9002 | 35.2 | 1.109 |
顺序写 | 34500 | 135 | 0.288 |
70%顺序读，30%顺序写 | 6946 | 27.1 | 1.219 |
随机读 | 420 | 1.6 | 23.750 |
随机写 | 1188 | 4.7 | 8.404 |
70%随机读，30%随机写 | 6645 | 25 | 1.279 |
由于4K随机读写速度有两种单位IOPS和MB/s，为方便我们比较，可以通过用下面公式来取得一个近似值。
总结以上论证就是说：存储的性能是由三个核心指标：iops、bw(Throughput)、await来定义存储是不是有性能瓶颈，通过FIO工具可以很直观的展现
三、观测磁盘I/O
1、iostat是常见的磁盘I/O性能观测工具，运维通过命令实时查看磁盘负荷情况
2、安装
yum install sysstat3、参数
-d: 显示该设备的状态的参数
-x：是一个比较常用的选项，该选项将用于显示和io相关的扩展数据
-m: 以M为单位显示所有信息
-d: sda/sdb 显示指定硬盘信息
-t: 报告每秒向终端读取和写入的字符数
数字2 ： 表示几秒刷新一次
[root@sre ~]# iostat -mtxd 2
Linux 3.10.0-1127.el7.x86_64 (sre) 2023年02月24日 _x86_64_ (8 CPU)
2023年02月24日 17时19分51秒
Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.01 0.32 1.63 69.04 0.13 1.27 40.54 0.17 2.37 161.48 3.30 2.03 14.34
scd0 0.00 0.00 0.00 0.00 0.00 0.00 114.22 0.00 0.11 0.11 0.00 0.11 0.00
dm-0 0.00 0.00 0.01 0.63 0.00 0.02 64.84 0.02 30.87 79.49 30.18 7.08 0.45
dm-1 0.00 0.00 0.01 0.02 0.00 0.00 8.01 0.00 138.56 43.93 193.34 4.93 0.01
dm-2 0.00 0.00 1.37 68.44 0.12 1.25 40.39 0.15 2.18 191.04 3.13 2.03 14.19
2023年02月24日 17时19分53秒
Device: rrqm/s wrqm/s r/s w/s rMB/s wMB/s avgrq-sz avgqu-sz await r_await w_await svctm %util
sda 0.00 0.00 0.00 56.50 0.00 0.56 20.13 0.10 1.77 0.00 1.77 0.68 3.85
scd0 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-0 0.00 0.00 0.00 35.50 0.00 0.25 14.58 0.02 0.54 0.00 0.54 0.52 1.85
dm-1 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00
dm-2 0.00 0.00 0.00 21.00 0.00 0.30 29.52 0.08 3.83 0.00 3.83 1.10 2.30
还是重点关注3个性能核心指标：iops、throughput、await与多个任务指标
网上找了一张图
如何通过以上指标数据判断磁盘有没有瓶颈呢？
(1)、如果 %util 接近 100%，说明产生的I/O请求太多，I/O系统已经满负荷，该磁盘可能存在瓶颈
(2)、如果 svctm的值与await很接近，说明IO任务提交之后，IO立即响应了，表示几乎没有I/O等待，磁盘性能很好，反之则有问题；
(3)、查看aqu-sz队列值，越小越好，反之差值越大，队列越大，说明有问题;
四、参考
https://blog.csdn.net/weixin_67470255/article/details/124090396
https://blog.csdn.net/qq_30654043/article/details/124464647
- 感谢你赐予我前进的力量
-
微信 -
支付宝
赞赏者名单
因为你们的支持让我意识到写文章的价值🙏
本文是原创文章，采用 CC BY-NC-ND 4.0 协议，完整转载请注明来自 运维小弟
阅读建议
