☰

使用软件 RAID 通过 Samba 传输速度慢

tag-icon • tag-icon linux ubuntu samba software-raid file-transfer
使用软件 RAID 通过 Samba 传输速度慢

我有一台迷你 PC（Intel Celeron J4005、4GB RAM、Intel 千兆网卡），配置为：

  • Ubuntu（5.4.0-81-generic，安装在 sda 上）
  • Samba（版本4.11.6-Ubuntu）
  • FTP（vsftpd，无加密）
  • RAID5（mdadm、md0：sdb-sdc-sdd、USB-SATA）

RAID阵列通过Samba和FTP共享，但我想消除FTP，所有主要客户端都是Windows机器。问题是我通过 Samba 共享获得的速度比 FTP 慢得多：

设备 方法 读取速度（Mbyte/s，1个大文件）
MD0 当地的 〜220
MD0 局域网、FTP ~115（网络限制）
MD0 局域网、桑巴舞 〜48
MD0 LAN、Samba、第二次运行（缓存在内存中） ~115（网络限制）
sda 局域网、桑巴舞 ~115（网络限制）

我尝试使用默认 Samba 设置和当前设置（见下文），但得到了相同的结果。我在测试之间刷新了缓存。

iostat 输出示例（LAN、Samba、首次运行）：

Device            r/s     rkB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wkB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dkB/s   drqm/s  %drqm d_await dareq-sz  aqu-sz  %util
md0            793.00 433408.00     0.00   0.00    0.00   546.54    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00   0.00
sda              0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00   0.00
sdb            254.00  16768.00     8.00   3.05   14.74    66.02    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    3.27  84.80
sdc            171.00  16896.00    93.00  35.23    2.99    98.81    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.32  60.80
sdd            161.00  16640.00   101.00  38.55   11.74   103.35    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    1.57  96.00

iostat 输出示例（LAN、FTP、首次运行）：

Device            r/s     rkB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wkB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dkB/s   drqm/s  %drqm d_await dareq-sz  aqu-sz  %util
md0           1828.00 292480.00     0.00   0.00    0.00   160.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00   0.00
sda              0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.00   0.00
sdb            458.00  39040.00   153.00  25.04    1.66    85.24    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.18  75.60
sdc            457.00  38976.00   152.00  24.96    1.45    85.29    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.09  70.40
sdd            457.00  38976.00   152.00  24.96    1.59    85.29    0.00      0.00     0.00   0.00    0.00     0.00    0.00      0.00     0.00   0.00    0.00     0.00    0.15  75.20

我不知道问题是什么，有人可以帮助我，或者至少我应该从哪里开始调查？

桑巴配置：

[global]
   workgroup = WORKGROUP
   min protocol = SMB3
   
   log level = 1
   socket options = TCP_NODELAY SO_RCVBUF=65536 SO_SNDBUF=65536 IPTOS_LOWDELAY SO_KEEPALIVE
   use sendfile = true
   aio read size = 65536
   aio write size = 65536
   read raw = yes
   write raw = yes
   getwd cache = yes
   acl allow execute always = true
   
   log file = /var/log/samba/log.%m
   max log size = 1000
   logging = file
   server role = standalone server
   obey pam restrictions = yes
   unix password sync = yes
   passwd program = /usr/bin/passwd %u
   passwd chat = *Enter\snew\s*\spassword:* %n\n *Retype\snew\s*\spassword:* %n\n *password\supdated\ssuccessfully* .
   pam password change = yes
   map to guest = bad user

[Share]
  path = /media/hdd
  writable = yes
  valid users = myuser
  directory mode = 0770
  create mode = 0660

RAID阵列配置：

/dev/md0:
           Version : 1.2
     Creation Time : Tue Sep  7 13:19:26 2021
        Raid Level : raid5
        Array Size : 976441344 (931.21 GiB 999.88 GB)
     Used Dev Size : 488220672 (465.60 GiB 499.94 GB)
      Raid Devices : 3
     Total Devices : 3
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Tue Sep 9 14:37:52 2021
             State : clean
    Active Devices : 3
   Working Devices : 3
    Failed Devices : 0
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 64K

Consistency Policy : bitmap

文件系统信息：

root@MiniPC:~# df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            1.8G     0  1.8G   0% /dev
tmpfs           371M   12M  360M   3% /run
/dev/sda2        58G  3.4G   55G   6% /
tmpfs           1.9G   12K  1.9G   1% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           1.9G     0  1.9G   0% /sys/fs/cgroup
tmpfs           500M   79M  422M  16% /var/cache/apt
tmpfs           500M     0  500M   0% /tmp
tmpfs           500M     0  500M   0% /var/backups
tmpfs           500M  2.2M  498M   1% /var/log
tmpfs           500M     0  500M   0% /var/tmp
/dev/sda1       511M  5.3M  506M   2% /boot/efi
/dev/md0        917G  356G  562G  39% /media/hdd

root@MiniPC:~# lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda       8:0    0  59.6G  0 disk
├─sda1    8:1    0   512M  0 part  /boot/efi
└─sda2    8:2    0  59.1G  0 part  /
sdb       8:16   0 465.7G  0 disk
└─sdb1    8:17   0 465.7G  0 part
  └─md0   9:0    0 931.2G  0 raid5 /media/hdd
sdc       8:32   0 465.8G  0 disk
└─sdc1    8:33   0 465.8G  0 part
  └─md0   9:0    0 931.2G  0 raid5 /media/hdd
sdd       8:48   0 465.8G  0 disk
└─sdd1    8:49   0 465.8G  0 part
  └─md0   9:0    0 931.2G  0 raid5 /media/hdd

答案1

https://www.samba.org/samba/docs/current/man-html/smb.conf.5.html

aio读取大小(S)

If this integer parameter is set to a non-zero value, Samba will read from files
asynchronously when the request size is bigger than this value.
Note that it happens only for non-chained and non-chaining reads and when
not using write cache.

The only reasonable values for this parameter are 0 (no async I/O) and 1 (always do async I/O).

same is said for aio write side

我建议从您的 smb.conf 中删除这两个。我从来没有在我的 smb.conf 中使用过这两个。

我通过 1gbps 有线网络与 win10 电脑全天获得 112 MB/秒的读/写速度，此处参考我的 smb.conf 全局部分

[global]
    workgroup = SAMBA
    security = user

    passdb backend = tdbsam

    printing = bsd
    printcap name = /dev/null
    load printers = no
    disable spoolss = yes
  # cups options = raw

    smb encrypt = required
    client min protocol = SMB3
    client max protocol = SMB3

在 RHEL 7 中，调整后的配置文件供参考

> tuned-adm list
    Available profiles:
    - atomic-guest                - Optimize virtual guests based on the Atomic variant
    - atomic-host                 - Optimize bare metal systems running the Atomic variant
    - balanced                    - General non-specialized tuned profile
    - cpu-partitioning            - Optimize for CPU partitioning
    - default                     - Legacy default tuned profile
    - desktop                     - Optimize for the desktop use-case
    - desktop-powersave           - Optmize for the desktop use-case with power saving
    - enterprise-storage          - Legacy profile for RHEL6, for RHEL7, please use throughput-performance profile
    - hpc-compute                 - Optimize for HPC compute workloads
    - laptop-ac-powersave         - Optimize for laptop with power savings
    - laptop-battery-powersave    - Optimize laptop profile with more aggressive power saving
    - latency-performance         - Optimize for deterministic performance at the cost of increased power consumption
    - mssql                       - Optimize for MS SQL Server
    - network-latency             - Optimize for deterministic performance at the cost of increased power consumption, focused on low latency network performance
    - network-throughput          - Optimize for streaming network throughput, generally only necessary on older CPUs or 40G+ networks
    - oracle                      - Optimize for Oracle RDBMS
    - powersave                   - Optimize for low power consumption
    - server-powersave            - Optimize for server power savings
    - spindown-disk               - Optimize for power saving by spinning-down rotational disks
    - throughput-performance      - Broadly applicable tuning that provides excellent performance across a variety of common server workloads
    - virtual-guest               - Optimize for running inside a virtual guest
    - virtual-host                - Optimize for running KVM guests
Current active profile: latency-performance

答案2

与 SMB 相比，FTP 在传输大文件时速度极快且高效。对于小文件来说可能会比较困难，但总的来说，FTP 文件传输协议的速度更好。在 SMB 中使用短消息会使网络延迟变得敏感，从而降低速度。

从：https://cloudinfrastructureservices.co.uk/ftp-vs-smb-whats-the-difference-performance-speed-security/#:~:text=FTP%20is%20extremely%20fast%20and,which%20can%20decrease% 20%20速度。

相关内容