您好,感谢您的阅读。
我正在开发一个分布式系统,有 8 个集群,每个集群都有一个 Postgres 实例。但是,其中一个 Postgres 实例与其他实例相比写入数据的速度非常慢。所有计算机都运行相同版本的 Ubuntu,具有相同的规格并安装相同的 Postgres 版本。
这是我的做法:
- 执行一些 Java 代码并通过 JDBC 连接将创建的数据写入数据库。所有操作都发生在一个 Java 实例中。确切的数据各不相同,但规模相同。
对于慢速集群,我有以下执行时间(以秒为单位):
Create Data with Java WriteData
92 11055
对于其他集群,我有更多类似这样的内容:
Create Data with Java WriteData
95 266
我已经检查过表空间是否有足够的磁盘空间,也完全重新安装了 Postgres 实例,但仍然不知道问题可能是什么。
感谢您对我可以寻找潜在问题的任何建议!
更新:
一些性能参数:
diskSlow: sudo hdparm -t /dev/sda6
/dev/sda6: Timing buffered disk reads: 76 MB in 3.05 seconds = 24.92 MB/sec
diskOther: sudo hdparm -t /dev/sda6
/dev/sda6: Timing buffered disk reads: 314 MB in 3.00 seconds = 104.62 MB/sec
缓存读取的时间是可比较的。
更新2: 好的,我发现慢集群最近已经重新安装了,并且有一个较新的Ubuntu版本。
SlowCluster OtherClusters:
Release: Ubuntu 10.10 Release: 9.10
Codename: maverick Codename: karmic
lspci 输出:
集群缓慢:
00:00.0 Host bridge: Intel Corporation 4 Series Chipset DRAM Controller (rev 03)
00:02.0 VGA compatible controller: Intel Corporation 4 Series Chipset Integrated Graphics Controller (rev 03)
00:02.1 Display controller: Intel Corporation 4 Series Chipset Integrated Graphics Controller (rev 03)
00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
00:1c.5 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 6
00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIB (ICH10) LPC Interface Controller
00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller #1
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller #2
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 01)
聚类其他:
00:00.0 Host bridge: Intel Corporation 4 Series Chipset DRAM Controller (rev 03)
00:01.0 PCI bridge: Intel Corporation 4 Series Chipset PCI Express Root Port (rev 03)
00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 1
00:1c.5 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 6
00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIB (ICH10) LPC Interface Controller
00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller
01:00.0 VGA compatible controller: nVidia Corporation GT200 [GeForce GT 220] (rev a2)
01:00.1 Audio device: nVidia Corporation Device 0be2 (rev a1)
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 01)
03:00.0 Network controller: RaLink RT2860
两者之间的主要区别:
03:00.0 Network controller: RaLink RT2860
01:00.1 Audio device: nVidia Corporation Device 0be2 (rev a1)
00:01.0 PCI bridge: Intel Corporation 4 Series Chipset PCI Express Root Port (rev 03)
来自 lshw:
集群缓慢
width: 64 bits
capabilities: vsyscall64 vsyscall32
*-core
description: Motherboard
physical id: 0
*-memory
description: System memory
physical id: 0
size: 7967MiB
*-cpu
product: Intel(R) Core(TM)2 Quad CPU Q8300 @ 2.50GHz
vendor: Intel Corp.
physical id: 1
bus info: cpu@0
size: 2003MHz
capacity: 2003MHz
width: 64 bits
capabilities: fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx x86-64 constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm cpufreq
*-pci
description: Host bridge
product: 4 Series Chipset DRAM Controller
vendor: Intel Corporation
physical id: 100
bus info: pci@0000:00:00.0
version: 03
width: 32 bits
clock: 33MHz
configuration: driver=agpgart-intel
resources: irq:0
其他集群:
width: 64 bits
capabilities: vsyscall64 vsyscall32
*-core
description: Motherboard
physical id: 0
*-memory
description: System memory
physical id: 0
size: 8960MiB
*-cpu
product: Intel(R) Core(TM)2 Quad CPU Q8300 @ 2.50GHz
vendor: Intel Corp.
physical id: 1
bus info: cpu@0
size: 2003MHz
capacity: 2003MHz
width: 64 bits
capabilities: fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx x86-64 constant_tsc arch_perfmon pebs bts rep_good pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm cpufreq
*-pci
description: Host bridge
product: 4 Series Chipset DRAM Controller
vendor: Intel Corporation
physical id: 100
bus info: pci@0000:00:00.0
version: 03
width: 32 bits
clock: 33MHz
*-pci:0
description: PCI bridge
product: 4 Series Chipset PCI Express Root Port
vendor: Intel Corporation
physical id: 1
bus info: pci@0000:00:01.0
version: 03
width: 32 bits
clock: 33MHz
capabilities: pci bus_master cap_list
configuration: driver=pcieport-driver
resources: irq:24 ioport:d000(size=4096) memory:fd000000-fe9fffff ioport:de000000(size=301989888)
*-display UNCLAIMED
description: VGA compatible controller
product: GT200 [GeForce GT 220]
vendor: nVidia Corporation
physical id: 0
bus info: pci@0000:01:00.0
version: a2
width: 64 bits
clock: 33MHz
capabilities: bus_master cap_list
configuration: latency=0
resources: memory:fd000000-fdffffff memory:e0000000-efffffff(prefetchable) memory:de000000-dfffffff(prefetchable) ioport:dc00(size=128) memory:fe900000-fe97ffff(prefetchable)
我将不胜感激任何提示。所有集群都具有(几乎)相同的硬件。可能某些驱动程序或配置有所不同,但我对这些主题并不熟悉。请告诉我该怎么做。
谢谢!
答案1
较新的 Linux 内核(例如 Ubuntu 10.10 中包含的内核)使用不同的方法在数据库写入时将信息刷新到驱动器。当数据库进行常规 fsync 调用之一时,Linux 使用所谓的写入屏障,每次您提交事务时都会发生这种情况。您的一个集群很慢,因为较新的写入屏障代码会更小心地将信息从各个硬盘驱动器写入缓存中推出。较旧的 Linux 系统让信息留在驱动器的缓存中,即使在数据库请求写入之后也是如此。
看可靠写入有关此主题的更多信息。较新、较慢的集群实际上在可靠的数据库写入方面做得正确。如果配置崩溃,较快的系统可能会丢失数据,而较慢的系统则不会出现此问题。
如果您担心崩溃时数据丢失的可能性,那么您应该更担心较快的系统而不是较慢的系统。无论如何,您都可以通过以下两种简单的方法加快较慢的系统的速度:
编辑 postgresql.conf 文件,关闭参数synchronous_commit,然后重新加载服务器。这将禁用数据库要求操作系统刷新数据的功能,因此禁用写屏障刷新功能,该功能在一台服务器上的性能差异很大。这可以避免新机器上的数据库损坏问题(旧机器仍然有风险),但您可能会在崩溃时丢失一些数据。
使用“nobarrier”选项重新安装文件系统。在这里,您将再次面临崩溃期间数据库损坏的可能性,就像您的旧系统一样。
您可能想查看代码,看看是否可以减少向数据库提交数据的频率。有时,如果您一次以大块行的形式进行数据写入,数据写入速度会更快。从您的问题来看,我猜您可能在每次写入数据库后都提交数据。