Postgres 运行非常慢

Postgres 运行非常慢

您好,感谢您的阅读。

我正在开发一个分布式系统,有 8 个集群,每个集群都有一个 Postgres 实例。但是,其中一个 Postgres 实例与其他实例相比写入数据的速度非常慢。所有计算机都运行相同版本的 Ubuntu,具有相同的规格并安装相同的 Postgres 版本。

这是我的做法:

  • 执行一些 Java 代码并通过 JDBC 连接将创建的数据写入数据库。所有操作都发生在一个 Java 实例中。确切的数据各不相同,但规模相同。

对于慢速集群,我有以下执行时间(以秒为单位):

Create Data with Java    WriteData
92                       11055

对于其他集群,我有更多类似这样的内容:

Create Data with Java    WriteData
95                       266

我已经检查过表空间是否有足够的磁盘空间,也完全重新安装了 Postgres 实例,但仍然不知道问题可能是什么。

感谢您对我可以寻找潜在问题的任何建议!

更新:

一些性能参数:

diskSlow: sudo hdparm -t /dev/sda6 
/dev/sda6: Timing buffered disk reads: 76 MB in 3.05 seconds = 24.92 MB/sec

diskOther: sudo hdparm -t /dev/sda6 
/dev/sda6: Timing buffered disk reads: 314 MB in 3.00 seconds = 104.62 MB/sec 

缓存读取的时间是可比较的。

更新2: 好的,我发现慢集群最近已经重新安装了,并且有一个较新的Ubuntu版本。

SlowCluster                       OtherClusters: 
Release:    Ubuntu 10.10          Release:    9.10
Codename:   maverick              Codename:   karmic

lspci 输出:

集群缓慢:

00:00.0 Host bridge: Intel Corporation 4 Series Chipset DRAM Controller (rev 03)
00:02.0 VGA compatible controller: Intel Corporation 4 Series Chipset Integrated Graphics Controller (rev 03)
00:02.1 Display controller: Intel Corporation 4 Series Chipset Integrated Graphics Controller (rev 03)
00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 1
00:1c.5 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Root Port 6
00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIB (ICH10) LPC Interface Controller
00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller #1
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller #2
01:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 01)

聚类其他:

00:00.0 Host bridge: Intel Corporation 4 Series Chipset DRAM Controller (rev 03)
00:01.0 PCI bridge: Intel Corporation 4 Series Chipset PCI Express Root Port (rev 03)
00:1a.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #4
00:1a.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #5
00:1a.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #6
00:1a.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #2
00:1b.0 Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller
00:1c.0 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 1
00:1c.5 PCI bridge: Intel Corporation 82801JI (ICH10 Family) PCI Express Port 6
00:1d.0 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #1
00:1d.1 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #2
00:1d.2 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB UHCI Controller #3
00:1d.7 USB Controller: Intel Corporation 82801JI (ICH10 Family) USB2 EHCI Controller #1
00:1e.0 PCI bridge: Intel Corporation 82801 PCI Bridge (rev 90)
00:1f.0 ISA bridge: Intel Corporation 82801JIB (ICH10) LPC Interface Controller
00:1f.2 IDE interface: Intel Corporation 82801JI (ICH10 Family) 4 port SATA IDE Controller
00:1f.3 SMBus: Intel Corporation 82801JI (ICH10 Family) SMBus Controller
00:1f.5 IDE interface: Intel Corporation 82801JI (ICH10 Family) 2 port SATA IDE Controller
01:00.0 VGA compatible controller: nVidia Corporation GT200 [GeForce GT 220] (rev a2)
01:00.1 Audio device: nVidia Corporation Device 0be2 (rev a1)
02:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 01)
03:00.0 Network controller: RaLink RT2860

两者之间的主要区别:

03:00.0 Network controller: RaLink RT2860
01:00.1 Audio device: nVidia Corporation Device 0be2 (rev a1)
00:01.0 PCI bridge: Intel Corporation 4 Series Chipset PCI Express Root Port (rev 03)

来自 lshw:

集群缓慢

width: 64 bits
    capabilities: vsyscall64 vsyscall32
  *-core
       description: Motherboard
       physical id: 0
     *-memory
          description: System memory
          physical id: 0
          size: 7967MiB
     *-cpu
          product: Intel(R) Core(TM)2 Quad CPU    Q8300  @ 2.50GHz
          vendor: Intel Corp.
          physical id: 1
          bus info: cpu@0
          size: 2003MHz
          capacity: 2003MHz
          width: 64 bits
          capabilities: fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx x86-64 constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm cpufreq
 *-pci
      description: Host bridge
      product: 4 Series Chipset DRAM Controller
      vendor: Intel Corporation
      physical id: 100
      bus info: pci@0000:00:00.0
      version: 03
      width: 32 bits
      clock: 33MHz
      configuration: driver=agpgart-intel
      resources: irq:0

其他集群:

    width: 64 bits
    capabilities: vsyscall64 vsyscall32
  *-core
       description: Motherboard
       physical id: 0
     *-memory
          description: System memory
          physical id: 0
          size: 8960MiB
     *-cpu
          product: Intel(R) Core(TM)2 Quad CPU    Q8300  @ 2.50GHz
          vendor: Intel Corp.
          physical id: 1
          bus info: cpu@0
          size: 2003MHz
          capacity: 2003MHz
          width: 64 bits
          capabilities: fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx x86-64 constant_tsc arch_perfmon pebs bts rep_good pni dtes64 monitor ds_cpl est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm cpufreq
 *-pci
      description: Host bridge
      product: 4 Series Chipset DRAM Controller
      vendor: Intel Corporation
      physical id: 100
      bus info: pci@0000:00:00.0
      version: 03
      width: 32 bits
      clock: 33MHz
    *-pci:0
         description: PCI bridge
         product: 4 Series Chipset PCI Express Root Port
         vendor: Intel Corporation
         physical id: 1
         bus info: pci@0000:00:01.0
         version: 03
         width: 32 bits
         clock: 33MHz
         capabilities: pci bus_master cap_list
         configuration: driver=pcieport-driver
         resources: irq:24 ioport:d000(size=4096) memory:fd000000-fe9fffff ioport:de000000(size=301989888)
       *-display UNCLAIMED
            description: VGA compatible controller
            product: GT200 [GeForce GT 220]
            vendor: nVidia Corporation
            physical id: 0
            bus info: pci@0000:01:00.0
            version: a2
            width: 64 bits
            clock: 33MHz
            capabilities: bus_master cap_list
            configuration: latency=0
            resources: memory:fd000000-fdffffff memory:e0000000-efffffff(prefetchable) memory:de000000-dfffffff(prefetchable) ioport:dc00(size=128) memory:fe900000-fe97ffff(prefetchable)

我将不胜感激任何提示。所有集群都具有(几乎)相同的硬件。可能某些驱动程序或配置有所不同,但我对这些主题并不熟悉。请告诉我该怎么做。

谢谢!

答案1

较新的 Linux 内核(例如 Ubuntu 10.10 中包含的内核)使用不同的方法在数据库写入时将信息刷新到驱动器。当数据库进行常规 fsync 调用之一时,Linux 使用所谓的写入屏障,每次您提交事务时都会发生这种情况。您的一个集群很慢,因为较新的写入屏障代码会更小心地将信息从各个硬盘驱动器写入缓存中推出。较旧的 Linux 系统让信息留在驱动器的缓存中,即使在数据库请求写入之后也是如此。

可靠写入有关此主题的更多信息。较新、较慢的集群实际上在可靠的数据库写入方面做得正确。如果配置崩溃,较快的系统可能会丢失数据,而较慢的系统则不会出现此问题。

如果您担心崩溃时数据丢失的可能性,那么您应该更担心较快的系统而不是较慢的系统。无论如何,您都可以通过以下两种简单的方法加快较慢的系统的速度:

  • 编辑 postgresql.conf 文件,关闭参数synchronous_commit,然后重新加载服务器。这将禁用数据库要求操作系统刷新数据的功能,因此禁用写屏障刷新功能,该功能在一台服务器上的性能差异很大。这可以避免新机器上的数据库损坏问题(旧机器仍然有风险),但您可能会在崩溃时丢失一些数据。

  • 使用“nobarrier”选项重新安装文件系统。在这里,您将再次面临崩溃期间数据库损坏的可能性,就像您的旧系统一样。

您可能想查看代码,看看是否可以减少向数据库提交数据的频率。有时,如果您一次以大块行的形式进行数据写入,数据写入速度会更快。从您的问题来看,我猜您可能在每次写入数据库后都提交数据。

相关内容