.tif我的目录中有大约 130000 个文件。我需要使用超正方体从中提取文本。我创建了一个tmpfs内存驱动器并将所有文件移到那里。下面是我的一张内页：
printf "%03d\n" {1..163} | parallel find . -name 'k-{}\*' -print0 | parallel -j100 -0 'tesseract {/} /mnt/ramdisk/output/{/.} > /dev/null 2>&1'
我的机器是运行 Ubuntu 16.04 的云虚拟机，具有 4 个 vCPU 核心和 16GB RAM。尽管我并行运行超过 100 个作业，但每分钟无法获取超过 25-30 个文件。
磁盘 I/O 是瓶颈吗？我对这个话题几乎一无所知。如何每分钟运行更多文件？
下面是ps -e输出：
21002 pts/0 00:00:06 tesseract <defunct>
21010 pts/0 00:00:05 tesseract <defunct>
21023 pts/0 00:00:06 tesseract <defunct>
21035 pts/0 00:00:04 tesseract <defunct>
21039 pts/0 00:00:06 tesseract <defunct>
21040 pts/0 00:00:04 tesseract <defunct>
21044 pts/0 00:00:00 tesseract <defunct>
21056 pts/0 00:00:04 tesseract <defunct>
21065 pts/0 00:00:05 tesseract <defunct>
21067 pts/0 00:00:06 tesseract <defunct>
21068 pts/0 00:00:06 tesseract <defunct>
21080 pts/0 00:00:06 tesseract <defunct>
21081 pts/0 00:00:06 tesseract <defunct>
21082 pts/0 00:00:05 tesseract <defunct>
21083 pts/0 00:00:06 tesseract <defunct>
21084 pts/0 00:00:06 tesseract <defunct>
21085 pts/0 00:00:00 tesseract <defunct>
21087 pts/0 00:00:06 tesseract <defunct>
21090 pts/0 00:00:05 tesseract <defunct>
21094 pts/0 00:00:00 tesseract <defunct>
21099 pts/0 00:00:05 tesseract <defunct>
21100 pts/0 00:00:02 tesseract <defunct>
21102 pts/0 00:00:06 tesseract <defunct>
21103 pts/0 00:00:05 tesseract <defunct>
21104 pts/0 00:00:06 tesseract <defunct>
21106 pts/0 00:00:08 tesseract <defunct>
21107 pts/0 00:00:06 tesseract <defunct>
21108 pts/0 00:00:04 tesseract <defunct>
21110 pts/0 00:00:06 tesseract <defunct>
21111 pts/0 00:00:08 tesseract <defunct>
21112 pts/0 00:00:06 tesseract <defunct>
21113 pts/0 00:00:04 tesseract <defunct>
21114 pts/0 00:00:06 tesseract <defunct>
21115 pts/0 00:00:06 tesseract <defunct>
21117 pts/0 00:00:01 tesseract <defunct>
21118 pts/0 00:00:00 tesseract <defunct>
21119 pts/0 00:00:04 tesseract <defunct>
21128 pts/0 00:00:05 tesseract <defunct>
21129 pts/0 00:00:00 tesseract <defunct>
21134 pts/0 00:00:06 tesseract <defunct>
21135 pts/0 00:00:04 tesseract <defunct>
21137 pts/0 00:00:06 tesseract <defunct>
21138 pts/0 00:00:00 tesseract <defunct>
21141 pts/0 00:00:06 tesseract <defunct>
21144 pts/0 00:00:05 tesseract <defunct>
21145 pts/0 00:00:06 tesseract <defunct>
21150 pts/0 00:00:06 tesseract <defunct>
21151 pts/0 00:00:02 tesseract <defunct>
21152 pts/0 00:00:06 tesseract <defunct>
21154 pts/0 00:00:00 tesseract <defunct>
21155 pts/0 00:00:05 tesseract <defunct>
21158 pts/0 00:00:05 tesseract <defunct>
21159 pts/0 00:00:06 tesseract <defunct>
21160 pts/0 00:00:06 tesseract <defunct>
21165 pts/0 00:00:06 tesseract <defunct>
21167 pts/0 00:00:05 tesseract <defunct>
21173 pts/0 00:00:00 tesseract <defunct>
21175 pts/0 00:00:00 tesseract <defunct>
21181 pts/0 00:00:06 tesseract <defunct>
21184 pts/0 00:00:05 tesseract <defunct>
21185 pts/0 00:00:06 tesseract <defunct>
21187 pts/0 00:00:05 tesseract <defunct>
21191 pts/0 00:00:06 tesseract <defunct>
21196 pts/0 00:00:00 tesseract <defunct>
21197 pts/0 00:00:05 tesseract <defunct>
21198 pts/0 00:00:06 tesseract <defunct>
21199 pts/0 00:00:08 tesseract
21200 pts/0 00:00:00 tesseract <defunct>
21201 pts/0 00:00:06 tesseract <defunct>
21202 pts/0 00:00:03 tesseract <defunct>
21204 pts/0 00:00:08 tesseract
21207 pts/0 00:00:06 tesseract <defunct>
21210 pts/0 00:00:00 tesseract <defunct>
21214 pts/0 00:00:00 tesseract <defunct>
21215 pts/0 00:00:06 tesseract <defunct>
21217 pts/0 00:00:05 tesseract <defunct>
21218 pts/0 00:00:03 tesseract <defunct>
21221 pts/0 00:00:06 tesseract <defunct>
21224 ? 00:00:00 kworker/0:1
21229 pts/0 00:00:06 tesseract <defunct>
21239 pts/0 00:00:06 tesseract <defunct>
21240 pts/0 00:00:04 tesseract <defunct>
21241 pts/0 00:00:06 tesseract <defunct>
21242 pts/0 00:00:02 tesseract <defunct>
21245 pts/0 00:00:06 tesseract <defunct>
21246 pts/0 00:00:05 tesseract <defunct>
21247 pts/0 00:00:04 tesseract <defunct>
21248 pts/0 00:00:00 tesseract <defunct>
21250 pts/0 00:00:06 tesseract <defunct>
21251 pts/0 00:00:05 tesseract <defunct>
21252 pts/0 00:00:01 tesseract <defunct>
21253 pts/0 00:00:06 tesseract <defunct>
21265 pts/0 00:00:06 tesseract <defunct>
21268 pts/0 00:00:06 tesseract <defunct>
21269 pts/0 00:00:06 tesseract <defunct>
21270 pts/0 00:00:01 tesseract <defunct>
21277 pts/0 00:00:06 tesseract <defunct>
21285 pts/0 00:00:05 tesseract
21289 pts/0 00:00:07 tesseract <defunct>
21291 pts/0 00:00:07 tesseract
21292 pts/0 00:00:06 tesseract
21293 pts/0 00:00:06 tesseract
21297 pts/0 00:00:06 tesseract
21299 pts/0 00:00:05 tesseract
21300 pts/0 00:00:06 tesseract <defunct>
21302 pts/0 00:00:05 tesseract
21303 pts/0 00:00:06 tesseract <defunct>
21308 pts/0 00:00:05 tesseract <defunct>
21320 pts/0 00:00:05 tesseract
21322 pts/0 00:00:00 tesseract <defunct>
21324 pts/0 00:00:05 tesseract <defunct>
21329 pts/0 00:00:05 tesseract <defunct>
21330 pts/0 00:00:05 tesseract <defunct>
21333 pts/0 00:00:05 tesseract
21334 pts/0 00:00:05 tesseract
21335 pts/0 00:00:06 tesseract
21337 pts/0 00:00:00 tesseract <defunct>
21338 pts/0 00:00:04 tesseract <defunct>
21340 pts/0 00:00:04 tesseract
21341 pts/0 00:00:05 tesseract
21344 pts/0 00:00:05 tesseract
21353 pts/0 00:00:06 tesseract <defunct>
21362 pts/0 00:00:06 tesseract <defunct>
21363 pts/0 00:00:00 tesseract <defunct>
21366 pts/0 00:00:03 tesseract
21372 pts/0 00:00:04 tesseract
21373 pts/0 00:00:04 tesseract
21377 pts/0 00:00:00 tesseract <defunct>
21378 pts/0 00:00:05 tesseract
21382 pts/0 00:00:05 tesseract <defunct>
21384 pts/0 00:00:04 tesseract
21385 pts/0 00:00:05 tesseract
21391 pts/0 00:00:05 tesseract <defunct>
21393 pts/0 00:00:01 tesseract <defunct>
21394 pts/0 00:00:04 tesseract
21396 pts/0 00:00:01 tesseract <defunct>
21397 pts/0 00:00:02 tesseract <defunct>
21399 pts/0 00:00:05 tesseract
21401 pts/0 00:00:06 tesseract <defunct>
21403 pts/0 00:00:00 tesseract <defunct>
21404 pts/0 00:00:01 tesseract <defunct>
21405 pts/0 00:00:00 tesseract <defunct>
21407 pts/0 00:00:04 tesseract
21408 pts/0 00:00:00 tesseract <defunct>
21409 pts/0 00:00:05 tesseract
21413 pts/0 00:00:02 tesseract <defunct>
21414 pts/0 00:00:04 tesseract
21415 pts/0 00:00:05 tesseract
21416 pts/0 00:00:04 tesseract
21417 pts/0 00:00:00 tesseract <defunct>
21418 pts/0 00:00:03 tesseract
21419 pts/0 00:00:03 tesseract
21422 pts/0 00:00:03 tesseract
21424 pts/0 00:00:04 tesseract
21426 pts/0 00:00:03 tesseract
21427 pts/0 00:00:04 tesseract
21429 pts/0 00:00:05 tesseract
21434 pts/0 00:00:01 tesseract <defunct>
21437 pts/0 00:00:04 tesseract
21438 pts/0 00:00:00 nest.sh
21440 pts/0 00:00:02 tesseract
21445 pts/0 00:00:03 tesseract
21446 pts/0 00:00:04 tesseract
21447 pts/0 00:00:04 tesseract
21448 pts/0 00:00:02 tesseract
21449 pts/0 00:00:03 tesseract
21450 pts/0 00:00:04 tesseract
21451 pts/0 00:00:04 tesseract
21452 pts/0 00:00:02 tesseract <defunct>
21454 pts/0 00:00:03 tesseract
21455 pts/0 00:00:03 tesseract
21457 pts/0 00:00:00 perl
21467 ? 00:00:00 kworker/2:2
21471 pts/0 00:00:03 tesseract
21472 pts/0 00:00:03 tesseract
21475 pts/0 00:00:00 tesseract <defunct>
21480 ? 00:00:00 kworker/u8:2
21485 pts/0 00:00:02 tesseract
21487 pts/0 00:00:03 tesseract
21488 pts/0 00:00:02 tesseract
21489 pts/0 00:00:03 tesseract
21493 pts/0 00:00:02 tesseract
21496 pts/0 00:00:03 tesseract
21498 pts/0 00:00:03 tesseract
21501 pts/0 00:00:03 tesseract
21508 pts/0 00:00:02 tesseract <defunct>
21510 pts/0 00:00:03 tesseract
21516 pts/0 00:00:00 tesseract <defunct>
21519 pts/0 00:00:02 tesseract
21521 pts/0 00:00:03 tesseract
21523 pts/0 00:00:02 tesseract
21531 pts/0 00:00:02 tesseract
21532 pts/0 00:00:02 tesseract
21536 pts/0 00:00:00 tesseract <defunct>
21537 pts/0 00:00:02 tesseract
21543 pts/0 00:00:01 tesseract
21545 pts/0 00:00:02 tesseract
21549 pts/0 00:00:02 tesseract
21553 pts/0 00:00:02 tesseract
21556 pts/0 00:00:00 tesseract <defunct>
21560 pts/0 00:00:00 tesseract
21561 pts/0 00:00:01 tesseract
21562 pts/0 00:00:01 tesseract
21566 pts/0 00:00:01 tesseract
21569 pts/0 00:00:01 tesseract
21570 pts/0 00:00:01 tesseract
21575 pts/0 00:00:00 tesseract <defunct>
21582 pts/0 00:00:02 tesseract
21583 pts/0 00:00:00 tesseract
21585 pts/0 00:00:01 tesseract
21586 pts/0 00:00:01 tesseract
21590 pts/0 00:00:01 tesseract
21595 pts/0 00:00:00 tesseract <defunct>
21596 pts/0 00:00:00 tesseract
21597 pts/0 00:00:01 tesseract
21598 pts/0 00:00:01 tesseract
21599 pts/0 00:00:00 tesseract
21600 pts/0 00:00:00 tesseract <defunct>
21606 pts/0 00:00:00 tesseract
21610 pts/0 00:00:00 tesseract
21611 pts/0 00:00:01 tesseract
21612 pts/0 00:00:00 tesseract
21615 pts/0 00:00:01 tesseract
21620 pts/0 00:00:01 tesseract
21621 pts/0 00:00:00 tesseract
21622 pts/0 00:00:00 tesseract
21624 pts/0 00:00:00 tesseract
21625 pts/0 00:00:00 tesseract
21626 pts/0 00:00:00 tesseract
21639 pts/0 00:00:01 tesseract
21641 pts/0 00:00:01 tesseract
21646 ? 00:00:00 mysql-systemd-s
21648 pts/0 00:00:00 tesseract
21653 pts/0 00:00:00 tesseract
21654 pts/0 00:00:00 tesseract
21655 pts/0 00:00:00 tesseract
21660 pts/0 00:00:00 tesseract
21665 pts/0 00:00:00 tesseract
21666 pts/0 00:00:00 tesseract
21670 pts/0 00:00:00 tesseract
21671 pts/0 00:00:00 tesseract
21674 pts/0 00:00:00 tesseract
21675 pts/0 00:00:00 tesseract
21677 pts/0 00:00:00 tesseract
21682 pts/0 00:00:00 tesseract
21683 pts/0 00:00:00 tesseract
21684 pts/0 00:00:00 tesseract
21688 pts/0 00:00:00 tesseract
21690 pts/0 00:00:00 tesseract
21691 pts/0 00:00:00 tesseract
21692 pts/0 00:00:00 tesseract
21693 pts/0 00:00:00 tesseract
21696 pts/0 00:00:00 tesseract
21697 pts/0 00:00:00 tesseract
21699 pts/0 00:00:00 tesseract
21703 pts/0 00:00:00 tesseract
21705 pts/0 00:00:00 tesseract
21708 pts/0 00:00:00 tesseract
21709 pts/0 00:00:00 tesseract
21710 pts/0 00:00:00 tesseract
21711 pts/0 00:00:00 tesseract
21712 pts/0 00:00:00 tesseract
21713 pts/0 00:00:00 tesseract
21714 pts/0 00:00:00 tesseract
21716 pts/0 00:00:00 tesseract
21717 pts/0 00:00:00 tesseract
21720 pts/0 00:00:00 tesseract
21724 pts/0 00:00:00 tesseract
21727 pts/0 00:00:00 tesseract
21730 pts/0 00:00:00 tesseract
21732 pts/0 00:00:00 tesseract
21733 pts/0 00:00:00 tesseract
21736 pts/0 00:00:00 tesseract
21739 pts/0 00:00:00 tesseract
21742 pts/0 00:00:00 tesseract
21745 pts/0 00:00:00 tesseract
21752 pts/0 00:00:00 tesseract
21755 pts/0 00:00:00 tesseract
21756 pts/0 00:00:00 tesseract
21758 pts/0 00:00:00 tesseract
21760 pts/0 00:00:00 tesseract
21762 pts/0 00:00:00 tesseract
21764 pts/0 00:00:00 tesseract
21765 pts/0 00:00:00 tesseract
21766 pts/0 00:00:00 tesseract
21769 pts/0 00:00:00 tesseract
21773 pts/0 00:00:00 tesseract
21778 pts/0 00:00:00 tesseract
21779 pts/0 00:00:00 tesseract
21780 pts/0 00:00:00 tesseract
21782 pts/0 00:00:00 tesseract
21784 pts/0 00:00:00 sleep
21785 pts/0 00:00:00 sleep
21786 pts/0 00:00:00 sleep
21787 pts/0 00:00:00 tesseract
21788 pts/0 00:00:00 bash
21789 pts/0 00:00:00 sleep
21790 pts/0 00:00:00 sleep
21791 pts/0 00:00:00 sleep
21792 pts/0 00:00:00 sleep
21794 pts/0 00:00:00 sleep
21795 pts/0 00:00:00 tesseract
21796 pts/0 00:00:00 tesseract
21801 pts/0 00:00:00 tesseract
21803 pts/0 00:00:00 bash
21804 pts/0 00:00:00 bash
21807 pts/0 00:00:00 sleep
21808 pts/0 00:00:00 tesseract
21809 pts/0 00:00:00 tesseract
21810 pts/0 00:00:00 bash
21811 pts/0 00:00:00 sleep
21812 pts/0 00:00:00 sleep
21813 pts/0 00:00:00 perl
21814 ? 00:00:00 sshd
21815 pts/0 00:00:00 perl
21816 ? 00:00:00 sleep
21817 pts/1 00:00:00 ps
答案1
如果您已经使用 tmpfs，那么您的 IO 应该不是瓶颈。您的脚本中明显错误的是您试图同时运行太多任务。由于您的任务不应该因 tmpfs 而等待任何事情，因此您应该尝试使用等于您拥有的核心数量的并行作业数量，可能是+1。另外，将您的测试集减少到 200 个文件，这样您就可以更快地进行测试。
答案2
您应该尝试确定导致此 I/O 的原因。读写不会tmpfs引起 I/O：
:> cd /run/user/1000
:> LC_ALL=C df -h .
Filesystem Size Used Avail Use% Mounted on
tmpfs 2.4G 128M 2.3G 6% /run/user/1000
:> dd if=/dev/urandom of=bigfile bs=1M count=100
:> ls -lh bigfile
-rw-r--r-- 1 hl hauke 100M 30. Dez 16:07 bigfile
:> while true; do cat <bigfile >bigfilecopy || break; echo -n .; done
其读取和写入数据的速率超过 1 GiB/s，但top显示 0.0% I/O 等待。
启动进程应该不是问题，因为这种情况每秒只发生 25 次，并且无论如何都应该被页面缓存覆盖。
如果您无法找到（并消除）I/O 的原因，那么您应该比较不同的少量并行进程的 I/O 等待。正如您已经被告知的那样，拥有比 CPU 核心更多的进程通常是没有意义的。但也许您的两个或三个进程的 I/O 等待时间非常低，而四个进程的负载则高得多，因此仅使用三个并行进程可能会更快。