我有一台虚拟服务器(AWS EC2 实例 c5a.4xlarge),上面装有 Ubuntu 20.04,用于运行 Playwright 脚本。我想执行大量脚本实例,但有些东西限制了它。服务器的资源有一半以上可用,但我无法启动超过一定数量的正在运行的实例。
我尝试通过在 /etc/security/limits.conf 中添加以下行来增加限制:
* soft sigpending 12701300
* hard sigpending 12701300
ubuntu soft sigpending 12701300
ubuntu hard sigpending 12701300
* soft memlock 6553600
* hard memlock 6553600
ubuntu soft memlock 6553600
ubuntu hard memlock 6553600
* soft nofile 102400
* hard nofile 102400
ubuntu soft nofile 102400
ubuntu hard nofile 102400
* soft nproc 12701300
* hard nproc 12701300
ubuntu soft nproc 12701300
ubuntu hard nproc 12701300
* soft stack 819200
* hard stack 819200
ubuntu soft stack 819200
ubuntu hard stack 819200
ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 12701300
max locked memory (kbytes, -l) 6553600
max memory size (kbytes, -m) unlimited
open files (-n) 102400
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 819200
cpu time (seconds, -t) unlimited
max user processes (-u) 12701300
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
但我仍然无法启动更多实例。
更新详情
我可以启动大约 40 个 Playwright 实例(headless:false)。CPU 负载约为 15%,RAM 负载约为 25%。当我尝试启动更多实例时,通常会抛出 2 个错误:
错误 1 - 常见错误
(node:84194) UnhandledPromiseRejectionWarning: browserType.launch: Protocol error (Browser.getVersion): Target closed.
=========================== logs ===========================
<launching> /home/ubuntu/.cache/ms-playwright/chromium-815036/chrome-linux/chrome --disable-background-networking --enable-features=NetworkService,NetworkServiceInProcess --disable-background-timer-throttling --disable-backgrounding-occluded-windows --disable-breakpad --disable-client-side-phishing-detection --disable-component-extensions-with-background-pages --disable-default-apps --disable-dev-shm-usage --disable-extensions --disable-features=TranslateUI,BlinkGenPropertyTrees,ImprovedCookieControls,SameSiteByDefaultCookies,LazyFrameLoading --disable-hang-monitor --disable-ipc-flooding-protection --disable-popup-blocking --disable-prompt-on-repost --disable-renderer-backgrounding --disable-sync --force-color-profile=srgb --metrics-recording-only --no-first-run --enable-automation --password-store=basic --use-mock-keychain --user-data-dir=/tmp/playwright_chromiumdev_profile-5Vv8kA --remote-debugging-pipe --no-sandbox --proxy-server=http://91.206.68.101:52068 --no-startup-window
<launched> pid=85115
[err] Maximum number of clients reached[85115:85115:1030/092321.045583:ERROR:browser_main_loop.cc(1434)] Unable to open X display.
============================================================
Note: use DEBUG=pw:api environment variable and rerun to capture Playwright logs.
at Connection.sendMessageToServer (/home/ubuntu/projectDir/node_modules/playwright-chromium/lib/client/connection.js:69:15)
at Proxy.<anonymous> (/home/ubuntu/projectDir/node_modules/playwright-chromium/lib/client/channelOwner.js:54:53)
at /home/ubuntu/projectDir/node_modules/playwright-chromium/lib/client/browserType.js:62:73
at BrowserType._wrapApiCall (/home/ubuntu/projectDir/node_modules/playwright-chromium/lib/client/channelOwner.js:80:34)
at BrowserType.launch (/home/ubuntu/projectDir/node_modules/playwright-chromium/lib/client/browserType.js:53:21)
at MyScript.initBrowser (/home/ubuntu/projectDir/scripts/myscript.js:115:33)
at MyScript.start (/home/ubuntu/projectDir/scripts/myscript.js:44:14)
at processTicksAndRejections (internal/process/task_queues.js:93:5)
(Use `node --trace-warnings ...` to show where the warning was created)
(node:84194) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag `--unhandled-rejections=strict` (see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 2)
(node:84194) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
错误 2 - 罕见
projectDir/node_modules/playwright-chromium/lib/client/connection.js:138
throw new Error(`Cannot find parent object ${parentGuid} to create ${guid}`);
^
Error: Cannot find parent object BrowserContext@542efe1178d54357c723ec96e9394def to create Frame@38a1562630571f41c86fea49af0a4b24
at Connection._createRemoteObject (projectDir/node_modules/playwright-chromium/lib/client/connection.js:138:19)
at Connection.dispatch (projectDir/node_modules/playwright-chromium/lib/client/connection.js:105:18)
at Immediate.<anonymous> (projectDir/node_modules/playwright-chromium/lib/inprocess.js:40:85)
at processImmediate (internal/timers.js:461:21)