使用 docker 部署的 Tomcat 9 在没有任何 webapps 的情况下随机达到 200% CPU

使用 docker 部署的 Tomcat 9 在没有任何 webapps 的情况下随机达到 200% CPU

您好,有一个 Tomcat 9 在 AWS 机器上运行,使用以下 docker compose 文件:

version: '3'
services:
  fstomcat:
    image: tomcat:9
    container_name: fstomcat
    ports:
      - 443:443
    volumes:
      - /opt/tomcat/webapps:/usr/local/tomcat/webapps
      - /opt/tomcat/conf:/usr/local/tomcat/conf
      - /opt/tomcat/logs:/usr/local/tomcat/logs

目前没有 webapps(webapps 为空)。此 EC2 仅具有 Tomcat。没有 Apache,没有其他 Web 服务器或数据库服务器。但是,AWS 报告 CPU 使用率随机飙升。当我进入容器时,java 的 CPU 使用率为 199%。最新一次峰值发生在 2021-06-20 13:30,我当天仅有的日志是:

卡塔利娜:

20-Jun-2021 09:45:04.595 INFO [https-openssl-nio-443-exec-6] org.apache.coyote.http11.Http11Processor.service Error parsing HTTP request header
 Note: further occurrences of HTTP request parsing errors will be logged at DEBUG level.
        java.lang.IllegalArgumentException: Invalid character found in the request target [/index.php?s=/Index/\think\app/invokefunction&function=call_user_func_$
                at org.apache.coyote.http11.Http11InputBuffer.parseRequestLine(Http11InputBuffer.java:490)
                at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:261)
                at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
                at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:888)
                at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1597)
                at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
                at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
                at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
                at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
                at java.base/java.lang.Thread.run(Thread.java:834)

使用权:

192.241.220.30 - - [20/Jun/2021:00:22:53 +0000] "-" 400 -
74.82.47.4 - - [20/Jun/2021:00:31:59 +0000] "-" 400 -
74.82.47.4 - - [20/Jun/2021:00:32:03 +0000] "-" 400 -
74.82.47.4 - - [20/Jun/2021:00:32:03 +0000] "-" 400 -
74.82.47.4 - - [20/Jun/2021:00:32:03 +0000] "-" 400 -
74.82.47.4 - - [20/Jun/2021:00:32:04 +0000] "-" 400 -
74.82.47.4 - - [20/Jun/2021:00:32:06 +0000] "-" 400 -
74.82.47.4 - - [20/Jun/2021:00:32:08 +0000] "-" 400 -
74.82.47.4 - - [20/Jun/2021:00:33:03 +0000] "-" 400 -
162.216.17.178 - - [20/Jun/2021:00:41:24 +0000] "-" 400 -
128.1.248.42 - - [20/Jun/2021:01:17:50 +0000] "GET / HTTP/1.1" 404 682
192.241.215.206 - - [20/Jun/2021:01:56:40 +0000] "GET /actuator/health HTTP/1.1" 404 682
45.33.79.16 - - [20/Jun/2021:02:19:19 +0000] "-" 400 -
209.17.97.98 - - [20/Jun/2021:02:57:39 +0000] "-" 400 -
162.216.17.71 - - [20/Jun/2021:04:19:13 +0000] "-" 400 -
45.83.67.150 - - [20/Jun/2021:04:58:00 +0000] "-" 400 -
66.240.205.34 - - [20/Jun/2021:06:08:25 +0000] "-" 400 -
45.33.79.16 - - [20/Jun/2021:06:18:56 +0000] "-" 400 -
162.62.123.46 - - [20/Jun/2021:08:04:09 +0000] "GET / HTTP/1.0" 404 682
192.241.218.53 - - [20/Jun/2021:08:12:25 +0000] "GET /owa/auth/logon.aspx?url=https%3a%2f%2f1%2fecp%2f HTTP/1.1" 404 682
162.216.17.71 - - [20/Jun/2021:08:18:54 +0000] "-" 400 -
45.146.165.123 - - [20/Jun/2021:09:44:57 +0000] "POST /Autodiscover/Autodiscover.xml HTTP/1.1" 404 682
45.146.165.123 - - [20/Jun/2021:09:44:59 +0000] "GET /_ignition/execute-solution HTTP/1.1" 404 682
45.146.165.123 - - [20/Jun/2021:09:45:00 +0000] "GET / HTTP/1.1" 404 682
45.146.165.123 - - [20/Jun/2021:09:45:02 +0000] "POST /vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php HTTP/1.1" 404 682
45.146.165.123 - - [20/Jun/2021:09:45:04 +0000] "GET null HTTP/1.1" 400 2273
45.146.165.123 - - [20/Jun/2021:09:45:06 +0000] "GET /vendor/phpunit/phpunit/src/Util/PHP/eval-stdin.php HTTP/1.1" 404 682
45.146.165.123 - - [20/Jun/2021:09:45:06 +0000] "GET /?XDEBUG_SESSION_START=phpstorm HTTP/1.1" 404 682
45.146.165.123 - - [20/Jun/2021:09:45:07 +0000] "GET /wp-content/plugins/wp-file-manager/readme.txt HTTP/1.1" 404 682
45.146.165.123 - - [20/Jun/2021:09:45:08 +0000] "POST /mifs/.;/services/LogService HTTP/1.1" 404 682
45.146.165.123 - - [20/Jun/2021:09:45:09 +0000] "GET /console/ HTTP/1.1" 404 682
45.33.79.16 - - [20/Jun/2021:10:19:25 +0000] "-" 400 -
193.118.53.210 - - [20/Jun/2021:10:20:10 +0000] "GET / HTTP/1.1" 404 682
162.216.17.71 - - [20/Jun/2021:12:19:00 +0000] "-" 400 -
138.68.175.207 - - [20/Jun/2021:13:28:31 +0000] "-" 400 -
138.68.175.207 - - [20/Jun/2021:13:28:35 +0000] "-" 400 -
138.68.175.207 - - [20/Jun/2021:13:28:35 +0000] "-" 400 -
138.68.175.207 - - [20/Jun/2021:13:28:35 +0000] "-" 400 -
138.68.175.207 - - [20/Jun/2021:13:28:35 +0000] "-" 400 -
165.22.86.42 - - [20/Jun/2021:13:56:50 +0000] "-" 400 -
162.142.125.39 - - [20/Jun/2021:14:01:08 +0000] "-" 400 -
162.142.125.39 - - [20/Jun/2021:14:01:10 +0000] "GET / HTTP/1.1" 404 682
162.142.125.39 - - [20/Jun/2021:14:01:10 +0000] "GET / HTTP/1.1" 404 682
60.217.75.69 - - [20/Jun/2021:14:22:42 +0000] "GET / HTTP/1.1" 404 682
172.105.172.151 - - [20/Jun/2021:14:35:22 +0000] "GET /owa/ HTTP/1.1" 404 682
192.241.214.26 - - [20/Jun/2021:15:04:40 +0000] "GET / HTTP/1.1" 404 682
34.90.100.245 - - [20/Jun/2021:15:18:59 +0000] "GET /.env HTTP/1.1" 404 682
34.90.100.245 - - [20/Jun/2021:15:19:00 +0000] "POST / HTTP/1.1" 404 682
128.14.134.170 - - [20/Jun/2021:16:01:33 +0000] "GET / HTTP/1.1" 404 682
97.107.132.27 - - [20/Jun/2021:16:19:28 +0000] "-" 400 -
173.255.234.116 - - [20/Jun/2021:16:30:04 +0000] "-" 400 -
23.90.160.130 - - [20/Jun/2021:16:37:09 +0000] "GET / HTTP/1.1" 404 682
23.95.191.195 - - [20/Jun/2021:16:50:06 +0000] "POST /GponForm/diag_Form?style/ HTTP/1.1" 404 682
162.216.17.71 - - [20/Jun/2021:18:18:33 +0000] "-" 400 -
193.118.53.210 - - [20/Jun/2021:18:29:39 +0000] "GET / HTTP/1.1" 404 682
51.159.23.43 - - [20/Jun/2021:18:44:34 +0000] "GET / HTTP/1.1" 404 682
45.79.168.6 - - [20/Jun/2021:20:19:38 +0000] "-" 400 -
192.241.220.87 - - [20/Jun/2021:20:48:59 +0000] "-" 400 -
192.241.220.87 - - [20/Jun/2021:20:48:59 +0000] "-" 400 -
192.241.220.87 - - [20/Jun/2021:20:48:59 +0000] "-" 400 -
192.241.220.87 - - [20/Jun/2021:20:48:59 +0000] "-" 400 -
192.241.220.87 - - [20/Jun/2021:20:48:59 +0000] "-" 400 -
192.241.220.87 - - [20/Jun/2021:20:48:59 +0000] "-" 400 -
192.241.220.87 - - [20/Jun/2021:20:49:00 +0000] "-" 400 -
192.241.220.87 - - [20/Jun/2021:20:49:00 +0000] "-" 400 -
192.241.212.36 - - [20/Jun/2021:21:03:09 +0000] "-" 400 -
128.14.209.162 - - [20/Jun/2021:21:36:20 +0000] "GET / HTTP/1.1" 404 682
192.241.218.97 - - [20/Jun/2021:22:11:38 +0000] "GET /ecp/Current/exporttool/microsoft.exchange.ediscovery.exporttool.application HTTP/1.1" 404 682
45.79.144.15 - - [20/Jun/2021:22:19:16 +0000] "-" 400 -
162.142.125.40 - - [20/Jun/2021:23:08:05 +0000] "-" 400 -
162.142.125.40 - - [20/Jun/2021:23:08:07 +0000] "GET / HTTP/1.1" 404 682
162.142.125.40 - - [20/Jun/2021:23:08:07 +0000] "GET / HTTP/1.1" 404 682
45.63.12.50 - - [20/Jun/2021:23:49:07 +0000] "-" 400 -

系统日志:

Jun 20 13:00:24 ip-172-30-1-110 systemd-timesyncd[21286]: Network configuration changed, trying to establish connection.
Jun 20 13:00:24 ip-172-30-1-110 systemd-networkd[13629]: ens5: Configured
Jun 20 13:00:24 ip-172-30-1-110 systemd-timesyncd[21286]: Synchronized to time server 91.189.89.198:123 (ntp.ubuntu.com).
Jun 20 13:17:01 ip-172-30-1-110 CRON[21362]: (root) CMD (   cd / && run-parts --report /etc/cron.hourly)
Jun 20 13:30:24 ip-172-30-1-110 systemd-networkd[13629]: ens5: Configured
Jun 20 13:30:24 ip-172-30-1-110 systemd-timesyncd[21286]: Network configuration changed, trying to establish connection.
Jun 20 13:30:24 ip-172-30-1-110 systemd-timesyncd[21286]: Synchronized to time server 91.189.89.198:123 (ntp.ubuntu.com).

服务器的启动方式如下:

23-Jun-2021 17:37:03.904 WARNING [main] org.apache.tomcat.util.digester.SetPropertiesRule.begin Match [Server/Service/Connector] failed to set property [maxSpareThreads] to [75]
23-Jun-2021 17:37:03.999 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version name:   Apache Tomcat/9.0.41
23-Jun-2021 17:37:04.000 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server built:          Dec 3 2020 11:43:00 UTC
23-Jun-2021 17:37:04.001 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Server version number: 9.0.41.0
23-Jun-2021 17:37:04.003 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Name:               Linux
23-Jun-2021 17:37:04.003 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log OS Version:            5.4.0-1029-aws
23-Jun-2021 17:37:04.004 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Architecture:          amd64
23-Jun-2021 17:37:04.004 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Java Home:             /usr/local/openjdk-11
23-Jun-2021 17:37:04.005 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Version:           11.0.10+9
23-Jun-2021 17:37:04.005 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log JVM Vendor:            Oracle Corporation
23-Jun-2021 17:37:04.006 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_BASE:         /usr/local/tomcat
23-Jun-2021 17:37:04.006 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log CATALINA_HOME:         /usr/local/tomcat
23-Jun-2021 17:37:04.007 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.lang=ALL-UNNAMED
23-Jun-2021 17:37:04.008 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.base/java.io=ALL-UNNAMED
23-Jun-2021 17:37:04.008 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: --add-opens=java.rmi/sun.rmi.transport=ALL-UNNAMED
23-Jun-2021 17:37:04.008 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.config.file=/usr/local/tomcat/conf/logging.properties
23-Jun-2021 17:37:04.009 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.util.logging.manager=org.apache.juli.ClassLoaderLogManager
23-Jun-2021 17:37:04.009 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djdk.tls.ephemeralDHKeySize=2048
23-Jun-2021 17:37:04.010 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.protocol.handler.pkgs=org.apache.catalina.webresources
23-Jun-2021 17:37:04.016 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dorg.apache.catalina.security.SecurityListener.UMASK=0027
23-Jun-2021 17:37:04.017 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dignore.endorsed.dirs=
23-Jun-2021 17:37:04.018 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.base=/usr/local/tomcat
23-Jun-2021 17:37:04.018 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Dcatalina.home=/usr/local/tomcat
23-Jun-2021 17:37:04.018 INFO [main] org.apache.catalina.startup.VersionLoggerListener.log Command line argument: -Djava.io.tmpdir=/usr/local/tomcat/temp
23-Jun-2021 17:37:04.025 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent Loaded Apache Tomcat Native library [1.2.25] using APR version [1.6.5].
23-Jun-2021 17:37:04.025 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR capabilities: IPv6 [true], sendfile [true], accept filters [false], random [true].
23-Jun-2021 17:37:04.026 INFO [main] org.apache.catalina.core.AprLifecycleListener.lifecycleEvent APR/OpenSSL configuration: useAprConnector [false], useOpenSSL [true]
23-Jun-2021 17:37:04.030 INFO [main] org.apache.catalina.core.AprLifecycleListener.initializeSSL OpenSSL successfully initialized [OpenSSL 1.1.1d  10 Sep 2019]
23-Jun-2021 17:37:04.634 INFO [main] org.apache.coyote.AbstractProtocol.init Initializing ProtocolHandler ["https-openssl-nio-443"]
23-Jun-2021 17:37:05.225 INFO [main] org.apache.catalina.startup.Catalina.load Server initialization in [1569] milliseconds
23-Jun-2021 17:37:05.324 INFO [main] org.apache.catalina.core.StandardService.startInternal Starting service [Catalina]
23-Jun-2021 17:37:05.324 INFO [main] org.apache.catalina.core.StandardEngine.startInternal Starting Servlet engine: [Apache Tomcat/9.0.41]
23-Jun-2021 17:37:05.365 INFO [main] org.apache.coyote.AbstractProtocol.start Starting ProtocolHandler ["https-openssl-nio-443"]
23-Jun-2021 17:37:05.396 INFO [main] org.apache.catalina.startup.Catalina.start Server startup in [170] milliseconds

我们停用了 EC2 机器上的所有自动更新,以消除 Docker 更新导致此问题的可能性。但我们唯一能做的就是重新启动它。

我想知道是否有人处理过类似的事情,并知道如何纠正它

答案1

您的服务器会定期被机器人网络/病毒扫描漏洞。如果这导致服务中断,您可以使用失败2ban400将 IP 列入黑名单,这会在短时间内导致大量错误。

您可以对错误执行相同的操作404,但请确保仅匹配那些从未存在于您的网站上的请求 URI,否则您可能会禁止搜索引擎爬虫。

相关内容