☰

Spotify Bot 在 NGINX 缓存服务器上使用大量带宽？

tag-icon • tag-icon nginx apache-2.4 bandwidth web-crawler
Spotify Bot 在 NGINX 缓存服务器上使用大量带宽？

我的网站上托管了几个播客，过去几个月我注意到一个令人不安的趋势：我的网站带宽使用量增加了 10 倍，但似乎大部分都是一系列 Google App Server 实例造成的，而不是听众数量的惊人增长。翻看日志，我认为它们中的大多数似乎都自称是“Spotify/1.0”，它每分钟都会多次访问我的服务器并下载不同的旧播客节目。

我想知道这是否与我最近在 Apache 前面实施 NGINX 作为缓存服务器有关，因为峰值与该更改大致一致。我回答 Spotify 机器人的方式是否存在问题？这是他们的索引器的一个已知问题，我需要以某种方式处理吗？

例如：

 35.240.121.201 - - [16/May/2021:12:16:14 -0500] "GET /littlehillschurch/resources/podcasts/20201214.mp3 HTTP/1.1" 200 15004716 "-" "Spotify/1.0" "-"
 35.195.247.67 - - [16/May/2021:12:16:15 -0500] "GET /littlehillschurch/resources/podcasts/20200803.mp3 HTTP/1.1" 200 14141931 "-" "Spotify/1.0" "-"
 146.148.19.22 - - [16/May/2021:12:16:15 -0500] "GET /littlehillschurch/resources/podcasts/20210503.mp3 HTTP/1.1" 200 14243142 "-" "Spotify/1.0" "-"
 35.240.121.201 - - [16/May/2021:12:16:15 -0500] "GET /littlehillschurch/resources/podcasts/20210125.mp3 HTTP/1.1" 200 15050067 "-" "Spotify/1.0" "-"
 35.195.91.128 - - [16/May/2021:12:16:15 -0500] "GET /littlehillschurch/resources/podcasts/20200817.mp3 HTTP/1.1" 200 15266593 "-" "Spotify/1.0" "-"
 35.187.181.74 - - [16/May/2021:12:16:15 -0500] "GET /littlehillschurch/resources/podcasts/20200921.mp3 HTTP/1.1" 200 14607340 "-" "Spotify/1.0" "-"
 35.195.247.67 - - [16/May/2021:12:16:15 -0500] "GET /littlehillschurch/resources/podcasts/20210222.mp3 HTTP/1.1" 200 15279536 "-" "Spotify/1.0" "-"
 35.195.91.128 - - [16/May/2021:12:16:15 -0500] "GET /littlehillschurch/resources/podcasts/20210208.mp3 HTTP/1.1" 200 15480738 "-" "Spotify/1.0" "-"
 35.189.225.190 - - [16/May/2021:12:16:15 -0500] "GET /littlehillschurch/resources/podcasts/20210510.mp3 HTTP/1.1" 200 16093457 "-" "Spotify/1.0" "-"
 35.205.135.106 - - [16/May/2021:12:16:15 -0500] "GET /littlehillschurch/resources/podcasts/20200907.mp3 HTTP/1.1" 200 14715203 "-" "Spotify/1.0" "-"
 35.205.135.106 - - [16/May/2021:12:16:16 -0500] "GET /littlehillschurch/resources/podcasts/20200720.mp3 HTTP/1.1" 200 14100420 "-" "Spotify/1.0" "-"

curl -v所请求的实际文件似乎报告了正确的修改日期等。如果我请求其中一个文件，以下是看似相关的输出：

* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
< HTTP/2 200 
< server: nginx/1.20.0
< date: Sun, 16 May 2021 17:38:40 GMT
< content-type: audio/mpeg
< content-length: 14100420
< strict-transport-security: max-age=16070400; includeSubDomains
< last-modified: Tue, 21 Jul 2020 00:52:50 GMT
< accept-ranges: bytes
< access-control-allow-origin: *

相关内容