☰

HTTP 错误 429：请求过多 - youtube-dl 停止工作

tag-icon • tag-icon python3 youtube youtube-dl
HTTP 错误 429：请求过多 - youtube-dl 停止工作

tldr：最新的 youtube-dl 在 4 台设备中的 1 台上表现异常，忽略了请求之间的超时

我已经使用 youtube-dl 下载视频一段时间了。有时当服务器端发生更改时，它就会停止工作，然后快速更新可以修复它。我有一台一直在使用的台式电脑，它又停止工作了。最后一次成功下载视频是在 2019-10-08。我等了一段时间才更新，但没有一个能解决这个问题。

当时我的电脑运行的是之前的 LTS（我认为是 16.04），所以我将其升级到 18.04，删除了 youtube-dl 包，清理了我之前的 pip 包。尝试了 pip 存储库中的 youtube-dl，并直接从他们的网站（使用wget https://yt-dl.org/latest/youtube-dl -O ~/bin/ytdl2）尝试了，但都不起作用。

这台 PC 位于我的路由器后面，具有相同的外部 IP 地址，还有一台 Raspberry Pi (1B+)，以及 - 当我在家时 - 一台笔记本电脑和一部安卓手机。pip 的 youtube-dl 和来自上述 URL 的 youtube-dl 都可以在所有其他设备上完美运行（在手机上的 Termux 的 python 中）。但我想在我的 PC 上运行它。

该程序在我的 PC 上也能访问不同的网站（已检查 bitchute），但不能访问 youtube。由于错误代码 429 表明服务器不喜欢我的下载速度，我尝试使用 --socket-timeout、--sleep-interval、--limit-rate 选项（几秒钟，降至 50kB/s）限制其下载速率，但没有任何变化。我注意到每个设备上的用户代理字符串略有不同，所以我也尝试修改它。--geo-bypass 不是问题。我尝试了大约 30 个不同的视频，只尝试了声音。甚至使用 --get-title 获取视频标题也以某种方式失败。在浏览器中观看相同的视频是可行的。我的 IP 显然也没有被禁止。

即使我只想获取标题，情况也是如此。还有一件事：在time ytdl2 --print-traffic --get-title --socket-timeout 5 --sleep-interval 5 --max-sleep-interval 10 https://www.youtube.com/watch?v=SuqRisgZfrU我的 PC 和笔记本电脑上运行并比较结果时，我可以清楚地看到我的笔记本电脑在请求之间等待的时间更长，但大约 2 秒即可完成该过程。但即使我使用 --ignore-config，某些东西也会导致不同的速度，从而导致不同的结果。（我的 PC 比我的笔记本电脑大约 8 年前，但我的手机和 RPi 显然更慢）

因此，尽管这 4 台设备上的代码完全相同，但两台运行 18.04 Ubuntu 的计算机都会忽略超时和配置，但其中一台不知何故会在请求之间等待足够长的时间才能成功，而另一台则不会。它曾经可以正常工作。

有什么建议要检查什么或如何修复吗？

我的电脑的完整输出：

nyos@hex:~$ time ytdl2 --print-traffic --get-title --socket-timeout 5 --sleep-interval 5 --max-sleep-interval 10 --ignore-config https://www.youtube.com/watch?v=SuqRisgZfrU
send: u'GET /watch?has_verified=1&bpctr=9999999999&hl=en&v=SuqRisgZfrU&disable_polymer=true&gl=US HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.38 Safari/537.36\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: PREF=f1=50000000&hl=en\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Cache-Control: no-cache
header: Strict-Transport-Security: max-age=31536000
header: Expires: Tue, 27 Apr 1971 19:44:06 EST
header: X-Frame-Options: SAMEORIGIN
header: X-Content-Type-Options: nosniff
header: Content-Encoding: gzip
header: Content-Type: text/html; charset=utf-8
header: P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
header: Date: Thu, 17 Oct 2019 00:51:35 GMT
header: Server: YouTube Frontend Proxy
header: X-XSS-Protection: 0
header: Set-Cookie: VISITOR_INFO1_LIVE=sn_5wz6eikA; path=/; domain=.youtube.com; expires=Tue, 14-Apr-2020 00:51:35 GMT; httponly
header: Set-Cookie: GPS=1; path=/; domain=.youtube.com; expires=Thu, 17-Oct-2019 01:21:35 GMT
header: Set-Cookie: s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; path=/; domain=.youtube.com
header: Set-Cookie: YSC=OGFieKP1qeE; path=/; domain=.youtube.com; httponly
header: Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
header: Connection: close
header: Transfer-Encoding: chunked
send: u'GET /get_video_info?ps=default&el=embedded&video_id=SuqRisgZfrU&hl=en&disable_polymer=true&gl=US&eurl= HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.38 Safari/537.36\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; VISITOR_INFO1_LIVE=sn_5wz6eikA; YSC=OGFieKP1qeE; PREF=f1=50000000&hl=en; GPS=1\r\n\r\n'
reply: 'HTTP/1.1 429 Too Many Requests\r\n'
header: Expires: Tue, 27 Apr 1971 19:44:06 EST
header: X-Frame-Options: SAMEORIGIN
header: Cache-Control: no-cache
header: Content-Type: text/html; charset=utf-8
header: Strict-Transport-Security: max-age=31536000
header: X-Content-Type-Options: nosniff
header: P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
header: Content-Length: 0
header: Date: Thu, 17 Oct 2019 00:51:35 GMT
header: Server: YouTube Frontend Proxy
header: X-XSS-Protection: 0
header: Set-Cookie: s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; path=/; domain=.youtube.com
header: Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
header: Connection: close
WARNING: unable to download video info webpage: HTTP Error 429: Too Many Requests
send: u'GET /get_video_info?ps=default&el=detailpage&video_id=SuqRisgZfrU&hl=en&disable_polymer=true&gl=US&eurl= HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.38 Safari/537.36\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; VISITOR_INFO1_LIVE=sn_5wz6eikA; YSC=OGFieKP1qeE; PREF=f1=50000000&hl=en; GPS=1\r\n\r\n'
reply: 'HTTP/1.1 429 Too Many Requests\r\n'
header: P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
header: Strict-Transport-Security: max-age=31536000
header: Content-Type: text/html; charset=utf-8
header: Cache-Control: no-cache
header: Content-Length: 0
header: X-Frame-Options: SAMEORIGIN
header: X-Content-Type-Options: nosniff
header: Expires: Tue, 27 Apr 1971 19:44:06 EST
header: Date: Thu, 17 Oct 2019 00:51:35 GMT
header: Server: YouTube Frontend Proxy
header: X-XSS-Protection: 0
header: Set-Cookie: s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; path=/; domain=.youtube.com
header: Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
header: Connection: close
WARNING: unable to download video info webpage: HTTP Error 429: Too Many Requests
send: u'GET /get_video_info?ps=default&el=vevo&video_id=SuqRisgZfrU&hl=en&disable_polymer=true&gl=US&eurl= HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.38 Safari/537.36\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; VISITOR_INFO1_LIVE=sn_5wz6eikA; YSC=OGFieKP1qeE; PREF=f1=50000000&hl=en; GPS=1\r\n\r\n'
reply: 'HTTP/1.1 429 Too Many Requests\r\n'
header: Content-Length: 0
header: X-Content-Type-Options: nosniff
header: Strict-Transport-Security: max-age=31536000
header: P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
header: Cache-Control: no-cache
header: Content-Type: text/html; charset=utf-8
header: Expires: Tue, 27 Apr 1971 19:44:06 EST
header: X-Frame-Options: SAMEORIGIN
header: Date: Thu, 17 Oct 2019 00:51:36 GMT
header: Server: YouTube Frontend Proxy
header: X-XSS-Protection: 0
header: Set-Cookie: s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; path=/; domain=.youtube.com
header: Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
header: Connection: close
WARNING: unable to download video info webpage: HTTP Error 429: Too Many Requests
send: u'GET /get_video_info?ps=default&video_id=SuqRisgZfrU&hl=en&disable_polymer=true&gl=US&eurl= HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.38 Safari/537.36\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; VISITOR_INFO1_LIVE=sn_5wz6eikA; YSC=OGFieKP1qeE; PREF=f1=50000000&hl=en; GPS=1\r\n\r\n'
reply: 'HTTP/1.1 429 Too Many Requests\r\n'
header: X-Frame-Options: SAMEORIGIN
header: Content-Type: text/html; charset=utf-8
header: Cache-Control: no-cache
header: P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
header: Expires: Tue, 27 Apr 1971 19:44:06 EST
header: Content-Length: 0
header: Strict-Transport-Security: max-age=31536000
header: X-Content-Type-Options: nosniff
header: Date: Thu, 17 Oct 2019 00:51:36 GMT
header: Server: YouTube Frontend Proxy
header: X-XSS-Protection: 0
header: Set-Cookie: s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; path=/; domain=.youtube.com
header: Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
header: Connection: close
WARNING: unable to download video info webpage: HTTP Error 429: Too Many Requests
ERROR: SuqRisgZfrU: YouTube said: Unable to extract video data

real    0m2,595s
user    0m1,494s
sys     0m0,054s

和我的笔记本电脑：

nyos@shodan:~$ time ytdl2 --print-traffic --get-title --socket-timeout 5 --sleep-interval 5 --max-sleep-interval 10 --ignore-config https://www.youtube.com/watch?v=SuqRisgZfrU
send: u'GET /watch?has_verified=1&bpctr=9999999999&hl=en&v=SuqRisgZfrU&disable_polymer=true&gl=US HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.108 Safari/537.36\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: PREF=f1=50000000&hl=en\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: Content-Encoding: gzip
header: Strict-Transport-Security: max-age=31536000
header: P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
header: X-Content-Type-Options: nosniff
header: X-Frame-Options: SAMEORIGIN
header: Content-Type: text/html; charset=utf-8
header: Cache-Control: no-cache
header: Expires: Tue, 27 Apr 1971 19:44:06 EST
header: Date: Thu, 17 Oct 2019 00:53:09 GMT
header: Server: YouTube Frontend Proxy
header: X-XSS-Protection: 0
header: Set-Cookie: s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==; path=/; domain=.youtube.com
header: Set-Cookie: VISITOR_INFO1_LIVE=HQNpuTQ4WDc; path=/; domain=.youtube.com; expires=Tue, 14-Apr-2020 00:53:09 GMT; httponly
header: Set-Cookie: YSC=F1swT5PPA-s; path=/; domain=.youtube.com; httponly
header: Set-Cookie: GPS=1; path=/; domain=.youtube.com; expires=Thu, 17-Oct-2019 01:23:09 GMT
header: Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
header: Connection: close
header: Transfer-Encoding: chunked
send: u'GET /get_video_info?ps=default&el=embedded&video_id=SuqRisgZfrU&hl=en&disable_polymer=true&gl=US&eurl= HTTP/1.1\r\nAccept-Language: en-us,en;q=0.5\r\nAccept-Encoding: gzip, deflate\r\nConnection: close\r\nAccept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8\r\nUser-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.108 Safari/537.36\r\nAccept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.7\r\nHost: www.youtube.com\r\nCookie: YSC=F1swT5PPA-s; VISITOR_INFO1_LIVE=HQNpuTQ4WDc; GPS=1; PREF=f1=50000000&hl=en; s_gl=1d69aac621b2f9c0a25dade722d6e24bcwIAAABVUw==\r\n\r\n'
reply: 'HTTP/1.1 200 OK\r\n'
header: X-Content-Type-Options: nosniff
header: Cache-Control: no-store
header: Content-Type: application/x-www-form-urlencoded
header: Content-Encoding: gzip
header: P3P: CP="This is not a P3P policy! See http://support.google.com/accounts/answer/151657?hl=en for more info."
header: Strict-Transport-Security: max-age=31536000
header: Expires: Tue, 27 Apr 1971 19:44:06 EST
header: X-Frame-Options: SAMEORIGIN
header: Date: Thu, 17 Oct 2019 00:53:10 GMT
header: Server: YouTube Frontend Proxy
header: X-XSS-Protection: 0
header: Alt-Svc: quic=":443"; ma=2592000; v="46,43",h3-Q048=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000
header: Connection: close
header: Transfer-Encoding: chunked
Itt a legcuccosabb lasagne a világon!

real    0m1,970s
user    0m1,028s
sys     0m0,044s

答案1

我的 openwrt 路由器上也出现了同样的问题，但在我的计算机上运行良好，所以没有 IP 禁止问题。

curl、aria2c 甚至 netcat 都响应了 429。

对我来说，当我升级到内核 4.14.150 时，问题就解决了...

（4.14.148 和 4.14.149 都有同样的问题）

答案2

该网站和/或代理不喜欢您的“User-Agent”标头，我只是告诉 wget 不要发送它，它就正常工作了。

从手册页中：使用 --user-agent="" 指定空用户代理指示 Wget 不要在 HTTP 请求中发送“User-Agent”标头。

相关内容