我目前正在尝试设置一个由 ASG、ELB 和 EC2 实例组成的 HA squid 代理。我已将 Internet Explorer 上的 LAN 设置中的代理服务器设置设置为 ELB DNS 名称。当尝试从允许的 URL 列表中加载网页时,我收到以下错误消息,其中实例配置为使用代理:
ERROR
The requested URL could not be retrieved
The following error was encountered while trying to retrieve the URL: /
Invalid URL
Some aspect of the requested URL is incorrect.
Some possible problems are:
•Missing or incorrect access protocol (should be “http://” or similar)
•Missing hostname
•Illegal double-escape in the URL-Path
•Illegal character in hostname; underscores are not allowed.
Your cache administrator is root.
问题似乎出在负载均衡器上,只要您将 Internet Explorer 代理设置设为代理私有 DNS 或私有 IP 地址,就不会出现问题,代理可以正常工作,即网页可以正常加载。
这是 squid.conf
# This file generated from a Chef template.
# squid/templates/default.squid.conf.erb
acl manager proto cache_object
acl localhost src 127.0.0.1/32
acl to_localhost dst 127.0.0.0/8 0.0.0.0/32
acl localnet src 10.0.0.0/8 # RFC1918 possible internal network
acl localnet src 172.16.0.0/12 # RFC1918 possible internal network
acl localnet src 192.168.0.0/16 # RFC1918 possible internal network
acl localnet src fc00::/7 # RFC4193 local private network range
acl localnet src fe80::/10 # RFC4291 link-local (directly-plugged) machine
acl SSL_ports port 443 # https
acl SSL_ports port 563 # snews
acl SSL_ports port 873 # rsync
acl Safe_ports port 80 # http
acl Safe_ports port 21 # ftp
acl Safe_ports port 443 # https
acl Safe_ports port 70 # gopher
acl Safe_ports port 210 # wais
acl Safe_ports port 1025-65535 # unregistered ports
acl Safe_ports port 280 # http-mgmt
acl Safe_ports port 488 # gss-http
acl Safe_ports port 591 # filemaker
acl Safe_ports port 777 # multiling http
acl Safe_ports port 631 # cups
acl Safe_ports port 873 # rsync
acl Safe_ports port 901 # SWAT
acl purge method PURGE
acl CONNECT method CONNECT
http_access allow all
# Managed with Chef
acl web-hosts src all
acl web-bd dstdomain .amazonaws.com
acl web-bd dstdomain .chef.io
acl web-bd dstdomain .rubygems.org
acl web-bd dstdomain .splunk.com
acl web-bd dstdomain .bintray.com
acl web-bd dstdomain .trendmicro.com
acl web-bd dstdomain .slproweb.com
acl web-bd dstdomain .fastly.net
http_access allow web-bd
# The line below blocks all websites which are not allowed in the squid_urls data bag
http_access deny !web-bd
http_access allow manager localhost
http_access deny manager
http_access allow purge localhost
http_access deny purge
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost
http_access allow localnet
http_access deny all
icp_access allow localnet
icp_access deny all
http_port 3130 protocol=HTTP
hierarchy_stoplist cgi-bin ?
access_log /var/log/squid/access.log squid
refresh_pattern ^ftp: 1440 20% 10080
refresh_pattern ^gopher: 1440 0% 1440
refresh_pattern -i (/cgi-bin/|\?) 0 0% 0
refresh_pattern (Release|Package(.gz)*)$ 0 20% 2880
# refresh_pattern \.deb$ 1440 20% 10080
# refresh_pattern \.rpm$ 1440 20% 10080
# refresh_pattern \.iso$ 1440 20% 10080
# refresh_pattern \.$ 1440 20% 10080
# refresh_pattern . 0 20% 4320
hosts_file /etc/hosts
maximum_object_size 1024 MB
coredump_dir /var/spool/squid
cache_mem 0 MB
debug_options ALL
我打开了 ELB 日志记录,以下是一些流量:
2016-09-02T10:58:59.552990Z ha-proxy 10.166.107.198:56190 10.166.106.20:3130 0.000036 0.000908 0.000026 400 400 0 3154 "GET http://youtube.com:3130/ HTTP/1.1" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0" - -
2016-09-02T10:58:59.572031Z ha-proxy 10.166.107.198:56190 10.166.106.20:3130 0.000023 0.000689 0.000018 400 400 0 3182 "GET http://www.squid-cache.org:3130/Artwork/SN.png HTTP/1.1" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0" - -
2016-09-02T10:59:01.844569Z ha-proxy 10.166.107.198:56190 10.166.106.20:3130 0.000046 0.001085 0.000021 400 400 0 3154 "GET http://google.com:3130/ HTTP/1.1" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0" - -
2016-09-02T10:59:01.862584Z ha-proxy 10.166.107.198:56190 10.166.106.20:3130 0.00004 0.000839 0.000023 400 400 0 3182 "GET http://www.squid-cache.org:3130/Artwork/SN.png HTTP/1.1" "Mozilla/5.0 (Windows NT 6.3; WOW64; rv:38.0) Gecko/20100101 Firefox/38.0" - -
从 squid 访问日志中:
1472813779.899 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472813793.898 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472813821.915 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472813828.898 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472813835.898 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472813849.898 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472813856.898 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472813863.899 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472813870.900 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472813884.876 0 10.166.106.29 NONE/400 4076 NONE error:invalid-request - NONE/- text/html
1472813898.899 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472813939.552 0 10.166.106.29 NONE/400 3533 GET / - NONE/- text/html
1472813939.570 0 10.166.106.29 NONE/400 3561 GET /Artwork/SN.png - NONE/- text/html
1472813940.899 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472813941.843 0 10.166.106.29 NONE/400 3533 GET / - NONE/- text/html
1472813941.861 0 10.166.106.29 NONE/400 3561 GET /Artwork/SN.png - NONE/- text/html
1472813961.899 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817013.903 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817020.898 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817026.226 0 10.166.106.80 NONE/400 4076 NONE error:invalid-request - NONE/- text/html
1472817026.332 0 10.166.106.80 NONE/400 4076 NONE error:invalid-request - NONE/- text/html
1472817034.899 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817048.898 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817083.899 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817097.898 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817104.898 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817153.904 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817160.899 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817174.898 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817195.920 0 10.166.106.29 NONE/400 4076 NONE error:invalid-request - NONE/- text/html
1472817202.898 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817223.898 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817237.900 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817251.899 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817265.899 0 10.166.106.80 NONE/400 4075 NONE error:invalid-request - NONE/- text/html
1472817275.160 0 10.166.106.80 NONE/400 3533 GET / - NONE/- text/html
1472817275.193 0 10.166.106.80 NONE/400 3561 GET /Artwork/SN.png - NONE/- text/html
1472817278.626 0 10.166.106.29 NONE/400 3677 GET /fwlink/?LinkID=403856&language=en-US&scale=100&contrast=gray - NONE/- text/html
1472817330.367 0 10.166.106.80 NONE/400 3533 GET / - NONE/- text/html
1472817330.397 0 10.166.106.80 NONE/400 3561 GET /Artwork/SN.png - NONE/- text/html
1472817330.448 0 10.166.106.80 NONE/400 3555 GET /favicon.ico - NONE/- text/html
1472817330.453 0 10.166.106.80 NONE/400 3555 GET /favicon.ico - NONE/- text/html
1472817337.300 0 10.166.106.29 NONE/400 3533 GET / - NONE/- text/html
1472817337.334 0 10.166.106.29 NONE/400 3561 GET /Artwork/SN.png - NONE/- text/html
1472818532.464 0 10.166.106.29 NONE/400 3533 GET / - NONE/- text/html
1472818532.478 0 10.166.106.29 NONE/400 3561 GET /Artwork/SN.png - NONE/- text/html
1472818533.259 0 10.166.106.29 NONE/400 3533 GET / - NONE/- text/html
1472818533.278 0 10.166.106.29 NONE/400 3561 GET /Artwork/SN.png - NONE/- text/html
1472818534.108 0 10.166.106.29 NONE/400 3533 GET / - NONE/- text/html
1472818534.137 0 10.166.106.29 NONE/400 3561 GET /Artwork/SN.png - NONE/- text/html
有人有什么想法吗?我快抓狂了。
答案1
你是对的 - 这是一个与 ELB 相关的问题。
tl;dr - 您可以通过从 HTTP 切换到 TCP 来与后端 squid 服务器通信来解决此问题。
当客户端向代理发送 HTTP 请求时,请求包含绝对 URI(例如 GET http ://host-url HTTP/1.1)。
但是,AWS ELB 强制执行最常见的 HTTP 请求形式并重写请求(GET / HTTP/1.1 HOST www.host-url.com)。
这次重写使得 Squid 无法读取它因此,目前无法使用 ELB HTTP 侦听器和“X-Forwarded-For”HTTP 标头来检索客户端 IP 地址。
为了解决这个问题,我改用了 TCP 协议,因为使用此协议,ELB 会按原样传递请求,而不会插入请求标头。
这样做的一个主要问题是,您无法获得常规 ELB 标头(例如 x-forwarded-for),并且无法记录客户端的 IP 地址(只能记录 ELB 的 IP 地址)。
这个问题有多种解决方案:
1. 使用代理传递协议和 TCP 协议配置 ELB。
你可以阅读更多相关信息这里.
2. 使用 squid 3.5 及以上版本。此版本支持代理传递协议,应从 ELB 解析 URL。阅读更多相关信息这里。