我正在尝试在我的 aws ubuntu ec2 中使用 spark-ec2 脚本启动 spark 2.4.5
我正在使用由 AWS_ACCESS_KEY_ID、AWS_SECRET_ACCESS_KEY、AWS_SESSION_TOKEN 组成的 AWS 临时凭证作为
export AWS_ACCESS_KEY_ID=MyAccessKey
export AWS_SECRET_ACCESS_KEY=MySecretAccessKey
export AWS_SESSION_TOKEN=MySessionToken
/opt/spark-2.4.5-bin-hadoop2.7/ec2/spark-ec2 --key-pair=mykey--identity-file=./mykey.pem --region=us-east-1 launch my-spark-cluster
但每当我运行这个我都会收到一个错误AWS 无法验证提供的凭证
/opt/spark-2.4.5-bin-hadoop2.7/ec2/lib/boto-2.34.0/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
/opt/spark-2.4.5-bin-hadoop2.7/ec2/lib/boto-2.34.0/boto/provider.py:197: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/ubuntu/.aws/credentials' mode='r' encoding='UTF-8'>
self.shared_credentials.load_from_path(shared_path)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
ERROR:boto:401 Unauthorized
ERROR:boto:b'<?xml version="1.0" encoding="UTF-8"?>\n<Response><Errors><Error><Code>AuthFailure</Code><Message>AWS was not able to validate the provided access credentials</Message></Error></Errors><RequestID>***********************</RequestID></Response>'
Traceback (most recent call last):
File "/opt/spark-2.4.5-bin-hadoop2.7/ec2/spark_ec2.py", line 1574, in <module>
main()
File "/opt/spark-2.4.5-bin-hadoop2.7/ec2/spark_ec2.py", line 1566, in main
real_main()
File "/opt/spark-2.4.5-bin-hadoop2.7/ec2/spark_ec2.py", line 1386, in real_main
opts.zone = random.choice(conn.get_all_zones()).name
File "/opt/spark-2.4.5-bin-hadoop2.7/ec2/lib/boto-2.34.0/boto/ec2/connection.py", line 1758, in get_all_zones
return self.get_list('DescribeAvailabilityZones', params,
File "/opt/spark-2.4.5-bin-hadoop2.7/ec2/lib/boto-2.34.0/boto/connection.py", line 1182, in get_list
raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 401 Unauthorized
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>AuthFailure</Code><Message>AWS was not able to validate the provided access credentials</Message></Error></Errors><RequestID>******************</RequestID></Response>
sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('********', *******), raddr=('********', ******)>
非常感谢您的帮助。