尝试使用 spark-ec2 脚本启动 spark:AWS 无法验证提供的访问凭据

尝试使用 spark-ec2 脚本启动 spark:AWS 无法验证提供的访问凭据

我正在尝试在我的 aws ubuntu ec2 中使用 spark-ec2 脚本启动 spark 2.4.5

我正在使用由 AWS_ACCESS_KEY_ID、AWS_SECRET_ACCESS_KEY、AWS_SESSION_TOKEN 组成的 AWS 临时凭证作为

export AWS_ACCESS_KEY_ID=MyAccessKey
export AWS_SECRET_ACCESS_KEY=MySecretAccessKey
export AWS_SESSION_TOKEN=MySessionToken

/opt/spark-2.4.5-bin-hadoop2.7/ec2/spark-ec2 --key-pair=mykey--identity-file=./mykey.pem --region=us-east-1 launch my-spark-cluster

但每当我运行这个我都会收到一个错误AWS 无法验证提供的凭证

/opt/spark-2.4.5-bin-hadoop2.7/ec2/lib/boto-2.34.0/boto/plugin.py:40: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp
/opt/spark-2.4.5-bin-hadoop2.7/ec2/lib/boto-2.34.0/boto/provider.py:197: ResourceWarning: unclosed file <_io.TextIOWrapper name='/home/ubuntu/.aws/credentials' mode='r' encoding='UTF-8'>
  self.shared_credentials.load_from_path(shared_path)
ResourceWarning: Enable tracemalloc to get the object allocation traceback
ERROR:boto:401 Unauthorized
ERROR:boto:b'<?xml version="1.0" encoding="UTF-8"?>\n<Response><Errors><Error><Code>AuthFailure</Code><Message>AWS was not able to validate the provided access credentials</Message></Error></Errors><RequestID>***********************</RequestID></Response>'
Traceback (most recent call last):
  File "/opt/spark-2.4.5-bin-hadoop2.7/ec2/spark_ec2.py", line 1574, in <module>
    main()
  File "/opt/spark-2.4.5-bin-hadoop2.7/ec2/spark_ec2.py", line 1566, in main
    real_main()
  File "/opt/spark-2.4.5-bin-hadoop2.7/ec2/spark_ec2.py", line 1386, in real_main
    opts.zone = random.choice(conn.get_all_zones()).name
  File "/opt/spark-2.4.5-bin-hadoop2.7/ec2/lib/boto-2.34.0/boto/ec2/connection.py", line 1758, in get_all_zones
    return self.get_list('DescribeAvailabilityZones', params,
  File "/opt/spark-2.4.5-bin-hadoop2.7/ec2/lib/boto-2.34.0/boto/connection.py", line 1182, in get_list
    raise self.ResponseError(response.status, response.reason, body)
boto.exception.EC2ResponseError: EC2ResponseError: 401 Unauthorized
<?xml version="1.0" encoding="UTF-8"?>
<Response><Errors><Error><Code>AuthFailure</Code><Message>AWS was not able to validate the provided access credentials</Message></Error></Errors><RequestID>******************</RequestID></Response>
sys:1: ResourceWarning: unclosed <ssl.SSLSocket fd=3, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('********', *******), raddr=('********', ******)>

非常感谢您的帮助。

相关内容