☰

Amazon AWS Elastic Beanstalk EBS 日志记录到 Cloudwatch（多 Docker 环境）

tag-icon • tag-icon amazon-web-services docker elastic-beanstalk amazon-cloudwatch
Amazon AWS Elastic Beanstalk EBS 日志记录到 Cloudwatch（多 Docker 环境）

我想在 cloudwatch 中查看我的所有日​​志。我目前有一个多 docker Elastic beanstalk 环境。

我已选择将日志推送到 cloudwatch，方法是：

Elastic Beanstalk > App > Env > Configuration > Software Configuration > CloudWatch Logs

此项已启用。

当我查看 CloudWatch 时，我看到了以下内容......

/aws/elasticbeanstalk/myapp-staging/var/log/docker-events.log
/aws/elasticbeanstalk/myapp-staging/var/log/eb-activity.log
/aws/elasticbeanstalk/myapp-staging/var/log/eb-ecs-mgr.log
/aws/elasticbeanstalk/myapp-staging/var/log/ecs/ecs-agent.log
/aws/elasticbeanstalk/myapp-staging/var/log/ecs/ecs-init.log

但是我没有看到 nginx 访问/错误日志。

我有这个Dockerrun.aws.json

"mountPoints":[
        {
          "sourceVolume": "awseb-logs-nginx",
          "containerPath": "/var/log/nginx"
        }

如果我通过 SSH 连接到包含此内容的实例，我可以看到这些日志（当我点击 URL 时）在本地路径等上生成（如预期的那样）/var/log/containers/nginx/access.log。

另外，如果我转到 EBS > 日志并请求最近的日志，我也可以看到访问日志中的内容，只是没有发送到 CloudWatch？

我想知道我是否必须设置Log Group正确的路径，我尝试过但没有填充？

我确信我缺少了一些东西来将其推送到 Cloudwatch，提前谢谢大家！

更新：我已添加以下内容，这是建议的必需内容。此文件位于.ebextensions名为的文件夹中nginx_logs.conf。

packages:
  yum:
    awslogs: []

files:
  "/etc/awslogs/config/nginx_logs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [/var/log/containers/nginx/access.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/containers/nginx/access.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/containers/nginx/access.log*
      [/var/log//containers/nginx/error.log]
      log_group_name = `{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/log/containers/nginx/error.log"]]}`
      log_stream_name = {instance_id}
      file = /var/log/containers/nginx/error.log*

commands:
  "01":
    command: chkconfig awslogs on
  "02":
    command: service awslogs restart

仍然没有惊喜，因为当我点击请求日志或请求最后 100 条日志时，我仍然可以看到它们。但 Cloudwatch 中什么都没有...

答案1

根据 AWS 官方文档，为 Docker Multicontainer EB 环境开箱即用地收集了以下文件：

  • /var/log/eb-activity.log
  • /var/log/ecs/ecs-init.log
  • /var/log/eb-ecs-mgr.log
  • /var/log/ecs/ecs-agent.log
  • /var/log/docker-events.log

由于这些路径不包含包含 nginx 日志的目录，因此它们不会被传输到 CloudWatch 是有道理的。

要流式传输日志，似乎必须配置 CloudWatch Logs 代理来收集容器目录中的文件。这里有示例配置这里。坦率地说，Docker Multicontainer 的默认配置不包含容器日志有点奇怪，但显然 AWS 就是这样实现的。

答案2

我们使用以下 ebextensions 将应用程序 (nginx/ror) 日志传输到 cloudwatch

$ cat .ebextensions/cw-logs.config
files:
  "/etc/awslogs/config/nginx-access-log.conf":
    mode: "000644"
    owner: root
    group: root
    encoding: plain
    content: |
        [nginx-access.log]
        datetime_format = %Y-%m-%dT%H:%M:%S%f
        file = /var/log/containers/project1-staging/nginx_access.log
        buffer_duration = 5000
        log_stream_name = project1_staging_{instance_id}
        initial_position = start_of_file
        log_group_name = PROJECT1_STAGING_NGINX

  "/etc/awslogs/config/app-log.conf":
    mode: "000644"
    owner: root
    group: root
    encoding: plain
    content: |
        [app-access.log]
        datetime_format = %Y-%m-%dT%H:%M:%S
        file = /var/log/containers/project1-staging/app.log
        buffer_duration = 5000
        log_stream_name = project1_staging_{instance_id}
        initial_position = start_of_file
        log_group_name = PROJECT1_STAGING_APP

commands:

    00-cmd:
        command: chkconfig --level 35 awslogs on
        test: "[ -s /usr/sbin/awslogsd ]"

container_commands:
    00-cmd:
        command: service awslogs restart
        test: "[ -s /etc/awslogs/config/nginx-access-log.conf -a -s /etc/awslogs/config/app-log.conf ]"

并轮换至 S3 策略

$ cat .ebextensions/cw-logs-s3-rotation.config
files:
  "/tmp/logrotate-project1-staging.conf":
    mode: "000644"
    owner: root
    group: root
    encoding: plain
    content: |
        /var/log/containers/project1-staging/*.log {
            size 512M
            weekly
            rotate 0
            missingok
            compress
            notifempty
            copytruncate
            dateext
            dateformat -%Y-%m-%d_%s
            olddir /var/log/containers/project1-staging/rotated
            lastaction
                aws s3 mv --recursive /var/log/containers/project1-staging/rotated/ \
                s3://app-logs-archive-s3-bucket/project1-staging/$(date '+%Y')/
            endscript
        }

container_commands:
    00-cmd:
        command: mv /tmp/logrotate-project1-staging.conf /etc/logrotate.elasticbeanstalk.hourly/logrotate.elasticbeanstalk.applogs-project1-staging.conf
        test: "[ -f /tmp/logrotate-project1-staging.conf ]"

我们的 Dockerrun.aws.json 看起来像

$ cat Dockerrun.aws.json 
{
    "AWSEBDockerrunVersion": 2,

    "containerDefinitions": [{
        "name": "project1-staging",
        "image": "0123456789.dkr.ecr.us-east-1.amazonaws.com/project1:staging_v.1234567",
        "cpu": {{ env['DOCKER_CPU'] }},
        "memory": {{ env['DOCKER_MEMORY'] }},
        "essential": true,

        "portMappings": [{
            "hostPort": 80,
            "containerPort": 80
        },
        {
            "hostPort": 443,
            "containerPort": 443
        }],

        "mountPoints": [{
            "sourceVolume": "awseb-logs-project1-staging",
            "containerPath": "/home/app/project1/log"
        }]
    }]
}

相关内容