☰

使用 Linux AMI 2（推荐）

tag-icon • tag-icon amazon-ec2 elastic-beanstalk queue worker-process
使用 Linux AMI 2（推荐）

我创建了 AWS Beanstalk 和 RDS 实例。我正在测试一个同步 FTP 和 S3 文件的项目。

如何在 AWS Beanstalk 中使其典型php artisan queue:work或工作？php artisan queue:listen

我知道 Redis、ElastiCache 等。我还没有测试过它们。但我正在尝试让它database只与驱动程序一起工作。

我也知道只需访问 SSH，但是有没有办法不使用 SSH 来处理排队作业？也许像使用 .ebconfig？

答案1

使用 Linux AMI 2（推荐）

自第一篇文章以来eb 路线图宣布 PHP 平台已更新，使事情变得更加简单。下面解释了需要设置的 .ebextensions 配置文件和 .platform 文件（否则 nginx 将在所有路由上抛出 404 错误）

此镜像使用 Systemd，这使得该过程变得更容易，因为不再需要主管。不幸的是，services新镜像尚不支持该关键字，因此必须使用该container_commands关键字启动和重新启动服务。

设置 .ebextensions 配置

此文件包含我在每个生产环境，请记住将它们更改为适合您的需求：

\.ebextension\01-setup.config

container_commands:
    01-no_dev:
        command: "composer.phar install --optimize-autoloader --no-dev"
    02-config_clear:
        command: "php artisan config:clear"
    03-view_clear:
        command: "php artisan view:clear"
    04-route_cache:
        command: "php artisan route:cache"
    05-view_cache:
        command: "php artisan view:cache"
    06-migrate: 
        command: "php artisan migrate --force"
        leader_only: true
    07-queue_service_restart:
        command: "systemctl restart laravel_worker"
files: 
    /opt/elasticbeanstalk/tasks/taillogs.d/laravel-logs.conf: 
        content: /var/app/current/storage/logs/laravel.log
        group: root
        mode: "000755"
        owner: root
    /etc/systemd/system/laravel_worker.service:
        mode: "000755"
        owner: root
        group: root
        content: |
            # Laravel queue worker using systemd
            # ----------------------------------
            #
            # /lib/systemd/system/queue.service
            #
            # run this command to enable service:
            # systemctl enable queue.service

            [Unit]
            Description=Laravel queue worker

            [Service]
            User=nginx
            Group=nginx
            Restart=always
            ExecStart=/usr/bin/nohup /usr/bin/php /var/www/html/laravel-project/artisan queue:work --daemon

            [Install]
            WantedBy=multi-user.target

第二个文件用于设置 Laravel 调度程序，它是一个php artisan schedule:run每分钟运行一次的 cron 作业。它必须以 root 身份执行，并且由于环境变量不可用，我们需要从中获取它们/opt/elasticbeanstalk/deployment/env。这是一个很好的答案关于该主题。

\.ebextension\cron-linux.config

files:
    "/etc/cron.d/schedule_run":
        mode: "000644"
        owner: root
        group: root
        content: |
            * * * * * root . /opt/elasticbeanstalk/deployment/env && /usr/bin/php /var/app/current/artisan schedule:run 1>> /dev/null 2>&1

commands:
    remove_old_cron:
        command: "rm -f /etc/cron.d/*.bak"

设置 .platform 配置

\.platform\nginx\conf.d\elasticbeanstalk\laravel.conf

location / {
    try_files $uri $uri/ /index.php?$query_string;
    gzip_static on;
}

使用旧的 Linux AMI（上一个图像）

最好的方法是运行 Supervisor 来管理服务下的队列，以确保它在重启后仍能继续运行。

设置 .ebextensions 配置

1- 使用软件包安装主管。python 关键字在后台使用 pip 和 easy_install：

packages:
    python:
        supervisor: []

2-创建主管配置文件：

files:
    /usr/local/etc/supervisord.conf:
        mode: "000755"
        owner: root
        group: root
        content: |
            [unix_http_server]
            file=/tmp/supervisor.sock   ; (the path to the socket file)

            [supervisord]
            logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
            logfile_maxbytes=50MB        ; (max main logfile bytes b4 rotation;default 50MB)
            logfile_backups=10           ; (num of main logfile rotation backups;default 10)
            loglevel=info                ; (log level;default info; others: debug,warn,trace)
            pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
            nodaemon=false               ; (start in foreground if true;default false)
            minfds=1024                  ; (min. avail startup file descriptors;default 1024)
            minprocs=200                 ; (min. avail process descriptors;default 200)

            [rpcinterface:supervisor]
            supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

            [supervisorctl]
            serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

            [include]
            files = /etc/supervisor/conf.d/*.conf

            [inet_http_server]
            port = 127.0.0.1:9001

3- 从创建主管进程文件配置主管laravel 文档中的部分：

files: 
    /etc/supervisor/conf.d/laravel-worker.conf: 
        content: |
            [program:laravel-worker]
            process_name=%(program_name)s_%(process_num)02d
            command=php /var/app/current/artisan queue:work database --sleep=3 --tries=3
            autostart=true
            autorestart=true
            ;user=root
            numprocs=1
            redirect_stderr=true
            ;stdout_logfile=/var/app/current/storage/logs/worker.log
            stopwaitsecs=3600

4- 创建运行主管的服务。它与这个答案添加了 chkconfig 和 processname 行。这些将允许我们稍后将其作为服务运行。

files:
    /etc/init.d/supervisord:
        mode: "000755"
        owner: root
        group: root
        content: |
            #!/bin/bash

            #chkconfig: 345 99 76
            # processname: supervisord

            # Source function library
            . /etc/rc.d/init.d/functions

            # Source system settings
            if [ -f /etc/sysconfig/supervisord ]; then
                . /etc/sysconfig/supervisord
            fi

            # Path to the supervisorctl script, server binary,
            # and short-form for messages.
            supervisorctl=/usr/local/bin/supervisorctl
            supervisord=${SUPERVISORD-/usr/local/bin/supervisord}
            prog=supervisord
            pidfile=${PIDFILE-/tmp/supervisord.pid}
            lockfile=${LOCKFILE-/var/lock/subsys/supervisord}
            STOP_TIMEOUT=${STOP_TIMEOUT-60}
            OPTIONS="${OPTIONS--c /usr/local/etc/supervisord.conf}"
            RETVAL=0

            start() {
                echo -n $"Starting $prog: "
                daemon --pidfile=${pidfile} $supervisord $OPTIONS
                RETVAL=$?
                echo
                if [ $RETVAL -eq 0 ]; then
                    touch ${lockfile}
                    $supervisorctl $OPTIONS status
                fi
                return $RETVAL
            }

            stop() {
                echo -n $"Stopping $prog: "
                killproc -p ${pidfile} -d ${STOP_TIMEOUT} $supervisord
                RETVAL=$?
                echo
                [ $RETVAL -eq 0 ] && rm -rf ${lockfile} ${pidfile}
            }

            reload() {
                echo -n $"Reloading $prog: "
                LSB=1 killproc -p $pidfile $supervisord -HUP
                RETVAL=$?
                echo
                if [ $RETVAL -eq 7 ]; then
                    failure $"$prog reload"
                else
                    $supervisorctl $OPTIONS status
                fi
            }

            restart() {
                stop
                start
            }

            case "$1" in
                start)
                    start
                    ;;
                stop)
                    stop
                    ;;
                status)
                    status -p ${pidfile} $supervisord
                    RETVAL=$?
                    [ $RETVAL -eq 0 ] && $supervisorctl $OPTIONS status
                    ;;
                restart)
                    restart
                    ;;
                condrestart|try-restart)
                    if status -p ${pidfile} $supervisord >&/dev/null; then
                    stop
                    start
                    fi
                    ;;
                force-reload|reload)
                    reload
                    ;;
                *)
                    echo $"Usage: $prog {start|stop|restart|condrestart|try-restart|force-reload|reload}"
                    RETVAL=2
                esac

                exit $RETVAL

5- 创建所有文件后，运行以下命令启动服务并添加服务以便进行管理：

commands:
  command-1: 
    command: "/etc/init.d/supervisord start"
  command-2:
    command: "chkconfig --add supervisord"

6-现在 services 关键字应该可以工作了，允许我们将 enabled 和 Ensurerunning 标志设置为 true。

services:
    sysvinit:
        supervisord:
            enabled: "true"
            ensureRunning: "true"
            files: 
                - "/usr/local/etc/supervisord.conf"

将所有这些放入 .config 文件中并部署以使队列正常工作。如果您想为旧镜像配置调度程序，也可以在这个答案，由于我还没有测试过，所以在这里就不更新了。

完整配置文件

记得更改 chkconfig 编号，并注意我正在运行迁移：新鲜命令

packages:
    python:
        supervisor: []
container_commands:
    01-migrate: 
        command: "php artisan migrate:fresh --seed"
        cwd: /var/app/ondeck
        leader_only: true
files: 
    /opt/elasticbeanstalk/tasks/taillogs.d/laravel-logs.conf: 
        content: /var/app/current/storage/logs/laravel.log
        group: root
        mode: "000755"
        owner: root
    /etc/supervisor/conf.d/laravel-worker.conf: 
        content: |
            [program:laravel-worker]
            process_name=%(program_name)s_%(process_num)02d
            command=php /var/app/current/artisan queue:work database --sleep=3 --tries=3
            autostart=true
            autorestart=true
            ;user=root
            numprocs=1
            redirect_stderr=true
            ;stdout_logfile=/var/app/current/storage/logs/worker.log
            stopwaitsecs=3600
    /usr/local/etc/supervisord.conf:
        mode: "000755"
        owner: root
        group: root
        content: |
            [unix_http_server]
            file=/tmp/supervisor.sock   ; (the path to the socket file)

            [supervisord]
            logfile=/tmp/supervisord.log ; (main log file;default $CWD/supervisord.log)
            logfile_maxbytes=50MB        ; (max main logfile bytes b4 rotation;default 50MB)
            logfile_backups=10           ; (num of main logfile rotation backups;default 10)
            loglevel=info                ; (log level;default info; others: debug,warn,trace)
            pidfile=/tmp/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
            nodaemon=false               ; (start in foreground if true;default false)
            minfds=1024                  ; (min. avail startup file descriptors;default 1024)
            minprocs=200                 ; (min. avail process descriptors;default 200)

            [rpcinterface:supervisor]
            supervisor.rpcinterface_factory = supervisor.rpcinterface:make_main_rpcinterface

            [supervisorctl]
            serverurl=unix:///tmp/supervisor.sock ; use a unix:// URL  for a unix socket

            [include]
            files = /etc/supervisor/conf.d/*.conf

            [inet_http_server]
            port = 127.0.0.1:9001
    /etc/init.d/supervisord:
        mode: "000755"
        owner: root
        group: root
        content: |
            #!/bin/bash

            #chkconfig: <number>
            # processname: supervisord

            # Source function library
            . /etc/rc.d/init.d/functions

            # Source system settings
            if [ -f /etc/sysconfig/supervisord ]; then
                . /etc/sysconfig/supervisord
            fi

            # Path to the supervisorctl script, server binary,
            # and short-form for messages.
            supervisorctl=/usr/local/bin/supervisorctl
            supervisord=${SUPERVISORD-/usr/local/bin/supervisord}
            prog=supervisord
            pidfile=${PIDFILE-/tmp/supervisord.pid}
            lockfile=${LOCKFILE-/var/lock/subsys/supervisord}
            STOP_TIMEOUT=${STOP_TIMEOUT-60}
            OPTIONS="${OPTIONS--c /usr/local/etc/supervisord.conf}"
            RETVAL=0

            start() {
                echo -n $"Starting $prog: "
                daemon --pidfile=${pidfile} $supervisord $OPTIONS
                RETVAL=$?
                echo
                if [ $RETVAL -eq 0 ]; then
                    touch ${lockfile}
                    $supervisorctl $OPTIONS status
                fi
                return $RETVAL
            }

            stop() {
                echo -n $"Stopping $prog: "
                killproc -p ${pidfile} -d ${STOP_TIMEOUT} $supervisord
                RETVAL=$?
                echo
                [ $RETVAL -eq 0 ] && rm -rf ${lockfile} ${pidfile}
            }

            reload() {
                echo -n $"Reloading $prog: "
                LSB=1 killproc -p $pidfile $supervisord -HUP
                RETVAL=$?
                echo
                if [ $RETVAL -eq 7 ]; then
                    failure $"$prog reload"
                else
                    $supervisorctl $OPTIONS status
                fi
            }

            restart() {
                stop
                start
            }

            case "$1" in
                start)
                    start
                    ;;
                stop)
                    stop
                    ;;
                status)
                    status -p ${pidfile} $supervisord
                    RETVAL=$?
                    [ $RETVAL -eq 0 ] && $supervisorctl $OPTIONS status
                    ;;
                restart)
                    restart
                    ;;
                condrestart|try-restart)
                    if status -p ${pidfile} $supervisord >&/dev/null; then
                    stop
                    start
                    fi
                    ;;
                force-reload|reload)
                    reload
                    ;;
                *)
                    echo $"Usage: $prog {start|stop|restart|condrestart|try-restart|force-reload|reload}"
                    RETVAL=2
                esac

                exit $RETVAL
                
commands:
  command-1: 
    command: "/etc/init.d/supervisord start"
  command-2:
    command: "chkconfig --add supervisord"
services:
    sysvinit:
        supervisord:
            enabled: "true"
            ensureRunning: "true"
            files: 
                - "/usr/local/etc/supervisord.conf"

答案2

@Aridez 的回答是正确的，但我必须做一些改变才能使我的队列正常工作。我希望这对其他人有帮助。

以下内容适用于我使用 Laravel 8、AWS​​ ElasticBeanstalk 中的 AWS SQS 和 Amazon Linux 2

我能够轻松地将作业推送到队列，但作业并没有被工作人员从队列中挑选出来。我花了一天时间才弄清楚，队列工作人员没有挑选环境变量，因此没有连接到 SQS。要解决这个问题，您需要EnvironmentFile在 中添加选项[Service]。

这里的后续问题是如何在文件中获取环境变量，因为我已经通过 AWS 控制台中的 EB 配置设置了它们。本文清楚地解释了如何通过钩子复制你的环境.platform/postdeploy。

这导致的后续问题是：Aridez 提供的配置通过启动工作程序，container_commands但在运行这些工作程序的阶段，不会像文章中描述的那样生成 env 文件。将命令移到 postdeploy 钩子中（如下所示）解决了这个问题。

我还将服务的用户和组更改为，webapp因为root感觉不安全，但一切运行正常webapp。

我想要为我的队列添加多个工作者，这很简单，只需@在服务名称末尾添加一个即可。检查使用一个 systemd 服务文件启动 N 个进程。下面的代码运行 3 个 worker。

.ebextensions/01_deploy.config

container_commands:

  01_run_migrations:
    command: "php artisan migrate --force"
    leader_only: true

files:
  /opt/elasticbeanstalk/tasks/taillogs.d/laravel-logs.conf:
      content: /var/app/current/storage/logs/laravel.log
      group: root
      mode: "000644"
      owner: root
  /etc/systemd/system/[email protected]:
      mode: "000644"
      owner: root
      group: root
      content: |
          [Unit]
          Description=Laravel queue worker

          [Service]
          User=webapp
          Group=webapp
          Restart=always
          EnvironmentFile=/opt/elasticbeanstalk/deployment/laravel_env
          ExecStart=/usr/bin/nohup /usr/bin/php /var/app/current/artisan queue:work

          [Install]
          WantedBy=multi-user.target

commands:
  remove_service_bak_file:
    command: "rm -f /etc/systemd/system/[email protected]"

.platform/hooks/postdeploy

#!/bin/bash
# https://aws.amazon.com/premiumsupport/knowledge-center/elastic-beanstalk-env-variables-linux2/

#Create a copy of the environment variable file.
cp /opt/elasticbeanstalk/deployment/env /opt/elasticbeanstalk/deployment/laravel_env

#Set permissions to the custom_env_var file so this file can be accessed by any user on the instance. You can restrict permissions as per your requirements.
chmod 644 /opt/elasticbeanstalk/deployment/laravel_env

#Remove duplicate files upon deployment.
rm -f /opt/elasticbeanstalk/deployment/*.bak

# Enable the workers
systemctl enable laravel_queue_worker@{1..3}.service

# Restart the workers
systemctl restart laravel_queue_worker@{1..3}.service

为了集成 Cloudwatch，你可以流式传输 laravel.log 文件，请检查此处关联 您需要设置 awslogs 包，然后.conf在中添加一个文件/etc/awslogs/config/。下面的配置文件执行此操作并将laravel.log文件流式传输到以 elastic beanstalk 环境名称为前缀的日志组中。

.ebextensions\03_logs.config

###################################################################################################
#### The following file installs and configures the AWS CloudWatch Logs agent to push logs to a Log
#### Group in CloudWatch Logs. The configuration below sets the logs to be pushed, the Log Group
#### name to push the logs to and the Log Stream name as the instance id.
####
#### /var/app/current/storage/logs/laravel.log
####
#### http://docs.aws.amazon.com/elasticbeanstalk/latest/dg/AWSHowTo.cloudwatchlogs.html
###################################################################################################

packages:
  yum:
    awslogs: []

files:
  "/etc/awslogs/awscli.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [plugins]
      cwlogs=cwlogs
      [default]
      region=`{"Ref":"AWS::Region"}`

  "/etc/awslogs/awslogs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [general]
      state_file=/var/lib/awslogs/agent-state

  "/etc/awslogs/config/logs.conf" :
    mode: "000600"
    owner: root
    group: root
    content: |
      [var/app/current/storage/log/laravel]
      log_group_name=`{"Fn::Join":["/", ["/aws/elasticbeanstalk", { "Ref":"AWSEBEnvironmentName" }, "var/app/current/storage/logs/laravel.log"]]}`
      log_stream_name={instance_id}
      file=/var/app/current/storage/logs/laravel.log

commands:
  "01":
    command: systemctl enable awslogsd.service
  "02":
    command: systemctl restart awslogsd

答案3

理想的做法是创建一个.ebextensions/01_queue_worker.config文件，其中包含启动命令的内容php artisan queue:work。

就像是：

container_commands:
  01_queue_worker:
    command: "php artisan queue:work"

现在，如果您不想在 Web 服务器上运行队列工作程序，而只想在单独的专用工作程序节点上运行，则可以创建一个名为“WORKER”的环境变量并将其设置为true。然后在 ebextensions 配置文件中，您可以测试该变量，并且仅在“WORKER”变量为“true”时运行脚本。它看起来像这样：

container_commands:
  01_queue_worker:
    test: '[ "${WORKER}" == "true" ]'
    command: "php artisan queue:work"

一般来说，任何时候你需要对 Elasticbeanstalk 上运行的内容进行任何修改，请尝试查看ebextensions。这是 AWS 用于进行修改的机制。

相关内容