Gitlab docker 更新后数据库迁移时出错(重启循环)

Gitlab docker 更新后数据库迁移时出错(重启循环)

我刚刚更新了我的 gitlab docker 实例,在主机上使用 docker-compose 运行。我遇到以下错误:

Recipe: gitlab::database_migrations
  * bash[migrate gitlab-rails database] action run
    [execute] rake aborted!
              PG::ConnectionBad: could not connect to server: No such file or directory
                Is the server running locally and accepting
                connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
              /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:48:in `block (3 levels) in <top (required)>'
              /opt/gitlab/embedded/bin/bundle:23:in `load'
              /opt/gitlab/embedded/bin/bundle:23:in `<main>'
              Tasks: TOP => gitlab:db:configure
              (See full trace by running task with --trace)
    
    ================================================================================
    Error executing action `run` on resource 'bash[migrate gitlab-rails database]'
    ================================================================================
    
    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    Expected process to exit with [0], but received '1'
    ---- Begin output of "bash"  "/tmp/chef-script20200707-23-1sorsih" ----
    STDOUT: rake aborted!
    PG::ConnectionBad: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
    /opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:48:in `block (3 levels) in <top (required)>'
    /opt/gitlab/embedded/bin/bundle:23:in `load'
    /opt/gitlab/embedded/bin/bundle:23:in `<main>'
    Tasks: TOP => gitlab:db:configure
    (See full trace by running task with --trace)
    STDERR: 
    ---- End output of "bash"  "/tmp/chef-script20200707-23-1sorsih" ----
    Ran "bash"  "/tmp/chef-script20200707-23-1sorsih" returned 1
    
    Resource Declaration:
    ---------------------
    # In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb
    
     55: bash "migrate gitlab-rails database" do
     56:   code <<-EOH
     57:     set -e
     58:     log_file="#{node['gitlab']['gitlab-rails']['log_directory']}/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log"
     59:     umask 077
     60:     /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}
     61:     STATUS=${PIPESTATUS[0]}
     62:     chown #{account_helper.gitlab_user}:#{account_helper.gitlab_group} ${log_file}
     63:     echo $STATUS > #{db_migrate_status_file}
     64:     exit $STATUS
     65:   EOH
     66:   environment env_variables unless env_variables.empty?
     67:   notifies :run, "execute[clear the gitlab-rails cache]", :immediately
     68:   dependent_services.each do |svc|
     69:     notifies :restart, svc, :immediately
     70:   end
     71:   not_if "(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)"
     72:   only_if { node['gitlab']['gitlab-rails']['auto_migrate'] }
     73: end
    
    Compiled Resource:
    ------------------
    # Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:55:in `from_file'
    
    bash("migrate gitlab-rails database") do
      action [:run]
      default_guard_interpreter :default
      command nil
      backup 5
      interpreter "bash"
      declared_type :bash
      cookbook_name "gitlab"
      recipe_name "database_migrations"
      code "    set -e\n    log_file=\"/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log\"\n    umask 077\n    /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}\n    STATUS=${PIPESTATUS[0]}\n    chown git:git ${log_file}\n    echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-8411f47246f\n    exit $STATUS\n"
      domain nil
      user nil
      not_if "(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-8411f47246f) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-8411f47246f | grep -Fx 0)"
      only_if { #code block }
    end
    
    System Info:
    ------------
    chef_version=14.14.29
    platform=ubuntu
    platform_version=16.04
    ruby=ruby 2.6.6p146 (2020-03-31 revision 67876) [x86_64-linux]
    program_name=/opt/gitlab/embedded/bin/chef-client
    executable=/opt/gitlab/embedded/bin/chef-client
    

Running handlers:
There was an error running gitlab-ctl reconfigure:

bash[migrate gitlab-rails database] (gitlab::database_migrations line 55) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash"  "/tmp/chef-script20200707-23-1sorsih" ----
STDOUT: rake aborted!
PG::ConnectionBad: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?
/opt/gitlab/embedded/service/gitlab-rails/lib/tasks/gitlab/db.rake:48:in `block (3 levels) in <top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:db:configure
(See full trace by running task with --trace)
STDERR: 
---- End output of "bash"  "/tmp/chef-script20200707-23-1sorsih" ----
Ran "bash"  "/tmp/chef-script20200707-23-1sorsih" returned 1

Running handlers complete
Chef Client failed. 9 resources updated in 18 seconds
Thank you for using GitLab Docker Image!
Current version: gitlab-ce=13.1.3-ce.0

Configure GitLab for your system by editing /etc/gitlab/gitlab.rb file
And restart this container to reload settings.
To do it use docker exec:

  docker exec -it gitlab vim /etc/gitlab/gitlab.rb
  docker restart gitlab

For a comprehensive list of configuration options please see the Omnibus GitLab readme
https://gitlab.com/gitlab-org/omnibus-gitlab/blob/master/README.md

If this container fails to start due to permission problems try to fix it by executing:

  docker exec -it gitlab update-permissions
  docker restart gitlab

我见过但没有答案可以解决我的问题。

如果需要更多信息,请随时发表评论。

更新:

根据@fuero 的回答,我遇到了几个问题:

  1. 我在尝试运行 docker 命令时遇到了 tty 错误(可能无关,但我还是找到了解决方法)。我必须先分离容器。

  2. /assets/wrapper 导致机器崩溃,因此我忽略了这一步,因为使用 docker-compose up 它可以正常运行。

  3. postgresql 包不存在于存储库中,所以我不得不手动添加它们。

  4. 由于我在 /,因此检查时出现权限错误。移至 /tmp 解决了该问题。

  5. 我现在仍然处于--检查另一个错误:Old cluster data and binary directories are from different major versions.我目前还找不到解决办法。

答案1

同样的事情也发生在我这里。查看 Postgresql 日志,它包含:

The data directory was initialized by PostgreSQL version 10, which is not compatible with this version 11.7.

似乎docker版本无法正确升级其数据库。

我现在已经恢复到旧版本(12.7)——如果不手动升级数据库,13.x 似乎就会损坏。以下是某人是如何做到这一点的,包括将内容作为要保留的引用:

在 gitlab-ce 13.1.2 容器内将 PostgreSQL 10 升级到 11

使用 docker-compose 启动我们的容器将在其中启动 gitlab。由于数据库故障,容器每分钟重新启动一次。我们不希望在迁移工作中出现这种情况。因此我们手动启动它并在其中访问 bash。

请至少用您的自定义配置替换主机名和其他选项!

docker run --rm \
--hostname gitlab.example.org \
--publish 8143:443 --publish 8180:80 \
--env GITLAB_OMNIBUS_CONFIG="external_url 'https://gitlab.example.org'; letsencrypt['enabled'] = false" \
--name gitlab_custom \
--volume /srv/gitlab/config:/etc/gitlab \
--volume /srv/gitlab/logs:/var/log/gitlab \
--volume /srv/gitlab/data:/var/opt/gitlab \
-it gitlab/gitlab-ce:latest /bin/bash

现在我们在docker容器中。

首先运行默认的入口脚本来检查它是否真的不起作用!

/assets/wrapper

如果 Recipe:gitlab::database_migrations 部分出现故障,且至少出现错误消息在资源“bash[migrate gitlab-rails database]”上执行操作“run”时出错,则继续迁移数据库。

我们需要安装默认的(旧版本和新版本)PostgreSQL 版本。

apt update
apt install postgresql-10 postgresql-11

创建迁移数据库的目标文件夹,并将目录所有者更新为 gitlab-psql。我们假设“旧”(版本 10)数据库数据位于 /var/opt/gitlab/postgresql/data。

mkdir -p /tmp/11/data
chown -R gitlab-psql /tmp/11 /var/opt/gitlab/postgresql/data

现在我们需要初始化新的数据库结构。

sudo -u gitlab-psql /usr/lib/postgresql/11/bin/initdb -D /tmp/11/data

查看 /var/opt/gitlab/postgresql/data/postgresql.conf 以找到正确的端口(并使用 -p 进行设置)。

在运行真正的迁移之前,我们应该检查数据库的访问权限和模式。如果出现错误,可以多次运行此命令。

sudo -u gitlab-psql /usr/lib/postgresql/11/bin/pg_upgrade -D /tmp/11/data -d /var/opt/gitlab/postgresql/data/ -v -b /usr/lib/postgresql/10/bin -B /usr/lib/postgresql/11/bin/ -p 5432 --check

如果检查过程中出现错误,您可能需要更新一些其他目录的权限。

如果检查成功,请使用以下命令执行数据库迁移。这可能需要一些时间,具体取决于您的 gitlab 实例的大小。

sudo -u gitlab-psql /usr/lib/postgresql/11/bin/pg_upgrade -D /tmp/11/data -d /var/opt/gitlab/postgresql/data/ -v -b /usr/lib/postgresql/10/bin -B /usr/lib/postgresql/11/bin/ -p 5432

停止所有gitlab服务,备份旧数据库并将迁移的数据移至原路径。

gitlab-ctl stop
rm -f /var/opt/gitlab/postgresql/.s.PGSQL.5432*
mv /var/opt/gitlab/postgresql/data /var/opt/gitlab/postgresql/data_old_10
mv /tmp/11/data /var/opt/gitlab/postgresql/data

现在运行默认入口脚本来完成内置的 gitlab 迁移。

/assets/wrapper

如果它有效(加载 gitlab 部分后出现一些日志输出,当然网站也可以正常工作),您可以按 Ctrl+C,然后输入 exit 离开并删除当前容器。

如果仍然有数据库错误,...哎呀。

如果出现这样的错误,

PG::ConnectionBad: could not connect to server: No such file or directory
     |                Is the server running locally and accepting
     |                connections on Unix domain socket "/var/opt/gitlab/postgresql/.s.PGSQL.5432"?

确保没有 postgresql 进程正在运行,删除套接字文件并重试。

此后,离开容器。

答案2

按照@fuero 的回答可以得到一个有效的安装,但是gitlab.rb必须手动更新以停止在不需要迁移时发生的数据库迁移。

按照这些步骤操作后出现错误Old cluster data and binary directories are from different major versions是因为数据库已经迁移了一半。解决方法是对新旧二进制文件都使用版本 11。

相关内容