据我了解层次结构 13 指向“name=systemd”,我一直在尝试使用以下命令在 docker 开发环境中运行一个稍微简单的 lxc-execute:
docker run -i -t --cap-add=SYS_PTRACE --cap-add=SYS_ADMIN lxc-build:phantomjs-test
lxc.conf
lxc.arch = x86_64
lxc.rootfs.path = /home/tester
lxc.uts.name = phantom_container
lxc.idmap = u 0 1000 65536
lxc.idmap = g 0 1000 65536
lxc.pty.max = 1
lxc.tty.max = 1
lxc.execute.cmd = phantomjs
lxc-执行
/home/tester # XDG_RUNTIME_DIR='' lxc-execute --name bork -f /home/tester/default.conf -l TRACE -o okay.log
痕迹
/home/tester # cat okay.log
INFO lxc_confile - confile.c:set_config_idmaps:1575 - Read uid map: type u nsid 0 hostid 1000 range 65536
INFO lxc_confile - confile.c:set_config_idmaps:1575 - Read uid map: type g nsid 0 hostid 1000 range 65536
TRACE lxc_commands - commands.c:lxc_cmd_init:1323 - Creating abstract unix socket "/usr/local/var/lib/lxc/bork/command"
TRACE lxc_start - start.c:lxc_init_handler:563 - Unix domain socket 4 for command server is ready
TRACE lxc_start - start.c:lxc_init:578 - initialized LSM
TRACE lxc_start - start.c:lxc_init:584 - read seccomp policy
TRACE lxc_start - start.c:lxc_serve_state_clients:350 - Set container state to STARTING
TRACE lxc_start - start.c:lxc_serve_state_clients:353 - No state clients registered
TRACE lxc_start - start.c:lxc_init:591 - set container state to "STARTING"
TRACE lxc_start - start.c:lxc_init:619 - set environment variables
TRACE lxc_start - start.c:lxc_init:625 - ran pre-start hooks
DEBUG lxc_start - start.c:setup_signal_fd:278 - Set SIGCHLD handler with file descriptor: 5.
TRACE lxc_start - start.c:lxc_init:636 - set up signal fd
DEBUG console - console.c:lxc_console_peer_default:513 - using "/dev/tty" as peer tty device
DEBUG console - console.c:lxc_console_signal_init:178 - Created signal fd 9
DEBUG console - console.c:lxc_console_winsz:86 - Set window size to 204 columns and 59 rows
TRACE lxc_start - start.c:lxc_init:643 - created console
TRACE lxc_conf - conf.c:lxc_ttys_shift_ids:2870 - chowned console "/dev/pts/1"
TRACE lxc_start - start.c:lxc_init:649 - shifted tty ids
INFO lxc_start - start.c:lxc_init:651 - container "bork" is initialized
DEBUG storage - storage/storage.c:storage_query:252 - Detected rootfs type "dir"
INFO lxc_cgroup - cgroups/cgroup.c:cgroup_init:67 - cgroup driver cgroupfs initing for bork
# I've been really focused on this error
ERROR lxc_cgfs - cgroups/cgfs.c:lxc_cgroupfs_create:901 - Could not find writable mount point for cgroup hierarchy 13 while trying to create cgroup.
ERROR lxc_start - start.c:lxc_spawn:1245 - Failed creating cgroups.
DEBUG lxc_network - network.c:lxc_delete_network:3123 - Deleted network devices
TRACE lxc_start - start.c:lxc_serve_state_clients:350 - Set container state to ABORTING
TRACE lxc_start - start.c:lxc_serve_state_clients:353 - No state clients registered
ERROR lxc_start - start.c:__lxc_start:1503 - Failed to spawn container "bork".
TRACE lxc_start - start.c:lxc_serve_state_clients:350 - Set container state to STOPPING
TRACE lxc_start - start.c:lxc_serve_state_clients:353 - No state clients registered
TRACE lxc_start - start.c:lxc_serve_state_clients:350 - Set container state to STOPPED
TRACE lxc_start - start.c:lxc_serve_state_clients:353 - No state clients registered
我一直在尝试将一个可执行文件容器化,并且我开始怀疑我是否应该只使用 cgroup,因为容器中的整个容器似乎有点令人困惑。
tester@04de411cd0fe:~$ cat /proc/1/cgroup
13:name=systemd:/docker-ce/docker/04de411cd0fe22b59d5508bbfc137631d1867b9a56104d9db80ca66b061f4b1f
12:pids:/docker-ce/docker/04de411cd0fe22b59d5508bbfc137631d1867b9a56104d9db80ca66b061f4b1f
11:hugetlb:/docker-ce/docker/04de411cd0fe22b59d5508bbfc137631d1867b9a56104d9db80ca66b061f4b1f
10:net_prio:/docker-ce/docker/04de411cd0fe22b59d5508bbfc137631d1867b9a56104d9db80ca66b061f4b1f
9:perf_event:/docker-ce/docker/04de411cd0fe22b59d5508bbfc137631d1867b9a56104d9db80ca66b061f4b1f
8:net_cls:/docker-ce/docker/04de411cd0fe22b59d5508bbfc137631d1867b9a56104d9db80ca66b061f4b1f
7:freezer:/docker-ce/docker/04de411cd0fe22b59d5508bbfc137631d1867b9a56104d9db80ca66b061f4b1f
6:devices:/docker-ce/docker/04de411cd0fe22b59d5508bbfc137631d1867b9a56104d9db80ca66b061f4b1f
5:memory:/docker-ce/docker/04de411cd0fe22b59d5508bbfc137631d1867b9a56104d9db80ca66b061f4b1f
4:blkio:/docker-ce/docker/04de411cd0fe22b59d5508bbfc137631d1867b9a56104d9db80ca66b061f4b1f
3:cpuacct:/docker-ce/docker/04de411cd0fe22b59d5508bbfc137631d1867b9a56104d9db80ca66b061f4b1f
2:cpu:/docker-ce/docker/04de411cd0fe22b59d5508bbfc137631d1867b9a56104d9db80ca66b061f4b1f
1:cpuset:/docker-ce/docker/04de411cd0fe22b59d5508bbfc137631d1867b9a56104d9db80ca66b061f4b1f
在什么情况下需要将 systemd 安装为可写?为什么 lxc/systemd 会找我挂载 systemd?
tester@04de411cd0fe:~$ mount | grep cgroup
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,relatime,mode=755)
cpuset on /sys/fs/cgroup/cpuset type cgroup (ro,nosuid,nodev,noexec,relatime,cpuset)
cpu on /sys/fs/cgroup/cpu type cgroup (ro,nosuid,nodev,noexec,relatime,cpu)
cpuacct on /sys/fs/cgroup/cpuacct type cgroup (ro,nosuid,nodev,noexec,relatime,cpuacct)
blkio on /sys/fs/cgroup/blkio type cgroup (ro,nosuid,nodev,noexec,relatime,blkio)
memory on /sys/fs/cgroup/memory type cgroup (ro,nosuid,nodev,noexec,relatime,memory)
devices on /sys/fs/cgroup/devices type cgroup (ro,nosuid,nodev,noexec,relatime,devices)
freezer on /sys/fs/cgroup/freezer type cgroup (ro,nosuid,nodev,noexec,relatime,freezer)
net_cls on /sys/fs/cgroup/net_cls type cgroup (ro,nosuid,nodev,noexec,relatime,net_cls)
perf_event on /sys/fs/cgroup/perf_event type cgroup (ro,nosuid,nodev,noexec,relatime,perf_event)
net_prio on /sys/fs/cgroup/net_prio type cgroup (ro,nosuid,nodev,noexec,relatime,net_prio)
hugetlb on /sys/fs/cgroup/hugetlb type cgroup (ro,nosuid,nodev,noexec,relatime,hugetlb)
pids on /sys/fs/cgroup/pids type cgroup (ro,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup type tmpfs (rw,relatime,mode=755)
cgroup on /sys/fs/cgroup type tmpfs (rw,relatime,mode=755)
/home/tester # cat /proc/cgroups
#subsys_name hierarchy num_cgroups enabled
cpuset 1 22 1
cpu 2 22 1
cpuacct 3 22 1
blkio 4 22 1
memory 5 23 1
devices 6 22 1
freezer 7 22 1
net_cls 8 22 1
perf_event 9 22 1
net_prio 10 22 1
hugetlb 11 22 1
pids 12 22 1
我尝试过在 ubuntu 17.04、16.04 和 alpine:latest 上使用 docker 镜像,结果都相同。我尝试过使用 lxc 2.07、2.08 和 2.1.1,大部分结果相同。
在我的学习过程中,我能够解决类似的错误:
- 尝试创建 cgroup 时找不到 cgroup 层次结构 4 的可写挂载点
- 尝试创建 cgroup 时找不到 cgroup 层次结构 5 的可写挂载点
- 尝试创建 cgroup 时找不到 cgroup 层次结构 6 的可写挂载点
- 尝试创建 cgroup 时找不到 cgroup 层次结构 7 的可写挂载点
- 尝试创建 cgroup 时找不到 cgroup 层次结构 8 的可写挂载点
- 尝试创建 cgroup 时找不到 cgroup 层次结构 9 的可写挂载点
- 尝试创建 cgroup 时找不到 cgroup 层次结构 10 的可写挂载点
- 尝试创建 cgroup 时找不到 cgroup 层次结构 11 的可写挂载点
- 尝试创建 cgroup 时找不到 cgroup 层次结构 12 的可写挂载点
使用 2 种不同的方法,我想我解决了前 12 个标准 cgroup 层次结构安装问题,但 13 个由于某种原因无法正常工作:
- 挂载-t cgroup cpuset -o cpuset /sys/fs/cgroup/cpuset/
- cd project && mkdir -p sys/fs/cgroup/{net_prio,pids,hugetlb,freezer,cpuset,cpu,devices,perf_event,memory,cpuacct,net_cls,blkio} # 然后使用 rootfs 或 mount.entry 类型配置
更多相关信息:
猫 /etc/*-release
3.6.2
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.6.2
PRETTY_NAME="Alpine Linux v3.6"
HOME_URL="http://alpinelinux.org"
BUG_REPORT_URL="http://bugs.alpinelinux.org"
uname -a
Linux 8fd6247b5665 4.9.60-linuxkit-aufs #1 SMP Mon Nov 6 16:00:12 UTC 2017 x86_64 Linux
lxc-执行--版本
2.1.0-devel
码头工人版本
Version 17.11.0-ce-mac40 (20561)
Channel: edge
d8fd0f1f4a
这些拉取请求的相关程度如何:
一些相关的想法/问题是我不应该在容器中使用容器,而应该使用完整的虚拟机(不能解决可用性问题)。
- 我可以尝试重新编译吗?系统?
- 也许与内核启动参数有关?
- 也许我使用了错误版本的 cgroupv1 和 cgroupv2?
- 是否可以禁止尝试挂载 cgroup 层次结构?
- 这与 docker 如何执行 /sbin/init 并启动初始 cgroup 命名空间有关吗?
任何帮助将不胜感激。
也可能相关:
- /sys/fs/cgroup/systemd/ && /sys/fs/cgroup/xxx/ 之间有什么区别
- https://stackoverflow.com/questions/29378051/inconsistency-runtime-kernel-parameters-in-docker-container-and-on-host
- https://github.com/systemd/systemd/issues/6092
- https://github.com/lxc/lxc/issues/1489#issuecomment-289971237
- https://github.com/moby/moby/issues/10176
因此,进行了更多研究,我也发现了此信息,其中显示了有关初始 cgroup cgfs 内容的更多信息,可能与以下内容相关name=systemd
:
/home/tester # cat /usr/share/doc/lxcfs/README.Debian
...
/home/tester # cat /usr/share/pam-configs/cgfs
Name: Create cgroups for user login sessions
Default: yes
Priority: 0
Session-Type: Additional
Session:
optional pam_cgfs.so -c freezer,memory,name=systemd
/home/tester # cat /etc/pam.d/common-session
session [default=1] pam_permit.so
session requisite pam_deny.so
session required pam_permit.so
session optional pam_umask.so
session required pam_unix.so
session optional pam_systemd.so
session optional pam_cgfs.so -c freezer,memory,name=systemd