程序员那些事儿

菜单

Henry
发布于 2024-01-17 / 268 阅读
0
0

Containerd 安装

#Containerd

背景简介

由于最新版的 k8s 不支持 docker了，因此学习 containerd 的使用，此篇介绍了如何安装 containerd。

环境配置

  1. 系统：Debian 6.1.69-1 (2023-12-30) x86_64 GNU/Linux
  2. cri-containerd-1.6.27

详细步骤

下载安装包

在 gitbub  中找到自己需要的 containerd 版本并下载。

参考链接：https://github.com/containerd/containerd

解压安装包

新建目录，以便存储解压内容

myserver@peag-k8s-master:~$ mkdir cri-containerd

解压指令（更具需要调整）

myserver@peag-k8s-master:~$ tar -xvzf cri-containerd-1.6.27-linux-amd64.tar.gz -C cri-containerd

Note:  也可直接解压到系统根目录 tar -C / -xzf cri-containerd-1.6.27-linux-amd64.tar.gz

解压后文件夹内容

myserver@peag-k8s-master:~$ tree cri-containerd
cri-containerd
├── cri-containerd.DEPRECATED.txt
├── etc
│   ├── crictl.yaml
│   └── systemd
│       └── system
│           └── containerd.service
├── opt
│   └── containerd
│       └── cluster
│           ├── gce
│           │   ├── cloud-init
│           │   │   ├── master.yaml
│           │   │   └── node.yaml
│           │   ├── cni.template
│           │   ├── configure.sh
│           │   └── env
│           └── version
└── usr
    └── local
        ├── bin
        │   ├── containerd
        │   ├── containerd-shim
        │   ├── containerd-shim-runc-v1
        │   ├── containerd-shim-runc-v2
        │   ├── containerd-stress
        │   ├── crictl
        │   ├── critest
        │   ├── ctd-decoder
        │   └── ctr
        └── sbin
            └── runc

13 directories, 19 files

安装 containerd

可执行文件配置

将 containerd 执行文件放入系统默认命令路径下

myserver@peag-k8s-master:~$ sudo cp -r cri-containerd/usr/local/* /usr/local/

配置环境变量

myserver@peag-k8s-master:~$ nano .bashrc

  ......
  
export PATH=$PATH:/usr/local/bin:/usr/local/sbin

执行指令使环境变量生效

myserver@peag-k8s-master:~$ source .bashrc

文件配置

复制 crictl.yaml

myserver@peag-k8s-master:~$ sudo cp cri-containerd/etc/crictl.yaml /etc/

复制 opt/containerd  配置文件至系统目录

myserver@peag-k8s-master:~$ sudo cp -r cri-containerd/opt/* /opt/

生成 containerd 默认配置文件

myserver@peag-k8s-master:~$ sudo mkdir /etc/containerd
myserver@peag-k8s-master:~$ sudo chown -R myserver /etc/containerd/
myserver@peag-k8s-master:~$ containerd config default > /etc/containerd/config.toml

启动containerd 服务

将 systemd 配置文件复制到系统目录

sudo cp cri-containerd/etc/systemd/system/* /etc/systemd/system/

启用 containerd 服务

myserver@peag-k8s-master:~$ sudo systemctl enable containerd
Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /etc/systemd/system/containerd.service.

Note: 也可使用 systemctl enable containerd --now 一键启动

查看状态

myserver@peag-k8s-master:~$ sudo systemctl status containerd
○ containerd.service - containerd container runtime
     Loaded: loaded (/etc/systemd/system/containerd.service; enabled; preset: enabled)
     Active: inactive (dead)
       Docs: https://containerd.io

启动服务

myserver@peag-k8s-master:~$ sudo systemctl start containerd
myserver@peag-k8s-master:~$ sudo systemctl status containerd
● containerd.service - containerd container runtime
     Loaded: loaded (/etc/systemd/system/containerd.service; enabled; preset: enabled)
     Active: active (running) since Mon 2024-01-15 16:49:10 HKT; 1min 20s ago
       Docs: https://containerd.io
    Process: 139601 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
   Main PID: 139603 (containerd)
      Tasks: 8
     Memory: 24.6M
        CPU: 204ms
     CGroup: /system.slice/containerd.service
             └─139603 /usr/local/bin/containerd

Jan 15 16:49:10 peag-k8s-master containerd[139603]: time="2024-01-15T16:49:10.395632027+08:00" level=info msg=serving... address=/run/containerd/containerd.sock.ttrpc
Jan 15 16:49:10 peag-k8s-master containerd[139603]: time="2024-01-15T16:49:10.395800862+08:00" level=info msg=serving... address=/run/containerd/containerd.sock
Jan 15 16:49:10 peag-k8s-master containerd[139603]: time="2024-01-15T16:49:10.396558811+08:00" level=info msg="Start subscribing containerd event"
Jan 15 16:49:10 peag-k8s-master containerd[139603]: time="2024-01-15T16:49:10.396820506+08:00" level=info msg="Start recovering state"
Jan 15 16:49:10 peag-k8s-master containerd[139603]: time="2024-01-15T16:49:10.397158171+08:00" level=info msg="Start event monitor"
Jan 15 16:49:10 peag-k8s-master containerd[139603]: time="2024-01-15T16:49:10.397381828+08:00" level=info msg="Start snapshots syncer"
Jan 15 16:49:10 peag-k8s-master containerd[139603]: time="2024-01-15T16:49:10.397429431+08:00" level=info msg="Start cni network conf syncer for default"
Jan 15 16:49:10 peag-k8s-master containerd[139603]: time="2024-01-15T16:49:10.397485088+08:00" level=info msg="Start streaming server"
Jan 15 16:49:10 peag-k8s-master containerd[139603]: time="2024-01-15T16:49:10.401048592+08:00" level=info msg="containerd successfully booted in 0.323465s"
Jan 15 16:49:10 peag-k8s-master systemd[1]: Started containerd.service - containerd container runtime.

启动完成后就可以使用 containerd 的本地 CLI 工具 ctr 指令了，比如查看版本：

myserver@peag-k8s-master:~$ sudo ctr version
Client:
  Version:  v1.6.27
  Revision: a1496014c916f9e62104b33d1bb5bd03b0858e59
  Go version: go1.20.13

Server:
  Version:  v1.6.27
  Revision: a1496014c916f9e62104b33d1bb5bd03b0858e59
  UUID: bd48bc0c-1a9c-4543-a79f-2104998537a5

查看 containerd 默认配置

配置文件内容较多，较为复杂，有需要请自行深入学习，本篇简单介绍一下 plugins。

myserver@peag-k8s-master:~$ cat /etc/containerd/config.toml 
disabled_plugins = []
imports = []
oom_score = 0
plugin_dir = ""
required_plugins = []
root = "/var/lib/containerd"
state = "/run/containerd"
temp = ""
version = 2

[cgroup]
  path = ""

[debug]
  address = ""
  format = ""
  gid = 0
  level = ""
  uid = 0

[grpc]
  address = "/run/containerd/containerd.sock"
  gid = 0
  max_recv_message_size = 16777216
  max_send_message_size = 16777216
  tcp_address = ""
  tcp_tls_ca = ""
  tcp_tls_cert = ""
  tcp_tls_key = ""
  uid = 0

[metrics]
  address = ""
  grpc_histogram = false

[plugins]

  [plugins."io.containerd.gc.v1.scheduler"]
    deletion_threshold = 0
    mutation_threshold = 100
    pause_threshold = 0.02
    schedule_delay = "0s"
    startup_delay = "100ms"

  [plugins."io.containerd.grpc.v1.cri"]
    device_ownership_from_security_context = false
    disable_apparmor = false
    disable_cgroup = false
    disable_hugetlb_controller = true
    disable_proc_mount = false
    disable_tcp_service = true
    enable_selinux = false
    enable_tls_streaming = false
    enable_unprivileged_icmp = false
    enable_unprivileged_ports = false
    ignore_image_defined_volumes = false
    max_concurrent_downloads = 3
    max_container_log_line_size = 16384
    netns_mounts_under_state_dir = false
    restrict_oom_score_adj = false
    sandbox_image = "registry.k8s.io/pause:3.6"
    selinux_category_range = 1024
    stats_collect_period = 10
    stream_idle_timeout = "4h0m0s"
    stream_server_address = "127.0.0.1"
    stream_server_port = "0"
    systemd_cgroup = false
    tolerate_missing_hugetlb_controller = true
    unset_seccomp_profile = ""

    [plugins."io.containerd.grpc.v1.cri".cni]
      bin_dir = "/opt/cni/bin"
      conf_dir = "/etc/cni/net.d"
      conf_template = ""
      ip_pref = ""
      max_conf_num = 1

    [plugins."io.containerd.grpc.v1.cri".containerd]
      default_runtime_name = "runc"
      disable_snapshot_annotations = true
      discard_unpacked_layers = false
      ignore_rdt_not_enabled_errors = false
      no_pivot = false
      snapshotter = "overlayfs"

      [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime]
        base_runtime_spec = ""
        cni_conf_dir = ""
        cni_max_conf_num = 0
        container_annotations = []
        pod_annotations = []
        privileged_without_host_devices = false
        runtime_engine = ""
        runtime_path = ""
        runtime_root = ""
        runtime_type = ""

        [plugins."io.containerd.grpc.v1.cri".containerd.default_runtime.options]

      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes]

        [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
          base_runtime_spec = ""
          cni_conf_dir = ""
          cni_max_conf_num = 0
          container_annotations = []
          pod_annotations = []
          privileged_without_host_devices = false
          runtime_engine = ""
          runtime_path = ""
          runtime_root = ""
          runtime_type = "io.containerd.runc.v2"

          [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]
            BinaryName = ""
            CriuImagePath = ""
            CriuPath = ""
            CriuWorkPath = ""
            IoGid = 0
            IoUid = 0
            NoNewKeyring = false
            NoPivotRoot = false
            Root = ""
            ShimCgroup = ""
            SystemdCgroup = false

      [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime]
        base_runtime_spec = ""
        cni_conf_dir = ""
        cni_max_conf_num = 0
        container_annotations = []
        pod_annotations = []
        privileged_without_host_devices = false
        runtime_engine = ""
        runtime_path = ""
        runtime_root = ""
        runtime_type = ""

        [plugins."io.containerd.grpc.v1.cri".containerd.untrusted_workload_runtime.options]

    [plugins."io.containerd.grpc.v1.cri".image_decryption]
      key_model = "node"

    [plugins."io.containerd.grpc.v1.cri".registry]
      config_path = ""

      [plugins."io.containerd.grpc.v1.cri".registry.auths]

      [plugins."io.containerd.grpc.v1.cri".registry.configs]

      [plugins."io.containerd.grpc.v1.cri".registry.headers]

      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]

    [plugins."io.containerd.grpc.v1.cri".x509_key_pair_streaming]
      tls_cert_file = ""
      tls_key_file = ""

  [plugins."io.containerd.internal.v1.opt"]
    path = "/opt/containerd"

  [plugins."io.containerd.internal.v1.restart"]
    interval = "10s"

  [plugins."io.containerd.internal.v1.tracing"]
    sampling_ratio = 1.0
    service_name = "containerd"

  [plugins."io.containerd.metadata.v1.bolt"]
    content_sharing_policy = "shared"

  [plugins."io.containerd.monitor.v1.cgroups"]
    no_prometheus = false

  [plugins."io.containerd.runtime.v1.linux"]
    no_shim = false
    runtime = "runc"
    runtime_root = ""
    shim = "containerd-shim"
    shim_debug = false

  [plugins."io.containerd.runtime.v2.task"]
    platforms = ["linux/amd64"]
    sched_core = false

  [plugins."io.containerd.service.v1.diff-service"]
    default = ["walking"]

  [plugins."io.containerd.service.v1.tasks-service"]
    rdt_config_file = ""

  [plugins."io.containerd.snapshotter.v1.aufs"]
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.btrfs"]
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.devmapper"]
    async_remove = false
    base_image_size = ""
    discard_blocks = false
    fs_options = ""
    fs_type = ""
    pool_name = ""
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.native"]
    root_path = ""

  [plugins."io.containerd.snapshotter.v1.overlayfs"]
    mount_options = []
    root_path = ""
    sync_remove = false
    upperdir_label = false

  [plugins."io.containerd.snapshotter.v1.zfs"]
    root_path = ""

  [plugins."io.containerd.tracing.processor.v1.otlp"]
    endpoint = ""
    insecure = false
    protocol = ""

[proxy_plugins]

[stream_processors]

  [stream_processors."io.containerd.ocicrypt.decoder.v1.tar"]
    accepts = ["application/vnd.oci.image.layer.v1.tar+encrypted"]
    args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
    env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
    path = "ctd-decoder"
    returns = "application/vnd.oci.image.layer.v1.tar"

  [stream_processors."io.containerd.ocicrypt.decoder.v1.tar.gzip"]
    accepts = ["application/vnd.oci.image.layer.v1.tar+gzip+encrypted"]
    args = ["--decryption-keys-path", "/etc/containerd/ocicrypt/keys"]
    env = ["OCICRYPT_KEYPROVIDER_CONFIG=/etc/containerd/ocicrypt/ocicrypt_keyprovider.conf"]
    path = "ctd-decoder"
    returns = "application/vnd.oci.image.layer.v1.tar+gzip"

[timeouts]
  "io.containerd.timeout.bolt.open" = "0s"
  "io.containerd.timeout.shim.cleanup" = "5s"
  "io.containerd.timeout.shim.load" = "5s"
  "io.containerd.timeout.shim.shutdown" = "3s"
  "io.containerd.timeout.task.state" = "2s"

[ttrpc]
  address = ""
  gid = 0
  uid = 0

我们可以发现每一个配置块的命名都是 plugins."io.containerd.xxx.vx.xxx" 这种形式，每一个配置块都表示一个插件，其中 io.containerd.xxx.vx 表示插件的类型，vx 后面的 xxx 表示插件的 ID，我们可以通过 ctr 查看插件列表：

myserver@peag-k8s-master:~$ sudo ctr plugin ls 
TYPE                                  ID                       PLATFORMS      STATUS    
io.containerd.content.v1              content                  -              ok        
io.containerd.snapshotter.v1          aufs                     linux/amd64    skip      
io.containerd.snapshotter.v1          btrfs                    linux/amd64    skip      
io.containerd.snapshotter.v1          devmapper                linux/amd64    error     
io.containerd.snapshotter.v1          native                   linux/amd64    ok        
io.containerd.snapshotter.v1          overlayfs                linux/amd64    ok        
io.containerd.snapshotter.v1          zfs                      linux/amd64    skip      
io.containerd.metadata.v1             bolt                     -              ok        
io.containerd.differ.v1               walking                  linux/amd64    ok        
io.containerd.event.v1                exchange                 -              ok        
io.containerd.gc.v1                   scheduler                -              ok        
io.containerd.warning.v1              deprecations             -              ok        
io.containerd.service.v1              introspection-service    -              ok        
io.containerd.service.v1              containers-service       -              ok        
io.containerd.service.v1              content-service          -              ok        
io.containerd.service.v1              diff-service             -              ok        
io.containerd.service.v1              images-service           -              ok        
io.containerd.service.v1              leases-service           -              ok        
io.containerd.service.v1              namespaces-service       -              ok        
io.containerd.service.v1              snapshots-service        -              ok        
io.containerd.runtime.v1              linux                    linux/amd64    ok        
io.containerd.runtime.v2              task                     linux/amd64    ok        
io.containerd.monitor.v1              cgroups                  linux/amd64    ok        
io.containerd.service.v1              tasks-service            -              ok        
io.containerd.grpc.v1                 introspection            -              ok        
io.containerd.internal.v1             restart                  -              ok        
io.containerd.grpc.v1                 containers               -              ok        
io.containerd.grpc.v1                 content                  -              ok        
io.containerd.grpc.v1                 diff                     -              ok        
io.containerd.grpc.v1                 events                   -              ok        
io.containerd.grpc.v1                 healthcheck              -              ok        
io.containerd.grpc.v1                 images                   -              ok        
io.containerd.grpc.v1                 leases                   -              ok        
io.containerd.grpc.v1                 namespaces               -              ok        
io.containerd.internal.v1             opt                      -              ok        
io.containerd.grpc.v1                 snapshots                -              ok        
io.containerd.grpc.v1                 tasks                    -              ok        
io.containerd.grpc.v1                 version                  -              ok        
io.containerd.tracing.processor.v1    otlp                     -              skip      
io.containerd.internal.v1             tracing                  -              ok        
io.containerd.grpc.v1                 cri                      linux/amd64    ok

默认存储路径配置

root = "/var/lib/containerd"
state = "/run/containerd"
  • root 是用来保存持久化数据，包括 Snapshots, Content, Metadata 以及各种插件的数据，每一个插件都有自己单独的目录，Containerd 本身不存储任何数据，它的所有功能都来自于已加载的插件。
  • state 是用来保存运行时的临时数据的，包括 sockets、pid、挂载点、运行时状态以及不需要持久化的插件数据。

配置镜像加速

添加 docker.io 和 k8s.gcr.io

    [plugins."io.containerd.grpc.v1.cri".registry]
      config_path = ""


		......

      [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
                endpoint = ["Https://registry.docker-cn.com"]
        [plugins."io.containerd.grpc.v1.cri".registry.mirrors."k8s.gcr.io"]
                endpoint = ["https://registry.aliyuncs.com/google_containers"]
                
        ......

crt 的使用简介

详细操作请参考笔记：ctr

指令使用帮助信息

 myserver@peag-k8s-master:~$ ctr --help
NAME:
   ctr - 
        __
  _____/ /______
 / ___/ __/ ___/
/ /__/ /_/ /
\___/\__/_/

containerd CLI


USAGE:
   ctr [global options] command [command options] [arguments...]

VERSION:
   v1.6.27

DESCRIPTION:
   
ctr is an unsupported debug and administrative client for interacting
with the containerd daemon. Because it is unsupported, the commands,
options, and operations are not guaranteed to be backward compatible or
stable from release to release of the containerd project.

COMMANDS:
   plugins, plugin            provides information about containerd plugins
   version                    print the client and server versions
   containers, c, container   manage containers
   content                    manage content
   events, event              display containerd events
   images, image, i           manage images
   leases                     manage leases
   namespaces, namespace, ns  manage namespaces
   pprof                      provide golang pprof outputs for containerd
   run                        run a container
   snapshots, snapshot        manage snapshots
   tasks, t, task             manage tasks
   install                    install a new package
   oci                        OCI tools
   deprecations               
   shim                       interact with a shim directly
   help, h                    Shows a list of commands or help for one command

GLOBAL OPTIONS:
   --debug                      enable debug output in logs
   --address value, -a value    address for containerd's GRPC server (default: "/run/containerd/containerd.sock") [$CONTAINERD_ADDRESS]
   --timeout value              total timeout for ctr commands (default: 0s)
   --connect-timeout value      timeout for connecting to containerd (default: 0s)
   --namespace value, -n value  namespace to use with commands (default: "default") [$CONTAINERD_NAMESPACE]
   --help, -h                   show help
   --version, -v                print the version
Containerd Pull a...
containerd ctr 指令

评论