使用 Nerdctl 构建多架构镜像

1. Nerdctl 安装

安装 Opscli

1
curl -sfL https://raw.githubusercontent.com/shaowenchen/ops/main/getcli.sh |VERSION=latest sh -

安装 Nerdctl

1
opscli task -f install-nerdctl --arch amd64

2. BuildKit

下载 Buildkit

1
wget https://github.com/moby/buildkit/releases/download/v0.19.0-rc2/buildkit-v0.19.0-rc2.linux-amd64.tar.gz

安装 Buildkit

1
2
tar xvf buildkit-*.tar.gz
mv bin/* /usr/local/bin/

配置 Buildkitd

1
mkdir -p /etc/buildkit /data/buildkit

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
cat > /etc/buildkit/buildkitd.toml <<EOF
debug = true
root = "/data/buildkit"
[worker.oci]
  enabled = false

[worker.containerd]
  address = "/run/containerd/containerd.sock"
  enabled = true
  platforms = [ "linux/amd64", "linux/arm64" ]
  namespace = "buildkit"
  gc = true
  gckeepstorage = 9000
  cniPoolSize = 16
EOF

生成 Systemd Unit 文件

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
cat > /etc/systemd/system/buildkitd.service << EOF
[Unit]
Description=buildkitd service
Documentation=https://github.com/moby/buildkit

[Service]
Environment="NYDUS_BUILDER=/usr/local/bin/nydus-image"
ExecStart=/usr/local/bin/buildkitd --config /etc/buildkit/buildkitd.toml

[Install]
WantedBy=multi-user.target
EOF

启动 Buildkitd 服务

1
2
3
systemctl enable buildkitd
systemctl start buildkitd
systemctl status buildkitd

3. 多架构配置

安装 Qemu

1
nerdctl run --privileged --rm registry.cn-beijing.aliyuncs.com/opshub/tonistiigi-binfmt:master --install all

查看 Qemu 配置

1
ls -1 /proc/sys/fs/binfmt_misc/qemu*

拉取指定架构镜像

1
nerdctl pull --platform=linux/arm64 registry.cn-beijing.aliyuncs.com/opshub/ubuntu:20.04

--all-platforms 可以拉取所有架构的镜像

运行指定架构容器

1
nerdctl run --rm --platform=linux/arm64 registry.cn-beijing.aliyuncs.com/opshub/ubuntu:20.04 uname -m

4. 镜像构建

创建一个测试 Dockerfile

1
2
3
4
cat << EOF >Dockerfile
FROM registry.cn-beijing.aliyuncs.com/opshub/ubuntu:20.04
RUN touch 123
EOF

构建多架构的镜像

1
nerdctl build --platform=amd64,arm64 -t registry-1.docker.io/shaowenchen/nerdctl-build:latest .

查看镜像信息

1
2
3
4
nerdctl images | grep nerdctl

registry-1.docker.io/shaowenchen/nerdctl-build                  latest                                                              3f2f8f7a36bd    About a minute ago    linux/amd64    77.9 MiB     26.2 MiB
registry-1.docker.io/shaowenchen/nerdctl-build                  latest                                                              3f2f8f7a36bd    About a minute ago    linux/arm64    0.0 B        24.8 MiB

推送多架构的镜像

1
nerdctl push --all-platforms registry-1.docker.io/shaowenchen/nerdctl-build:latest

5. Dockerfile 中的多架构变量

借助以下变量，可以下载不同架构的二进制文件，避免出现 exec format error 报错，镜像无法运行的问题。

TARGETPLATFORM

构建镜像的目标平台，例如 linux/amd64, linux/arm/v7, windows/amd64。

TARGETOS

TARGETPLATFORM 的 OS 类型，例如 linux, windows

TARGETARCH

TARGETPLATFORM 的架构类型，例如 amd64, arm

TARGETVARIANT

TARGETPLATFORM 的变种，该变量可能为空，例如 v7

BUILDPLATFORM

构建镜像主机平台，例如 linux/amd64

BUILDOS

BUILDPLATFORM 的 OS 类型，例如 linux

BUILDARCH

BUILDPLATFORM 的架构类型，例如 amd64

BUILDVARIANT

BUILDPLATFORM 的变种，该变量可能为空，例如 v7

使用方式:

Dockerfile 中声明需要使用的 ARG 变量
使用 ${TARGETARCH} 引用变量

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
FROM alpine:latest as builder
ARG TARGETARCH
WORKDIR /data
RUN wget https://github.com/git-lfs/git-lfs/releases/download/v3.4.0/git-lfs-linux-${TARGETARCH}-v3.4.0.tar.gz && \
    tar xvfz git-lfs-linux-${TARGETARCH}-v3.4.0.tar.gz

FROM alpine:latest
RUN apk update && \
    apk upgrade && \
    apk add --update alpine-sdk && \
    apk add --no-cache \
    git \
    openssh

COPY --from=builder /data/git-lfs-3.4.0/git-lfs /usr/local/bin/git-lfs
RUN git lfs install
WORKDIR /runtime
ENTRYPOINT ["git"]