问题背景

为了保持系统更新,笔者每隔一周会更新一次软件源。今天在尝试更新时,发现 APT 无法获取 MySql 的 Apt repository url,报错如下:

apt_mysql_public_key_is_not_available.webp
图 1: 公钥未找到,无法验签

Err:4 http://repo.mysql.com/apt/debian bullseye InRelease
  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 467B942D3A79BD29

W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://repo.mysql.com/apt/debian bullseye InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 467B942D3A79BD29
W: Failed to fetch http://repo.mysql.com/apt/debian/dists/bullseye/InRelease  The following signatures couldn't be verified because the public key is not available: NO_PUBKEY 467B942D3A79BD29
W: Some index files failed to download. They have been ignored, or old ones used instead.

简短回答

TL;DR

从问题的关键词来看,原因就是我们缺少对应的公钥,无法验证该软件源的签名。本着缺啥就补啥的修复思路,重新从公钥服务器(Key Server)获取 Public Key 即可。

Key Servers

已知的几个 Key Server 可供下载:

这些 Key Server 一般带有 Web 版的搜索界面,可以先在网页端查找库中是否存有我们想要的 Key ID。这个 ID 可以理解为该证书与其它证书的区别性特征,即 Key Fingerprint。

ubuntu_key_server_web_ui.jpg
图 2: Ubuntu Key Server

笔者先从报错信息中拿到 Key ID,在上述几个 Server 都进行了搜索,但只有 Ubuntu Key Server 找到了匹配的结果。

须注意:搜索的关键词若为十六进制的 Key ID,则需在开头加上 0x,比如 0x467B942D3A79BD29 ,不然系统当作普通字符串去搜索了。

key_id_search_result.jpg
图 3: Key ID Search Result

从 KS 获取公钥

我们将上面能找到匹配结果的 Key Server 和 Key ID 分别填到 apt-key 的命令行关键字参数后,开始下载公钥。相关的参数说明如下:

  • adv:意指 advanced options,这些高级选项将会被传递给 gpg 命令处理,即用于下载 key。
  • --keyserver:其中 Key Server 的 URL 是使用独有的 OpenPGP HTTP Keyserver Protocol (HKP) 协议类型,其以 hkp// 开头;类似于 httphttps 的关系,hkp 也有受 SSL/TLS 加密保护的 hkps(HKP over TLS)。
  • --recv-keys:要获取的 Key ID

receive_key.jpg
图 4: 获取公钥并加入 trusted 证书库

$ sudo apt-key adv --keyserver hkps://keyserver.ubuntu.com --recv-keys 0x467B942D3A79BD29

Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).
Executing: /tmp/apt-key-gpghome.4UClNKT1Z2/gpg.1.sh --keyserver hkps://keyserver.ubuntu.com --recv-keys 0x467B942D3A79BD29
gpg: key 467B942D3A79BD29: public key "MySQL Release Engineering <[email protected]>" imported
gpg: Total number processed: 1
gpg:               imported: 1

查看已导入的 Key

从下面命令的输出可以看到,我们成功导入一个有效期更长的 MySql 公钥,另一个旧公钥临近过期了(2022-02-16),再过半个月就失效了,想必这就是 MySQL 维护者 Oracle 更换 APT 包公钥的原因。

$ sudo apt-key list

/etc/apt/trusted.gpg
--------------------
pub   dsa1024 2003-02-03 [SCA] [expires: 2022-02-16]
      A4A9 4068 76FC BD3C 4567  70C8 8C71 8D3B 5072 E1F5
uid           [ unknown] MySQL Release Engineering <[email protected]>

pub   rsa4096 2021-12-14 [SC] [expires: 2023-12-14]
      859B E8D7 C586 F538 430B  19C2 467B 942D 3A79 BD29
uid           [ unknown] MySQL Release Engineering <[email protected]>
sub   rsa4096 2021-12-14 [E] [expires: 2023-12-14]

/etc/apt/trusted.gpg.d/debian-archive-bullseye-automatic.gpg
------------------------------------------------------------
...

重新 APT Update

在将缺少的公钥导入到系统信任库中后,笔者重新执行 apt update 这次没有报错,可以下载更新,可以验证签名,可以安装更新。Bingo~ 🎉

apt_update_successful.jpg
图 5: 正常更新软件源索引

$ sudo apt update
Hit:1 http://repo.mysql.com/apt/debian bullseye InRelease
Hit:2 http://deb.debian.org/debian bullseye InRelease
Hit:3 http://deb.debian.org/debian bullseye-updates InRelease
Hit:4 http://security.debian.org bullseye-security InRelease
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
9 packages can be upgraded. Run 'apt list --upgradable' to see them.

更好的答案

Deprecation warnings

细心的读者可能注意到,在执行 apt-key 获取公钥时,出现了这行警告,提示该命令已经弃用,建议我们直接使用 gpg 去管理 /etc/apt/trusted.gpg.d/ 下的密钥文件。

Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)).

在弃用 APT key 命令之前,我们先看看它背后帮我们做了些什么工作。在切换到 gpg 命令后,这些工作中哪部分是需要我们自己手动做的。

APT Key 背后原理

首先,利用 which 命令定位一下 apt-key 命令的出处:

$ which apt-key
/usr/bin/apt-key

拷贝一份出来,查看其内部实现,它是一个 8 百多行的 shell 脚本。它本身支持的命令比较多,我们直奔解析 adv 高级选项的相关代码。

解析命令行参数

# command 变量,它的值就是第一个命令行参数
command="$1"
# 必要参数如未传递,显示本命令的用法
if [ -z "$command" ]; then
    usage
    exit 1
fi
shift

# 若是请求非 `帮助` 或 `验证` 的话,准备 GPG 要用到的主目录。
if [ "$command" != 'help' ] && [ "$command" != 'verify' ]; then
    prepare_gpg_home
fi

case "$command" in
    add)
        ;;
    del|rm|remove)
        ;;
    update)
        ;;
    net-update)
        ;;
    list|finger*)
        ;;
    export|exportall)
        ;;
    # 解析将要传递给 gpg 的高级选项
    adv*)
        warn_on_script_usage
        # 环境准备:配置环境变量,文件系统等
        setup_merged_keyring
        # 准备执行:将待执行的命令和参数,通过 echo 输出到终端
        aptkey_echo "Executing: $GPG" "[email protected]"
        # 实际执行:从 Key Server 获取公钥
        aptkey_execute "$GPG" "[email protected]"
        # 将变更(若有)合并到本地的证书库
        merge_back_changes
        ;;
    verify)
        ;;
    help)
        usage
        ;;
    *)
        usage
        exit 1
        ;;
esac

实际执行的代码为 aptkey_execute "$GPG" "[email protected]"。其中 [email protected] 表示的是剩余的命令行参数,在我们的例子中就是后面两个具名参数 --keyserver hkps://keyserver.ubuntu.com --recv-keys 0x467B942D3A79BD29,那么 $GPG 代表什么呢?

$GPG 是什么?

# now tell gpg that it shouldn't try to maintain this trustdb file
echo "#!/bin/sh
exec '$(escape_shell "${GPG_EXE}")' --ignore-time-conflict --no-options --no-default-keyring \\
--homedir '$(escape_shell "${GPGHOMEDIR}")' --no-auto-check-trustdb --trust-model always \"\[email protected]\"" > "${GPGHOMEDIR}/gpg.0.sh"
GPG_SH="${GPGHOMEDIR}/gpg.0.sh"
# GPG 临时可执行文件.0
GPG="$GPG_SH"

# ... skip some lines of code ...

setup_merged_keyring() {
    # FORCE_KEYID 字符串变量的长度 > 0?
    if [ -n "$FORCED_KEYID" ]; then
        merge_all_trusted_keyrings_into_pubring
        FORCED_KEYRING="${GPGHOMEDIR}/forcedkeyid.gpg"
        TRUSTEDFILE="${FORCED_KEYRING}"
        echo "#!/bin/sh
exec sh '($(escape_shell "${GPG}")' --keyring '$(escape_shell "${TRUSTEDFILE}")' \"\[email protected]\"" > "${GPGHOMEDIR}/gpg.1.sh"
        GPG="${GPGHOMEDIR}/gpg.1.sh"
        # ignore error as this "just" means we haven't found the forced keyid and the keyring will be empty
        import_keyring_into_keyring '' "$TRUSTEDFILE" "$FORCED_KEYID" || true
    # FORCE_KEYID 字符串变量的长度 = 0?(✅匹配我们的命令行输入)
    elif [ -z "$FORCED_KEYRING" ]; then
        merge_all_trusted_keyrings_into_pubring
        if [ -r "${GPGHOMEDIR}/pubring.gpg" ]; then
            cp -a "${GPGHOMEDIR}/pubring.gpg" "${GPGHOMEDIR}/pubring.orig.gpg"
        else
           touch "${GPGHOMEDIR}/pubring.gpg" "${GPGHOMEDIR}/pubring.orig.gpg"
        fi
        # 构造临时可执行文件.1
        echo "#!/bin/sh
exec sh '$(escape_shell "${GPG}")' --keyring '$(escape_shell "${GPGHOMEDIR}/pubring.gpg")' \"\[email protected]\"" > "${GPGHOMEDIR}/gpg.1.sh"
        GPG="${GPGHOMEDIR}/gpg.1.sh"
    else
        TRUSTEDFILE="$(dearmor_filename "$FORCED_KEYRING")"
        create_new_keyring "$TRUSTEDFILE"
        echo "#!/bin/sh
exec sh '$(escape_shell "${GPG}")' --keyring '$(escape_shell "${TRUSTEDFILE}")' \"\[email protected]\"" > "${GPGHOMEDIR}/gpg.1.sh"
        GPG="${GPGHOMEDIR}/gpg.1.sh"
    fi
}

从上面的定义可知,GPG 先是赋值为 /tmp-path/gpg.0.sh,后面指向 /tmp-path/gpg.1.sh。而后者又调用了前者。

结合终端输出

还记得上面 aptkey_echo "Executing: $GPG" "[email protected]" 有记录日志到终端吗?摘选出来:

Executing: /tmp/apt-key-gpghome.4UClNKT1Z2/gpg.1.sh --keyserver hkps://keyserver.ubuntu.com --recv-keys 0x467B942D3A79BD29

结合其终端输出,可知 $GPG 最终指向 /tmp/apt-key-gpghome.4UClNKT1Z2/gpg.1.sh 这个临时文件。笔者尝试想把查看这些临时文件的内容,但是 apt-key 命令自带清理机制——执行完毕它们就自动删除了。自动删除临时文件的代码写得很棒👍,留在下一节分析🔬。

截留临时 SH 文件

为了截留这些临时可执行文件,最容易想到的方法就是在变动前各拷贝一份。笔者选在 setup_merged_keyring 调用的前、后时刻,分别保存 $GPG 变量指向的临时文件的内容,并记录如下:

$ cat /tmp/apt-key-gpghome.4UClNKT1Z2/gpg.1.sh
#!/bin/sh
exec sh '/tmp/apt-key-gpghome.4UClNKT1Z2/gpg.0.sh' --keyring '/tmp/apt-key-gpghome.4UClNKT1Z2/pubring.gpg' "[email protected]"

$ cat /tmp/apt-key-gpghome.4UClNKT1Z2/gpg.0.sh
#!/bin/sh
exec 'gpg' --ignore-time-conflict --no-options --no-default-keyring \
--homedir '/tmp/apt-key-gpghome.4UClNKT1Z2' --no-auto-check-trustdb --trust-model always "[email protected]"

以上两个嵌套调用的 sh 便是 apt-key 实际执行的 GPG 调用命令。

合并变更的 Keys

merge_back_changes() {
    if [ -n "$FORCED_KEYRING" ]; then
        # if the keyring was forced merge is already done
        if [ "$FORCED_KEYRING" != "$TRUSTEDFILE" ]; then
            mv -f "$FORCED_KEYRING" "${FORCED_KEYRING}~"
            export_key_from_to "$TRUSTEDFILE" "$FORCED_KEYRING" --armor
        fi
        return
    fi
    # -s 判断目标文件是否存在,且文件大小 > 0
    if [ -s "${GPGHOMEDIR}/pubring.gpg" ]; then
        # merge all updated keys
        foreach_keyring_do 'merge_keys_into_keyrings' "${GPGHOMEDIR}/pubring.gpg"
    fi
    # look for keys which were added or removed
    get_fingerprints_of_keyring "${GPGHOMEDIR}/pubring.orig.gpg" > "${GPGHOMEDIR}/pubring.orig.keylst"
    get_fingerprints_of_keyring "${GPGHOMEDIR}/pubring.gpg" > "${GPGHOMEDIR}/pubring.keylst"
    comm -3 "${GPGHOMEDIR}/pubring.keylst" "${GPGHOMEDIR}/pubring.orig.keylst" > "${GPGHOMEDIR}/pubring.diff"
    # key isn't part of new keyring, so remove
    cut -f 2 "${GPGHOMEDIR}/pubring.diff" | while read key; do
        if [ -z "$key" ]; then continue; fi
        foreach_keyring_do 'remove_key_from_keyring' "$key"
    done
    # key is only part of new keyring, so we need to import it
    cut -f 1 "${GPGHOMEDIR}/pubring.diff" | while read key; do
        if [ -z "$key" ]; then continue; fi
        import_keyring_into_keyring '' "$TRUSTEDFILE" "$key"
    done
}

临时可执行文件的生成

# gpg needs (in different versions more or less) files to function correctly,
# so we give it its own homedir and generate some valid content for it later on
create_gpg_home() {
    # for cases in which we want to cache a homedir due to expensive setup
    if [ -n "$GPGHOMEDIR" ]; then
        return
    fi
    if [ -n "$TMPDIR" ]; then
        # tmpdir is a directory and current user has rwx access to it
        # same tests as in apt-pkg/contrib/fileutl.cc GetTempDir()
        if [ ! -d "$TMPDIR" ] || [ ! -r "$TMPDIR" ] || [ ! -w "$TMPDIR" ] || [ ! -x "$TMPDIR" ]; then
            unset TMPDIR
        fi
    fi
    GPGHOMEDIR="$(mktemp --directory --tmpdir 'apt-key-gpghome.XXXXXXXXXX')"
    CURRENTTRAP="${CURRENTTRAP} cleanup_gpg_home;"
    trap "${CURRENTTRAP}" 0 HUP INT QUIT ILL ABRT FPE SEGV PIPE TERM
    if [ -z "$GPGHOMEDIR" ]; then
        apt_error "Could not create temporary gpg home directory in $TMPDIR (wrong permissions?)"
        exit 28
    fi
    chmod 700 "$GPGHOMEDIR"
}

(未完待续)

参考链接

  1. MySQL PPA Invalid Signature - Ask Ubuntu
  2. How do I fix the GPG error "NO_PUBKEY"? - Ask Ubuntu

添加新评论