improve docker cross-compilation

* support cross-compiling an x86-64 target on an arm64 host. This
  it turns out is a matter of *removing* an unnecessary dependency.
  (aarch64-linux-gnu-pkg-config exists but x86_64-linux-gnu-pkg-config
  doesn't. Turns out neither is necessary.) Added a comment explaining
  where ${gcc_target}-pkg-config comes from now.
* documentation tweaks
* improve debug output a bit
This commit is contained in:
Scott Lamb 2023-01-10 21:03:01 -08:00
parent 58e19265ef
commit 6ed23e90e8
No known key found for this signature in database
7 changed files with 32 additions and 27 deletions

View File

@ -4,7 +4,7 @@
# SPDX-License-Identifier: GPL-v3.0-or-later WITH GPL-3.0-linking-exception.
# See documentation here:
# https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/syntax.md
# https://github.com/moby/buildkit/blob/master/frontend/dockerfile/docs/reference.md
# "dev-common" is the portion of "dev" (see below) which isn't specific to the
# target arch. It's sufficient for building the non-arch-specific webpack.

View File

@ -13,9 +13,8 @@ mkdir /docker-build-debug/build-server
exec > >(tee -i /docker-build-debug/build-server/output) 2>&1
date
uname -a
ls -laFR /cargo-cache > /docker-build-debug/build-server/cargo-cache-before
ls -laFR /var/lib/moonfire-nvr/target \
> /docker-build-debug/build-server/target-before
find /cargo-cache -ls > /docker-build-debug/build-server/cargo-cache-before
find ~/target -ls > /docker-build-debug/build-server/target-before
source ~/.buildrc
@ -26,12 +25,13 @@ sudo chmod 777 /cargo-cache /var/lib/moonfire-nvr/target
mkdir -p /cargo-cache/{git,registry}
ln -s /cargo-cache/{git,registry} ~/.cargo
build_profile=release-lto
cd src/server
time cargo test
time cargo build --profile=release-lto
sudo install -m 755 ~/moonfire-nvr /usr/local/bin/moonfire-nvr
ls -laFR /cargo-cache > /docker-build-debug/build-server/cargo-cache-after
ls -laFR /var/lib/moonfire-nvr/target \
> /docker-build-debug/build-server/target-after
time cargo build --profile=$build_profile
find /cargo-cache -ls > /docker-build-debug/build-server/cargo-cache-after
find ~/target -ls > /docker-build-debug/build-server/target-after
sudo install -m 755 \
~/platform-target/$build_profile/moonfire-nvr \
/usr/local/bin/moonfire-nvr
date

View File

@ -19,6 +19,6 @@ npm --version
time npm ci
time npm run build
ls -laFR /var/lib/moonfire-nvr/src/ui/node_modules \
find /var/lib/moonfire-nvr/src/ui/node_modules -ls \
> /docker-build-debug/build-ui/node_modules-after
date

View File

@ -11,8 +11,7 @@ set -o xtrace
mkdir -p /docker-build-debug/deploy
exec > >(tee -i /docker-build-debug/deploy/output) 2>&1
ls -laFR /var/cache/apt \
> /docker-build-debug/deploy/var-cache-apt-before
find /var/cache/apt -ls > /docker-build-debug/deploy/var-cache-apt-before
date
uname -a
@ -29,6 +28,5 @@ time apt-get install --assume-yes --no-install-recommends \
rm -rf /var/lib/apt/lists/*
ln -s moonfire-nvr /usr/local/bin/nvr
ls -laFR /var/cache/apt \
> /docker-build-debug/deploy/var-cache-apt-after
find /var/cache/apt -ls > /docker-build-debug/deploy/var-cache-apt-after
date

View File

@ -15,7 +15,7 @@ exec > >(tee -i /docker-build-debug/dev-common/output) 2>&1
date
uname -a
ls -laFR /var/cache/apt > /docker-build-debug/dev-common/var-cache-apt-before
find /var/cache/apt -ls > /docker-build-debug/dev-common/var-cache-apt-before
export DEBIAN_FRONTEND=noninteractive
@ -78,5 +78,5 @@ export CARGO_BUILD_TARGET_DIR=/var/lib/moonfire-nvr/target
EOF
chown moonfire-nvr:moonfire-nvr /var/lib/moonfire-nvr/.buildrc
ls -laFR /var/cache/apt > /docker-build-debug/dev-common/var-cache-apt-after
find /var/cache/apt -ls > /docker-build-debug/dev-common/var-cache-apt-after
date

View File

@ -14,7 +14,7 @@ exec > >(tee -i /docker-build-debug/dev/output) 2>&1
date
uname -a
ls -laFR /var/cache/apt > /docker-build-debug/dev/var-cache-apt-before
find /var/cache/apt -ls > /docker-build-debug/dev/var-cache-apt-before
export DEBIAN_FRONTEND=noninteractive
@ -69,7 +69,6 @@ if [[ "${BUILDARCH}" != "${TARGETARCH}" ]]; then
packages+=(
g++-${gcc_target/_/-}
libc6-dev-${dpkg_arch}-cross
pkg-config-${gcc_target}
qemu-user
)
fi
@ -85,12 +84,12 @@ time apt-get update
time apt-get install --assume-yes --no-install-recommends "${packages[@]}"
# Set environment variables for cross-compiling.
# Also set up a symlink that points to the release binary, because the
# release binary's location varies when cross-compiling, as described here:
# Also set up a symlink that points to the output platform's target dir, because
# the target directory layout varies when cross-compiling, as described here:
# https://doc.rust-lang.org/cargo/guide/build-cache.html
if [[ -n "${rust_target}" ]]; then
su moonfire-nvr -lc "rustup target install ${rust_target} &&
ln -s target/${rust_target}/release-lto/moonfire-nvr ."
ln -s target/${rust_target} platform-target"
underscore_rust_target="${rust_target//-/_}"
uppercase_underscore_rust_target="${underscore_rust_target^^}"
cat >> /var/lib/moonfire-nvr/.buildrc <<EOF
@ -99,6 +98,10 @@ if [[ -n "${rust_target}" ]]; then
export CARGO_BUILD_TARGET=${rust_target}
export CARGO_TARGET_${uppercase_underscore_rust_target}_LINKER=${gcc_target}-gcc
# Use a pkg-config wrapper for the target architecture. This wrapper was
# automatically created on "dpkg --add-architecture"; see
# /etc/dpkg/dpkg.cfg.d/pkgconf-hook-config.
#
# https://github.com/rust-lang/pkg-config-rs uses the "PKG_CONFIG"
# variable to to select the pkg-config binary to use. As of pkg-config 0.3.19,
# it unfortunately doesn't understand the <target>_ prefix that the README.md
@ -112,8 +115,8 @@ export CC_${underscore_rust_target}=${gcc_target}-gcc
export CXX_${underscore_rust_target}=${gcc_target}-g++
EOF
else
su moonfire-nvr -lc "ln -s target/release-lto/moonfire-nvr ."
su moonfire-nvr -lc "ln -s target platform-target"
fi
ls -laFR /var/cache/apt > /docker-build-debug/dev/var-cache-apt-after
find /var/cache/apt -ls > /docker-build-debug/dev/var-cache-apt-after
date

View File

@ -114,8 +114,11 @@ caveats:
On Linux hosts (as opposed to when using Docker Desktop on macOS/Windows),
you'll likely see errors like the ones below. The solution is to [install
emulators](https://github.com/tonistiigi/binfmt#installing-emulators).
You may need to reinstall emulators on each boot of the host.
```
Exec format error
Error while loading /usr/sbin/dpkg-split: No such file or directory
Error while loading /usr/sbin/dpkg-deb: No such file or directory
```
@ -127,7 +130,7 @@ Moonfire NVR's `Dockerfile` has some built-in debugging tools:
info includes:
* output (stdout + stderr) from the build script, running long operations
through the `time` command.
* `ls -laFR` of cache mounts before and after.
* `find -ls` output on cache mounts before and after.
* Each stage accepts a `INVALIDATE_CACHE_<stage>` argument. You can use eg
`--build-arg=INVALIDATE_CACHE_BUILD_SERVER=$(date +%s)` to force the
`build-server` stage to be rebuilt rather than use cached Docker layers.
@ -191,8 +194,9 @@ $ sudo apt-get install \
tzdata
```
Ubuntu 20.04 (the latest LTS as of this writing) bundles node 10, which has
reached end-of-life (see [node.js: releases](https://nodejs.org/en/about/releases/)).
Ubuntu 20.04 LTS (still popular, supported by Ubuntu until April 2025) bundles
node 10, which has reached end-of-life (see
[node.js: releases](https://nodejs.org/en/about/releases/)).
So rather than install the `nodejs` and `npm` packages from the built-in
repository, see [Installing Node.js via package
manager](https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions).