Optimizations and build speedup (#6339)

* Optimizations and build speedup

With this commit I have changed several components to be more efficient.
This can be less llvm-lines generated or less `clone()` calls.

 ### Config
- Re-ordered the `make_config` macro to be more efficient
- Created a custom Deserializer for `ConfigBuilder` less code and more efficient
- Use struct's for the `prepare_json` function instead of generating a custom JSON object.
  This generates less code and is more efficient.
- Updated the `get_support_string` function to handle the masking differently.
  This generates less code and also was able to remove some sub-macro-calls

 ### Error
- Added an extra new call to prevent duplicate Strings in generated macro code.
  This generated less llvm-lines and seems to be more efficient.
- Created a custom Serializer for `ApiError` and `CompactApiError`
  This makes that struct smaller in size, so better for memory, but also less llvm-lines.

 ### General
- Removed `once_lock` and replace it all with Rust's std LazyLock
- Added and fixed some Clippy lints which reduced `clone()` calls for example.
- Updated build profiles for more efficiency
  Also added a new profile specifically for CI, which should decrease the build check
- Updated several GitHub Workflows for better security and use the new `ci` build profile
- Updated to Rust v1.90.0 which uses a new linker `rust-lld` which should help in faster building
- Updated the Cargo.toml for all crates to better use the `workspace` variables
- Added a `typos` Workflow and Pre-Commit, which should help in detecting spell error's.
  Also fixed a few found by it.

Signed-off-by: BlackDex <black.dex@gmail.com>

* Fix release profile

Signed-off-by: BlackDex <black.dex@gmail.com>

* Update typos and remove mimalloc check from pre-commit checks

Signed-off-by: BlackDex <black.dex@gmail.com>

* Misc fixes and updated typos

Signed-off-by: BlackDex <black.dex@gmail.com>

* Update crates and workflows

Signed-off-by: BlackDex <black.dex@gmail.com>

* Fix formating and pre-commit

Signed-off-by: BlackDex <black.dex@gmail.com>

* Update to Rust v1.91 and update crates

Signed-off-by: BlackDex <black.dex@gmail.com>

* Update web-vault to v2025.10.1 and xx to v1.8.0

Signed-off-by: BlackDex <black.dex@gmail.com>

---------

Signed-off-by: BlackDex <black.dex@gmail.com>
This commit is contained in:
Mathijs van Veluw
2025-11-01 22:21:04 +01:00
committed by GitHub
parent 8d30285160
commit 9017ca265a
47 changed files with 762 additions and 488 deletions

View File

@@ -14,6 +14,7 @@ on:
- "diesel.toml"
- "docker/Dockerfile.j2"
- "docker/DockerSettings.yaml"
- "macros/**"
pull_request:
paths:
@@ -27,13 +28,11 @@ on:
- "diesel.toml"
- "docker/Dockerfile.j2"
- "docker/DockerSettings.yaml"
- "macros/**"
jobs:
build:
name: Build and Test ${{ matrix.channel }}
permissions:
actions: write
contents: read
runs-on: ubuntu-24.04
timeout-minutes: 120
# Make warnings errors, this is to prevent warnings slipping through.
@@ -81,7 +80,7 @@ jobs:
# Only install the clippy and rustfmt components on the default rust-toolchain
- name: "Install rust-toolchain version"
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master @ Aug 23, 2025, 3:20 AM GMT+2
uses: dtolnay/rust-toolchain@6d653acede28d24f02e3cd41383119e8b1b35921 # master @ Sep 16, 2025, 8:37 PM GMT+2
if: ${{ matrix.channel == 'rust-toolchain' }}
with:
toolchain: "${{steps.toolchain.outputs.RUST_TOOLCHAIN}}"
@@ -91,7 +90,7 @@ jobs:
# Install the any other channel to be used for which we do not execute clippy and rustfmt
- name: "Install MSRV version"
uses: dtolnay/rust-toolchain@e97e2d8cc328f1b50210efc529dca0028893a2d9 # master @ Aug 23, 2025, 3:20 AM GMT+2
uses: dtolnay/rust-toolchain@6d653acede28d24f02e3cd41383119e8b1b35921 # master @ Sep 16, 2025, 8:37 PM GMT+2
if: ${{ matrix.channel != 'rust-toolchain' }}
with:
toolchain: "${{steps.toolchain.outputs.RUST_TOOLCHAIN}}"
@@ -121,43 +120,55 @@ jobs:
# Use a custom prefix-key to force a fresh start. This is sometimes needed with bigger changes.
# Like changing the build host from Ubuntu 20.04 to 22.04 for example.
# Only update when really needed! Use a <year>.<month>[.<inc>] format.
prefix-key: "v2023.07-rust"
prefix-key: "v2025.09-rust"
# End Enable Rust Caching
# Run cargo tests
# First test all features together, afterwards test them separately.
- name: "test features: sqlite,mysql,postgresql,enable_mimalloc,s3"
id: test_sqlite_mysql_postgresql_mimalloc_s3
if: ${{ !cancelled() }}
run: |
cargo test --profile ci --features sqlite,mysql,postgresql,enable_mimalloc,s3
- name: "test features: sqlite,mysql,postgresql,enable_mimalloc"
id: test_sqlite_mysql_postgresql_mimalloc
if: ${{ !cancelled() }}
run: |
cargo test --profile ci --features sqlite,mysql,postgresql,enable_mimalloc
- name: "test features: sqlite,mysql,postgresql"
id: test_sqlite_mysql_postgresql
if: ${{ !cancelled() }}
run: |
cargo test --features sqlite,mysql,postgresql
cargo test --profile ci --features sqlite,mysql,postgresql
- name: "test features: sqlite"
id: test_sqlite
if: ${{ !cancelled() }}
run: |
cargo test --features sqlite
cargo test --profile ci --features sqlite
- name: "test features: mysql"
id: test_mysql
if: ${{ !cancelled() }}
run: |
cargo test --features mysql
cargo test --profile ci --features mysql
- name: "test features: postgresql"
id: test_postgresql
if: ${{ !cancelled() }}
run: |
cargo test --features postgresql
cargo test --profile ci --features postgresql
# End Run cargo tests
# Run cargo clippy, and fail on warnings
- name: "clippy features: sqlite,mysql,postgresql,enable_mimalloc"
- name: "clippy features: sqlite,mysql,postgresql,enable_mimalloc,s3"
id: clippy
if: ${{ !cancelled() && matrix.channel == 'rust-toolchain' }}
run: |
cargo clippy --features sqlite,mysql,postgresql,enable_mimalloc
cargo clippy --profile ci --features sqlite,mysql,postgresql,enable_mimalloc,s3
# End Run cargo clippy
@@ -175,6 +186,8 @@ jobs:
- name: "Some checks failed"
if: ${{ failure() }}
env:
TEST_DB_M_S3: ${{ steps.test_sqlite_mysql_postgresql_mimalloc_s3.outcome }}
TEST_DB_M: ${{ steps.test_sqlite_mysql_postgresql_mimalloc.outcome }}
TEST_DB: ${{ steps.test_sqlite_mysql_postgresql.outcome }}
TEST_SQLITE: ${{ steps.test_sqlite.outcome }}
TEST_MYSQL: ${{ steps.test_mysql.outcome }}
@@ -186,11 +199,13 @@ jobs:
echo "" >> "${GITHUB_STEP_SUMMARY}"
echo "|Job|Status|" >> "${GITHUB_STEP_SUMMARY}"
echo "|---|------|" >> "${GITHUB_STEP_SUMMARY}"
echo "|test (sqlite,mysql,postgresql,enable_mimalloc,s3)|${TEST_DB_M_S3}|" >> "${GITHUB_STEP_SUMMARY}"
echo "|test (sqlite,mysql,postgresql,enable_mimalloc)|${TEST_DB_M}|" >> "${GITHUB_STEP_SUMMARY}"
echo "|test (sqlite,mysql,postgresql)|${TEST_DB}|" >> "${GITHUB_STEP_SUMMARY}"
echo "|test (sqlite)|${TEST_SQLITE}|" >> "${GITHUB_STEP_SUMMARY}"
echo "|test (mysql)|${TEST_MYSQL}|" >> "${GITHUB_STEP_SUMMARY}"
echo "|test (postgresql)|${TEST_POSTGRESQL}|" >> "${GITHUB_STEP_SUMMARY}"
echo "|clippy (sqlite,mysql,postgresql,enable_mimalloc)|${CLIPPY}|" >> "${GITHUB_STEP_SUMMARY}"
echo "|clippy (sqlite,mysql,postgresql,enable_mimalloc,s3)|${CLIPPY}|" >> "${GITHUB_STEP_SUMMARY}"
echo "|fmt|${FMT}|" >> "${GITHUB_STEP_SUMMARY}"
echo "" >> "${GITHUB_STEP_SUMMARY}"
echo "Please check the failed jobs and fix where needed." >> "${GITHUB_STEP_SUMMARY}"