MyFavMirrors
/
moonfire-nvr
mirror of https://github.com/scottlamb/moonfire-nvr.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
moonfire-nvr/server/Cargo.toml

125 lines
4.1 KiB
TOML
Raw Normal View History

Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
[package]
name = "moonfire-nvr"
stop using cargo pkg version One fewer thing to manage at release time.
2023-10-17 12:36:32 -04:00
version = "0.0.0"
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
authors = ["Scott Lamb <slamb@slamb.org>"]
switch from ancient clap/structopt release to bpaf Improves #70: this reduces binary size from 12.3 MiB to 11.9 MiB (3%) on macOS/arm64. The user experience is almost the same. (The help output's `Usage:` lines lack the e.g. `moonfire-nvr run` prefix of argv[0] and subcommand, which isn't ideal, but I guess it's pretty minor in the grand scheme of things.)
2023-02-11 14:38:12 -05:00
edition = "2021"
use newer cargo version resolver This trims ~700KiB off moonfire-nvr's text section, much of it by eliminating h2. See #70.
2021-10-27 16:18:52 -04:00
resolver = "2"
shorten per-file copyright headers I'm tired of all the boilerplate, so use the new GPL-3.0-linking-exception license identifier instead in all the server components. I left the ui stuff alone because I'm just going to replace it (#111). Add a checker for the header because it's easy to forget.
2021-02-17 16:28:48 -05:00
license-file = "../LICENSE.txt"
upgrade to retina 0.4.9 This is a step toward H.265 recording support (#33) by using Retina for the `VisualSampleEncoding` instead of the prior H.264-specific logic. The only further change to Moonfire itself needed for H.265 is to add `| "h265"` to the check for the right video stream. This also adds support for recording MJPEG, although browser playback almost certainly won't work.
2024-08-14 23:43:33 -04:00
rust-version = "1.79"
prep 0.7.6 release
2023-07-08 13:05:32 -04:00
publish = false
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
compile with stable Rust The benchmarks now require "cargo bench --features=nightly". The extra #[cfg(nightly)] switches in the code needed for it are a bit annoying; I may move the benches to a separate directory to avoid this. But for now, this works.
2016-12-10 01:04:35 -05:00
[features]
upgrade to async hyper serve_generated_bytes is >3X faster. One caveat is that the reactor thread may stall when reading from the memory-mapped slice. Moonfire NVR is basically a single-user program, so that may not be so bad, but we'll see.
2017-03-02 22:29:28 -05:00
# The nightly feature is used within moonfire-nvr itself to gate the
# benchmarks. Also pass it along to crates that can benefit from it.
upgrade various Rust dependencies This stops using parking_lot entirely. Since Rust 1.62, the std implementations on Linux are direct futexes, not the boxed pthread mutexes they used to be. No real reason to use parking_lot anymore, so shed this dependency.
2022-09-29 01:19:35 -04:00
nightly = ["db/nightly"]
compile with stable Rust The benchmarks now require "cargo bench --features=nightly". The extra #[cfg(nightly)] switches in the code needed for it are a bit annoying; I may move the benches to a separate directory to avoid this. But for now, this works.
2016-12-10 01:04:35 -05:00
bundle UI files into the binary This is optional but now enabled for release builds. Why? * It shrinks the release docker images a bit, as the binary includes only the gzipped version of files and uncompressed into RAM at startup (which should be fast). * It's a step toward #160.
2023-08-04 13:52:05 -04:00
# The bundled feature aims to make a single executable file that is deployable,
# including statically linked libraries and embedded UI files.
bundled = ["rusqlite/bundled", "bundled-ui"]
bundled-ui = []
upgrade rusqlite and support bundled library SQLite3 has gotten some noticeable speed improvements in recent releases. This is a modest improvement on reasonably new platforms, and a pretty significant one on Raspbian Jessie (which has an incredibly old SQLite3).
2017-03-04 01:42:13 -05:00
initial split of database to a separate crate It should reduce compile time / memory usage to put quite a bit of the code into a separate crate. I also intend to limit visibility of some things to only within the db crate, but that's for a future change. This is the smallest move that will compile.
2018-02-21 02:15:39 -05:00
[workspace]
use extracted ffmpeg library
2020-03-28 03:59:25 -04:00
members = ["base", "db"]
initial split of database to a separate crate It should reduce compile time / memory usage to put quite a bit of the code into a separate crate. I also intend to limit visibility of some things to only within the db crate, but that's for a future change. This is the smallest move that will compile.
2018-02-21 02:15:39 -05:00
massive error overhaul * fully stop using ancient `failure` crate in favor of own error type * set an `ErrorKind` on everything
2023-07-10 01:04:17 -04:00
[workspace.dependencies]
upgrade Rust deps including Retina
2023-12-29 18:17:10 -05:00
base64 = "0.21.0"
h264-reader = "0.7.0"
update some Rust dependencies I didn't go to quite the latest version of everything, in an effort to minimize duplicates in the cargo tree.
2024-01-06 13:43:20 -05:00
itertools = "0.12.0"
nix = "0.27.0"
tolerate bad sps/pps, continued
2024-05-30 21:17:04 -04:00
pretty-hex = "0.4.0"
update some Rust dependencies I didn't go to quite the latest version of everything, in an effort to minimize duplicates in the cargo tree.
2024-01-06 13:43:20 -05:00
ring = "0.17.0"
update some deps
2024-08-24 00:34:04 -04:00
rusqlite = "0.32.1"
tracing = { version = "0.1" }
tracing-core = "0.1.30"
tracing-futures = { version = "0.2.5", features = ["futures-03", "std-future"] }
tolerate bad sps/pps, continued
2024-05-30 21:17:04 -04:00
tracing-log = "0.2"
update some deps
2024-08-24 00:34:04 -04:00
tracing-subscriber = { version = "0.3.16" }
massive error overhaul * fully stop using ancient `failure` crate in favor of own error type * set an `ErrorKind` on everything
2023-07-10 01:04:17 -04:00
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
[dependencies]
lose "extern crate" everywhere (Rust 2018 edition)
2018-12-28 22:53:29 -05:00
base = { package = "moonfire-base", path = "base" }
upgrade Rust deps including Retina
2023-12-29 18:17:10 -05:00
base64 = { workspace = true }
bump versions of blake3, nom, tokio-tungstenite
2021-08-31 16:04:26 -04:00
blake3 = "1.0.0"
use bpaf 0.9.1
2023-07-08 13:01:45 -04:00
bpaf = { version = "0.9.1", features = ["autocomplete", "bright-color", "derive"]}
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 14:47:52 -05:00
bytes = "1"
update several dependencies I left serde alone because uuid hasn't been updated for the new version.
2017-01-27 23:58:04 -05:00
byteorder = "1.0"
no more Docker! * use `termion` rather than `ncurses` to limit runtime deps * cross-compile with `cross` instead of our own dockerfiles/scripts * update instructions * update release procedure and GitHub actions to match * prep changelog for `v0.7.8` Fixes #160 Closes #265
2023-10-14 18:52:06 -04:00
cursive = { version = "0.20.0", default-features = false, features = ["termion-backend"] }
lose "extern crate" everywhere (Rust 2018 edition)
2018-12-28 22:53:29 -05:00
db = { package = "moonfire-db", path = "db" }
upgrade to hyper 0.13 ecosystem This doesn't take much advantage of async fns so far. For example, the with_{form,json}_body functions are still designed to be used with future combinators when it'd be more natural to call them from async fns now. But it's a start. Similarly, this still uses the old version of reqwest. Small steps. Requires Rust 1.40 now. (1.39 is a requirement of async, and 1.40 is a requirement of http-serve 0.2.0.)
2020-01-09 02:04:36 -05:00
futures = "0.3"
upgrade Rust deps including Retina
2023-12-29 18:17:10 -05:00
h264-reader = { workspace = true }
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 14:47:52 -05:00
http = "0.2.3"
read sample files from dedicated threads Reading from the mmap()ed region in the tokio threads could cause them to stall: * That could affect UI serving when there were concurrent UI requests (i.e., not just requests that needed the reads in question anyway). * If there's a faulty disk, it could cause the UI to totally hang. Better to not mix disks between threads. * Soon, I want to handle RTSP from the tokio threads (#37). Similarly, we don't want RTSP streaming to block on operations from unrelated disks. I went with just one thread per disk which I think is sufficient. But it'd be possible to do a fixed-size pool instead which might improve latency when some pages are already cached. I also dropped the memmap dependency. I had to compute the page alignment anyway to get mremap to work, and Moonfire NVR already is Unix-specific, so there wasn't much value from the memmap or memmap2 crates. Fixes #88
2021-06-04 22:17:32 -04:00
http-serve = { version = "0.3.1", features = ["dir"] }
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 14:47:52 -05:00
hyper = { version = "0.14.2", features = ["http1", "server", "stream", "tcp"] }
update some Rust dependencies I didn't go to quite the latest version of everything, in an effort to minimize duplicates in the cargo tree.
2024-01-06 13:43:20 -05:00
itertools = { workspace = true }
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
libc = "0.2"
logging improvements * allow debug/trace logging on release builds again * enable log messages from hyper. I didn't notice they went away with 0.14.0, although there's a breaking change in the log: https://github.com/hyperium/hyper/blob/master/CHANGELOG.md#v0140-2020-12-23 * downgrade some particularly spammy messages
2021-06-05 02:28:48 -04:00
log = { version = "0.4" }
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-26 00:31:50 -05:00
memchr = "2.0.2"
update some Rust dependencies I didn't go to quite the latest version of everything, in an effort to minimize duplicates in the cargo tree.
2024-01-06 13:43:20 -05:00
nix = { workspace = true, features = ["time", "user"] }
bump versions of blake3, nom, tokio-tungstenite
2021-08-31 16:04:26 -04:00
nom = "7.0.0"
update some Rust dependencies I didn't go to quite the latest version of everything, in an effort to minimize duplicates in the cargo tree.
2024-01-06 13:43:20 -05:00
password-hash = "0.5.0"
tolerate bad sps/pps, continued
2024-05-30 21:17:04 -04:00
pretty-hex = { workspace = true }
update a few deps
2022-05-02 14:22:14 -04:00
protobuf = "3.0"
upgrade reffers dependency
2022-03-08 14:41:00 -05:00
reffers = "0.7.0"
upgrade to retina 0.4.9 This is a step toward H.265 recording support (#33) by using Retina for the `VisualSampleEncoding` instead of the prior H.264-specific logic. The only further change to Moonfire itself needed for H.265 is to add `| "h265"` to the check for the right video stream. This also adds support for recording MJPEG, although browser playback almost certainly won't work.
2024-08-14 23:43:33 -04:00
retina = "0.4.9"
update some Rust dependencies I didn't go to quite the latest version of everything, in an effort to minimize duplicates in the cargo tree.
2024-01-06 13:43:20 -05:00
ring = { workspace = true }
massive error overhaul * fully stop using ancient `failure` crate in favor of own error type * set an `ErrorKind` on everything
2023-07-10 01:04:17 -04:00
rusqlite = { workspace = true }
lose "extern crate" everywhere (Rust 2018 edition)
2018-12-28 22:53:29 -05:00
serde = { version = "1.0", features = ["derive"] }
update dependencies * The mylog update fixes a couple bad bugs. * Otherwise, just keep up with the Rust ecosystem.
2017-06-11 15:57:55 -04:00
serde_json = "1.0"
always use smallvec unions They only require Rust 1.51, and our minimum is 1.52 now.
2021-10-27 11:05:07 -04:00
smallvec = { version = "1.7", features = ["union"] }
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 14:47:52 -05:00
sync_wrapper = "0.1.0"
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
time = "0.1"
Bump tokio from 1.24.0 to 1.24.2 in /server Bumps [tokio](https://github.com/tokio-rs/tokio) from 1.24.0 to 1.24.2. - [Release notes](https://github.com/tokio-rs/tokio/releases) - [Commits](https://github.com/tokio-rs/tokio/commits) --- updated-dependencies: - dependency-name: tokio dependency-type: direct:production ... Signed-off-by: dependabot[bot] <support@github.com>
2023-02-03 19:56:27 -05:00
tokio = { version = "1.24", features = ["macros", "rt-multi-thread", "signal", "sync", "time"] }
upgrade Rust deps including Retina
2023-12-29 18:17:10 -05:00
tokio-tungstenite = "0.20.0"
update indexmap
2024-01-06 14:54:33 -05:00
toml = "0.8"
update some deps
2024-08-24 00:34:04 -04:00
tracing = { workspace = true, features = ["log"] }
switch from `log` to `tracing` I think this is a big improvement in readability. I removed the `lnav` config, which is a little sad, but I don't think it supports this structured logging format well. Still seems worthwhile on balance.
2023-02-16 02:14:54 -05:00
tracing-subscriber = { version = "0.3.16", features = ["env-filter", "json"] }
tracing-core = "0.1.30"
upgrade to Rust 1.70, use `std::sync::OnceLock` The most notable part of this is that `db::auth` no longer holds a lock during password hashing operations. That was probably never a great idea...
2023-07-04 16:45:33 -04:00
tracing-futures = { version = "0.2.5", features = ["futures-03", "std-future"] }
update some Rust dependencies I didn't go to quite the latest version of everything, in an effort to minimize duplicates in the cargo tree.
2024-01-06 13:43:20 -05:00
tracing-log = { workspace = true }
switch from `log` to `tracing` I think this is a big improvement in readability. I removed the `lnav` config, which is a little sad, but I don't think it supports this structured logging format well. Still seems worthwhile on balance.
2023-02-16 02:14:54 -05:00
ulid = "1.0.0"
upgrade to Rust 1.70, use `std::sync::OnceLock` The most notable part of this is that `db::auth` no longer holds a lock during password hashing operations. That was probably never a great idea...
2023-07-04 16:45:33 -04:00
url = "2.1.1"
uuid = { version = "1.1.2", features = ["serde", "std", "v4"] }
bundle UI files into the binary This is optional but now enabled for release builds. Why? * It shrinks the release docker images a bit, as the binary includes only the gzipped version of files and uncompressed into RAM at startup (which should be fast). * It's a step toward #160.
2023-08-04 13:52:05 -04:00
flate2 = "1.0.26"
stop using cargo pkg version One fewer thing to manage at release time.
2023-10-17 12:36:32 -04:00
git-version = "0.3.5"
bundle UI files into the binary This is optional but now enabled for release builds. Why? * It shrinks the release docker images a bit, as the binary includes only the gzipped version of files and uncompressed into RAM at startup (which should be fast). * It's a step toward #160.
2023-08-04 13:52:05 -04:00
systemd Ready/Stopping notification
2023-10-20 00:34:28 -04:00
[target.'cfg(target_os = "linux")'.dependencies]
update some Rust dependencies I didn't go to quite the latest version of everything, in an effort to minimize duplicates in the cargo tree.
2024-01-06 13:43:20 -05:00
libsystemd = "0.7.0"
systemd Ready/Stopping notification
2023-10-20 00:34:28 -04:00
bundle UI files into the binary This is optional but now enabled for release builds. Why? * It shrinks the release docker images a bit, as the binary includes only the gzipped version of files and uncompressed into RAM at startup (which should be fast). * It's a step toward #160.
2023-08-04 13:52:05 -04:00
[build-dependencies]
update some Rust dependencies I didn't go to quite the latest version of everything, in an effort to minimize duplicates in the cargo tree.
2024-01-06 13:43:20 -05:00
ahash = "0.8"
bundle UI files into the binary This is optional but now enabled for release builds. Why? * It shrinks the release docker images a bit, as the binary includes only the gzipped version of files and uncompressed into RAM at startup (which should be fast). * It's a step toward #160.
2023-08-04 13:52:05 -04:00
blake3 = "1.0.0"
walkdir = "2.3.3"
fully implement json handling as in spec This is a significant milestone; now the Rust branch matches the C++ branch's features. In the process, I switched from using serde_derive (which requires nightly Rust) to serde_codegen (which does not). It was easier than I thought it'd be. I'm getting close to no longer requiring nightly Rust.
2016-12-09 00:28:50 -05:00
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
[dev-dependencies]
drop ffmpeg support * switch the config interface over to use Retina and make the test button honor rtsp_transport = udp. * adjust the threading model of the Retina streaming code. Before, it spawned a background future that read from the runtime and wrote to a channel. Other calls read from this channel. After, it does work directly from within the block_on calls (no channels). The immediate motivation was that the config interface didn't have another runtime handy. And passing in a current thread runtime deadlocked. I later learned this is a difference between Runtime::block_on and Handle::block_on. The former will drive IO and timers; the latter will not. But this is also more efficient to avoid so many thread hand-offs. Both the context switches and the extra spinning that tokio appears to do as mentioned here: https://github.com/scottlamb/retina/issues/5#issuecomment-871971550 This may not be the final word on the threading model. Eventually I may not have per-stream writing threads at all. But I think it will be easier to look at this after getting rid of the separate `moonfire-nvr config` subcommand in favor of a web interface. * in tests, read `.mp4` files via the `mp4` crate rather than ffmpeg. The annoying part is that this doesn't parse edit lists; oh well. * simplify the `Opener` interface. Formerly, it'd take either a RTSP URL or a path to a `.mp4` file, and they'd share some code because they both sometimes used ffmpeg. Now, they're totally different libraries (`retina` vs `mp4`). Pull the latter out to a `testutil` module with a different interface that exposes more of the `mp4` stuff. Now `Opener` is just for RTSP. * simplify the h264 module. It had a lot of logic to deal with Annex B. Retina doesn't use this encoding. Fixes #36 Fixes #126
2022-03-18 13:30:23 -04:00
mp4 = { git = "https://github.com/scottlamb/mp4-rust", branch = "moonfire" }
upgrade some server deps I avoided rtcp 0.2.2->0.2.3 because of an accidental semver break.
2021-07-09 18:01:15 -04:00
num-rational = { version = "0.4.0", default-features = false, features = ["std"] }
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 14:47:52 -05:00
reqwest = { version = "0.11.0", default-features = false, features = ["json"] }
stop using old tempdir crate
2021-05-17 16:08:01 -04:00
tempfile = "3.2.0"
switch from `log` to `tracing` I think this is a big improvement in readability. I removed the `lnav` config, which is a little sad, but I don't think it supports this structured logging format well. Still seems worthwhile on balance.
2023-02-16 02:14:54 -05:00
tracing-test = "0.2.4"
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
always optimize scrypt password hashes
2020-11-24 10:53:52 -05:00
[profile.dev.package.scrypt]
# On an Intel i3-6100U @ 2.30 GHz, a single scrypt password hash takes 7.6
# seconds at opt-level=0, or 0.096 seconds at opt-level=2. Always optimize this
# crate to avoid seeming hung / being annoyingly slow when debugging.
opt-level = 2
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
[profile.release]
reduce debug output in release/bench builds This reduces the binary size from 154 MiB to 70 MiB (#70 progress). Tools like `cargo flamegraph` still work fine. As suggested by "EarthFeet" on reddit: https://www.reddit.com/r/rust/comments/rw0jad/cargos_strip_profile_option_has_been_stabilized/hra193k/
2022-01-05 12:55:30 -05:00
debug = 1
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
build docker image with lto
2022-04-14 13:53:03 -04:00
[profile.release-lto]
inherits = "release"
lto = true
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
[profile.bench]
reduce debug output in release/bench builds This reduces the binary size from 154 MiB to 70 MiB (#70 progress). Tools like `cargo flamegraph` still work fine. As suggested by "EarthFeet" on reddit: https://www.reddit.com/r/rust/comments/rw0jad/cargos_strip_profile_option_has_been_stabilized/hra193k/
2022-01-05 12:55:30 -05:00
debug = 1
upgrade Rust deps
2023-01-05 18:35:26 -05:00
[patch.crates-io]
update indexmap
2024-01-06 14:54:33 -05:00
update some deps
2024-08-24 00:34:04 -04:00
# Override the `tracing` crate versions with a branch that updates the
# `matchers` dependency to avoid duplicate `regex-automata` crate versions.
# This branch is based on tracing's `0.1.x` branch with changes similar to
# <https://github.com/tokio-rs/tracing/pull/3033> applied.
tracing = { git = "https://github.com/scottlamb/tracing", rev = "861b443d7b2da400ca7b09111957f33c80135908" }
tracing-core = { git = "https://github.com/scottlamb/tracing", rev = "861b443d7b2da400ca7b09111957f33c80135908" }
tracing-log = { git = "https://github.com/scottlamb/tracing", rev = "861b443d7b2da400ca7b09111957f33c80135908" }
tracing-subscriber = { git = "https://github.com/scottlamb/tracing", rev = "861b443d7b2da400ca7b09111957f33c80135908" }