MyFavMirrors
/
moonfire-nvr
mirror of https://github.com/scottlamb/moonfire-nvr.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
moonfire-nvr/server/Cargo.toml

81 lines
2.5 KiB
TOML
Raw Normal View History

Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
[package]
name = "moonfire-nvr"
Prepare v0.6.3 release ...including changelog and new screenshots in the README.
2021-03-31 18:21:09 -04:00
version = "0.6.3"
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
authors = ["Scott Lamb <slamb@slamb.org>"]
lose "extern crate" everywhere (Rust 2018 edition)
2018-12-28 22:53:29 -05:00
edition = "2018"
shorten per-file copyright headers I'm tired of all the boilerplate, so use the new GPL-3.0-linking-exception license identifier instead in all the server components. I left the ui stuff alone because I'm just going to replace it (#111). Add a checker for the header because it's easy to forget.
2021-02-17 16:28:48 -05:00
license-file = "../LICENSE.txt"
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
compile with stable Rust The benchmarks now require "cargo bench --features=nightly". The extra #[cfg(nightly)] switches in the code needed for it are a bit annoying; I may move the benches to a separate directory to avoid this. But for now, this works.
2016-12-10 01:04:35 -05:00
[features]
upgrade to async hyper serve_generated_bytes is >3X faster. One caveat is that the reactor thread may stall when reading from the memory-mapped slice. Moonfire NVR is basically a single-user program, so that may not be so bad, but we'll see.
2017-03-02 22:29:28 -05:00
# The nightly feature is used within moonfire-nvr itself to gate the
# benchmarks. Also pass it along to crates that can benefit from it.
fix #64 (extraneous flushes) Now each syncer has a binary heap of the times it plans to do a flush. When one of those times arrives, it rechecks if there's something to do. Seems more straightforward than rechecking each stream's first uncommitted recording, especially with the logic to retry failed flushes every minute. Also improved the info! log for each flush to see the actual recordings being flushed for better debuggability. No new tests right now. :-( They're tricky to write. One problem is that it's hard to get the timing right: a different flush has to happen after Syncer::save's database operations and before Syncer::run calls SimulatedClocks::recv_timeout with an empty channel[*], advancing the time. I've thought of a few ways of doing this: * adding a new SyncerCommand to run something, but it's messy (have to add it from the mock of one of the actions done by the save), and Box<dyn FnOnce() + 'static> not working (see rust-lang/rust#28796) makes it especially annoying. * replacing SimulatedClocks with something more like MockClocks. Lots of boilerplate. Maybe I need to find a good general-purpose Rust mock library. (mockers sounds good but I want something that works on stable Rust.) * bypassing the Syncer::run loop, instead manually running iterations from the test. Maybe the last way is the best for now. I'm likely to try it soon. [*] actually, it's calling Receiver::recv_timeout directly; Clocks::recv_timeout is dead code now? oops.
2019-01-04 14:56:15 -05:00
nightly = ["db/nightly", "parking_lot/nightly", "smallvec/union"]
compile with stable Rust The benchmarks now require "cargo bench --features=nightly". The extra #[cfg(nightly)] switches in the code needed for it are a bit annoying; I may move the benches to a separate directory to avoid this. But for now, this works.
2016-12-10 01:04:35 -05:00
upgrade rusqlite and support bundled library SQLite3 has gotten some noticeable speed improvements in recent releases. This is a modest improvement on reasonably new platforms, and a pretty significant one on Raspbian Jessie (which has an incredibly old SQLite3).
2017-03-04 01:42:13 -05:00
# The bundled feature includes bundled (aka statically linked) versions of
# native libraries where possible.
bundled = ["rusqlite/bundled"]
first step toward object detection (#30) When compiled with cargo build --features=analytics and enabled via moonfire-nvr run --object-detection, this runs object detection on every sub stream frame through an Edge TPU (a Coral USB accelerator) and logs the result. This is a very small step toward a working system. It doesn't actually record the result in the database or send it out on the live stream yet. It doesn't support running object detection at a lower frame rate than the sub streams come in at either. To address those problems, I need to do some refactoring. Currently moonfire_db::writer::Writer::Write is the only place that knows the duration of the frame it's about to flush, before it gets added to the index or sent out on the live stream. I don't want to do the detection from there; I'd prefer the moonfire_nvr crate. So I either need to introduce an analytics callback or move a bunch of that logic to the other crate. Once I do that, I need to add database support (although I have some experiments for that in moonfire-playground) and API support, then some kind of useful frontend. Note edgetpu.tflite is taken from the Apache 2.0-licensed https://github.com/google-coral/edgetpu, test_data/mobilenet_ssd_v2_coco_quant_postprocess_edgetpu.tflite. The following page says it's fine to include Apache 2.0 stuff in GPLv3 projects: https://www.apache.org/licenses/GPL-compatibility.html
2020-04-14 02:03:49 -04:00
analytics = ["moonfire-tflite", "ffmpeg/swscale"]
initial split of database to a separate crate It should reduce compile time / memory usage to put quite a bit of the code into a separate crate. I also intend to limit visibility of some things to only within the db crate, but that's for a future change. This is the smallest move that will compile.
2018-02-21 02:15:39 -05:00
[workspace]
use extracted ffmpeg library
2020-03-28 03:59:25 -04:00
members = ["base", "db"]
initial split of database to a separate crate It should reduce compile time / memory usage to put quite a bit of the code into a separate crate. I also intend to limit visibility of some things to only within the db crate, but that's for a future change. This is the smallest move that will compile.
2018-02-21 02:15:39 -05:00
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
[dependencies]
lose "extern crate" everywhere (Rust 2018 edition)
2018-12-28 22:53:29 -05:00
base = { package = "moonfire-base", path = "base" }
update various deps This brings most things reasonably up-to-date. libpasta's deps are dragging a bit, keeping us on an older ring to avoid duplication, and causing us to use three versions of base64. And I need to update a few of my companion crates' parking_lot dep to match tokio.
2020-11-23 03:23:03 -05:00
base64 = "0.13.0"
blake3 = "0.3.7"
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 14:47:52 -05:00
bytes = "1"
update several dependencies I left serde alone because uuid hasn't been updated for the new version.
2017-01-27 23:58:04 -05:00
byteorder = "1.0"
update various deps This brings most things reasonably up-to-date. libpasta's deps are dragging a bit, keeping us on an older ring to avoid duplication, and causing us to use three versions of base64. And I need to update a few of my companion crates' parking_lot dep to match tokio.
2020-11-23 03:23:03 -05:00
cstr = "0.2.5"
upgrade some other deps
2021-01-27 15:11:36 -05:00
cursive = "0.16.3"
lose "extern crate" everywhere (Rust 2018 edition)
2018-12-28 22:53:29 -05:00
db = { package = "moonfire-db", path = "db" }
replace homegrown Error with failure crate This reduces boilerplate, making it a bit easier for me to split the db stuff out into its own crate.
2018-02-21 01:46:14 -05:00
failure = "0.1.1"
use extracted ffmpeg library
2020-03-28 03:59:25 -04:00
ffmpeg = { package = "moonfire-ffmpeg", git = "https://github.com/scottlamb/moonfire-ffmpeg" }
upgrade to hyper 0.13 ecosystem This doesn't take much advantage of async fns so far. For example, the with_{form,json}_body functions are still designed to be used with future combinators when it'd be more natural to call them from async fns now. But it's a start. Similarly, this still uses the old version of reqwest. Small steps. Requires Rust 1.40 now. (1.39 is a requirement of async, and 1.40 is a requirement of http-serve 0.2.0.)
2020-01-09 02:04:36 -05:00
futures = "0.3"
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
fnv = "1.0"
schema version 6 with pixel aspect ratio This makes anamorphic sub streams display correctly, even ones from old Hikvision cameras that don't properly set the aspect ratio at the H.264 layer.
2020-03-20 00:35:42 -04:00
h264-reader = { git = "https://github.com/dholroyd/h264-reader" }
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 14:47:52 -05:00
http = "0.2.3"
http-serve = { version = "0.3.0", features = ["dir"] }
hyper = { version = "0.14.2", features = ["http1", "server", "stream", "tcp"] }
upgrade deps
2018-01-31 01:05:39 -05:00
lazy_static = "1.0"
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
libc = "0.2"
upgrade deps
2018-01-31 01:05:39 -05:00
log = { version = "0.4", features = ["release_max_level_info"] }
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-26 00:31:50 -05:00
memchr = "2.0.2"
update most deps, notably including reqwest Fixes #60 The reqwest dependency is significant because the old version required an old version of openssl, complicating compilation on newer platforms. reqwest also pulled in old/duplicate versions of hyper, tokio, etc. Nice to drop a lot of that cruft. I left rusqlite and uuid alone because they had breaking changes I didn't want to mess with at the moment. Bumped the minimum Rust version to 1.30.0, as required by the new encoding_rs crate (and perhaps other things).
2018-11-20 12:32:55 -05:00
memmap = "0.7"
Merge branch 'master' into new-schema
2020-04-18 02:33:46 -04:00
moonfire-tflite = { git = "https://github.com/scottlamb/moonfire-tflite", features = ["edgetpu"], optional = true }
specify git path to mylog
2017-03-26 03:03:24 -04:00
mylog = { git = "https://github.com/scottlamb/mylog" }
update various deps This brings most things reasonably up-to-date. libpasta's deps are dragging a bit, keeping us on an older ring to avoid duplication, and causing us to use three versions of base64. And I need to update a few of my companion crates' parking_lot dep to match tokio.
2020-11-23 03:23:03 -05:00
nix = "0.19.0"
nom = "6.0.0"
upgrade some deps including parking_lot Now there's a single parking_lot version, yay.
2020-11-24 01:46:05 -05:00
parking_lot = { version = "0.11.1", features = [] }
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 18:17:50 -04:00
protobuf = { git = "https://github.com/stepancheg/rust-protobuf" }
upgrade some deps, including reqwest The reqwest one is particularly notable because it means not having two versions of hyper/http/tokio/futures/bytes. It also drops a number of transitive deps; with some work I think I could stop depending on regex now.
2020-01-09 23:06:30 -05:00
reffers = "0.6.0"
update libpasta/ring This uses my (hopefully short-lived) libpasta fork.
2020-11-24 00:47:02 -05:00
ring = "0.16.2"
update rusqlite The big difference here is query_named and execute_named have gone away. Fair number of lines changes but straightforward.
2021-05-17 13:50:12 -04:00
rusqlite = "0.25.3"
lose "extern crate" everywhere (Rust 2018 edition)
2018-12-28 22:53:29 -05:00
serde = { version = "1.0", features = ["derive"] }
update dependencies * The mylog update fixes a couple bad bugs. * Otherwise, just keep up with the Rust ecosystem.
2017-06-11 15:57:55 -04:00
serde_json = "1.0"
upgrade some deps, including reqwest The reqwest one is particularly notable because it means not having two versions of hyper/http/tokio/futures/bytes. It also drops a number of transitive deps; with some work I think I could stop depending on regex now.
2020-01-09 23:06:30 -05:00
smallvec = "1.0"
Merge branch 'master' into new-schema
2020-04-18 02:33:46 -04:00
structopt = { version = "0.3.13", features = ["default", "wrap_help"] }
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 14:47:52 -05:00
sync_wrapper = "0.1.0"
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
time = "0.1"
send keepalives on live.m4s Chrome appears to time out at 60 seconds of inactivity otherwise. I think it's better to keep the stream open, even if the camera is broken. The implementation looks awkward, but that might be the state of Rust async right now.
2021-03-26 02:01:38 -04:00
tokio = { version = "1.0", features = ["macros", "parking_lot", "rt-multi-thread", "signal", "time"] }
tokio-stream = "0.1.5"
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 14:47:52 -05:00
tokio-tungstenite = "0.13.0"
upgrade some deps, including reqwest The reqwest one is particularly notable because it means not having two versions of hyper/http/tokio/futures/bytes. It also drops a number of transitive deps; with some work I think I could stop depending on regex now.
2020-01-09 23:06:30 -05:00
url = "2.1.1"
uuid = { version = "0.8", features = ["serde", "std", "v4"] }
fully implement json handling as in spec This is a significant milestone; now the Rust branch matches the C++ branch's features. In the process, I switched from using serde_derive (which requires nightly Rust) to serde_codegen (which does not). It was easier than I thought it'd be. I'm getting close to no longer requiring nightly Rust.
2016-12-09 00:28:50 -05:00
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
[dev-dependencies]
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 14:47:52 -05:00
reqwest = { version = "0.11.0", default-features = false, features = ["json"] }
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
tempdir = "0.3"
always optimize scrypt password hashes
2020-11-24 10:53:52 -05:00
[profile.dev.package.scrypt]
# On an Intel i3-6100U @ 2.30 GHz, a single scrypt password hash takes 7.6
# seconds at opt-level=0, or 0.096 seconds at opt-level=2. Always optimize this
# crate to avoid seeming hung / being annoyingly slow when debugging.
opt-level = 2
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
[profile.release]
debug = true
[profile.bench]
debug = true