MyFavMirrors/moonfire-nvr
1
0
Fork 0
mirror of https://github.com/scottlamb/moonfire-nvr.git synced 2025-02-23 03:22:30 -05:00
Code Issues Packages Projects Releases Wiki Activity
moonfire-nvr/server/src/web/mod.rs

783 lines
28 KiB
Rust
Raw Normal View History

upgrade copyright notices * As discussed in #48, say "The Moonfire NVR Authors" at the top of every file rather than whoever created that file. Have one AUTHORS file listing everyone. * Consistently call it a "security camera network video recorder" rather than "security camera digital video recorder".
2020-03-01 22:53:41 -08:00
// This file is part of Moonfire NVR, a security camera network video recorder.
start splitting up web.rs It's getting huge and hard to work with. The path stuff is easy to pull out.
2021-10-28 12:38:29 -07:00
// Copyright (C) 2021 The Moonfire NVR Authors; see AUTHORS and LICENSE.txt.
shorten per-file copyright headers I'm tired of all the boilerplate, so use the new GPL-3.0-linking-exception license identifier instead in all the server components. I left the ui stuff alone because I'm just going to replace it (#111). Add a checker for the header because it's easy to forget.
2021-02-17 13:28:48 -08:00
// SPDX-License-Identifier: GPL-v3.0-or-later WITH GPL-3.0-linking-exception.
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
separate live view into its own file
2021-10-28 13:07:39 -07:00
mod live;
start splitting up web.rs It's getting huge and hard to work with. The path stuff is easy to pull out.
2021-10-28 12:38:29 -07:00
mod path;
extract /api/{login,logout} to their own file
2021-10-28 13:57:32 -07:00
mod session;
separate out signals API to own file
2021-10-28 14:09:30 -07:00
mod signals;
split static file serving into its own file
2021-10-28 12:49:50 -07:00
mod static_file;
move /view.{mp4,m4s} to their own file
2021-10-28 13:23:49 -07:00
mod view;
start splitting up web.rs It's getting huge and hard to work with. The path stuff is easy to pull out.
2021-10-28 12:38:29 -07:00
use self::path::Path;
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
use crate::body::Body;
lose "extern crate" everywhere (Rust 2018 edition)
2018-12-28 21:53:29 -06:00
use crate::json;
use crate::mp4;
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
use base::{bail_t, ErrorKind};
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
use base::{clock::Clocks, format_err_t};
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
use core::borrow::Borrow;
use core::str::FromStr;
lose "extern crate" everywhere (Rust 2018 edition)
2018-12-28 21:53:29 -06:00
use db::dir::SampleFileDir;
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
use db::{auth, recording};
use typed errors in /view.mp4 path This fixes #178. Before, everything got translated to 5xx status; now it produces the correct type in several cases. Ideally I'd get rid of the untyped errors in all of web.rs; this is a small step.
2021-10-21 10:25:37 -07:00
use failure::{format_err, Error};
support multiple sample file directories This is still pretty basic support. There's no config UI support for renaming/moving the sample file directories after they are created, and no error checking that the files are still in the expected place. I can imagine sysadmins getting into trouble trying to change things. I hope to address at least some of that in a follow-up change to introduce a versioning/locking scheme that ensures databases and sample file dirs match in some way. A bonus change that kinda got pulled along for the ride: a dialog pops up in the config UI while a stream is being tested. The experience was pretty bad before; there was no indication the button worked at all until it was done, sometimes many seconds later.
2018-02-11 22:45:51 -08:00
use fnv::FnvHashMap;
upgrade to hyper 0.12.x Just one (intentional) functional change---now the streamers start shutting down while the webserver shuts down gracefully.
2018-08-29 22:26:19 -07:00
use http::header::{self, HeaderValue};
UI preferences: #153 #155
2021-09-01 15:01:42 -07:00
use http::method::Method;
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
use http::{status::StatusCode, Request, Response};
overhaul HTTP serving and caching * use content-hashed paths for static resources (except the top-level request), with immutable Cache-Control headers. This should improve cache behavior in both directions: avoid preventable HTTP requests and cause immediate refresh when needed. I had some staleness when browsing with my phone. * set up the favicons properly while I'm at it (closes #50). I used the convenient favicons-webpack-plugin to build everything from a .svg. I've hit an error similar to lovell/sharp#1593 at least once though so I might change my mind about that part if it continues to be problematic. * use http-serve's new directory traversal code for static file serving. This removes the odd behavior where files that weren't present at server startup couldn't be served. (I wasn't comfortable switching to the content-hashed paths before doing this.) It also means the static files can be served compressed. JSON API responses were already served compressed, so this closes #25. * for a given API URL, decide if we want it to be cached or not server-side. Stop using jQuery's kludgy cache-defeating _=<timestamp> URL parameter. I might start setting etags on some of these things and could serve 304 Not Modified responses if it's genuinely unmodified.
2020-05-29 21:20:14 -07:00
use http_serve::dir::FsDir;
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
use hyper::body::Bytes;
extract /api/{login,logout} to their own file
2021-10-28 13:57:32 -07:00
use log::{debug, warn};
support proxy forwarded headers I went with legacy headers (X-Real-IP, X-Forwarded-Proto) because they appear to be more widely supported than the RFC 7239 Forwarded header.
2018-11-28 14:22:30 -08:00
use std::net::IpAddr;
style improvements * remove stuttering: mp4::Mp4Foo -> mp4::Foo * stop using a &MutexGuard<Foo> where a &Foo will do
2017-02-24 21:33:26 -08:00
use std::sync::Arc;
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
use url::form_urlencoded;
use uuid::Uuid;
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
/// An HTTP error response.
/// This is a thin wrapper over the hyper response type; it doesn't even verify
/// that the response actually uses a non-2xx status code. Its purpose is to
/// allow automatic conversion from `base::Error`. Rust's orphan rule prevents
/// this crate from defining a direct conversion from `base::Error` to
/// `hyper::Response`.
struct HttpError(Response<Body>);
impl From<Response<Body>> for HttpError {
fn from(response: Response<Body>) -> Self {
HttpError(response)
}
}
impl From<base::Error> for HttpError {
fn from(err: base::Error) -> Self {
HttpError(from_base_error(err))
}
}
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
fn plain_response<B: Into<Body>>(status: http::StatusCode, body: B) -> Response<Body> {
Response::builder()
.status(status)
.header(header::CONTENT_TYPE, HeaderValue::from_static("text/plain"))
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
.body(body.into())
.expect("hardcoded head should be valid")
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
}
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
fn not_found<B: Into<Body>>(body: B) -> HttpError {
HttpError(plain_response(StatusCode::NOT_FOUND, body))
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
}
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
fn bad_req<B: Into<Body>>(body: B) -> HttpError {
HttpError(plain_response(StatusCode::BAD_REQUEST, body))
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
}
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
fn internal_server_err<E: Into<Error>>(err: E) -> HttpError {
HttpError(plain_response(
StatusCode::INTERNAL_SERVER_ERROR,
err.into().to_string(),
))
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
}
introduce typed errors and use in mp4 code Fixes #46. If there are no video_sample_entries, it returns InvalidArgument, which gets mapped to a HTTP 400. Various other failures turn into non-500s as well. There are many places that can & should be using typed errors, but it's a start.
2018-12-28 17:30:33 -06:00
fn from_base_error(err: base::Error) -> Response<Body> {
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
use ErrorKind::*;
introduce typed errors and use in mp4 code Fixes #46. If there are no video_sample_entries, it returns InvalidArgument, which gets mapped to a HTTP 400. Various other failures turn into non-500s as well. There are many places that can & should be using typed errors, but it's a start.
2018-12-28 17:30:33 -06:00
let status_code = match err.kind() {
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
Unauthenticated => StatusCode::UNAUTHORIZED,
PermissionDenied => StatusCode::FORBIDDEN,
InvalidArgument | FailedPrecondition => StatusCode::BAD_REQUEST,
NotFound => StatusCode::NOT_FOUND,
introduce typed errors and use in mp4 code Fixes #46. If there are no video_sample_entries, it returns InvalidArgument, which gets mapped to a HTTP 400. Various other failures turn into non-500s as well. There are many places that can & should be using typed errors, but it's a start.
2018-12-28 17:30:33 -06:00
_ => StatusCode::INTERNAL_SERVER_ERROR,
};
plain_response(status_code, err.to_string())
}
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 15:17:50 -07:00
struct Caller {
permissions: db::Permissions,
UI preferences: #153 #155
2021-09-01 15:01:42 -07:00
user: Option<json::ToplevelUser>,
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 15:17:50 -07:00
}
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
type ResponseResult = Result<Response<Body>, HttpError>;
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
read-only signals support (#28) This is mostly untested and useless by itself, but it's a starting point. In particular: * there's no way to set up signals or add/remove/update events yet except by manual changes to the database. * if you associate a signal with a camera then remove the camera, hitting /api/ will error out.
2019-06-06 16:18:13 -07:00
fn serve_json<T: serde::ser::Serialize>(req: &Request<hyper::Body>, out: &T) -> ResponseResult {
let (mut resp, writer) = http_serve::streaming_body(&req).build();
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
resp.headers_mut().insert(
header::CONTENT_TYPE,
HeaderValue::from_static("application/json"),
);
read-only signals support (#28) This is mostly untested and useless by itself, but it's a starting point. In particular: * there's no way to set up signals or add/remove/update events yet except by manual changes to the database. * if you associate a signal with a camera then remove the camera, hitting /api/ will error out.
2019-06-06 16:18:13 -07:00
if let Some(mut w) = writer {
serde_json::to_writer(&mut w, out).map_err(internal_server_err)?;
}
Ok(resp)
}
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
fn csrf_matches(csrf: &str, session: auth::SessionHash) -> bool {
let mut b64 = [0u8; 32];
session.encode_base64(&mut b64);
::ring::constant_time::verify_slices_are_equal(&b64[..], csrf.as_bytes()).is_ok()
}
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-21 21:54:27 -07:00
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
/// Extracts `s` cookie from the HTTP request. Does not authenticate.
fn extract_sid(req: &Request<hyper::Body>) -> Option<auth::RawSessionId> {
let hdr = match req.headers().get(header::COOKIE) {
None => return None,
Some(c) => c,
};
for mut cookie in hdr.as_bytes().split(|&b| b == b';') {
if cookie.starts_with(b" ") {
cookie = &cookie[1..];
}
if cookie.starts_with(b"s=") {
let s = &cookie[2..];
if let Ok(s) = auth::RawSessionId::decode_base64(s) {
return Some(s);
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 15:17:50 -07:00
}
}
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
}
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
None
}
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
/// Extracts an `application/json` POST body from a request.
///
/// This returns the request body as bytes rather than performing
/// deserialization. Keeping the bytes allows the caller to use a `Deserialize`
/// that borrows from the bytes.
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
async fn extract_json_body(req: &mut Request<hyper::Body>) -> Result<Bytes, HttpError> {
UI preferences: #153 #155
2021-09-01 15:01:42 -07:00
if *req.method() != Method::POST {
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
return Err(plain_response(StatusCode::METHOD_NOT_ALLOWED, "POST expected").into());
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
}
let correct_mime_type = match req.headers().get(header::CONTENT_TYPE) {
Some(t) if t == "application/json" => true,
Some(t) if t == "application/json; charset=UTF-8" => true,
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
_ => false,
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
};
if !correct_mime_type {
return Err(bad_req("expected application/json request body"));
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
}
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
let b = ::std::mem::replace(req.body_mut(), hyper::Body::empty());
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
hyper::body::to_bytes(b)
.await
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
.map_err(|e| internal_server_err(format_err!("unable to read request body: {}", e)))
}
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
pub struct Config<'a> {
pub db: Arc<db::Database>,
pub ui_dir: Option<&'a std::path::Path>,
pub trust_forward_hdrs: bool,
pub time_zone_name: String,
pub allow_unauthenticated_permissions: Option<db::Permissions>,
}
pub struct Service {
db: Arc<db::Database>,
ui_dir: Option<Arc<FsDir>>,
dirs_by_stream_id: Arc<FnvHashMap<i32, Arc<SampleFileDir>>>,
time_zone_name: String,
allow_unauthenticated_permissions: Option<db::Permissions>,
trust_forward_hdrs: bool,
}
/// Useful HTTP `Cache-Control` values to set on successful (HTTP 200) API responses.
enum CacheControl {
/// For endpoints which have private data that may change from request to request.
PrivateDynamic,
/// For endpoints which rarely change for a given URL.
/// E.g., a fixed segment of video. The underlying video logically never changes; there may
/// rarely be some software change to the actual bytes (which would result in a new etag) so
/// (unlike the content-hashed static content) it's not entirely immutable.
PrivateStatic,
None,
}
impl Service {
pub fn new(config: Config) -> Result<Self, Error> {
let mut ui_dir = None;
if let Some(d) = config.ui_dir {
match FsDir::builder().for_path(&d) {
Err(e) => {
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
warn!(
"Unable to load --ui-dir={}; will serve no static files: {}",
d.display(),
e
);
}
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
Ok(d) => ui_dir = Some(d),
};
}
let dirs_by_stream_id = {
let l = config.db.lock();
let mut d =
FnvHashMap::with_capacity_and_hasher(l.streams_by_id().len(), Default::default());
for (&id, s) in l.streams_by_id().iter() {
let dir_id = match s.sample_file_dir_id {
Some(d) => d,
None => continue,
support run splitting in json api
2017-10-17 09:00:05 -07:00
};
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
d.insert(id, l.sample_file_dirs_by_id().get(&dir_id).unwrap().get()?);
support run splitting in json api
2017-10-17 09:00:05 -07:00
}
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
Arc::new(d)
reorganize /recordings JSON response I want to start returning the pixel aspect ratio of each video sample entry. It's silly to duplicate it for each returned recording, so let's instead return a videoSampleEntryId and then put all the information about each VSE once. This change doesn't actually handle pixel aspect ratio server-side yet. Most likely I'll require a new schema version for that, to store it as a new column in the database. Codec-specific logic in the database layer is awkward and I'd like to avoid it. I did a similar schema change to add the rfc6381_codec. I also adjusted ui-src/lib/models/Recording.js in a few ways: * fixed a couple mismatches between its field name and the key defined in the API. Consistency aids understanding. * dropped all the getters in favor of just setting the fields (with type annotations) as described here: https://google.github.io/styleguide/jsguide.html#features-classes-fields * where the wire format used undefined (to save space), translate it to a more natural null or false.
2020-03-13 21:20:51 -07:00
};
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
Ok(Service {
db: config.db,
dirs_by_stream_id,
ui_dir,
allow_unauthenticated_permissions: config.allow_unauthenticated_permissions,
trust_forward_hdrs: config.trust_forward_hdrs,
time_zone_name: config.time_zone_name,
})
support segmented mp4s This is intended to support HTML5 Media Source Extensions, which I expect to be the most practical way to make a good web UI with a proper scrub bar and such. This feature has had very limited testing on Chrome and Firefox, and that was not entirely successful. More work is needed before it's usable, but this seems like a helpful progress checkpoint.
2017-10-01 15:29:22 -07:00
}
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
/// Serves an HTTP request.
/// Note that the `serve` wrapper handles responses the same whether they
/// are `Ok` or `Err`. But returning `Err` here with the `?` operator is
/// convenient for error paths.
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
async fn serve_inner(
self: Arc<Self>,
req: Request<::hyper::Body>,
p: Path,
caller: Caller,
) -> ResponseResult {
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
let (cache, mut response) = match p {
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
Path::InitSegment(sha1, debug) => (
CacheControl::PrivateStatic,
self.init_segment(sha1, debug, &req)?,
),
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
Path::TopLevel => (CacheControl::PrivateDynamic, self.top_level(&req, caller)?),
Path::Request => (CacheControl::PrivateDynamic, self.request(&req)?),
Path::Camera(uuid) => (CacheControl::PrivateDynamic, self.camera(&req, uuid)?),
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
Path::StreamRecordings(uuid, type_) => (
CacheControl::PrivateDynamic,
self.stream_recordings(&req, uuid, type_)?,
),
Path::StreamViewMp4(uuid, type_, debug) => (
CacheControl::PrivateStatic,
self.stream_view_mp4(&req, caller, uuid, type_, mp4::Type::Normal, debug)?,
),
Path::StreamViewMp4Segment(uuid, type_, debug) => (
CacheControl::PrivateStatic,
self.stream_view_mp4(&req, caller, uuid, type_, mp4::Type::MediaSegment, debug)?,
),
Path::StreamLiveMp4Segments(uuid, type_) => (
CacheControl::PrivateDynamic,
self.stream_live_m4s(req, caller, uuid, type_)?,
),
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
Path::NotFound => return Err(not_found("path not understood")),
Path::Login => (CacheControl::PrivateDynamic, self.login(req).await?),
Path::Logout => (CacheControl::PrivateDynamic, self.logout(req).await?),
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
Path::Signals => (
CacheControl::PrivateDynamic,
self.signals(req, caller).await?,
),
Path::Static => (CacheControl::None, self.static_file(req).await?),
UI preferences: #153 #155
2021-09-01 15:01:42 -07:00
Path::User(id) => (
CacheControl::PrivateDynamic,
self.user(req, caller, id).await?,
),
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
};
match cache {
CacheControl::PrivateStatic => {
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
response.headers_mut().insert(
header::CACHE_CONTROL,
HeaderValue::from_static("private, max-age=3600"),
);
}
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
CacheControl::PrivateDynamic => {
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
response.headers_mut().insert(
header::CACHE_CONTROL,
HeaderValue::from_static("private, no-cache"),
);
}
CacheControl::None => {}
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
}
Ok(response)
}
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
/// Serves an HTTP request.
/// An error return from this method causes hyper to abruptly drop the
/// HTTP connection rather than respond. That's not terribly useful, so this
/// method always returns `Ok`. It delegates to a `serve_inner` which is
/// allowed to generate `Err` results with the `?` operator, but returns
/// them to hyper as `Ok` results.
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
pub async fn serve(
self: Arc<Self>,
req: Request<::hyper::Body>,
) -> Result<Response<Body>, std::convert::Infallible> {
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
let p = Path::decode(req.uri().path());
address some no-op clippy warnings
2021-05-17 14:31:50 -07:00
let always_allow_unauthenticated = matches!(
p,
Path::NotFound | Path::Request | Path::Login | Path::Logout | Path::Static
);
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
debug!("request on: {}: {:?}", req.uri(), p);
let caller = match self.authenticate(&req, always_allow_unauthenticated) {
Ok(c) => c,
Err(e) => return Ok(from_base_error(e)),
};
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
Ok(self
.serve_inner(req, p, caller)
.await
.unwrap_or_else(|e| e.0))
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
}
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 15:17:50 -07:00
fn top_level(&self, req: &Request<::hyper::Body>, caller: Caller) -> ResponseResult {
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-21 21:54:27 -07:00
let mut days = false;
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 15:17:50 -07:00
let mut camera_configs = false;
gzip json responses, handle HEAD properly
2018-01-23 11:22:23 -08:00
if let Some(q) = req.uri().query() {
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-21 21:54:27 -07:00
for (key, value) in form_urlencoded::parse(q.as_bytes()) {
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
let (key, value): (_, &str) = (key.borrow(), value.borrow());
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-21 21:54:27 -07:00
match key {
"days" => days = value == "true",
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 15:17:50 -07:00
"cameraConfigs" => camera_configs = value == "true",
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
_ => {}
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-21 21:54:27 -07:00
};
}
}
address some no-op clippy warnings
2021-05-17 14:31:50 -07:00
if camera_configs && !caller.permissions.read_camera_configs {
bail_t!(PermissionDenied, "read_camera_configs required");
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 15:17:50 -07:00
}
read-only signals support (#28) This is mostly untested and useless by itself, but it's a starting point. In particular: * there's no way to set up signals or add/remove/update events yet except by manual changes to the database. * if you associate a signal with a camera then remove the camera, hitting /api/ will error out.
2019-06-06 16:18:13 -07:00
let db = self.db.lock();
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
serve_json(
req,
&json::TopLevel {
time_zone_name: &self.time_zone_name,
add server version to /api response Fixes #171
2021-10-27 13:09:20 -07:00
server_version: env!("CARGO_PKG_VERSION"),
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
cameras: (&db, days, camera_configs),
UI preferences: #153 #155
2021-09-01 15:01:42 -07:00
user: caller.user,
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
signals: (&db, days),
signal_types: &db,
},
)
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
}
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
fn camera(&self, req: &Request<::hyper::Body>, uuid: Uuid) -> ResponseResult {
read-only signals support (#28) This is mostly untested and useless by itself, but it's a starting point. In particular: * there's no way to set up signals or add/remove/update events yet except by manual changes to the database. * if you associate a signal with a camera then remove the camera, hitting /api/ will error out.
2019-06-06 16:18:13 -07:00
let db = self.db.lock();
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
let camera = db
.get_camera(uuid)
.ok_or_else(|| not_found(format!("no such camera {}", uuid)))?;
serve_json(
req,
&json::Camera::wrap(camera, &db, true, false).map_err(internal_server_err)?,
)
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
}
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
fn stream_recordings(
&self,
req: &Request<::hyper::Body>,
uuid: Uuid,
type_: db::StreamType,
) -> ResponseResult {
support run splitting in json api
2017-10-17 09:00:05 -07:00
let (r, split) = {
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
let mut time = recording::Time::min_value()..recording::Time::max_value();
support run splitting in json api
2017-10-17 09:00:05 -07:00
let mut split = recording::Duration(i64::max_value());
gzip json responses, handle HEAD properly
2018-01-23 11:22:23 -08:00
if let Some(q) = req.uri().query() {
support run splitting in json api
2017-10-17 09:00:05 -07:00
for (key, value) in form_urlencoded::parse(q.as_bytes()) {
let (key, value) = (key.borrow(), value.borrow());
match key {
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
"startTime90k" => {
time.start = recording::Time::parse(value)
.map_err(|_| bad_req("unparseable startTime90k"))?
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
}
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
"endTime90k" => {
time.end = recording::Time::parse(value)
.map_err(|_| bad_req("unparseable endTime90k"))?
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
}
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
"split90k" => {
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
split = recording::Duration(
i64::from_str(value)
.map_err(|_| bad_req("unparseable split90k"))?,
)
}
_ => {}
support run splitting in json api
2017-10-17 09:00:05 -07:00
}
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
}
support run splitting in json api
2017-10-17 09:00:05 -07:00
}
(time, split)
};
reorganize /recordings JSON response I want to start returning the pixel aspect ratio of each video sample entry. It's silly to duplicate it for each returned recording, so let's instead return a videoSampleEntryId and then put all the information about each VSE once. This change doesn't actually handle pixel aspect ratio server-side yet. Most likely I'll require a new schema version for that, to store it as a new column in the database. Codec-specific logic in the database layer is awkward and I'd like to avoid it. I did a similar schema change to add the rfc6381_codec. I also adjusted ui-src/lib/models/Recording.js in a few ways: * fixed a couple mismatches between its field name and the key defined in the API. Consistency aids understanding. * dropped all the getters in favor of just setting the fields (with type annotations) as described here: https://google.github.io/styleguide/jsguide.html#features-classes-fields * where the wire format used undefined (to save space), translate it to a more natural null or false.
2020-03-13 21:20:51 -07:00
let db = self.db.lock();
let mut out = json::ListRecordings {
recordings: Vec::new(),
video_sample_entries: (&db, Vec::new()),
};
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
let camera = db.get_camera(uuid).ok_or_else(|| {
plain_response(StatusCode::NOT_FOUND, format!("no such camera {}", uuid))
})?;
let stream_id = camera.streams[type_.index()].ok_or_else(|| {
plain_response(
StatusCode::NOT_FOUND,
format!("no such stream {}/{}", uuid, type_),
)
})?;
reorganize /recordings JSON response I want to start returning the pixel aspect ratio of each video sample entry. It's silly to duplicate it for each returned recording, so let's instead return a videoSampleEntryId and then put all the information about each VSE once. This change doesn't actually handle pixel aspect ratio server-side yet. Most likely I'll require a new schema version for that, to store it as a new column in the database. Codec-specific logic in the database layer is awkward and I'd like to avoid it. I did a similar schema change to add the rfc6381_codec. I also adjusted ui-src/lib/models/Recording.js in a few ways: * fixed a couple mismatches between its field name and the key defined in the API. Consistency aids understanding. * dropped all the getters in favor of just setting the fields (with type annotations) as described here: https://google.github.io/styleguide/jsguide.html#features-classes-fields * where the wire format used undefined (to save space), translate it to a more natural null or false.
2020-03-13 21:20:51 -07:00
db.list_aggregated_recordings(stream_id, r, split, &mut |row| {
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
let end = row.ids.end - 1; // in api, ids are inclusive.
reorganize /recordings JSON response I want to start returning the pixel aspect ratio of each video sample entry. It's silly to duplicate it for each returned recording, so let's instead return a videoSampleEntryId and then put all the information about each VSE once. This change doesn't actually handle pixel aspect ratio server-side yet. Most likely I'll require a new schema version for that, to store it as a new column in the database. Codec-specific logic in the database layer is awkward and I'd like to avoid it. I did a similar schema change to add the rfc6381_codec. I also adjusted ui-src/lib/models/Recording.js in a few ways: * fixed a couple mismatches between its field name and the key defined in the API. Consistency aids understanding. * dropped all the getters in favor of just setting the fields (with type annotations) as described here: https://google.github.io/styleguide/jsguide.html#features-classes-fields * where the wire format used undefined (to save space), translate it to a more natural null or false.
2020-03-13 21:20:51 -07:00
out.recordings.push(json::Recording {
start_id: row.ids.start,
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
end_id: if end == row.ids.start {
None
} else {
Some(end)
},
reorganize /recordings JSON response I want to start returning the pixel aspect ratio of each video sample entry. It's silly to duplicate it for each returned recording, so let's instead return a videoSampleEntryId and then put all the information about each VSE once. This change doesn't actually handle pixel aspect ratio server-side yet. Most likely I'll require a new schema version for that, to store it as a new column in the database. Codec-specific logic in the database layer is awkward and I'd like to avoid it. I did a similar schema change to add the rfc6381_codec. I also adjusted ui-src/lib/models/Recording.js in a few ways: * fixed a couple mismatches between its field name and the key defined in the API. Consistency aids understanding. * dropped all the getters in favor of just setting the fields (with type annotations) as described here: https://google.github.io/styleguide/jsguide.html#features-classes-fields * where the wire format used undefined (to save space), translate it to a more natural null or false.
2020-03-13 21:20:51 -07:00
start_time_90k: row.time.start.0,
end_time_90k: row.time.end.0,
sample_file_bytes: row.sample_file_bytes,
open_id: row.open_id,
first_uncommitted: row.first_uncommitted,
video_samples: row.video_samples,
minor api fixes * videoSampleEntryId should be a number, not a string * fix some markdown formatting * fix missing description of sampleFileBytes
2021-02-22 13:46:51 -08:00
video_sample_entry_id: row.video_sample_entry_id,
reorganize /recordings JSON response I want to start returning the pixel aspect ratio of each video sample entry. It's silly to duplicate it for each returned recording, so let's instead return a videoSampleEntryId and then put all the information about each VSE once. This change doesn't actually handle pixel aspect ratio server-side yet. Most likely I'll require a new schema version for that, to store it as a new column in the database. Codec-specific logic in the database layer is awkward and I'd like to avoid it. I did a similar schema change to add the rfc6381_codec. I also adjusted ui-src/lib/models/Recording.js in a few ways: * fixed a couple mismatches between its field name and the key defined in the API. Consistency aids understanding. * dropped all the getters in favor of just setting the fields (with type annotations) as described here: https://google.github.io/styleguide/jsguide.html#features-classes-fields * where the wire format used undefined (to save space), translate it to a more natural null or false.
2020-03-13 21:20:51 -07:00
growing: row.growing,
UNTESTED: note trailing zeros in /recordings reply
2021-10-10 16:13:57 -07:00
has_trailing_zero: row.has_trailing_zero,
reorganize /recordings JSON response I want to start returning the pixel aspect ratio of each video sample entry. It's silly to duplicate it for each returned recording, so let's instead return a videoSampleEntryId and then put all the information about each VSE once. This change doesn't actually handle pixel aspect ratio server-side yet. Most likely I'll require a new schema version for that, to store it as a new column in the database. Codec-specific logic in the database layer is awkward and I'd like to avoid it. I did a similar schema change to add the rfc6381_codec. I also adjusted ui-src/lib/models/Recording.js in a few ways: * fixed a couple mismatches between its field name and the key defined in the API. Consistency aids understanding. * dropped all the getters in favor of just setting the fields (with type annotations) as described here: https://google.github.io/styleguide/jsguide.html#features-classes-fields * where the wire format used undefined (to save space), translate it to a more natural null or false.
2020-03-13 21:20:51 -07:00
});
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
if !out
.video_sample_entries
.1
.contains(&row.video_sample_entry_id)
{
reorganize /recordings JSON response I want to start returning the pixel aspect ratio of each video sample entry. It's silly to duplicate it for each returned recording, so let's instead return a videoSampleEntryId and then put all the information about each VSE once. This change doesn't actually handle pixel aspect ratio server-side yet. Most likely I'll require a new schema version for that, to store it as a new column in the database. Codec-specific logic in the database layer is awkward and I'd like to avoid it. I did a similar schema change to add the rfc6381_codec. I also adjusted ui-src/lib/models/Recording.js in a few ways: * fixed a couple mismatches between its field name and the key defined in the API. Consistency aids understanding. * dropped all the getters in favor of just setting the fields (with type annotations) as described here: https://google.github.io/styleguide/jsguide.html#features-classes-fields * where the wire format used undefined (to save space), translate it to a more natural null or false.
2020-03-13 21:20:51 -07:00
out.video_sample_entries.1.push(row.video_sample_entry_id);
}
Ok(())
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
})
.map_err(internal_server_err)?;
read-only signals support (#28) This is mostly untested and useless by itself, but it's a starting point. In particular: * there's no way to set up signals or add/remove/update events yet except by manual changes to the database. * if you associate a signal with a camera then remove the camera, hitting /api/ will error out.
2019-06-06 16:18:13 -07:00
serve_json(req, &out)
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
}
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
fn init_segment(&self, id: i32, debug: bool, req: &Request<::hyper::Body>) -> ResponseResult {
support segmented mp4s This is intended to support HTML5 Media Source Extensions, which I expect to be the most practical way to make a good web UI with a proper scrub bar and such. This feature has had very limited testing on Chrome and Firefox, and that was not entirely successful. More work is needed before it's usable, but this seems like a helpful progress checkpoint.
2017-10-01 15:29:22 -07:00
let mut builder = mp4::FileBuilder::new(mp4::Type::InitSegment);
let db = self.db.lock();
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
let ent = db
.video_sample_entries_by_id()
.get(&id)
use Blake3 instead of SHA-1 or Blake2b Benefits: * Blake3 is faster. This is most noticeable for the hashing of the sample file data. * we no longer need OpenSSL, which helps with shrinking the binary size (#70). sha1 basically forced OpenSSL usage; ring deliberately doesn't support this old algorithm, and the pure-Rust sha1 crate is painfully slow. OpenSSL might still be a better choice than ring/rustls for TLS but it's nice to have the option. For the video sample entries, I decided we don't need to hash at all. I think the id number is sufficiently stable, and it's okay---perhaps even desirable---if an existing init segment changes for fixes like e5b83c2.
2020-03-20 20:52:30 -07:00
.ok_or_else(|| not_found("not such init segment"))?;
builder.append_video_sample_entry(ent.clone());
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
let mp4 = builder
.build(self.db.clone(), self.dirs_by_stream_id.clone())
use Blake3 instead of SHA-1 or Blake2b Benefits: * Blake3 is faster. This is most noticeable for the hashing of the sample file data. * we no longer need OpenSSL, which helps with shrinking the binary size (#70). sha1 basically forced OpenSSL usage; ring deliberately doesn't support this old algorithm, and the pure-Rust sha1 crate is painfully slow. OpenSSL might still be a better choice than ring/rustls for TLS but it's nice to have the option. For the video sample entries, I decided we don't need to hash at all. I think the id number is sufficiently stable, and it's okay---perhaps even desirable---if an existing init segment changes for fixes like e5b83c2.
2020-03-20 20:52:30 -07:00
.map_err(from_base_error)?;
if debug {
Ok(plain_response(StatusCode::OK, format!("{:#?}", mp4)))
} else {
Ok(http_serve::serve(mp4, req))
support segmented mp4s This is intended to support HTML5 Media Source Extensions, which I expect to be the most practical way to make a good web UI with a proper scrub bar and such. This feature has had very limited testing on Chrome and Firefox, and that was not entirely successful. More work is needed before it's usable, but this seems like a helpful progress checkpoint.
2017-10-01 15:29:22 -07:00
}
}
UI preferences: #153 #155
2021-09-01 15:01:42 -07:00
async fn user(&self, req: Request<hyper::Body>, caller: Caller, id: i32) -> ResponseResult {
if caller.user.map(|u| u.id) != Some(id) {
bail_t!(Unauthenticated, "must be authenticated as supplied user");
}
match *req.method() {
Method::POST => self.post_user(req, id).await,
_ => Err(plain_response(StatusCode::METHOD_NOT_ALLOWED, "POST expected").into()),
}
}
async fn post_user(&self, mut req: Request<hyper::Body>, id: i32) -> ResponseResult {
let r = extract_json_body(&mut req).await?;
let r: json::PostUser = serde_json::from_slice(&r).map_err(|e| bad_req(e.to_string()))?;
let mut db = self.db.lock();
let user = db
.users_by_id()
.get(&id)
.ok_or_else(|| format_err_t!(Internal, "can't find currently authenticated user"))?;
if let Some(precondition) = r.precondition {
add config json to user table
2021-10-26 13:08:45 -07:00
if matches!(precondition.preferences, Some(p) if p != user.config.preferences) {
UI preferences: #153 #155
2021-09-01 15:01:42 -07:00
bail_t!(FailedPrecondition, "preferences mismatch");
}
}
if let Some(update) = r.update {
let mut change = user.change();
if let Some(preferences) = update.preferences {
add config json to user table
2021-10-26 13:08:45 -07:00
change.config.preferences = preferences;
UI preferences: #153 #155
2021-09-01 15:01:42 -07:00
}
db.apply_user_change(change).map_err(internal_server_err)?;
}
Ok(plain_response(StatusCode::NO_CONTENT, &b""[..]))
}
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
fn authreq(&self, req: &Request<::hyper::Body>) -> auth::Request {
auth::Request {
when_sec: Some(self.db.clocks().realtime().sec),
support proxy forwarded headers I went with legacy headers (X-Real-IP, X-Forwarded-Proto) because they appear to be more widely supported than the RFC 7239 Forwarded header.
2018-11-28 14:22:30 -08:00
addr: if self.trust_forward_hdrs {
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
req.headers()
.get("X-Real-IP")
.and_then(|v| v.to_str().ok())
.and_then(|v| IpAddr::from_str(v).ok())
} else {
None
},
user_agent: req
.headers()
.get(header::USER_AGENT)
.map(|ua| ua.as_bytes().to_vec()),
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
}
}
endpoint to debug --trust-forward-hdrs for #26
2018-12-01 00:44:19 -08:00
fn request(&self, req: &Request<::hyper::Body>) -> ResponseResult {
let authreq = self.authreq(req);
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
let host = req
.headers()
.get(header::HOST)
.map(|h| String::from_utf8_lossy(h.as_bytes()));
let agent = authreq
.user_agent
.as_ref()
.map(|u| String::from_utf8_lossy(&u[..]));
Ok(plain_response(
StatusCode::OK,
format!(
"when: {}\n\
document proxy setup in guide/secure.md (for #26) The guide is not as quick to follow and amateur-friendly as I'd like. A few things that might improve matters: * complete #27 (built-in https+letsencrypt), so that when not sharing the port, users don't need to use nginx or certbot. * more ubiquitous IPv6 (out of my control but should happen over time) to reduce need to share the port * embed a dynamic DNS client * support UPnP Internet Gateway Device Control Protocol (if common routers have this enabled? probably not for security reasons.) It's progress, though. Enough that I think I'll merge the auth branch into master shortly.
2018-12-27 16:00:15 -06:00
host: {:?}\n\
endpoint to debug --trust-forward-hdrs for #26
2018-12-01 00:44:19 -08:00
addr: {:?}\n\
user_agent: {:?}\n\
secure: {:?}",
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
time::at(time::Timespec {
sec: authreq.when_sec.unwrap(),
nsec: 0
})
.strftime("%FT%T")
.map(|f| f.to_string())
.unwrap_or_else(|e| e.to_string()),
cleanup some awkward Option call chains
2021-05-06 09:47:07 -07:00
host.as_deref(),
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
&authreq.addr,
cleanup some awkward Option call chains
2021-05-06 09:47:07 -07:00
agent.as_deref(),
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
self.is_secure(req)
),
))
endpoint to debug --trust-forward-hdrs for #26
2018-12-01 00:44:19 -08:00
}
small readability improvements to web.rs I think this has a minor behavior change: permission denied replies change to HTTP 403 where they were HTTP 401. The new behavior seems more correct, as these errors can occur when authentication has succeeded but the session in question is not authorized for the given operation. The UI currently doesn't care about this distinction.
2021-03-10 16:18:47 -08:00
/// Returns true iff the client is connected over `https`.
/// Moonfire NVR currently doesn't directly serve `https`, but it supports
/// proxies which set the `X-Forwarded-Proto` header. See `guide/secure.md`
/// for more information.
support proxy forwarded headers I went with legacy headers (X-Real-IP, X-Forwarded-Proto) because they appear to be more widely supported than the RFC 7239 Forwarded header.
2018-11-28 14:22:30 -08:00
fn is_secure(&self, req: &Request<::hyper::Body>) -> bool {
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
self.trust_forward_hdrs
&& req
.headers()
.get("X-Forwarded-Proto")
.map(|v| v.as_bytes() == b"https")
.unwrap_or(false)
support proxy forwarded headers I went with legacy headers (X-Real-IP, X-Forwarded-Proto) because they appear to be more widely supported than the RFC 7239 Forwarded header.
2018-11-28 14:22:30 -08:00
}
better error handling for authentication
2021-03-06 05:16:09 -08:00
/// Authenticates the session (if any) and returns a Caller.
///
/// If there's no session,
/// 1. if `allow_unauthenticated_permissions` is configured, returns okay
/// with those permissions.
/// 2. if the caller specifies `unauth_path`, returns okay with no
/// permissions.
/// 3. returns `Unauthenticated` error otherwise.
///
/// Does no authorization. That is, this doesn't check that the returned
/// permissions are sufficient for whatever operation the caller is
/// performing.
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
fn authenticate(
&self,
req: &Request<hyper::Body>,
unauth_path: bool,
) -> Result<Caller, base::Error> {
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
if let Some(sid) = extract_sid(req) {
let authreq = self.authreq(req);
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 15:17:50 -07:00
address some no-op clippy warnings
2021-05-17 14:31:50 -07:00
match self.db.lock().authenticate_session(authreq, &sid.hash()) {
better error handling for authentication
2021-03-06 05:16:09 -08:00
Ok((s, u)) => {
return Ok(Caller {
permissions: s.permissions.clone(),
UI preferences: #153 #155
2021-09-01 15:01:42 -07:00
user: Some(json::ToplevelUser {
id: s.user_id,
name: u.username.clone(),
add config json to user table
2021-10-26 13:08:45 -07:00
preferences: u.config.preferences.clone(),
UI preferences: #153 #155
2021-09-01 15:01:42 -07:00
session: Some(json::Session { csrf: s.csrf() }),
better error handling for authentication
2021-03-06 05:16:09 -08:00
}),
})
}
Err(e) if e.kind() == base::ErrorKind::Unauthenticated => {
// Log the specific reason this session is unauthenticated.
// Don't let the API client see it, as it may have a
// revocation reason that isn't for their eyes.
warn!("Session authentication failed: {:?}", &e);
}
Err(e) => return Err(e),
};
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
}
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 15:17:50 -07:00
if let Some(s) = self.allow_unauthenticated_permissions.as_ref() {
return Ok(Caller {
permissions: s.clone(),
UI preferences: #153 #155
2021-09-01 15:01:42 -07:00
user: None,
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 15:17:50 -07:00
});
}
if unauth_path {
return Ok(Caller {
permissions: db::Permissions::default(),
UI preferences: #153 #155
2021-09-01 15:01:42 -07:00
user: None,
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
});
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 15:17:50 -07:00
}
bail_t!(Unauthenticated, "unauthenticated");
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
}
}
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
#[cfg(test)]
mod tests {
lose "extern crate" everywhere (Rust 2018 edition)
2018-12-28 21:53:29 -06:00
use db::testutil::{self, TestDb};
upgrade to hyper 0.13 ecosystem This doesn't take much advantage of async fns so far. For example, the with_{form,json}_body functions are still designed to be used with future combinators when it'd be more natural to call them from async fns now. But it's a start. Similarly, this still uses the old version of reqwest. Small steps. Requires Rust 1.40 now. (1.39 is a requirement of async, and 1.40 is a requirement of http-serve 0.2.0.)
2020-01-08 23:04:36 -08:00
use futures::future::FutureExt;
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
use std::sync::Arc;
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
move /view.{mp4,m4s} to their own file
2021-10-28 13:23:49 -07:00
pub(super) struct Server {
pub(super) db: TestDb<base::clock::RealClocks>,
pub(super) base_url: String,
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
//test_camera_uuid: Uuid,
handle: Option<::std::thread::JoinHandle<()>>,
upgrade to hyper 0.13 ecosystem This doesn't take much advantage of async fns so far. For example, the with_{form,json}_body functions are still designed to be used with future combinators when it'd be more natural to call them from async fns now. But it's a start. Similarly, this still uses the old version of reqwest. Small steps. Requires Rust 1.40 now. (1.39 is a requirement of async, and 1.40 is a requirement of http-serve 0.2.0.)
2020-01-08 23:04:36 -08:00
shutdown_tx: Option<futures::channel::oneshot::Sender<()>>,
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
}
impl Server {
move /view.{mp4,m4s} to their own file
2021-10-28 13:23:49 -07:00
pub(super) fn new(allow_unauthenticated_permissions: Option<db::Permissions>) -> Server {
lose "extern crate" everywhere (Rust 2018 edition)
2018-12-28 21:53:29 -06:00
let db = TestDb::new(base::clock::RealClocks {});
upgrade to hyper 0.13 ecosystem This doesn't take much advantage of async fns so far. For example, the with_{form,json}_body functions are still designed to be used with future combinators when it'd be more natural to call them from async fns now. But it's a start. Similarly, this still uses the old version of reqwest. Small steps. Requires Rust 1.40 now. (1.39 is a requirement of async, and 1.40 is a requirement of http-serve 0.2.0.)
2020-01-08 23:04:36 -08:00
let (shutdown_tx, shutdown_rx) = futures::channel::oneshot::channel::<()>();
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
let service = Arc::new(
super::Service::new(super::Config {
db: db.db.clone(),
ui_dir: None,
allow_unauthenticated_permissions,
trust_forward_hdrs: true,
time_zone_name: "".to_owned(),
})
.unwrap(),
);
upgrade to hyper 0.13 ecosystem This doesn't take much advantage of async fns so far. For example, the with_{form,json}_body functions are still designed to be used with future combinators when it'd be more natural to call them from async fns now. But it's a start. Similarly, this still uses the old version of reqwest. Small steps. Requires Rust 1.40 now. (1.39 is a requirement of async, and 1.40 is a requirement of http-serve 0.2.0.)
2020-01-08 23:04:36 -08:00
let make_svc = hyper::service::make_service_fn(move |_conn| {
futures::future::ok::<_, std::convert::Infallible>(hyper::service::service_fn({
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
let s = Arc::clone(&service);
move |req| Arc::clone(&s).serve(req)
upgrade to hyper 0.13 ecosystem This doesn't take much advantage of async fns so far. For example, the with_{form,json}_body functions are still designed to be used with future combinators when it'd be more natural to call them from async fns now. But it's a start. Similarly, this still uses the old version of reqwest. Small steps. Requires Rust 1.40 now. (1.39 is a requirement of async, and 1.40 is a requirement of http-serve 0.2.0.)
2020-01-08 23:04:36 -08:00
}))
});
upgrade some deps, including reqwest The reqwest one is particularly notable because it means not having two versions of hyper/http/tokio/futures/bytes. It also drops a number of transitive deps; with some work I think I could stop depending on regex now.
2020-01-09 20:06:30 -08:00
let (tx, rx) = std::sync::mpsc::channel();
let handle = ::std::thread::spawn(move || {
upgrade to hyper 0.13 ecosystem This doesn't take much advantage of async fns so far. For example, the with_{form,json}_body functions are still designed to be used with future combinators when it'd be more natural to call them from async fns now. But it's a start. Similarly, this still uses the old version of reqwest. Small steps. Requires Rust 1.40 now. (1.39 is a requirement of async, and 1.40 is a requirement of http-serve 0.2.0.)
2020-01-08 23:04:36 -08:00
let addr = ([127, 0, 0, 1], 0).into();
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 11:47:52 -08:00
let rt = tokio::runtime::Runtime::new().unwrap();
let srv = {
let _guard = rt.enter();
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
hyper::server::Server::bind(&addr)
.tcp_nodelay(true)
.serve(make_svc)
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 11:47:52 -08:00
};
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
let addr = srv.local_addr(); // resolve port 0 to a real ephemeral port number.
upgrade some deps, including reqwest The reqwest one is particularly notable because it means not having two versions of hyper/http/tokio/futures/bytes. It also drops a number of transitive deps; with some work I think I could stop depending on regex now.
2020-01-09 20:06:30 -08:00
tx.send(addr).unwrap();
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
rt.block_on(srv.with_graceful_shutdown(shutdown_rx.map(|_| ())))
.unwrap();
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
});
upgrade some deps, including reqwest The reqwest one is particularly notable because it means not having two versions of hyper/http/tokio/futures/bytes. It also drops a number of transitive deps; with some work I think I could stop depending on regex now.
2020-01-09 20:06:30 -08:00
let addr = rx.recv().unwrap();
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
// Create a user.
let mut c = db::UserChange::add_user("slamb".to_owned());
c.set_password("hunter2".to_owned());
db.db.lock().apply_user_change(c).unwrap();
Server {
db,
base_url: format!("http://{}:{}", addr.ip(), addr.port()),
handle: Some(handle),
shutdown_tx: Some(shutdown_tx),
}
}
}
impl Drop for Server {
fn drop(&mut self) {
self.shutdown_tx.take().unwrap().send(()).unwrap();
self.handle.take().unwrap().join().unwrap()
}
}
upgrade some deps, including reqwest The reqwest one is particularly notable because it means not having two versions of hyper/http/tokio/futures/bytes. It also drops a number of transitive deps; with some work I think I could stop depending on regex now.
2020-01-09 20:06:30 -08:00
#[tokio::test]
async fn unauthorized_without_cookie() {
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
testutil::init();
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 15:17:50 -07:00
let s = Server::new(None);
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
let cli = reqwest::Client::new();
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
let resp = cli
.get(&format!("{}/api/", &s.base_url))
.send()
.await
.unwrap();
upgrade to hyper 0.13 ecosystem This doesn't take much advantage of async fns so far. For example, the with_{form,json}_body functions are still designed to be used with future combinators when it'd be more natural to call them from async fns now. But it's a start. Similarly, this still uses the old version of reqwest. Small steps. Requires Rust 1.40 now. (1.39 is a requirement of async, and 1.40 is a requirement of http-serve 0.2.0.)
2020-01-08 23:04:36 -08:00
assert_eq!(resp.status(), reqwest::StatusCode::UNAUTHORIZED);
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-25 21:31:50 -08:00
}
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 14:34:00 -08:00
}
benchmark camera page, fix broken schema This page was noticeably slower than necessary because the recording_cover index wasn't actually covering the query. Both the schema for new databases and the upgrade query were broken (and not even in the same way). No new schema version to correct this, at least for now. I'll probably have another reason to change the schema soon anyway and can throw this in.
2017-02-12 20:37:03 -08:00
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
#[cfg(all(test, feature = "nightly"))]
benchmark camera page, fix broken schema This page was noticeably slower than necessary because the recording_cover index wasn't actually covering the query. Both the schema for new databases and the upgrade query were broken (and not even in the same way). No new schema version to correct this, at least for now. I'll probably have another reason to change the schema soon anyway and can throw this in.
2017-02-12 20:37:03 -08:00
mod bench {
extern crate test;
initial split of database to a separate crate It should reduce compile time / memory usage to put quite a bit of the code into a separate crate. I also intend to limit visibility of some things to only within the db crate, but that's for a future change. This is the smallest move that will compile.
2018-02-20 23:15:39 -08:00
use db::testutil::{self, TestDb};
benchmark camera page, fix broken schema This page was noticeably slower than necessary because the recording_cover index wasn't actually covering the query. Both the schema for new databases and the upgrade query were broken (and not even in the same way). No new schema version to correct this, at least for now. I'll probably have another reason to change the schema soon anyway and can throw this in.
2017-02-12 20:37:03 -08:00
use hyper;
lose "extern crate" everywhere (Rust 2018 edition)
2018-12-28 21:53:29 -06:00
use lazy_static::lazy_static;
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
use std::sync::Arc;
use add_camera in tests, not direct db inserts This is a wash in terms of lines of code now, but it makes it a bit easier to maintain as I make changes to the schema (such as separating out streams from cameras), and it helps ensure the tests reflect reality.
2018-02-03 21:56:04 -08:00
use uuid::Uuid;
benchmark camera page, fix broken schema This page was noticeably slower than necessary because the recording_cover index wasn't actually covering the query. Both the schema for new databases and the upgrade query were broken (and not even in the same way). No new schema version to correct this, at least for now. I'll probably have another reason to change the schema soon anyway and can throw this in.
2017-02-12 20:37:03 -08:00
struct Server {
base_url: String,
use add_camera in tests, not direct db inserts This is a wash in terms of lines of code now, but it makes it a bit easier to maintain as I make changes to the schema (such as separating out streams from cameras), and it helps ensure the tests reflect reality.
2018-02-03 21:56:04 -08:00
test_camera_uuid: Uuid,
benchmark camera page, fix broken schema This page was noticeably slower than necessary because the recording_cover index wasn't actually covering the query. Both the schema for new databases and the upgrade query were broken (and not even in the same way). No new schema version to correct this, at least for now. I'll probably have another reason to change the schema soon anyway and can throw this in.
2017-02-12 20:37:03 -08:00
}
impl Server {
fn new() -> Server {
refine flush_if_sec behavior The new behavior eliminates a couple unpleasant edge cases in which it would never flush: * if all recording stops, whatever was unflushed would stay that way * if every recording attempt produces a 0-duration recording (such as if the camera sends only one frame and thus no PTS delta can be calculated), the list of recordings to flush would continue to grow
2018-03-23 15:16:43 -07:00
let db = TestDb::new(::base::clock::RealClocks {});
use add_camera in tests, not direct db inserts This is a wash in terms of lines of code now, but it makes it a bit easier to maintain as I make changes to the schema (such as separating out streams from cameras), and it helps ensure the tests reflect reality.
2018-02-03 21:56:04 -08:00
let test_camera_uuid = db.test_camera_uuid;
benchmark camera page, fix broken schema This page was noticeably slower than necessary because the recording_cover index wasn't actually covering the query. Both the schema for new databases and the upgrade query were broken (and not even in the same way). No new schema version to correct this, at least for now. I'll probably have another reason to change the schema soon anyway and can throw this in.
2017-02-12 20:37:03 -08:00
testutil::add_dummy_recordings_to_db(&db.db, 1440);
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
let service = Arc::new(
super::Service::new(super::Config {
db: db.db.clone(),
ui_dir: None,
allow_unauthenticated_permissions: Some(db::Permissions::default()),
trust_forward_hdrs: false,
time_zone_name: "".to_owned(),
})
.unwrap(),
);
upgrade to hyper 0.13 ecosystem This doesn't take much advantage of async fns so far. For example, the with_{form,json}_body functions are still designed to be used with future combinators when it'd be more natural to call them from async fns now. But it's a start. Similarly, this still uses the old version of reqwest. Small steps. Requires Rust 1.40 now. (1.39 is a requirement of async, and 1.40 is a requirement of http-serve 0.2.0.)
2020-01-08 23:04:36 -08:00
let make_svc = hyper::service::make_service_fn(move |_conn| {
futures::future::ok::<_, std::convert::Infallible>(hyper::service::service_fn({
more readable async web code This uses "async fn" throughout rather than a mix of async and the older futures style. And it takes advantage of the "self: Arc<Self>" syntax to avoid having a ServiceInner. It was confusing to have some methods on Service and some on ServiceInner; now that distinction is gone. One downside is there's a little more atomic reference-counting. Before, service_fn essentially took an &Arc<Self>, which means it could call Arc::clone where its use of self actually outlived the future (see stream_live_m4s) but didn't need to otherwise. After, it calls an async fn that takes Arc<Self>. Using &Arc<Self> is apparently possible (as of Rust 1.41) but using that with "async fn" means the returned future is tied to its lifetime. The workaround is to use async blocks as described here: <https://rust-lang.github.io/async-book/03_async_await/01_chapter.html> but that's really ugly: it brings back the explicit Future reference, requires futures::future::Either in some cases, and introduces another level of indenting. I think it's better to just pay the arc costs which are probably negligible, or at least cheaper than the boxing was before. Oh, and I make this compile on Rust 1.40 again as it claimed to. http-serve accidentally used the &Arc<Self> thing which broke this. Update to a freshly-pushed commit which doesn't do that.
2020-05-30 19:08:54 -07:00
let s = Arc::clone(&service);
move |req| Arc::clone(&s).serve(req)
upgrade to hyper 0.13 ecosystem This doesn't take much advantage of async fns so far. For example, the with_{form,json}_body functions are still designed to be used with future combinators when it'd be more natural to call them from async fns now. But it's a start. Similarly, this still uses the old version of reqwest. Small steps. Requires Rust 1.40 now. (1.39 is a requirement of async, and 1.40 is a requirement of http-serve 0.2.0.)
2020-01-08 23:04:36 -08:00
}))
});
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 11:47:52 -08:00
let rt = tokio::runtime::Runtime::new().unwrap();
let srv = {
let _guard = rt.enter();
upgrade to hyper 0.13 ecosystem This doesn't take much advantage of async fns so far. For example, the with_{form,json}_body functions are still designed to be used with future combinators when it'd be more natural to call them from async fns now. But it's a start. Similarly, this still uses the old version of reqwest. Small steps. Requires Rust 1.40 now. (1.39 is a requirement of async, and 1.40 is a requirement of http-serve 0.2.0.)
2020-01-08 23:04:36 -08:00
let addr = ([127, 0, 0, 1], 0).into();
hyper::server::Server::bind(&addr)
.tcp_nodelay(true)
.serve(make_svc)
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 11:47:52 -08:00
};
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
let addr = srv.local_addr(); // resolve port 0 to a real ephemeral port number.
benchmark camera page, fix broken schema This page was noticeably slower than necessary because the recording_cover index wasn't actually covering the query. Both the schema for new databases and the upgrade query were broken (and not even in the same way). No new schema version to correct this, at least for now. I'll probably have another reason to change the schema soon anyway and can throw this in.
2017-02-12 20:37:03 -08:00
::std::thread::spawn(move || {
upgrade to hyper 0.13 ecosystem This doesn't take much advantage of async fns so far. For example, the with_{form,json}_body functions are still designed to be used with future combinators when it'd be more natural to call them from async fns now. But it's a start. Similarly, this still uses the old version of reqwest. Small steps. Requires Rust 1.40 now. (1.39 is a requirement of async, and 1.40 is a requirement of http-serve 0.2.0.)
2020-01-08 23:04:36 -08:00
rt.block_on(srv).unwrap();
benchmark camera page, fix broken schema This page was noticeably slower than necessary because the recording_cover index wasn't actually covering the query. Both the schema for new databases and the upgrade query were broken (and not even in the same way). No new schema version to correct this, at least for now. I'll probably have another reason to change the schema soon anyway and can throw this in.
2017-02-12 20:37:03 -08:00
});
use add_camera in tests, not direct db inserts This is a wash in terms of lines of code now, but it makes it a bit easier to maintain as I make changes to the schema (such as separating out streams from cameras), and it helps ensure the tests reflect reality.
2018-02-03 21:56:04 -08:00
Server {
base_url: format!("http://{}:{}", addr.ip(), addr.port()),
test_camera_uuid,
}
benchmark camera page, fix broken schema This page was noticeably slower than necessary because the recording_cover index wasn't actually covering the query. Both the schema for new databases and the upgrade query were broken (and not even in the same way). No new schema version to correct this, at least for now. I'll probably have another reason to change the schema soon anyway and can throw this in.
2017-02-12 20:37:03 -08:00
}
}
lazy_static! {
live stream frame-by-frame rather than GOP-by-GOP (#59) This should reduce live stream latency by two seconds when my cameras are at their default setting (I frame interval = 2 * frame rate)! I was under the impression that every HTML5 Media Source Extensions media segment had to start with a Random Access Point. This used to be true, but apparently changed quite a while ago: https://bugs.chromium.org/p/chromium/issues/detail?id=229412 Support generating segments that don't start with a key frame, and plumb this through the mp4 media segment generation logic. Add some extra error checking in mp4 slice handling, as my first attempts had a mismatch between expected and actual lengths that silently returned corrupted .m4s files. Also pull everything from the most recent key frame on along with the first live segment to reduce startup latency. Live view is quite a bit more pleasant now.
2020-08-07 15:30:22 -07:00
static ref SERVER: Server = Server::new();
benchmark camera page, fix broken schema This page was noticeably slower than necessary because the recording_cover index wasn't actually covering the query. Both the schema for new databases and the upgrade query were broken (and not even in the same way). No new schema version to correct this, at least for now. I'll probably have another reason to change the schema soon anyway and can throw this in.
2017-02-12 20:37:03 -08:00
}
#[bench]
lose "extern crate" everywhere (Rust 2018 edition)
2018-12-28 21:53:29 -06:00
fn serve_stream_recordings(b: &mut test::Bencher) {
benchmark camera page, fix broken schema This page was noticeably slower than necessary because the recording_cover index wasn't actually covering the query. Both the schema for new databases and the upgrade query were broken (and not even in the same way). No new schema version to correct this, at least for now. I'll probably have another reason to change the schema soon anyway and can throw this in.
2017-02-12 20:37:03 -08:00
testutil::init();
let server = &*SERVER;
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
let url = reqwest::Url::parse(&format!(
"{}/api/cameras/{}/main/recordings",
server.base_url, server.test_camera_uuid
))
.unwrap();
update dependencies
2017-11-16 23:01:09 -08:00
let client = reqwest::Client::new();
upgrade tokio, bytes, hyper, reqwest, http-serve This was mostly straightforward. The most confusing part waas the Sync bound change on body streams. I copied what hyper did and it seemed to work. /shruggie
2021-01-27 11:47:52 -08:00
let rt = tokio::runtime::Runtime::new().unwrap();
let f = || {
fix some reqwest upgrade brokenness in benches
2020-02-16 23:58:07 -08:00
rt.block_on(async {
let resp = client.get(url.clone()).send().await.unwrap();
assert_eq!(resp.status(), reqwest::StatusCode::OK);
let _b = resp.bytes().await.unwrap();
});
reuse reqwest client in serve_camera_html bench This makes a huge difference in the reported time - 863 usec rather than 6 milliseconds on my laptop. Part of the difference is in reqwest client setup (it apparently initializes a SSL_CTX that is never used in this test), part fresh connections vs keepalive, part I don't know what. None of it seems relevant to the logic I want to test.
2017-03-03 22:26:29 -08:00
};
rustfmt everything I want to make the project more accessible by not expecting folks to match my idiosyncratic style. Now almost [1] everything is written in the "standard" style. CI enforces this. [1] "Almost": I used #[rustfmt::skip] in a few sections where I felt aligning things in columns significantly improves readability.
2021-02-16 22:15:54 -08:00
f(); // warm.
reuse reqwest client in serve_camera_html bench This makes a huge difference in the reported time - 863 usec rather than 6 milliseconds on my laptop. Part of the difference is in reqwest client setup (it apparently initializes a SSL_CTX that is never used in this test), part fresh connections vs keepalive, part I don't know what. None of it seems relevant to the logic I want to test.
2017-03-03 22:26:29 -08:00
b.iter(f);
benchmark camera page, fix broken schema This page was noticeably slower than necessary because the recording_cover index wasn't actually covering the query. Both the schema for new databases and the upgrade query were broken (and not even in the same way). No new schema version to correct this, at least for now. I'll probably have another reason to change the schema soon anyway and can throw this in.
2017-02-12 20:37:03 -08:00
}
}
Reference in New Issue Copy Permalink