223 Commits

Author SHA1 Message Date
Scott Lamb
49a8e5c5a1 Add "moonfire-nvr login username" command
This should be useful when creating sessions for robot users without
ever having to set a password for them.
2019-06-20 09:07:01 -07:00
Scott Lamb
004aa5d6ce extend "moonfire-nvr sql" to pass args to sqlite3 2019-06-19 21:15:25 -07:00
Scott Lamb
a6ebf2d10d add "sql" subcommand
This delegates to the "sqlite3" CLI but has a couple benefits over using
sqlite3 directly:

  * safer because it does the same locking as other moonfire-nvr invocations
  * more convenient because it takes the same argument format as other
    moonfire-nvr subcommands:
      * --db-dir rather than full path including /db suffix
      * has the --db-dir default value
      * --read-only rather than file:...?mode=ro

Use like "moonfire-nvr sql" or "moonfire-nvr sql --read-only".
2019-06-19 17:20:44 -07:00
Scott Lamb
fda7e4ca2b add concept of user/session permissions
(I also considered the names "capabilities" and "scopes", but I think
"permissions" is the most widely understood.)

This is increasingly necessary as the web API becomes more capable.
Among other things, it allows:

* non-administrator users who can view but not access camera passwords
  or change any state
* workers that update signal state based on cameras' built-in motion
  detection or a security system's events but don't need to view videos
* control over what can be done without authenticating

Currently session permissions are just copied from user permissions, but
you can also imagine admin sessions vs not, as a checkbox when signing
in. This would match the standard Unix workflow of using a
non-administrative session most of the time.

Relevant to my current signals work (#28) and to the addition of an
administrative API (#35, including #66).
2019-06-19 15:34:20 -07:00
Scott Lamb
6d4b06f7d2 web api glue for updating signals
This is very lightly tested, but it at least sometimes works.
2019-06-14 16:11:12 -07:00
Scott Lamb
7fe9d34655 cargo fix --all
* it added "dyn" to trait objects
* it changed "..." in patterns to "..="

cargo --version says: "cargo 1.37.0-nightly (545f35425 2019-05-23)"
2019-06-14 08:47:11 -07:00
Scott Lamb
7dd98bb76a db crate support for updating signals (#28)
This is a definite work in progress. In particular,

* there's no src/web.rs support yet so it can't be used,
* the code is surprisingly complex, and there's almost no tests so far.
  I want to at least get complete branch coverage.
* I may still go back to time_sec rather than time_90k to save RAM and
  flash.

I simplified the approach a bit from the earlier goal in design/api.md.
In particular, there's no longer the separate concept of "observation"
vs "prediction". Now the predictions are just observations that extend a
bit beyond now. They may be flushed prematurely and I'll try living with
that to avoid making things even more complex.
2019-06-13 22:25:55 -07:00
Scott Lamb
6f2c63ffac read-only signals support (#28)
This is mostly untested and useless by itself, but it's a starting
point. In particular:

* there's no way to set up signals or add/remove/update events yet
  except by manual changes to the database.
* if you associate a signal with a camera then remove the camera,
  hitting /api/ will error out.
2019-06-06 16:20:44 -07:00
Scott Lamb
cb1bb5d810 upgrade cursive to latest major version 2019-05-31 16:35:07 -07:00
Scott Lamb
428f5a3ba4 update a few deps
cursive & rusqlite are more significant; I'll do those separately
2019-05-31 15:08:49 -07:00
Scott Lamb
579150c9d5 redact URLs within stream.rs; fixes #13 2019-02-13 22:34:19 -08:00
Scott Lamb
091217b1a4 configure ffmpeg to only stream video
This works around #36 for now. I'll need to do something different when
I actually implement audio support.
2019-02-11 22:58:09 -08:00
Scott Lamb
c271cfa2b5 make Writer enforce maximum recording duration
My installation recently somehow ended up with a recording with a
duration of 503793844 90,000ths of a second, way over the maximum of 5
minutes. (Looks like the machine was pretty unresponsive at the time
and/or having network problems.)

When this happens, the system really spirals. Every flush afterward (12
per minute with my installation) fails with a CHECK constraint failure
on the recording table. It never gives up on that recording. /var/log
fills pretty quickly as this failure is extremely verbose (a stack
trace, and a line for each byte of video_index). Eventually the sample
file dirs fill up too as it continues writing video samples while GC is
stuck. The video samples are useless anyway; given that they're not
referenced in the database, they'll be deleted on next startup.

This ensures the offending recording is never added to the database, so
we don't get the same persistent problem. Instead, writing to the
recording will fail. The stream will drop and be retried. If the
underlying condition that caused a too-long recording (many
non-key-frames, or the camera returning a crazy duration, or the
monotonic clock jumping forward extremely, or something) has gone away,
the system should recover.
2019-01-29 08:26:36 -08:00
Scott Lamb
3ba3bf2b18 backend support for live stream (#59)
This is so far completely untested, for use by a new UI prototype.

It creates a new URL endpoint which sends one video/mp4 media segment
per key frame, with the dependent frames included. This means there will
be about one key frame interval of latency (typically about a second).
This seems hard to avoid, as mentioned in issue #59.
2019-01-21 15:58:52 -08:00
Scott Lamb
95a8c2e78d support .mp4 files > 13.25 hours
Use version 1 of the mvhd, tkhd, and mdhd boxes to support 64-bit
durations. 2^32 units / 90,000 units/sec / 60 sec/min / 60 min/hr ~=
13.25 hrs.

Compatibility: looks like Chrome, Firefox, VLC, and ffmepg all support
version 1 with no problem.
2019-01-07 00:59:32 -08:00
Scott Lamb
de643f9f8d include segments in debug output 2018-12-29 13:15:01 -06:00
Scott Lamb
eb8a51aecb add a url for getting debug info about a .mp4 file
and add a unit test of path decoding along the way
2018-12-29 13:09:16 -06:00
Scott Lamb
b5387af3d4 lose "extern crate" everywhere (Rust 2018 edition) 2018-12-28 21:59:39 -06:00
Scott Lamb
f5703b9968 introduce typed errors and use in mp4 code
Fixes #46. If there are no video_sample_entries, it returns
InvalidArgument, which gets mapped to a HTTP 400. Various other failures
turn into non-500s as well.

There are many places that can & should be using typed errors, but it's
a start.
2018-12-28 17:30:33 -06:00
Scott Lamb
699ec87968 upgrade to 2018 Rust edition
This is mostly just "cargo fix --edition" + Cargo.toml changes.
There's one fix for upgrading to NLL in db/writer.rs:
Writer::previously_opened wouldn't build with NLL because of a
double-borrow the previous borrow checker somehow didn't catch.
Restructure to avoid it.

I'll put elective NLL changes in a following commit.
2018-12-28 14:59:06 -06:00
Scott Lamb
3644548018 fix some outdated comments in slices.rs 2018-12-28 10:04:47 -06:00
Scott Lamb
89fa35a2f7 be slightly more graceful on bad /view.mp4 (#46)
Before, this would panic from the reactor thread. After, it returns a
internal server error. Still not ideal, but better.

To return "bad request" as it should, mp4::FileBuilder::build() should
return a new error type that distinguishes "invalid argument" from
"internal" and the like. I'm thinking of using a ErrorKind enum
throughout the program that's similar to grpc::StatusCode.
2018-12-28 09:01:47 -06:00
Scott Lamb
4580038013 fix --require-auth flag
Apparently with docopt, --require-auth=false doesn't work, so booleans
with a default value of true can't be turned off. Toggle the default to
false to deal with this, for now. I'd prefer the default be true, but
I also would prefer to not use a negative --no-require-auth or
--allow-unauthenticated flag. I think I'll switch from docopt to clap
in the near future; it seems to be what the cool kids use.
2018-12-28 08:39:50 -06:00
Scott Lamb
24674f5b50 document proxy setup in guide/secure.md (for #26)
The guide is not as quick to follow and amateur-friendly as I'd like. A
few things that might improve matters:

   * complete #27 (built-in https+letsencrypt), so that when not sharing
     the port, users don't need to use nginx or certbot.
   * more ubiquitous IPv6 (out of my control but should happen over
     time) to reduce need to share the port
   * embed a dynamic DNS client
   * support UPnP Internet Gateway Device Control Protocol (if common
     routers have this enabled? probably not for security reasons.)

It's progress, though. Enough that I think I'll merge the auth branch
into master shortly.
2018-12-27 16:00:15 -06:00
Scott Lamb
3c1163dfe2 use SameSite=Strict (for #26)
I initially chose SameSite=Lax because I thought if a user followed a
link to the landing page, the landing page's ajax requests wouldn't send
the cookie. But I just did an experiment, and that's not true. Only the
initial page load (of a .html file) lacks the cookie. All of its
resources and ajax requests send the cookie. I'm not sure about
document.cookie accesses, but my cookie is HttpOnly anyway, so it's
irrelevant. So no reason to be lax.
2018-12-01 22:04:54 -08:00
Scott Lamb
3f76096a81 add Cache-Control: private hdr to non-static stuff
for #26
2018-12-01 16:55:10 -08:00
Scott Lamb
4f87c16c31 Merge branch 'master' into auth 2018-12-01 15:27:54 -08:00
Scott Lamb
35e6891221 update all Rust deps 2018-12-01 15:20:19 -08:00
Scott Lamb
087fdafc61 endpoint to debug --trust-forward-hdrs for #26 2018-12-01 00:44:19 -08:00
Scott Lamb
7a81d36562 support proxy forwarded headers
I went with legacy headers (X-Real-IP, X-Forwarded-Proto) because they
appear to be more widely supported than the RFC 7239 Forwarded header.
2018-11-28 14:49:56 -08:00
Scott Lamb
422cd2a75e preliminary web support for auth (#26)
Some caveats:

  * it doesn't record the peer IP yet, which makes it harder to verify
    sessions are valid. This is a little annoying to do in hyper now
    (see hyperium/hyper#1410). The direct peer might not be what we want
    right now anyway because there's no TLS support yet (see #27).  In
    the meantime, the sane way to expose Moonfire NVR to the Internet is
    via a proxy server, and recording the proxy's IP is not useful.
    Maybe better to interpret a RFC 7239 Forwarded header (and/or
    the older X-Forwarded-{For,Proto} headers).

  * it doesn't ever use Secure (https-only) cookies, for a similar reason.
    It's not safe to use even with a tls proxy until this is fixed.

  * there's no "moonfire-nvr config" support for inspecting/invalidating
    sessions yet.

  * in debug builds, logging in is crazy slow. See libpasta/libpasta#9.

Some notes:

  * I removed the Javascript "no-use-before-defined" lint, as some of
    the functions form a cycle.

  * Fixed #20 along the way. I needed to add support for properly
    returning non-OK HTTP statuses to signal unauthorized and such.

  * I removed the Access-Control-Allow-Origin header support, which was
    at odds with the "SameSite=lax" in the cookie header. The "yarn
    start" method for running a local proxy server accomplishes the same
    thing as the Access-Control-Allow-Origin support in a more secure
    manner.
2018-11-27 11:08:33 -08:00
Scott Lamb
61af963a64 Merge branch 'master' into auth 2018-11-20 11:10:47 -08:00
Scott Lamb
071be03c6f update most deps, notably including reqwest
Fixes #60

The reqwest dependency is significant because the old version required
an old version of openssl, complicating compilation on newer platforms.
reqwest also pulled in old/duplicate versions of hyper, tokio, etc.
Nice to drop a lot of that cruft.

I left rusqlite and uuid alone because they had breaking changes I
didn't want to mess with at the moment.

Bumped the minimum Rust version to 1.30.0, as required by the
new encoding_rs crate (and perhaps other things).
2018-11-20 09:32:55 -08:00
Scott Lamb
8a5056b253 "moonfire-nvr config" support for users (for #26) 2018-11-02 07:15:48 -07:00
Scott Lamb
aa81eae65a more robust timezone detection (fixes #12) 2018-08-31 17:19:24 -07:00
Scott Lamb
955a0a8c15 upgrade to hyper 0.12.x
Just one (intentional) functional change---now the streamers start
shutting down while the webserver shuts down gracefully.
2018-08-29 22:26:19 -07:00
Scott Lamb
a10e77d98e update cursive from 0.7 to 0.9 2018-08-24 22:14:03 -07:00
Scott Lamb
8dc5d64333 make with_recording_playback less monomorphized
This is a minor code size reduction - instead of being monomorphized
into four variants (according to "cargo llvm-lines"), it's now
monomorphized into two. The stripped release binary on macOS is about
8kB smaller (0.15%). Not a huge improvement but better than nothing.

Benchmarks seem unchanged (though they have a lot of variance).
2018-08-24 15:34:42 -07:00
Scott Lamb
b0071515e0 update deps
I want to use hyper::server::Request::bytes_mut(), so an update is
needed. Update everything at once. Most notably, the http-serve update
starts using the http crate types for some things. (More to come.)
2018-04-06 15:54:52 -07:00
Scott Lamb
f720f6acd4 fix timezone fetching on macOS High Sierra 2018-04-06 13:49:18 -07:00
Scott Lamb
97d831e054 move strutil to base crate
I plan to use strutil::hex in db/auth.rs.
2018-03-30 08:54:20 -07:00
Scott Lamb
299c0b1802 Merge branch 'master' (early part) into new-schema
Catch the new-schema branch up with everything up to (but not including) the
big UI refactoring. I'll merge that separately.
2018-03-24 22:29:40 -07:00
Scott Lamb
91636d3193 refine flush_if_sec behavior
The new behavior eliminates a couple unpleasant edge cases in which it
would never flush:

* if all recording stops, whatever was unflushed would stay that way
* if every recording attempt produces a 0-duration recording (such as if the
  camera sends only one frame and thus no PTS delta can be calculated),
  the list of recordings to flush would continue to grow
2018-03-23 15:16:43 -07:00
Scott Lamb
addeb9d2f6 add a TimerGuard around db locks & ops
I moved the clocks member from LockedDatabase to Database to make this happen,
so the new DatabaseGuard (replacing a direct MutexGuard<LockedDatabase>) can
access it before acquiring the lock.

I also made the type of clock a type parameter of Database (and so several
other things throughout the system). This allowed me to drop the Arc<>, but
more importantly it means that the Clocks trait doesn't need to stay
object-safe. I plan to take advantage of that shortly.
2018-03-23 13:31:23 -07:00
Scott Lamb
320374c6e9 favicon: security camera in orange/red circle
This is derived from the following icon:
https://thenounproject.com/term/security-camera/72530
by iconsmind.com. I paid for the royalty-free license.

Fixes #50.
2018-03-12 22:47:44 -07:00
Scott Lamb
4c8daa6d24 save timestamps along with opens 2018-03-10 16:15:36 -08:00
Scott Lamb
5854b31b76 serve .map files as Javascript
This fixes #47 for now at least.
2018-03-10 16:04:37 -08:00
Scott Lamb
d6fa470713 tests and fixes for Writer and Syncer
* separate these out into a new file, writer.rs, as dir.rs was getting
  unwieldy.
* extract traits for the parts of SampleFileDir and std::fs::File they needed;
  set up mock implementations.
* move clock.rs to a new base crate to be accessible from the db crate.
* add tests that exercise all the retry paths.
* bugfix: account for the new recording's bytes when calculating how much to
  delete.
* bugfix: when retrying an unlink failure in collect_garbage, we shouldn't
  warn about all the recordings no longer existing. Do this by retrying each
  step rather than the whole procedure again.
* avoid double-panic scenarios, which I hit while tweaking the mocks. These
  are quite annoying to debug as Rust doesn't print information about either
  panic. I ended up using lldb to get a backtrace. Better to be cautious about
  what we're doing when already panicking.
* give more context on raw::insert_recording errors, which I hit as well while
  tweaking the new tests.
2018-03-07 04:42:46 -08:00
Scott Lamb
672a327ee2
support serving Access-Control-Allow-Origin header (#19)
support serving Access-Control-Allow-Origin header

Closes #17.
2018-03-03 06:43:36 -08:00
Scott Lamb
b78ffc3808 view in-progress recordings!
The time from recorded to viewable was previously 60-120 sec for the first
recording of a RTSP session, 0-60 sec otherwise. Now it's one frame.
2018-03-02 15:40:32 -08:00