Commit Graph

527 Commits

Author SHA1 Message Date
Scott Lamb
b5387af3d4 lose "extern crate" everywhere (Rust 2018 edition) 2018-12-28 21:59:39 -06:00
Scott Lamb
f5703b9968 introduce typed errors and use in mp4 code
Fixes #46. If there are no video_sample_entries, it returns
InvalidArgument, which gets mapped to a HTTP 400. Various other failures
turn into non-500s as well.

There are many places that can & should be using typed errors, but it's
a start.
2018-12-28 17:30:33 -06:00
Scott Lamb
0b0f4ec9ed NLL-inspired simplifications to db.rs
* remove intermediate bool from adjust_day.

* rewrite LockedDatabase::list_aggregate_recordings.
  I started by collapsing the flush into the first part of the if, in a
  similar way to adjust_day. But then I refactored more and ended up
  with a structure that probably would have been allowed with the old
  lexical borrow checker. I think it's more readable, and it does 1
  btree operation per row where before it did 2 or 3.
2018-12-28 15:10:12 -06:00
Scott Lamb
699ec87968 upgrade to 2018 Rust edition
This is mostly just "cargo fix --edition" + Cargo.toml changes.
There's one fix for upgrading to NLL in db/writer.rs:
Writer::previously_opened wouldn't build with NLL because of a
double-borrow the previous borrow checker somehow didn't catch.
Restructure to avoid it.

I'll put elective NLL changes in a following commit.
2018-12-28 14:59:06 -06:00
Scott Lamb
ff58f24785 update deps 2018-12-28 10:13:03 -06:00
Scott Lamb
3644548018 fix some outdated comments in slices.rs 2018-12-28 10:04:47 -06:00
Scott Lamb
89fa35a2f7 be slightly more graceful on bad /view.mp4 (#46)
Before, this would panic from the reactor thread. After, it returns a
internal server error. Still not ideal, but better.

To return "bad request" as it should, mp4::FileBuilder::build() should
return a new error type that distinguishes "invalid argument" from
"internal" and the like. I'm thinking of using a ErrorKind enum
throughout the program that's similar to grpc::StatusCode.
2018-12-28 09:01:47 -06:00
Scott Lamb
4580038013 fix --require-auth flag
Apparently with docopt, --require-auth=false doesn't work, so booleans
with a default value of true can't be turned off. Toggle the default to
false to deal with this, for now. I'd prefer the default be true, but
I also would prefer to not use a negative --no-require-auth or
--allow-unauthenticated flag. I think I'll switch from docopt to clap
in the near future; it seems to be what the cool kids use.
2018-12-28 08:39:50 -06:00
Scott Lamb
54ebc6ec2f install the tzdata package from scripts
Fixes #53
2018-12-27 16:34:35 -06:00
Scott Lamb
278a87d5fd tweaks to guide/secure.md
after reading the rendered version online
2018-12-27 16:29:26 -06:00
Scott Lamb
24674f5b50 document proxy setup in guide/secure.md (for #26)
The guide is not as quick to follow and amateur-friendly as I'd like. A
few things that might improve matters:

   * complete #27 (built-in https+letsencrypt), so that when not sharing
     the port, users don't need to use nginx or certbot.
   * more ubiquitous IPv6 (out of my control but should happen over
     time) to reduce need to share the port
   * embed a dynamic DNS client
   * support UPnP Internet Gateway Device Control Protocol (if common
     routers have this enabled? probably not for security reasons.)

It's progress, though. Enough that I think I'll merge the auth branch
into master shortly.
2018-12-27 16:00:15 -06:00
Scott Lamb
3c1163dfe2 use SameSite=Strict (for #26)
I initially chose SameSite=Lax because I thought if a user followed a
link to the landing page, the landing page's ajax requests wouldn't send
the cookie. But I just did an experiment, and that's not true. Only the
initial page load (of a .html file) lacks the cookie. All of its
resources and ajax requests send the cookie. I'm not sure about
document.cookie accesses, but my cookie is HttpOnly anyway, so it's
irrelevant. So no reason to be lax.
2018-12-01 22:04:54 -08:00
Scott Lamb
3f76096a81 add Cache-Control: private hdr to non-static stuff
for #26
2018-12-01 16:55:10 -08:00
Scott Lamb
4f87c16c31 Merge branch 'master' into auth 2018-12-01 15:27:54 -08:00
Scott Lamb
35e6891221 update all Rust deps 2018-12-01 15:20:19 -08:00
Scott Lamb
087fdafc61 endpoint to debug --trust-forward-hdrs for #26 2018-12-01 00:44:19 -08:00
Scott Lamb
d35a4592e3 Merge branch 'master' into auth 2018-12-01 00:06:43 -08:00
Scott Lamb
b2bdccb507 Javascript fix for unauthenticated case
newTimeFormat didn't handle newTimeZone not having been called well.
Restore the prior behavior of having called newTimeZone(null), which was
apparently good enough.
2018-12-01 00:04:43 -08:00
Scott Lamb
131c5e0640 Fix "no garbage row for <id>" flush failure loops
Add some comments along the way.

Fixes #63.
2018-12-01 00:03:43 -08:00
Scott Lamb
7a81d36562 support proxy forwarded headers
I went with legacy headers (X-Real-IP, X-Forwarded-Proto) because they
appear to be more widely supported than the RFC 7239 Forwarded header.
2018-11-28 14:49:56 -08:00
Scott Lamb
4daf618c29 fix a couple compile errors in 422cd2a
I just ran a "cargo test" on this after a round of tweaks, not
"cargo test --all", so I missed compile errors in the db crate,
and a Javascript lint config error. travis-ci caught these.
2018-11-27 12:23:44 -08:00
Scott Lamb
422cd2a75e preliminary web support for auth (#26)
Some caveats:

  * it doesn't record the peer IP yet, which makes it harder to verify
    sessions are valid. This is a little annoying to do in hyper now
    (see hyperium/hyper#1410). The direct peer might not be what we want
    right now anyway because there's no TLS support yet (see #27).  In
    the meantime, the sane way to expose Moonfire NVR to the Internet is
    via a proxy server, and recording the proxy's IP is not useful.
    Maybe better to interpret a RFC 7239 Forwarded header (and/or
    the older X-Forwarded-{For,Proto} headers).

  * it doesn't ever use Secure (https-only) cookies, for a similar reason.
    It's not safe to use even with a tls proxy until this is fixed.

  * there's no "moonfire-nvr config" support for inspecting/invalidating
    sessions yet.

  * in debug builds, logging in is crazy slow. See libpasta/libpasta#9.

Some notes:

  * I removed the Javascript "no-use-before-defined" lint, as some of
    the functions form a cycle.

  * Fixed #20 along the way. I needed to add support for properly
    returning non-OK HTTP statuses to signal unauthorized and such.

  * I removed the Access-Control-Allow-Origin header support, which was
    at odds with the "SameSite=lax" in the cookie header. The "yarn
    start" method for running a local proxy server accomplishes the same
    thing as the Access-Control-Allow-Origin support in a more secure
    manner.
2018-11-27 11:08:33 -08:00
Scott Lamb
679370c77a Merge branch 'master' into auth 2018-11-20 11:26:26 -08:00
Scott Lamb
61af963a64 Merge branch 'master' into auth 2018-11-20 11:10:47 -08:00
Scott Lamb
496f7d7e3a upgrade some JS deps to work with node 11
Fixes #62

* added travis config for latest node as well as 8.

* ran "yarn upgrade -P webpack-dev-server", which caused the upath
  dependency to be upgraded. I arrived at this by inspecting yarn.lock
  for the things depending on upack, along with some trial and error.
  ("yarn upgrade -P chokidar" was less successful.)
2018-11-20 11:06:20 -08:00
Scott Lamb
071be03c6f update most deps, notably including reqwest
Fixes #60

The reqwest dependency is significant because the old version required
an old version of openssl, complicating compilation on newer platforms.
reqwest also pulled in old/duplicate versions of hyper, tokio, etc.
Nice to drop a lot of that cruft.

I left rusqlite and uuid alone because they had breaking changes I
didn't want to mess with at the moment.

Bumped the minimum Rust version to 1.30.0, as required by the
new encoding_rs crate (and perhaps other things).
2018-11-20 09:32:55 -08:00
Scott Lamb
8a5056b253 "moonfire-nvr config" support for users (for #26) 2018-11-02 07:15:48 -07:00
Scott Lamb
f9d4b5bb8a fix accidental dependency on rust 1.30.0
travis-ci's 1.27.0 build failed with:

error[E0658]: access to extern crates through prelude is experimental (see issue #44660)
   --> db/auth.rs:159:6
    |
159 | impl rusqlite::types::FromSql for FromSqlIpAddr {
    |      ^^^^^^^^
2018-11-02 07:02:08 -07:00
Scott Lamb
75f233da79 initial db layer work for authentication (#26) 2018-11-01 23:25:06 -07:00
Scott Lamb
aa81eae65a more robust timezone detection (fixes #12) 2018-08-31 17:19:24 -07:00
Scott Lamb
fc0bc51bed build and lint the UI in travis-ci
This is a separate item in the matrix, so it doesn't go through this for
each of the three Rust versions.
2018-08-31 08:19:56 -07:00
Scott Lamb
ee3da33470 fix trailing whitespace in Javascript
I want to make travis-ci enforce that everything passes lint.
Get it in a good state first.
2018-08-31 07:31:22 -07:00
Scott Lamb
8adf6f4bc7 extend cache timeout
300 is insufficient for nightly, perhaps due to the extra weight of
the optimized build for the benchmark.
2018-08-31 00:01:30 -07:00
Scott Lamb
d7e0fcc3ba cache ffmpeg build in travis-ci
Hopefully this will significantly speed up builds.
2018-08-30 23:10:12 -07:00
Scott Lamb
955a0a8c15 upgrade to hyper 0.12.x
Just one (intentional) functional change---now the streamers start
shutting down while the webserver shuts down gracefully.
2018-08-29 22:26:19 -07:00
Scott Lamb
91ef07b9a7 bump required rust to 1.27
1.26 doesn't work with the updated rusqlite:

error[E0658]: use of unstable library feature 'duration_extras' (see issue #46507)
  --> /home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/rusqlite-0.14.0/src/busy.rs:26:49
   |
26 |             .and_then(|t| t.checked_add(timeout.subsec_millis().into()))
   |                                                 ^^^^^^^^^^^^^
2018-08-28 21:47:02 -07:00
Scott Lamb
0cc4d191c0 bump minimum Rust version to 1.26
1.25 also fails with the upgraded reffers because u128 isn't stable:

error[E0658]: 128-bit type is unstable (see issue #35118)
   --> /home/travis/.cargo/git/checkouts/reffers-rs-0d00fc7f893338b3/49a4d75/src/rc_bitmask.rs:194:34
    |
194 | rc_bit_mask_internal!(primitive, u128, 42, 42, 42);
    |                                  ^^^^
2018-08-27 21:32:51 -07:00
Scott Lamb
6ab416caed bump minimum Rust version
travis-ci pointed out that building with 1.21 broke with a recent dep
upgrade (8c52c36). reffers now uses nested groups of imports, which is a
feature introduced with Rust 1.25. Prior to 1.25, it fails as follows:

error: expected one of `,` or `as`, found `::`
 --> /home/travis/.cargo/git/checkouts/reffers-rs-0d00fc7f893338b3/49a4d75/src/arc.rs:6:46
  |
6 | use std::sync::atomic::{AtomicUsize, Ordering::SeqCst};
  |                                              ^^ expected one of `,` or `as` here
2018-08-25 06:33:50 -07:00
Scott Lamb
a10e77d98e update cursive from 0.7 to 0.9 2018-08-24 22:14:03 -07:00
Scott Lamb
8c52c36b51 upgrade a few deps 2018-08-24 22:06:14 -07:00
Scott Lamb
5bba71345c few small markdown tweaks 2018-08-24 21:04:13 -07:00
Scott Lamb
489fda6abe add missing guide/install-scripted.md
I mistakenly lost a "git mv easy-install.md install-scripted.md" when
reworking the commit, I suppose.
2018-08-24 20:59:56 -07:00
Scott Lamb
c5345c1e11 simplify and fix installation instructions
* install.md, install-manual.md, and easy-install.md had a lot of
  redundancy. Rework them so the common prefix and suffix are in
  install.md and it's clear when to navigate back and forth. This
  removes from very stale references to prep.sh and cameras.sql in
  install-manual.md (which never should have mentioned these scripts
  anyway).

* remove all the SAMPLE_MEDIA_DIR, SAMPLE_FILE_DIR, and
  SAMPLE_FILE_PATH stuff from the scripts. This was too complicated
  (one variable will suffice) and inconsistent in terminology (a
  couple "samples dir" occurrences slipped through review; they
  should have been "sample file dir"). It also wasn't really useful
  enough because the procedure for a mount point is manual anyway,
  and because some installs will have multiple sample file dirs
  anyway.

* in the mount point procedure, fix the paths to be consistent. Also
  describe the "nofail" and "Requires=" config I have on my machine.

* fix some incorrect info about how to use "moonfire-nvr config" and
  describe "flush_if_sec".
2018-08-24 20:45:46 -07:00
Scott Lamb
8dc5d64333 make with_recording_playback less monomorphized
This is a minor code size reduction - instead of being monomorphized
into four variants (according to "cargo llvm-lines"), it's now
monomorphized into two. The stripped release binary on macOS is about
8kB smaller (0.15%). Not a huge improvement but better than nothing.

Benchmarks seem unchanged (though they have a lot of variance).
2018-08-24 15:34:42 -07:00
Scott Lamb
d7a94956eb deflake writer tests
There was a race condition here because it wasn't waiting for the db
flush to complete. This made write_path_retries sometimes not reflect
the consequence of the flush, causing an assertion failure. I assume it
was also responsible for gc_path_retries timeouts under travis-ci.
2018-08-07 21:58:40 -05:00
Scott Lamb
f3127f563a fifth attempt at travis config
typo: it's --jobs=2, not -jobs=2.
2018-08-07 16:01:43 -05:00
Scott Lamb
01e8dab67f fourth attempt at travis config
* upgrade min required rust to 1.21; crossbeam-deque requires the
  ord_max_min feature, apparently stabilized in this version.

* use "make --jobs=2" to build ffmpeg so it goes faster.
  https://docs.travis-ci.com/user/reference/overview/ says there are 2
  cores available.
2018-08-07 15:51:19 -05:00
Scott Lamb
62a1bad8b9 third attempt at travis config
* upgrade minimum required Rust from 1.17 to 1.20; reffers 0.4.2
  apparently uses std::mem::ManuallyDrop, introduced in 1.20

* install ffmpeg from source (requiring sudo access) rather than using
  the ancient one from Ubuntu Trusty to meet the minimum version
  requirements specified in ffmpeg/build.rs.
2018-08-07 15:28:34 -05:00
Scott Lamb
a81de3f620 second attempt at travis config
* list apt packages to install
* apparently it's rust 1.17.0, not rust 1.17
2018-05-01 08:38:25 -07:00
Scott Lamb
7bb33522e2 first attempt at a travis config
This just does a build+test. I'll try the install scripts (for #58) in a
following commit.
2018-05-01 08:28:14 -07:00