Commit Graph

37 Commits

Author SHA1 Message Date
Scott Lamb
6b5359b7cb use SQLite3 "extra synchronous" pragma
See
https://github.com/scottlamb/moonfire-nvr/issues/84#issuecomment-640862000
https://www.sqlite.org/pragma.html#pragma_synchronous
2020-06-08 13:35:45 -07:00
Scott Lamb
e8eb764b90 switch from docopt to structopt
A couple reasons for this:

* the docopt crate is "unlikely to see significant future evolution",
  and the wider docopt project is "mostly unmaintained at this point".
  clap/structopt is more full-featured, has more natural subcommand
  support, etc.

* it may allow me to shrink the binary (#70). This change alone seems
  to be a slight regression, but it's a step toward getting rid of
  regex, which is pretty large. And I feel less ridiculous now that I
  don't have two parsing crates anyway; prettydiff was pulling in
  structopt.

There are some behavior changes here:

* misc --help output changes and such as you'd expect from switching
  argument-parsing libraries

* I properly used PathBuf and OsString for stuff that theoretically
  could be non-UTF-8. I haven't tested that it actually made any
  difference. I'm also still storing the sample file dirname as "text"
  in the database to avoid causing a diff when not doing a schema
  change.
2020-04-17 21:53:37 -07:00
Scott Lamb
066c086050 style: use rusqlite's {named_,}params! everywhere 2020-03-19 20:46:25 -07:00
Scott Lamb
317a620e6e upgrade copyright notices
* As discussed in #48, say "The Moonfire NVR Authors" at the top of
  every file rather than whoever created that file. Have one AUTHORS
  file listing everyone.
* Consistently call it a "security camera network video recorder" rather
  than "security camera digital video recorder".
2020-03-01 22:53:41 -08:00
Scott Lamb
fce0c5b014 use a released version of nix 2019-12-30 07:46:53 -06:00
Scott Lamb
2bd8963961 garbage collection for signals 2019-12-28 07:48:08 -06:00
Scott Lamb
54e06a5326 clean up old garbage files on v5 upgrade 2019-07-24 22:18:44 -07:00
Scott Lamb
433be217ac improve test of upgrade from v0 on up; fix bugs
Now the test actually has a recording and garbage with matching files.
This caught a few problems in the upgrade procedure:

* it didn't work with foreign keys enabled because the new recording
  table was set up after the new camera table, and the old recording
  table was destroyed after the old camera table. And now I enable
  foreign keys all the time. Reorder the procedure to fix.

* the pathname manipulation in the v2 to v3 procedure was incorrect
  since my introduction of nix because I gave it a &[u8] with the
  trailing nul, where I should have used CStr::from_bytes_with_nul.

* it wasn't removing garbage files. It'd be most natural to do this
  in the v2 to v3 upgrade (with the rename) but I historically removed
  the table when upgrading to v2. I can't redefine the schema now, so
  do it unnaturally.

  I'm considering also renaming all uuid-like files on upgrade to v4/v5
  to clean up this mess automatically for installations that have
  already done this upgrade.
2019-07-21 22:49:50 -07:00
Scott Lamb
18c693fa46 update deps
The immediate motivation is that Cargo.lock referred to a commit version
in a PR branch of my nix fork that no longer exists. (I didn't know, but
it makes sense, that "git push -f" not only forcibly updates the branch
to refer to a new commit but also gets rid of orphaned commits.) Use a
moonfire branch that I'll keep stable until I'm ready to move on.

I also updated parking_lot and rusqlite to new major versions (nothing
in the interface that I care about has changed) and did a full cargo
update.
2019-07-17 14:32:09 -07:00
Scott Lamb
e52e725958 s/std::fs::read_dir/nix::dir::Dir/ in a few spots
This is nicer in a few ways:

   * I can use openat so there's no possibility of any kind of a race
     involving scanning a different directory than the one used in
     other ways (locking, metadata file, adding/removing sample files)
   * filename() doesn't need to allocate memory, so it's a bit more
     efficient
   * dogfooding - I wrote nix::dir.
2019-07-12 11:07:14 -07:00
Scott Lamb
bb227491b6 use nix to remove many uses of unsafe 2019-07-11 21:59:01 -07:00
Scott Lamb
d7a918d397 schema comparison in new upgrade tests, "moonfire-nvr check"
The .sql files here are copied from earlier revisions:

v0.sql  fee4141:src/schema.sql
v1.sql  0d69f4f:src/schema.sql
v3.sql  422cd2a:db/schema.sql
2019-07-11 13:31:33 -07:00
Scott Lamb
8159acf703 Merge branch 'master' into new-schema 2019-07-10 01:11:25 -07:00
Scott Lamb
f80c99a43f set sqlite3 options for full durability 2019-07-10 01:09:52 -07:00
Scott Lamb
d61b5e1bdd Use fixed-size directory meta files
Add a new schema version 5; now 4 means the directory meta may or may
not be upgraded.

Fixes #65: now it's possible to open the directory even if it lies on a
completely full disk.
2019-07-04 23:30:37 -05:00
Scott Lamb
06d7815f9c fix upgrade procedure with newer SQLite
Newer SQLite library versions (such as what you get when using
--features=bundled) actually enforce foreign keys. Unfortunately there's
no way to drop foreign key constraints, so you have to transitively
recreate all the tables with foreign key constraints on the table you're
recreating.
2019-07-01 00:33:36 -05:00
Scott Lamb
a9f64798d6 store full rtsp urls
My dad's "GW-GW4089IP" cameras use separate ports for the main and sub
streams:

rtsp://192.168.1.110:5050/H264?channel=0&subtype=0&unicast=true&proto=Onvif
rtsp://192.168.1.110:5049/H264?channel=0&subtype=1&unicast=true&proto=Onvif

Previously I could get one of the streams to work by including :5050 or
:5049 in the host field of the camera. But not both. Now make the
camera's host field reflect the ONVIF port (which is also non-standard
on these cameras, :85). It's not directly used yet but probably will be
sooner or later. Make each stream know its full URL.
2019-06-30 23:54:52 -05:00
Scott Lamb
eaf6608597 tweak and document version 3->4 upgrade 2019-06-20 16:03:59 -07:00
Scott Lamb
fda7e4ca2b add concept of user/session permissions
(I also considered the names "capabilities" and "scopes", but I think
"permissions" is the most widely understood.)

This is increasingly necessary as the web API becomes more capable.
Among other things, it allows:

* non-administrator users who can view but not access camera passwords
  or change any state
* workers that update signal state based on cameras' built-in motion
  detection or a security system's events but don't need to view videos
* control over what can be done without authenticating

Currently session permissions are just copied from user permissions, but
you can also imagine admin sessions vs not, as a checkbox when signing
in. This would match the standard Unix workflow of using a
non-administrative session most of the time.

Relevant to my current signals work (#28) and to the addition of an
administrative API (#35, including #66).
2019-06-19 15:34:20 -07:00
Scott Lamb
7fe9d34655 cargo fix --all
* it added "dyn" to trait objects
* it changed "..." in patterns to "..="

cargo --version says: "cargo 1.37.0-nightly (545f35425 2019-05-23)"
2019-06-14 08:47:11 -07:00
Scott Lamb
7dd98bb76a db crate support for updating signals (#28)
This is a definite work in progress. In particular,

* there's no src/web.rs support yet so it can't be used,
* the code is surprisingly complex, and there's almost no tests so far.
  I want to at least get complete branch coverage.
* I may still go back to time_sec rather than time_90k to save RAM and
  flash.

I simplified the approach a bit from the earlier goal in design/api.md.
In particular, there's no longer the separate concept of "observation"
vs "prediction". Now the predictions are just observations that extend a
bit beyond now. They may be flushed prematurely and I'll try living with
that to avoid making things even more complex.
2019-06-13 22:25:55 -07:00
Scott Lamb
6f2c63ffac read-only signals support (#28)
This is mostly untested and useless by itself, but it's a starting
point. In particular:

* there's no way to set up signals or add/remove/update events yet
  except by manual changes to the database.
* if you associate a signal with a camera then remove the camera,
  hitting /api/ will error out.
2019-06-06 16:20:44 -07:00
Scott Lamb
b629fe6ac1 upgrade rusqlite, bump required Rust to 1.33
The new rusqlite requires the transpose_result feature, stabilized in
this Rust version.
2019-05-31 16:19:04 -07:00
Scott Lamb
b5387af3d4 lose "extern crate" everywhere (Rust 2018 edition) 2018-12-28 21:59:39 -06:00
Scott Lamb
699ec87968 upgrade to 2018 Rust edition
This is mostly just "cargo fix --edition" + Cargo.toml changes.
There's one fix for upgrading to NLL in db/writer.rs:
Writer::previously_opened wouldn't build with NLL because of a
double-borrow the previous borrow checker somehow didn't catch.
Restructure to avoid it.

I'll put elective NLL changes in a following commit.
2018-12-28 14:59:06 -06:00
Scott Lamb
35e6891221 update all Rust deps 2018-12-01 15:20:19 -08:00
Scott Lamb
9982c0b080 small adjustments to auth schema
Nothing uses the user and user_session tables yet; I'm trying to
anticipate what auth will need before freezing schema version 3.
2018-04-27 06:24:02 -07:00
Scott Lamb
0701121586 a couple refinements to the new user_session table 2018-03-25 07:23:40 -07:00
Scott Lamb
c0da1ef880 make v1->v3 upgrade work with --features=bundled
--features=bundled enables -DSQLITE_DEFAULT_FOREIGN_KEYS=1, and so some
operations have to be done in the proper order.

* enable foreign key enforcement all the time, so I test this more reliably.
* reorder some parts of the v1->v3 order. foreign key enforcement is
  immediate (rather than deferred) by default. and ensure
  old_recording_playback isn't left with a dangling reference to old_recording
  at the v2 stage. Instead, wait until v3 to delete tables it depends on.
2018-03-22 09:05:40 -07:00
Scott Lamb
c46c50af8f fix another upgrade error in dfee66c 2018-03-22 00:08:49 -07:00
Scott Lamb
2ff7ecb6f4 fix upgrade procedure broken in dfee66c 2018-03-22 00:00:39 -07:00
Scott Lamb
1c9f2a4d83 initial schema for user authentication (#26)
This is only the database schema, which I'm adding now in the hopes of
freezing schema version 3. There's no way yet to create users, much less
actually authenticate.
2018-03-21 23:57:45 -07:00
Scott Lamb
dfee66c84b support additional recording_integrity timestamps
These are not actually populated by the code yet. I'm trying to get the
v3 schema frozen as soon as possible; actually using the fields can come
later.

Add some explanation of their value in time.md, along with some general
musing on leap seconds, and a correction on the frequency error of my cameras.
2018-03-21 22:32:41 -07:00
Scott Lamb
4c8daa6d24 save timestamps along with opens 2018-03-10 16:15:36 -08:00
Scott Lamb
f81d699c8c new recording_integrity table
A couple rarely-used fields move to here, and I expect I'll add more.
Redo the check command to just put everything in RAM for simplicity.
2018-03-09 13:37:30 -08:00
Scott Lamb
f01f523c2c refine 1->3 upgrade process
In hindsight, the "post_tx" step in the upgrade process introduced in
e7f5733 doesn't make sense. If the procedure fails at this stage, nothing says
it still needs to be completed. If the sample file dirs have to be updated
after the database, then there should be another database version to mark that
it's fully completed, and indeed that's the purpose version 3 serves. So get
rid of the Upgrader trait and just go back to a simple run function per
version.

In the case of the sample file dir metadata, it actually can happen before the
database transaction; the stuff written to the database later just needs to be
consistent with what it finds if there's an existing metadata file from a
half-completed update.

For safety, ensure there are no unexpected directory contents before
upgrading 1->2, and ensure the metadata matches before upgrading 2->3.
2018-03-01 09:47:56 -08:00
Scott Lamb
bcf42fe02c move db upgrade logic into db crate
This allows shrinking db's API surface.
2018-02-28 21:21:47 -08:00