Commit Graph

384 Commits

Author SHA1 Message Date
Scott Lamb
d75157af56 fix d7a918d regression in SQLite required version
The pragma_* table-valued-functions require SQLite 3.16.0 (2017-01-02).
2019-07-11 14:00:09 -07:00
Scott Lamb
d7a918d397 schema comparison in new upgrade tests, "moonfire-nvr check"
The .sql files here are copied from earlier revisions:

v0.sql  fee4141:src/schema.sql
v1.sql  0d69f4f:src/schema.sql
v3.sql  422cd2a:db/schema.sql
2019-07-11 13:31:33 -07:00
Scott Lamb
f1112031c2 Merge branch 'master' into new-schema 2019-07-10 17:03:30 -07:00
Scott Lamb
856c01918b Update test to allow ffmpeg 4.x edit list behavior
Fixes #10
2019-07-10 17:02:45 -07:00
Scott Lamb
d2e18ca5e2 improve flush_if_sec docs 2019-07-10 15:17:55 -07:00
Scott Lamb
fb50617a7b add links to the wiki 2019-07-10 11:45:25 -07:00
Scott Lamb
34253fb96a add links to the wiki 2019-07-10 11:43:58 -07:00
Scott Lamb
106cf8cb4b Merge branch 'master' into new-schema 2019-07-10 02:03:13 -07:00
Scott Lamb
81d4fd67d4 suggest RequireMountsFor in systemd service file
Besides using one line instead of two, this avoids the need to do hex
escaping of characters like hyphens.
2019-07-10 02:00:10 -07:00
Scott Lamb
8159acf703 Merge branch 'master' into new-schema 2019-07-10 01:11:25 -07:00
Scott Lamb
f80c99a43f set sqlite3 options for full durability 2019-07-10 01:09:52 -07:00
Scott Lamb
71c64908ec Merge branch 'master' into new-schema 2019-07-10 00:59:42 -07:00
Scott Lamb
1c904b925a many improvements to install docs/procedures
* in markdown files, use code fences rather than indented blocks.
    This is harder to screw up (one of them was off by a space so didn't
    render properly) and allows me to add info strings.

  * uniformly use "useradd" to create the user and group in all three
    places (install-manual.md, script-functions.sh, Dockerfile) rather
    than addgroup + adduser. Create a full home dir, which I suspect was
    the problem in #67. Don't allow customizing group name; it's always
    the same as the user.

  * install the sqlite3 package so that the "moonfire-nvr sql" command
    works properly.

  * remove "setup_db" function, which was out of place. Since the
    creation of the "moonfire-nvr init" command, this has to happen
    after installation of the binary. install.md gives instructions on
    this part anyway so remove it from the script.

  * give a proper command to create the db dir. It was creating it
    within the current directory, not within /var/lib/moonfire-nvr.
    Don't bother creating sample directory; "moonfire-nvr config"
    will do this.

  * when setting owners on a newly created directory, use a single
    "install -d" command rather than "mkdir" + "chown".

  * address confusion about whether sample file dirs need to be
    precreated. (Only when Moonfire NVR doesn't have write permissions
    on the parent.)

  * always just install the packaged version of ffmpeg rather than
    building our own. This has been usable since Debian/Raspbian 9
    Stretch; Debian/Raspbian 10 Buster is out now so there's no excuse
    for still running Debian/Raspbian 8 Jessie.

  * don't chown the UI directory; it can be owned by root as with
    the binary.

  * in scripts/install.sh, don't enable/start the service yet. It hasn't
    been configured.
2019-07-10 00:56:43 -07:00
Scott Lamb
d61b5e1bdd Use fixed-size directory meta files
Add a new schema version 5; now 4 means the directory meta may or may
not be upgraded.

Fixes #65: now it's possible to open the directory even if it lies on a
completely full disk.
2019-07-04 23:30:37 -05:00
Scott Lamb
13b192949d use cstr crate rather than unsafe
This removes a few uses of unsafe, and it verifies statically that there
are no interior NUL bytes.
2019-07-04 16:51:38 -05:00
Scott Lamb
06d7815f9c fix upgrade procedure with newer SQLite
Newer SQLite library versions (such as what you get when using
--features=bundled) actually enforce foreign keys. Unfortunately there's
no way to drop foreign key constraints, so you have to transitively
recreate all the tables with foreign key constraints on the table you're
recreating.
2019-07-01 00:33:36 -05:00
Scott Lamb
866a8cc4fc fix "cargo test" errors and warnings in a9f6479 2019-07-01 00:06:22 -05:00
Scott Lamb
a9f64798d6 store full rtsp urls
My dad's "GW-GW4089IP" cameras use separate ports for the main and sub
streams:

rtsp://192.168.1.110:5050/H264?channel=0&subtype=0&unicast=true&proto=Onvif
rtsp://192.168.1.110:5049/H264?channel=0&subtype=1&unicast=true&proto=Onvif

Previously I could get one of the streams to work by including :5050 or
:5049 in the host field of the camera. But not both. Now make the
camera's host field reflect the ONVIF port (which is also non-standard
on these cameras, :85). It's not directly used yet but probably will be
sooner or later. Make each stream know its full URL.
2019-06-30 23:54:52 -05:00
Scott Lamb
afe693ef95 fix obsolete flag name 2019-06-20 16:22:46 -07:00
Scott Lamb
eaf6608597 tweak and document version 3->4 upgrade 2019-06-20 16:03:59 -07:00
Scott Lamb
4f25412c3f remove debug logs mistakenly added in 644ea4e6 2019-06-20 14:59:04 -07:00
Scott Lamb
abf4eba42d unbreak test --features=nightly 2019-06-20 14:57:13 -07:00
Scott Lamb
644ea4e6ea expose signal id in api
...and update api.md which described a different format than before or
after.
2019-06-20 12:10:23 -07:00
Scott Lamb
1c1a823759 fix db auth tests for domain being optional
These broke in 49a8e5c
2019-06-20 12:06:57 -07:00
Scott Lamb
49a8e5c5a1 Add "moonfire-nvr login username" command
This should be useful when creating sessions for robot users without
ever having to set a password for them.
2019-06-20 09:07:01 -07:00
Scott Lamb
004aa5d6ce extend "moonfire-nvr sql" to pass args to sqlite3 2019-06-19 21:15:25 -07:00
Scott Lamb
a6ebf2d10d add "sql" subcommand
This delegates to the "sqlite3" CLI but has a couple benefits over using
sqlite3 directly:

  * safer because it does the same locking as other moonfire-nvr invocations
  * more convenient because it takes the same argument format as other
    moonfire-nvr subcommands:
      * --db-dir rather than full path including /db suffix
      * has the --db-dir default value
      * --read-only rather than file:...?mode=ro

Use like "moonfire-nvr sql" or "moonfire-nvr sql --read-only".
2019-06-19 17:20:44 -07:00
Scott Lamb
80306064a3 add missing build.rs 2019-06-19 16:53:14 -07:00
Scott Lamb
a5ffe3ca2c bump minimum Rust version
The master version of rust-protobuf requires the TryFrom feature,
stabilized in 1.34.
2019-06-19 15:56:04 -07:00
Scott Lamb
fda7e4ca2b add concept of user/session permissions
(I also considered the names "capabilities" and "scopes", but I think
"permissions" is the most widely understood.)

This is increasingly necessary as the web API becomes more capable.
Among other things, it allows:

* non-administrator users who can view but not access camera passwords
  or change any state
* workers that update signal state based on cameras' built-in motion
  detection or a security system's events but don't need to view videos
* control over what can be done without authenticating

Currently session permissions are just copied from user permissions, but
you can also imagine admin sessions vs not, as a checkbox when signing
in. This would match the standard Unix workflow of using a
non-administrative session most of the time.

Relevant to my current signals work (#28) and to the addition of an
administrative API (#35, including #66).
2019-06-19 15:34:20 -07:00
Scott Lamb
d8b8d5d5e0 fix dumb typo bug in signal/state input validation
The effect was that it'd allow any state for signal 0, when it should
have allowed state 0 for any signal.
2019-06-14 16:41:18 -07:00
Scott Lamb
1312349ca1 stop using deprecated libc::int64_t
The cool kids just use i64 now.
2019-06-14 16:14:07 -07:00
Scott Lamb
6d4b06f7d2 web api glue for updating signals
This is very lightly tested, but it at least sometimes works.
2019-06-14 16:11:12 -07:00
Scott Lamb
7fe9d34655 cargo fix --all
* it added "dyn" to trait objects
* it changed "..." in patterns to "..="

cargo --version says: "cargo 1.37.0-nightly (545f35425 2019-05-23)"
2019-06-14 08:47:11 -07:00
Scott Lamb
7dd98bb76a db crate support for updating signals (#28)
This is a definite work in progress. In particular,

* there's no src/web.rs support yet so it can't be used,
* the code is surprisingly complex, and there's almost no tests so far.
  I want to at least get complete branch coverage.
* I may still go back to time_sec rather than time_90k to save RAM and
  flash.

I simplified the approach a bit from the earlier goal in design/api.md.
In particular, there's no longer the separate concept of "observation"
vs "prediction". Now the predictions are just observations that extend a
bit beyond now. They may be flushed prematurely and I'll try living with
that to avoid making things even more complex.
2019-06-13 22:25:55 -07:00
Scott Lamb
d232ca55fa document proposed API for updating signals (#28) 2019-06-07 10:19:38 -07:00
Scott Lamb
6f2c63ffac read-only signals support (#28)
This is mostly untested and useless by itself, but it's a starting
point. In particular:

* there's no way to set up signals or add/remove/update events yet
  except by manual changes to the database.
* if you associate a signal with a camera then remove the camera,
  hitting /api/ will error out.
2019-06-06 16:20:44 -07:00
Scott Lamb
cb1bb5d810 upgrade cursive to latest major version 2019-05-31 16:35:07 -07:00
Scott Lamb
225e1fd75b upgrade db's parking_lot dependency too 2019-05-31 16:21:10 -07:00
Scott Lamb
b629fe6ac1 upgrade rusqlite, bump required Rust to 1.33
The new rusqlite requires the transpose_result feature, stabilized in
this Rust version.
2019-05-31 16:19:04 -07:00
Scott Lamb
3668c69d4b bump required Rust version to 1.32
travis-ci pointed out that the dependency bump broke 1.31:

   Compiling docopt v1.1.0
error[E0658]: imports can only refer to extern crate names passed with `--extern` on stable channel (see issue #53130)
   --> /home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/docopt-1.1.0/src/parse.rs:48:5
    |
48  |   use regex;
    |       ^^^^^
    |

Looks like uniform_paths was stabilized in 1.32, and I verified locally that
version builds.
2019-05-31 16:09:42 -07:00
Scott Lamb
428f5a3ba4 update a few deps
cursive & rusqlite are more significant; I'll do those separately
2019-05-31 15:08:49 -07:00
Scott Lamb
6f0c8c19de honor removing a time constraint
Looks like a bug got introduced with the great UI rewrite: when you add
a (start or end) time constraint, then remove one, the change wouldn't
be reflected. Within CalendarTSRange, it used null to mean to keep a
value, and || to check if it was null. These meant empty strings turned
into the existing value, instead of no constraint as they should be.
This was unnecessarily clever; stop doing that.

Also keep the console logging in the deployed config; it's harmless and
eases debugging.
2019-02-19 13:42:26 -08:00
Scott Lamb
36f3bda9c6 update deps 2019-02-13 22:43:30 -08:00
Scott Lamb
579150c9d5 redact URLs within stream.rs; fixes #13 2019-02-13 22:34:19 -08:00
Scott Lamb
d7a0cb9a7c Remove mention of #36 from troubleshooting guide
The 091217b workaround of telling ffmpeg to only request the video
stream works perfectly fine for now. I'll revisit when adding audio
support (#34).

Fixes #36
2019-02-13 22:25:33 -08:00
Scott Lamb
091217b1a4 configure ffmpeg to only stream video
This works around #36 for now. I'll need to do something different when
I actually implement audio support.
2019-02-11 22:58:09 -08:00
Scott Lamb
c271cfa2b5 make Writer enforce maximum recording duration
My installation recently somehow ended up with a recording with a
duration of 503793844 90,000ths of a second, way over the maximum of 5
minutes. (Looks like the machine was pretty unresponsive at the time
and/or having network problems.)

When this happens, the system really spirals. Every flush afterward (12
per minute with my installation) fails with a CHECK constraint failure
on the recording table. It never gives up on that recording. /var/log
fills pretty quickly as this failure is extremely verbose (a stack
trace, and a line for each byte of video_index). Eventually the sample
file dirs fill up too as it continues writing video samples while GC is
stuck. The video samples are useless anyway; given that they're not
referenced in the database, they'll be deleted on next startup.

This ensures the offending recording is never added to the database, so
we don't get the same persistent problem. Instead, writing to the
recording will fail. The stream will drop and be retried. If the
underlying condition that caused a too-long recording (many
non-key-frames, or the camera returning a crazy duration, or the
monotonic clock jumping forward extremely, or something) has gone away,
the system should recover.
2019-01-29 08:26:36 -08:00
Scott Lamb
3ba3bf2b18 backend support for live stream (#59)
This is so far completely untested, for use by a new UI prototype.

It creates a new URL endpoint which sends one video/mp4 media segment
per key frame, with the dependent frames included. This means there will
be about one key frame interval of latency (typically about a second).
This seems hard to avoid, as mentioned in issue #59.
2019-01-21 15:58:52 -08:00
Scott Lamb
95a8c2e78d support .mp4 files > 13.25 hours
Use version 1 of the mvhd, tkhd, and mdhd boxes to support 64-bit
durations. 2^32 units / 90,000 units/sec / 60 sec/min / 60 min/hr ~=
13.25 hrs.

Compatibility: looks like Chrome, Firefox, VLC, and ffmepg all support
version 1 with no problem.
2019-01-07 00:59:32 -08:00