new database/sample file dir interlock scheme

The idea is to avoid the problems described in src/schema.proto; those
possibilities have bothered me for a while. A bonus is that (in a future
commit) it can replace the sample file uuid scheme in favor of using
<camera_uuid>-<stream_type>/<recording_id> for several advantages:

  * on data integrity problems (specifically, extra sample files), more
    information to use to understand what happened.
  * no more reserving sample files prior to using them. This avoids some extra
    database transactions on startup (now there's an extra two total rather
    than an extra one per stream). It also simplifies an upcoming change I
    want to make in which some streams are not flushed immediately, reducing
    the write load significantly (maybe one per minute total rather than one
    per stream per minute).
  * get rid of eight bytes per playback cache entry in RAM (and nine bytes
    per recording_playback row on flash).

The implementation is still pretty rough in places:

  * Lack of tests.
  * Poor ode organization. In particular, SampleFileDirectory::write_meta
    shouldn't be exposed beyond db. I'm thinking about moving db.rs and
    SampleFileDirectory to a new crate, moonfire_nvr_db. This would improve
    compile times as well.
  * No tooling for renaming a sample file directory.
  * Config subcommand still panics in conditions that can be reasonably
    expected to happen.
This commit is contained in:
Scott Lamb
2018-02-14 23:10:10 -08:00
parent 89b6bccaa3
commit e7f5733f29
19 changed files with 1508 additions and 344 deletions

View File

@@ -199,6 +199,8 @@ Version 2 adds:
* recording of sub streams (splits a new `stream` table out of `camera`)
* support for multiple sample file directories, to take advantage of
multiple hard drives (or multiple RAID volumes).
* interlock between database and sample file directories to avoid various
mixups that could cause data integrity problems.
* records the RFC-6381 codec associated with a video sample entry, so that
logic for determining this is no longer needed as part of the database
layer.