Commit Graph

509 Commits

Author SHA1 Message Date
Scott Lamb dc402bdc01 schema version 2: support sub streams
This allows each camera to have a main and a sub stream. Previously there was
a field in the schema for the sub stream's url, but it didn't do anything. Now
you can configure individual retention for main and sub streams. They show up
grouped in the UI.

No support for upgrading from schema version 1 yet.
2018-02-03 22:15:54 -08:00
Scott Lamb 0d69f4f49b use add_camera in tests, not direct db inserts
This is a wash in terms of lines of code now, but it makes it a bit easier to
maintain as I make changes to the schema (such as separating out streams from
cameras), and it helps ensure the tests reflect reality.
2018-02-03 21:56:04 -08:00
Scott Lamb c43fb80639 warn if a streamer op takes too long
My odroid setup has been occasionally (about once a week) losing about 15
seconds of recordings on all cameras. I'm not sure why. So I'm labelling all
the likely suspect spots and logging if any of them takes longer than a
second. I think this will give me more information; hopefully narrow it down
to network or local disk I/O.
2018-01-31 14:20:30 -08:00
Scott Lamb 6902be1981 upgrade deps 2018-01-30 22:05:39 -08:00
Scott Lamb d192d98129 more fixes to the upgrade guide 2018-01-30 21:14:53 -08:00
Scott Lamb 9a8ed69047 fix more formatting in upgrade guide 2018-01-30 17:01:44 -08:00
Scott Lamb 7566b6a38b fix several formatting problems in upgrade guide 2018-01-30 17:01:03 -08:00
Scott Lamb 529aad9982 upgrade deps, including http-serve bugfix
This was including the wrong Content-Encoding: header on gzipped json
responses, so Chrome wouldn't understand them at all.
2018-01-30 16:51:32 -08:00
Scott Lamb 2c62d977b0 gzip json responses, handle HEAD properly 2018-01-23 11:24:40 -08:00
Scott Lamb 8caa2e5d0e crate rename: http-(entity|file) -> http-serve 2018-01-23 11:08:21 -08:00
Scott Lamb 10550bc35f simplify ffmpeg wrapper crate
This was using PhantomData to enforce lifetimes + raw pointers. Simpler to
convert to a reference. This also enforces non-null.
2017-11-30 14:40:31 -08:00
Scott Lamb 5c8970fe8a update dependencies 2017-11-16 23:01:09 -08:00
Scott Lamb 16ed7f73ba remove redundant panic 2017-11-16 22:54:44 -08:00
Scott Lamb b9ebb74a58 explicitly test ffmpeg library compatibility
Makes the problem in #11 more obvious.
2017-10-24 07:26:18 -07:00
Scott Lamb 99f10dfba6 fix instructions for building UI
"yarn build" doesn't actually fetch packages. When starting from a clean
client, you have to run "yarn" (with no arguments) first.
2017-10-23 21:10:58 -07:00
Scott Lamb 2c026e1b6b minor cleanups to ffmpeg build setup
* the "lib: {}" print didn't do anything. It turns out that the pkg-config
  crate emits the necessary metadata for linking automatically. I had the
  wrong format and didn't notice because something else did it correctly.

* gcc::Config is deprecated; the new name is Build.

* and the crate is now called cc, version 1.0.

Stuff found while looking at #11. Still haven't figured that issue out.
2017-10-23 21:07:07 -07:00
Scott Lamb 8de7e391f8 populate timeZoneName as expected by UI
This works by a nasty hack, but it seems to work well enough for now.
Fingers crossed.
2017-10-21 23:57:13 -07:00
Scott Lamb 2966cf59b0 couple more js fixes
* make "yarn build" cmd work on first run.
  (it was installing a hardlinked file where the dir should go, yuck)

* remove an obsolete ui/index.html; it's ui-src/index.html now
2017-10-21 23:01:35 -07:00
Scott Lamb 084c44f110 update to http-file with Linux/arm fix 2017-10-21 22:44:04 -07:00
Scott Lamb 92962b9ee6 fix broken Cargo.toml
I'd temporarily pointed this to a local path for development and didn't notice
it was still in place when committing. Back to the git path that works for
everyone.
2017-10-21 22:35:24 -07:00
Scott Lamb 6b369a4cf0 3rd try at screenshot in README 2017-10-21 22:03:12 -07:00
Scott Lamb 43874c6a43 second try at screenshot in README 2017-10-21 21:57:04 -07:00
Scott Lamb 315f3594c2 add a basic Javascript UI
The Javascript is pretty amateurish I'm sure but at least it's something to
iterate from. It's already much more pleasant for browsing through videos in
several ways:

* more responsive to load only a day at a time rather than 90+ days
* much easier to see the same time segment on several cameras
* more pleasant to have the videos load as a popup rather than a link
  that blows away your position in an enormous list
* exposes the fancier .mp4 generation options: splitting at lengths
  other than the default, trimming to an arbitrary start and end time,
  including a subtitle track with timestamps.

There's a slight regression in functionality: I didn't match the former
top-level page which showed how much camera used of its disk allocation and
the total duration of video. This is exposed in the JSON API, so it shouldn't
be too hard to add back.
2017-10-21 21:54:27 -07:00
Scott Lamb 6eda26a9cc support run splitting in json api 2017-10-17 09:00:05 -07:00
Scott Lamb 9041eeb907 fix panic when requesting zero segment duration
The recording::Segment was constructing a segment with no frames in it, which
was causing a panic when appending a zero-length stts to the Slices. Fix this
in a couple ways:

* Slices::append should return Err rather than panic. No reason to crash the
  whole program when we have trouble serving a single .mp4 request.
* recording::Segment shouldn't produce zero-frame segments
2017-10-17 08:55:21 -07:00
Scott Lamb 1d08698d0c debug, fix panic with zero-duration recording
I had an assert that fired in this case, dating back to when I hadn't plumbed
Result returns through much of .mp4 construction. Now I have, so there's no
excuse in having an assert here. Change to an error return, and tweak it to
not fire in the zero-duration case.

Also fix a problem in the test harness; I hadn't finished converting it for
multi-recording tests, and it was returning the wrong recording.

Because of that, I seem to have stumbled across a related problem in which
asking for zero duration of a non-zero duration recording will return a
recording::Segment with no frames, which will cause panics because its
corresponding .mp4 slices are zero-length. I just adjusted the panic message
here; I'll follow up with changes to address that.
2017-10-17 06:14:47 -07:00
Scott Lamb 1b1ae0bf0a fix link to users list, mention github tracker 2017-10-16 22:20:29 -07:00
Scott Lamb 2bdb2eca5d fix a couple time problems
* CameraDayKey::bounds (used to generate the start and end times of days in
  the returned JSON) returned UTC, not matching what recordings were mapped
  into that day. So fetching a day with its given bounds would return
  something different. Test and fix it.

* Several time-related tests weren't calling testutil::init(), so they weren't
  fixing the time zone to the expected America/Los_Angeles. If the machine
  time is set to something else, they would break.
2017-10-11 20:08:26 -07:00
Scott Lamb bbe04f909c fix ClockAdjuster logic
Small negative deltas caused every_minus_1 to be negative, which caused
underflow errors in debug builds. Fix this, test more comprehensively.
2017-10-09 22:13:38 -07:00
Scott Lamb 1e4d7d5ad9 make json api more idiomatic
* camelCase
* lose the "days":null in the overall cameras dict
2017-10-09 21:58:44 -07:00
Scott Lamb 5bb3dde74e work around #10 with advanced_editlist=false
I think this is an ffmpeg bug, which I plan to report. In the meantime, this
makes the tests pass. Long-term, even if ffmpeg fixes this, I probably don't
want to continue doing acceptance tests against whatever version of ffmpeg
happens to be installed - my real targets of interest are the latest versions
of Chrome, Firefox, Safari, QuickTime, and VLC.
2017-10-09 21:44:48 -07:00
Scott Lamb 711f7b3409 fix with-editlist hash
I missed this because I was running with ffmpeg 3 and had grown to expect this
test to fail. Quick fix on that coming shortly.
2017-10-09 21:00:45 -07:00
Scott Lamb af282c309e fix corrupt stss on segments after trimmed segment
This was causing Firefox to fail to play multipart .mp4s which trimmed away a
prefix. In the developer console, it said NS_ERROR_DOM_MEDIA_METADATA_ERR
without giving any RESULT_DETAIL, making it a pain to diagnose. Given that the
stss is supposed to be needed for seeking, I'm surprised it didn't have any
immediately obvious impact on Chrome or VLC.  Maybe they just took longer to
seek than otherwise necessary.

The bug was that when keeping track of the "next frame num" while constructing
the .mp4, I appended the number in the underlying recording, not the number
post-trimming. That meant following segments used the wrong numbers. In some
cases, it caused it to exceed the total number of samples in the generated
.mp4, which seems to be what Firefox was complaining about. Running the result
through "ffmpeg -i bad.mp4 -c copy -f mp4 good.mp4" just trimmed away the most
obviously invalid ones, leaving others that didn't point to the frames they
meant to. That was enough to make Firefox start playing the file. /shruggie

The existing tests were all with a single segment, so I added a new one to
catch this. I also added a Debug implementation to recording::Segment and
mp4::Segment.
2017-10-09 06:32:43 -07:00
Scott Lamb 919e9a6deb remove extraneous debug logging 2017-10-04 22:55:29 -07:00
Scott Lamb cb18ba44d8 fix /view.mp4 with rel_start
This was totally broken in commit 1cf27c18. It would serve bytes from the
beginning of the sample file in question, not from the start of the given
range.
2017-10-04 22:51:16 -07:00
Scott Lamb 57985079cc bugfix: in /recordings, end_id should be inclusive 2017-10-04 06:36:30 -07:00
Scott Lamb 5ea2c2fed1 fix media rate in edit list
it should be exactly 1, but was slightly more because the fraction was
incorrectly 1 rather than 0. I'm not sure if any actual players care about
this, but it was something I noticed when looking into strange edit list
behavior.
2017-10-04 00:03:33 -07:00
Scott Lamb bd4104b446 add start_id and end_id to .../recordings json
This was added to the API documentation in eee887b9 but never actually
implemented then. It's necessary to actually fetch the .mp4 in question.
2017-10-04 00:00:56 -07:00
Scott Lamb 7673a00bd9 serve 'video/mp4; codecs="avc1.xxxxxx"' mime type
This can be used when constructing a HTML5 SourceBuffer.
2017-10-03 23:25:58 -07:00
Scott Lamb 9eff91f7da fix test clip's mvhd timebase
I manually fixed the offending timebases and durations with a hex editor.
This addresses most of the failing tests described in #10.
2017-10-02 20:43:37 -07:00
Scott Lamb cbd8f7d3d2 Reorganize and expand documentation 2017-10-01 22:02:39 -07:00
Scott Lamb 04e9f3f160 support segmented mp4s
This is intended to support HTML5 Media Source Extensions, which I expect to
be the most practical way to make a good web UI with a proper scrub bar and
such.

This feature has had very limited testing on Chrome and Firefox, and that was
not entirely successful. More work is needed before it's usable, but this
seems like a helpful progress checkpoint.
2017-10-01 15:29:22 -07:00
Scott Lamb cb689b2ec8 Linux/arm compilation fix
libc::c_char is u8 rather than i8 there, unlike Linux/x86_64 or OS X.
Use correct type to compile on all platforms.
2017-09-23 21:12:17 -07:00
Scott Lamb 45b508dff6 fix moonfire_ffmpeg::Error formatting
This would return a string with trailing nuls up to the buffer size (64
bytes) which would cause problems later. One in particular: in
"moonfire-nvr config", if testing a camera failed, displaying the error would
panic with the backtrace below.

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: NulError(33, [69, 114, 114, 111, 114, 58, 32, 102, 102, 109, 112, 101, 103, 58, 32, 67, 111, 110, 110, 101, 99, 116, 105, 111, 110, 32, 114, 101, 102, 117, 115, 101, 100, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0])', src/libcore/result.rs:860:4
stack backtrace:
   0: std::sys:👿:backtrace::tracing:👿:unwind_backtrace
   1: std::panicking::default_hook::{{closure}}
   2: std::panicking::default_hook
   3: std::panicking::rust_panic_with_hook
   4: std::panicking::begin_panic_new
   5: std::panicking::begin_panic_fmt
   6: rust_begin_unwind
   7: core::panicking::panic_fmt
   8: core::result::unwrap_failed
   9: <core::result::Result<T, E>>::unwrap
  10: <&'a str as ncurses::ToCStr>::to_c_str
  11: ncurses::addstr
  12: ncurses::mvaddstr
  13: <cursive::backend::curses::n::Concrete as cursive::backend::Backend>::print_at
  14: cursive:🖨️:Printer::print
  15: <cursive::views::text_view::TextView as cursive::view::View>::draw::{{closure}}
  16: cursive::view:📜:ScrollBase::draw
  17: <cursive::views::text_view::TextView as cursive::view::View>::draw
  18: <cursive::views::dialog::Dialog as cursive::view::View>::draw
  19: <cursive::views::layer::Layer<T> as cursive::view::view_wrapper::ViewWrapper>::wrap_draw
  20: cursive::view::view_wrapper::<impl cursive::view::View for T>::draw
  21: <cursive::views::shadow_view::ShadowView<T> as cursive::view::view_wrapper::ViewWrapper>::wrap_draw
  22: cursive::view::view_wrapper::<impl cursive::view::View for T>::draw
  23: <cursive::views::stack_view::StackView as cursive::view::View>::draw::{{closure}}
  24: cursive:🖨️:Printer::with_color::{{closure}}
  25: <cursive::backend::curses::n::Concrete as cursive::backend::Backend>::with_color
  26: cursive:🖨️:Printer::with_color
  27: <cursive::views::stack_view::StackView as cursive::view::View>::draw
  28: cursive::Cursive::draw
  29: cursive::Cursive::step
  30: cursive::Cursive::run
  31: moonfire_nvr::cmds::config::run
  32: moonfire_nvr::cmds::Command::run
  33: moonfire_nvr::main
  34: __rust_maybe_catch_panic
  35: std::rt::lang_start
  36: main
2017-09-22 06:47:08 -07:00
Scott Lamb 71e2930ced use the latest libc crate to build on OS X 2017-09-22 06:20:40 -07:00
Scott Lamb 11420df065 update deps (particularly hyper) + fix warnings 2017-09-21 21:51:58 -07:00
Scott Lamb 857a66f29c use my own ffmpeg crate
This significantly improves safety of the ffmpeg interface. The complex
ABIs aren't accessed directly from Rust. Instead, I have a small C
wrapper which uses the ffmpeg C API and the C headers at compile-time to
determine the proper ABI in the same way any C program using ffmpeg
would, so that the ABI doesn't have to be duplicated in Rust code.
I've tested with ffmpeg 2.x and ffmpeg 3.x; it seems to work properly
with both where before ffmpeg 3.x caused segfaults.

It still depends on ABI compatibility between the compiled and running
versions. C programs need this, too, and normal shared library
versioning practices provide this guarantee. But log both versions on
startup for diagnosing problems with this.

Fixes #7
2017-09-20 21:06:06 -07:00
Scott Lamb 8ff1d0dcb8 workaround config crash since cursive 0.5.1
https://github.com/gyscos/Cursive/issues/144
2017-07-02 22:03:16 -07:00
Scott Lamb ac43e7fe17 fix bench --features=nightly, broken by upgrade
The benchmarks don't get compiled with the standard "cargo test";
they require "cargo +nightly bench --features=nightly", so I didn't notice
they were broken in the previous commit. Now fixed.
2017-06-11 19:40:36 -07:00
Scott Lamb bebd6ee79a update dependencies
* The mylog update fixes a couple bad bugs.
* Otherwise, just keep up with the Rust ecosystem.
2017-06-11 12:57:55 -07:00