MyFavMirrors
/
moonfire-nvr
mirror of https://github.com/scottlamb/moonfire-nvr.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
moonfire-nvr/README.md

75 lines
3.9 KiB
Markdown
Raw Normal View History

Initial commit, with basic functionality.
2016-01-02 01:06:47 -05:00
# Introduction
Moonfire NVR is an open-source security camera network video recorder, started
fix a couple README formatting mistakes
2016-11-25 16:03:58 -05:00
by Scott Lamb &lt;<slamb@slamb.org>&gt;. It saves H.264-over-RTSP streams from
IP cameras to disk into a hybrid format: video frames in a directory on
update the README to reflect the SQLite schema
2016-11-25 16:00:27 -05:00
spinning disk, other data in a SQLite3 database on flash. It can construct
`.mp4` files for arbitrary time ranges on-the-fly. It does not decode,
analyze, or re-encode video frames, so it requires little CPU. It handles six
1080p/30fps streams on a [Raspberry Pi
2](https://www.raspberrypi.org/products/raspberry-pi-2-model-b/), using
less than 10% of the machine's total CPU.
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-22 00:54:27 -04:00
So far, the web interface is basic: a filterable list of video segments,
with support for trimming them to arbitrary time ranges. No scrub bar yet.
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-26 00:31:50 -05:00
There's also no support for motion detection, no https/SSL/TLS support (you'll
document proxy setup in guide/secure.md (for #26) The guide is not as quick to follow and amateur-friendly as I'd like. A few things that might improve matters: * complete #27 (built-in https+letsencrypt), so that when not sharing the port, users don't need to use nginx or certbot. * more ubiquitous IPv6 (out of my control but should happen over time) to reduce need to share the port * embed a dynamic DNS client * support UPnP Internet Gateway Device Control Protocol (if common routers have this enabled? probably not for security reasons.) It's progress, though. Enough that I think I'll merge the auth branch into master shortly.
2018-12-27 17:00:15 -05:00
need a proxy server, as described [here](guide/secure.md)), and only a
console-based (rather than web-based) configuration UI.
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-22 00:54:27 -04:00
3rd try at screenshot in README
2017-10-22 01:03:12 -04:00
![screenshot](screenshot.png)
Initial commit, with basic functionality.
2016-01-02 01:06:47 -05:00
This is version 0.1, the initial release. Until version 1.0, there will be no
compatibility guarantees: configuration and storage formats may change from
schema version 1 The advantages of the new schema are: * overlapping recordings can be unambiguously described and viewed. This is a significant problem right now; the clock on my cameras appears to run faster than the (NTP-synchronized) clock on my NVR. Thus, if an RTSP session drops and is quickly reconnected, there's likely to be overlap. * less I/O is required to view mp4s when there are multiple cameras. This is a pretty dramatic difference in the number of database read syscalls with pragma page_size = 1024 (605 -> 39 in one test), although I'm not sure how much of that maps to actual I/O wait time. That's probably as dramatic as it is due to overflow page chaining. But even with larger page sizes, there's an improvement. It helps to stop interleaving the video_index fields from different cameras. There are changes to the JSON API to take advantage of this, described in design/api.md. There's an upgrade procedure, described in guide/schema.md.
2016-12-21 01:08:18 -05:00
version to version. There is an [upgrade procedure](guide/schema.md) but it is
not for the faint of heart.
Initial commit, with basic functionality.
2016-01-02 01:06:47 -05:00
I hope to add features such as salient motion detection. It's way too early to
make promises, but it seems possible to build a full-featured
hobbyist-oriented multi-camera NVR that requires nothing but a cheap machine
with a big hard drive. I welcome help; see [Getting help and getting
involved](#help) below. There are many exciting techniques we could use to
make this possible:
* avoiding CPU-intensive H.264 encoding in favor of simply continuing to use the
camera's already-encoded video streams. Cheap IP cameras these days provide
pre-encoded H.264 streams in both "main" (full-sized) and "sub" (lower
resolution, compression quality, and/or frame rate) varieties. The "sub"
stream is more suitable for fast computer vision work as well as
remote/mobile streaming. Disk space these days is quite cheap (with 3 TB
drives costing about $100), so we can afford to keep many camera-months of
both streams on disk.
* decoding and analyzing only select "key" video frames (see
few small markdown tweaks
2018-08-25 00:04:13 -04:00
[wikipedia](https://en.wikipedia.org/wiki/Video_compression_picture_types)).
Initial commit, with basic functionality.
2016-01-02 01:06:47 -05:00
* off-loading expensive work to a GPU. Even the Raspberry Pi has a
surprisingly powerful GPU.
* using [HTTP Live Streaming](https://en.wikipedia.org/wiki/HTTP_Live_Streaming)
rather than requiring custom browser plug-ins.
* taking advantage of cameras' built-in motion detection. This is
the most obvious way to reduce motion detection CPU. It's a last resort
because these cheap cameras' proprietary algorithms are awful compared to
those described on [changedetection.net](http://changedetection.net).
Cameras have high false-positive and false-negative rates, are hard to
experiment with (as opposed to rerunning against saved video files), and
don't provide any information beyond if motion exceeded the threshold or
not.
Reorganize and expand documentation
2017-10-02 01:02:39 -04:00
# Documentation
Initial commit, with basic functionality.
2016-01-02 01:06:47 -05:00
Reorganize and expand documentation
2017-10-02 01:02:39 -04:00
* [License](LICENSE.txt) — GPLv3
* [Building and installing](guide/install.md)
Settings can now be taken from separate file with local override. * Various settings in settings-nvr.js module * settings-nvr-local.js can override settings-nvr.js * settings-nvr-local is unchecked file * Both files can be straight maps, or functions returning maps * webpack env and args available to those functions
2018-03-07 16:09:43 -05:00
* [UI Development](guide/developing-ui.md)
Reorganize and expand documentation
2017-10-02 01:02:39 -04:00
* [Troubleshooting](guide/troubleshooting.md)
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
Initial commit, with basic functionality.
2016-01-02 01:06:47 -05:00
# <a name="help"></a> Getting help and getting involved
Please email the
fix link to users list, mention github tracker
2017-10-17 01:20:29 -04:00
[moonfire-nvr-users](https://groups.google.com/d/forum/moonfire-nvr-users)
mailing list with questions, or just to say you love/hate the software and
why. You can also file bugs and feature requests on the
[github issue tracker](https://github.com/scottlamb/moonfire-nvr/issues).
Initial commit, with basic functionality.
2016-01-02 01:06:47 -05:00
Rust rewrite I should have submitted/pushed more incrementally but just played with it on my computer as I was learning the language. The new Rust version more or less matches the functionality of the current C++ version, although there are many caveats listed below. Upgrade notes: when moving from the C++ version, I recommend dropping and recreating the "recording_cover" index in SQLite3 to pick up the addition of the "video_sync_samples" column: $ sudo systemctl stop moonfire-nvr $ sudo -u moonfire-nvr sqlite3 /var/lib/moonfire-nvr/db/db sqlite> drop index recording_cover; sqlite3> create index ...rest of command as in schema.sql...; sqlite3> ^D Some known visible differences from the C++ version: * .mp4 generation queries SQLite3 differently. Before it would just get all video indexes in a single query. Now it leads with a query that should be satisfiable by the covering index (assuming the index has been recreated as noted above), then queries individual recording's indexes as needed to fill a LRU cache. I believe this is roughly similar speed for the initial hit (which generates the moov part of the file) and significantly faster when seeking. I would have done it a while ago with the C++ version but didn't want to track down a lru cache library. It was easier to find with Rust. * On startup, the Rust version cleans up old reserved files. This is as in the design; the C++ version was just missing this code. * The .html recording list output is a little different. It's in ascending order, with the most current segment shorten than an hour rather than the oldest. This is less ergonomic, but it was easy. I could fix it or just wait to obsolete it with some fancier JavaScript UI. * commandline argument parsing and logging have changed formats due to different underlying libraries. * The JSON output isn't quite right (matching the spec / C++ implementation) yet. Additional caveats: * I haven't done any proof-reading of prep.sh + install instructions. * There's a lot of code quality work to do: adding (back) comments and test coverage, developing a good Rust style. * The ffmpeg foreign function interface is particularly sketchy. I'd eventually like to switch to something based on autogenerated bindings. I'd also like to use pure Rust code where practical, but once I do on-NVR motion detection I'll need to existing C/C++ libraries for speed (H.264 decoding + OpenCL-based analysis).
2016-11-25 17:34:00 -05:00
I'd welcome help with testing, development (in Rust, JavaScript, and HTML),
user interface/graphic design, and documentation. Please email the mailing
Reorganize and expand documentation
2017-10-02 01:02:39 -04:00
list if interested. Pull requests are welcome, but I encourage you to discuss
large changes on the mailing list or in a github issue first to save effort.