diff --git a/README.md b/README.md index 154d972..e8b1712 100644 --- a/README.md +++ b/README.md @@ -50,209 +50,11 @@ make this possible: don't provide any information beyond if motion exceeded the threshold or not. -# Downloading +# Documentation -See the [github page](https://github.com/scottlamb/moonfire-nvr) (in case -you're not reading this text there already). You can download the bleeding -edge version from the command line via git: - - $ git clone https://github.com/scottlamb/moonfire-nvr.git - -# Building from source - -There are no binary packages of Moonfire NVR available yet, so it must be built -from source. - -Moonfire NVR is written in the [Rust Programming -Language](https://www.rust-lang.org/en-US/). In the long term, I expect this -will result in a more secure, full-featured, easy-to-install software. In the -short term, there will be growing pains. Rust is a new programming language. -Moonfire NVR's primary author is new to Rust. And Moonfire NVR is a young -project. - -You will need the following C libraries installed: - -* [ffmpeg](http://ffmpeg.org/) version 2.x or 3.x, including `libavutil`, - `libavcodec` (to inspect H.264 frames), and `libavformat` (to connect to RTSP - servers and write `.mp4` files). - - Note ffmpeg library versions older than 55.1.101, along with all versions of - the competing project [libav](http://libav.org), don't not support socket - timeouts for RTSP. For reliable reconnections on error, it's strongly - recommended to use ffmpeg library versions >= 55.1.101. - -* [SQLite3](https://www.sqlite.org/). - -* [`ncursesw`](https://www.gnu.org/software/ncurses/), the UTF-8 version of - the `ncurses` library. - -On Ubuntu 16.04.1 LTS or Raspbian Jessie, the following command will install -all non-Rust dependencies: - - $ sudo apt-get install \ - build-essential \ - libavcodec-dev \ - libavformat-dev \ - libavutil-dev \ - libncurses5-dev \ - libncursesw5-dev \ - libsqlite3-dev \ - libssl-dev \ - pkgconf - -Next, you need Rust 1.15+ and Cargo. The easiest way to install them is by following -the instructions at [rustup.rs](https://www.rustup.rs/). - -You can continue to follow the build/install instructions below for a manual -build and install, or alternatively you can run the prep script called `prep.sh`. - - $ cd moonfire-nvr - $ ./prep.sh - -The script will take the following command line options, should you need them: - -* `-S`: Skip updating and installing dependencies through apt-get. This too can be - useful on repeated builds. - -You can edit variables at the start of the script to influence names and -directories, but defaults should suffice in most cases. For details refer to -the script itself. We will mention just one option, needed when you follow the -suggestion to separate database and samples between flash storage and a hard disk. -If you have the hard disk mounted on, lets say `/media/nvr`, and you want to -store the video samples inside a directory named `samples` there, you would set: - - SAMPLES_DIR=/media/nvr/samples - -The script will perform all necessary steps to leave you with a fully built, -installed moonfire-nvr binary. The only thing -you'll have to do manually is add your camera configuration(s) to the database. -Alternatively, before running the script, you can create a file named `cameras.sql` -in the same directory as the `prep.sh` script and it will be automatically -included for you. -For instructions, you can skip to "[Camera configuration and hard disk mounting](#camera)". - -Once prerequisites are installed, Moonfire NVR can be built as follows: - - $ cargo test - $ cargo build --release - $ sudo install -m 755 target/release/moonfire-nvr /usr/local/bin - -# Further configuration - -Moonfire NVR should be run under a dedicated user. It keeps two kinds of -state: - - * a SQLite database, typically <1 GiB. It should be stored on flash if - available. - * the "sample file directory", which holds the actual samples/frames of - H.264 video. This should be quite large and typically is stored on a hard - drive. - -(See [guide/schema.md](guide/schema.md) for more information.) - -Both kinds of state are intended to be accessed only by Moonfire NVR itself. -However, the interface for adding new cameras is not yet written, so you will -have to manually insert cameras with the `sqlite3` command line tool prior to -starting Moonfire NVR. - -Manual commands would look something like this: - - $ sudo addgroup --system moonfire-nvr - $ sudo adduser --system moonfire-nvr --home /var/lib/moonfire-nvr - $ sudo mkdir /var/lib/moonfire-nvr - $ sudo -u moonfire-nvr -H mkdir db sample - $ sudo -u moonfire-nvr moonfire-nvr init - -## Camera configuration and hard drive mounting - -If a dedicated hard drive is available, set up the mount point: - - $ sudo vim /etc/fstab - $ sudo mount /var/lib/moonfire-nvr/sample - -Once setup is complete, it is time to add camera configurations to the -database. If the daemon is running, you will need to stop it temporarily: - - $ sudo systemctl stop moonfire-nvr - -You can configure the system through a text-based user interface: - - $ sudo -u moonfire-nvr moonfire-nvr config 2>debug-log - -In the user interface, add your cameras under the "Edit cameras" dialog. -There's a "Test" button to verify your settings directly from the dialog. - -After the cameras look correct, go to "Edit retention" to assign disk space to -each camera. Leave a little slack (at least 100 MB per camera) between the total -limit and the filesystem capacity, even if you store nothing else on the disk. -There are several reasons this is needed: - - * The limit currently controls fully-written files only. There will be up - to two minutes of video per camera of additional video. - * The rotation happens after the limit is exceeded, not proactively. - * Moonfire NVR currently doesn't account for the unused space in the final - filesystem block at the end of each file. - * Moonfire NVR doesn't account for the space used for directory listings. - * If a file is open when it is deleted (such as if a HTTP client is - downloading it), it stays around until the file is closed. Moonfire NVR - currently doesn't account for this. - -When finished, start the daemon: - - $ sudo systemctl start moonfire-nvr - -## System Service - -Moonfire NVR can be run as a systemd service. If you used `prep.sh` this has -been done for you. If not, Create -`/etc/systemd/system/moonfire-nvr.service`: - - [Unit] - Description=Moonfire NVR - After=network-online.target - - [Service] - ExecStart=/usr/local/bin/moonfire-nvr run \ - --sample-file-dir=/var/lib/moonfire-nvr/sample \ - --db-dir=/var/lib/moonfire-nvr/db \ - --http-addr=0.0.0.0:8080 - Environment=TZ=:/etc/localtime - Environment=MOONFIRE_FORMAT=google-systemd - Environment=MOONFIRE_LOG=info - Type=simple - User=moonfire-nvr - Nice=-20 - Restart=on-abnormal - CPUAccounting=true - MemoryAccounting=true - BlockIOAccounting=true - - [Install] - WantedBy=multi-user.target - -Note that the HTTP port currently has no authentication; it should not be -directly exposed to the Internet. - -Complete the installation through `systemctl` commands: - - $ sudo systemctl daemon-reload - $ sudo systemctl start moonfire-nvr - $ sudo systemctl status moonfire-nvr - $ sudo systemctl enable moonfire-nvr - -See the [systemd](http://www.freedesktop.org/wiki/Software/systemd/) -documentation for more information. The [manual -pages](http://www.freedesktop.org/software/systemd/man/) for `systemd.service` -and `systemctl` may be of particular interest. - -# Troubleshooting - -While Moonfire NVR is running, logs will be written to stderr. The -`MOONFIRE_LOG` environmental variable controls the log level; -`MOONFIRE_LOG=info` is the default. `MOONFIRE_FORMAT` controls the -logging style; options are `google` (default, like the Google glog package) -or `google-systemd` (formatted for the systemd journal). If running through -systemd, try `sudo journalctl --unit moonfire-nvr` to view the logs. +* [License](LICENSE.txt) — GPLv3 +* [Building and installing](guide/install.md) +* [Troubleshooting](guide/troubleshooting.md) # Getting help and getting involved @@ -263,37 +65,5 @@ you love/hate the software and why. I'd welcome help with testing, development (in Rust, JavaScript, and HTML), user interface/graphic design, and documentation. Please email the mailing -list if interested. Patches are welcome, but I encourage you to discuss large -changes on the mailing list first to save effort. - -# License - -This file is part of Moonfire NVR, a security camera digital video recorder. -Copyright (C) 2016 Scott Lamb - -This program is free software: you can redistribute it and/or modify -it under the terms of the GNU General Public License as published by -the Free Software Foundation, either version 3 of the License, or -(at your option) any later version. - -In addition, as a special exception, the copyright holders give -permission to link the code of portions of this program with the -OpenSSL library under certain conditions as described in each -individual source file, and distribute linked combinations including -the two. - -You must obey the GNU General Public License in all respects for all -of the code used other than OpenSSL. If you modify file(s) with this -exception, you may extend this exception to your version of the -file(s), but you are not obligated to do so. If you do not wish to do -so, delete this exception statement from your version. If you delete -this exception statement from all source files in the program, then -also delete it here. - -This program is distributed in the hope that it will be useful, -but WITHOUT ANY WARRANTY; without even the implied warranty of -MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the -GNU General Public License for more details. - -You should have received a copy of the GNU General Public License -along with this program. If not, see . +list if interested. Pull requests are welcome, but I encourage you to discuss +large changes on the mailing list or in a github issue first to save effort. diff --git a/guide/install.md b/guide/install.md new file mode 100644 index 0000000..884bfb5 --- /dev/null +++ b/guide/install.md @@ -0,0 +1,198 @@ +# Installing Moonfire NVR + +This document describes how to install Moonfire NVR on a Linux system. + +## Downloading + +See the [github page](https://github.com/scottlamb/moonfire-nvr) (in case +you're not reading this text there already). You can download the bleeding +edge version from the command line via git: + + $ git clone https://github.com/scottlamb/moonfire-nvr.git + +## Building from source + +There are no binary packages of Moonfire NVR available yet, so it must be built +from source. + +Moonfire NVR is written in the [Rust Programming +Language](https://www.rust-lang.org/en-US/). In the long term, I expect this +will result in a more secure, full-featured, easy-to-install software. In the +short term, there will be growing pains. Rust is a new programming language. +Moonfire NVR's primary author is new to Rust. And Moonfire NVR is a young +project. + +You will need the following C libraries installed: + +* [ffmpeg](http://ffmpeg.org/) version 2.x or 3.x, including `libavutil`, + `libavcodec` (to inspect H.264 frames), and `libavformat` (to connect to RTSP + servers and write `.mp4` files). + + Note ffmpeg library versions older than 55.1.101, along with all versions of + the competing project [libav](http://libav.org), don't not support socket + timeouts for RTSP. For reliable reconnections on error, it's strongly + recommended to use ffmpeg library versions >= 55.1.101. + +* [SQLite3](https://www.sqlite.org/). + +* [`ncursesw`](https://www.gnu.org/software/ncurses/), the UTF-8 version of + the `ncurses` library. + +On recent Ubuntu or Raspbian, the following command will install +all non-Rust dependencies: + + $ sudo apt-get install \ + build-essential \ + libavcodec-dev \ + libavformat-dev \ + libavutil-dev \ + libncurses5-dev \ + libncursesw5-dev \ + libsqlite3-dev \ + libssl-dev \ + pkgconf + +Next, you need Rust 1.17+ and Cargo. The easiest way to install them is by following +the instructions at [rustup.rs](https://www.rustup.rs/). + +You can continue to follow the build/install instructions below for a manual +build and install, or alternatively you can run the prep script called `prep.sh`. + + $ cd moonfire-nvr + $ ./prep.sh + +The script will take the following command line options, should you need them: + +* `-S`: Skip updating and installing dependencies through apt-get. This too can be + useful on repeated builds. + +You can edit variables at the start of the script to influence names and +directories, but defaults should suffice in most cases. For details refer to +the script itself. We will mention just one option, needed when you follow the +suggestion to separate database and samples between flash storage and a hard disk. +If you have the hard disk mounted on, lets say `/media/nvr`, and you want to +store the video samples inside a directory named `samples` there, you would set: + + SAMPLES_DIR=/media/nvr/samples + +The script will perform all necessary steps to leave you with a fully built, +installed moonfire-nvr binary. The only thing +you'll have to do manually is add your camera configuration(s) to the database. +Alternatively, before running the script, you can create a file named `cameras.sql` +in the same directory as the `prep.sh` script and it will be automatically +included for you. +For instructions, you can skip to "[Camera configuration and hard disk mounting](#camera)". + +Once prerequisites are installed, Moonfire NVR can be built as follows: + + $ cargo test + $ cargo build --release + $ sudo install -m 755 target/release/moonfire-nvr /usr/local/bin + +## Further configuration + +Moonfire NVR should be run under a dedicated user. It keeps two kinds of +state: + + * a SQLite database, typically <1 GiB. It should be stored on flash if + available. + * the "sample file directory", which holds the actual samples/frames of + H.264 video. This should be quite large and typically is stored on a hard + drive. + +(See [schema.md](schema.md) for more information.) + +Both kinds of state are intended to be accessed only by Moonfire NVR itself. +However, the interface for adding new cameras is not yet written, so you will +have to manually insert cameras with the `sqlite3` command line tool prior to +starting Moonfire NVR. + +Manual commands would look something like this: + + $ sudo addgroup --system moonfire-nvr + $ sudo adduser --system moonfire-nvr --home /var/lib/moonfire-nvr + $ sudo mkdir /var/lib/moonfire-nvr + $ sudo -u moonfire-nvr -H mkdir db sample + $ sudo -u moonfire-nvr moonfire-nvr init + +### Camera configuration and hard drive mounting + +If a dedicated hard drive is available, set up the mount point: + + $ sudo vim /etc/fstab + $ sudo mount /var/lib/moonfire-nvr/sample + +Once setup is complete, it is time to add camera configurations to the +database. If the daemon is running, you will need to stop it temporarily: + + $ sudo systemctl stop moonfire-nvr + +You can configure the system through a text-based user interface: + + $ sudo -u moonfire-nvr moonfire-nvr config 2>debug-log + +In the user interface, add your cameras under the "Edit cameras" dialog. +There's a "Test" button to verify your settings directly from the dialog. + +After the cameras look correct, go to "Edit retention" to assign disk space to +each camera. Leave a little slack (at least 100 MB per camera) between the total +limit and the filesystem capacity, even if you store nothing else on the disk. +There are several reasons this is needed: + + * The limit currently controls fully-written files only. There will be up + to two minutes of video per camera of additional video. + * The rotation happens after the limit is exceeded, not proactively. + * Moonfire NVR currently doesn't account for the unused space in the final + filesystem block at the end of each file. + * Moonfire NVR doesn't account for the space used for directory listings. + * If a file is open when it is deleted (such as if a HTTP client is + downloading it), it stays around until the file is closed. Moonfire NVR + currently doesn't account for this. + +When finished, start the daemon: + + $ sudo systemctl start moonfire-nvr + +### System Service + +Moonfire NVR can be run as a systemd service. If you used `prep.sh` this has +been done for you. If not, Create +`/etc/systemd/system/moonfire-nvr.service`: + + [Unit] + Description=Moonfire NVR + After=network-online.target + + [Service] + ExecStart=/usr/local/bin/moonfire-nvr run \ + --sample-file-dir=/var/lib/moonfire-nvr/sample \ + --db-dir=/var/lib/moonfire-nvr/db \ + --http-addr=0.0.0.0:8080 + Environment=TZ=:/etc/localtime + Environment=MOONFIRE_FORMAT=google-systemd + Environment=MOONFIRE_LOG=info + Type=simple + User=moonfire-nvr + Nice=-20 + Restart=on-abnormal + CPUAccounting=true + MemoryAccounting=true + BlockIOAccounting=true + + [Install] + WantedBy=multi-user.target + +Note that the HTTP port currently has no authentication, encryption, or +logging; it should not be directly exposed to the Internet. + +Complete the installation through `systemctl` commands: + + $ sudo systemctl daemon-reload + $ sudo systemctl start moonfire-nvr + $ sudo systemctl status moonfire-nvr + $ sudo systemctl enable moonfire-nvr + +See the [systemd](http://www.freedesktop.org/wiki/Software/systemd/) +documentation for more information. The [manual +pages](http://www.freedesktop.org/software/systemd/man/) for `systemd.service` +and `systemctl` may be of particular interest. diff --git a/guide/troubleshooting.md b/guide/troubleshooting.md new file mode 100644 index 0000000..1538a68 --- /dev/null +++ b/guide/troubleshooting.md @@ -0,0 +1,70 @@ +# Troubleshooting + +## Logs + +While Moonfire NVR is running, logs will be written to stderr. + + * When running `moonfire-nvr config`, you typically should redirect stderr + to a text file to avoid poor interaction between the interactive stdout + output and the logging. + * When running through systemd, stderr will be redirected to the journal. + Try `sudo journalctl --unit moonfire-nvr` to view the logs. You also + likely want to set `MOONFIRE_FORMAT=google-systemd` to format logs as + expected by systemd. + +Logging options are controlled by environmental variables: + + * `MOONFIRE_LOG` controls the log level. Its format is similar to the + `RUST_LOG` variable used by the + [env-logger](http://rust-lang-nursery.github.io/log/env_logger/) crate. + `MOONFIRE_LOG=info` is the default. + `MOONFIRE_LOG=info,moonfire_nvr=debug` gives more detailed logging of the + `moonfire_nvr` crate itself. + * `MOONFIRE_FORMAT` selects the output format. The two options currently + accepted are `google` (the default, like the Google + [glog](https://github.com/google/glog) package) and `google-systemd` (a + variation for better systemd compatibility). + +## Problems + +### `Error: pts not monotonically increasing; got 26615520 then 26539470` + +If your streams cut out with an error message like this one, there are a +couple possibilities. + +One is that your camera outputs [B +frames](https://en.wikipedia.org/wiki/Video_compression_picture_types#Bi-directional_predicted_.28B.29_frames.2Fslices_.28macroblocks.29). +If you believe this is the case, file a feature request; Moonfire NVR +currently doesn't support B frames. You may be able to configure your camera +to disable B frames in the meantime. + +A more subtle problem occurs in cameras such as the Dahua Starlight series +when the following is true: + + * Audio is enabled (thus a single RTSP session has two streams). + * The camera's clock changes abruptly. Note that many cameras use SNTP + rather than NTP to adjust time, so they consistently step time rather + than slew it. + * They send RTCP Sender Reports (these include the NTP time). + +Moonfire NVR currently uses the ffmpeg library to talk to the cameras. ffmpeg +doesn't properly support this situation. It uses the NTP time to adjust the +PTS and DTS, and thus experiences jumps forward and backward. The forward +jumps cause one frame to be artificially lengthened. The backward jumps create +an impossible situation which causes Moonfire NVR to abort the session and +retry. + +In the long term, Moonfire NVR will likely implement its own RTSP support. + +In the short term, you can use either of two workarounds: + + * Disable audio in the camera settings. Note that Moonfire NVR doesn't + yet support recording audio anyway. + * Disable time adjustment. You'll likely want to disable in-picture + timestamps as well as they will become untrustworthy. + +### `moonfire-nvr config` displays garbage + +This happens if your machine is configured to a non-UTF-8 locale, due to +gyscos/Cursive#13. As a workaround, type `export LC_ALL=en_US.UTF-8` prior to +running `moonfire-nvr config`.