From e7456643cdc47c3e377307d720d270f7616db1bf Mon Sep 17 00:00:00 2001 From: Dolf Starreveld Date: Sun, 7 Feb 2016 22:59:29 -0800 Subject: [PATCH] Added prep.sh script for automated builds * Changed README.md commensurately * Add cameras.sql to .gitignore to not commit personal camera data * Change CMakeLists.txt to explicitly refer to hand-built libevent dirs --- .gitignore | 1 + README.md | 106 ++++++++++++++----- prep.sh | 254 +++++++++++++++++++++++++++++++++++++++++++++ src/CMakeLists.txt | 4 +- 4 files changed, 340 insertions(+), 25 deletions(-) create mode 100755 prep.sh diff --git a/.gitignore b/.gitignore index 3dd51ca..42b5fb9 100644 --- a/.gitignore +++ b/.gitignore @@ -2,6 +2,7 @@ build debug obj-* +cameras.sql debian/files debian/moonfire-nvr.debhelper.log debian/moonfire-nvr.postinst.debhelper diff --git a/README.md b/README.md index 9d1bb05..3872a3d 100644 --- a/README.md +++ b/README.md @@ -47,7 +47,7 @@ make this possible: See the [github page](https://github.com/scottlamb/moonfire-nvr) (in case you're not reading this text there already). You can download the bleeding -edge version from the commandline via git: +edge version from the command line via git: $ git clone https://github.com/scottlamb/moonfire-nvr.git @@ -69,7 +69,7 @@ from source. It requires several packages to build: [nghttp2](https://github.com/tatsuhiro-t/nghttp2) in the future.) Unfortunately, the libevent 2.0 bundled with current Debian releases is unsuitable. -* [gflags](http://gflags.github.io/gflags/), for commandline flag parsing. +* [gflags](http://gflags.github.io/gflags/), for command line flag parsing. * [glog](https://github.com/google/glog), for debug logging. * [gperftools](https://github.com/gperftools/gperftools), for debugging. * [googletest](https://github.com/google/googletest), for automated testing. @@ -92,14 +92,50 @@ pre-requisites (see also the `Build-Depends` field in `debian/control`): libgoogle-glog-dev \ libgoogle-perftools-dev \ libre2-dev \ + sqlite3 \ libsqlite3-dev \ pkgconf \ + uuid-runtime \ uuid-dev libevent 2.1 will have to be installed from source. In the future, this dependency may be replaced or support may be added for automatically building libevent in-tree to avoid the inconvenience. +uuid-runtime is only necessary if you wish to use the uuid command to generate +uuids for your cameras (see below). If you obtain them elsewhere, you can skip this +package. + +You can continue to follow the build/install instructions below for a manual +build and install, or alternatively you can run the prep script called `prep.sh`. + + $ cd moonfire-nvr + $ ./prep.sh + +The script will take the following command line options, should you need them: + +* `-E`: Forcibly purge all existing libevent packages. You would only do this + if there is some apparent conflict (see remarks about building libevent + from source). +* `-f`: Force a build even if the binary appears to be installed. This can be useful + on repeat builds. +* `-S`: Skip updating and installing dependencies through apt-get. This too can be + useful on repeated builds. + +You can edit variables at the start of the script to influence names and +directories, but defaults should suffice in most cases. For details refer to +the script itself. We will mention just one option, needed when you follow the +suggestion to separate database and samples between flash storage and a hard disk. +If you have the hard disk mounted on, lets say `/media/nvr`, and you want to +store the video samples inside a directory named `samples` there, you would set: + + SAMPLES_DIR=/media/nvr/samples + +The script will perform all necessary steps to leave you with a fully built, +installed moonfire-nvr binary and (running) system service. The only thing +you'll have to do manually is add your camera configuration(s) to the database. +For instructions, you can skip to "[Camera configuration and hard disk mounting](#camera)". + Once prerequisites are installed, Moonfire NVR can be built as follows: $ mkdir build @@ -114,23 +150,43 @@ installed, you may be able to prepare a `.deb` package: $ sudo apt-get install devscripts dh-systemd $ debuild -us -uc -# Installation +# Further configuration Moonfire NVR should be run under a dedicated user. It keeps two kinds of state: -* a SQLite3 database, typically <1 GiB. It should be stored on flash if +* a SQLite database, typically <1 GiB. It should be stored on flash if available. * the "sample file directory", which holds the actual samples/frames of H.264 video. This should be quite large and typically is stored on a hard drive. Both are intended to be accessed only by Moonfire NVR itself. However, the interface for adding new cameras is not yet written, so you will have to -manually create the database and insert cameras with the `sqlite3` commandline +manually create the database and insert cameras with the `sqlite3` command line tool prior to starting Moonfire NVR. +Manual commands would look something like this: + + $ sudo addgroup --system moonfire-nvr + $ sudo adduser --system moonfire-nvr --home /var/lib/moonfire-nvr + $ sudo mkdir /var/lib/moonfire-nvr + $ sudo -u moonfire-nvr -H mkdir db sample + $ sudo -u moonfire-nvr sqlite3 ~moonfire-nvr/db/db < path/to/schema.sql + +## Camera configuration and hard drive mounting + +If a dedicated hard drive is available, set up the mount point: + + $ sudo vim /etc/fstab + $ sudo mount /var/lib/moonfire-nvr/sample + +Once setup is complete, it is time to add camera configurations to the +database. However, the interface for adding new cameras is not yet written, +so you will have to manually insert cameras configurations with the `sqlite3` +command line tool prior to starting Moonfire NVR. + Before setting up a camera, it may be helpful to test settings with the -`ffmpeg` commandline tool: +`ffmpeg` command line tool: $ ffmpeg \ -i "rtsp://admin:12345@192.168.1.101:554/Streaming/Channels/1" \ @@ -140,16 +196,21 @@ Before setting up a camera, it may be helpful to test settings with the -flags:v +global_header \ test.mp4 -Once you have a working `ffmpeg` commandline, set up Moonfire NVR as follows: +Once you have a working `ffmpeg` command line, insert the camera config as +follows. See the schema SQL file's comments for more information. +Note that the sum of `retain_bytes` for all cameras combined should be +somewhat less than the available bytes on the sample file directory's +filesystem, as the currently-writing sample files are not included in +this sum. Be sure also to subtract out the filesystem's reserve for root +(typically 5%). + +In the following example, we generate a uuid which is then later used +to uniquely identify this camera. Thus, you will generate a new one for +each camera you insert using this method. - $ sudo addgroup --system moonfire-nvr - $ sudo adduser --system moonfire-nvr --home /var/lib/moonfire-nvr - $ sudo mkdir /var/lib/moonfire-nvr - $ sudo -u moonfire-nvr -H mkdir db sample $ uuidgen | sed -e 's/-//g' b47f48706d91414591cd6c931bf836b4 - $ sudo -u moonfire-nvr sqlite3 db/db - sqlite3> .read path/to/schema.sql + $ sudo -u moonfire-nvr sqlite3 ~moonfire-nvr/db/db sqlite3> insert into camera ( ...> uuid, short_name, description, host, username, password, ...> main_rtsp_path, sub_rtsp_path, retain_bytes) values ( @@ -159,18 +220,10 @@ Once you have a working `ffmpeg` commandline, set up Moonfire NVR as follows: ...> '/Streaming/Channels/2', 104857600); sqlite3> ^D -See the schema SQL file's comments for more information. Note that the sum of -`retain_bytes` for all cameras should be somewhat less than the available -bytes on the sample file directory's filesystem, as the currently-writing -sample files are not included in this sum. Be sure also to subtract out the -filesystem's reserve for root (typically 5%). +## System Service -If a dedicated hard drive is available, set up the mount point: - - $ sudo vim /etc/fstab - $ sudo mount /var/lib/moonfire-nvr/sample - -Moonfire NVR can be run as a systemd service. Create +Moonfire NVR can be run as a systemd service. If you used `prep.sh` this has +been done for you. If not, Create `/etc/systemd/system/moonfire-nvr.service`: [Unit] @@ -208,7 +261,12 @@ documentation for more information. The [manual pages](http://www.freedesktop.org/software/systemd/man/) for `systemd.service` and `systemctl` may be of particular interest. +# Troubleshooting + While Moonfire NVR is running, logs will be written to `/tmp/moonfire-nvr.INFO`. +Also available will be `/tmp/moonfire-nvr.WARNING` and `/tmp/moonfire-nvr.ERROR`. +These latter to contain only warning or more serious messages, respectively only +error messages. # Getting help and getting involved diff --git a/prep.sh b/prep.sh new file mode 100755 index 0000000..460c5b6 --- /dev/null +++ b/prep.sh @@ -0,0 +1,254 @@ +#!/bin/bash +# +# This file is part of Moonfire NVR, a security camera network video recorder. +# Copyright (C) 2016 Scott Lamb +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# In addition, as a special exception, the copyright holders give +# permission to link the code of portions of this program with the +# OpenSSL library under certain conditions as described in each +# individual source file, and distribute linked combinations including +# the two. +# +# You must obey the GNU General Public License in all respects for all +# of the code used other than OpenSSL. If you modify file(s) with this +# exception, you may extend this exception to your version of the +# file(s), but you are not obligated to do so. If you do not wish to do +# so, delete this exception statement from your version. If you delete +# this exception statement from all source files in the program, then +# also delete it here. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +# +# +# Script to prepare for moonfire-nvr operations +# +# Command line options: +# -f: Force clean build, even if binary already installed +# -S: Skip apt-get update and install +# + +# Configuration variables. Should only need minimal, or zero, changes. +# Empty values will use defaults. +# + +# User and group +# Default: or whatever is in $NVR_USER (default "moonfire-nvr") +# +NVR_USER= +NVR_GROUP= + +# Port for web server +# Default: 8080 +# +NVR_PORT= + +# This should, ideally, be a location on flash storage under which the +# moonfire user's home directory will be created. +# Default: "/var/lib" +# +NVR_HOME_BASE= + +# Set to mountpoint of media directory, empty to stay in home directory +# Default: empty +SAMPLES_DIR= + +# Set to path for media directory relative to mountpoint +# Default: "samples" +# +SAMPLES_DIR_NAME= + +# Binary location +# Default: "/usr/local/bin/moonfire-nvr" +# +SERVICE_BIN= + +# Service name +# Default: "moonfire-nvr" +# +SERVICE_NAME= + +# Service Description +# Default: "Moonfire NVR" +# +SERVICE_DESC= + +# -------------------------------------------------------------------- +# Derived variables: Do not modify! +# +# Determine directory path of this script +# +SOURCE="${BASH_SOURCE[0]}" +while [ -h "$SOURCE" ]; do # resolve $SOURCE until file no longer a symlink + DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )" + SOURCE="$(readlink "$SOURCE")" + # if $SOURCE was relative symlink, resolve relative to path of symlink + [[ "$SOURCE" != /* ]] && SOURCE="$DIR/$SOURCE" +done + +SRC_DIR="$( cd -P "$( dirname "$SOURCE" )" && pwd )"/src +NVR_USER="${NVR_USER:-moonfire-nvr}" +NVR_GROUP="${NVR_GROUP:-$NVR_USER}" +NVR_PORT="${NVR_PORT:-8080}" +NVR_HOME_BASE="${NVR_HOME_BASE:-/var/lib}" +NVR_HOME="${NVR_HOME_BASE}/${NVR_USER}" +DB_NAME="${DB_NAME:-db}" +DB_DIR="${DB_DIR:-$NVR_HOME/db}" +SAMPLES_DIR_NAME="${SAMPLES_DIR_NAME:-samples}" +SAMPLES_DIR="${SAMPLES_DIR:-$NVR_HOME/$SAMPLES_DIR_NAME}" +SERVICE_NAME="${SERVICE_NAME:-moonfire-nvr}" +SERVICE_DESC="${SERVICE_DESC:-Moonfire NVR}" +SERVICE_BIN="${SERVICE_BIN:-/usr/local/bin/moonfire-nvr}" + +# Process command line options +# +while getopts ":DEfS" opt; do + case $opt in + D) SKIP_DB=1 + ;; + E) PURGE_LIBEVENT=1 + ;; + f) FORCE_BUILD=1 + ;; + S) SKIP_APT=1 + ;; + :) + echo "Option -$OPTARG requires an argument." >&2 + exit 1 + ;; + \?) + echo "Invalid option: -$OPTARG" >&2 + exit 1 + ;; + esac +done + +# Setup all packages we need +# +echo 'Preparing and downloading packages we need...'; echo +if [ "${SKIP_APT:-0}" != 1 ]; then + sudo apt-get update + [ "${PURGE_LIBEVENT:-0}" == 1] && sudo apt-get --purge remove libevent-* + sudo apt-get install \ + build-essential \ + cmake \ + libavcodec-dev \ + libavformat-dev \ + libavutil-dev \ + libgflags-dev \ + libgoogle-glog-dev \ + libgoogle-perftools-dev \ + libre2-dev \ + sqlite3 \ + libsqlite3-dev \ + pkgconf \ + uuid-runtime \ + uuid-dev +fi + +# Check if binary is installed. Setup for build if it is not +# +if [ ! -x "${SERVICE_BIN}" ]; then + echo "Binary not installed, building..."; echo + FORCE_BUILD=1 +fi + +# Build if requested +# +if [ "${FORCE_BUILD:-0}" -eq 1 ]; then + # Remove previous build, if any + [ -d build ] && rm -fr build 2>/dev/null + mkdir build; cd build + cmake .. && make && sudo make install + if [ -x "${SERVICE_BIN}" ]; then + echo "Binary installed..."; echo + else + echo "Build failed to install..."; echo + exit 1 + fi +fi + +# Create user and groups +# +echo 'Create user/group and directories we need...'; echo +sudo addgroup --quiet --system ${NVR_GROUP} +sudo adduser --quiet --system ${NVR_USER} --home "${NVR_HOME}" +if [ ! -d "${NVR_HOME}" ]; then + sudo mkdir "${NVR_HOME}" +fi +sudo chown ${NVR_USER}:${NVR_GROUP} "${NVR_HOME}" + +# Prepare samples directory +# +if [ -z "${SAMPLES_DIR}" ]; then + SAMPLES_PATH="${NVR_HOME}/${SAMPLES_DIR_NAME}" + if [ ! -d "${SAMPLES_PATH}" ]; then + sudo -u ${NVR_USER} -H mkdir "${SAMPLES_PATH}" + fi +else + SAMPLES_PATH="${SAMPLES_DIR}" + if [ ! -d "${SAMPLES_PATH}" ]; then + sudo mkdir "${SAMPLES_PATH}" + echo ">>> Do not forget to edit /etc/fstab to mount samples media"; echo + fi +fi + + +# Prepare for sqlite directory and set schema into db +# +echo 'Create database...'; echo +if [ ! -d "${DB_DIR}" ]; then + sudo -u ${NVR_USER} -H mkdir "${DB_DIR}" +fi +DB_PATH="${DB_DIR}/${DB_NAME}" +CAMERAS_PATH="${SRC_DIR}/../cameras.sql" +[ "${SKIP_DB:-0}" == 0 ] && sudo -u ${NVR_USER} -H sqlite3 "${DB_PATH}" < "${SRC_DIR}/schema.sql" +if [ -r "${CAMERAS_PATH}" ]; then + echo 'Add cameras...'; echo + sudo -u ${NVR_USER} -H sqlite3 "${DB_PATH}" < "${CAMERAS_PATH}" +fi + +# Prepare service files +# +SERVICE_PATH="/etc/systemd/system/${SERVICE_NAME}.service" +sudo sh -c "cat > ${SERVICE_PATH}" <