MyFavMirrors/moonfire-nvr
1
0
Fork 0
mirror of https://github.com/scottlamb/moonfire-nvr.git synced 2024-12-26 07:05:56 -05:00
Code Issues Packages Projects Releases Wiki Activity

Add a simple JSON API.

Browse Source 
This is a work in progress. There are no tests yet.
This commit is contained in:
Scott Lamb 2016-04-23 13:55:36 -07:00
parent 8ab2edb970
commit 5dd0dca51f
8 changed files with 358 additions and 43 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
CMakeLists.txt
View File

@ -59,6 +59,7 @@ find_library(PROFILER_LIBRARIES profiler)
find_package(PkgConfig)
pkg_check_modules(FFMPEG REQUIRED libavutil libavcodec libavformat)
pkg_check_modules(LIBEVENT REQUIRED libevent>=2.1)
pkg_check_modules(JSONCPP REQUIRED jsoncpp)
pkg_check_modules(GLOG REQUIRED libglog)
pkg_check_modules(OPENSSL REQUIRED libcrypto)
pkg_check_modules(SQLITE REQUIRED sqlite3)

1
README.md
View File

@ -91,6 +91,7 @@ pre-requisites (see also the `Build-Depends` field in `debian/control`):
libgflags-dev \
libgoogle-glog-dev \
libgoogle-perftools-dev \
libjsoncpp-dev \
libre2-dev \
sqlite3 \
libsqlite3-dev \

16
debian/control vendored
View File

@ -3,10 +3,20 @@ Maintainer: Scott Lamb <slamb@slamb.org>
Section: video
Priority: optional
Standards-Version: 3.9.6.1
Build-Depends: debhelper (>= 9), dh-systemd, cmake, libprotobuf-dev, libavcodec-dev, libavformat-dev, libevent-dev (>= 2.1), libgflags-dev, libgoogle-glog-dev, libgoogle-perftools-dev, libre2-dev, pkgconf, protobuf-compiler, uuid-dev, libsqlite3-dev
Build-Depends: cmake,
libavcodec-dev,
libavformat-dev,
libavutil-dev,
libgflags-dev,
libgoogle-glog-dev,
libgoogle-perftools-dev,
libjsoncpp-dev,
libre2-dev,
libsqlite3-dev,
pkgconf,
uuid-dev
Package: moonfire-nvr
Architecture: any
Depends: ${shlibs:Depends}, ${misc:Depends}, adduser
Depends: ${shlibs:Depends}, ${misc:Depends}, adduser, sqlite3, uuid-runtime
Description: security camera network video recorder
moonfire-nvr records video files from IP security cameras.

159
design/api.md Normal file
View File

@ -0,0 +1,159 @@
# Moonfire NVR API
Status: **unstable**. This is an early draft; the API may change without
warning.
## Objective
Allow a JavaScript-based web interface to list cameras and view recordings.
In the future, this is likely to be expanded:
* configuration support
* commandline tool over a UNIX-domain socket
(at least for bootstrapping web authentication)
* mobile interface
## Detailed design
All requests for JSON data should be sent with the header `Accept:
application/json` (exactly). Without this header, replies will generally be in
HTML rather than JSON.
TODO(slamb): authentication.
### `/cameras`
A `GET` request on this URL returns basic information about all cameras. The
`application/json` response will have a top-level `cameras` with a list of
attributes about each camera:
* `uuid`: in text format
* `short\_name`: a short name (typically one or two words)
* `description`: a longer description (typically a phrase or paragraph)
* `retain\_bytes`: the configured total number of bytes of completed
recordings to retain.
* `min\_start\_time\_90k`: the start time of the earliest recording for this
camera, in 90kHz units since 1970-01-01 00:00:00 UTC.
* `max\_end\_time\_90k`: the end time of the latest recording for this
camera, in 90kHz units since 1970-01-01 00:00:00 UTC.
* `total\_duration\_90k`: the total duration recorded, in 90 kHz units.
This is no greater than `max\_end\_time\_90k - max\_start\_time\_90k`; it
will be lesser if there are gaps in the recorded data.
* `total\_sample\_file\_bytes`: the total number of bytes of sample data (the
`mdat` portion of a `.mp4` file).
Example response:
```json
{
"cameras": [
{
"uuid": "fd20f7a2-9d69-4cb3-94ed-d51a20c3edfe",
"short_name": "driveway",
"description": "Hikvision DS-2CD2032 overlooking the driveway from east",
"retain_bytes": 536870912000,
"min_start_time_90k": 130888729442361,
"max_end_time_90k": 130985466591817
"total_duration_90k": 96736169725,
"total_sample_file_bytes": 446774393937,
},
...
],
}
```
### `/cameras/<uuid>`
A GET returns information for the camera with the given URL. The information
returned is a superset of that returned by the camera list.
TODO(slamb): this should likely return a list of calendar days with data in the
server's time zone, along with the associated `start\_time\_90k` and
`end\_time\_90k`. The server will calculate this on startup and maintain it
as recordings are updated.
### `/camera/<uuid>/recordings`
A GET returns information about recordings, in descending order.
TODO(slamb): once we support annotations, should they be included in the same
URI or as a separate `/annotations`?
TODO(slamb): this should support paging. The client can limit the range via
the URI parameters `start\_time\_90k` and `end\_time\_90k`. If the range is
too large, the server will return some fraction of the data along with a
continuation key to pass in for the next request.
TODO(slamb): There might be some irregularity in the order if there are
overlapping recordings (such as if the server's clock jumped while running)
but I haven't thought about the details. In general, I'm not really sure how
to handle this case, other than ideally to keep recording stuff no matter what
and present some UI to help the user to fix it after the
fact.
In the property `recordings`, returns a list of recordings. Each recording
object has the following properties:
* `start\_time\_90k`
* `end\_time\_90k`
* `sample\_file\_bytes`
* `video\_sample\_entry\_sha1`
* `video\_sample\_entry\_width`
* `video\_sample\_entry\_height`
TODO(slamb): consider ways to reduce the data size; this is in theory quite
compressible but I'm not sure how effective gzip will be without some tweaks.
One simple approach would be to just combine some adjacent list entries if
one's start matches the other's end exactly and the `video\_sample\_entry\_*`
parameters are the same. So you might get one entry that represents 2 hours of
video instead of 120 entries representing a minute each.
Example request URI (with added whitespace between parameters):
```
/camera/fd20f7a2-9d69-4cb3-94ed-d51a20c3edfe/recordings
?start_time_90k=130888729442361
&end_time_90k=130985466591817
```
Example response:
```json
{
"recordings": [
{
"end_time_90k": 130985466591817,
"start_time_90k": 130985461191810,
"sample_file_bytes": 8405564,
"video_sample_entry_sha1": "81710c9c51a02cc95439caa8dd3bc12b77ffe767",
"video_sample_entry_width": 1280,
"video_sample_entry_height": 720,
},
{
"end_time_90k": 130985461191810,
...
},
...
],
"continuation_key": "<opaque blob>",
}
```
### `/camera/<uuid>/view.mp4`
A GET returns a .mp4 file, with an etag and support for range requests.
Expected query parameters:
* `start\_time\_90k`
* `end\_time\_90k`
* TODO(slamb): possibly `overlap` to indicate what to do about segments of
recording with overlapping wall times. Values might include:
* `error` (return an HTTP error)
* `include_all` (include all, in order of the recording ids)
* `include_latest` (include only the latest by recording id for a
particular segment of time)
* TODO(slamb): gaps allowed or not? maybe a parameter for this also?
* TODO(slamb): parameter to indicate if a caption track should be included
with timestamps?

1
prep.sh
View File

@ -148,6 +148,7 @@ if [ "${SKIP_APT:-0}" != 1 ]; then
libgflags-dev \
libgoogle-glog-dev \
libgoogle-perftools-dev \
libjsoncpp-dev \
libre2-dev \
sqlite3 \
libsqlite3-dev \

4
src/CMakeLists.txt
View File

@ -28,13 +28,15 @@
# along with this program. If not, see <http://www.gnu.org/licenses/>.
include_directories(${CMAKE_CURRENT_BINARY_DIR})
include_directories(${JSONCPP_INCLUDE_DIRS})
set(MOONFIRE_DEPS
${CMAKE_THREAD_LIBS_INIT}
${FFMPEG_LIBRARIES}
${LIBEVENT_LIBRARIES}
${GFLAGS_LIBRARIES}
${GLOG_LIBRARIES}
${JSONCPP_LIBRARIES}
${LIBEVENT_LIBRARIES}
${OPENSSL_LIBRARIES}
${PROFILER_LIBRARIES}
${RE2_LIBRARIES}

196
src/web.cc
View File

@ -33,20 +33,77 @@
#include "web.h"
#include <glog/logging.h>
#include <json/value.h>
#include <json/writer.h>
#include <re2/re2.h>
#include "recording.h"
#include "string.h"
namespace moonfire_nvr {
void WebInterface::Register(evhttp *http) {
evhttp_set_cb(http, "/", &WebInterface::HandleCameraList, this);
evhttp_set_cb(http, "/camera", &WebInterface::HandleCameraDetail, this);
evhttp_set_cb(http, "/view.mp4", &WebInterface::HandleMp4View, this);
namespace {
static const char kJsonMimeType[] = "application/json";
void ReplyWithJson(evhttp_request *req, const Json::Value &value) {
EvBuffer buf;
buf.Add(Json::writeString(Json::StreamWriterBuilder(), value));
evhttp_add_header(evhttp_request_get_output_headers(req), "Content-Type",
kJsonMimeType);
evhttp_send_reply(req, HTTP_OK, "OK", buf.get());
}
void WebInterface::HandleCameraList(evhttp_request *req, void *arg) {
// RE2::Arg::Parser for uuids.
bool ParseUuid(const char *str, int n, void *dest) {
auto *uuid = reinterpret_cast<Uuid *>(dest);
return uuid->ParseText(re2::StringPiece(str, n));
}
} // namespace
void WebInterface::Register(evhttp *http) {
evhttp_set_gencb(http, &WebInterface::DispatchHttpRequest, this);
}
void WebInterface::DispatchHttpRequest(evhttp_request *req, void *arg) {
static const RE2 kCameraUri("/cameras/([^/]+)/");
static const RE2 kCameraRecordingsUri("/cameras/([^/]+)/recordings");
static const RE2 kCameraViewUri("/cameras/([^/]+)/view.mp4");
re2::StringPiece accept =
evhttp_find_header(evhttp_request_get_input_headers(req), "Accept");
bool json = accept == kJsonMimeType;
auto *this_ = reinterpret_cast<WebInterface *>(arg);
const evhttp_uri *uri = evhttp_request_get_evhttp_uri(req);
re2::StringPiece path = evhttp_uri_get_path(uri);
Uuid camera_uuid;
RE2::Arg camera_uuid_arg(&camera_uuid, &ParseUuid);
if (path == "/" || path == "/cameras/") {
if (json) {
this_->HandleJsonCameraList(req);
} else {
this_->HandleHtmlCameraList(req);
}
} else if (RE2::FullMatch(path, kCameraUri, camera_uuid_arg)) {
if (json) {
this_->HandleJsonCameraDetail(req, camera_uuid);
} else {
this_->HandleHtmlCameraDetail(req, camera_uuid);
}
} else if (RE2::FullMatch(path, kCameraRecordingsUri, camera_uuid_arg)) {
// The HTML version includes this in the top-level camera view.
// So only support JSON at this URI.
this_->HandleJsonCameraRecordings(req, camera_uuid);
} else if (RE2::FullMatch(path, kCameraViewUri, camera_uuid_arg)) {
this_->HandleMp4View(req, camera_uuid);
} else {
evhttp_send_error(req, HTTP_NOTFOUND, "path not understood");
}
}
void WebInterface::HandleHtmlCameraList(evhttp_request *req) {
EvBuffer buf;
buf.Add(
"<!DOCTYPE html>\n"
@ -70,7 +127,7 @@ void WebInterface::HandleCameraList(evhttp_request *req, void *arg) {
? std::string("n/a")
: PrettyTimestamp(row.max_end_time_90k);
buf.AddPrintf(
"<tr class=header><td colspan=2><a href=\"/camera?uuid=%s\">%s</a>"
"<tr class=header><td colspan=2><a href=\"/cameras/%s/\">%s</a>"
"</td></tr>\n"
"<tr><td>description</td><td>%s</td></tr>\n"
"<tr><td>space</td><td>%s / %s (%.1f%%)</td></tr>\n"
@ -90,7 +147,7 @@ void WebInterface::HandleCameraList(evhttp_request *req, void *arg) {
EscapeHtml(HumanizeDuration(seconds)).c_str());
return IterationControl::kContinue;
};
this_->env_->mdb->ListCameras(row_cb);
env_->mdb->ListCameras(row_cb);
buf.Add(
"</table>\n"
"</body>\n"
@ -98,17 +155,37 @@ void WebInterface::HandleCameraList(evhttp_request *req, void *arg) {
evhttp_send_reply(req, HTTP_OK, "OK", buf.get());
}
void WebInterface::HandleCameraDetail(evhttp_request *req, void *arg) {
auto *this_ = reinterpret_cast<WebInterface *>(arg);
Uuid camera_uuid;
QueryParameters params(evhttp_request_get_uri(req));
if (!params.ok() || !camera_uuid.ParseText(params.Get("uuid"))) {
return evhttp_send_error(req, HTTP_BADREQUEST, "bad query parameters");
}
void WebInterface::HandleJsonCameraList(evhttp_request *req) {
Json::Value cameras(Json::arrayValue);
auto row_cb = [&](const ListCamerasRow &row) {
Json::Value camera(Json::objectValue);
camera["uuid"] = row.uuid.UnparseText();
camera["short_name"] = row.short_name;
camera["description"] = row.description;
camera["retain_bytes"] = static_cast<Json::Int64>(row.retain_bytes);
camera["total_duration_90k"] =
static_cast<Json::Int64>(row.total_duration_90k);
camera["total_sample_file_bytes"] =
static_cast<Json::Int64>(row.total_sample_file_bytes);
if (row.min_start_time_90k != -1) {
camera["min_start_time_90k"] =
static_cast<Json::Int64>(row.min_start_time_90k);
}
if (row.max_end_time_90k != -1) {
camera["max_end_time_90k"] =
static_cast<Json::Int64>(row.max_end_time_90k);
}
cameras.append(camera);
return IterationControl::kContinue;
};
env_->mdb->ListCameras(row_cb);
ReplyWithJson(req, cameras);
}
void WebInterface::HandleHtmlCameraDetail(evhttp_request *req,
Uuid camera_uuid) {
GetCameraRow camera_row;
if (!this_->env_->mdb->GetCamera(camera_uuid, &camera_row)) {
if (!env_->mdb->GetCamera(camera_uuid, &camera_row)) {
return evhttp_send_error(req, HTTP_NOTFOUND, "no such camera");
}
@ -148,12 +225,11 @@ void WebInterface::HandleCameraDetail(evhttp_request *req, void *arg) {
aggregated.start_time_90k) /
kTimeUnitsPerSecond;
buf.AddPrintf(
"<tr><td><a href=\"/view.mp4?camera_uuid=%s&start_time_90k=%" PRId64
"<tr><td><a href=\"view.mp4?start_time_90k=%" PRId64
"&end_time_90k=%" PRId64
"\">%s</a></td><td>%s</td><td>%dx%d</td>"
"<td>%.0f</td><td>%s</td><td>%s</td></tr>\n",
camera_uuid.UnparseText().c_str(), aggregated.start_time_90k,
aggregated.end_time_90k,
aggregated.start_time_90k, aggregated.end_time_90k,
PrettyTimestamp(aggregated.start_time_90k).c_str(),
PrettyTimestamp(aggregated.end_time_90k).c_str(),
static_cast<int>(aggregated.width), static_cast<int>(aggregated.height),
@ -183,9 +259,9 @@ void WebInterface::HandleCameraDetail(evhttp_request *req, void *arg) {
int64_t start_time_90k = 0;
int64_t end_time_90k = std::numeric_limits<int64_t>::max();
std::string error_message;
if (!this_->env_->mdb->ListCameraRecordings(camera_uuid, start_time_90k,
end_time_90k, handle_sql_row,
&error_message)) {
if (!env_->mdb->ListCameraRecordings(camera_uuid, start_time_90k,
end_time_90k, handle_sql_row,
&error_message)) {
return evhttp_send_error(
req, HTTP_INTERNAL,
StrCat("sqlite query failed: ", EscapeHtml(error_message)).c_str());
@ -197,14 +273,78 @@ void WebInterface::HandleCameraDetail(evhttp_request *req, void *arg) {
evhttp_send_reply(req, HTTP_OK, "OK", buf.get());
}
void WebInterface::HandleMp4View(evhttp_request *req, void *arg) {
auto *this_ = reinterpret_cast<WebInterface *>(arg);
void WebInterface::HandleJsonCameraDetail(evhttp_request *req,
Uuid camera_uuid) {
GetCameraRow camera_row;
if (!env_->mdb->GetCamera(camera_uuid, &camera_row)) {
return evhttp_send_error(req, HTTP_NOTFOUND, "no such camera");
}
Uuid camera_uuid;
Json::Value camera(Json::objectValue);
camera["short_name"] = camera_row.short_name;
camera["description"] = camera_row.description;
camera["retain_bytes"] = static_cast<Json::Int64>(camera_row.retain_bytes);
camera["total_duration_90k"] =
static_cast<Json::Int64>(camera_row.total_duration_90k);
camera["total_sample_file_bytes"] =
static_cast<Json::Int64>(camera_row.total_sample_file_bytes);
if (camera_row.min_start_time_90k != -1) {
camera["min_start_time_90k"] =
static_cast<Json::Int64>(camera_row.min_start_time_90k);
}
if (camera_row.max_end_time_90k != -1) {
camera["max_end_time_90k"] =
static_cast<Json::Int64>(camera_row.max_end_time_90k);
}
// TODO(slamb): include list of calendar days with data.
ReplyWithJson(req, camera);
}
void WebInterface::HandleJsonCameraRecordings(evhttp_request *req,
Uuid camera_uuid) {
GetCameraRow camera_row;
if (!env_->mdb->GetCamera(camera_uuid, &camera_row)) {
return evhttp_send_error(req, HTTP_NOTFOUND, "no such camera");
}
// TODO(slamb): paging support.
Json::Value recordings(Json::arrayValue);
auto handle_row = [&](const ListCameraRecordingsRow &row) {
Json::Value recording(Json::objectValue);
recording["end_time_90k"] = static_cast<Json::Int64>(row.end_time_90k);
recording["start_time_90k"] = static_cast<Json::Int64>(row.start_time_90k);
recording["video_samples"] = static_cast<Json::Int64>(row.video_samples);
recording["sample_file_bytes"] =
static_cast<Json::Int64>(row.sample_file_bytes);
recording["video_sample_entry_sha1"] = ToHex(row.video_sample_entry_sha1);
recording["video_sample_entry_width"] = row.width;
recording["video_sample_entry_height"] = row.height;
recordings.append(recording);
return IterationControl::kContinue;
};
int64_t start_time_90k = 0;
int64_t end_time_90k = std::numeric_limits<int64_t>::max();
std::string error_message;
if (!env_->mdb->ListCameraRecordings(camera_uuid, start_time_90k,
end_time_90k, handle_row,
&error_message)) {
return evhttp_send_error(
req, HTTP_INTERNAL,
StrCat("sqlite query failed: ", EscapeHtml(error_message)).c_str());
}
Json::Value response(Json::objectValue);
response["recordings"] = recordings;
ReplyWithJson(req, response);
}
void WebInterface::HandleMp4View(evhttp_request *req, Uuid camera_uuid) {
int64_t start_time_90k;
int64_t end_time_90k;
QueryParameters params(evhttp_request_get_uri(req));
if (!params.ok() || !camera_uuid.ParseText(params.Get("camera_uuid")) ||
if (!params.ok() ||
!Atoi64(params.Get("start_time_90k"), 10, &start_time_90k) ||
!Atoi64(params.Get("end_time_90k"), 10, &end_time_90k) ||
start_time_90k < 0 || start_time_90k >= end_time_90k) {
@ -212,8 +352,8 @@ void WebInterface::HandleMp4View(evhttp_request *req, void *arg) {
}
std::string error_message;
auto file = this_->BuildMp4(camera_uuid, start_time_90k, end_time_90k,
&error_message);
auto file =
BuildMp4(camera_uuid, start_time_90k, end_time_90k, &error_message);
if (file == nullptr) {
// TODO: more nuanced HTTP status codes.
LOG(WARNING) << "BuildMp4 failed: " << error_message;

23
src/web.h
View File

@ -28,15 +28,11 @@
// You should have received a copy of the GNU General Public License
// along with this program. If not, see <http://www.gnu.org/licenses/>.
//
// web.h: web (HTTP/HTML) interface to the SQLite-based recording schema.
// Currently, during the transition from the old bunch-of-.mp4-files schema to
// the SQLite-based schema, it's convenient for this to be a separate class
// that interacts with the recording system only through the SQLite database
// and filesystem. In fact, the only advantage of being in-process is that it
// shares the same database mutex and avoids hitting SQLITE_BUSY.
// web.h: web (HTTP/HTML/JSON) interface to the SQLite-based recording schema.
// See design/api.md for a description of the JSON API.
//
// In the future, the interface will be reworked for tighter integration to
// support more features:
// In the future, the interface will be reworked for tighter integration with
// the recording system to support more features:
//
// * including the recording currently being written in the web interface
// * subscribing to changes
@ -67,9 +63,14 @@ class WebInterface {
void Register(evhttp *http);
private:
static void HandleCameraList(evhttp_request *req, void *arg);
static void HandleCameraDetail(evhttp_request *req, void *arg);
static void HandleMp4View(evhttp_request *req, void *arg);
static void DispatchHttpRequest(evhttp_request *req, void *arg);
void HandleHtmlCameraList(evhttp_request *req);
void HandleJsonCameraList(evhttp_request *req);
void HandleHtmlCameraDetail(evhttp_request *req, Uuid camera_uuid);
void HandleJsonCameraDetail(evhttp_request *req, Uuid camera_uuid);
void HandleJsonCameraRecordings(evhttp_request *req, Uuid camera_uuid);
void HandleMp4View(evhttp_request *req, Uuid camera_uuid);
// TODO: more nuanced error code for HTTP.
std::shared_ptr<VirtualFile> BuildMp4(Uuid camera_uuid,