MyFavMirrors
/
moonfire-nvr
mirror of https://github.com/scottlamb/moonfire-nvr.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
moonfire-nvr/ref/api.md

977 lines
38 KiB
Markdown
Raw Normal View History

mass markdown reformatting Add tables of contents (using the VS Code Markdown All-In-One extension) and reformat lists to consistently use 4-space indents. No content changes.
2021-04-01 15:10:43 -04:00
# Moonfire NVR API <!-- omit in toc -->
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
update design docs for new-schema branch changes
2018-03-24 23:51:30 -04:00
Status: **current**.
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
make `GET /api/` return current permissions This is useful for e.g. deciding whether or not to present the user admin UI in navigation. As part of this change, I adjusted the casing in Permissions, and then all the toml stuff for consistency. Noted in changelog.
2022-12-31 12:08:26 -05:00
* [Summary](#summary)
* [Endpoints](#endpoints)
extend POST /users/:id Now you can set a password for a user while the server is running, e.g. via the following command: ```shell curl \ -H 'Content-Type: application/json' \ -d '{"update": {"password": "asdf"}}' \ --unix-socket /var/lib/moonfire-nvr/sock \ http://nvr/api/users/1 ```
2022-12-23 15:43:00 -05:00
* [Authentication](#authentication)
* [`POST /api/login`](#post-apilogin)
* [`POST /api/logout`](#post-apilogout)
mass markdown reformatting Add tables of contents (using the VS Code Markdown All-In-One extension) and reformat lists to consistently use 4-space indents. No content changes.
2021-04-01 15:10:43 -04:00
* [`GET /api/`](#get-api)
* [`GET /api/cameras/<uuid>/`](#get-apicamerasuuid)
* [`GET /api/cameras/<uuid>/<stream>/recordings`](#get-apicamerasuuidstreamrecordings)
* [`GET /api/cameras/<uuid>/<stream>/view.mp4`](#get-apicamerasuuidstreamviewmp4)
* [`GET /api/cameras/<uuid>/<stream>/view.mp4.txt`](#get-apicamerasuuidstreamviewmp4txt)
* [`GET /api/cameras/<uuid>/<stream>/view.m4s`](#get-apicamerasuuidstreamviewm4s)
* [`GET /api/cameras/<uuid>/<stream>/view.m4s.txt`](#get-apicamerasuuidstreamviewm4stxt)
* [`GET /api/cameras/<uuid>/<stream>/live.m4s`](#get-apicamerasuuidstreamlivem4s)
* [`GET /api/init/<id>.mp4`](#get-apiinitidmp4)
* [`GET /api/init/<id>.mp4.txt`](#get-apiinitidmp4txt)
* [`GET /api/signals`](#get-apisignals)
* [`POST /api/signals`](#post-apisignals)
* [Request 1](#request-1)
* [Request 2](#request-2)
* [Request 3](#request-3)
add GET /users/ endpoint
2022-12-24 13:09:05 -05:00
* [User management](#user-management)
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
* [`GET /api/users/`](#get-apiusers)
user admin api improvements
2023-01-08 04:14:03 -05:00
* [`POST /api/users/`](#post-apiusers)
add GET /users/ endpoint
2022-12-24 13:09:05 -05:00
* [`GET /api/users/<id>`](#get-apiusersid)
user admin api improvements
2023-01-08 04:14:03 -05:00
* [`PATCH /api/users/<id>`](#patch-apiusersid)
more user admin actions
2022-12-24 15:21:06 -05:00
* [`DELETE /api/users/<id>`](#delete-apiusersid)
make `GET /api/` return current permissions This is useful for e.g. deciding whether or not to present the user admin UI in navigation. As part of this change, I adjusted the casing in Permissions, and then all the toml stuff for consistency. Noted in changelog.
2022-12-31 12:08:26 -05:00
* [Types](#types)
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
* [UserSubset](#usersubset)
make `GET /api/` return current permissions This is useful for e.g. deciding whether or not to present the user admin UI in navigation. As part of this change, I adjusted the casing in Permissions, and then all the toml stuff for consistency. Noted in changelog.
2022-12-31 12:08:26 -05:00
* [Permissions](#permissions)
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
* [Cross-site request forgery (CSRF) protection](#cross-site-request-forgery-csrf-protection)
mass markdown reformatting Add tables of contents (using the VS Code Markdown All-In-One extension) and reformat lists to consistently use 4-space indents. No content changes.
2021-04-01 15:10:43 -04:00
make `GET /api/` return current permissions This is useful for e.g. deciding whether or not to present the user admin UI in navigation. As part of this change, I adjusted the casing in Permissions, and then all the toml stuff for consistency. Noted in changelog.
2022-12-31 12:08:26 -05:00
## Summary
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
make `GET /api/` return current permissions This is useful for e.g. deciding whether or not to present the user admin UI in navigation. As part of this change, I adjusted the casing in Permissions, and then all the toml stuff for consistency. Noted in changelog.
2022-12-31 12:08:26 -05:00
A JavaScript-based web interface to list cameras and view recordings.
Supports external analytics.
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
In the future, this is likely to be expanded:
* configuration support
* commandline tool over a UNIX-domain socket
(at least for bootstrapping web authentication)
start splitting wall and media duration for #34 This splits the schema and playback path. The recording path still adjusts the frame durations and always says the wall and media durations are the same. I expect to change that in a following commit. I wouldn't be surprised if that shakes out some bugs in this portion.
2020-08-05 00:44:01 -04:00
*Note:* italicized terms in this document are defined in the [glossary](glossary.md).
include ext stream in API response; better docs In particular, the docs now explicitly call out that API changes can happen at any version, and from 0.7.0 onward they're described in the changelog.
2021-11-23 16:01:20 -05:00
Currently the API is considered an internal contract between the server and the
UI which are bundled together. Thus, breaking changes in the API may happen in
any release of Moonfire NVR, even a "minor" or "patch" release. From version
0.7.0 onward, API changes should be described in the
[changelog](../CHANGELOG.md).
Future work may introduce versioning to improve compatibility with externally
developed tools.
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-26 00:31:50 -05:00
All requests for JSON data should be sent with the header
`Accept: application/json` (exactly).
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
make `GET /api/` return current permissions This is useful for e.g. deciding whether or not to present the user admin UI in navigation. As part of this change, I adjusted the casing in Permissions, and then all the toml stuff for consistency. Noted in changelog.
2022-12-31 12:08:26 -05:00
## Endpoints
extend POST /users/:id Now you can set a password for a user while the server is running, e.g. via the following command: ```shell curl \ -H 'Content-Type: application/json' \ -d '{"update": {"password": "asdf"}}' \ --unix-socket /var/lib/moonfire-nvr/sock \ http://nvr/api/users/1 ```
2022-12-23 15:43:00 -05:00
### Authentication
#### `POST /api/login`
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-26 00:31:50 -05:00
UI preferences: #153 #155
2021-09-01 18:01:42 -04:00
The request should have an `application/json` body containing a JSON object with
use application/json for login and logout
2020-01-09 02:23:58 -05:00
`username` and `password` keys.
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-26 00:31:50 -05:00
On successful authentication, the server will return an HTTP 204 (no content)
use SameSite=Lax instead of SameSite=Strict To improve reliability of live streams (#59) on Safari. Safari was dropping the cookie from websocket update requests. (But it worked sometimes. I don't get why.) I saw folks on the Internet thinking this related to HttpOnly: * https://developer.apple.com/forums/thread/104488 * https://stackoverflow.com/q/47742807/23584 but I still see this behavior without HttpOnly. SameSite=Strict vs SameSite=Lax appears to make a difference. Try that instead. SameSite=Strict is pointless for us anyway as noted in a new comment. Turning off HttpOnly would be more unfortunate security-wise.
2021-03-31 13:44:08 -04:00
with a `Set-Cookie` header for the `s` cookie, which is an opaque, `HttpOnly`
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-26 00:31:50 -05:00
(unavailable to Javascript) session identifier.
If authentication or authorization fails, the server will return a HTTP 403
(forbidden) response. Currently the body will be a `text/plain` error message;
future versions will likely be more sophisticated.
extend POST /users/:id Now you can set a password for a user while the server is running, e.g. via the following command: ```shell curl \ -H 'Content-Type: application/json' \ -d '{"update": {"password": "asdf"}}' \ --unix-socket /var/lib/moonfire-nvr/sock \ http://nvr/api/users/1 ```
2022-12-23 15:43:00 -05:00
#### `POST /api/logout`
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-26 00:31:50 -05:00
use application/json for login and logout
2020-01-09 02:23:58 -05:00
The request should have an `application/json` body containing
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
a `csrf` parameter copied from the `session.csrf` of the
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-26 00:31:50 -05:00
top-level API request.
On success, returns an HTTP 204 (no content) responses. On failure, returns a
4xx response with `text/plain` error message.
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
### `GET /api/`
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-22 00:54:27 -04:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
Returns basic information about the server, including all cameras. Valid
request parameters:
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-22 00:54:27 -04:00
* `days`: a boolean indicating if the days parameter described below
should be included.
API change: cameraConfigs should include rtsp urls
2020-06-22 03:38:23 -04:00
* `cameraConfigs`: a boolean indicating if the `camera.config` and
`camera.stream[].config` parameters described below should be included.
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
This requires the `readCameraConfigs` permission.
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-22 00:54:27 -04:00
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 18:17:50 -04:00
Example request URI (with added whitespace between parameters):
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-22 00:54:27 -04:00
```
/api/?days=true
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 18:17:50 -04:00
&cameraConfigs=true
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-22 00:54:27 -04:00
```
UI preferences: #153 #155
2021-09-01 18:01:42 -04:00
The `application/json` response will have a JSON object as follows:
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-22 00:54:27 -04:00
* `timeZoneName`: the name of the IANA time zone the server is using
to divide recordings into days as described further below.
add server version to /api response Fixes #171
2021-10-27 16:09:20 -04:00
* `serverVersion`: the version of the server in use, eg `0.7.0`.
UI preferences: #153 #155
2021-09-01 18:01:42 -04:00
* `cameras`: a list of cameras. Each is a JSON object as follows:
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-22 00:54:27 -04:00
* `uuid`: in text format
include ids in api responses for debugging
2021-08-31 15:05:03 -04:00
* `id`: an integer. The client doesn't ever need to send the id
back in API requests, but camera ids are helpful to know when debugging
by reading logs or directly examining the filesystem/database.
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-22 00:54:27 -04:00
* `shortName`: a short name (typically one or two words)
* `description`: a longer description (typically a phrase or paragraph)
include ext stream in API response; better docs In particular, the docs now explicitly call out that API changes can happen at any version, and from 0.7.0 onward they're described in the changelog.
2021-11-23 16:01:20 -05:00
* `config`: (only included if request parameter `cameraConfigs` is
true) a JSON object describing the configuration of the camera.
See doc comments on the `CameraConfig` type in
[`server/db/json.rs`](../server/db.json.rs).
* `streams`: a JSON object. Maps each configured stream type (valid types
are `main`, `sub`, and `ext`), a JSON object describing the stream:
include ids in api responses for debugging
2021-08-31 15:05:03 -04:00
* `id`: an integer. The client doesn't ever need to send the id
back in API requests, but stream ids are helpful to know when
debugging by reading logs or directly examining the
filesystem/database.
schema version 2: support sub streams This allows each camera to have a main and a sub stream. Previously there was a field in the schema for the sub stream's url, but it didn't do anything. Now you can configure individual retention for main and sub streams. They show up grouped in the UI. No support for upgrading from schema version 1 yet.
2018-01-23 14:05:07 -05:00
* `retainBytes`: the configured total number of bytes of completed
include ext stream in API response; better docs In particular, the docs now explicitly call out that API changes can happen at any version, and from 0.7.0 onward they're described in the changelog.
2021-11-23 16:01:20 -05:00
recordings to retain. This is copied from the `config` to make it
available when the client doesn't have permission to view
the full configuration.
schema version 2: support sub streams This allows each camera to have a main and a sub stream. Previously there was a field in the schema for the sub stream's url, but it didn't do anything. Now you can configure individual retention for main and sub streams. They show up grouped in the UI. No support for upgrading from schema version 1 yet.
2018-01-23 14:05:07 -05:00
* `minStartTime90k`: the start time of the earliest recording for
this camera, in 90kHz units since 1970-01-01 00:00:00 UTC.
* `maxEndTime90k`: the end time of the latest recording for this
camera, in 90kHz units since 1970-01-01 00:00:00 UTC.
* `totalDuration90k`: the total duration recorded, in 90 kHz units.
This is no greater than `maxEndTime90k - maxStartTime90k`; it will
be lesser if there are gaps in the recorded data.
* `totalSampleFileBytes`: the total number of bytes of sample data
(the `mdat` portion of a `.mp4` file).
track "assumed" filesystem usage (#89) As described in #89, we need to refactor a bit before we can get the actual filesystem block size. Assuming 4096 for now. Small steps.
2020-07-12 19:51:39 -04:00
* `fsBytes`: the total number of bytes on the filesystem used by
this stream. This is slightly more than `totalSampleFileBytes`
because it also includes the wasted portion of the final
filesystem block allocated to each file.
present signal days in API requests I also enforced some invariants in the signals code, fixing a couple bugs. The signals code is more complex than I'd like, but hopefully is working now.
2021-03-23 23:22:29 -04:00
* `days`: (only included if request parameter `days` is true)
UI preferences: #153 #155
2021-09-01 18:01:42 -04:00
JSON object representing calendar days (in the server's time zone)
include all recordings in days map (fixes #57) This is a quick fix to a problem that gives a confusing/poor initial experience, as in this thread: https://groups.google.com/g/moonfire-nvr-users/c/WB-TIW3bBZI/m/Gqh-L6I9BgAJ I don't think it's a permanent solution. In particular, when we implement an event stream (#40), I don't want to have a separate event for every frame, so having the days map change that often won't work. The client side will likely manipulate the days map then to include a special entry for a growing recording, representing "from this time to now".
2020-07-18 14:57:17 -04:00
with non-zero total duration of recordings for that day. Currently
this includes uncommitted and growing recordings. This is likely
to change in a future release for
[#40](https://github.com/scottlamb/moonfire-nvr/issues/40). The
keys are of the form `YYYY-mm-dd`; the values are objects with the
schema version 2: support sub streams This allows each camera to have a main and a sub stream. Previously there was a field in the schema for the sub stream's url, but it didn't do anything. Now you can configure individual retention for main and sub streams. They show up grouped in the UI. No support for upgrading from schema version 1 yet.
2018-01-23 14:05:07 -05:00
following attributes:
* `totalDuration90k` is the total duration recorded during that
day. If a recording spans a day boundary, some portion of it
is accounted to each day.
* `startTime90k` is the start of that calendar day in the
server's time zone.
* `endTime90k` is the end of that calendar day in the server's
time zone. It is usually 24 hours after the start time. It
might be 23 hours or 25 hours during spring forward or fall
back, respectively.
API change: cameraConfigs should include rtsp urls
2020-06-22 03:38:23 -04:00
* `config`: (only included if request parameter `cameraConfigs` is
include ext stream in API response; better docs In particular, the docs now explicitly call out that API changes can happen at any version, and from 0.7.0 onward they're described in the changelog.
2021-11-23 16:01:20 -05:00
true) a JSON object describing the configuration of the stream.
See doc comments on the `StreamConfig` type in
[`server/db/json.rs`](../server/db.json.rs).
UI preferences: #153 #155
2021-09-01 18:01:42 -04:00
* `signals`: a list of all *signals* known to the server. Each is a JSON
object with the following properties:
minor api fixes * videoSampleEntryId should be a number, not a string * fix some markdown formatting * fix missing description of sampleFileBytes
2021-02-22 16:46:51 -05:00
* `id`: an integer identifier.
more flexible signals Now there's room to add arbitrary configuration to signals and types. Several things are no longer fixed columns/tables but instead within the configuration types.
2021-10-26 13:12:19 -04:00
* `uuid`: a UUID identifier.
minor api fixes * videoSampleEntryId should be a number, not a string * fix some markdown formatting * fix missing description of sampleFileBytes
2021-02-22 16:46:51 -05:00
* `shortName`: a unique, human-readable description of the signal
* `cameras`: a map of associated cameras' UUIDs to the type of association:
`direct` or `indirect`. See `db/schema.sql` for more description.
* `type`: a UUID, expected to match one of `signalTypes`.
present signal days in API requests I also enforced some invariants in the signals code, fixing a couple bugs. The signals code is more complex than I'd like, but hopefully is working now.
2021-03-23 23:22:29 -04:00
* `days`: (only included if request parameter `days` is true) similar to
`cameras.days` above. Values are objects with the following attributes:
clarify a couple points of the signal API Thanks to @clydebarrow for pointing out these missing/ambiguous pieces.
2021-11-23 15:14:56 -05:00
* `states`: an array of the total time (in 90,000ths of a second) the
signal was state 1, state 2, and so on during the day. These may not
sum to the entire day; if so, the rest of the day is in state 0
(`unknown`).
read-only signals support (#28) This is mostly untested and useless by itself, but it's a starting point. In particular: * there's no way to set up signals or add/remove/update events yet except by manual changes to the database. * if you associate a signal with a camera then remove the camera, hitting /api/ will error out.
2019-06-06 19:18:13 -04:00
* `signalTypes`: a list of all known signal types.
minor api fixes * videoSampleEntryId should be a number, not a string * fix some markdown formatting * fix missing description of sampleFileBytes
2021-02-22 16:46:51 -05:00
* `uuid`: in text format.
clarify a couple points of the signal API Thanks to @clydebarrow for pointing out these missing/ambiguous pieces.
2021-11-23 15:14:56 -05:00
* `states`: an array of all possible states of the enumeration to more
information about them. Each holds a JSON object:
* `value`: an integer used to refer to this state, 1 or higher.
Value 0 always means the `unknown` state.
* `name`: a human-readable name of this state.
minor api fixes * videoSampleEntryId should be a number, not a string * fix some markdown formatting * fix missing description of sampleFileBytes
2021-02-22 16:46:51 -05:00
* `motion`: if present and true, directly associated cameras will be
considered to have motion when this signal is in this state.
clarify a couple points of the signal API Thanks to @clydebarrow for pointing out these missing/ambiguous pieces.
2021-11-23 15:14:56 -05:00
* `color` (optional): a recommended color to use in UIs to represent
this state, as in the [HTML specification](https://html.spec.whatwg.org/#colours).
make `GET /api/` return current permissions This is useful for e.g. deciding whether or not to present the user admin UI in navigation. As part of this change, I adjusted the casing in Permissions, and then all the toml stuff for consistency. Noted in changelog.
2022-12-31 12:08:26 -05:00
* `permissions`: the caller's current `Permissions` object (defined below).
include ext stream in API response; better docs In particular, the docs now explicitly call out that API changes can happen at any version, and from 0.7.0 onward they're described in the changelog.
2021-11-23 16:01:20 -05:00
* `user`: an object, present only when authenticated:
UI preferences: #153 #155
2021-09-01 18:01:42 -04:00
* `name`: a human-readable name
* `id`: an integer
* `preferences`: a JSON object
* `session`: an object, present only if authenticated via session cookie.
* `csrf`: a cross-site request forgery token for use in `POST` requests.
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
Example response:
```json
{
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-22 00:54:27 -04:00
"timeZoneName": "America/Los_Angeles",
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
"cameras": [
{
"uuid": "fd20f7a2-9d69-4cb3-94ed-d51a20c3edfe",
make json api more idiomatic * camelCase * lose the "days":null in the overall cameras dict
2017-10-10 00:58:44 -04:00
"shortName": "driveway",
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
"description": "Hikvision DS-2CD2032 overlooking the driveway from east",
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 18:17:50 -04:00
"config": {
store full rtsp urls My dad's "GW-GW4089IP" cameras use separate ports for the main and sub streams: rtsp://192.168.1.110:5050/H264?channel=0&subtype=0&unicast=true&proto=Onvif rtsp://192.168.1.110:5049/H264?channel=0&subtype=1&unicast=true&proto=Onvif Previously I could get one of the streams to work by including :5050 or :5049 in the host field of the camera. But not both. Now make the camera's host field reflect the ONVIF port (which is also non-standard on these cameras, :85). It's not directly used yet but probably will be sooner or later. Make each stream know its full URL.
2019-07-01 00:54:52 -04:00
"onvif_host": "192.168.1.100",
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 18:17:50 -04:00
"user": "admin",
"password": "12345",
},
schema version 2: support sub streams This allows each camera to have a main and a sub stream. Previously there was a field in the schema for the sub stream's url, but it didn't do anything. Now you can configure individual retention for main and sub streams. They show up grouped in the UI. No support for upgrading from schema version 1 yet.
2018-01-23 14:05:07 -05:00
"streams": {
"main": {
"retainBytes": 536870912000,
"minStartTime90k": 130888729442361,
"maxEndTime90k": 130985466591817,
"totalDuration90k": 96736169725,
"totalSampleFileBytes": 446774393937,
better error msg on live view when misconfigured Improves (but doesn't fix) #119 and #120.
2021-08-13 15:02:42 -04:00
"record": true,
schema version 2: support sub streams This allows each camera to have a main and a sub stream. Previously there was a field in the schema for the sub stream's url, but it didn't do anything. Now you can configure individual retention for main and sub streams. They show up grouped in the UI. No support for upgrading from schema version 1 yet.
2018-01-23 14:05:07 -05:00
"days": {
"2016-05-01": {
"endTime90k": 131595516000000,
"startTime90k": 131587740000000,
"totalDuration90k": 52617609
},
"2016-05-02": {
"endTime90k": 131603292000000,
"startTime90k": 131595516000000,
"totalDuration90k": 20946022
}
}
add a basic Javascript UI The Javascript is pretty amateurish I'm sure but at least it's something to iterate from. It's already much more pleasant for browsing through videos in several ways: * more responsive to load only a day at a time rather than 90+ days * much easier to see the same time segment on several cameras * more pleasant to have the videos load as a popup rather than a link that blows away your position in an enormous list * exposes the fancier .mp4 generation options: splitting at lengths other than the default, trimming to an arbitrary start and end time, including a subtitle track with timestamps. There's a slight regression in functionality: I didn't match the former top-level page which showed how much camera used of its disk allocation and the total duration of video. This is exposed in the JSON API, so it shouldn't be too hard to add back.
2017-10-22 00:54:27 -04:00
}
schema version 2: support sub streams This allows each camera to have a main and a sub stream. Previously there was a field in the schema for the sub stream's url, but it didn't do anything. Now you can configure individual retention for main and sub streams. They show up grouped in the UI. No support for upgrading from schema version 1 yet.
2018-01-23 14:05:07 -05:00
}
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
},
...
],
expose signal id in api ...and update api.md which described a different format than before or after.
2019-06-20 15:10:23 -04:00
"signals": [
{
"id": 1,
read-only signals support (#28) This is mostly untested and useless by itself, but it's a starting point. In particular: * there's no way to set up signals or add/remove/update events yet except by manual changes to the database. * if you associate a signal with a camera then remove the camera, hitting /api/ will error out.
2019-06-06 19:18:13 -04:00
"shortName": "driveway motion",
"cameras": {
"fd20f7a2-9d69-4cb3-94ed-d51a20c3edfe": "direct"
},
"type": "ee66270f-d9c6-4819-8b33-9720d4cbca6b",
"days": {
"2016-05-01": {
"endTime90k": 131595516000000,
"startTime90k": 131587740000000,
present signal days in API requests I also enforced some invariants in the signals code, fixing a couple bugs. The signals code is more complex than I'd like, but hopefully is working now.
2021-03-23 23:22:29 -04:00
"states": [5400000]
read-only signals support (#28) This is mostly untested and useless by itself, but it's a starting point. In particular: * there's no way to set up signals or add/remove/update events yet except by manual changes to the database. * if you associate a signal with a camera then remove the camera, hitting /api/ will error out.
2019-06-06 19:18:13 -04:00
}
}
}
],
"signalTypes": [
{
"uuid": "ee66270f-d9c6-4819-8b33-9720d4cbca6b",
"states": {
0: {
"name": "unknown",
"color": "#000000"
},
1: {
"name": "off",
"color": "#888888"
},
2: {
"name": "on",
"color": "#ff8888",
"motion": true
}
}
}
],
UI preferences: #153 #155
2021-09-01 18:01:42 -04:00
"user": {
"id": 1,
"name": "slamb",
"session": {
"csrf": "2DivvlnKUQ9JD4ao6YACBJm8XK4bFmOc"
}
preliminary web support for auth (#26) Some caveats: * it doesn't record the peer IP yet, which makes it harder to verify sessions are valid. This is a little annoying to do in hyper now (see hyperium/hyper#1410). The direct peer might not be what we want right now anyway because there's no TLS support yet (see #27). In the meantime, the sane way to expose Moonfire NVR to the Internet is via a proxy server, and recording the proxy's IP is not useful. Maybe better to interpret a RFC 7239 Forwarded header (and/or the older X-Forwarded-{For,Proto} headers). * it doesn't ever use Secure (https-only) cookies, for a similar reason. It's not safe to use even with a tls proxy until this is fixed. * there's no "moonfire-nvr config" support for inspecting/invalidating sessions yet. * in debug builds, logging in is crazy slow. See libpasta/libpasta#9. Some notes: * I removed the Javascript "no-use-before-defined" lint, as some of the functions form a cycle. * Fixed #20 along the way. I needed to add support for properly returning non-OK HTTP statuses to signal unauthorized and such. * I removed the Access-Control-Allow-Origin header support, which was at odds with the "SameSite=lax" in the cookie header. The "yarn start" method for running a local proxy server accomplishes the same thing as the Access-Control-Allow-Origin support in a more secure manner.
2018-11-26 00:31:50 -05:00
}
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
}
```
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
### `GET /api/cameras/<uuid>/`
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 18:17:50 -04:00
Returns information for the camera with the given URL. As in the like section
of `GET /api/` with the `days` parameter set and the `cameraConfigs` parameter
unset.
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
Export the calendar days map.
2016-05-02 11:38:52 -04:00
Example response:
```json
{
schema version 2: support sub streams This allows each camera to have a main and a sub stream. Previously there was a field in the schema for the sub stream's url, but it didn't do anything. Now you can configure individual retention for main and sub streams. They show up grouped in the UI. No support for upgrading from schema version 1 yet.
2018-01-23 14:05:07 -05:00
"description": "",
"streams": {
"main": {
"days": {
"2016-05-01": {
"endTime90k": 131595516000000,
"startTime90k": 131587740000000,
"totalDuration90k": 52617609
},
"2016-05-02": {
"endTime90k": 131603292000000,
"startTime90k": 131595516000000,
"totalDuration90k": 20946022
}
},
"maxEndTime90k": 131598273666690,
"minStartTime90k": 131590386129355,
"retainBytes": 104857600,
"totalDuration90k": 73563631,
"totalSampleFileBytes": 98901406
Export the calendar days map.
2016-05-02 11:38:52 -04:00
}
},
schema version 2: support sub streams This allows each camera to have a main and a sub stream. Previously there was a field in the schema for the sub stream's url, but it didn't do anything. Now you can configure individual retention for main and sub streams. They show up grouped in the UI. No support for upgrading from schema version 1 yet.
2018-01-23 14:05:07 -05:00
"shortName": "driveway"
Export the calendar days map.
2016-05-02 11:38:52 -04:00
}
```
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
### `GET /api/cameras/<uuid>/<stream>/recordings`
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
start splitting wall and media duration for #34 This splits the schema and playback path. The recording path still adjusts the frame durations and always says the wall and media durations are the same. I expect to change that in a following commit. I wouldn't be surprised if that shakes out some bugs in this portion.
2020-08-05 00:44:01 -04:00
Returns information about *recordings*. Valid request parameters:
Support limiting the range when listing recordings Now it's possible to quickly determine what calendar days have data and then query recordings for just the day(s) of interest with their returned {start,end}_time_usec.
2016-05-03 08:17:06 -04:00
make json api more idiomatic * camelCase * lose the "days":null in the overall cameras dict
2017-10-10 00:58:44 -04:00
* `startTime90k` and and `endTime90k` limit the data returned to only
start splitting wall and media duration for #34 This splits the schema and playback path. The recording path still adjusts the frame durations and always says the wall and media durations are the same. I expect to change that in a following commit. I wouldn't be surprised if that shakes out some bugs in this portion.
2020-08-05 00:44:01 -04:00
recordings with wall times overlapping with the given half-open interval.
Either or both may be absent; they default to the beginning and end of time,
respectively.
support run splitting in json api
2017-10-17 12:00:05 -04:00
* `split90k` causes long runs of recordings to be split at the next
convenient boundary after the given duration.
Support limiting the range when listing recordings Now it's possible to quickly determine what calendar days have data and then query recordings for just the day(s) of interest with their returned {start,end}_time_usec.
2016-05-03 08:17:06 -04:00
* TODO(slamb): `continue` to support paging. (If data is too large, the
server should return a `continue` key which is expected to be returned on
following requests.)
reorganize /recordings JSON response I want to start returning the pixel aspect ratio of each video sample entry. It's silly to duplicate it for each returned recording, so let's instead return a videoSampleEntryId and then put all the information about each VSE once. This change doesn't actually handle pixel aspect ratio server-side yet. Most likely I'll require a new schema version for that, to store it as a new column in the database. Codec-specific logic in the database layer is awkward and I'd like to avoid it. I did a similar schema change to add the rfc6381_codec. I also adjusted ui-src/lib/models/Recording.js in a few ways: * fixed a couple mismatches between its field name and the key defined in the API. Consistency aids understanding. * dropped all the getters in favor of just setting the fields (with type annotations) as described here: https://google.github.io/styleguide/jsguide.html#features-classes-fields * where the wire format used undefined (to save space), translate it to a more natural null or false.
2020-03-14 00:20:51 -04:00
Returns a JSON object. Under the key `recordings` is an array of recordings in
arbitrary order. Each recording object has the following properties:
schema version 1 The advantages of the new schema are: * overlapping recordings can be unambiguously described and viewed. This is a significant problem right now; the clock on my cameras appears to run faster than the (NTP-synchronized) clock on my NVR. Thus, if an RTSP session drops and is quickly reconnected, there's likely to be overlap. * less I/O is required to view mp4s when there are multiple cameras. This is a pretty dramatic difference in the number of database read syscalls with pragma page_size = 1024 (605 -> 39 in one test), although I'm not sure how much of that maps to actual I/O wait time. That's probably as dramatic as it is due to overflow page chaining. But even with larger page sizes, there's an improvement. It helps to stop interleaving the video_index fields from different cameras. There are changes to the JSON API to take advantage of this, described in design/api.md. There's an upgrade procedure, described in guide/schema.md.
2016-12-21 01:08:18 -05:00
make json api more idiomatic * camelCase * lose the "days":null in the overall cameras dict
2017-10-10 00:58:44 -04:00
* `startId`. The id of this recording, which can be used with `/view.mp4`
schema version 1 The advantages of the new schema are: * overlapping recordings can be unambiguously described and viewed. This is a significant problem right now; the clock on my cameras appears to run faster than the (NTP-synchronized) clock on my NVR. Thus, if an RTSP session drops and is quickly reconnected, there's likely to be overlap. * less I/O is required to view mp4s when there are multiple cameras. This is a pretty dramatic difference in the number of database read syscalls with pragma page_size = 1024 (605 -> 39 in one test), although I'm not sure how much of that maps to actual I/O wait time. That's probably as dramatic as it is due to overflow page chaining. But even with larger page sizes, there's an improvement. It helps to stop interleaving the video_index fields from different cameras. There are changes to the JSON API to take advantage of this, described in design/api.md. There's an upgrade procedure, described in guide/schema.md.
2016-12-21 01:08:18 -05:00
to retrieve its content.
make json api more idiomatic * camelCase * lose the "days":null in the overall cameras dict
2017-10-10 00:58:44 -04:00
* `endId` (optional). If absent, this object describes a single recording.
If present, this indicates that recordings `startId-endId` (inclusive)
schema version 1 The advantages of the new schema are: * overlapping recordings can be unambiguously described and viewed. This is a significant problem right now; the clock on my cameras appears to run faster than the (NTP-synchronized) clock on my NVR. Thus, if an RTSP session drops and is quickly reconnected, there's likely to be overlap. * less I/O is required to view mp4s when there are multiple cameras. This is a pretty dramatic difference in the number of database read syscalls with pragma page_size = 1024 (605 -> 39 in one test), although I'm not sure how much of that maps to actual I/O wait time. That's probably as dramatic as it is due to overflow page chaining. But even with larger page sizes, there's an improvement. It helps to stop interleaving the video_index fields from different cameras. There are changes to the JSON API to take advantage of this, described in design/api.md. There's an upgrade procedure, described in guide/schema.md.
2016-12-21 01:08:18 -05:00
together are as described. Adjacent recordings from the same RTSP session
may be coalesced in this fashion to reduce the amount of redundant data
transferred.
seamlessly merge minor VSE changes Improves #302.
2024-02-12 20:35:27 -05:00
* `runStartId`. The id of the first recording in this run.
allow listing and viewing uncommitted recordings There may be considerable lag between being fully written and being committed when using the flush_if_sec feature. Additionally, this is a step toward listing and viewing recordings before they're fully written. That's a considerable delay: 60 to 120 seconds for the first recording of a run, 0 to 60 seconds for subsequent recordings. These recordings aren't yet included in the information returned by /api/?days=true. They probably should be, but small steps.
2018-03-02 14:38:11 -05:00
* `firstUncommitted` (optional). If this range is not fully committed to the
database, the first id that is uncommitted. This is significant because
it's possible that after a crash and restart, this id will refer to a
completely different recording. That recording will have a different
`openId`.
view in-progress recordings! The time from recorded to viewable was previously 60-120 sec for the first recording of a RTSP session, 0-60 sec otherwise. Now it's one frame.
2018-03-02 18:40:32 -05:00
* `growing` (optional). If this boolean is true, the recording `endId` is
still being written to. Accesses to this id (such as `view.mp4`) may
retrieve more data than described here if not bounded by duration.
Additionally, if `startId` == `endId`, the start time of the recording is
"unanchored" and may change in subsequent accesses.
allow listing and viewing uncommitted recordings There may be considerable lag between being fully written and being committed when using the flush_if_sec feature. Additionally, this is a step toward listing and viewing recordings before they're fully written. That's a considerable delay: 60 to 120 seconds for the first recording of a run, 0 to 60 seconds for subsequent recordings. These recordings aren't yet included in the information returned by /api/?days=true. They probably should be, but small steps.
2018-03-02 14:38:11 -05:00
* `openId`. Each time Moonfire NVR starts in read-write mode, it is assigned
an increasing "open id". This field is the open id as of when these
recordings were written. This can be used to disambiguate ids referring to
uncommitted recordings.
start splitting wall and media duration for #34 This splits the schema and playback path. The recording path still adjusts the frame durations and always says the wall and media durations are the same. I expect to change that in a following commit. I wouldn't be surprised if that shakes out some bugs in this portion.
2020-08-05 00:44:01 -04:00
* `startTime90k`: the start time of the given recording, in the wall time
scale. Note this may be less than the requested `startTime90k` if this
recording was ongoing at the requested time.
* `endTime90k`: the end time of the given recording, in the wall time scale.
Note this may be greater than the requested `endTime90k` if this recording
was ongoing at the requested time.
reorganize /recordings JSON response I want to start returning the pixel aspect ratio of each video sample entry. It's silly to duplicate it for each returned recording, so let's instead return a videoSampleEntryId and then put all the information about each VSE once. This change doesn't actually handle pixel aspect ratio server-side yet. Most likely I'll require a new schema version for that, to store it as a new column in the database. Codec-specific logic in the database layer is awkward and I'd like to avoid it. I did a similar schema change to add the rfc6381_codec. I also adjusted ui-src/lib/models/Recording.js in a few ways: * fixed a couple mismatches between its field name and the key defined in the API. Consistency aids understanding. * dropped all the getters in favor of just setting the fields (with type annotations) as described here: https://google.github.io/styleguide/jsguide.html#features-classes-fields * where the wire format used undefined (to save space), translate it to a more natural null or false.
2020-03-14 00:20:51 -04:00
* `videoSampleEntryId`: a reference to an entry in the `videoSampleEntries`
trim recordings to match the requested range This matches the default behavior of the old UI. It's not configurable. I'm not sure anyone ever deliberately unchecked that box.
2021-03-14 01:25:05 -05:00
object.
make json api more idiomatic * camelCase * lose the "days":null in the overall cameras dict
2017-10-10 00:58:44 -04:00
* `videoSamples`: the number of samples (aka frames) of video in this
Coalesce adjacent recordings for efficiency
2016-05-10 20:37:53 -04:00
recording.
minor api fixes * videoSampleEntryId should be a number, not a string * fix some markdown formatting * fix missing description of sampleFileBytes
2021-02-22 16:46:51 -05:00
* `sampleFileBytes`: the number of bytes of video in this recording.
UNTESTED: note trailing zeros in /recordings reply
2021-10-10 19:13:57 -04:00
* `hasTrailingZero`: the final frame of the final recording id described by
this row (`endId` if present, otherwise `startId`) has a duration of 0.
A frame's duration is calculated by subtracting its timestamp from the
following frame's timestamp. When a run ends, there's no following frame
and Moonfire NVR fills in a duration of 0. When using `/view.mp4`, it's
not possible to append additional segments after such frames, as noted
below.
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
reorganize /recordings JSON response I want to start returning the pixel aspect ratio of each video sample entry. It's silly to duplicate it for each returned recording, so let's instead return a videoSampleEntryId and then put all the information about each VSE once. This change doesn't actually handle pixel aspect ratio server-side yet. Most likely I'll require a new schema version for that, to store it as a new column in the database. Codec-specific logic in the database layer is awkward and I'd like to avoid it. I did a similar schema change to add the rfc6381_codec. I also adjusted ui-src/lib/models/Recording.js in a few ways: * fixed a couple mismatches between its field name and the key defined in the API. Consistency aids understanding. * dropped all the getters in favor of just setting the fields (with type annotations) as described here: https://google.github.io/styleguide/jsguide.html#features-classes-fields * where the wire format used undefined (to save space), translate it to a more natural null or false.
2020-03-14 00:20:51 -04:00
Under the property `videoSampleEntries`, an object mapping ids to objects with
the following properties:
* `width`: the stored width in pixels.
* `height`: the stored height in pixels.
* `pixelHSpacing`: the relative width of a pixel, as in a ISO/IEC 14496-12
section 12.1.4.3 `PixelAspectRatioBox`. If absent, assumed to be 1.
* `pixelVSpacing`: the relative height of a pixel, as in a ISO/IEC 14496-12
section 12.1.4.3 `PixelAspectRatioBox`. If absent, assumed to be 1.
UI: improve aspect ratio handling As written in the changelog: Live streams formerly worked around a Firefox pixel aspect ratio bug by forcing all videos to 16:9, which dramatically distorted 9:16 camera views. Playback didn't, so anamorphic videos looked correct on Chrome but slightly stretched on Firefox. Now both live streams and playback are fully correct on all browsers.
2021-08-12 16:32:01 -04:00
* `aspectWidth`: the width component of the aspect ratio. (The aspect ratio
can be computed from the dimensions and pixel spacing; it's included as a
convenience.)
* `aspectHeight`: the height component of the aspect ratio.
reorganize /recordings JSON response I want to start returning the pixel aspect ratio of each video sample entry. It's silly to duplicate it for each returned recording, so let's instead return a videoSampleEntryId and then put all the information about each VSE once. This change doesn't actually handle pixel aspect ratio server-side yet. Most likely I'll require a new schema version for that, to store it as a new column in the database. Codec-specific logic in the database layer is awkward and I'd like to avoid it. I did a similar schema change to add the rfc6381_codec. I also adjusted ui-src/lib/models/Recording.js in a few ways: * fixed a couple mismatches between its field name and the key defined in the API. Consistency aids understanding. * dropped all the getters in favor of just setting the fields (with type annotations) as described here: https://google.github.io/styleguide/jsguide.html#features-classes-fields * where the wire format used undefined (to save space), translate it to a more natural null or false.
2020-03-14 00:20:51 -04:00
use Blake3 instead of SHA-1 or Blake2b Benefits: * Blake3 is faster. This is most noticeable for the hashing of the sample file data. * we no longer need OpenSSL, which helps with shrinking the binary size (#70). sha1 basically forced OpenSSL usage; ring deliberately doesn't support this old algorithm, and the pure-Rust sha1 crate is painfully slow. OpenSSL might still be a better choice than ring/rustls for TLS but it's nice to have the option. For the video sample entries, I decided we don't need to hash at all. I think the id number is sufficiently stable, and it's okay---perhaps even desirable---if an existing init segment changes for fixes like e5b83c2.
2020-03-20 23:52:30 -04:00
The full initialization segment data for a given video sample entry can be
retrieved at the URL `/api/init/<id>.mp4`.
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
Example request URI (with added whitespace between parameters):
```
schema version 2: support sub streams This allows each camera to have a main and a sub stream. Previously there was a field in the schema for the sub stream's url, but it didn't do anything. Now you can configure individual retention for main and sub streams. They show up grouped in the UI. No support for upgrading from schema version 1 yet.
2018-01-23 14:05:07 -05:00
/api/cameras/fd20f7a2-9d69-4cb3-94ed-d51a20c3edfe/main/recordings
make json api more idiomatic * camelCase * lose the "days":null in the overall cameras dict
2017-10-10 00:58:44 -04:00
?startTime90k=130888729442361
&endTime90k=130985466591817
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
```
Example response:
```json
{
"recordings": [
{
make json api more idiomatic * camelCase * lose the "days":null in the overall cameras dict
2017-10-10 00:58:44 -04:00
"startId": 1,
"startTime90k": 130985461191810,
"endTime90k": 130985466591817,
"sampleFileBytes": 8405564,
minor api fixes * videoSampleEntryId should be a number, not a string * fix some markdown formatting * fix missing description of sampleFileBytes
2021-02-22 16:46:51 -05:00
"videoSampleEntryId": 1,
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
},
{
make json api more idiomatic * camelCase * lose the "days":null in the overall cameras dict
2017-10-10 00:58:44 -04:00
"endTime90k": 130985461191810,
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
...
},
...
],
reorganize /recordings JSON response I want to start returning the pixel aspect ratio of each video sample entry. It's silly to duplicate it for each returned recording, so let's instead return a videoSampleEntryId and then put all the information about each VSE once. This change doesn't actually handle pixel aspect ratio server-side yet. Most likely I'll require a new schema version for that, to store it as a new column in the database. Codec-specific logic in the database layer is awkward and I'd like to avoid it. I did a similar schema change to add the rfc6381_codec. I also adjusted ui-src/lib/models/Recording.js in a few ways: * fixed a couple mismatches between its field name and the key defined in the API. Consistency aids understanding. * dropped all the getters in favor of just setting the fields (with type annotations) as described here: https://google.github.io/styleguide/jsguide.html#features-classes-fields * where the wire format used undefined (to save space), translate it to a more natural null or false.
2020-03-14 00:20:51 -04:00
"videoSampleEntries": {
"1": {
"width": 1280,
"height": 720
}
},
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
}
```
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
### `GET /api/cameras/<uuid>/<stream>/view.mp4`
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
Requires the `viewVideo` permission.
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 18:17:50 -04:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
Returns a `.mp4` file, with an etag and support for range requests. The MIME
type will be `video/mp4`, with a `codecs` parameter as specified in
[RFC 6381][rfc-6381].
Add a simple JSON API. This is a work in progress. There are no tests yet.
2016-04-23 16:55:36 -04:00
Expected query parameters:
schema version 1 The advantages of the new schema are: * overlapping recordings can be unambiguously described and viewed. This is a significant problem right now; the clock on my cameras appears to run faster than the (NTP-synchronized) clock on my NVR. Thus, if an RTSP session drops and is quickly reconnected, there's likely to be overlap. * less I/O is required to view mp4s when there are multiple cameras. This is a pretty dramatic difference in the number of database read syscalls with pragma page_size = 1024 (605 -> 39 in one test), although I'm not sure how much of that maps to actual I/O wait time. That's probably as dramatic as it is due to overflow page chaining. But even with larger page sizes, there's an improvement. It helps to stop interleaving the video_index fields from different cameras. There are changes to the JSON API to take advantage of this, described in design/api.md. There's an upgrade procedure, described in guide/schema.md.
2016-12-21 01:08:18 -05:00
* `s` (one or more): a string of the form
allow listing and viewing uncommitted recordings There may be considerable lag between being fully written and being committed when using the flush_if_sec feature. Additionally, this is a step toward listing and viewing recordings before they're fully written. That's a considerable delay: 60 to 120 seconds for the first recording of a run, 0 to 60 seconds for subsequent recordings. These recordings aren't yet included in the information returned by /api/?days=true. They probably should be, but small steps.
2018-03-02 14:38:11 -05:00
`START_ID[-END_ID][@OPEN_ID][.[REL_START_TIME]-[REL_END_TIME]]`. This
start splitting wall and media duration for #34 This splits the schema and playback path. The recording path still adjusts the frame durations and always says the wall and media durations are the same. I expect to change that in a following commit. I wouldn't be surprised if that shakes out some bugs in this portion.
2020-08-05 00:44:01 -04:00
specifies *segments* to include. The produced `.mp4` file will be a
concatenation of the segments indicated by all `s` parameters. The ids to
fix live view This broke with the media vs wall duration split, part of #34.
2020-08-07 13:16:06 -04:00
retrieve are as returned by the `/recordings` URL. The *open id* is
optional and will be enforced if present; it's recommended for
disambiguation when the requested range includes uncommitted recordings.
The optional start and end times are in 90k units of wall time and relative
to the start of the first specified id. These can be used to clip the
returned segments. Note they can be used to skip over some ids entirely;
this is allowed so that the caller doesn't need to know the start time of
each interior id. If there is no key frame at the desired relative start
time, frames back to the last key frame will be included in the returned
data, and an edit list will instruct the viewer to skip to the desired
start time.
schema version 1 The advantages of the new schema are: * overlapping recordings can be unambiguously described and viewed. This is a significant problem right now; the clock on my cameras appears to run faster than the (NTP-synchronized) clock on my NVR. Thus, if an RTSP session drops and is quickly reconnected, there's likely to be overlap. * less I/O is required to view mp4s when there are multiple cameras. This is a pretty dramatic difference in the number of database read syscalls with pragma page_size = 1024 (605 -> 39 in one test), although I'm not sure how much of that maps to actual I/O wait time. That's probably as dramatic as it is due to overflow page chaining. But even with larger page sizes, there's an improvement. It helps to stop interleaving the video_index fields from different cameras. There are changes to the JSON API to take advantage of this, described in design/api.md. There's an upgrade procedure, described in guide/schema.md.
2016-12-21 01:08:18 -05:00
* `ts` (optional): should be set to `true` to request a subtitle track be
added with human-readable recording timestamps.
Example request URI to retrieve all of recording id 1 from the given camera:
```
schema version 2: support sub streams This allows each camera to have a main and a sub stream. Previously there was a field in the schema for the sub stream's url, but it didn't do anything. Now you can configure individual retention for main and sub streams. They show up grouped in the UI. No support for upgrading from schema version 1 yet.
2018-01-23 14:05:07 -05:00
/api/cameras/fd20f7a2-9d69-4cb3-94ed-d51a20c3edfe/main/view.mp4?s=1
schema version 1 The advantages of the new schema are: * overlapping recordings can be unambiguously described and viewed. This is a significant problem right now; the clock on my cameras appears to run faster than the (NTP-synchronized) clock on my NVR. Thus, if an RTSP session drops and is quickly reconnected, there's likely to be overlap. * less I/O is required to view mp4s when there are multiple cameras. This is a pretty dramatic difference in the number of database read syscalls with pragma page_size = 1024 (605 -> 39 in one test), although I'm not sure how much of that maps to actual I/O wait time. That's probably as dramatic as it is due to overflow page chaining. But even with larger page sizes, there's an improvement. It helps to stop interleaving the video_index fields from different cameras. There are changes to the JSON API to take advantage of this, described in design/api.md. There's an upgrade procedure, described in guide/schema.md.
2016-12-21 01:08:18 -05:00
```
Example request URI to retrieve all of recording ids 1–5 from the given camera,
with timestamp subtitles:
```
schema version 2: support sub streams This allows each camera to have a main and a sub stream. Previously there was a field in the schema for the sub stream's url, but it didn't do anything. Now you can configure individual retention for main and sub streams. They show up grouped in the UI. No support for upgrading from schema version 1 yet.
2018-01-23 14:05:07 -05:00
/api/cameras/fd20f7a2-9d69-4cb3-94ed-d51a20c3edfe/main/view.mp4?s=1-5&ts=true
schema version 1 The advantages of the new schema are: * overlapping recordings can be unambiguously described and viewed. This is a significant problem right now; the clock on my cameras appears to run faster than the (NTP-synchronized) clock on my NVR. Thus, if an RTSP session drops and is quickly reconnected, there's likely to be overlap. * less I/O is required to view mp4s when there are multiple cameras. This is a pretty dramatic difference in the number of database read syscalls with pragma page_size = 1024 (605 -> 39 in one test), although I'm not sure how much of that maps to actual I/O wait time. That's probably as dramatic as it is due to overflow page chaining. But even with larger page sizes, there's an improvement. It helps to stop interleaving the video_index fields from different cameras. There are changes to the JSON API to take advantage of this, described in design/api.md. There's an upgrade procedure, described in guide/schema.md.
2016-12-21 01:08:18 -05:00
```
Example request URI to retrieve recording id 1, skipping its first 26
90,000ths of a second:
```
fix example URI to match grammar Fixes #177
2021-10-09 11:13:17 -04:00
/api/cameras/fd20f7a2-9d69-4cb3-94ed-d51a20c3edfe/main/view.mp4?s=1.26-
schema version 1 The advantages of the new schema are: * overlapping recordings can be unambiguously described and viewed. This is a significant problem right now; the clock on my cameras appears to run faster than the (NTP-synchronized) clock on my NVR. Thus, if an RTSP session drops and is quickly reconnected, there's likely to be overlap. * less I/O is required to view mp4s when there are multiple cameras. This is a pretty dramatic difference in the number of database read syscalls with pragma page_size = 1024 (605 -> 39 in one test), although I'm not sure how much of that maps to actual I/O wait time. That's probably as dramatic as it is due to overflow page chaining. But even with larger page sizes, there's an improvement. It helps to stop interleaving the video_index fields from different cameras. There are changes to the JSON API to take advantage of this, described in design/api.md. There's an upgrade procedure, described in guide/schema.md.
2016-12-21 01:08:18 -05:00
```
support segmented mp4s This is intended to support HTML5 Media Source Extensions, which I expect to be the most practical way to make a good web UI with a proper scrub bar and such. This feature has had very limited testing on Chrome and Firefox, and that was not entirely successful. More work is needed before it's usable, but this seems like a helpful progress checkpoint.
2017-10-01 18:29:22 -04:00
start splitting wall and media duration for #34 This splits the schema and playback path. The recording path still adjusts the frame durations and always says the wall and media durations are the same. I expect to change that in a following commit. I wouldn't be surprised if that shakes out some bugs in this portion.
2020-08-05 00:44:01 -04:00
Note carefully the distinction between *wall duration* and *media duration*.
It's normal for `/view.mp4` to return a media presentation with a length
slightly different from the *wall duration* of the backing recording or
portion that was requested.
improve API docs, including caveats and bugs For #178
2021-10-10 18:58:55 -04:00
Bugs and limitations:
* If the `s=` parameter references a recording id that doesn't exist when the
server starts processing the `/view.mp4` request, the server will return a
`404` with a text error message. This commonly happens when the oldest
recording was deleted between the `/recordings` request and the `/view.mp4`
request. The server probably should return a structured JSON document
describing exactly which recordings have been deleted. For now, the client
will have to retry from `/recordings` and again race against deletion.
* If a recording is deleted after the server starts processing `/view.mp4`
but before the request advances to the recording's byte position, the server
will abruptly drop the HTTP connection. The client must then retry to see
a proper 404 error. It'd be better if the server would prevent recordings
from being deleted while there are `/view.mp4` requests in progress which
reference them.
* The final recording in every "run" ends with a frame that has duration 0.
It's not possible to append additional segments after such a frame;
use typed errors in /view.mp4 path This fixes #178. Before, everything got translated to 5xx status; now it produces the correct type in several cases. Ideally I'd get rid of the untyped errors in all of web.rs; this is a small step.
2021-10-21 13:25:37 -04:00
the server will return a 400 error like `Invalid argument: unable to append
UNTESTED: note trailing zeros in /recordings reply
2021-10-10 19:13:57 -04:00
recording 2/16672 after recording 2/16671 with trailing zero`. See also
`hasTrailingZero` above, and
[#178](https://github.com/scottlamb/moonfire-nvr/issues/178).
support segmented mp4s This is intended to support HTML5 Media Source Extensions, which I expect to be the most practical way to make a good web UI with a proper scrub bar and such. This feature has had very limited testing on Chrome and Firefox, and that was not entirely successful. More work is needed before it's usable, but this seems like a helpful progress checkpoint.
2017-10-01 18:29:22 -04:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
### `GET /api/cameras/<uuid>/<stream>/view.mp4.txt`
add a url for getting debug info about a .mp4 file and add a unit test of path decoding along the way
2018-12-29 14:06:44 -05:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
Returns a `text/plain` debugging string for the `.mp4` generated by the
add a url for getting debug info about a .mp4 file and add a unit test of path decoding along the way
2018-12-29 14:06:44 -05:00
same URL minus the `.txt` suffix.
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
### `GET /api/cameras/<uuid>/<stream>/view.m4s`
support segmented mp4s This is intended to support HTML5 Media Source Extensions, which I expect to be the most practical way to make a good web UI with a proper scrub bar and such. This feature has had very limited testing on Chrome and Firefox, and that was not entirely successful. More work is needed before it's usable, but this seems like a helpful progress checkpoint.
2017-10-01 18:29:22 -04:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
Returns a `.mp4` suitable for use as a [HTML5 Media Source Extensions
serve 'video/mp4; codecs="avc1.xxxxxx"' mime type This can be used when constructing a HTML5 SourceBuffer.
2017-10-04 02:25:58 -04:00
media segment][media-segment]. The MIME type will be `video/mp4`, with a
pass prev duration and runs through API layer Builds on f3ddbfe, for #32 and #59.
2020-06-10 01:06:03 -04:00
`codecs` parameter as specified in [RFC 6381][rfc-6381]. Note that these
can't include edit lists, so (unlike `/view.mp4`) the caller must manually
trim undesired leading portions.
This response will include the following additional headers:
start splitting wall and media duration for #34 This splits the schema and playback path. The recording path still adjusts the frame durations and always says the wall and media durations are the same. I expect to change that in a following commit. I wouldn't be surprised if that shakes out some bugs in this portion.
2020-08-05 00:44:01 -04:00
* `X-Prev-Media-Duration`: the total *media duration* (in 90 kHz units) of all
*recordings* before the first requested recording in the `s` parameter.
Browser-based callers may use this to place this at the correct position in
the source buffer via `SourceBuffer.timestampOffset`.
pass prev duration and runs through API layer Builds on f3ddbfe, for #32 and #59.
2020-06-10 01:06:03 -04:00
* `X-Runs`: the cumulative number of "runs" of recordings. If this recording
starts a new run, it is included in the count. Browser-based callers may
use this to force gaps in the source buffer timeline by adjusting the
timestamp offset if desired.
start splitting wall and media duration for #34 This splits the schema and playback path. The recording path still adjusts the frame durations and always says the wall and media durations are the same. I expect to change that in a following commit. I wouldn't be surprised if that shakes out some bugs in this portion.
2020-08-05 00:44:01 -04:00
* `X-Leading-Media-Duration`: if present, the total duration (in 90 kHz
units) of additional leading video included before the caller's first
requested timestamp. This happens when the caller's requested timestamp
does not fall exactly on a key frame. Media segments can't include edit
lists, so unlike with the `/api/.../view.mp4` endpoint the caller is
responsible for trimming this portion. Browser-based callers may use
pass prev duration and runs through API layer Builds on f3ddbfe, for #32 and #59.
2020-06-10 01:06:03 -04:00
`SourceBuffer.appendWindowStart`.
support segmented mp4s This is intended to support HTML5 Media Source Extensions, which I expect to be the most practical way to make a good web UI with a proper scrub bar and such. This feature has had very limited testing on Chrome and Firefox, and that was not entirely successful. More work is needed before it's usable, but this seems like a helpful progress checkpoint.
2017-10-01 18:29:22 -04:00
Expected query parameters:
pass prev duration and runs through API layer Builds on f3ddbfe, for #32 and #59.
2020-06-10 01:06:03 -04:00
* `s` (one or more): as with the `.mp4` URL.
support segmented mp4s This is intended to support HTML5 Media Source Extensions, which I expect to be the most practical way to make a good web UI with a proper scrub bar and such. This feature has had very limited testing on Chrome and Firefox, and that was not entirely successful. More work is needed before it's usable, but this seems like a helpful progress checkpoint.
2017-10-01 18:29:22 -04:00
It's recommended that each `.m4s` retrieval be for at most one Moonfire NVR
improve API docs, including caveats and bugs For #178
2021-10-10 18:58:55 -04:00
recording. The fundamental reason is that the Media Source Extension API appears
structured for adding a complete segment at a time. Large media segments thus
impose significant latency on seeking. Additionally, because of this fundamental
reason Moonfire NVR makes no effort to make multiple-segment `.m4s` requests
practical:
support segmented mp4s This is intended to support HTML5 Media Source Extensions, which I expect to be the most practical way to make a good web UI with a proper scrub bar and such. This feature has had very limited testing on Chrome and Firefox, and that was not entirely successful. More work is needed before it's usable, but this seems like a helpful progress checkpoint.
2017-10-01 18:29:22 -04:00
* There is currently a hard limit of 4 GiB of data because the `.m4s` uses a
single `moof` followed by a single `mdat`; the former references the
latter with 32-bit offsets.
* There's currently no way to generate an initialization segment for more
than one video sample entry, so a `.m4s` that uses more than one video
sample entry can't be used.
fix live view This broke with the media vs wall duration split, part of #34.
2020-08-07 13:16:06 -04:00
* The `X-Prev-Media-Duration` and `X-Leading-Media-Duration` headers only
describe the first segment.
start splitting wall and media duration for #34 This splits the schema and playback path. The recording path still adjusts the frame durations and always says the wall and media durations are the same. I expect to change that in a following commit. I wouldn't be surprised if that shakes out some bugs in this portion.
2020-08-05 00:44:01 -04:00
Timestamp tracks (see the `ts` parameter to `.mp4` URIs) aren't supported
today. Most likely browser clients will implement timestamp subtitles via
WebVTT API calls anyway.
support segmented mp4s This is intended to support HTML5 Media Source Extensions, which I expect to be the most practical way to make a good web UI with a proper scrub bar and such. This feature has had very limited testing on Chrome and Firefox, and that was not entirely successful. More work is needed before it's usable, but this seems like a helpful progress checkpoint.
2017-10-01 18:29:22 -04:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
### `GET /api/cameras/<uuid>/<stream>/view.m4s.txt`
add a url for getting debug info about a .mp4 file and add a unit test of path decoding along the way
2018-12-29 14:06:44 -05:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
Returns a `text/plain` debugging string for the `.mp4` generated by the same
URL minus the `.txt` suffix.
add a url for getting debug info about a .mp4 file and add a unit test of path decoding along the way
2018-12-29 14:06:44 -05:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
### `GET /api/cameras/<uuid>/<stream>/live.m4s`
backend support for live stream (#59) This is so far completely untested, for use by a new UI prototype. It creates a new URL endpoint which sends one video/mp4 media segment per key frame, with the dependent frames included. This means there will be about one key frame interval of latency (typically about a second). This seems hard to avoid, as mentioned in issue #59.
2019-01-21 18:58:52 -05:00
switch to websocket for live stream (#59) The multipart stream / hanging GET approach worked in a prototype for a single stream, but Chrome has a per-host limit of six connections. If I try streaming all my cameras at once, I hit that limit. I can't open all the streams, much less additional connections to load init segments and such. Websockets apparently has a much higher limit of 256.
2020-02-29 00:41:31 -05:00
Initiate a WebSocket stream for chunks of video. Expects the standard
WebSocket headers as described in [RFC 6455][rfc-6455] and (if authentication
is required) the `s` cookie.
rework WebSocket error return protocol This gives much better information to the UI layer, getting rid of a whole troubleshooting guide entry. See #119 #132 #218 #219 I also restructured the code in anticipation of a new WebSocket event stream (#40).
2023-02-15 10:04:50 -05:00
The server will send messages as follows:
* text: a plaintext error message, followed by the end of stream.
* binary: video data, repeatedly, as described below.
* ping: every 30 seconds.
Each binary message corresponds to one or more frames of video. The first
message is guaranteed to start with a "key" (IDR) frame; others may not. The
message will contain HTTP headers followed by by a `.mp4` media segment. The
following headers will be included:
switch to websocket for live stream (#59) The multipart stream / hanging GET approach worked in a prototype for a single stream, but Chrome has a per-host limit of six connections. If I try streaming all my cameras at once, I hit that limit. I can't open all the streams, much less additional connections to load init segments and such. Websockets apparently has a much higher limit of 256.
2020-02-29 00:41:31 -05:00
Documentation updates
2021-09-29 21:39:18 -04:00
* `X-Video-Sample-Entry-Id`: An id to use when fetching an initialization segment.
switch to websocket for live stream (#59) The multipart stream / hanging GET approach worked in a prototype for a single stream, but Chrome has a per-host limit of six connections. If I try streaming all my cameras at once, I hit that limit. I can't open all the streams, much less additional connections to load init segments and such. Websockets apparently has a much higher limit of 256.
2020-02-29 00:41:31 -05:00
* `X-Recording-Id`: the open id, a period, and the recording id of the
recording these frames belong to.
* `X-Recording-Start`: the timestamp (in Moonfire NVR's usual 90,000ths
of a second) of the start of the recording. Note that if the recording
is "unanchored" (as described in `GET /api/.../recordings`), the
recording's start time may change before it is completed.
fix live view This broke with the media vs wall duration split, part of #34.
2020-08-07 13:16:06 -04:00
* `X-Prev-Media-Duration`: as in `/.../view.m4s`.
pass prev duration and runs through API layer Builds on f3ddbfe, for #32 and #59.
2020-06-10 01:06:03 -04:00
* `X-Runs`: as in `/.../view.m4s`.
fix live view This broke with the media vs wall duration split, part of #34.
2020-08-07 13:16:06 -04:00
* `X-Media-Time-Range`: the relative media start and end times of these
frames within the recording, as a half-open interval.
backend support for live stream (#59) This is so far completely untested, for use by a new UI prototype. It creates a new URL endpoint which sends one video/mp4 media segment per key frame, with the dependent frames included. This means there will be about one key frame interval of latency (typically about a second). This seems hard to avoid, as mentioned in issue #59.
2019-01-21 18:58:52 -05:00
rework WebSocket error return protocol This gives much better information to the UI layer, getting rid of a whole troubleshooting guide entry. See #119 #132 #218 #219 I also restructured the code in anticipation of a new WebSocket event stream (#40).
2023-02-15 10:04:50 -05:00
The WebSocket will always open immediately but will receive messages only while
the backing RTSP stream is connected.
backend support for live stream (#59) This is so far completely untested, for use by a new UI prototype. It creates a new URL endpoint which sends one video/mp4 media segment per key frame, with the dependent frames included. This means there will be about one key frame interval of latency (typically about a second). This seems hard to avoid, as mentioned in issue #59.
2019-01-21 18:58:52 -05:00
Example request URI:
```
/api/cameras/fd20f7a2-9d69-4cb3-94ed-d51a20c3edfe/main/live.m4s
```
switch to websocket for live stream (#59) The multipart stream / hanging GET approach worked in a prototype for a single stream, but Chrome has a per-host limit of six connections. If I try streaming all my cameras at once, I hit that limit. I can't open all the streams, much less additional connections to load init segments and such. Websockets apparently has a much higher limit of 256.
2020-02-29 00:41:31 -05:00
Example binary message sequence:
backend support for live stream (#59) This is so far completely untested, for use by a new UI prototype. It creates a new URL endpoint which sends one video/mp4 media segment per key frame, with the dependent frames included. This means there will be about one key frame interval of latency (typically about a second). This seems hard to avoid, as mentioned in issue #59.
2019-01-21 18:58:52 -05:00
```
Content-Type: video/mp4; codecs="avc1.640028"
switch to websocket for live stream (#59) The multipart stream / hanging GET approach worked in a prototype for a single stream, but Chrome has a per-host limit of six connections. If I try streaming all my cameras at once, I hit that limit. I can't open all the streams, much less additional connections to load init segments and such. Websockets apparently has a much higher limit of 256.
2020-02-29 00:41:31 -05:00
X-Recording-Id: 42.5680
X-Recording-Start: 130985461191810
fix live view This broke with the media vs wall duration split, part of #34.
2020-08-07 13:16:06 -04:00
X-Prev-Media-Duration: 10000000
X-Media-Time-Range: 5220058-5400061
use Blake3 instead of SHA-1 or Blake2b Benefits: * Blake3 is faster. This is most noticeable for the hashing of the sample file data. * we no longer need OpenSSL, which helps with shrinking the binary size (#70). sha1 basically forced OpenSSL usage; ring deliberately doesn't support this old algorithm, and the pure-Rust sha1 crate is painfully slow. OpenSSL might still be a better choice than ring/rustls for TLS but it's nice to have the option. For the video sample entries, I decided we don't need to hash at all. I think the id number is sufficiently stable, and it's okay---perhaps even desirable---if an existing init segment changes for fixes like e5b83c2.
2020-03-20 23:52:30 -04:00
X-Video-Sample-Entry-Id: 4
backend support for live stream (#59) This is so far completely untested, for use by a new UI prototype. It creates a new URL endpoint which sends one video/mp4 media segment per key frame, with the dependent frames included. This means there will be about one key frame interval of latency (typically about a second). This seems hard to avoid, as mentioned in issue #59.
2019-01-21 18:58:52 -05:00
binary mp4 data
switch to websocket for live stream (#59) The multipart stream / hanging GET approach worked in a prototype for a single stream, but Chrome has a per-host limit of six connections. If I try streaming all my cameras at once, I hit that limit. I can't open all the streams, much less additional connections to load init segments and such. Websockets apparently has a much higher limit of 256.
2020-02-29 00:41:31 -05:00
```
backend support for live stream (#59) This is so far completely untested, for use by a new UI prototype. It creates a new URL endpoint which sends one video/mp4 media segment per key frame, with the dependent frames included. This means there will be about one key frame interval of latency (typically about a second). This seems hard to avoid, as mentioned in issue #59.
2019-01-21 18:58:52 -05:00
switch to websocket for live stream (#59) The multipart stream / hanging GET approach worked in a prototype for a single stream, but Chrome has a per-host limit of six connections. If I try streaming all my cameras at once, I hit that limit. I can't open all the streams, much less additional connections to load init segments and such. Websockets apparently has a much higher limit of 256.
2020-02-29 00:41:31 -05:00
```
backend support for live stream (#59) This is so far completely untested, for use by a new UI prototype. It creates a new URL endpoint which sends one video/mp4 media segment per key frame, with the dependent frames included. This means there will be about one key frame interval of latency (typically about a second). This seems hard to avoid, as mentioned in issue #59.
2019-01-21 18:58:52 -05:00
Content-Type: video/mp4; codecs="avc1.640028"
switch to websocket for live stream (#59) The multipart stream / hanging GET approach worked in a prototype for a single stream, but Chrome has a per-host limit of six connections. If I try streaming all my cameras at once, I hit that limit. I can't open all the streams, much less additional connections to load init segments and such. Websockets apparently has a much higher limit of 256.
2020-02-29 00:41:31 -05:00
X-Recording-Id: 42.5681
X-Recording-Start: 130985461191822
fix live view This broke with the media vs wall duration split, part of #34.
2020-08-07 13:16:06 -04:00
X-Prev-Media-Duration: 10180003
X-Media-Time-Range: 0-180002
use Blake3 instead of SHA-1 or Blake2b Benefits: * Blake3 is faster. This is most noticeable for the hashing of the sample file data. * we no longer need OpenSSL, which helps with shrinking the binary size (#70). sha1 basically forced OpenSSL usage; ring deliberately doesn't support this old algorithm, and the pure-Rust sha1 crate is painfully slow. OpenSSL might still be a better choice than ring/rustls for TLS but it's nice to have the option. For the video sample entries, I decided we don't need to hash at all. I think the id number is sufficiently stable, and it's okay---perhaps even desirable---if an existing init segment changes for fixes like e5b83c2.
2020-03-20 23:52:30 -04:00
X-Video-Sample-Entry-Id: 4
backend support for live stream (#59) This is so far completely untested, for use by a new UI prototype. It creates a new URL endpoint which sends one video/mp4 media segment per key frame, with the dependent frames included. This means there will be about one key frame interval of latency (typically about a second). This seems hard to avoid, as mentioned in issue #59.
2019-01-21 18:58:52 -05:00
binary mp4 data
switch to websocket for live stream (#59) The multipart stream / hanging GET approach worked in a prototype for a single stream, but Chrome has a per-host limit of six connections. If I try streaming all my cameras at once, I hit that limit. I can't open all the streams, much less additional connections to load init segments and such. Websockets apparently has a much higher limit of 256.
2020-02-29 00:41:31 -05:00
```
backend support for live stream (#59) This is so far completely untested, for use by a new UI prototype. It creates a new URL endpoint which sends one video/mp4 media segment per key frame, with the dependent frames included. This means there will be about one key frame interval of latency (typically about a second). This seems hard to avoid, as mentioned in issue #59.
2019-01-21 18:58:52 -05:00
switch to websocket for live stream (#59) The multipart stream / hanging GET approach worked in a prototype for a single stream, but Chrome has a per-host limit of six connections. If I try streaming all my cameras at once, I hit that limit. I can't open all the streams, much less additional connections to load init segments and such. Websockets apparently has a much higher limit of 256.
2020-02-29 00:41:31 -05:00
```
backend support for live stream (#59) This is so far completely untested, for use by a new UI prototype. It creates a new URL endpoint which sends one video/mp4 media segment per key frame, with the dependent frames included. This means there will be about one key frame interval of latency (typically about a second). This seems hard to avoid, as mentioned in issue #59.
2019-01-21 18:58:52 -05:00
Content-Type: video/mp4; codecs="avc1.640028"
switch to websocket for live stream (#59) The multipart stream / hanging GET approach worked in a prototype for a single stream, but Chrome has a per-host limit of six connections. If I try streaming all my cameras at once, I hit that limit. I can't open all the streams, much less additional connections to load init segments and such. Websockets apparently has a much higher limit of 256.
2020-02-29 00:41:31 -05:00
X-Recording-Id: 42.5681
X-Recording-Start: 130985461191822
fix live view This broke with the media vs wall duration split, part of #34.
2020-08-07 13:16:06 -04:00
X-Prev-Media-Duration: 10360005
X-Media-Time-Range: 180002-360004
use Blake3 instead of SHA-1 or Blake2b Benefits: * Blake3 is faster. This is most noticeable for the hashing of the sample file data. * we no longer need OpenSSL, which helps with shrinking the binary size (#70). sha1 basically forced OpenSSL usage; ring deliberately doesn't support this old algorithm, and the pure-Rust sha1 crate is painfully slow. OpenSSL might still be a better choice than ring/rustls for TLS but it's nice to have the option. For the video sample entries, I decided we don't need to hash at all. I think the id number is sufficiently stable, and it's okay---perhaps even desirable---if an existing init segment changes for fixes like e5b83c2.
2020-03-20 23:52:30 -04:00
X-Video-Sample-Entry-Id: 4
backend support for live stream (#59) This is so far completely untested, for use by a new UI prototype. It creates a new URL endpoint which sends one video/mp4 media segment per key frame, with the dependent frames included. This means there will be about one key frame interval of latency (typically about a second). This seems hard to avoid, as mentioned in issue #59.
2019-01-21 18:58:52 -05:00
binary mp4 data
```
live stream frame-by-frame rather than GOP-by-GOP (#59) This should reduce live stream latency by two seconds when my cameras are at their default setting (I frame interval = 2 * frame rate)! I was under the impression that every HTML5 Media Source Extensions media segment had to start with a Random Access Point. This used to be true, but apparently changed quite a while ago: https://bugs.chromium.org/p/chromium/issues/detail?id=229412 Support generating segments that don't start with a key frame, and plumb this through the mp4 media segment generation logic. Add some extra error checking in mp4 slice handling, as my first attempts had a mismatch between expected and actual lengths that silently returned corrupted .m4s files. Also pull everything from the most recent key frame on along with the first live segment to reduce startup latency. Live view is quite a bit more pleasant now.
2020-08-07 18:30:22 -04:00
These roughly correspond to the `.m4s` files available at the following URLs:
backend support for live stream (#59) This is so far completely untested, for use by a new UI prototype. It creates a new URL endpoint which sends one video/mp4 media segment per key frame, with the dependent frames included. This means there will be about one key frame interval of latency (typically about a second). This seems hard to avoid, as mentioned in issue #59.
2019-01-21 18:58:52 -05:00
fix live view This broke with the media vs wall duration split, part of #34.
2020-08-07 13:16:06 -04:00
* `/api/cameras/fd20f7a2-9d69-4cb3-94ed-d51a20c3edfe/main/view.m4s?s=5680@42.5220058-5400061`
* `/api/cameras/fd20f7a2-9d69-4cb3-94ed-d51a20c3edfe/main/view.m4s?s=5681@42.0-180002`
* `/api/cameras/fd20f7a2-9d69-4cb3-94ed-d51a20c3edfe/main/view.m4s?s=5681@42.180002-360004`
backend support for live stream (#59) This is so far completely untested, for use by a new UI prototype. It creates a new URL endpoint which sends one video/mp4 media segment per key frame, with the dependent frames included. This means there will be about one key frame interval of latency (typically about a second). This seems hard to avoid, as mentioned in issue #59.
2019-01-21 18:58:52 -05:00
live stream frame-by-frame rather than GOP-by-GOP (#59) This should reduce live stream latency by two seconds when my cameras are at their default setting (I frame interval = 2 * frame rate)! I was under the impression that every HTML5 Media Source Extensions media segment had to start with a Random Access Point. This used to be true, but apparently changed quite a while ago: https://bugs.chromium.org/p/chromium/issues/detail?id=229412 Support generating segments that don't start with a key frame, and plumb this through the mp4 media segment generation logic. Add some extra error checking in mp4 slice handling, as my first attempts had a mismatch between expected and actual lengths that silently returned corrupted .m4s files. Also pull everything from the most recent key frame on along with the first live segment to reduce startup latency. Live view is quite a bit more pleasant now.
2020-08-07 18:30:22 -04:00
However, there are two important differences:
* The `/view.m4s` endpoint accepts offsets within a recording as wall durations;
the `/live.m4s` endpoint's `X-Media-Time-Range` header returns them as
media durations. Thus the URLs above are only exactly correct if the wall
and media durations of the recording are identical.
* The `/view.m4s` endpoint always returns a time range that starts with a key frame;
`/live.m4s` messages may not include a key frame.
switch to websocket for live stream (#59) The multipart stream / hanging GET approach worked in a prototype for a single stream, but Chrome has a per-host limit of six connections. If I try streaming all my cameras at once, I hit that limit. I can't open all the streams, much less additional connections to load init segments and such. Websockets apparently has a much higher limit of 256.
2020-02-29 00:41:31 -05:00
Note: an earlier version of this API used a `multipart/mixed` segment instead,
compatible with the [multipart-stream-js][multipart-stream-js] library. The
problem with this approach is that browsers have low limits on the number of
active HTTP/1.1 connections: six in Chrome's case. The WebSocket limit is much
higher (256), allowing browser-side Javascript to stream all active camera
streams simultaneously as well as making other simultaneous HTTP requests.
use Blake3 instead of SHA-1 or Blake2b Benefits: * Blake3 is faster. This is most noticeable for the hashing of the sample file data. * we no longer need OpenSSL, which helps with shrinking the binary size (#70). sha1 basically forced OpenSSL usage; ring deliberately doesn't support this old algorithm, and the pure-Rust sha1 crate is painfully slow. OpenSSL might still be a better choice than ring/rustls for TLS but it's nice to have the option. For the video sample entries, I decided we don't need to hash at all. I think the id number is sufficiently stable, and it's okay---perhaps even desirable---if an existing init segment changes for fixes like e5b83c2.
2020-03-20 23:52:30 -04:00
### `GET /api/init/<id>.mp4`
support segmented mp4s This is intended to support HTML5 Media Source Extensions, which I expect to be the most practical way to make a good web UI with a proper scrub bar and such. This feature has had very limited testing on Chrome and Firefox, and that was not entirely successful. More work is needed before it's usable, but this seems like a helpful progress checkpoint.
2017-10-01 18:29:22 -04:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
Returns a `.mp4` suitable for use as a [HTML5 Media Source Extensions
serve 'video/mp4; codecs="avc1.xxxxxx"' mime type This can be used when constructing a HTML5 SourceBuffer.
2017-10-04 02:25:58 -04:00
initialization segment][init-segment]. The MIME type will be `video/mp4`, with
Documentation updates
2021-09-29 21:39:18 -04:00
a `codecs` parameter as specified in [RFC 6381][rfc-6381]. The `<id>` should be a value
previously extracted from the `X-Video-Sample-Entry-Id` header returned in a `.../live.m4s` response.
support segmented mp4s This is intended to support HTML5 Media Source Extensions, which I expect to be the most practical way to make a good web UI with a proper scrub bar and such. This feature has had very limited testing on Chrome and Firefox, and that was not entirely successful. More work is needed before it's usable, but this seems like a helpful progress checkpoint.
2017-10-01 18:29:22 -04:00
UI: improve aspect ratio handling As written in the changelog: Live streams formerly worked around a Firefox pixel aspect ratio bug by forcing all videos to 16:9, which dramatically distorted 9:16 camera views. Playback didn't, so anamorphic videos looked correct on Chrome but slightly stretched on Firefox. Now both live streams and playback are fully correct on all browsers.
2021-08-12 16:32:01 -04:00
An `X-Aspect` HTTP header will include the aspect ratio as width:height,
eg `16:9` (most cameras) or `9:16` (rotated 90 degrees).
This is redundant with the returned `.mp4` but is far easier to parse from
Javascript.
use Blake3 instead of SHA-1 or Blake2b Benefits: * Blake3 is faster. This is most noticeable for the hashing of the sample file data. * we no longer need OpenSSL, which helps with shrinking the binary size (#70). sha1 basically forced OpenSSL usage; ring deliberately doesn't support this old algorithm, and the pure-Rust sha1 crate is painfully slow. OpenSSL might still be a better choice than ring/rustls for TLS but it's nice to have the option. For the video sample entries, I decided we don't need to hash at all. I think the id number is sufficiently stable, and it's okay---perhaps even desirable---if an existing init segment changes for fixes like e5b83c2.
2020-03-20 23:52:30 -04:00
### `GET /api/init/<id>.mp4.txt`
add a url for getting debug info about a .mp4 file and add a unit test of path decoding along the way
2018-12-29 14:06:44 -05:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
Returns a `text/plain` debugging string for the `.mp4` generated by the
add a url for getting debug info about a .mp4 file and add a unit test of path decoding along the way
2018-12-29 14:06:44 -05:00
same URL minus the `.txt` suffix.
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
### `GET /api/signals`
read-only signals support (#28) This is mostly untested and useless by itself, but it's a starting point. In particular: * there's no way to set up signals or add/remove/update events yet except by manual changes to the database. * if you associate a signal with a camera then remove the camera, hitting /api/ will error out.
2019-06-06 19:18:13 -04:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
Returns an `application/json` response with state of every signal for the
requested timespan.
read-only signals support (#28) This is mostly untested and useless by itself, but it's a starting point. In particular: * there's no way to set up signals or add/remove/update events yet except by manual changes to the database. * if you associate a signal with a camera then remove the camera, hitting /api/ will error out.
2019-06-06 19:18:13 -04:00
Valid request parameters:
* `startTime90k` and and `endTime90k` limit the data returned to only
events relevant to the given half-open interval. Either or both
may be absent; they default to the beginning and end of time, respectively.
This will return the current state as of the latest change (to any signal)
before the start time (if any), then all changes in the interval. This
allows the caller to determine the state at every moment during the
present signal days in API requests I also enforced some invariants in the signals code, fixing a couple bugs. The signals code is more complex than I'd like, but hopefully is working now.
2021-03-23 23:22:29 -04:00
selected timespan, as well as observe all events.
read-only signals support (#28) This is mostly untested and useless by itself, but it's a starting point. In particular: * there's no way to set up signals or add/remove/update events yet except by manual changes to the database. * if you associate a signal with a camera then remove the camera, hitting /api/ will error out.
2019-06-06 19:18:13 -04:00
Responses are several parallel arrays for each observation:
* `times90k`: the time of each event. Events are given in ascending order.
* `signalIds`: the id of the relevant signal; expected to match one in the
`signals` field of the `/api/` response.
* `states`: the new state.
Example request URI (with added whitespace between parameters):
```
/api/signals
?startTime90k=130888729442361
&endTime90k=130985466591817
```
Example response:
```json
{
"signalIds": [1, 1, 1],
"states": [1, 2, 1],
"times90k": [130888729440000, 130985424000000, 130985418600000]
}
```
This represents the following observations:
1. time 130888729440000 was the last change before the requested start;
signal 1 (`driveway motion`) was in state 1 (`off`).
2. signal 1 entered state 2 (`on`) at time 130985424000000.
3. signal 1 entered state 1 (`off`) at time 130985418600000.
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
### `POST /api/signals`
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
Requires the `updateSignals` permission.
add concept of user/session permissions (I also considered the names "capabilities" and "scopes", but I think "permissions" is the most widely understood.) This is increasingly necessary as the web API becomes more capable. Among other things, it allows: * non-administrator users who can view but not access camera passwords or change any state * workers that update signal state based on cameras' built-in motion detection or a security system's events but don't need to view videos * control over what can be done without authenticating Currently session permissions are just copied from user permissions, but you can also imagine admin sessions vs not, as a checkbox when signing in. This would match the standard Unix workflow of using a non-administrative session most of the time. Relevant to my current signals work (#28) and to the addition of an administrative API (#35, including #66).
2019-06-19 18:17:50 -04:00
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
Alters the state of a signal.
db crate support for updating signals (#28) This is a definite work in progress. In particular, * there's no src/web.rs support yet so it can't be used, * the code is surprisingly complex, and there's almost no tests so far. I want to at least get complete branch coverage. * I may still go back to time_sec rather than time_90k to save RAM and flash. I simplified the approach a bit from the earlier goal in design/api.md. In particular, there's no longer the separate concept of "observation" vs "prediction". Now the predictions are just observations that extend a bit beyond now. They may be flushed prematurely and I'll try living with that to avoid making things even more complex.
2019-06-14 00:55:15 -04:00
A typical client might be a subscriber of a camera's built-in motion
detection event stream or of a security system's zone status event stream.
It makes a request on every event or on every 30 second timeout, predicting
that the state will last for a minute. This prediction may be changed later.
Writing to the near future in this way ensures that the UI never displays
`unknown` when the client is actively managing the signal.
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
Some requests may instead backfill earlier history, such as when a video
analytics client starts up and analyzes all video segments recorded since it
db crate support for updating signals (#28) This is a definite work in progress. In particular, * there's no src/web.rs support yet so it can't be used, * the code is surprisingly complex, and there's almost no tests so far. I want to at least get complete branch coverage. * I may still go back to time_sec rather than time_90k to save RAM and flash. I simplified the approach a bit from the earlier goal in design/api.md. In particular, there's no longer the separate concept of "observation" vs "prediction". Now the predictions are just observations that extend a bit beyond now. They may be flushed prematurely and I'll try living with that to avoid making things even more complex.
2019-06-14 00:55:15 -04:00
last ran. These will specify beginning and end times.
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
The request should have an `application/json` body describing the change to
UI preferences: #153 #155
2021-09-01 18:01:42 -04:00
make. It should be a JSON object with these attributes:
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
* `csrf`: a CSRF token, required when using session authentication.
db crate support for updating signals (#28) This is a definite work in progress. In particular, * there's no src/web.rs support yet so it can't be used, * the code is surprisingly complex, and there's almost no tests so far. I want to at least get complete branch coverage. * I may still go back to time_sec rather than time_90k to save RAM and flash. I simplified the approach a bit from the earlier goal in design/api.md. In particular, there's no longer the separate concept of "observation" vs "prediction". Now the predictions are just observations that extend a bit beyond now. They may be flushed prematurely and I'll try living with that to avoid making things even more complex.
2019-06-14 00:55:15 -04:00
* `signalIds`: a list of signal ids to change. Must be sorted.
* `states`: a list (one per `signalIds` entry) of states to set.
UI preferences: #153 #155
2021-09-01 18:01:42 -04:00
* `start`: the starting time of the change, as a JSON object of the form
refine timestamps in json signals api * API change: in update signals, allow setting a start time relative to now. This is an accuracy improvement in the case where the client has been retrying an initial request for a while. Kind of an obscure corner case but easy enough to address. And use a more convenient enum representation. * in update signals, choose `now` before acquiring the database lock. If lock acquisition takes a long time, this more accurately reflects the time the caller intended. * in general, make Time and Duration (de)serializable and use them in json types. This makes the types more self-describing, with better debug printing on both the server side and on the client library (in moonfire-playground). To make this work, base has to import serde which initially seemed like poor layering to me, but serde seems to be imported in some pretty foundational Rust crates for this reason. I'll go with it.
2021-04-21 13:44:01 -04:00
`{'base': 'epoch', 'rel90k': t}` or `{'base': 'now', 'rel90k': t}`. In
the `epoch` form, `rel90k` is 90 kHz units since 1970-01-01 00:00:00 UTC.
In the `now` form, `rel90k` is relative to current time and may be
negative.
* `end`: the ending time of the change, in the same form as `start`.
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
UI preferences: #153 #155
2021-09-01 18:01:42 -04:00
The response will be an `application/json` body JSON object with the following
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
attributes:
db crate support for updating signals (#28) This is a definite work in progress. In particular, * there's no src/web.rs support yet so it can't be used, * the code is surprisingly complex, and there's almost no tests so far. I want to at least get complete branch coverage. * I may still go back to time_sec rather than time_90k to save RAM and flash. I simplified the approach a bit from the earlier goal in design/api.md. In particular, there's no longer the separate concept of "observation" vs "prediction". Now the predictions are just observations that extend a bit beyond now. They may be flushed prematurely and I'll try living with that to avoid making things even more complex.
2019-06-14 00:55:15 -04:00
* `time90k`: the current time. When the request's `startTime90k` is absent
and/or its `endBase` is `now`, this is needed to know the effect of the
earlier request.
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
Example request sequence:
#### Request 1
The client responsible for reporting live driveway motion has just started. It
observes motion now. It records no history and predicts there will be motion
for the next minute.
Request:
```json
{
db crate support for updating signals (#28) This is a definite work in progress. In particular, * there's no src/web.rs support yet so it can't be used, * the code is surprisingly complex, and there's almost no tests so far. I want to at least get complete branch coverage. * I may still go back to time_sec rather than time_90k to save RAM and flash. I simplified the approach a bit from the earlier goal in design/api.md. In particular, there's no longer the separate concept of "observation" vs "prediction". Now the predictions are just observations that extend a bit beyond now. They may be flushed prematurely and I'll try living with that to avoid making things even more complex.
2019-06-14 00:55:15 -04:00
"signalIds": [1],
"states": [2],
refine timestamps in json signals api * API change: in update signals, allow setting a start time relative to now. This is an accuracy improvement in the case where the client has been retrying an initial request for a while. Kind of an obscure corner case but easy enough to address. And use a more convenient enum representation. * in update signals, choose `now` before acquiring the database lock. If lock acquisition takes a long time, this more accurately reflects the time the caller intended. * in general, make Time and Duration (de)serializable and use them in json types. This makes the types more self-describing, with better debug printing on both the server side and on the client library (in moonfire-playground). To make this work, base has to import serde which initially seemed like poor layering to me, but serde seems to be imported in some pretty foundational Rust crates for this reason. I'll go with it.
2021-04-21 13:44:01 -04:00
"start": {"base": "now", "rel90k": 0},
"end": {"base": "now", "rel90k": 5400000}
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
}
```
Response:
```json
{
db crate support for updating signals (#28) This is a definite work in progress. In particular, * there's no src/web.rs support yet so it can't be used, * the code is surprisingly complex, and there's almost no tests so far. I want to at least get complete branch coverage. * I may still go back to time_sec rather than time_90k to save RAM and flash. I simplified the approach a bit from the earlier goal in design/api.md. In particular, there's no longer the separate concept of "observation" vs "prediction". Now the predictions are just observations that extend a bit beyond now. They may be flushed prematurely and I'll try living with that to avoid making things even more complex.
2019-06-14 00:55:15 -04:00
"time90k": 140067468000000
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
}
```
#### Request 2
30 seconds later (half the prediction interval), the client still observes
db crate support for updating signals (#28) This is a definite work in progress. In particular, * there's no src/web.rs support yet so it can't be used, * the code is surprisingly complex, and there's almost no tests so far. I want to at least get complete branch coverage. * I may still go back to time_sec rather than time_90k to save RAM and flash. I simplified the approach a bit from the earlier goal in design/api.md. In particular, there's no longer the separate concept of "observation" vs "prediction". Now the predictions are just observations that extend a bit beyond now. They may be flushed prematurely and I'll try living with that to avoid making things even more complex.
2019-06-14 00:55:15 -04:00
motion. It leaves the prior data alone and predicts the motion will continue.
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
Request:
```json
{
db crate support for updating signals (#28) This is a definite work in progress. In particular, * there's no src/web.rs support yet so it can't be used, * the code is surprisingly complex, and there's almost no tests so far. I want to at least get complete branch coverage. * I may still go back to time_sec rather than time_90k to save RAM and flash. I simplified the approach a bit from the earlier goal in design/api.md. In particular, there's no longer the separate concept of "observation" vs "prediction". Now the predictions are just observations that extend a bit beyond now. They may be flushed prematurely and I'll try living with that to avoid making things even more complex.
2019-06-14 00:55:15 -04:00
"signalIds": [1],
"states": [2],
refine timestamps in json signals api * API change: in update signals, allow setting a start time relative to now. This is an accuracy improvement in the case where the client has been retrying an initial request for a while. Kind of an obscure corner case but easy enough to address. And use a more convenient enum representation. * in update signals, choose `now` before acquiring the database lock. If lock acquisition takes a long time, this more accurately reflects the time the caller intended. * in general, make Time and Duration (de)serializable and use them in json types. This makes the types more self-describing, with better debug printing on both the server side and on the client library (in moonfire-playground). To make this work, base has to import serde which initially seemed like poor layering to me, but serde seems to be imported in some pretty foundational Rust crates for this reason. I'll go with it.
2021-04-21 13:44:01 -04:00
"start": {"base": "epoch", "rel90k": 140067468000000},
"end": {"base": "now", "rel90k": 5400000}
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
}
```
Response:
```json
{
db crate support for updating signals (#28) This is a definite work in progress. In particular, * there's no src/web.rs support yet so it can't be used, * the code is surprisingly complex, and there's almost no tests so far. I want to at least get complete branch coverage. * I may still go back to time_sec rather than time_90k to save RAM and flash. I simplified the approach a bit from the earlier goal in design/api.md. In particular, there's no longer the separate concept of "observation" vs "prediction". Now the predictions are just observations that extend a bit beyond now. They may be flushed prematurely and I'll try living with that to avoid making things even more complex.
2019-06-14 00:55:15 -04:00
"time90k": 140067470700000
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
}
```
mass markdown reformatting Add tables of contents (using the VS Code Markdown All-In-One extension) and reformat lists to consistently use 4-space indents. No content changes.
2021-04-01 15:10:43 -04:00
#### Request 3
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
db crate support for updating signals (#28) This is a definite work in progress. In particular, * there's no src/web.rs support yet so it can't be used, * the code is surprisingly complex, and there's almost no tests so far. I want to at least get complete branch coverage. * I may still go back to time_sec rather than time_90k to save RAM and flash. I simplified the approach a bit from the earlier goal in design/api.md. In particular, there's no longer the separate concept of "observation" vs "prediction". Now the predictions are just observations that extend a bit beyond now. They may be flushed prematurely and I'll try living with that to avoid making things even more complex.
2019-06-14 00:55:15 -04:00
5 seconds later, the client observes motion has ended. It leaves the prior
data alone and predicts no more motion.
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
Request:
```json
{
db crate support for updating signals (#28) This is a definite work in progress. In particular, * there's no src/web.rs support yet so it can't be used, * the code is surprisingly complex, and there's almost no tests so far. I want to at least get complete branch coverage. * I may still go back to time_sec rather than time_90k to save RAM and flash. I simplified the approach a bit from the earlier goal in design/api.md. In particular, there's no longer the separate concept of "observation" vs "prediction". Now the predictions are just observations that extend a bit beyond now. They may be flushed prematurely and I'll try living with that to avoid making things even more complex.
2019-06-14 00:55:15 -04:00
"signalIds": [1],
"states": [2],
refine timestamps in json signals api * API change: in update signals, allow setting a start time relative to now. This is an accuracy improvement in the case where the client has been retrying an initial request for a while. Kind of an obscure corner case but easy enough to address. And use a more convenient enum representation. * in update signals, choose `now` before acquiring the database lock. If lock acquisition takes a long time, this more accurately reflects the time the caller intended. * in general, make Time and Duration (de)serializable and use them in json types. This makes the types more self-describing, with better debug printing on both the server side and on the client library (in moonfire-playground). To make this work, base has to import serde which initially seemed like poor layering to me, but serde seems to be imported in some pretty foundational Rust crates for this reason. I'll go with it.
2021-04-21 13:44:01 -04:00
"start": {"base": "now", "rel90k": 0},
"end": {"base": "now", "rel90k": 5400000}
db crate support for updating signals (#28) This is a definite work in progress. In particular, * there's no src/web.rs support yet so it can't be used, * the code is surprisingly complex, and there's almost no tests so far. I want to at least get complete branch coverage. * I may still go back to time_sec rather than time_90k to save RAM and flash. I simplified the approach a bit from the earlier goal in design/api.md. In particular, there's no longer the separate concept of "observation" vs "prediction". Now the predictions are just observations that extend a bit beyond now. They may be flushed prematurely and I'll try living with that to avoid making things even more complex.
2019-06-14 00:55:15 -04:00
}
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
}
```
Response:
```json
{
db crate support for updating signals (#28) This is a definite work in progress. In particular, * there's no src/web.rs support yet so it can't be used, * the code is surprisingly complex, and there's almost no tests so far. I want to at least get complete branch coverage. * I may still go back to time_sec rather than time_90k to save RAM and flash. I simplified the approach a bit from the earlier goal in design/api.md. In particular, there's no longer the separate concept of "observation" vs "prediction". Now the predictions are just observations that extend a bit beyond now. They may be flushed prematurely and I'll try living with that to avoid making things even more complex.
2019-06-14 00:55:15 -04:00
"time90k": 140067471150000
document proposed API for updating signals (#28)
2019-06-07 13:19:38 -04:00
}
```
add GET /users/ endpoint
2022-12-24 13:09:05 -05:00
### User management
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
#### `GET /api/users/`
add GET /users/ endpoint
2022-12-24 13:09:05 -05:00
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
Requires the `adminUsers` permission.
add GET /users/ endpoint
2022-12-24 13:09:05 -05:00
Lists all users. Currently there's no paging. Returns a JSON object with
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
a `users` key with an array of objects, each with the following keys:
* `id`: a number.
user admin api improvements
2023-01-08 04:14:03 -05:00
* `user`: a `UserSubset`.
add GET /users/ endpoint
2022-12-24 13:09:05 -05:00
user admin api improvements
2023-01-08 04:14:03 -05:00
#### `POST /api/users/`
more user admin actions
2022-12-24 15:21:06 -05:00
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
Requires the `adminUsers` permission.
more user admin actions
2022-12-24 15:21:06 -05:00
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
Adds a user. Expects a JSON object as follows:
more user admin actions
2022-12-24 15:21:06 -05:00
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
* `csrf`: a CSRF token, required when using session authentication.
* `user`: a `UserSubset` as defined below.
more user admin actions
2022-12-24 15:21:06 -05:00
Returns status 204 (No Content) on success.
add GET /users/ endpoint
2022-12-24 13:09:05 -05:00
#### `GET /api/users/<id>`
retrieve and set users' permissions
2022-12-24 12:38:13 -05:00
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
Retrieves the user. Requires the `adminUsers` permission if the caller is
retrieve and set users' permissions
2022-12-24 12:38:13 -05:00
not authenticated as the user in question.
user admin api improvements
2023-01-08 04:14:03 -05:00
Returns a HTTP status 200 on success with a JSON `UserSubset`.
retrieve and set users' permissions
2022-12-24 12:38:13 -05:00
user admin api improvements
2023-01-08 04:14:03 -05:00
#### `PATCH /api/users/<id>`
UI preferences: #153 #155
2021-09-01 18:01:42 -04:00
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
Updates the given user. Requires the `adminUsers` permission if the caller is
not authenticated as the user in question.
UI preferences: #153 #155
2021-09-01 18:01:42 -04:00
Expects a JSON object:
extend POST /users/:id Now you can set a password for a user while the server is running, e.g. via the following command: ```shell curl \ -H 'Content-Type: application/json' \ -d '{"update": {"password": "asdf"}}' \ --unix-socket /var/lib/moonfire-nvr/sock \ http://nvr/api/users/1 ```
2022-12-23 15:43:00 -05:00
* `csrf`: a CSRF token, required when using session authentication.
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
* `update`: `UserSubset`, sets the provided fields. Field-specific notes:
support disabling/enabling users by http api
2023-01-31 09:47:25 -05:00
* `disabled`: requires `adminUsers` permission.
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
* `password`: when updating the password, the previous password must
user admin api improvements
2023-01-08 04:14:03 -05:00
be supplied as a precondition, unless the caller has `adminUsers`
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
permission.
* `permissions`: requires `adminUsers` permission. Note that updating a
user's permissions currently neither adds nor limits permissions of
existing sessions; it only changes what is available to newly created
sessions.
support disabling/enabling users by http api
2023-01-31 09:47:25 -05:00
* `username`: requires `adminUsers` permission.
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
* `precondition`: `UserSubset`, forces the request to fail with HTTP status
412 (Precondition failed) if the provided fields don't have the given
values.
UI preferences: #153 #155
2021-09-01 18:01:42 -04:00
Returns HTTP status 204 (No Content) on success.
more user admin actions
2022-12-24 15:21:06 -05:00
#### `DELETE /api/users/<id>`
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
Deletes the given user. Requires the `adminUsers` permission.
Expects a JSON object body with the following parameters:
* `csrf`: a CSRF token, required when using session authentication.
more user admin actions
2022-12-24 15:21:06 -05:00
Returns HTTP status 204 (No Content) on success.
make `GET /api/` return current permissions This is useful for e.g. deciding whether or not to present the user admin UI in navigation. As part of this change, I adjusted the casing in Permissions, and then all the toml stuff for consistency. Noted in changelog.
2022-12-31 12:08:26 -05:00
## Types
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
### UserSubset
A JSON object with any of the following parameters:
support disabling/enabling users by http api
2023-01-31 09:47:25 -05:00
* `disabled`, boolean indicating if all logins from the user are rejected.
user admin api improvements
2023-01-08 04:14:03 -05:00
* `password`
* on retrieval, a placeholder string to indicate a password is set,
or null.
* in preconditions, may be left absent to ignore, set to null to require
no password, or set to a plaintext string.
* in updates, may be left absent to keep as-is, set to null to disable
session creation, or set to a plaintext string.
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
* `permissions`, a `Permissions` as described below.
support disabling/enabling users by http api
2023-01-31 09:47:25 -05:00
* `preferences`, a JSON object which the server stores without interpreting.
This field is meant for user-level preferences meaningful to the UI.
* `username`
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
make `GET /api/` return current permissions This is useful for e.g. deciding whether or not to present the user admin UI in navigation. As part of this change, I adjusted the casing in Permissions, and then all the toml stuff for consistency. Noted in changelog.
2022-12-31 12:08:26 -05:00
### Permissions
A JSON object of permissions to perform various actions:
* `adminUsers`: bool
* `readCameraConfigs`: bool, read camera configs including credentials
* `updateSignals`: bool
* `viewVideo`: bool
See endpoints above for more details on the contexts in which these are
required.
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
## Cross-site request forgery (CSRF) protection
The API includes several standard protections against [cross-site request
forgery](https://en.wikipedia.org/wiki/Cross-site_request_forgery) attacks, in
which a malicious third-party website convinces the user's browser to send
requests on its behalf.
The following protections apply regardless of method of authentication (session,
no authentication + `allowUnauthenticatedPermissions`, or the browser proxying
to a Unix socket with `ownUidIsPrivileged`):
* The `GET` method is always "safe". Actions that have significant side
effects require another method such as `DELETE`, `POST`, or `PUT`. This
prevents simple hyperlinks from causing damage.
* Mutations always require some non-default request header (e.g.
`Content-Type: application/json`) so that a `<form method="POST">` will be
rejected.
* The server does *not* override the default
[CORS](https://en.wikipedia.org/wiki/Cross-origin_resource_sharing) policy.
Thus, cross-domain Ajax requests (via `XMLHTTPRequest` or `fetch`) should
fail.
rework WebSocket error return protocol This gives much better information to the UI layer, getting rid of a whole troubleshooting guide entry. See #119 #132 #218 #219 I also restructured the code in anticipation of a new WebSocket event stream (#40).
2023-02-15 10:04:50 -05:00
* WebSocket upgrade requests are rejected if the `Origin` header is present
and does not match `Host`. This is the sole protection against
[Cross-Site WebSocket Hijacting (CSWSH)](https://christian-schneider.net/CrossSiteWebSocketHijacking.html).
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
The following additional protections apply only when using session
authentication:
* Session cookies are set with the [`SameSite=Lax` attribute](samesite-lax),
so that sufficiently modern web browsers will never send the session cookie
on subrequests. (Note they still send session cookies when following links
and on WebSocket upgrade. In these cases, we rely on the protections
described above.)
* Mutations use a `csrf` token, requiring the caller to prove it is able
to read the `GET /api/` response. (This is subect to change. We may decide
to implement these tokens in a way that doesn't require session
authentication or decide they're entirely unnecessary.)
support segmented mp4s This is intended to support HTML5 Media Source Extensions, which I expect to be the most practical way to make a good web UI with a proper scrub bar and such. This feature has had very limited testing on Chrome and Firefox, and that was not entirely successful. More work is needed before it's usable, but this seems like a helpful progress checkpoint.
2017-10-01 18:29:22 -04:00
[media-segment]: https://w3c.github.io/media-source/isobmff-byte-stream-format.html#iso-media-segments
[init-segment]: https://w3c.github.io/media-source/isobmff-byte-stream-format.html#iso-init-segments
[rfc-6381]: https://tools.ietf.org/html/rfc6381
switch to websocket for live stream (#59) The multipart stream / hanging GET approach worked in a prototype for a single stream, but Chrome has a per-host limit of six connections. If I try streaming all my cameras at once, I hit that limit. I can't open all the streams, much less additional connections to load init segments and such. Websockets apparently has a much higher limit of 256.
2020-02-29 00:41:31 -05:00
[rfc-6455]: https://tools.ietf.org/html/rfc6455
[multipart-mixed-js]: https://github.com/scottlamb/multipart-mixed-js
tweaks to api and docs In particular, the docs now talk about the CSRF protection. This is increasing relevant as we start having more mutation endpoints. And make the signals api expect a csrf for session auth to match the newer users api.
2023-01-05 13:11:28 -05:00
[samesite-lax]: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Set-Cookie/SameSite#lax