Commit Graph

527 Commits

Author SHA1 Message Date
Scott Lamb
84406a8123 Add a fast path to Mp4SampleTablePieces::Init.
This avoids iteration through the video index for the "interior" recordings of
a virtual file. This takes generating the size of a ~8-hour / 15 fps file from
about 60 ms to about 10 ms. I expect better savings on a Raspberry Pi 2, for
longer records, and for higher frame rates. The total time here can be
significant; one one ~day-long recording on the Pi, it was several seconds.
I'm optimistic this will help with that.

It'd also be possible to optimize DecodeVar32 (perhaps by unrolling the loop)
but better to remove a call than to optimize one.

To add the fast path, we need a new field "video_sync_samples" in the
recording table to calculate the length of the stss table. Storage cost should
be minimal; I think typically two bytes in SQLite's record format (serial type
1, value < 128), described here: <https://www.sqlite.org/fileformat2.html>.
2016-01-14 15:41:45 -08:00
Scott Lamb
78c3b8dafa Fixes/improvements to mp4 VirtualFile generation.
* Fix the mdat box size, which was not properly including the length of the
  header itself. (The "mp4file" tool nicely diagnosed this corruption.)

* Fix the stsc box. The first number of each entry is meant to be a chunk
  index, not a sample index. This was causing strange behavior in basically
  any video player for multi-recording videos.

* Populate etag and last-modified so that Range: requests can work properly.
  The etag must be changed every time the generated file format changes.
  There's a serial number constant for this purpose and a test meant to help
  catch such problems.
2016-01-13 07:50:13 -08:00
Scott Lamb
d38eb9103e Helper for making large values human-readable. 2016-01-13 07:20:39 -08:00
Scott Lamb
95523c3522 ToHex shouldn't require padding between bytes.
This was getting obnoxious for SHA-1s, and particularly so when serving them
as etags.
2016-01-13 06:51:23 -08:00
Scott Lamb
29696688b5 Small Uuid class wrapping libuuid.
This will be used to generate the names of sample files,
as well as camera ids.
2016-01-12 09:46:21 -08:00
Scott Lamb
85b7027803 Logic for generating .mp4 virtual files.
This is still pretty rough. For example, there's no test coverage of virtual
files based on multiple recordings. The etag and last modified code are stubs.
And various other conditions aren't tested at all. But it does appear to work
in a test that does a round-trip from a .mp4 file, so it should be a decent
starting point.
2016-01-11 00:17:56 -08:00
Scott Lamb
798f1db039 h264.cc: handle both kinds of ffmpeg extradata. 2016-01-10 21:28:07 -08:00
Scott Lamb
29054d42a0 A few types of FileSlice useful for mp4 building. 2016-01-09 23:26:02 -08:00
Scott Lamb
eed2a69f7a Work around libevent bug 306 in evbuffer_add_file.
See <https://github.com/libevent/libevent/issues/306> for details.
2016-01-09 22:51:27 -08:00
Scott Lamb
bb7fb95b57 Helper for composing a VirtualFile from "slices". 2016-01-09 22:15:22 -08:00
Scott Lamb
c89907d785 Nit: fix typo in reserve. 2016-01-09 21:42:00 -08:00
Scott Lamb
1ca6e2a665 Bit less string copying in H.264 code. 2016-01-09 21:41:12 -08:00
Scott Lamb
15b1ee54a6 Compute the full AVCSampleEntry (avc1 box).
This code isn't pretty exactly---particularly the hardcoded lengths---but it
does work. I'll have a different mechanism for calculating the length and
nesting structure forthe more dynamic parts of the moov atom. This way is
convenient when generating a single string of mostly static data.
2016-01-09 21:13:05 -08:00
Scott Lamb
48d0473a4c Small helper for writing sample files safely.
Handles partial writes + checksumming.
2016-01-09 17:16:55 -08:00
Scott Lamb
30e0f73ae0 First portion of .mp4 generation logic. 2016-01-09 12:02:36 -08:00
Scott Lamb
c294d751b6 Add logic to create an AVCDecoderConfiguration. 2016-01-08 21:44:19 -08:00
Scott Lamb
dca9642c51 Remove unused/untested ffmpeg-test.cc.
This was a half-done thing added by mistake.
2016-01-08 21:40:59 -08:00
Scott Lamb
d00c0b2f12 Add ReadFileOrDie test util function.
This fixes a compilation error; I'd left it out of the previous commit adding
a sqlite-test.cc method which depends on it.
2016-01-08 21:39:42 -08:00
Scott Lamb
9af7eb8c14 Add small sqlite3 wrapper + start of schema. 2016-01-07 22:59:34 -08:00
Scott Lamb
a46df2c2e5 Minor File interface changes.
These are intended to make the sample file writing easier.
2016-01-06 23:38:46 -08:00
Scott Lamb
7b45f48027 Add (openssl-based) SHA-1 hashing. 2016-01-06 23:27:44 -08:00
Scott Lamb
60988f0646 Add sample index codec; fix schema doc. 2016-01-05 11:01:36 -08:00
Scott Lamb
23ba5e0049 Add util functions for binary encoding/decoding.
These will be used by the new sample index format.
2016-01-05 08:29:12 -08:00
Scott Lamb
cc0adc327b Rough draft of schema design doc. 2016-01-04 23:52:05 -08:00
Scott Lamb
320c4afa94 Fix incorrect path, commandline in README.md.
From Dolf Starreveld <dolf@starreveld.com>.
2016-01-04 22:06:55 -08:00
Scott Lamb
7968952295 Add a Filesystem interface for testability.
Not immediately adding any tests that take advantage of it.
The new storage schema should handle ENOSPC correctly, and this will aid in
testing it.
2016-01-02 10:51:04 -08:00
Scott Lamb
c9eda8ac15 Initial commit, with basic functionality. 2016-01-01 22:06:47 -08:00