Scott Lamb 4c7eed293f Construct HTTP responses incrementally. ... This isn't as much of a speed-up as you might imagine; most of the large HTTP content was mmap()ed files which are relatively efficient. The big improvement here is that it's now possible to serve large files (4 GiB and up) on 32-bit machines. This actually works: I was just able to browse a 25-hour, 37 GiB .mp4 file on my Raspberry Pi 2 Model B. It takes about 400 ms to start serving each request, which isn't exactly zippy but might be forgivable for such a large file. I still intend for the common request from the web interface to be for much smaller fragmented .mp4 files. Speed could be improved later through caching. Right now my test code is creating a fresh VirtualFile from a database query on each request, even though it hasn't changed. The tricky part will be doing cache invalidation cleanly if it does change---new recordings are added to the requested time range, recordings are deleted, or existing recordings' timestamps are changed. The downside to the approach here is that it requires libevent 2.1 for evhttp_send_reply_chunk_with_cb. Unfortunately, Ubuntu 15.10 and Debian Jessie still bundle libevent 2.0. There are a few possible improvements here: 1. fall back to assuming chunks are added immediately, so that people with libevent 2.0 get the old bad behavior and people with libevent 2.1 get the better behavior. This is kind of lame, though; it's easy to go through the whole address space pretty fast, particularly when the browsers send out requests so quickly so there may be some unintentional concurrency. 2. alter the FileSlice interface to return a pointer/destructor rather than add something to the evbuffer. HttpServe would then add each chunk via evbuffer_add_reference, and it'd supply a cleanupfn that (in addition to calling the FileSlice-supplied destructor) notes that this chunk has been fully sent. For all the currently-used FileSlices, this shouldn't be too hard, and there are a few other reasons it might be beneficial: * RealFileSlice could call madvise() to control the OS buffering * RealFileSlice could track when file descriptors are open and thus FileManager's unlink() calls don't actually free up space * It feels dirty to expose libevent stuff through the otherwise-nice FileSlice interface. 3. support building libevent 2.1 statically in-tree if the OS-supplied libevent is unsuitable. I'm tempted to go with #2, but probably not right now. More urgent to commit support for writing the new format and the wrapper bits for viewing it.		2016-01-14 22:41:49 -08:00
..
source	Initial commit, with basic functionality.	2016-01-01 22:06:47 -08:00
changelog	Initial commit, with basic functionality.	2016-01-01 22:06:47 -08:00
compat	Initial commit, with basic functionality.	2016-01-01 22:06:47 -08:00
control	Construct HTTP responses incrementally.	2016-01-14 22:41:49 -08:00
copyright	Initial commit, with basic functionality.	2016-01-01 22:06:47 -08:00
moonfire-nvr.postinst	Initial commit, with basic functionality.	2016-01-01 22:06:47 -08:00
moonfire-nvr.service	Initial commit, with basic functionality.	2016-01-01 22:06:47 -08:00
rules	Initial commit, with basic functionality.	2016-01-01 22:06:47 -08:00