Forward RPC errors from crawler (#9569)

The `keepHTTPResponseAlive` would cause errors to be 
returned with status OK.

- Add '32' as a filler byte until a response is ready
- '0' to indicate the response is ready to be consumed
- '1' to indicate response has an error which needs
to be returned to the caller

Clear out 'file not found' errors from dir walker, since it may be 
in a folder that has been deleted since it was scanned.

cmd/fastwalk.go

@@ -19,6 +19,9 @@ var errSkipFile = errors.New("fastwalk: skip this file")
 func readDirFn(dirName string, fn func(entName string, typ os.FileMode) error) error {
 	fis, err := readDir(dirName)
 	if err != nil {
+		if os.IsNotExist(err) || err == errFileNotFound {
+			return nil
+		}
 		return err
 	}
 	for _, fi := range fis {