ListObjects Metadata Caching (#10648)

Design: https://gist.github.com/klauspost/025c09b48ed4a1293c917cecfabdf21c

Gist of improvements:

* Cross-server caching and listing will use the same data across servers and requests.
* Lists can be arbitrarily resumed at a constant speed.
* Metadata for all files scanned is stored for streaming retrieval.
* The existing bloom filters controlled by the crawler is used for validating caches.
* Concurrent requests for the same data (or parts of it) will not spawn additional walkers.
* Listing a subdirectory of an existing recursive cache will use the cache.
* All listing operations are fully streamable so the number of objects in a bucket no 
  longer dictates the amount of memory.
* Listings can be handled by any server within the cluster.
* Caches are cleaned up when out of date or superseded by a more recent one.

cmd/handler-utils.go

@@ -424,6 +424,17 @@ func extractAPIVersion(r *http.Request) string {
 	return regexVersion.FindString(r.URL.Path)
 }
 
+func methodNotAllowedHandler(api string) func(w http.ResponseWriter, r *http.Request) {
+	return func(w http.ResponseWriter, r *http.Request) {
+		code := "XMinio" + api + "VersionMismatch"
+		writeErrorResponseString(r.Context(), w, APIError{
+			Code:           code,
+			Description:    "Not allowed (" + r.Method + " " + r.URL.String() + " on " + api + " API)",
+			HTTPStatusCode: http.StatusMethodNotAllowed,
+		}, r.URL)
+	}
+}
+
 // If none of the http routes match respond with appropriate errors
 func errorResponseHandler(w http.ResponseWriter, r *http.Request) {
 	if r.Method == http.MethodOptions {