1
0
mirror of https://github.com/minio/minio.git synced 2025-02-09 12:48:08 -05:00

5 Commits

Author SHA1 Message Date
Klaus Post
038fdeea83
snowball: return errors on failures ()
Return errors when untar fails at once.

Current error handling was quite a mess. Errors are written 
to the stream, but processing continues.

Instead, return errors when they occur and transform 
internal errors to bad request errors, since it is likely a 
problem with the input.

Fixes 
2021-12-06 09:45:23 -08:00
Klaus Post
c2eb60df4a
bz2: limit max concurrent CPU ()
Ensure that bz2 decompression will never take more than 50% CPU.
2021-10-18 08:44:36 -07:00
Klaus Post
5e53f767c4
Use concurrent bz2 decompression ()
Testing with `mc sql --compression BZIP2 --csv-input "rd=\n,fh=USE,fd=;" --query="select COUNT(*) from S3Object" local2/testbucket/nyc-taxi-data-10M.csv.bz2`

Before 96.98s, after 10.79s. Uses about 70% CPU while running.
2021-10-14 11:11:07 -07:00
Harshavardhana
069432566f update license change for MinIO
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-23 11:58:53 -07:00
Harshavardhana
d8bda2dd92
[feat] Add targz transparent extract support ()
This feature brings in support for auto extraction
of objects onto MinIO's namespace from an incoming
tar gzipped stream, the only expected metadata sent
by the client is to set `snowball-auto-extract`.

All the contents from the tar stream are saved as
folders and objects on the namespace.

fixes 
2021-03-26 17:15:09 -07:00