0bc34952eb
fix: under FanOut API avoid repeated md5sum calculation ( #17572 )
...
md5sum calculation has a high CPU overhead, avoid calculating
it repeatedly for similar fanOut calls.
To fix following CPU profiler result
```
(pprof) top10
Showing nodes accounting for 678.68s, 84.67% of 801.54s total
Dropped 1072 nodes (cum <= 4.01s)
Showing top 10 nodes out of 156
flat flat% sum% cum cum%
332.54s 41.49% 41.49% 332.54s 41.49% runtime/internal/syscall.Syscall6
228.39s 28.49% 69.98% 228.39s 28.49% crypto/md5.block
48.07s 6.00% 75.98% 48.07s 6.00% runtime.memmove
28.91s 3.61% 79.59% 28.91s 3.61% github.com/minio/highwayhash.updateAVX2
8.25s 1.03% 80.61% 8.25s 1.03% runtime.futex
8.25s 1.03% 81.64% 10.81s 1.35% runtime.step
6.99s 0.87% 82.52% 22.35s 2.79% runtime.pcvalue
6.67s 0.83% 83.35% 38.90s 4.85% runtime.mallocgc
5.77s 0.72% 84.07% 32.61s 4.07% runtime.gentraceback
4.84s 0.6% 84.67% 10.49s 1.31% runtime.lock2
```
2023-07-05 03:16:05 -07:00
d0a0eb9738
support fan-out objects via PostUpload() ( #17233 )
2023-05-24 22:51:07 -07:00
5677f73794
Add PostObject Checksum ( #17244 )
2023-05-23 07:58:33 -07:00
aaf1abc993
simplify HardLimitReader by using LimitReader for internal usage ( #17218 )
2023-05-16 13:14:37 -07:00
ef2fc0f99e
fix: reduce using memory and temporary files. ( #17206 )
2023-05-15 14:08:54 -07:00
e07c2ab868
Use hash.NewLimitReader for internal multipart calls ( #17191 )
2023-05-12 11:19:08 -07:00
76913a9fd5
Signed trailers for signature v4 ( #16484 )
2023-05-05 19:53:12 -07:00
328d660106
support CRC32 Checksums on single drive setup ( #15873 )
2022-10-15 11:58:47 -07:00
a9f1ad7924
Add extended checksum support ( #15433 )
2022-08-29 16:57:16 -07:00
9d07cde385
use crypto/sha256 only for FIPS 140-2 compliance ( #14983 )
...
It would seem like the PR #11623 had chewed more
than it wanted to, non-fips build shouldn't really
be forced to use slower crypto/sha256 even for
presumed "non-performance" codepaths. In MinIO
there are really no "non-performance" codepaths.
This assumption seems to have had an adverse
effect in certain areas of CPU usage.
This PR ensures that we stick to sha256-simd
on all non-FIPS builds, our most common build
to ensure we get the best out of the CPU at
any given point in time.
2022-05-27 06:00:19 -07:00
1f262daf6f
rename all remaining packages to internal/ ( #12418 )
...
This is to ensure that there are no projects
that try to import `minio/minio/pkg` into
their own repo. Any such common packages should
go to `https://github.com/minio/pkg `
2021-06-01 14:59:40 -07:00
Harshavardhana
Klaus Post
Poorna