1
0
mirror of https://github.com/minio/minio.git synced 2025-01-18 10:13:16 -05:00

25 Commits

Author SHA1 Message Date
Aditya Manthramurthy
5f78691fcf
ldap: Add user DN attributes list config param ()
This change uses the updated ldap library in minio/pkg (bumped
up to v3). A new config parameter is added for LDAP configuration to
specify extra user attributes to load from the LDAP server and to store
them as additional claims for the user.

A test is added in sts_handlers.go that shows how to access the LDAP
attributes as a claim.

This is in preparation for adding SSH pubkey authentication to MinIO's SFTP
integration.
2024-05-24 16:05:23 -07:00
Harshavardhana
08d74819b6
handle racy updates to globalSite config ()
```
==================
WARNING: DATA RACE
Read at 0x0000082be990 by goroutine 205:
  github.com/minio/minio/cmd.setCommonHeaders()

Previous write at 0x0000082be990 by main goroutine:
  github.com/minio/minio/cmd.lookupConfigs()
```
2024-05-16 16:13:47 -07:00
Harshavardhana
a03ca80269
support 'mc support perf object' with root login disabled ()
It is expected that whoever is using the credentials which has
the proper set of permissions must be able to run.

`mc support perf object`

While the root login is disabled.
2024-05-06 02:45:10 -07:00
Harshavardhana
1d3bd02089
avoid close 'nil' panics if any ()
brings a generic implementation that
prints a stack trace for 'nil' channel
closes(), if not safely closes it.
2024-01-28 10:04:17 -08:00
Zhou Ting
31d16f6cc2
allow sha256 payload to be configurable for object perf test ()
Signed-off-by: Zhou Ting <ting.z.zhou@intel.com>
2023-12-29 23:56:50 -08:00
Harshavardhana
754f7a8a39
replace io.Discard usage to fix some NUMA copy() latencies ()
replace io.Discard usage to fix NUMA copy() latencies

On NUMA systems copying from 8K buffer allocated via
io.Discard leads to large latency build-up for every

```
copy(new8kbuf, largebuf)
```

can in-cur upto 1ms worth of latencies on NUMA systems
due to memory sharding across NUMA nodes.
2023-11-06 14:26:08 -08:00
Klaus Post
7926df0b80
Fix globalDeploymentID race ()
globalDeploymentID was being read while it was being set.

Fixes race:

```
WARNING: DATA RACE
Write at 0x0000079605a0 by main goroutine:
  github.com/minio/minio/cmd.connectLoadInitFormats()
      github.com/minio/minio/cmd/prepare-storage.go:269 +0x14f0
  github.com/minio/minio/cmd.waitForFormatErasure()
      github.com/minio/minio/cmd/prepare-storage.go:294 +0x21d
...

Previous read at 0x0000079605a0 by goroutine 105:
  github.com/minio/minio/cmd.newContext()
      github.com/minio/minio/cmd/utils.go:817 +0x31e
  github.com/minio/minio/cmd.adminMiddleware.func1()
      github.com/minio/minio/cmd/admin-router.go:110 +0x96
  net/http.HandlerFunc.ServeHTTP()
      net/http/server.go:2136 +0x47
  github.com/minio/minio/cmd.setBucketForwardingMiddleware.func1()
      github.com/minio/minio/cmd/generic-handlers.go:460 +0xb1a
  net/http.HandlerFunc.ServeHTTP()
      net/http/server.go:2136 +0x47
...
```
2023-10-18 08:06:57 -07:00
Anis Eleuch
69c0e18685
perf net: Add the endpoint name related to the perf net error ()
In a perf test, one node will run speed test with all nodes. If there is
an error with a peer node, the peer node name is not included in the
error hence confusing the user.

This commit will add the peer endpoint string to the netperf error.
2023-09-19 22:41:06 -07:00
Aditya Manthramurthy
1c99fb106c
Update to minio/pkg/v2 () 2023-09-04 12:57:37 -07:00
jiuker
6aeca54ece
fix: replace context by timeout-context from parent-context when selfSpeedTest () 2023-08-25 07:58:38 -07:00
jiuker
02cc18ff29 refactor the perf client for TTFB and TotalResponseTime () 2023-08-24 10:21:08 -07:00
jiuker
2dbb1cff4a
feat: support perf site replication () 2023-07-05 22:28:26 -07:00
Aditya Manthramurthy
5a1612fe32
Bump up madmin-go and pkg deps () 2023-06-19 17:53:08 -07:00
Aditya Manthramurthy
a30cfdd88f
Bump up madmin-go to v2 () 2022-12-06 13:46:50 -08:00
jiuker
bdcb485740
netPerfRX Reset() should use write Lock() () 2022-11-10 19:44:20 -08:00
Klaus Post
ff12080ff5
Remove deprecated io/ioutil () 2022-09-19 11:05:16 -07:00
Harshavardhana
8f7c739328
feat: add SpeedTest ResponseTimes and TTFB ()
Capture average, p50, p99, p999 response times
and ttfb values. These are needed for latency
measurements and overall understanding of our
speedtest results.
2022-08-05 09:40:03 -07:00
Harshavardhana
b4eb74f5ff
allow custom speedtest bucket ()
this allows for specifying existing buckets with

- object replication enabled
- object encryption enabled
- object versioning enabled
- object locking enabled
2022-07-12 10:12:47 -07:00
Harshavardhana
e1afac9439
reduce sha256 CPU usage by turning it off for speedtests ()
continuation of the PR , keeping signature v4 for
the headers however avoiding sha256 for the body.
2022-06-23 11:26:53 -07:00
Harshavardhana
1a40c7c27c
use signature-v2 for 'object perf' tests to avoid CPU using sha256 ()
It is observed in a local 8 drive system the CPU seems to be
bottlenecked at

```
(pprof) top
Showing nodes accounting for 1385.31s, 88.47% of 1565.88s total
Dropped 1304 nodes (cum <= 7.83s)
Showing top 10 nodes out of 159
      flat  flat%   sum%        cum   cum%
      724s 46.24% 46.24%       724s 46.24%  crypto/sha256.block
   219.04s 13.99% 60.22%    226.63s 14.47%  syscall.Syscall
   158.04s 10.09% 70.32%    158.04s 10.09%  runtime.memmove
   127.58s  8.15% 78.46%    127.58s  8.15%  crypto/md5.block
    58.67s  3.75% 82.21%     58.67s  3.75%  github.com/minio/highwayhash.updateAVX2
    40.07s  2.56% 84.77%     40.07s  2.56%  runtime.epollwait
    33.76s  2.16% 86.93%     33.76s  2.16%  github.com/klauspost/reedsolomon._galMulAVX512Parallel84
     8.88s  0.57% 87.49%     11.56s  0.74%  runtime.step
     7.84s   0.5% 87.99%      7.84s   0.5%  runtime.memclrNoHeapPointers
     7.43s  0.47% 88.47%     22.18s  1.42%  runtime.pcvalue
```

Bonus changes:

- re-use transport for bucket replication clients, also site replication clients.
- use 32KiB buffer for all read and writes at transport layer seems to help
  TLS read connections.
- Do not have 'MaxConnsPerHost' this is problematic to be used with net/http
  connection pooling 'MaxIdleConnsPerHost' is enough.
2022-06-22 16:28:25 -07:00
Harshavardhana
d2a10dbe69
fix: simplify healthcheck code to freeze calls only once ()
- currently subnet health check was freezing and calling
  locks at multiple locations, avoid them.

- throw errors if first attempt itself fails with no results
2022-06-14 11:22:07 -07:00
Krishna Srinivas
b35b9dcff7
Use S3 client for uplooads/downloads during perf test () 2022-04-07 21:20:40 -07:00
Minio Trusted
ffcadcd99e Revert "Use S3 client for uplooads/downloads during perf test ()"
This reverts commit ff811f594baa343c10580f845c31868aef1bc01c.

Speedtest is broken need to fix this more cleanly.
2022-03-16 23:34:49 -07:00
Krishna Srinivas
ff811f594b
Use S3 client for uplooads/downloads during perf test () 2022-03-16 16:58:46 -07:00
Krishna Srinivas
4d0715d226
Implement netperf for "mc support perf net" ()
Co-authored-by: Klaus Post <klauspost@gmail.com>
2022-03-08 09:54:38 -08:00