Commit Graph

28 Commits

Author SHA1 Message Date
Aditya Manthramurthy bb6921bf9c
Send AuditLog via new middleware fn for admin APIs (#17632)
A new middleware function is added for admin handlers, including options
for modifying certain behaviors. This admin middleware:

- sets the handler context via reflection in the request and sends AuditLog
- checks for object API availability (skipping it if a flag is passed)
- enables gzip compression (skipping it if a flag is passed)
- enables header tracing (adding body tracing if a flag is passed)

While the new function is a middleware, due to the flags used for
conditional behavior modification, which is used in each route registration
call.

To try to ensure that no regressions are introduced, the following
changes were done mechanically mostly with `sed` and regexp:

- Remove defer logger.AuditLog in admin handlers
- Replace newContext() calls with r.Context()
- Update admin routes registration calls

Bonus: remove unused NetSpeedtestHandler

Since the new adminMiddleware function checks for object layer presence
by default, we need to pass the `noObjLayerFlag` explicitly to admin
handlers that should work even when it is not available. The following
admin handlers do not require it:

- ServerInfoHandler
- StartProfilingHandler
- DownloadProfilingHandler
- ProfileHandler
- SiteReplicationDevNull
- SiteReplicationNetPerf
- TraceHandler

For these handlers adminMiddleware does not check for the object layer
presence (disabled by passing the `noObjLayerFlag`), and for all other
handlers, the pre-check ensures that the handler is not called when the
object layer is not available - the client would get a
ErrServerNotInitialized and can retry later.

This `noObjLayerFlag` is added based on existing behavior for these
handlers only.
2023-07-13 14:52:21 -07:00
Klaus Post ff5988f4e0
Reduce allocations (#17584)
* Reduce allocations

* Add stringsHasPrefixFold which can compare string prefixes, while ignoring case and not allocating.
* Reuse all msgp.Readers
* Reuse metadata buffers when not reading data.

* Make type safe. Make buffer 4K instead of 8.

* Unslice
2023-07-06 16:02:08 -07:00
guangwu 87b6fb37d6
chore: pkg imported more than once (#17444) 2023-06-26 09:21:29 -07:00
Aditya Manthramurthy 5a1612fe32
Bump up madmin-go and pkg deps (#17469) 2023-06-19 17:53:08 -07:00
Poorna 2920b0fc6d
allow specification of path/virtual style bucket lookup in batch replication (#17201) 2023-05-21 15:16:31 -07:00
Klaus Post aaf1abc993
simplify HardLimitReader by using LimitReader for internal usage (#17218) 2023-05-16 13:14:37 -07:00
Poorna e07c2ab868
Use hash.NewLimitReader for internal multipart calls (#17191) 2023-05-12 11:19:08 -07:00
Poorna ec84bad882
batch replication now supports arbitrary S3 targets (#17113) 2023-05-02 22:52:35 -07:00
Poorna 98c26df53e
fix: allow past retention headers to be copied in batch replication (#17095) 2023-04-27 13:43:18 -07:00
Praveen raj Mani 72802a5972
Use 'minio/pkg/sync/errgroup' and 'minio/pkg/workers' (#17069) 2023-04-25 22:57:40 -07:00
Harshavardhana 6825bd7e75
fix: inlined objects don't need to honor long locks (#17039) 2023-04-17 12:16:37 -07:00
Poorna 3158f2d12e
Add support for batch key rotation (#16844) 2023-04-04 10:56:54 -07:00
Poorna 699a24f7e5
batch: validate versioning on src/tgt buckets (#16955) 2023-04-04 10:50:11 -07:00
Poorna 407c9ddcbf
feat: add batch replicate from remote MinIO to local cluster (#16922) 2023-03-31 10:48:36 -07:00
Harshavardhana 6c11dbffd5
add crash protection from backend modifications (#16846) 2023-03-20 09:08:42 -07:00
Poorna 440ad20c1d
Add support for batch job cancellation (#16843) 2023-03-17 23:42:43 -07:00
Harshavardhana 0c1f8b4e0f
add user-agent for all minio.Client usage (#16619) 2023-02-14 13:19:30 -08:00
Harshavardhana 15a75bd79b
ignore preconditionFailed error in batch replication (#16615) 2023-02-14 07:22:08 -08:00
Harshavardhana 31188e9327
add parallel workers in batch replication (#16609) 2023-02-13 12:07:58 -08:00
jiuker a15b6f21b8
remove incorrect use of WaitGroup (#16596) 2023-02-12 20:59:45 -08:00
Aditya Manthramurthy a30cfdd88f
Bump up madmin-go to v2 (#16162) 2022-12-06 13:46:50 -08:00
Harshavardhana 374abd1e7d
add filter support for tags and metadata in batch replication (#15885) 2022-10-18 21:22:21 -07:00
Harshavardhana 9fff315555
do not need to trace ignored objects (#15894) 2022-10-18 13:47:55 -07:00
Harshavardhana c68910005b
validate bucket before attempting batch replication (#15861) 2022-10-15 11:58:31 -07:00
Harshavardhana 44097faec1
support deleteMarkers and all versions in batch replication (#15858) 2022-10-13 14:42:10 -07:00
Harshavardhana 97112c69be
fix: replication stats() to not crash under any situation (#15851)
Co-authored-by: Daniel Valdivia <18384552+dvaldivia@users.noreply.github.com>
2022-10-12 15:47:41 -07:00
Harshavardhana e3cb0278ce
honor specified target prefix under batch replication (#15834) 2022-10-11 14:36:06 -07:00
Harshavardhana 2a13cc28f2 feat: implement support batch replication (#15554) 2022-10-05 23:00:43 -07:00