Commit Graph

50 Commits

Author SHA1 Message Date
Harshavardhana 997ba3a574
introduce reader deadlines for net.Conn (#19023)
Bonus: set "retry-after" header for AWS SDKs if possible to honor them.
2024-02-09 13:25:16 -08:00
Sveinn 9b8ba97f9f
feat: add support for GetObjectAttributes API (#18732) 2024-01-05 10:43:06 -08:00
Aditya Manthramurthy 0a284a1a10
fix: SR: Add more info when IAM config differs (#18302)
Provide details on what IAM info mismatched when the validation fails
2023-10-23 21:16:40 -07:00
Harshavardhana cb089dcb52
error out by default beyond 10000 versions per object (#17803)
```
You've exceeded the limit on the number of versions you can create on this object
```
2023-08-04 10:40:21 -07:00
Poorna fb49aead9b
replication: add validation API (#17520)
To check if replication is set up properly on a bucket.
2023-07-10 20:09:20 -07:00
Harshavardhana 7605d07bb2
add support for bucket level request count per API (#17468)
New metrics added to calculate API request count
per bucket, per API.  Captures errors, including
4xx, 5xx HTTP status codes separately.
2023-06-21 09:41:59 -07:00
Harshavardhana 1443b5927a
allow quorum fileInfo to pick same parityBlocks (#17454)
Bonus: allow replication to proceed for 503 errors such as
with error code SlowDownRead
2023-06-18 18:20:15 -07:00
Harshavardhana 26a0803388
various compliance related fixes (#17401)
- getObjectTagging to be allowed for anonymous policies
- return correct errors for invalid retention period
- return sorted list of tags for an object
- putObjectTagging must return 200 OK not 204 OK
- return 409 ErrObjectLockConfigurationNotAllowed for existing buckets
2023-06-12 13:22:07 -07:00
Harshavardhana 43468f4d47
return InvalidRequest when no parts are provided (#17395) 2023-06-10 21:59:51 -07:00
Aditya Manthramurthy 65cba212e8
Remove older policy attach behavior for LDAP (#17240) 2023-05-26 06:31:24 -07:00
Krishnan Parthasarathi 62df731006
Add updatedAt for GetBucketLifecycleConfig (#17271) 2023-05-24 22:52:39 -07:00
Poorna ef7177ebbd
disallow bucket replication setup with site replication (#17189) 2023-05-11 15:48:40 -07:00
Klaus Post 76913a9fd5
Signed trailers for signature v4 (#16484) 2023-05-05 19:53:12 -07:00
Aditya Manthramurthy 58266c9e2c
Add enable flag for LDAP IDP config (#16805) 2023-03-16 11:58:59 -07:00
Harshavardhana 901887e6bf
feat: add lambda transformation functions target (#16507) 2023-03-07 08:12:41 -08:00
Harshavardhana 0a17acdb34
return error if policy changes on disabled groups (#16766) 2023-03-06 10:46:24 -08:00
Harshavardhana ae029191a3
liveness returns "busy" if queued requests > available capacity (#16719) 2023-02-27 08:34:52 -08:00
Anis Elleuch 94c6cb1323
tests: Add test for S3 API error codes (#16705) 2023-02-25 08:06:29 +05:30
Allan Roger Reid 8bfe972bab
Set meaningful message from minio with env variable KMS_SECRET_KEY (#16584) 2023-02-22 07:13:01 +05:30
Poorna c33a237067
fix: under site replication disallow remote target modification (#16628) 2023-02-15 20:22:13 -08:00
Harshavardhana e64b9f6751
fix: disallow SSE-C encrypted objects on replicated buckets (#16467) 2023-01-24 15:46:33 -08:00
Taran Pelkey 709eb283d9
Add endpoints for managing IAM policies (#15897)
Co-authored-by: Taran <taran@minio.io>
Co-authored-by: ¨taran-p¨ <¨taran@minio.io¨>
Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2022-12-13 12:13:23 -08:00
Aditya Manthramurthy e06127566d
Add IAM API to attach/detach policies for LDAP (#16182) 2022-12-09 13:08:33 -08:00
Harshavardhana b03dd1af17
remove hard limit for number of buckets (#16194) 2022-12-08 12:24:03 -08:00
Anis Elleuch 44735be38e
s3: Return correct error when Version is invalid in policy document (#16178) 2022-12-06 08:07:24 -08:00
Aditya Manthramurthy 4fb47cd568
fix: update admin IDP APIs to be more RESTful (#15896) 2022-10-31 14:52:26 -07:00
Krishnan Parthasarathi 4523da6543
feat: introduce pool-level rebalance (#15483) 2022-10-25 12:36:57 -07:00
Aditya Manthramurthy 2d16e74f38
Add LDAP IDP Configuration APIs (#15840) 2022-10-19 11:00:10 -07:00
Harshavardhana 2a13cc28f2 feat: implement support batch replication (#15554) 2022-10-05 23:00:43 -07:00
Harshavardhana b579163802
limit number of buckets to 500k (#15668)
500k is a reasonable limit for any single MinIO
cluster deployment, in future we may increase this
value.

However for now we are going to keep this limit.
2022-09-09 03:06:34 -07:00
Klaus Post a9f1ad7924
Add extended checksum support (#15433) 2022-08-29 16:57:16 -07:00
Harshavardhana 48640b1de2
fix: trim arn:aws:kms from incoming SSE aws-kms-key-id (#15540) 2022-08-16 11:28:30 -07:00
Aditya Manthramurthy af9bc7ea7d
Add external IDP management Admin API for OpenID (#15152) 2022-07-05 18:18:04 -07:00
Harshavardhana 48e367ff7d
reject resync start on misconfigured replication rules (#15041)
we expect resync to start on buckets with replication
rule ExistingObjects enabled, if not we reject such
calls.
2022-06-06 02:54:39 -07:00
Poorna 523670ba0d
fix: site removal API error handling (#14870)
when the site is being removed is missing replication config. This can
happen when a new deployment is brought in place of a site that
is lost/destroyed and needs to delink old deployment from site
replication.
2022-05-06 12:40:34 -07:00
Harshavardhana 16bc11e72e
fix: disallow newer policies, users & groups with space characters (#14845)
space characters at the beginning or at the end can lead to
confusion under various UI elements in differentiating the
actual name of "policy, user or group" - to avoid this behavior
this PR onwards we shall reject such inputs for newer entries.

existing saved entries will behave as is and are going to be
operable until they are removed/renamed to something more
meaningful.
2022-05-02 09:27:35 -07:00
Poorna 93af4a4864
Handle non existent kms key correctly (#14329)
- in PutBucketEncryption API
- admin APIs for  `mc admin KMS key [create|info]`
- PutObject API when invalid KMS key is specified
2022-02-17 11:36:14 -08:00
Harshavardhana 1cf726348f
return meaningful error for disabled users (#13968)
fixes #13958
2021-12-22 11:40:21 -08:00
Poorna K b42cfcea60
Disallow versioning/replication change in cluster replication setup (#13910) 2021-12-15 10:37:08 -08:00
Krishnan Parthasarathi 3da9ee15d3
Add MaxNoncurrentVersions to NoncurrentExpiration action (#13580)
This unit allows users to limit the maximum number of noncurrent 
versions of an object.

To enable this rule you need the following *ilm.json*
```
cat >> ilm.json <<EOF
{
    "Rules": [
        {
            "ID": "test-max-noncurrent",
            "Status": "Enabled",
            "Filter": {
                "Prefix": "user-uploads/"
            },
            "NoncurrentVersionExpiration": {
                "MaxNoncurrentVersions": 5
            }
        }
    ]
}
EOF
mc ilm import myminio/mybucket < ilm.json
```
2021-11-19 17:54:10 -08:00
Aditya Manthramurthy 3a7c79e2c7
Add new site replication feature (#13311)
This change allows a set of MinIO sites (clusters) to be configured 
for mutual replication of all buckets (including bucket policies, tags, 
object-lock configuration and bucket encryption), IAM policies, 
LDAP service accounts and LDAP STS accounts.
2021-10-06 16:36:31 -07:00
Harshavardhana 035882d292
fix: remove parentIsObject() check (#12851)
we will allow situations such as

```
a/b/1.txt
a/b
```

and

```
a/b
a/b/1.txt
```

we are going to document that this usecase is
not supported and we will never support it, if
any application does this users have to delete
the top level parent to make sure namespace is
accessible at lower level.

rest of the situations where the prefixes get
created across sets are supported as is.
2021-08-03 13:26:57 -07:00
Poorna Krishnamoorthy d00783c923
Use rate.Limiter for bandwidth monitoring (#12506)
Bonus: fixes a hang when bandwidth caps are enabled for
synchronous replication
2021-06-24 18:29:30 -07:00
Poorna Krishnamoorthy dbea8d2ee0
Add support for existing object replication. (#12109)
Also adding an API to allow resyncing replication when
existing object replication is enabled and the remote target
is entirely lost. With the `mc replicate reset` command, the
objects that are eligible for replication as per the replication
config will be resynced to target if existing object replication
is enabled on the rule.
2021-06-01 19:59:11 -07:00
Krishnan Parthasarathi c829e3a13b Support for remote tier management (#12090)
With this change, MinIO's ILM supports transitioning objects to a remote tier.
This change includes support for Azure Blob Storage, AWS S3 compatible object
storage incl. MinIO and Google Cloud Storage as remote tier storage backends.

Some new additions include:

 - Admin APIs remote tier configuration management

 - Simple journal to track remote objects to be 'collected'
   This is used by object API handlers which 'mutate' object versions by
   overwriting/replacing content (Put/CopyObject) or removing the version
   itself (e.g DeleteObjectVersion).

 - Rework of previous ILM transition to fit the new model
   In the new model, a storage class (a.k.a remote tier) is defined by the
   'remote' object storage type (one of s3, azure, GCS), bucket name and a
   prefix.

* Fixed bugs, review comments, and more unit-tests

- Leverage inline small object feature
- Migrate legacy objects to the latest object format before transitioning
- Fix restore to particular version if specified
- Extend SharedDataDirCount to handle transitioned and restored objects
- Restore-object should accept version-id for version-suspended bucket (#12091)
- Check if remote tier creds have sufficient permissions
- Bonus minor fixes to existing error messages

Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io>
Co-authored-by: Krishna Srinivas <krishna@minio.io>
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-23 11:58:53 -07:00
Harshavardhana 069432566f update license change for MinIO
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-23 11:58:53 -07:00
Andreas Auernhammer 3455f786fa kms: encrypt IAM/config data with the KMS (#12041)
This commit changes the config/IAM encryption
process. Instead of encrypting config data
(users, policies etc.) with the root credentials
MinIO now encrypts this data with a KMS - if configured.

Therefore, this PR moves the MinIO-KMS configuration (via
env. variables) to a "top-level" configuration.
The KMS configuration cannot be stored in the config file
since it is used to decrypt the config file in the first
place.

As a consequence, this commit also removes support for
Hashicorp Vault - which has been deprecated anyway.

Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-04-22 09:51:09 -07:00
Anis Elleuch 291d2793ca
ldap: Create services accounts for LDAP and STS temp accounts (#11808) 2021-04-14 22:51:14 -07:00
Harshavardhana 0b33fa50ae
fix: calculate correct content-range with partNumber query (#11992)
fixes #11989
fixes #11824
2021-04-07 14:37:10 -07:00
Harshavardhana 18dee6a333
add stringer for ErrorCodes (#11933) 2021-03-31 09:30:52 -07:00