48e367ff7d
reject resync start on misconfigured replication rules ( #15041 )
...
we expect resync to start on buckets with replication
rule ExistingObjects enabled, if not we reject such
calls.
2022-06-06 02:54:39 -07:00
3a398775fb
Update yaml files to latest version RELEASE.2022-06-03T01-40-53Z
2022-06-03 02:36:56 +00:00
197d6fb644
Update yaml files to latest version RELEASE.2022-06-02T16-16-26Z
2022-06-02 17:46:32 +00:00
1fce2b180f
Update yaml files to latest version RELEASE.2022-06-02T02-11-04Z
2022-06-02 02:42:14 +00:00
befbf48563
fix: s3-check-md5 to not panic for incomplete md5
2022-05-30 20:58:42 -07:00
f1abb92f0c
feat: Single drive XL implementation ( #14970 )
...
Main motivation is move towards a common backend format
for all different types of modes in MinIO, allowing for
a simpler code and predictable behavior across all features.
This PR also brings features such as versioning, replication,
transitioning to single drive setups.
2022-05-30 10:58:37 -07:00
c2630bb3a3
add total usage pie chart based on total/free bytes
2022-05-28 09:53:53 -07:00
464b9d7c80
Add support for Identity Management Plugin ( #14913 )
...
- Adds an STS API `AssumeRoleWithCustomToken` that can be used to
authenticate via the Id. Mgmt. Plugin.
- Adds a sample identity manager plugin implementation
- Add doc for plugin and STS API
- Add an example program using go SDK for AssumeRoleWithCustomToken
2022-05-26 17:58:09 -07:00
62cd643868
Add `--insecure` flag to skip TLS verification in s3-md5-check tool ( #14980 )
2022-05-26 06:02:05 -07:00
1b7dd70f72
Update yaml files to latest version RELEASE.2022-05-26T05-48-41Z
2022-05-26 06:27:03 +00:00
fc08fcab52
hash-set: Add file input for debug tool ( #14965 )
...
Add input option for a file list to display total distribution.
2022-05-24 09:05:39 -07:00
5be76856bd
Update yaml files to latest version RELEASE.2022-05-23T18-45-11Z
2022-05-24 00:29:45 +00:00
18a4276e25
Update yaml files to latest version RELEASE.2022-05-19T18-20-59Z
2022-05-19 20:18:49 +00:00
4629abd5a2
Add tests for Access Management Plugin ( #14909 )
2022-05-12 15:24:19 -07:00
389ec21d0c
Update documentation for /minio/health/cluster ( #14889 )
2022-05-12 09:54:07 -07:00
c293c2e9a3
docs: update new name for MINIO_POLICY_OPA_URL ( #14898 )
2022-05-11 13:04:15 -07:00
83071a3459
Add support for Access Management Plugin ( #14875 )
...
- This change renames the OPA integration as Access Management Plugin - there is
nothing specific to OPA in the integration, it is just a webhook.
- OPA configuration is automatically migrated to Access Management Plugin and
OPA specific configuration is marked as deprecated.
- OPA doc is updated and moved.
2022-05-10 17:14:55 -07:00
95d1a12422
Update yaml files to latest version RELEASE.2022-05-08T23-50-31Z
2022-05-09 03:46:40 +00:00
62aa42cccf
avoid replication proxy on version excluded paths ( #14878 )
...
no need to attempt proxying objects that were
never replicated, but do have local `null`
versions on them.
2022-05-08 16:50:31 -07:00
5cffd3780a
fix: multiple fixes in prefix exclude implementation ( #14877 )
...
- do not need to restrict prefix exclusions that do not
have `/` as suffix, relax this requirement as spark may
have staging folders with other autogenerated characters
, so we are better off doing full prefix March and skip.
- multiple delete objects was incorrectly creating a
null delete marker on a versioned bucket instead of
creating a proper versioned delete marker.
- do not suspend paths on the excluded prefixes during
delete operations to avoid creating `null` delete markers,
honor suspension of versioning only at bucket level for
delete markers.
2022-05-07 22:06:44 -07:00
ad8e611098
feat: implement prefix-level versioning exclusion ( #14828 )
...
Spark/Hadoop workloads which use Hadoop MR
Committer v1/v2 algorithm upload objects to a
temporary prefix in a bucket. These objects are
'renamed' to a different prefix on Job commit.
Object storage admins are forced to configure
separate ILM policies to expire these objects
and their versions to reclaim space.
Our solution:
This can be avoided by simply marking objects
under these prefixes to be excluded from versioning,
as shown below. Consequently, these objects are
excluded from replication, and don't require ILM
policies to prune unnecessary versions.
- MinIO Extension to Bucket Version Configuration
```xml
<VersioningConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/ ">
<Status>Enabled</Status>
<ExcludeFolders>true</ExcludeFolders>
<ExcludedPrefixes>
<Prefix>app1-jobs/*/_temporary/</Prefix>
</ExcludedPrefixes>
<ExcludedPrefixes>
<Prefix>app2-jobs/*/__magic/</Prefix>
</ExcludedPrefixes>
<!-- .. up to 10 prefixes in all -->
</VersioningConfiguration>
```
Note: `ExcludeFolders` excludes all folders in a bucket
from versioning. This is required to prevent the parent
folders from accumulating delete markers, especially
those which are shared across spark workloads
spanning projects/teams.
- To enable version exclusion on a list of prefixes
```
mc version enable --excluded-prefixes "app1-jobs/*/_temporary/,app2-jobs/*/_magic," --exclude-prefix-marker myminio/test
```
2022-05-06 19:05:28 -07:00
2b7e75e079
Add OPA doc and remove deprecation marking ( #14863 )
2022-05-04 23:53:42 -07:00
2fc65dcb99
Update yaml files to latest version RELEASE.2022-05-04T07-45-27Z
2022-05-04 08:54:16 +00:00
0a256053ee
Update yaml files to latest version RELEASE.2022-05-03T20-36-08Z
2022-05-03 21:27:19 +00:00
f427dbbd60
Update yaml files to latest version RELEASE.2022-04-30T22-23-53Z
2022-05-01 01:33:28 +00:00
85f3a9f3b0
Remove Azure gateway implementation ( #14418 )
...
refer #14331
2022-04-29 12:51:23 -07:00
96f27a4965
Update yaml files to latest version RELEASE.2022-04-29T01-27-09Z
2022-04-29 06:32:50 +00:00
0e502899a8
Add support for multiple OpenID providers with role policies ( #14223 )
...
- When using multiple providers, claim-based providers are not allowed. All
providers must use role policies.
- Update markdown config to allow `details` HTML element
2022-04-28 18:27:09 -07:00
990fbeb3a4
rename true/false to on/off in bucket notification docs
2022-04-27 23:51:31 -07:00
8a6b2b4447
Update yaml files to latest version RELEASE.2022-04-26T01-20-24Z
2022-04-26 02:08:20 +00:00
0cc993f403
Remove GCS, HDFS gateway implementations #14418
...
refer #14331
2022-04-24 10:19:17 -07:00
1494ba2e6e
Update yaml files to latest version RELEASE.2022-04-16T04-26-02Z
2022-04-16 05:03:00 +00:00
646350fa7f
Update yaml files to latest version RELEASE.2022-04-12T06-55-35Z
2022-04-12 07:23:20 +00:00
33d267fa1b
Update yaml files to latest version RELEASE.2022-04-09T15-09-52Z
2022-04-09 20:23:18 +00:00
f630d7c3fa
Update yaml files to latest version RELEASE.2022-04-08T19-44-35Z
2022-04-08 23:35:38 +00:00
c7962118f8
Update yaml files to latest version RELEASE.2022-04-01T03-41-39Z
2022-04-01 08:23:40 +00:00
e955aa7f2a
kes: add support for encrypted private keys ( #14650 )
...
This commit adds support for encrypted KES
client private keys.
Now, it is possible to encrypt the KES client
private key (`MINIO_KMS_KES_KEY_FILE`) with
a password.
For example, KES CLI already supports the
creation of encrypted private keys:
```
kes identity new --encrypt --key client.key --cert client.crt MinIO
```
To decrypt an encrypted private key, the password
needs to be provided:
```
MINIO_KMS_KES_KEY_PASSWORD=<password>
```
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-03-29 09:53:33 -07:00
81d2b54dfd
doc: typo fix for ttfb entry in table ( #14647 )
2022-03-29 09:42:02 -07:00
be5d394e56
Update yaml files to latest version RELEASE.2022-03-26T06-49-28Z
2022-03-26 07:32:25 +00:00
849a27ee61
update hotfixes instructions and fix some typo
2022-03-25 23:49:28 -07:00
4d2fc530d0
add support for SSE-S3 bulk ETag decryption ( #14627 )
...
This commit adds support for bulk ETag
decryption for SSE-S3 encrypted objects.
If KES supports a bulk decryption API, then
MinIO will check whether its policy grants
access to this API. If so, MinIO will use
a bulk API call instead of sending encrypted
ETags serially to KES.
Note that MinIO will not use the KES bulk API
if its client certificate is an admin identity.
MinIO will process object listings in batches.
A batch has a configurable size that can be set
via `MINIO_KMS_KES_BULK_API_BATCH_SIZE=N`.
It defaults to `500`.
This env. variable is experimental and may be
renamed / removed in the future.
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2022-03-25 15:01:41 -07:00
cf220be9b5
Update yaml files to latest version RELEASE.2022-03-24T00-43-44Z
2022-03-24 01:28:05 +00:00
751e9fb7be
Update yaml files to latest version RELEASE.2022-03-22T02-05-10Z
2022-03-22 02:45:24 +00:00
e9bc0e7e98
Update yaml files to latest version RELEASE.2022-03-17T06-34-49Z
2022-03-17 00:11:59 -07:00
7a733a8d54
Update yaml files to latest version RELEASE.2022-03-17T02-57-36Z
2022-03-16 22:27:48 -07:00
ce97313fda
Add extra LDAP configuration validation ( #14535 )
...
- The result now contains suggestions on fixing common configuration issues.
- These suggestions will subsequently be exposed in console/mc
2022-03-16 19:57:36 -07:00
721c053712
Update yaml files to latest version RELEASE.2022-03-14T18-25-24Z
2022-03-14 19:32:22 +00:00
57b7c3494f
Update yaml files to latest version RELEASE.2022-03-11T23-57-45Z
2022-03-13 08:47:27 +00:00
f8d6eaaa96
fix: regression from range GET proxy on replicated buckets #14345 ( #14532 )
...
Fixes : #14531
2022-03-11 15:56:49 -08:00
80039f60d5
Update yaml files to latest version RELEASE.2022-03-11T11-08-23Z
2022-03-11 11:47:17 +00:00
b7ed3b77bd
Indicate required fields in LDAP configuration correctly ( #14526 )
2022-03-10 19:03:38 -08:00
fce380a044
Update yaml files to latest version RELEASE.2022-03-08T22-28-51Z
2022-03-09 01:36:59 +00:00
1f11af42f1
Update yaml files to latest version RELEASE.2022-03-05T06-32-39Z
2022-03-05 09:27:28 +00:00
a026c8748f
Update nginx.conf for large file uploads ( #14481 )
2022-03-04 22:32:39 -08:00
b0c84e3de7
fix: deleteVersions causing xl.meta to have empty Versions[] slice ( #14483 )
...
This is a side-affect of the optimization done in PR #13544 which
causes a certain type of delete operations on given object versions
can cause lastVersion indication to be skipped, which leads to
an `xl.meta` where Versions[] slice is empty while the entire
file is intact by itself.
This PR tries to ensure that such files are visible and deletable
by regular means of listing as null 'delete-marker' and also
avoid the situation where this potential issue might arise.
2022-03-04 20:01:26 -08:00
04bbede17d
Update yaml files to latest version RELEASE.2022-03-03T21-21-16Z
2022-03-03 22:16:10 +00:00
27fad98179
Replace HeadBucket permission with GetBucketAcl ( #14436 )
...
Resolves https://github.com/minio/minio/issues/14379
2022-03-01 21:18:23 -08:00
7d4579e737
Update yaml files to latest version RELEASE.2022-02-26T02-54-46Z
2022-02-26 03:36:08 +00:00
890e526bde
rename 'mc admin inspect' to 'mc support inspect'
2022-02-24 17:17:53 -08:00
16ce455fca
update docker release to RELEASE.2022-02-24T22-12-01Z
2022-02-24 15:35:14 -08:00
b23b19e5c3
Update yaml files to latest version RELEASE.2022-02-18T01-50-10Z
2022-02-17 19:12:27 -08:00
1dbb3f6f43
Update yaml files to latest version RELEASE.2022-02-17T23-22-26Z
2022-02-18 00:33:01 +00:00
27d94c64ed
Update yaml files to latest version RELEASE.2022-02-16T00-35-27Z
2022-02-16 05:34:56 +00:00
d0862ddf86
doc: add console-address on all example ( #14307 )
...
--console-address ":9001" is missing on docker example for regular user.
2022-02-15 09:26:04 -08:00
f088e8960b
docs: turn-on more markdown rules and fix them ( #14301 )
2022-02-14 08:50:42 -08:00
1f92fc3fc0
Always check for root disks unless MINIO_CI_CD is set ( #14232 )
...
The current code considers a pool with all root disks to be as part
of a testing environment even if there are other pools with mounted
disks. This will result to illegitimate writing in root disks.
Fix this by simplifing the logic: require MINIO_CI_CD in order to skip
root disk check.
2022-02-13 15:42:07 -08:00
f71b114a84
Update yaml files to latest version RELEASE.2022-02-12T00-51-25Z
2022-02-13 19:31:37 +00:00
e3e0532613
cleanup markdown docs across multiple files ( #14296 )
...
enable markdown-linter
2022-02-11 16:51:25 -08:00
97b5bf1fb7
update gateway docs to indicate code-freeze
2022-02-10 11:39:41 -08:00
ed3418c046
Refactor replication resync to be an active process ( #14266 )
...
When resync is triggered, walk the bucket namespace and
resync objects that are unreplicated. This PR also adds
an API to report resync progress.
2022-02-10 10:16:52 -08:00
a2230868e0
remove all stale old docs about 2018 releases
2022-02-10 09:54:27 -08:00
0ee2933234
Export tier metrics via Prometheus ( #13413 )
...
e.g
```
minio_cluster_ilm_transitioned_bytes{server="minio3:9000",tier="S3TIER-1"} 1.36317772e+08
minio_cluster_ilm_transitioned_bytes{server="minio3:9000",tier="S3TIER-2"} 2892
minio_cluster_ilm_transitioned_bytes{server="minio3:9000",tier="STANDARD"}
1.3631488e+08
minio_cluster_ilm_transitioned_objects{server="minio3:9000",tier="S3TIER-1"} 1
minio_cluster_ilm_transitioned_objects{server="minio3:9000",tier="S3TIER-2"} 0
minio_cluster_ilm_transitioned_objects{server="minio3:9000",tier="STANDARD"} 1
minio_cluster_ilm_transitioned_versions{server="minio3:9000",tier="S3TIER-1"} 3
minio_cluster_ilm_transitioned_versions{server="minio3:9000",tier="S3TIER-2"} 2
minio_cluster_ilm_transitioned_versions{server="minio3:9000",tier="STANDARD"} 1
```
2022-02-08 12:45:28 -08:00
2480c66857
Update yaml files to latest version RELEASE.2022-02-07T08-17-33Z
2022-02-07 09:19:24 +00:00
570670be8c
Update yaml files to latest version RELEASE.2022-02-05T04-40-59Z
2022-02-05 18:33:46 +00:00
9417fd933e
Update yaml files to latest version RELEASE.2022-02-01T18-00-14Z
2022-02-01 23:37:07 +00:00
74faed166a
Add quota usage as part of prometheus metrics ( #14222 )
...
Bonus: pass caller context when needed to all bucket metadata handling calls.
2022-01-31 17:27:43 -08:00
dbd05d6e82
remove FIFO bucket quota, use ILM expiration instead ( #14206 )
2022-01-31 11:07:04 -08:00
c7df5fb119
Update yaml files to latest version RELEASE.2022-01-28T02-28-16Z
2022-01-28 05:12:35 +00:00
a66071099c
Update yaml files to latest version RELEASE.2022-01-27T03-53-02Z
2022-01-27 04:47:45 +00:00
7dfa565d00
Identity LDAP: Allow multiple search base DNs ( #14191 )
...
This change allows the MinIO server to lookup users in different directory
sub-trees by allowing specification of multiple search bases separated by
semicolons.
2022-01-26 15:05:59 -08:00
affe51cb19
docs: add casdoor as identity provider ( #14185 )
2022-01-26 09:49:10 -08:00
f556a72fe2
Update yaml files to latest version RELEASE.2022-01-25T19-56-04Z
2022-01-25 20:42:46 +00:00
cd7a5cab8a
update docs for Decommission
2022-01-25 11:56:04 -08:00
295730408b
Disallow delete replication for tag based rules ( #14167 )
2022-01-24 15:22:20 -08:00
f30afa4956
docs: add decommission docs about pool removal ( #14159 )
2022-01-24 09:47:06 -08:00
8fb4ae916c
update decommission docs
2022-01-21 18:34:06 -08:00
07df9eecda
Update xl.meta docs ( #14150 )
2022-01-21 08:47:46 -08:00
9d588319dd
support site replication to replicate IAM users,groups ( #14128 )
...
- Site replication was missing replicating users,
groups when an empty site was added.
- Add site replication for groups and users when they
are disabled and enabled.
- Add support for replicating bucket quota config.
2022-01-19 20:02:24 -08:00
54a98773f8
fix: replication of tag removal ( #14056 )
...
Currently tag removal leaves replication state as `PENDING`
because the `HEAD` api returns just a tag count but not the
actual tags, and this is treated as a no-op
2022-01-10 19:06:10 -08:00
76b21de0c6
feat: decommission feature for pools ( #14012 )
...
```
λ mc admin decommission start alias/ http://minio{1...2}/data{1...4}
```
```
λ mc admin decommission status alias/
┌─────┬─────────────────────────────────┬──────────────────────────────────┬────────┐
│ ID │ Pools │ Capacity │ Status │
│ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Active │
│ 2nd │ http://minio{3...4}/data{1...4} │ 329 GiB (used) / 421 GiB (total) │ Active │
└─────┴─────────────────────────────────┴──────────────────────────────────┴────────┘
```
```
λ mc admin decommission status alias/ http://minio{1...2}/data{1...4}
Progress: ===================> [1GiB/sec] [15%] [4TiB/50TiB]
Time Remaining: 4 hours (started 3 hours ago)
```
```
λ mc admin decommission status alias/ http://minio{1...2}/data{1...4}
ERROR: This pool is not scheduled for decommissioning currently.
```
```
λ mc admin decommission cancel alias/
┌─────┬─────────────────────────────────┬──────────────────────────────────┬──────────┐
│ ID │ Pools │ Capacity │ Status │
│ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Draining │
└─────┴─────────────────────────────────┴──────────────────────────────────┴──────────┘
```
> NOTE: Canceled decommission will not make the pool active again, since we might have
> Potentially partial duplicate content on the other pools, to avoid this scenario be
> very sure to start decommissioning as a planned activity.
```
λ mc admin decommission cancel alias/ http://minio{1...2}/data{1...4}
┌─────┬─────────────────────────────────┬──────────────────────────────────┬────────────────────┐
│ ID │ Pools │ Capacity │ Status │
│ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Draining(Canceled) │
└─────┴─────────────────────────────────┴──────────────────────────────────┴────────────────────┘
```
2022-01-10 09:07:49 -08:00
f394313fee
Update yaml files to latest version RELEASE.2022-01-08T03-11-54Z
2022-01-08 05:53:59 +00:00
b7c5e45fff
heal: isObjectDangling should return false when it cannot decide ( #14053 )
...
In a multi-pool setup when disks are coming up, or in a single pool
setup let's say with 100's of erasure sets with a slow network.
It's possible when healing is attempted on `.minio.sys/config`
folder, it can lead to healing unexpectedly deleting some policy
files as dangling due to a mistake in understanding when `isObjectDangling`
is considered to be 'true'.
This issue happened in commit 30135eed86
2022-01-07 19:11:54 -08:00
0a224654c2
fix: progagation of service accounts for site replication ( #14054 )
...
- Only non-root-owned service accounts are replicated for now.
- Add integration tests for OIDC with site replication
2022-01-07 17:41:43 -08:00
e420a1de4d
Update yaml files to latest version RELEASE.2022-01-07T01-53-23Z
2022-01-07 05:57:36 +00:00
62dc0f7698
Update site replication docs ( #14044 )
...
- Now OpenID external IDP is also supported for site replication.
- IAM users/groups are also replicated
2022-01-06 17:53:23 -08:00
f4c03e56b8
Update yaml files to latest version RELEASE.2022-01-04T07-41-07Z
2022-01-04 23:01:29 +00:00
5dd40b9377
Update yaml files to latest version RELEASE.2022-01-03T18-22-58Z
2022-01-04 00:12:07 +00:00
a60ac7ca17
fix: audit log to support object names in multipleObjectNames() handler ( #14017 )
2022-01-03 01:28:52 -08:00
c9e49f4366
Update yaml files to latest version RELEASE.2021-12-29T06-49-06Z
2021-12-29 07:38:12 +00:00
3e28af1723
docs: update TLS doc use -keyout instead of -key ( #14001 )
2021-12-28 12:51:38 -08:00
6aa0574a53
Update yaml files to latest version RELEASE.2021-12-27T07-23-18Z
2021-12-27 19:52:13 +00:00
7e3a7d7044
add healing for invalid shards by skipping the blocks ( #13978 )
...
Built on top of #13945 , now we need to simply skip the
shards and its automated.
2021-12-23 23:01:46 -08:00
416977436e
rename MINIO_CACHE_.._MASTER_KEY to MINIO_CACHE_.._SECRET_KEY
...
fixes #13975
2021-12-22 12:11:07 -08:00
0e3037631f
skip inconsistent shards if possible ( #13945 )
...
data shards were wrong due to a healing bug
reported in #13803 mainly with unaligned object
sizes.
This PR is an attempt to automatically avoid
these shards, with available information about
the `xl.meta` and actually disk mtime.
2021-12-21 10:08:26 -08:00
f3602d7d08
Update yaml files to latest version RELEASE.2021-12-20T22-07-16Z
2021-12-20 23:13:48 +00:00
0b21734571
update helm chart link
2021-12-20 13:20:59 -08:00
cfd66ab8c3
Update yaml files to latest version RELEASE.2021-12-18T04-42-33Z
2021-12-19 22:19:30 +00:00
3ddb501190
Add docs for NewerNoncurrentVersions ( #13944 )
2021-12-17 19:32:29 -08:00
4fa250a6a1
add hash-set debugging currently supports SIPMOD ( #13911 )
...
fixes a bug in s3-check-md5 for single part
uploaded multipart objects.
2021-12-15 12:07:15 -08:00
88ad742da0
fix: error handling cases in site-replication ( #13901 )
...
- Allow proper SRError to be propagated to
handlers and converted appropriately.
- Make sure to enable object locking on buckets
when requested in MakeBucketHook.
- When DNSConfig is enabled attempt to delete it
first before deleting buckets locally.
2021-12-14 14:09:57 -08:00
40dbe243d9
update: keycloak binding documentation ( #13894 )
2021-12-13 18:23:16 -08:00
109c927dad
docs/debug: log any corruption and continue debug tool
2021-12-13 17:45:34 -08:00
878d368cea
Update yaml files to latest version RELEASE.2021-12-10T23-03-39Z
2021-12-11 00:02:59 -08:00
518612492c
xl-meta: Add header titles ( #13880 )
...
Add type for headers and create custom marshal to make
it easier to read. Group headers and metadata.
Restore functionality that will read `xl.meta` in the current dir with no params.
Before:
```
{
"Headers": [
[
"8M04bTiYRDmEMQGeAsk1yg==",
1639150471630100400,
"rLD1Rw==",
1,
6
],
],
"Versions": [
{
"Type": 1,
"V2Obj": {
"CSumAlgo": 1,
"DDir": "oC1Xpg4tRfW03g8o8w7Bzg==",
"EcAlgo": 1,
"EcBSize": 1048576,
"EcDist": [
7,
8,
1,
2,
3,
4,
5,
6
],
"EcIndex": 1,
"EcM": 4,
"EcN": 4,
"ID": "8M04bTiYRDmEMQGeAsk1yg==",
"MTime": 1639150471630100400,
"MetaSys": {
"x-minio-internal-inline-data": "dHJ1ZQ=="
},
"MetaUsr": {
"content-type": "application/octet-stream",
"etag": "b8252c86fad2d8937300aa92b467a3aa"
},
"PartASizes": [
1000
],
"PartETags": null,
"PartNums": [
1
],
"PartSizes": [
1000
],
"Size": 1000
}
}
]
}
```
After:
```
{
"Versions": [
{
"Header": {
"Flags": 6,
"ModTime": "2021-12-10T16:34:31.6301004+01:00",
"Signature": "acb0f547",
"Type": 1,
"VersionID": "f0cd386d389844398431019e02c935ca"
},
"Idx": 0,
"Metadata": {
"Type": 1,
"V2Obj": {
"CSumAlgo": 1,
"DDir": "oC1Xpg4tRfW03g8o8w7Bzg==",
"EcAlgo": 1,
"EcBSize": 1048576,
"EcDist": [
7,
8,
1,
2,
3,
4,
5,
6
],
"EcIndex": 1,
"EcM": 4,
"EcN": 4,
"ID": "8M04bTiYRDmEMQGeAsk1yg==",
"MTime": 1639150471630100400,
"MetaSys": {
"x-minio-internal-inline-data": "dHJ1ZQ=="
},
"MetaUsr": {
"content-type": "application/octet-stream",
"etag": "b8252c86fad2d8937300aa92b467a3aa"
},
"PartASizes": [
1000
],
"PartETags": null,
"PartNums": [
1
],
"PartSizes": [
1000
],
"Size": 1000
}
}
}
]
}
```
2021-12-10 15:03:25 -08:00
c76f86fdbd
Clarify example for Standalone Docker instructions ( #13879 )
...
Closes #13868
Also points users to the web documentation for docker installation.
2021-12-10 09:43:02 -08:00
4be8023408
Update yaml files to latest version RELEASE.2021-12-09T06-19-41Z
2021-12-09 08:40:46 +00:00
dcff6c996d
fix: do not list delete-marked objects ( #13864 )
...
delete marked objects should not be considered
for listing when listing is delimited, this issue
as introduced in PR #13804 which was mainly to
address listing of directories in listing when
delimited.
This PR fixes this properly and adds tests to
ensure that we behave in accordance with how
an S3 API behaves for ListObjects() without
versions.
2021-12-08 17:34:52 -08:00
e82a5c5c54
fix: site replication issues and add tests ( #13861 )
...
- deleting policies was deleting all LDAP
user mapping, this was a regression introduced
in #13567
- deleting of policies is properly sent across
all sites.
- remove unexpected errors instead embed the real
errors as part of the 500 error response.
2021-12-08 11:50:15 -08:00
92fdcafb66
add verification tests for ETag on replicated content ( #13857 )
2021-12-07 10:08:26 -08:00
be34fc9134
fix: kms-id header should have arn:aws:kms: prefix ( #13833 )
...
arn:aws:kms: is a must for KMS keyID.
2021-12-06 00:39:32 -08:00
20c60aae68
Update hotfix documentation and container building
2021-12-02 17:52:46 -08:00
f840080e5b
cleanup site-replication docs ( #13812 )
2021-12-02 13:27:01 -08:00
ffe9acfe4a
docs: Add a markdown documentation on hotfix branches and process
2021-12-01 01:00:18 -08:00
9ec197f2e8
Add support for adding new site(s) to site replication ( #13696 )
...
Currently, the new site is expected to be empty
2021-11-30 13:16:37 -08:00
4c0f48c548
Add role ARN support for OIDC identity provider ( #13651 )
...
- Allows setting a role policy parameter when configuring OIDC provider
- When role policy is set, the server prints a role ARN usable in STS API requests
- The given role policy is applied to STS API requests when the roleARN parameter is provided.
- Service accounts for role policy are also possible and work as expected.
2021-11-26 19:22:40 -08:00
4ce6d35e30
Add new `site` config sub-system intended to replace `region` ( #13672 )
...
- New sub-system has "region" and "name" fields.
- `region` subsystem is marked as deprecated, however still works, unless the
new region parameter under `site` is set - in this case, the region subsystem is
ignored. `region` subsystem is hidden from top-level help (i.e. from `mc admin
config set myminio`), but appears when specifically requested (i.e. with `mc
admin config set myminio region`).
- MINIO_REGION, MINIO_REGION_NAME are supported as legacy environment variables for server region.
- Adds MINIO_SITE_REGION as the current environment variable to configure the
server region and MINIO_SITE_NAME for the site name.
2021-11-25 13:06:25 -08:00
a5db4ca092
Update yaml files to latest version RELEASE.2021-11-24T23-19-33Z
2021-11-25 07:39:00 +00:00
ef0b8367b5
Update minio-overview.json data source panel ( #13730 )
...
Add missing datasource in `Healing` panel.
2021-11-23 09:01:07 -08:00
3da9ee15d3
Add MaxNoncurrentVersions to NoncurrentExpiration action ( #13580 )
...
This unit allows users to limit the maximum number of noncurrent
versions of an object.
To enable this rule you need the following *ilm.json*
```
cat >> ilm.json <<EOF
{
"Rules": [
{
"ID": "test-max-noncurrent",
"Status": "Enabled",
"Filter": {
"Prefix": "user-uploads/"
},
"NoncurrentVersionExpiration": {
"MaxNoncurrentVersions": 5
}
}
]
}
EOF
mc ilm import myminio/mybucket < ilm.json
```
2021-11-19 17:54:10 -08:00
7b82411e6f
change the unit of measurement from TB to TiB ( #13686 )
2021-11-18 20:06:37 -08:00
79b3a1fe4e
remove object torrent, AWS S3 removed support for torrent API
2021-11-18 12:21:48 -08:00
faf013ec84
Improve performance on multiple versions ( #13573 )
...
Existing:
```go
type xlMetaV2 struct {
Versions []xlMetaV2Version `json:"Versions" msg:"Versions"`
}
```
Serialized as regular MessagePack.
```go
//msgp:tuple xlMetaV2VersionHeader
type xlMetaV2VersionHeader struct {
VersionID [16]byte
ModTime int64
Type VersionType
Flags xlFlags
}
```
Serialize as streaming MessagePack, format:
```
int(headerVersion)
int(xlmetaVersion)
int(nVersions)
for each version {
binary blob, xlMetaV2VersionHeader, serialized
binary blob, xlMetaV2Version, serialized.
}
```
xlMetaV2VersionHeader is <= 30 bytes serialized. Deserialized struct
can easily be reused and does not contain pointers, so efficient as a
slice (single allocation)
This allows quickly parsing everything as slices of bytes (no copy).
Versions are always *saved* sorted by modTime, newest *first*.
No more need to sort on load.
* Allows checking if a version exists.
* Allows reading single version without unmarshal all.
* Allows reading latest version of type without unmarshal all.
* Allows reading latest version without unmarshal of all.
* Allows checking if the latest is deleteMarker by reading first entry.
* Allows adding/updating/deleting a version with only header deserialization.
* Reduces allocations on conversion to FileInfo(s).
2021-11-18 12:15:22 -08:00
3d2bc15e9a
Add grafana json file for replication metrics ( #13678 )
2021-11-17 14:49:46 -08:00
661b263e77
add gocritic/ruleguard checks back again, cleanup code. ( #13665 )
...
- remove some duplicated code
- reported a bug, separately fixed in #13664
- using strings.ReplaceAll() when needed
- using filepath.ToSlash() use when needed
- remove all non-Go style comments from the codebase
Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2021-11-16 09:28:29 -08:00
82d73f387d
add tool to read healing.bin for debugging ( #13650 )
2021-11-12 16:18:53 -08:00
5acc8c0134
add multi-site replication tests ( #13631 )
2021-11-10 18:18:09 -08:00
03725dc015
Default multipart caching to writethrough ( #13613 )
...
when `MINIO_CACHE_COMMIT` is set.
- `writeback` caching applies only to single
uploads. When cache commit mode is
`writeback`, default multipart caching to be
synchronous.
- Add writethrough caching for single uploads
2021-11-10 08:12:03 -08:00
edf1f4233b
Update yaml files to latest version RELEASE.2021-11-09T03-21-45Z
2021-11-09 04:51:05 +00:00
5aef492b4c
update disk-caching design guide
2021-11-07 01:21:34 -08:00
5d7ed8ff7d
update S3 gateway limitation docs
2021-11-06 23:24:48 -07:00
cbb0828ab8
Update yaml files to latest version RELEASE.2021-11-05T09-16-26Z
2021-11-05 10:03:56 +00:00
9f3c151c3c
Update yaml files to latest version RELEASE.2021-11-03T03-36-36Z
2021-11-03 06:48:34 +00:00
e106070640
update docs to mention the expected behavior for requests_max
...
fixes #13561
2021-11-02 14:10:21 -07:00
1aa08f594d
Update README.md prometheus ( #13514 )
...
Modify the doc to warn users about Prometheus sending `domain:port`
2021-11-02 12:27:30 -07:00
15dcacc1fc
Add support for caching multipart in writethrough mode ( #13507 )
2021-11-01 08:11:58 -07:00
3c1220adca
add tests for default governance replication
2021-10-30 08:57:59 -07:00
2af5445309
update 3-site replication tests
2021-10-29 22:09:55 -07:00
abb1916bda
update list objects limit to match S3 spec
2021-10-28 18:21:51 -07:00
8e29ae8c44
Update yaml files to latest version RELEASE.2021-10-27T16-29-42Z
2021-10-28 02:45:22 +00:00
0c76fb57f2
Update yaml files to latest version RELEASE.2021-10-23T03-28-24Z
2021-10-23 07:46:29 +00:00
52c5f6e152
remove extraneous whitespaces
2021-10-21 14:43:13 -07:00
26ca59859f
update LDAP doc ( #13492 )
...
- clarify the login flow
- add some sections on configuration
- minor fixes to improve readability
2021-10-21 14:41:32 -07:00
1642867136
Add documentation for debugging tools ( #13484 )
...
Move `xl-meta` so it can be installed out-of-repo with a single command.
2021-10-20 10:12:46 -07:00
bedf739d16
update required IAM policies
2021-10-13 12:28:53 -07:00
6299e42aa9
Update yaml files to latest version RELEASE.2021-10-13T00-23-17Z
2021-10-13 01:14:23 +00:00
9ea45399ce
fix: enable AssumeRoleWithCertificate API only when asked ( #13410 )
...
This is a breaking change but we need to do this to avoid
issues discussed in #13409 based on discussions from #13371
fixes #13371
fixes #13409
2021-10-11 14:23:51 -07:00
e0db822a9b
Update yaml files to latest version RELEASE.2021-10-10T16-53-30Z
2021-10-10 21:24:01 +00:00
c49ebaaf1a
Update yaml files to latest version RELEASE.2021-10-08T23-58-24Z
2021-10-09 00:50:16 +00:00
d57b57bddc
feat: Add RX/TX to audit logging ( #13382 )
...
add additional values for audit logging
2021-10-07 19:03:46 -07:00
f81a188ef6
update site replication doc ( #13377 )
...
make pre-requisites clear
2021-10-07 15:21:01 -07:00
4ce6830a7b
Update yaml files to latest version RELEASE.2021-10-06T23-36-31Z
2021-10-07 04:13:46 +00:00
3a7c79e2c7
Add new site replication feature ( #13311 )
...
This change allows a set of MinIO sites (clusters) to be configured
for mutual replication of all buckets (including bucket policies, tags,
object-lock configuration and bucket encryption), IAM policies,
LDAP service accounts and LDAP STS accounts.
2021-10-06 16:36:31 -07:00
d6a3215fe2
Update yaml files to latest version RELEASE.2021-10-02T16-31-05Z
2021-10-03 21:43:13 +00:00
bc6067d195
Add admin inspect Glob support ( #13328 )
...
* Add admin Glob support
Allow returning multiple files on inspect calls.
```
λ mc admin inspect --json local2/testbucket/nyc-taxi-data-10M.csv.zst/*
...
λ unzip -l inspect.5f0643b2.zip
Archive: inspect.5f0643b2.zip
Length Date Time Name
--------- ---------- ----- ----
0 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
802 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
0 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
802 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
0 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
802 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
0 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
802 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
--------- -------
3208 8 files
```
Using fully recursive:
```
λ mc admin inspect local2/testbucket/nyc-taxi-data-10M.csv.zst/**
...
Archive: inspect.79c261cb.zip
Length Date Time Name
--------- ---------- ----- ----
0 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/
0 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.1
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.10
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.11
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.12
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.13
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.14
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.15
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.16
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.17
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.18
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.19
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.2
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.20
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.21
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.22
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.23
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.24
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.25
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.26
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.27
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.28
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.29
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.3
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.30
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.31
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.32
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.33
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.34
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.35
3439368 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.36
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.4
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.5
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.6
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.7
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.8
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.9
802 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
0 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/
0 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.1
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.10
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.11
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.12
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.13
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.14
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.15
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.16
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.17
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.18
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.19
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.2
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.20
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.21
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.22
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.23
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.24
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.25
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.26
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.27
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.28
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.29
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.3
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.30
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.31
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.32
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.33
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.34
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.35
3439368 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.36
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.4
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.5
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.6
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.7
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.8
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.9
802 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
0 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/
0 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.1
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.10
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.11
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.12
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.13
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.14
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.15
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.16
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.17
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.18
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.19
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.2
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.20
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.21
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.22
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.23
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.24
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.25
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.26
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.27
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.28
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.29
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.3
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.30
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.31
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.32
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.33
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.34
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.35
3439368 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.36
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.4
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.5
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.6
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.7
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.8
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.9
802 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
0 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/
0 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.1
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.10
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.11
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.12
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.13
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.14
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.15
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.16
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.17
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.18
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.19
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.2
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.20
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.21
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.22
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.23
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.24
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.25
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.26
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.27
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.28
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.29
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.3
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.30
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.31
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.32
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.33
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.34
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.35
3439368 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.36
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.4
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.5
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.6
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.7
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.8
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.9
802 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
--------- -------
601034920 156 files
```
Furthermore allow `inspect` to do direct decode from `mc`, for example:
```
λ mc admin inspect --json local2/testbucket/nyc-taxi-data-10M.csv.zst/*|inspect -json
Output decrypted to inspect.5f0643b2.zip
```
- Correct error, forward non-EOF errors.
- Add some extra safety. Log FNF when no files.
- Add `xl-meta` zip support.
For `xl-meta` multiple inputs output object with names as key.
Automatically switches `xl-meta` to single-line output when multiple objects.
Add double-star wildcard support to xl-meta input.
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-10-01 11:50:00 -07:00
7f6ed35347
Allow null versions to be replicated ( #13310 )
...
for pre-existing objects present in a bucket
prior to enabling existing object replication.
Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io>
2021-09-28 10:26:12 -07:00
a5923a5d51
Update yaml files to latest version RELEASE.2021-09-24T00-24-24Z
2021-09-24 19:20:44 +00:00
f4d1b7c603
Update yaml files to latest version RELEASE.2021-09-23T04-46-24Z
2021-09-23 07:42:55 +00:00
0b55a0423e
fix: cache usage deserialization from v5 to v6 ( #13258 )
2021-09-21 09:01:51 -07:00
f492f72154
add 3site replication script ( #13256 )
2021-09-20 18:24:24 -07:00
c4373ef290
Add support for multi site replication ( #12880 )
2021-09-18 13:31:35 -07:00
0b8c5a6872
Update yaml files to latest version RELEASE.2021-09-18T18-09-59Z
2021-09-18 20:28:07 +00:00
bf409936e7
Update yaml files to latest version RELEASE.2021-09-15T04-54-25Z
2021-09-15 08:04:57 +00:00
af78c3925a
add userinfo support for OpenID ( #12469 )
...
Some identity providers like GitLab do not provide
information about group membership as part of the
identity token claims. They only expose it via OIDC compatible
'/oauth/userinfo' endpoint, as described in the OpenID
Connect 1.0 sepcification.
But this of course requires application to make sure to add
additional accessToken, since idToken cannot be re-used to
perform the same 'userinfo' call. This is why this is specialized
requirement. Gitlab seems to be the only OpenID vendor that requires
this support for the time being.
fixes #12367
2021-09-13 16:22:14 -07:00
90e505e58f
calculate API requests/error as increase() intervals not as rate()
2021-09-12 11:28:28 -07:00
a216583d95
Update yaml files to latest version RELEASE.2021-09-09T21-37-07Z
2021-09-09 23:50:26 +00:00
950fe73c4f
fix: minio console nginx config ( #13156 )
2021-09-08 23:41:06 -07:00
aaa3fc3805
fix: remove deprecated LDAP username format support ( #13165 )
2021-09-08 13:31:51 -07:00
e438dccf19
sts: add support for certificate-based authentication ( #12748 )
...
This commit adds a new STS API for X.509 certificate
authentication.
A client can make an HTTP POST request over a TLS connection
and MinIO will verify the provided client certificate, map it to an
S3 policy and return temp. S3 credentials to the client.
So, this STS API allows clients to authenticate with X.509
certificates over TLS and obtain temp. S3 credentials.
For more details and examples refer to the docs/sts/tls.md
documentation.
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2021-09-07 19:03:48 -07:00
b2c92cdaaa
select: Add more compression formats ( #13142 )
...
Support Zstandard, LZ4, S2, and snappy as additional
compression formats for S3 Select.
2021-09-06 09:09:53 -07:00
1250312287
fail ready/liveness if etcd is unhealthy in gateway mode ( #13146 )
2021-09-03 17:05:41 -07:00
e2b6fb0a6a
Update yaml files to latest version RELEASE.2021-09-03T03-56-13Z
2021-09-03 06:45:54 +00:00
f486cfae86
point all docs for containers to quay.io
2021-09-01 18:48:26 -07:00
5c6dc63577
Update yaml files to latest version RELEASE.2021-08-31T05-46-54Z
2021-08-31 07:27:43 +00:00
470553ff5d
Tweak readall allocation and renameData buffer reuse ( #13108 )
...
Use a single allocation for reading the file, not the growing buffer of `io.ReadAll`.
Reuse the write buffer if we can when writing metadata in RenameData.
2021-08-30 08:38:11 -07:00
abf079135e
Update yaml files to latest version RELEASE.2021-08-25T00-41-18Z
2021-08-25 02:01:05 +00:00
60394ddf83
Add support for changing job name in Grafana dashboard ( #13050 )
2021-08-24 09:51:09 -07:00
7802088e71
Update yaml files to latest version RELEASE.2021-08-20T18-32-01Z
2021-08-20 20:42:00 +00:00
202d0b64eb
fix: enable go1.17 github ci/cd ( #12997 )
2021-08-18 18:35:22 -07:00
16aeb68c28
Update yaml files to latest version RELEASE.2021-08-17T20-53-08Z
2021-08-17 22:34:33 +00:00
30b77f59b1
doc: Add ilm prometheus metrics information ( #12994 )
2021-08-17 12:19:36 -07:00
9b7d593e28
Add helper script to call assume role ( #12978 )
2021-08-17 01:46:59 -07:00
a526ad2e80
Add headers into AMQP notifications ( #12911 )
...
Signed-off-by: Ricardo Katz <rkatz@vmware.com>
2021-08-11 22:24:19 -07:00
39f81d2c5b
update max_delay to max_sleep under healing docs
2021-08-10 08:52:42 -07:00
7b0b0f9101
Update yaml files to latest version RELEASE.2021-08-05T22-01-19Z
2021-08-06 02:19:42 +00:00
c13cbc64d1
fix multiple typos in documentation
2021-08-04 14:15:45 -07:00
109c8acf4f
fixed typo in metrics README.md ( #12874 )
2021-08-04 12:48:57 -07:00
035882d292
fix: remove parentIsObject() check ( #12851 )
...
we will allow situations such as
```
a/b/1.txt
a/b
```
and
```
a/b
a/b/1.txt
```
we are going to document that this usecase is
not supported and we will never support it, if
any application does this users have to delete
the top level parent to make sure namespace is
accessible at lower level.
rest of the situations where the prefixes get
created across sets are supported as is.
2021-08-03 13:26:57 -07:00
ea64a9263c
add deprecation notice for LDAP username format ( #12849 )
2021-08-02 18:20:06 -07:00
7281e86d9e
update docker-compose to RELEASE.2021-07-30T00-02-00Z
2021-07-29 18:07:06 -07:00
a78bc7bfdb
rename all access token to id_tokens
2021-07-27 22:38:12 -07:00
3735450e7e
fix: allow audience claim to be an array ( #12810 )
...
Some incorrect setups might have multiple audiences
where they are trying to use a single authentication
endpoint for multiple services.
Nevertheless OpenID spec allows it to make it
even more confusin for no good reason.
> It MUST contain the OAuth 2.0 client_id of the
> Relying Party as an audience value. It MAY also
> contain identifiers for other audiences. In the
> general case, the aud value is an array of case
> sensitive strings. In the common special case
> when there is one audience, the aud value MAY
> be a single case sensitive string.
fixes #12809
2021-07-27 18:37:51 -07:00
471b4fd0c9
Update yaml files to latest version RELEASE.2021-07-27T02-40-15Z
2021-07-27 05:02:00 +00:00
7106e9394c
[LDAP] Document new LDAP synchronization behavior ( #12786 )
2021-07-25 10:45:53 -07:00
9a31030e74
DEPRECATION Warning: Avoid configuring default LDAP STS Expiry ( #12781 )
...
- Show notice when `MINIO_IDENTITY_LDAP_STS_EXPIRY` or the
corresponding to the configuration option is used at server startup.
- Once support is removed, the default will be fixed at 1 hour.
- Users may specify expiry directly in the STS API.
- Update docs and help message
- Adds example in ldap.go to configure expiry in STS API.
2021-07-22 16:43:57 -07:00
a3079a7de2
fix: Add support for DurationSeconds in LDAP STS API ( #12778 )
2021-07-22 12:13:21 -07:00
026a005b64
Update yaml files to latest version RELEASE.2021-07-22T05-23-32Z
2021-07-22 06:12:29 +00:00
dbd7f74bb9
Update yaml files to latest version RELEASE.2021-07-21T22-15-23Z
2021-07-21 23:46:28 +00:00
c25ad94b7e
Refactor docker-compose, Use YAML variable for common minio configs ( #12735 )
2021-07-18 18:06:40 -07:00
ead8778305
Update yaml files to latest version RELEASE.2021-07-15T22-27-34Z
2021-07-16 04:27:54 +00:00
eae9c2f65b
Add changes to ensure session policy is enforced in LDAP STS ( #12716 )
...
- Bonus: Fix bug in webidentity sts that doesnt parse session policy correctly.
- update ldap.go to support session policy argument
2021-07-15 15:27:34 -07:00
ce800ed347
DOCS: Remove http links in favor of HTTPS ( #12725 )
2021-07-15 13:31:59 -07:00
8d19efe7e0
feat: increase allowed maximum STS expiration timeout to 365 days ( #12704 )
2021-07-14 00:08:22 -07:00
e316873f84
feat: Add support for kakfa audit logger target ( #12678 )
2021-07-13 09:39:13 -07:00
e25ae1c01f
add console-address flag for spoonfed users
2021-07-12 17:05:38 -07:00
bc8962102f
Update yaml files to latest version RELEASE.2021-07-12T02-44-53Z
2021-07-12 03:45:14 +00:00
55b08403cb
update docker docs with --console-address
...
also update the orchestration docs for nginx
proxy for console.
fixes #12660
2021-07-11 19:44:53 -07:00
d6a2fe02d3
Add admin file inspector ( #12635 )
...
Download files from *any* bucket/path as an encrypted zip file.
The key is included in the response but can be separated so zip
and the key doesn't have to be sent on the same channel.
Requires https://github.com/minio/pkg/pull/6
2021-07-09 11:29:16 -07:00
28adb29db3
feat: Add support to poll users on external SSO ( #12592 )
...
Additional support for vendor-specific admin API
integrations for OpenID, to ensure validity of
credentials on MinIO.
Every 5minutes check for validity of credentials
on MinIO with vendor specific IDP.
2021-07-09 11:17:21 -07:00
b6dd9b55a7
Update yaml files to latest version RELEASE.2021-07-08T19-43-25Z
2021-07-08 22:53:49 -07:00
95d29a6a53
fix: Mention GetBucketyPolicy in S3 gateway docs policy ( #12642 )
...
That action is needed for the gateway to obtain the policy
setting for public access.
Resolves: https://github.com/minio/minio/issues/12638
2021-07-08 10:34:56 -07:00
c14f965293
Update yaml files to latest version RELEASE.2021-07-08T01-15-01Z
2021-07-08 03:12:44 +00:00
2ce60d772b
fix: update README.md for new release ( #12637 )
...
rename all 'docker run' commands to 'podman run'
fixes #12633
2021-07-07 18:07:29 -07:00
3137dc2eb3
support implicit flow in web-identity.go example ( #12600 )
...
when a client secret is not provided,
automatically assume implicit flow
for authentication and invoke
relevant code accordingly.
2021-06-30 07:43:04 -07:00
b043e61e8e
remove deprecated warning for 'mc encrypt'
2021-06-25 08:12:40 -07:00
d40268d899
docs: add limits for bucket and object name length ( #12564 )
2021-06-23 12:51:31 -07:00
cdeccb5510
feat: Deprecate embedded browser and import console ( #12460 )
...
This feature also changes the default port where
the browser is running, now the port has moved
to 9001 and it can be configured with
```
--console-address ":9001"
```
2021-06-17 20:27:04 -07:00
e4fbc6a9ff
update to RELEASE.2021-06-17T00-10-46Z
2021-06-16 17:50:32 -07:00
0d1d26a4ea
Simplify s3zip example ( #12500 )
...
There is no need to create a custom transport wrapper.
Just set the header in the options.
2021-06-14 11:04:14 -07:00
264ee97219
update to RELEASE.2021-06-14T01-29-23Z release
2021-06-13 20:58:56 -07:00
92e4c8be10
Update replication docs to fix mc reference ( #12490 )
...
Signed-off-by: Poorna Krishnamoorthy <poorna@minio.io>
2021-06-13 11:37:22 -07:00
f2a3872301
Update design.md for replication ( #12486 )
...
Fixes #12483
2021-06-10 16:05:39 -07:00
ba5fb2365c
feat: support of ZIP list/get/head as S3 extension ( #12267 )
...
When enabled, it is possible to list/get files
inside a zip file without uncompressing it.
Signed-off-by: Anis Elleuch <anis@min.io>
2021-06-10 08:17:03 -07:00
22c58be421
Update yaml files to latest version RELEASE.2021-06-09T18-51-39Z
2021-06-09 19:21:21 +00:00
dd5e9493f1
docs: add missing admin actions ( #12442 )
2021-06-09 10:37:20 -07:00
32017454ee
fix typo in Grafana dashboard json ( #12471 )
2021-06-09 08:04:12 -07:00
1bf80a6eea
Update yaml files to latest version RELEASE.2021-06-07T21-40-51Z
2021-06-07 22:13:41 +00:00
00c5d7e1b3
Add healing related metrics in official dashboard ( #12456 )
2021-06-07 12:46:54 -07:00
f199afcd6c
tiering: add aws role support for s3 ( #12424 )
...
Signed-off-by: Poorna Krishnamoorthy <poorna@minio.io>
2021-06-04 12:47:00 -07:00
dbea8d2ee0
Add support for existing object replication. ( #12109 )
...
Also adding an API to allow resyncing replication when
existing object replication is enabled and the remote target
is entirely lost. With the `mc replicate reset` command, the
objects that are eligible for replication as per the replication
config will be resynced to target if existing object replication
is enabled on the rule.
2021-06-01 19:59:11 -07:00
1f262daf6f
rename all remaining packages to internal/ ( #12418 )
...
This is to ensure that there are no projects
that try to import `minio/minio/pkg` into
their own repo. Any such common packages should
go to `https://github.com/minio/pkg `
2021-06-01 14:59:40 -07:00
3690de0c6b
Drop Pending size and count from replication metrics ( #12378 )
...
Real-time metrics calculated in-memory rely on the initial
replication metrics saved with data usage. However, this can
lag behind the actual state of the cluster at the time of server
restart leading to inaccurate Pending size/counts reported to
Prometheus. Dropping the Pending metrics as this can be more
reliably monitored by applications with replication notifications.
Signed-off-by: Poorna Krishnamoorthy <poorna@minio.io>
2021-05-31 20:26:52 -07:00
ab7410af11
docs: add hostname entry for docker stack deploy
2021-05-31 19:11:52 -07:00
5cd6253508
docs: mention docker stack deploy doesn't work with compose.yml
2021-05-31 18:59:32 -07:00
f27513ff28
remove all docker swarm references
2021-05-31 18:55:21 -07:00
4444ba13a4
support ldap:username for policy substitution ( #12390 )
...
LDAPusername is the simpler form of LDAPUser (userDN),
using a simpler version is convenient from policy
conditions point of view, since these are unique id's
used for LDAP login.
2021-05-28 10:33:07 -07:00
62b6615a2a
Add xl-meta data extraction ( #12396 )
...
* Add `-export` flag to export data.
* Support wildcard inputs.
2021-05-28 08:10:41 -07:00
ba75281d5a
Update yaml files to latest version RELEASE.2021-05-27T22-06-31Z
2021-05-27 22:35:50 +00:00
acc452b7ce
Add more erasure codes on degraded systems. ( #11852 )
...
In cases where a cluster is degraded, we do not uphold our consistency
guarantee and we will write fewer erasure codes and rely on healing
to recreate the missing shards.
In some cases replacing known bad disks in practice take days.
We want to change the behavior of a known degraded system to keep
the erasure code promise of the storage class for each object.
This will create the objects with the same confidence as a fully
functional cluster. The tradeoff will be that objects created
during a partial outage will take up slightly more space.
This means that when the storage class is EC:4, there should
always be written 4 parity shards, even if some disks are unavailable.
When an object is created on a set, the disks are immediately
checked. If any disks are unavailable additional parity shards
will be made for each offline disk, up to 50% of the number of disks.
We add an internal metadata field with the actual and intended
erasure code level, this can optionally be picked up later by
the scanner if we decide that data like this should be re-sharded.
2021-05-27 11:38:09 -07:00
9773b16f6f
update TLS docs to use new certgen tool
2021-05-26 10:41:29 -07:00
7b06b754a6
Update yaml files to latest version RELEASE.2021-05-26T00-22-46Z
2021-05-26 00:52:38 +00:00
Harshavardhana
Minio Trusted
Aditya Manthramurthy
Praveen raj Mani
Klaus Post
Krishna Srinivas
Eric Qiu
Krishnan Parthasarathi
Andreas Auernhammer
Eco
Poorna
Jan Madera
Emmet McPoland
Pierre Kancir
Anis Elleuch
Bingchang Chen
Yoann Guillerme
fpaupier
Ravind Kumar
chrisbecke
Mani
Ashish Kumar Sinha
jandres - moscardo
soudhaf
Nitish Tiwari
Ricardo Katz
Matt Sarrel
Ali Malek
Christian Weiske
iternity-dotcom