76b21de0c6
feat: decommission feature for pools ( #14012 )
...
```
λ mc admin decommission start alias/ http://minio{1...2}/data{1...4}
```
```
λ mc admin decommission status alias/
┌─────┬─────────────────────────────────┬──────────────────────────────────┬────────┐
│ ID │ Pools │ Capacity │ Status │
│ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Active │
│ 2nd │ http://minio{3...4}/data{1...4} │ 329 GiB (used) / 421 GiB (total) │ Active │
└─────┴─────────────────────────────────┴──────────────────────────────────┴────────┘
```
```
λ mc admin decommission status alias/ http://minio{1...2}/data{1...4}
Progress: ===================> [1GiB/sec] [15%] [4TiB/50TiB]
Time Remaining: 4 hours (started 3 hours ago)
```
```
λ mc admin decommission status alias/ http://minio{1...2}/data{1...4}
ERROR: This pool is not scheduled for decommissioning currently.
```
```
λ mc admin decommission cancel alias/
┌─────┬─────────────────────────────────┬──────────────────────────────────┬──────────┐
│ ID │ Pools │ Capacity │ Status │
│ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Draining │
└─────┴─────────────────────────────────┴──────────────────────────────────┴──────────┘
```
> NOTE: Canceled decommission will not make the pool active again, since we might have
> Potentially partial duplicate content on the other pools, to avoid this scenario be
> very sure to start decommissioning as a planned activity.
```
λ mc admin decommission cancel alias/ http://minio{1...2}/data{1...4}
┌─────┬─────────────────────────────────┬──────────────────────────────────┬────────────────────┐
│ ID │ Pools │ Capacity │ Status │
│ 1st │ http://minio{1...2}/data{1...4} │ 439 GiB (used) / 561 GiB (total) │ Draining(Canceled) │
└─────┴─────────────────────────────────┴──────────────────────────────────┴────────────────────┘
```
2022-01-10 09:07:49 -08:00
f394313fee
Update yaml files to latest version RELEASE.2022-01-08T03-11-54Z
2022-01-08 05:53:59 +00:00
b7c5e45fff
heal: isObjectDangling should return false when it cannot decide ( #14053 )
...
In a multi-pool setup when disks are coming up, or in a single pool
setup let's say with 100's of erasure sets with a slow network.
It's possible when healing is attempted on `.minio.sys/config`
folder, it can lead to healing unexpectedly deleting some policy
files as dangling due to a mistake in understanding when `isObjectDangling`
is considered to be 'true'.
This issue happened in commit 30135eed86
2022-01-07 19:11:54 -08:00
0a224654c2
fix: progagation of service accounts for site replication ( #14054 )
...
- Only non-root-owned service accounts are replicated for now.
- Add integration tests for OIDC with site replication
2022-01-07 17:41:43 -08:00
e420a1de4d
Update yaml files to latest version RELEASE.2022-01-07T01-53-23Z
2022-01-07 05:57:36 +00:00
62dc0f7698
Update site replication docs ( #14044 )
...
- Now OpenID external IDP is also supported for site replication.
- IAM users/groups are also replicated
2022-01-06 17:53:23 -08:00
f4c03e56b8
Update yaml files to latest version RELEASE.2022-01-04T07-41-07Z
2022-01-04 23:01:29 +00:00
5dd40b9377
Update yaml files to latest version RELEASE.2022-01-03T18-22-58Z
2022-01-04 00:12:07 +00:00
a60ac7ca17
fix: audit log to support object names in multipleObjectNames() handler ( #14017 )
2022-01-03 01:28:52 -08:00
c9e49f4366
Update yaml files to latest version RELEASE.2021-12-29T06-49-06Z
2021-12-29 07:38:12 +00:00
3e28af1723
docs: update TLS doc use -keyout instead of -key ( #14001 )
2021-12-28 12:51:38 -08:00
6aa0574a53
Update yaml files to latest version RELEASE.2021-12-27T07-23-18Z
2021-12-27 19:52:13 +00:00
7e3a7d7044
add healing for invalid shards by skipping the blocks ( #13978 )
...
Built on top of #13945 , now we need to simply skip the
shards and its automated.
2021-12-23 23:01:46 -08:00
416977436e
rename MINIO_CACHE_.._MASTER_KEY to MINIO_CACHE_.._SECRET_KEY
...
fixes #13975
2021-12-22 12:11:07 -08:00
0e3037631f
skip inconsistent shards if possible ( #13945 )
...
data shards were wrong due to a healing bug
reported in #13803 mainly with unaligned object
sizes.
This PR is an attempt to automatically avoid
these shards, with available information about
the `xl.meta` and actually disk mtime.
2021-12-21 10:08:26 -08:00
f3602d7d08
Update yaml files to latest version RELEASE.2021-12-20T22-07-16Z
2021-12-20 23:13:48 +00:00
0b21734571
update helm chart link
2021-12-20 13:20:59 -08:00
cfd66ab8c3
Update yaml files to latest version RELEASE.2021-12-18T04-42-33Z
2021-12-19 22:19:30 +00:00
3ddb501190
Add docs for NewerNoncurrentVersions ( #13944 )
2021-12-17 19:32:29 -08:00
4fa250a6a1
add hash-set debugging currently supports SIPMOD ( #13911 )
...
fixes a bug in s3-check-md5 for single part
uploaded multipart objects.
2021-12-15 12:07:15 -08:00
88ad742da0
fix: error handling cases in site-replication ( #13901 )
...
- Allow proper SRError to be propagated to
handlers and converted appropriately.
- Make sure to enable object locking on buckets
when requested in MakeBucketHook.
- When DNSConfig is enabled attempt to delete it
first before deleting buckets locally.
2021-12-14 14:09:57 -08:00
40dbe243d9
update: keycloak binding documentation ( #13894 )
2021-12-13 18:23:16 -08:00
109c927dad
docs/debug: log any corruption and continue debug tool
2021-12-13 17:45:34 -08:00
878d368cea
Update yaml files to latest version RELEASE.2021-12-10T23-03-39Z
2021-12-11 00:02:59 -08:00
518612492c
xl-meta: Add header titles ( #13880 )
...
Add type for headers and create custom marshal to make
it easier to read. Group headers and metadata.
Restore functionality that will read `xl.meta` in the current dir with no params.
Before:
```
{
"Headers": [
[
"8M04bTiYRDmEMQGeAsk1yg==",
1639150471630100400,
"rLD1Rw==",
1,
6
],
],
"Versions": [
{
"Type": 1,
"V2Obj": {
"CSumAlgo": 1,
"DDir": "oC1Xpg4tRfW03g8o8w7Bzg==",
"EcAlgo": 1,
"EcBSize": 1048576,
"EcDist": [
7,
8,
1,
2,
3,
4,
5,
6
],
"EcIndex": 1,
"EcM": 4,
"EcN": 4,
"ID": "8M04bTiYRDmEMQGeAsk1yg==",
"MTime": 1639150471630100400,
"MetaSys": {
"x-minio-internal-inline-data": "dHJ1ZQ=="
},
"MetaUsr": {
"content-type": "application/octet-stream",
"etag": "b8252c86fad2d8937300aa92b467a3aa"
},
"PartASizes": [
1000
],
"PartETags": null,
"PartNums": [
1
],
"PartSizes": [
1000
],
"Size": 1000
}
}
]
}
```
After:
```
{
"Versions": [
{
"Header": {
"Flags": 6,
"ModTime": "2021-12-10T16:34:31.6301004+01:00",
"Signature": "acb0f547",
"Type": 1,
"VersionID": "f0cd386d389844398431019e02c935ca"
},
"Idx": 0,
"Metadata": {
"Type": 1,
"V2Obj": {
"CSumAlgo": 1,
"DDir": "oC1Xpg4tRfW03g8o8w7Bzg==",
"EcAlgo": 1,
"EcBSize": 1048576,
"EcDist": [
7,
8,
1,
2,
3,
4,
5,
6
],
"EcIndex": 1,
"EcM": 4,
"EcN": 4,
"ID": "8M04bTiYRDmEMQGeAsk1yg==",
"MTime": 1639150471630100400,
"MetaSys": {
"x-minio-internal-inline-data": "dHJ1ZQ=="
},
"MetaUsr": {
"content-type": "application/octet-stream",
"etag": "b8252c86fad2d8937300aa92b467a3aa"
},
"PartASizes": [
1000
],
"PartETags": null,
"PartNums": [
1
],
"PartSizes": [
1000
],
"Size": 1000
}
}
}
]
}
```
2021-12-10 15:03:25 -08:00
c76f86fdbd
Clarify example for Standalone Docker instructions ( #13879 )
...
Closes #13868
Also points users to the web documentation for docker installation.
2021-12-10 09:43:02 -08:00
4be8023408
Update yaml files to latest version RELEASE.2021-12-09T06-19-41Z
2021-12-09 08:40:46 +00:00
dcff6c996d
fix: do not list delete-marked objects ( #13864 )
...
delete marked objects should not be considered
for listing when listing is delimited, this issue
as introduced in PR #13804 which was mainly to
address listing of directories in listing when
delimited.
This PR fixes this properly and adds tests to
ensure that we behave in accordance with how
an S3 API behaves for ListObjects() without
versions.
2021-12-08 17:34:52 -08:00
e82a5c5c54
fix: site replication issues and add tests ( #13861 )
...
- deleting policies was deleting all LDAP
user mapping, this was a regression introduced
in #13567
- deleting of policies is properly sent across
all sites.
- remove unexpected errors instead embed the real
errors as part of the 500 error response.
2021-12-08 11:50:15 -08:00
92fdcafb66
add verification tests for ETag on replicated content ( #13857 )
2021-12-07 10:08:26 -08:00
be34fc9134
fix: kms-id header should have arn:aws:kms: prefix ( #13833 )
...
arn:aws:kms: is a must for KMS keyID.
2021-12-06 00:39:32 -08:00
20c60aae68
Update hotfix documentation and container building
2021-12-02 17:52:46 -08:00
f840080e5b
cleanup site-replication docs ( #13812 )
2021-12-02 13:27:01 -08:00
ffe9acfe4a
docs: Add a markdown documentation on hotfix branches and process
2021-12-01 01:00:18 -08:00
9ec197f2e8
Add support for adding new site(s) to site replication ( #13696 )
...
Currently, the new site is expected to be empty
2021-11-30 13:16:37 -08:00
4c0f48c548
Add role ARN support for OIDC identity provider ( #13651 )
...
- Allows setting a role policy parameter when configuring OIDC provider
- When role policy is set, the server prints a role ARN usable in STS API requests
- The given role policy is applied to STS API requests when the roleARN parameter is provided.
- Service accounts for role policy are also possible and work as expected.
2021-11-26 19:22:40 -08:00
4ce6d35e30
Add new `site` config sub-system intended to replace `region` ( #13672 )
...
- New sub-system has "region" and "name" fields.
- `region` subsystem is marked as deprecated, however still works, unless the
new region parameter under `site` is set - in this case, the region subsystem is
ignored. `region` subsystem is hidden from top-level help (i.e. from `mc admin
config set myminio`), but appears when specifically requested (i.e. with `mc
admin config set myminio region`).
- MINIO_REGION, MINIO_REGION_NAME are supported as legacy environment variables for server region.
- Adds MINIO_SITE_REGION as the current environment variable to configure the
server region and MINIO_SITE_NAME for the site name.
2021-11-25 13:06:25 -08:00
a5db4ca092
Update yaml files to latest version RELEASE.2021-11-24T23-19-33Z
2021-11-25 07:39:00 +00:00
ef0b8367b5
Update minio-overview.json data source panel ( #13730 )
...
Add missing datasource in `Healing` panel.
2021-11-23 09:01:07 -08:00
3da9ee15d3
Add MaxNoncurrentVersions to NoncurrentExpiration action ( #13580 )
...
This unit allows users to limit the maximum number of noncurrent
versions of an object.
To enable this rule you need the following *ilm.json*
```
cat >> ilm.json <<EOF
{
"Rules": [
{
"ID": "test-max-noncurrent",
"Status": "Enabled",
"Filter": {
"Prefix": "user-uploads/"
},
"NoncurrentVersionExpiration": {
"MaxNoncurrentVersions": 5
}
}
]
}
EOF
mc ilm import myminio/mybucket < ilm.json
```
2021-11-19 17:54:10 -08:00
7b82411e6f
change the unit of measurement from TB to TiB ( #13686 )
2021-11-18 20:06:37 -08:00
79b3a1fe4e
remove object torrent, AWS S3 removed support for torrent API
2021-11-18 12:21:48 -08:00
faf013ec84
Improve performance on multiple versions ( #13573 )
...
Existing:
```go
type xlMetaV2 struct {
Versions []xlMetaV2Version `json:"Versions" msg:"Versions"`
}
```
Serialized as regular MessagePack.
```go
//msgp:tuple xlMetaV2VersionHeader
type xlMetaV2VersionHeader struct {
VersionID [16]byte
ModTime int64
Type VersionType
Flags xlFlags
}
```
Serialize as streaming MessagePack, format:
```
int(headerVersion)
int(xlmetaVersion)
int(nVersions)
for each version {
binary blob, xlMetaV2VersionHeader, serialized
binary blob, xlMetaV2Version, serialized.
}
```
xlMetaV2VersionHeader is <= 30 bytes serialized. Deserialized struct
can easily be reused and does not contain pointers, so efficient as a
slice (single allocation)
This allows quickly parsing everything as slices of bytes (no copy).
Versions are always *saved* sorted by modTime, newest *first*.
No more need to sort on load.
* Allows checking if a version exists.
* Allows reading single version without unmarshal all.
* Allows reading latest version of type without unmarshal all.
* Allows reading latest version without unmarshal of all.
* Allows checking if the latest is deleteMarker by reading first entry.
* Allows adding/updating/deleting a version with only header deserialization.
* Reduces allocations on conversion to FileInfo(s).
2021-11-18 12:15:22 -08:00
3d2bc15e9a
Add grafana json file for replication metrics ( #13678 )
2021-11-17 14:49:46 -08:00
661b263e77
add gocritic/ruleguard checks back again, cleanup code. ( #13665 )
...
- remove some duplicated code
- reported a bug, separately fixed in #13664
- using strings.ReplaceAll() when needed
- using filepath.ToSlash() use when needed
- remove all non-Go style comments from the codebase
Co-authored-by: Aditya Manthramurthy <donatello@users.noreply.github.com>
2021-11-16 09:28:29 -08:00
82d73f387d
add tool to read healing.bin for debugging ( #13650 )
2021-11-12 16:18:53 -08:00
5acc8c0134
add multi-site replication tests ( #13631 )
2021-11-10 18:18:09 -08:00
03725dc015
Default multipart caching to writethrough ( #13613 )
...
when `MINIO_CACHE_COMMIT` is set.
- `writeback` caching applies only to single
uploads. When cache commit mode is
`writeback`, default multipart caching to be
synchronous.
- Add writethrough caching for single uploads
2021-11-10 08:12:03 -08:00
edf1f4233b
Update yaml files to latest version RELEASE.2021-11-09T03-21-45Z
2021-11-09 04:51:05 +00:00
5aef492b4c
update disk-caching design guide
2021-11-07 01:21:34 -08:00
5d7ed8ff7d
update S3 gateway limitation docs
2021-11-06 23:24:48 -07:00
cbb0828ab8
Update yaml files to latest version RELEASE.2021-11-05T09-16-26Z
2021-11-05 10:03:56 +00:00
9f3c151c3c
Update yaml files to latest version RELEASE.2021-11-03T03-36-36Z
2021-11-03 06:48:34 +00:00
e106070640
update docs to mention the expected behavior for requests_max
...
fixes #13561
2021-11-02 14:10:21 -07:00
1aa08f594d
Update README.md prometheus ( #13514 )
...
Modify the doc to warn users about Prometheus sending `domain:port`
2021-11-02 12:27:30 -07:00
15dcacc1fc
Add support for caching multipart in writethrough mode ( #13507 )
2021-11-01 08:11:58 -07:00
3c1220adca
add tests for default governance replication
2021-10-30 08:57:59 -07:00
2af5445309
update 3-site replication tests
2021-10-29 22:09:55 -07:00
abb1916bda
update list objects limit to match S3 spec
2021-10-28 18:21:51 -07:00
8e29ae8c44
Update yaml files to latest version RELEASE.2021-10-27T16-29-42Z
2021-10-28 02:45:22 +00:00
0c76fb57f2
Update yaml files to latest version RELEASE.2021-10-23T03-28-24Z
2021-10-23 07:46:29 +00:00
52c5f6e152
remove extraneous whitespaces
2021-10-21 14:43:13 -07:00
26ca59859f
update LDAP doc ( #13492 )
...
- clarify the login flow
- add some sections on configuration
- minor fixes to improve readability
2021-10-21 14:41:32 -07:00
1642867136
Add documentation for debugging tools ( #13484 )
...
Move `xl-meta` so it can be installed out-of-repo with a single command.
2021-10-20 10:12:46 -07:00
bedf739d16
update required IAM policies
2021-10-13 12:28:53 -07:00
6299e42aa9
Update yaml files to latest version RELEASE.2021-10-13T00-23-17Z
2021-10-13 01:14:23 +00:00
9ea45399ce
fix: enable AssumeRoleWithCertificate API only when asked ( #13410 )
...
This is a breaking change but we need to do this to avoid
issues discussed in #13409 based on discussions from #13371
fixes #13371
fixes #13409
2021-10-11 14:23:51 -07:00
e0db822a9b
Update yaml files to latest version RELEASE.2021-10-10T16-53-30Z
2021-10-10 21:24:01 +00:00
c49ebaaf1a
Update yaml files to latest version RELEASE.2021-10-08T23-58-24Z
2021-10-09 00:50:16 +00:00
d57b57bddc
feat: Add RX/TX to audit logging ( #13382 )
...
add additional values for audit logging
2021-10-07 19:03:46 -07:00
f81a188ef6
update site replication doc ( #13377 )
...
make pre-requisites clear
2021-10-07 15:21:01 -07:00
4ce6830a7b
Update yaml files to latest version RELEASE.2021-10-06T23-36-31Z
2021-10-07 04:13:46 +00:00
3a7c79e2c7
Add new site replication feature ( #13311 )
...
This change allows a set of MinIO sites (clusters) to be configured
for mutual replication of all buckets (including bucket policies, tags,
object-lock configuration and bucket encryption), IAM policies,
LDAP service accounts and LDAP STS accounts.
2021-10-06 16:36:31 -07:00
d6a3215fe2
Update yaml files to latest version RELEASE.2021-10-02T16-31-05Z
2021-10-03 21:43:13 +00:00
bc6067d195
Add admin inspect Glob support ( #13328 )
...
* Add admin Glob support
Allow returning multiple files on inspect calls.
```
λ mc admin inspect --json local2/testbucket/nyc-taxi-data-10M.csv.zst/*
...
λ unzip -l inspect.5f0643b2.zip
Archive: inspect.5f0643b2.zip
Length Date Time Name
--------- ---------- ----- ----
0 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
802 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
0 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
802 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
0 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
802 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
0 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
802 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
--------- -------
3208 8 files
```
Using fully recursive:
```
λ mc admin inspect local2/testbucket/nyc-taxi-data-10M.csv.zst/**
...
Archive: inspect.79c261cb.zip
Length Date Time Name
--------- ---------- ----- ----
0 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/
0 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.1
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.10
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.11
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.12
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.13
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.14
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.15
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.16
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.17
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.18
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.19
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.2
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.20
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.21
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.22
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.23
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.24
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.25
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.26
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.27
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.28
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.29
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.3
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.30
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.31
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.32
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.33
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.34
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.35
3439368 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.36
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.4
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.5
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.6
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.7
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.8
4194816 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.9
802 2021-09-03 12:50 192.168.1.78:9001/a221edde-48fe-45f5-ad32-3bc7131c7659/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
0 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/
0 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.1
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.10
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.11
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.12
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.13
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.14
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.15
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.16
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.17
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.18
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.19
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.2
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.20
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.21
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.22
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.23
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.24
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.25
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.26
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.27
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.28
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.29
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.3
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.30
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.31
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.32
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.33
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.34
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.35
3439368 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.36
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.4
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.5
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.6
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.7
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.8
4194816 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.9
802 2021-09-03 12:50 192.168.1.78:9001/cb7440ef-f0d9-42a8-b137-f00f519276ca/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
0 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/
0 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.1
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.10
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.11
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.12
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.13
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.14
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.15
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.16
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.17
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.18
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.19
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.2
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.20
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.21
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.22
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.23
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.24
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.25
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.26
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.27
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.28
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.29
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.3
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.30
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.31
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.32
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.33
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.34
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.35
3439368 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.36
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.4
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.5
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.6
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.7
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.8
4194816 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.9
802 2021-09-03 12:50 192.168.1.78:9001/759cd5ac-7860-4cf3-acad-a375fcbae338/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
0 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/
0 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.1
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.10
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.11
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.12
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.13
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.14
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.15
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.16
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.17
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.18
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.19
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.2
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.20
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.21
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.22
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.23
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.24
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.25
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.26
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.27
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.28
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.29
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.3
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.30
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.31
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.32
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.33
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.34
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.35
3439368 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.36
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.4
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.5
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.6
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.7
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.8
4194816 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/18a50b3e-3c56-418e-a045-ad5c58c1d44b/part.9
802 2021-09-09 15:56 192.168.1.78:9001/2b48619c-c2fa-4e69-839e-58fc82c1b43e/testbucket/nyc-taxi-data-10M.csv.zst/xl.meta
--------- -------
601034920 156 files
```
Furthermore allow `inspect` to do direct decode from `mc`, for example:
```
λ mc admin inspect --json local2/testbucket/nyc-taxi-data-10M.csv.zst/*|inspect -json
Output decrypted to inspect.5f0643b2.zip
```
- Correct error, forward non-EOF errors.
- Add some extra safety. Log FNF when no files.
- Add `xl-meta` zip support.
For `xl-meta` multiple inputs output object with names as key.
Automatically switches `xl-meta` to single-line output when multiple objects.
Add double-star wildcard support to xl-meta input.
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-10-01 11:50:00 -07:00
7f6ed35347
Allow null versions to be replicated ( #13310 )
...
for pre-existing objects present in a bucket
prior to enabling existing object replication.
Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io>
2021-09-28 10:26:12 -07:00
a5923a5d51
Update yaml files to latest version RELEASE.2021-09-24T00-24-24Z
2021-09-24 19:20:44 +00:00
f4d1b7c603
Update yaml files to latest version RELEASE.2021-09-23T04-46-24Z
2021-09-23 07:42:55 +00:00
0b55a0423e
fix: cache usage deserialization from v5 to v6 ( #13258 )
2021-09-21 09:01:51 -07:00
f492f72154
add 3site replication script ( #13256 )
2021-09-20 18:24:24 -07:00
c4373ef290
Add support for multi site replication ( #12880 )
2021-09-18 13:31:35 -07:00
0b8c5a6872
Update yaml files to latest version RELEASE.2021-09-18T18-09-59Z
2021-09-18 20:28:07 +00:00
bf409936e7
Update yaml files to latest version RELEASE.2021-09-15T04-54-25Z
2021-09-15 08:04:57 +00:00
af78c3925a
add userinfo support for OpenID ( #12469 )
...
Some identity providers like GitLab do not provide
information about group membership as part of the
identity token claims. They only expose it via OIDC compatible
'/oauth/userinfo' endpoint, as described in the OpenID
Connect 1.0 sepcification.
But this of course requires application to make sure to add
additional accessToken, since idToken cannot be re-used to
perform the same 'userinfo' call. This is why this is specialized
requirement. Gitlab seems to be the only OpenID vendor that requires
this support for the time being.
fixes #12367
2021-09-13 16:22:14 -07:00
90e505e58f
calculate API requests/error as increase() intervals not as rate()
2021-09-12 11:28:28 -07:00
a216583d95
Update yaml files to latest version RELEASE.2021-09-09T21-37-07Z
2021-09-09 23:50:26 +00:00
950fe73c4f
fix: minio console nginx config ( #13156 )
2021-09-08 23:41:06 -07:00
aaa3fc3805
fix: remove deprecated LDAP username format support ( #13165 )
2021-09-08 13:31:51 -07:00
e438dccf19
sts: add support for certificate-based authentication ( #12748 )
...
This commit adds a new STS API for X.509 certificate
authentication.
A client can make an HTTP POST request over a TLS connection
and MinIO will verify the provided client certificate, map it to an
S3 policy and return temp. S3 credentials to the client.
So, this STS API allows clients to authenticate with X.509
certificates over TLS and obtain temp. S3 credentials.
For more details and examples refer to the docs/sts/tls.md
documentation.
Signed-off-by: Andreas Auernhammer <hi@aead.dev>
2021-09-07 19:03:48 -07:00
b2c92cdaaa
select: Add more compression formats ( #13142 )
...
Support Zstandard, LZ4, S2, and snappy as additional
compression formats for S3 Select.
2021-09-06 09:09:53 -07:00
1250312287
fail ready/liveness if etcd is unhealthy in gateway mode ( #13146 )
2021-09-03 17:05:41 -07:00
e2b6fb0a6a
Update yaml files to latest version RELEASE.2021-09-03T03-56-13Z
2021-09-03 06:45:54 +00:00
f486cfae86
point all docs for containers to quay.io
2021-09-01 18:48:26 -07:00
5c6dc63577
Update yaml files to latest version RELEASE.2021-08-31T05-46-54Z
2021-08-31 07:27:43 +00:00
470553ff5d
Tweak readall allocation and renameData buffer reuse ( #13108 )
...
Use a single allocation for reading the file, not the growing buffer of `io.ReadAll`.
Reuse the write buffer if we can when writing metadata in RenameData.
2021-08-30 08:38:11 -07:00
abf079135e
Update yaml files to latest version RELEASE.2021-08-25T00-41-18Z
2021-08-25 02:01:05 +00:00
60394ddf83
Add support for changing job name in Grafana dashboard ( #13050 )
2021-08-24 09:51:09 -07:00
7802088e71
Update yaml files to latest version RELEASE.2021-08-20T18-32-01Z
2021-08-20 20:42:00 +00:00
202d0b64eb
fix: enable go1.17 github ci/cd ( #12997 )
2021-08-18 18:35:22 -07:00
16aeb68c28
Update yaml files to latest version RELEASE.2021-08-17T20-53-08Z
2021-08-17 22:34:33 +00:00
30b77f59b1
doc: Add ilm prometheus metrics information ( #12994 )
2021-08-17 12:19:36 -07:00
9b7d593e28
Add helper script to call assume role ( #12978 )
2021-08-17 01:46:59 -07:00
a526ad2e80
Add headers into AMQP notifications ( #12911 )
...
Signed-off-by: Ricardo Katz <rkatz@vmware.com>
2021-08-11 22:24:19 -07:00
39f81d2c5b
update max_delay to max_sleep under healing docs
2021-08-10 08:52:42 -07:00
7b0b0f9101
Update yaml files to latest version RELEASE.2021-08-05T22-01-19Z
2021-08-06 02:19:42 +00:00
c13cbc64d1
fix multiple typos in documentation
2021-08-04 14:15:45 -07:00
109c8acf4f
fixed typo in metrics README.md ( #12874 )
2021-08-04 12:48:57 -07:00
035882d292
fix: remove parentIsObject() check ( #12851 )
...
we will allow situations such as
```
a/b/1.txt
a/b
```
and
```
a/b
a/b/1.txt
```
we are going to document that this usecase is
not supported and we will never support it, if
any application does this users have to delete
the top level parent to make sure namespace is
accessible at lower level.
rest of the situations where the prefixes get
created across sets are supported as is.
2021-08-03 13:26:57 -07:00
ea64a9263c
add deprecation notice for LDAP username format ( #12849 )
2021-08-02 18:20:06 -07:00
7281e86d9e
update docker-compose to RELEASE.2021-07-30T00-02-00Z
2021-07-29 18:07:06 -07:00
a78bc7bfdb
rename all access token to id_tokens
2021-07-27 22:38:12 -07:00
3735450e7e
fix: allow audience claim to be an array ( #12810 )
...
Some incorrect setups might have multiple audiences
where they are trying to use a single authentication
endpoint for multiple services.
Nevertheless OpenID spec allows it to make it
even more confusin for no good reason.
> It MUST contain the OAuth 2.0 client_id of the
> Relying Party as an audience value. It MAY also
> contain identifiers for other audiences. In the
> general case, the aud value is an array of case
> sensitive strings. In the common special case
> when there is one audience, the aud value MAY
> be a single case sensitive string.
fixes #12809
2021-07-27 18:37:51 -07:00
471b4fd0c9
Update yaml files to latest version RELEASE.2021-07-27T02-40-15Z
2021-07-27 05:02:00 +00:00
7106e9394c
[LDAP] Document new LDAP synchronization behavior ( #12786 )
2021-07-25 10:45:53 -07:00
9a31030e74
DEPRECATION Warning: Avoid configuring default LDAP STS Expiry ( #12781 )
...
- Show notice when `MINIO_IDENTITY_LDAP_STS_EXPIRY` or the
corresponding to the configuration option is used at server startup.
- Once support is removed, the default will be fixed at 1 hour.
- Users may specify expiry directly in the STS API.
- Update docs and help message
- Adds example in ldap.go to configure expiry in STS API.
2021-07-22 16:43:57 -07:00
a3079a7de2
fix: Add support for DurationSeconds in LDAP STS API ( #12778 )
2021-07-22 12:13:21 -07:00
026a005b64
Update yaml files to latest version RELEASE.2021-07-22T05-23-32Z
2021-07-22 06:12:29 +00:00
dbd7f74bb9
Update yaml files to latest version RELEASE.2021-07-21T22-15-23Z
2021-07-21 23:46:28 +00:00
c25ad94b7e
Refactor docker-compose, Use YAML variable for common minio configs ( #12735 )
2021-07-18 18:06:40 -07:00
ead8778305
Update yaml files to latest version RELEASE.2021-07-15T22-27-34Z
2021-07-16 04:27:54 +00:00
eae9c2f65b
Add changes to ensure session policy is enforced in LDAP STS ( #12716 )
...
- Bonus: Fix bug in webidentity sts that doesnt parse session policy correctly.
- update ldap.go to support session policy argument
2021-07-15 15:27:34 -07:00
ce800ed347
DOCS: Remove http links in favor of HTTPS ( #12725 )
2021-07-15 13:31:59 -07:00
8d19efe7e0
feat: increase allowed maximum STS expiration timeout to 365 days ( #12704 )
2021-07-14 00:08:22 -07:00
e316873f84
feat: Add support for kakfa audit logger target ( #12678 )
2021-07-13 09:39:13 -07:00
e25ae1c01f
add console-address flag for spoonfed users
2021-07-12 17:05:38 -07:00
bc8962102f
Update yaml files to latest version RELEASE.2021-07-12T02-44-53Z
2021-07-12 03:45:14 +00:00
55b08403cb
update docker docs with --console-address
...
also update the orchestration docs for nginx
proxy for console.
fixes #12660
2021-07-11 19:44:53 -07:00
d6a2fe02d3
Add admin file inspector ( #12635 )
...
Download files from *any* bucket/path as an encrypted zip file.
The key is included in the response but can be separated so zip
and the key doesn't have to be sent on the same channel.
Requires https://github.com/minio/pkg/pull/6
2021-07-09 11:29:16 -07:00
28adb29db3
feat: Add support to poll users on external SSO ( #12592 )
...
Additional support for vendor-specific admin API
integrations for OpenID, to ensure validity of
credentials on MinIO.
Every 5minutes check for validity of credentials
on MinIO with vendor specific IDP.
2021-07-09 11:17:21 -07:00
b6dd9b55a7
Update yaml files to latest version RELEASE.2021-07-08T19-43-25Z
2021-07-08 22:53:49 -07:00
95d29a6a53
fix: Mention GetBucketyPolicy in S3 gateway docs policy ( #12642 )
...
That action is needed for the gateway to obtain the policy
setting for public access.
Resolves: https://github.com/minio/minio/issues/12638
2021-07-08 10:34:56 -07:00
c14f965293
Update yaml files to latest version RELEASE.2021-07-08T01-15-01Z
2021-07-08 03:12:44 +00:00
2ce60d772b
fix: update README.md for new release ( #12637 )
...
rename all 'docker run' commands to 'podman run'
fixes #12633
2021-07-07 18:07:29 -07:00
3137dc2eb3
support implicit flow in web-identity.go example ( #12600 )
...
when a client secret is not provided,
automatically assume implicit flow
for authentication and invoke
relevant code accordingly.
2021-06-30 07:43:04 -07:00
b043e61e8e
remove deprecated warning for 'mc encrypt'
2021-06-25 08:12:40 -07:00
d40268d899
docs: add limits for bucket and object name length ( #12564 )
2021-06-23 12:51:31 -07:00
cdeccb5510
feat: Deprecate embedded browser and import console ( #12460 )
...
This feature also changes the default port where
the browser is running, now the port has moved
to 9001 and it can be configured with
```
--console-address ":9001"
```
2021-06-17 20:27:04 -07:00
e4fbc6a9ff
update to RELEASE.2021-06-17T00-10-46Z
2021-06-16 17:50:32 -07:00
0d1d26a4ea
Simplify s3zip example ( #12500 )
...
There is no need to create a custom transport wrapper.
Just set the header in the options.
2021-06-14 11:04:14 -07:00
264ee97219
update to RELEASE.2021-06-14T01-29-23Z release
2021-06-13 20:58:56 -07:00
92e4c8be10
Update replication docs to fix mc reference ( #12490 )
...
Signed-off-by: Poorna Krishnamoorthy <poorna@minio.io>
2021-06-13 11:37:22 -07:00
f2a3872301
Update design.md for replication ( #12486 )
...
Fixes #12483
2021-06-10 16:05:39 -07:00
ba5fb2365c
feat: support of ZIP list/get/head as S3 extension ( #12267 )
...
When enabled, it is possible to list/get files
inside a zip file without uncompressing it.
Signed-off-by: Anis Elleuch <anis@min.io>
2021-06-10 08:17:03 -07:00
22c58be421
Update yaml files to latest version RELEASE.2021-06-09T18-51-39Z
2021-06-09 19:21:21 +00:00
dd5e9493f1
docs: add missing admin actions ( #12442 )
2021-06-09 10:37:20 -07:00
32017454ee
fix typo in Grafana dashboard json ( #12471 )
2021-06-09 08:04:12 -07:00
1bf80a6eea
Update yaml files to latest version RELEASE.2021-06-07T21-40-51Z
2021-06-07 22:13:41 +00:00
00c5d7e1b3
Add healing related metrics in official dashboard ( #12456 )
2021-06-07 12:46:54 -07:00
f199afcd6c
tiering: add aws role support for s3 ( #12424 )
...
Signed-off-by: Poorna Krishnamoorthy <poorna@minio.io>
2021-06-04 12:47:00 -07:00
dbea8d2ee0
Add support for existing object replication. ( #12109 )
...
Also adding an API to allow resyncing replication when
existing object replication is enabled and the remote target
is entirely lost. With the `mc replicate reset` command, the
objects that are eligible for replication as per the replication
config will be resynced to target if existing object replication
is enabled on the rule.
2021-06-01 19:59:11 -07:00
1f262daf6f
rename all remaining packages to internal/ ( #12418 )
...
This is to ensure that there are no projects
that try to import `minio/minio/pkg` into
their own repo. Any such common packages should
go to `https://github.com/minio/pkg `
2021-06-01 14:59:40 -07:00
3690de0c6b
Drop Pending size and count from replication metrics ( #12378 )
...
Real-time metrics calculated in-memory rely on the initial
replication metrics saved with data usage. However, this can
lag behind the actual state of the cluster at the time of server
restart leading to inaccurate Pending size/counts reported to
Prometheus. Dropping the Pending metrics as this can be more
reliably monitored by applications with replication notifications.
Signed-off-by: Poorna Krishnamoorthy <poorna@minio.io>
2021-05-31 20:26:52 -07:00
ab7410af11
docs: add hostname entry for docker stack deploy
2021-05-31 19:11:52 -07:00
5cd6253508
docs: mention docker stack deploy doesn't work with compose.yml
2021-05-31 18:59:32 -07:00
f27513ff28
remove all docker swarm references
2021-05-31 18:55:21 -07:00
4444ba13a4
support ldap:username for policy substitution ( #12390 )
...
LDAPusername is the simpler form of LDAPUser (userDN),
using a simpler version is convenient from policy
conditions point of view, since these are unique id's
used for LDAP login.
2021-05-28 10:33:07 -07:00
62b6615a2a
Add xl-meta data extraction ( #12396 )
...
* Add `-export` flag to export data.
* Support wildcard inputs.
2021-05-28 08:10:41 -07:00
ba75281d5a
Update yaml files to latest version RELEASE.2021-05-27T22-06-31Z
2021-05-27 22:35:50 +00:00
acc452b7ce
Add more erasure codes on degraded systems. ( #11852 )
...
In cases where a cluster is degraded, we do not uphold our consistency
guarantee and we will write fewer erasure codes and rely on healing
to recreate the missing shards.
In some cases replacing known bad disks in practice take days.
We want to change the behavior of a known degraded system to keep
the erasure code promise of the storage class for each object.
This will create the objects with the same confidence as a fully
functional cluster. The tradeoff will be that objects created
during a partial outage will take up slightly more space.
This means that when the storage class is EC:4, there should
always be written 4 parity shards, even if some disks are unavailable.
When an object is created on a set, the disks are immediately
checked. If any disks are unavailable additional parity shards
will be made for each offline disk, up to 50% of the number of disks.
We add an internal metadata field with the actual and intended
erasure code level, this can optionally be picked up later by
the scanner if we decide that data like this should be re-sharded.
2021-05-27 11:38:09 -07:00
9773b16f6f
update TLS docs to use new certgen tool
2021-05-26 10:41:29 -07:00
7b06b754a6
Update yaml files to latest version RELEASE.2021-05-26T00-22-46Z
2021-05-26 00:52:38 +00:00
2baabd455b
docs: fix per tenant limits docs formatting
2021-05-24 09:37:17 -07:00
41e9c6572f
fix: docs links use non-raw links for markdown
2021-05-22 10:52:47 -07:00
0da84a0c72
Update yaml files to latest version RELEASE.2021-05-22T02-34-39Z
2021-05-21 20:32:14 -07:00
df4914b6f3
fix: update docs, fix wording and links
2021-05-21 12:36:03 -07:00
783ea5eb5c
Update yaml files to latest version RELEASE.2021-05-20T22-31-44Z
2021-05-20 23:02:31 +00:00
a27339826c
Fix replication README.md docs ( #12330 )
...
Signed-off-by: Poorna Krishnamoorthy <poorna@minio.io>
2021-05-20 08:17:14 -07:00
bb7fbcdc09
fix: generating service accounts for group only LDAP accounts ( #12318 )
...
fixes #12315
2021-05-18 15:19:20 -07:00
267f12a2a1
Update to release RELEASE.2021-05-18T00-53-28Z
2021-05-17 18:24:38 -07:00
3d9873106d
feat: distributed setup can start now with default credentials ( #12303 )
...
In lieu of new changes coming for server command line, this
change is to deprecate strict requirement for distributed setups
to provide root credentials.
Bonus: remove MINIO_WORM warning from April 2020, it is time to
remove this warning.
2021-05-17 08:45:22 -07:00
ce3d9dc9fa
Update yaml files to latest version RELEASE.2021-05-16T05-32-34Z
2021-05-16 06:03:09 +00:00
7334247c98
update docs about NFS consistency model
2021-05-14 11:34:56 -07:00
951acf561c
Add support for syncing replica modifications ( #11104 )
...
when bidirectional replication is set up.
If ReplicaModifications is enabled in the replication
configuration, sync metadata updates to source if
replication rules are met. By default, if this
configuration is unset, MinIO automatically sync's
metadata updates on replica back to the source.
2021-05-13 19:20:45 -07:00
a592d3be19
fix the dashboard to use $rate_interval ( #12277 )
...
refer https://grafana.com/blog/2020/09/28/new-in-grafana-7.2-__rate_interval-for-prometheus-rate-queries-that-just-work/
for further information
2021-05-12 08:06:47 -07:00
f63eedb2b4
Update yaml files to latest version RELEASE.2021-05-11T23-27-41Z
2021-05-12 00:16:44 +00:00
477cd85bef
purge deprecate docker swarm documentation
...
documentation license moved to CC-BY-4.0
2021-05-10 09:50:06 -07:00
d09806c887
add docs LICENSE file
2021-05-07 09:32:51 -07:00
2fd9c13b50
rename minio-cluster to minio-job as per prometheus config
2021-05-06 12:39:58 -07:00
ddc1e4b5b3
Update Grafana dashboard to use the new v2 cluster metrics ( #12220 )
...
Fixes #11543
2021-05-06 14:44:03 +05:30
804a23a06d
update docs to remove _OLD credential references
...
also update the docs about config, IAM on encryption.
2021-05-04 10:27:51 -07:00
f7a87b30bf
Revert "deprecate embedded browser ( #12163 )"
...
This reverts commit 736d8cbac4
2021-04-30 08:50:39 -07:00
e5ec1325fc
docs: add QuickStart section to KMS encryption of IAM data ( #12190 )
...
This commit enhances the docs about IAM encryption.
It adds a quick-start section that explains how to
get started quickly with `MINIO_KMS_SECRET_KEY`
instead of setting up KES.
It also removes the startup message that gets printed
when the server migrates IAM data to plaintext.
We will point this out in the release notes.
Signed-off-by: Andreas Auernhammer <aead@mail.de>
2021-04-29 14:20:28 -07:00
736d8cbac4
deprecate embedded browser ( #12163 )
...
https://github.com/minio/console takes over the functionality for the
future object browser development
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-27 10:52:12 -07:00
f420996dfa
fix: allow parsing keys in both new and old format ( #12144 )
...
Bonus fix fallback to decrypt previously
encrypted content as well using older master
key ciphertext format.
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-24 19:05:25 -07:00
c829e3a13b
Support for remote tier management ( #12090 )
...
With this change, MinIO's ILM supports transitioning objects to a remote tier.
This change includes support for Azure Blob Storage, AWS S3 compatible object
storage incl. MinIO and Google Cloud Storage as remote tier storage backends.
Some new additions include:
- Admin APIs remote tier configuration management
- Simple journal to track remote objects to be 'collected'
This is used by object API handlers which 'mutate' object versions by
overwriting/replacing content (Put/CopyObject) or removing the version
itself (e.g DeleteObjectVersion).
- Rework of previous ILM transition to fit the new model
In the new model, a storage class (a.k.a remote tier) is defined by the
'remote' object storage type (one of s3, azure, GCS), bucket name and a
prefix.
* Fixed bugs, review comments, and more unit-tests
- Leverage inline small object feature
- Migrate legacy objects to the latest object format before transitioning
- Fix restore to particular version if specified
- Extend SharedDataDirCount to handle transitioned and restored objects
- Restore-object should accept version-id for version-suspended bucket (#12091 )
- Check if remote tier creds have sufficient permissions
- Bonus minor fixes to existing error messages
Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io>
Co-authored-by: Krishna Srinivas <krishna@minio.io>
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-23 11:58:53 -07:00
069432566f
update license change for MinIO
...
Signed-off-by: Harshavardhana <harsha@minio.io>
2021-04-23 11:58:53 -07:00
e05e14309c
Update yaml files to latest version RELEASE.2021-04-22T15-44-28Z
2021-04-22 16:32:11 +00:00
28f0ded1a4
Update replication design.md for sync mode ( #12100 )
2021-04-20 17:31:36 -07:00
49b7923309
Update yaml files to latest version RELEASE.2021-04-18T19-26-29Z
2021-04-18 20:15:32 +00:00
ca9b48b3b4
Update Replication README to point at new docs ( #12069 )
...
This is a minor change to call out the new documentation and warn
users to change their bookmarks. Once we are ready to set up
some redirects, we can remove this page from Gluegun TOC.
2021-04-15 16:32:44 -07:00
d50193d30f
xl-meta.go dump skip CRC check ( #12031 )
...
Skip CRC of metadata.
2021-04-09 17:38:23 -07:00
05a9108c24
Update etcd doc to clarify that any STS setup should work ( #12022 )
2021-04-08 17:30:17 -07:00
0e4794ea50
fix: allow S3 gateway passthrough for SSE-S3 header ( #12020 )
...
only in case of S3 gateway we have a case where we
need to allow for SSE-S3 headers as passthrough,
If SSE-C headers are passed then they are rejected
if KMS is not configured.
2021-04-08 16:40:38 -07:00
2899cc92b4
Update replication docs for required permission ( #12010 )
2021-04-07 15:56:02 -07:00
a772379dc5
Update yaml files to latest version RELEASE.2021-04-06T23-11-00Z
2021-04-06 17:59:45 -07:00
8a9d15ace2
update prometheus metrics with failed_count
2021-04-04 09:52:37 -07:00
47c09a1e6f
Various improvements in replication ( #11949 )
...
- collect real time replication metrics for prometheus.
- add pending_count, failed_count metric for total pending/failed replication operations.
- add API to get replication metrics
- add MRF worker to handle spill-over replication operations
- multiple issues found with replication
- fixes an issue when client sends a bucket
name with `/` at the end from SetRemoteTarget
API call make sure to trim the bucket name to
avoid any extra `/`.
- hold write locks in GetObjectNInfo during replication
to ensure that object version stack is not overwritten
while reading the content.
- add additional protection during WriteMetadata() to
ensure that we always write a valid FileInfo{} and avoid
ever writing empty FileInfo{} to the lowest layers.
Co-authored-by: Poorna Krishnamoorthy <poorna@minio.io>
Co-authored-by: Harshavardhana <harsha@minio.io>
2021-04-03 09:03:42 -07:00
2623338dc5
Inline small file data in xl.meta file ( #11758 )
2021-03-29 17:00:55 -07:00
91eb1fe2ef
Update yaml files to latest version RELEASE.2021-03-26T00-00-41Z
2021-03-26 00:23:59 +00:00
23b03dadb8
Add process uptime metric ( #11844 )
2021-03-20 21:23:27 -07:00
b379ca3bb0
Update yaml files to latest version RELEASE.2021-03-17T02-33-02Z
2021-03-17 02:56:28 +00:00
980311fdfd
Fix `STANDARD` defaults, point to new docs site. ( #11800 )
2021-03-16 12:04:28 -07:00
afd346417d
Update yaml files to latest version RELEASE.2021-03-12T00-00-47Z
2021-03-12 00:23:57 +00:00
9d118b372e
Update yaml files to latest version RELEASE.2021-03-10T05-11-33Z
2021-03-10 05:34:48 +00:00
464fa08f2e
Update yaml files to latest version RELEASE.2021-03-04T00-53-13Z
2021-03-04 01:15:49 +00:00
2c198ae7b6
fix: prometheus metrics disks_online count when disks are down ( #11689 )
...
prometheus metrics was using total disks instead
of online disk count, when disks were down, this
PR fixes this and also adds a new metric for
total_disk_count
2021-03-03 11:18:41 -08:00
bbd1244a88
Add support for mTLS for Audit log target ( #11645 )
2021-03-01 09:19:13 -08:00
e3f8830ab7
Update yaml files to latest version RELEASE.2021-03-01T04-20-55Z
2021-03-01 04:43:28 +00:00
c67d1bf120
add unauthenticated lookup-bind mode to LDAP identity ( #11655 )
...
Closes #11646
2021-02-28 12:57:31 -08:00
9171d6ef65
rename all references from crawl -> scanner ( #11621 )
2021-02-26 15:11:42 -08:00
b912e9ab41
Update yaml files to latest version RELEASE.2021-02-24T18-44-45Z
2021-02-24 19:08:36 +00:00
1b63291ee2
Update yaml files to latest version RELEASE.2021-02-23T20-05-01Z
2021-02-23 20:28:30 +00:00
02e7de6367
LDAP config: fix substitution variables ( #11586 )
...
- In username search filter and username format variables we support %s for
replacing with the username.
- In group search filter we support %s for username and %d for the full DN of
the username.
2021-02-22 13:20:36 -08:00
8cad407e0b
fix: Bring support for symlink on regular files on NAS ( #11383 )
...
fixes #11203
2021-02-20 00:30:12 -08:00
c70240b893
fix incorrect values in sizing guide ( #11583 )
2021-02-19 10:05:04 -08:00
8ba2136e06
Update yaml files to latest version RELEASE.2021-02-19T04-38-02Z
2021-02-18 21:02:25 -08:00
8e8a792d9d
Allow delete marker replication from replica ( #11566 )
...
in the case of active-active replication.
This PR also has the following changes:
- add docs on replication design
- fix corner case of completing versioned delete on a delete marker
when the target is down and `mc rm --vid` is performed repeatedly. Instead
the version should still be retained in the `PENDING|FAILED` state until
replication sync completes.
- remove `s3:Replication:OperationCompletedReplication` and
`s3:Replication:OperationFailedReplication` from ObjectCreated
events type
2021-02-18 00:33:51 -08:00
ffea6fcf09
fix: rename crawler as scanner in config ( #11549 )
2021-02-17 12:04:11 -08:00
cfc8b92dff
Update yaml files to latest version RELEASE.2021-02-14T04-01-33Z
2021-02-14 04:25:52 +00:00
0ef3e359d8
Update yaml files to latest version RELEASE.2021-02-11T08-23-43Z
2021-02-11 08:47:10 +00:00
466e95bb59
Return group DN instead of group name in LDAP STS ( #11501 )
...
- Additionally, check if the user or their groups has a policy attached during
the STS call.
- Remove the group name attribute configuration value.
2021-02-10 16:52:49 -08:00
876b79b8d8
read-health check endpoint returns success if cluster can serve read requests ( #11310 )
2021-02-09 01:00:44 -08:00
152d7cd95b
HDFS support keytab ( #11473 )
2021-02-07 17:29:47 -08:00
647a209c73
Update yaml files to latest version RELEASE.2021-02-07T01-31-02Z
2021-02-07 01:53:27 +00:00
9497dfd804
docs: add deprecation notice for federation
2021-02-04 17:18:37 -08:00
f2d49ec21a
Update ldap.md: add a link to ldap.go ( #11409 )
2021-02-02 15:47:04 -08:00
c4848f9b4f
Add process start time to cluster metrics. ( #11405 )
2021-02-01 23:02:18 -08:00
8c663f93f7
fix: typo in chinese docs ( #11401 )
2021-02-01 18:42:58 -08:00
b4cb7edf85
Update yaml files to latest version RELEASE.2021-02-01T22-56-52Z
2021-02-01 23:28:23 +00:00
33554651e9
crypto: deprecate native Hashicorp Vault support ( #11352 )
...
This commit deprecates the native Hashicorp Vault
support and removes the legacy Vault documentation.
The native Hashicorp Vault documentation is marked as
outdated and deprecated for over a year now. We give
another 6 months before we start removing Hashicorp Vault
support and show a deprecation warning when a MinIO server
starts with a native Vault configuration.
2021-01-29 17:55:37 -08:00
451d9057f3
Update yaml files to latest version RELEASE.2021-01-30T00-20-58Z
2021-01-30 00:45:11 +00:00
6717295e18
fix: rename audit log docs and datastructure
2021-01-26 13:39:55 -08:00
00cff1aac5
audit: per object send pool number, set number and servers per operation ( #11233 )
2021-01-26 13:21:51 -08:00
5f51ef0b40
Add LDAP Lookup-Bind mode ( #11318 )
...
This change allows the MinIO server to be configured with a special (read-only)
LDAP account to perform user DN lookups.
The following configuration parameters are added (along with corresponding
environment variables) to LDAP identity configuration (under `identity_ldap`):
- lookup_bind_dn / MINIO_IDENTITY_LDAP_LOOKUP_BIND_DN
- lookup_bind_password / MINIO_IDENTITY_LDAP_LOOKUP_BIND_PASSWORD
- user_dn_search_base_dn / MINIO_IDENTITY_LDAP_USER_DN_SEARCH_BASE_DN
- user_dn_search_filter / MINIO_IDENTITY_LDAP_USER_DN_SEARCH_FILTER
This lookup-bind account is a service account that is used to lookup the user's
DN from their username provided in the STS API. When configured, searching for
the user DN is enabled and configuration of the base DN and filter for search is
required. In this "lookup-bind" mode, the username format is not checked and must
not be specified. This feature is to support Active Directory setups where the
DN cannot be simply derived from the username.
When the lookup-bind is not configured, the old behavior is enabled: the minio
server performs LDAP lookups as the LDAP user making the STS API request and the
username format is checked and configuring it is required.
2021-01-25 14:26:10 -08:00
0bf2d84f96
update new metrics url docs ( #11342 )
2021-01-25 01:03:07 -08:00
7575c24037
Add open FD and FD limit to cluster metrics ( #11328 )
2021-01-22 18:30:16 -08:00
c080f04e66
fix: prometheus metrics link typo update to latest
2021-01-22 01:53:23 -08:00
b4add82bb6
Updated Prometheus metrics ( #11141 )
...
* Add metrics for nodes online and offline
* Add cluster capacity metrics
* Introduce v2 metrics
2021-01-18 20:35:38 -08:00
3163a660aa
Fix support for multiple LDAP user formats ( #11276 )
...
Fixes support for using multiple base DNs for user search in the LDAP directory
allowing users from different subtrees in the LDAP hierarchy to request
credentials.
- The username in the produced credentials is now the full DN of the LDAP user
to disambiguate users in different base DNs.
2021-01-17 21:54:32 -08:00
98f76008c7
fix: bucket lifecycle again to remove Days parameter
2021-01-17 01:50:56 -08:00
8da0b7cf03
fix: lifecycle documentation for DeleteMarker
2021-01-17 01:37:25 -08:00
f903cae6ff
Support variable server pools ( #11256 )
...
Current implementation requires server pools to have
same erasure stripe sizes, to facilitate same SLA
and expectations.
This PR allows server pools to be variadic, i.e they
do not have to be same erasure stripe sizes - instead
they should have SLA for parity ratio.
If the parity ratio cannot be guaranteed by the new
server pool, the deployment is rejected i.e server
pool expansion is not allowed.
2021-01-16 12:08:02 -08:00
40d59c1961
Update yaml files to latest version RELEASE.2021-01-16T02-19-44Z
2021-01-16 02:43:53 +00:00
7090bcc8e0
fix: doc links and delete replication permissions enforcement ( #11285 )
2021-01-15 15:22:55 -08:00
cc2d887e0e
fix: whitespace and formatting in replication docs
2021-01-14 22:58:53 -08:00
c1b4b24236
Update replication docs ( #11279 )
2021-01-15 10:22:57 +05:30
feaf8dfb9a
Fix replication status reported on completion ( #11273 )
...
Fixes : #11272
2021-01-13 11:52:28 -08:00
b97d53b29c
fix remote target healthcheck ( #11267 )
2021-01-12 20:48:04 -08:00
00af9881b0
LDAP doc fix: remove repeated paragraph and add emphasis ( #11266 )
2021-01-12 15:44:31 -08:00
7824e19d20
Allow synchronous replication if enabled. ( #11165 )
...
Synchronous replication can be enabled by setting the --sync
flag while adding a remote replication target.
This PR also adds proxying on GET/HEAD to another node in a
active-active replication setup in the event of a 404 on the current node.
2021-01-11 22:36:51 -08:00
e8176fe978
Use `-new` during OpenSSL certificate generation ( #11199 )
...
As per https://stackoverflow.com/a/3758443/8156177 , OpenSSL expects a certificate via STDIN.
`-new` will allow a new certificate to be generated instead.
2021-01-11 02:24:50 -08:00
d9224fbc65
Update yaml files to latest version RELEASE.2021-01-08T21-18-21Z
2021-01-08 21:37:35 +00:00
76e2713ffe
fix: use buffers only when necessary for io.Copy() ( #11229 )
...
Use separate sync.Pool for writes/reads
Avoid passing buffers for io.CopyBuffer()
if the writer or reader implement io.WriteTo or io.ReadFrom
respectively then its useless for sync.Pool to allocate
buffers on its own since that will be completely ignored
by the io.CopyBuffer Go implementation.
Improve this wherever we see this to be optimal.
This allows us to be more efficient on memory usage.
```
385 // copyBuffer is the actual implementation of Copy and CopyBuffer.
386 // if buf is nil, one is allocated.
387 func copyBuffer(dst Writer, src Reader, buf []byte) (written int64, err error) {
388 // If the reader has a WriteTo method, use it to do the copy.
389 // Avoids an allocation and a copy.
390 if wt, ok := src.(WriterTo); ok {
391 return wt.WriteTo(dst)
392 }
393 // Similarly, if the writer has a ReadFrom method, use it to do the copy.
394 if rt, ok := dst.(ReaderFrom); ok {
395 return rt.ReadFrom(src)
396 }
```
From readahead package
```
// WriteTo writes data to w until there's no more data to write or when an error occurs.
// The return value n is the number of bytes written.
// Any error encountered during the write is also returned.
func (a *reader) WriteTo(w io.Writer) (n int64, err error) {
if a.err != nil {
return 0, a.err
}
n = 0
for {
err = a.fill()
if err != nil {
return n, err
}
n2, err := w.Write(a.cur.buffer())
a.cur.inc(n2)
n += int64(n2)
if err != nil {
return n, err
}
```
2021-01-06 09:36:55 -08:00
b5d291ea88
fix: rename remaining zone -> pool ( #11231 )
2021-01-06 09:35:47 -08:00
eb9172eecb
Allow Compression + encryption ( #11103 )
2021-01-05 20:08:35 -08:00
cb0eaeaad8
feat: migrate to ROOT_USER/PASSWORD from ACCESS/SECRET_KEY ( #11185 )
2021-01-05 10:22:57 -08:00
f3f0041ad0
Update yaml files to latest version RELEASE.2021-01-05T05-22-38Z
2021-01-05 05:42:54 +00:00
c1f6ca6697
Fix caddy project url ( #11198 )
2020-12-31 09:44:07 -08:00
cde801282d
chore: enabled syntax highlighting in docs ( #11182 )
2020-12-29 17:38:28 -08:00
6cf0008469
fix: docs typo in object lock docs ( #11181 )
2020-12-29 16:14:10 -08:00
7b0330a98c
Update yaml files to latest version RELEASE.2020-12-29T23-29-29Z
2020-12-29 23:46:24 +00:00
d3c853a3be
Update yaml files to latest version RELEASE.2020-12-26T01-35-54Z
2020-12-26 01:53:30 +00:00
7b8a456f68
Update lifecycle README docs ( #11160 )
...
Removing reference to transition feature in docs as this
feature is being revamped to provide better extensibility
across different cloud targets.
2020-12-23 19:56:55 -08:00
b43906f6ee
fix: docs typos and keywords
2020-12-23 11:59:20 -08:00
bfb92a27b7
Update yaml files to latest version RELEASE.2020-12-23T02-24-12Z
2020-12-23 02:43:25 +00:00
386dd56856
Update yaml files to latest version RELEASE.2020-12-18T03-27-42Z
2020-12-18 03:45:13 +00:00
26624552be
Update yaml files to latest version RELEASE.2020-12-16T05-05-17Z
2020-12-16 05:23:19 +00:00
31bf6f0c25
Update yaml files to latest version RELEASE.2020-12-12T08-39-07Z
2020-12-12 08:56:30 +00:00
14792cdbc6
docs: fix the metrics formatting ( #11081 )
2020-12-10 18:15:47 -08:00
97856bfebf
fix: grafana double counting for bucket usage, histrogram and objects ( #11070 )
2020-12-09 20:30:37 -08:00
83a67a1d21
Update yaml files to latest version RELEASE.2020-12-10T01-54-29Z
2020-12-10 02:12:01 +00:00
54d243cd98
fix: grafana dashboard calculating online nodes ( #11041 )
...
Also use a generic name instead of diff names per revision
2020-12-09 00:26:42 -08:00
04848dfa1c
Add documentation for bucket replication related metrics ( #11055 )
2020-12-08 12:48:10 -08:00
4a564336fe
Revert "Add metrics for nodes online and offline ( #11050 )"
...
This reverts commit f60bbdf86b
2020-12-08 09:23:35 -08:00
f60bbdf86b
Add metrics for nodes online and offline ( #11050 )
2020-12-08 01:06:27 -08:00
8c79f87f02
add dynamic config docs ( #11048 )
...
Co-authored-by: Eco <41090896+eco-minio@users.noreply.github.com>
2020-12-07 19:02:20 -08:00
f3beb1236a
Add cache usage, total capacity to prometheus metrics ( #11026 )
2020-12-07 16:35:11 -08:00
934bed47fa
Add transition event notification ( #11047 )
...
This is a MinIO specific extension to allow monitoring of transition events.
2020-12-07 13:53:28 -08:00
6ff12f5f01
Add the dashboard json file ( #11028 )
...
This will allow users to contribute to the dashboard as needed.
2020-12-04 16:27:41 -08:00
de9b64834e
fix: update grafana dashboard docs ( #11023 )
...
Refer to the official Grafana dashboard
2020-12-03 15:56:15 -08:00
d54cf77356
Update yaml files to latest version RELEASE.2020-12-03T05-49-24Z
2020-12-03 06:05:56 +00:00
1ab4d6a6aa
Update yaml files to latest version RELEASE.2020-12-03T00-03-10Z
2020-12-03 00:24:16 +00:00
fe11e9047d
deprecate CommonName from TLS docs ( #11017 )
...
CommonName is not supported anymore in Go 1.15
fix the TLS documentation to use subjAltNames
2020-12-02 10:18:39 -08:00
3ca046b408
Added set keyword in the command set to enable encryption on buckets ( #11010 )
2020-12-01 16:00:49 -08:00
4ec45753e6
rename server sets to server pools
2020-12-01 13:50:33 -08:00
790833f3b2
Revert "Support variable server sets ( #10314 )"
...
This reverts commit aabf053d2f
2020-12-01 12:02:29 -08:00
aabf053d2f
Support variable server sets ( #10314 )
2020-11-25 16:28:47 -08:00
f839bb5a0a
Update yaml files to latest version RELEASE.2020-11-25T22-36-25Z
2020-11-25 22:53:02 +00:00
75a8e81f8f
azure: Specify different Azure storage in the shell env ( #10943 )
...
AZURE_STORAGE_ACCOUNT and AZURE_STORAGE_KEY are used in
azure CLI to specify the azure blob storage access & secret keys. With this commit,
it is possible to set them if you want the gateway's own credentials to be
different from the Azure blob credentials.
Co-authored-by: Harshavardhana <harsha@minio.io>
2020-11-23 16:45:56 -08:00
39f3d5493b
Show Delete replication status header ( #10946 )
...
X-Minio-Replication-Delete-Status header shows the
status of the replication of a permanent delete of a version.
All GETs are disallowed and return 405 on this object version.
In the case of replicating delete markers.
X-Minio-Replication-DeleteMarker-Status shows the status
of replication, and would similarly return 405.
Additionally, this PR adds reporting of delete marker event completion
and updates documentation
2020-11-21 23:48:50 -08:00
7bc47a14cc
Rename OBD to Health ( #10842 )
...
Also, Remove thread stats and openfds from the health report
as we already have process stats and numfds
2020-11-20 12:52:53 -08:00
9263be8cca
docs: fix missing event types in notifications ( #10944 )
2020-11-20 11:27:27 -08:00
1ebf6f146a
Add support for ILM transition ( #10565 )
...
This PR adds transition support for ILM
to transition data to another MinIO target
represented by a storage class ARN. Subsequent
GET or HEAD for that object will be streamed from
the transition tier. If PostRestoreObject API is
invoked, the transitioned object can be restored for
duration specified to the source cluster.
2020-11-19 18:47:17 -08:00
9a34fd5c4a
Revert "Revert "Add delete marker replication support ( #10396 )""
...
This reverts commit 267d7bf0a9
2020-11-19 18:43:58 -08:00
b9e3a8b5ac
Update yaml files to latest version RELEASE.2020-11-19T23-48-16Z
2020-11-20 00:06:03 +00:00
0784a0c33a
Update yaml files to latest version RELEASE.2020-11-13T20-10-18Z
2020-11-13 20:27:01 +00:00
3595cb1267
Update yaml files to latest version RELEASE.2020-11-12T22-33-34Z
2020-11-12 22:51:11 +00:00
07859ef48b
docs: clarify notifications support for gateways ( #10729 )
...
Co-authored-by: Harshavardhana <harsha@minio.io>
2020-11-12 12:19:04 -08:00
267d7bf0a9
Revert "Add delete marker replication support ( #10396 )"
...
This reverts commit 50c10a5087
2020-11-12 11:43:14 -08:00
2c1e37197b
fix: bad example json for policy in replication docs ( #10869 )
2020-11-10 17:49:49 -08:00
50c10a5087
Add delete marker replication support ( #10396 )
...
Delete marker replication is implemented for V2
configuration specified in AWS spec (though AWS
allows it only in the V1 configuration).
This PR also brings in a MinIO only extension of
replicating permanent deletes, i.e. deletes specifying
version id are replicated to target cluster.
2020-11-10 15:24:14 -08:00
9f4ad873bc
Update yaml files to latest version RELEASE.2020-11-10T21-02-24Z
2020-11-10 21:18:35 +00:00
8e8ddf7233
doc: Add definition of 1KB and 1MB in prometheus ( #10857 )
2020-11-09 10:05:01 -08:00
21016265e5
Update yaml files to latest version RELEASE.2020-11-06T23-17-07Z
2020-11-06 23:34:19 +00:00
51222cc664
Update yaml files to latest version RELEASE.2020-10-28T08-16-50Z
2020-10-28 08:33:22 +00:00
ca77ee1c0e
Update yaml files to latest version RELEASE.2020-10-27T04-03-55Z
2020-10-27 04:20:10 +00:00
4442382c16
update STS examples to use latest v7 APIs
2020-10-24 11:16:59 -07:00
6fd088f448
docs: Update ILM doc with versioning features ( #10714 )
2020-10-20 09:23:27 -07:00
ba5215561f
Update yaml files to latest version RELEASE.2020-10-18T21-54-12Z
2020-10-18 22:15:30 +00:00
4eb45c9a0f
fix: dex getting started guide URL ( #10701 )
2020-10-17 16:47:28 -07:00
ad726b49b4
rename zones to serverSets to avoid terminology conflict ( #10679 )
...
we are bringing in availability zones, we should avoid
zones as per server expansion concept.
2020-10-15 14:28:50 -07:00
9c042a503b
remove deprecate readiness from healthcheck docs ( #10659 )
2020-10-12 18:56:03 -07:00
4a678ad70f
Update yaml files to latest version RELEASE.2020-10-12T21-53-21Z
2020-10-12 22:15:06 +00:00
071c004f8b
Update yaml files to latest version RELEASE.2020-10-09T22-55-05Z
2020-10-09 23:16:30 +00:00
1738eb24b1
fix: caching doc README.md missing high watermark ( #10646 )
2020-10-09 08:32:51 -07:00
01498a3e34
fix: add docs for new event types in notification ( #10636 )
2020-10-06 13:33:23 -07:00
57f0176759
Update KES table to include additional supported KMS providers ( #10631 )
2020-10-06 11:09:43 -07:00
017954e7ea
Update yaml files to latest version RELEASE.2020-10-03T02-19-42Z
2020-10-03 02:36:54 +00:00
511424a287
Update yaml files to latest version RELEASE.2020-09-26T03-44-56Z
2020-09-26 04:02:42 +00:00
b880796aef
Set the maximum open connections limit in PG and MySQL target configs ( #10558 )
...
As the bulk/recursive delete will require multiple connections to open at an instance,
The default open connections limit will be reached which results in the following error
```FATAL: sorry, too many clients already```
By setting the open connections to a reasonable value - `2`, We ensure that the max open connections
will not be exhausted and lie under bounds.
The queries are simple inserts/updates/deletes which is operational and sufficient with the
the maximum open connection limit is 2.
Fixes #10553
Allow user configuration for MaxOpenConnections
2020-09-24 22:20:30 -07:00
e6ab4db6b8
Fix minimum replication workers started ( #10560 )
...
This PR also fixes GetReplicationConfiguration permission
in web-handlers.go to use bucket as resource
2020-09-24 12:25:41 -07:00
df08fd1f03
Update Kubernetes Helm Readme.md ( #10559 )
...
The link to the Minio Helm Chart repository was wrong.
2020-09-24 08:38:46 -07:00
ac82f416a4
Update yaml files to latest version RELEASE.2020-09-23T19-18-30Z
2020-09-23 19:38:14 +00:00
5fdf47b118
Add Chinese documentation for distributed design ( #10549 )
2020-09-23 09:26:47 -07:00
8b74a72b21
fix: rename READY deadline to CLUSTER deadline ENV ( #10535 )
2020-09-23 09:14:33 -07:00
a4006e23a0
Update replication docs to clarify permissions ( #10536 )
...
Co-authored-by: Klaus Post <klauspost@gmail.com>
2020-09-22 11:58:04 -07:00
d73c4f09f3
Update yaml files to latest version RELEASE.2020-09-21T22-31-59Z
2020-09-21 22:58:40 +00:00
dd4a2d7419
update distributed zh_CN document. ( #10531 )
2020-09-21 10:54:56 -07:00
3168e93730
fix typo in healthcheck README.md ( #10518 )
2020-09-18 09:52:37 -07:00
6f45e303f5
fix tls flag ( #10487 )
2020-09-18 15:35:54 +05:30
dff37aa33d
update disk caching zh_CN document. ( #10472 )
...
add disk caching DESIGN zh_CN document.
2020-09-18 00:07:48 -07:00
48d2c03250
Update yaml files to latest version RELEASE.2020-09-17T04-49-20Z
2020-09-17 05:07:58 +00:00
b2536476c9
Update yaml files to latest version RELEASE.2020-09-16T04-22-35Z
2020-09-16 04:41:00 +00:00
9d95937018
update KMS docs indicating deprecation of AUTO_ENCRYPTION env
2020-09-13 16:23:28 -07:00
eb3ded420e
Update yaml files to latest version RELEASE.2020-09-10T22-02-45Z
2020-09-10 22:22:28 +00:00
a966ccd17d
Support nginx LB with docker-compose example ( #10434 )
2020-09-10 09:19:17 -07:00
1dce6918c2
fix sysctl.sh quotes which are incompatible with sysctl ( #10446 )
2020-09-09 17:29:23 -07:00
6019628f7d
Update yaml files to latest version RELEASE.2020-09-08T23-05-18Z
2020-09-08 23:25:40 +00:00
a694ba93d9
update kernel tuning zh_CN document ( #10433 )
2020-09-07 19:21:52 -07:00
a9aaea0d67
Update yaml files to latest version RELEASE.2020-09-05T07-14-49Z
2020-09-05 07:32:10 +00:00
746f1585eb
add debugging zh_CN document. ( #10401 )
2020-09-04 13:03:46 -07:00
b4e3956e69
update KES docs to talk about 'mc encrypt' command ( #10400 )
...
add a deprecation notice for KMS_AUTO_ENCRYPTION
2020-09-03 12:43:45 -07:00
c51229493b
azure docs remove default chunk size example ( #10406 )
2020-09-03 01:31:56 -07:00
650dccfa9e
cache: Only start at high watermark ( #10403 )
...
Currently, cache purges are triggered as soon as the low watermark is exceeded.
To reduce IO this should only be done when reaching the high watermark.
This simplifies checks and reduces all calls for a GC to go through
`dcache.diskSpaceAvailable(size)`. While a comment claims that
`dcache.triggerGC <- struct{}{}` was non-blocking I don't see how
that was possible. Instead, we add a 1 size to the queue channel
and use channel semantics to avoid blocking when a GC has
already been requested.
`bytesToClear` now takes the high watermark into account to it will
not request any bytes to be cleared until that is reached.
2020-09-02 17:48:44 -07:00
d08b4b147d
Update yaml files to latest version RELEASE.2020-09-02T18-19-50Z
2020-09-02 18:40:57 +00:00
4b6585d249
support 'ldap:user' variable replacement properly ( #10391 )
...
also update `ldap.go` examples with latest
minio-go changes
Fixes #10367
2020-09-01 12:26:22 +05:30
cb61e50b51
since its an internal RPC API we do not need to document it. ( #10382 )
2020-08-31 10:20:07 -07:00
cd380251b3
Support custom paths for secret files in docker-entrypoint.sh ( #10344 )
2020-08-28 14:04:29 -07:00
92cd1eed45
Clarify zone example ( #10374 )
2020-08-28 14:03:29 -07:00
db32a24cb6
update kernel tuning docs to indicate sane values
2020-08-28 12:07:55 -07:00
73a6b4ea11
fix typo in replication docs ( #10366 )
2020-08-27 12:54:23 -07:00
c1b88c17cc
Update yaml files to latest version RELEASE.2020-08-27T05-16-20Z
2020-08-27 05:36:01 +00:00
5a974fb10c
Update yaml files to latest version RELEASE.2020-08-26T00-00-49Z
2020-08-26 00:19:30 +00:00
7d50a0cfea
Update yaml files to latest version RELEASE.2020-08-25T00-21-20Z
2020-08-25 00:44:39 +00:00
2a8e40f19f
add chroot zh_CN document ( #10337 )
2020-08-24 13:58:19 -07:00
21e8440423
add compression zh_CN document ( #10330 )
2020-08-23 22:07:36 -07:00
d0c910a6f3
Support https and basic-auth for elasticsearch notification target ( #10332 )
2020-08-23 09:43:48 -07:00
0ff246653b
CN bucket policy document that is no longer needed can be removed, ( #10310 )
...
which is consistent with the English document.
2020-08-22 11:05:21 -07:00
113bcbdb78
update minio limit zh_CN document ( #10320 )
2020-08-22 11:04:31 -07:00
331c517a5b
translate bucket versioning design document to zh_CN ( #10300 )
2020-08-20 13:18:51 -07:00
2eb5f934d8
add bucket versioning zh_CN document ( #10281 )
2020-08-18 13:10:26 -07:00
b43d376a87
Update yaml files to latest version RELEASE.2020-08-18T19-41-00Z
2020-08-18 20:04:22 +00:00
adca28801d
feat: disable Parquet by default (breaking change) ( #9920 )
...
I have built a fuzz test and it crashes heavily in seconds and will OOM shortly after.
It seems like supporting Parquet is basically a completely open way to crash the
server if you can upload a file and run s3 select on it.
Until Parquet is more hardened it is DISABLED by default since hostile
crafted input can easily crash the server.
If you are in a controlled environment where it is safe to assume no hostile
content can be uploaded to your cluster you can safely enable Parquet.
To enable Parquet set the environment variable `MINIO_API_SELECT_PARQUET=on`
while starting the MinIO server.
Furthermore, we guard parquet by recover functions.
2020-08-18 10:23:28 -07:00
ede86845e5
docs: Add policy variables for resource and conditions ( #10278 )
...
Bonus fix adds LDAP policy variable and clarifies the
usage of policy variables for temporary credentials.
fixes #10197
2020-08-17 17:39:55 -07:00
670724184c
Update yaml files to latest version RELEASE.2020-08-16T18-39-38Z
2020-08-16 18:56:41 +00:00
01a2ccc52f
add bucket retention zh_CN document ( #10264 )
2020-08-14 22:12:03 -07:00
1c865dd119
Update yaml files to latest version RELEASE.2020-08-13T02-39-50Z
2020-08-13 02:57:25 +00:00
6914b2c99d
Add bucket replication zh_CN document ( #10243 )
2020-08-11 11:33:01 -07:00
b4b7a18497
add bucket quota zh_CN document ( #10232 )
2020-08-10 10:55:34 -07:00
a49e3647b6
add bucket lifecycle zh_CN document ( #10231 )
2020-08-09 02:29:25 -07:00
954e17c3d0
Update yaml files to latest version RELEASE.2020-08-08T04-50-06Z
2020-08-07 22:32:27 -07:00
8049184dcc
fix: documentation changes in replication docs ( #10209 )
2020-08-07 13:30:52 -07:00
19c4f3082b
update and improve bucket notifications zh_CN document ( #10224 )
2020-08-07 12:36:27 -07:00
6d64aab420
Update yaml files to latest version RELEASE.2020-08-07T01-23-07Z
2020-08-07 01:43:14 +00:00
adcaa6f9de
fix: Change ListBucketTargets handler ( #10217 )
...
to list all targets across a tenant.
Also fixing some validations.
2020-08-06 17:10:21 -07:00
ce129efa09
update and improve config,throttle zh_CN document ( #10214 )
2020-08-06 11:43:30 -07:00
121164db56
fix: relax some replication validations ( #10210 )
...
Also inherit storage class from source object
if replication configuration does not have a storage
class specified for destination bucket.
2020-08-05 20:01:20 -07:00
195f95196e
Update yaml files to latest version RELEASE.2020-08-05T21-34-13Z
2020-08-05 21:58:17 +00:00
3acc0ebb81
fix: Change service name in Arn for replication ( #10205 )
2020-08-05 00:43:18 -07:00
1cdaced8b6
Update yaml files to latest version RELEASE.2020-08-04T23-10-51Z
2020-08-04 23:46:06 +00:00
d90ab904e7
fix: multi-tenant deployment guide to use ellipses format ( #10192 )
2020-08-04 08:15:37 -07:00
0cf4539fe8
fix: update and improve zh_CN documentation ( #10174 )
2020-08-01 11:52:46 -07:00
2174a22835
Update yaml files to latest version RELEASE.2020-07-31T03-39-05Z
2020-07-31 03:58:04 +00:00
a8dd7b3eda
Refactor replication target management. ( #10154 )
...
Generalize replication target management so
that remote targets for a bucket can be
managed with ARNs. `mc admin bucket remote`
command will be used to manage targets.
2020-07-30 19:55:22 -07:00
27266f8a54
fix: if OPA set do not enforce policy claim ( #10149 )
2020-07-28 11:47:57 -07:00
98691f75bc
Update yaml files to latest version RELEASE.2020-07-27T18-37-02Z
2020-07-27 18:57:10 +00:00
416ec316bd
Add clarification for minimum drives per server ( #10130 )
2020-07-24 22:09:01 -07:00
8a23988711
Update yaml files to latest version RELEASE.2020-07-24T22-43-05Z
2020-07-24 23:00:32 +00:00
0b5d1bc91d
fix: bucket replication docs ( #10104 )
...
* fix: bucket replication docs
* Update docs/bucket/replication/README.md
Co-authored-by: kannappanr <30541348+kannappanr@users.noreply.github.com>
Co-authored-by: kannappanr <30541348+kannappanr@users.noreply.github.com>
2020-07-21 22:19:30 -07:00
c43da3005a
Add support for server side bucket replication ( #9882 )
2020-07-21 17:49:56 -07:00
ca4c15bc63
Update yaml files to latest version RELEASE.2020-07-22T00-26-33Z
2020-07-22 00:44:03 +00:00
ec06089eda
fix: re-implement cluster healthcheck ( #10101 )
2020-07-20 18:31:22 -07:00
38f60b3c1d
Update yaml files to latest version RELEASE.2020-07-20T02-25-16Z
2020-07-20 02:41:52 +00:00
30104cb12b
docs: fix veeam document formatting
2020-07-18 18:38:12 -07:00
68aaa5bbc3
Update yaml files to latest version RELEASE.2020-07-18T18-48-16Z
2020-07-18 19:04:01 +00:00
5e8392c8ef
Update Veeam integration doc with immutability references ( #10067 )
2020-07-16 17:16:53 -07:00
07eb24b775
add absolute path for images ( #10056 )
2020-07-16 00:06:14 -07:00
cd849bc2ff
update STS docs with new values ( #10055 )
...
Co-authored-by: Poorna <poornas@users.noreply.github.com>
2020-07-15 14:36:14 -07:00
ec91fa55db
docs: Add more STS docs with dex and python example ( #10047 )
2020-07-15 17:25:55 +05:30
a97ce3c96e
Update yaml files to latest version RELEASE.2020-07-14T19-14-30Z
2020-07-14 19:32:27 +00:00
Harshavardhana
Minio Trusted
Aditya Manthramurthy
Yoann Guillerme
Krishnan Parthasarathi
fpaupier
Klaus Post
Ravind Kumar
Poorna K
chrisbecke
Mani
Ashish Kumar Sinha
jandres - moscardo
soudhaf
Andreas Auernhammer
Nitish Tiwari
Ricardo Katz
Matt Sarrel
Ali Malek
Christian Weiske
iternity-dotcom
Anis Elleuch
Andreas Auernhammer
Ritesh H Shukla
cbows
WangYuMu
Krishna Srinivas
Sarasa Kisaragi
Bahram Aghaei
swartz-k
George Tsatsis
Baptiste Mille-Mathias
0xflotus
Sudarshan (Sid)
Shireesh Anjal
Omar Alvarez
Anton Melser
Márk Sági-Kazár
miraculli
Praveen raj Mani
Espen Finnesand
飞雪无情
Derek Bender
Aleksey Pogibelev
Li Yi
unlimitedbits
Eco