Commit Graph

29 Commits

Author SHA1 Message Date
Harshavardhana adf9a9d300 Remove all unused variables and functions (#5823) 2018-04-15 19:26:04 +05:30
Andreas Auernhammer 2f20d90465 set storage class in FS mode (#5779)
This change sets the storage class of the object-info if a storage
class was specified during PUT. The server now replies with the
storage class which was set during uploading the object in FS mode.

Fixes #5777
2018-04-06 20:50:02 +05:30
kannappanr f8a3fd0c2a
Create logger package and rename errorIf to LogIf (#5678)
Removing message from error logging
Replace errors.Trace with LogIf
2018-04-05 15:04:40 -07:00
poornas 76d1e8bbcd change fs.json format to include checksum fields (#5685) 2018-03-27 17:23:10 -07:00
Anis Elleuch 120b061966 Add multipart support in SSE-C encryption (#5576)
*) Add Put/Get support of multipart in encryption
*) Add GET Range support for encryption
*) Add CopyPart encrypted support
*) Support decrypting of large single PUT object
2018-03-01 11:37:57 -08:00
Krishna Srinivas 047b7aff0c Seek to offset 0 after Truncate() (#5375) 2018-02-06 15:37:48 -08:00
Krishna Srinivas 3b2486ebaf Lock free multipart backend implementation for FS (#5401) 2018-01-31 13:17:24 -08:00
Krishna Srinivas 7c72d14027 Separate the codebase for XL and FS format.json related code (#5317) 2018-01-08 14:30:55 -08:00
Nitish Tiwari 1e5fb4b79a
Fix storage class related issues (#5338)
- Update startup banner to print storage class in capitals. This
makes it easier to identify different storage classes available.

- Update response metadata to not send STANDARD storage class.
This is in accordance with AWS S3 behaviour.

- Update minio-go library to bring in storage class related
changes. This is needed to make transparent translation of
storage class headers for Minio S3 Gateway.
2018-01-04 11:44:45 +05:30
Harshavardhana 8efa82126b
Convert errors tracer into a separate package (#5221) 2017-11-25 11:58:29 -08:00
Harshavardhana a4d6195244 Add public data-types for easier external loading (#5170)
This change brings public data-types such that
we can ask projects to implement gateway projects
externally than maintaining in our repo.

All publicly exported structs are maintained in object-api-datatypes.go

completePart --> CompletePart
uploadMetadata --> MultipartInfo

All other exported errors are at object-api-errors.go
2017-11-14 13:55:10 +05:30
Harshavardhana b2cbade477 Support creating empty directories. (#5049)
Every so often we get requirements for creating
directories/prefixes and we end up rejecting
such requirements. This PR implements this and
allows empty directories without any new file
addition to backend.

Existing lower APIs themselves are leveraged to provide
this behavior. Only FS backend supports this for
the time being as desired.
2017-10-16 17:20:54 -07:00
Harshavardhana d864e00e24 posix: Deprecate custom removeAll/mkdirAll implementations. (#4808)
Since go1.8 os.RemoveAll and os.MkdirAll both support long
path names i.e UNC path on windows. The code we are carrying
was directly borrowed from `pkg/os` package and doesn't need
to be in our repo anymore. As a side affect this also
addresses our codecoverage issue.

Refer #4658
2017-08-12 19:25:43 -07:00
Harshavardhana 353f2d3a6e fs: Hold `format.json` readLock ref to avoid GC. (#4532)
Looks like if we follow pattern such as

```
_ = rlk
```

Go can potentially kick in GC and close the fd when
the reference is lost, only speculation is that
the cause here is `SetFinalizer` which is set on
`os.close()` internally in `os` stdlib.

This is unexpected and unsual endeavour for Go, but
we have to make sure the reference is never lost
and always dies with the server.

Fixes #4530
2017-06-13 08:29:07 -07:00
Harshavardhana 075b8903d7 fs: Add safe locking semantics for `format.json` (#4523)
This patch also reverts previous changes which were
merged for migration to the newer disk format. We will
be bringing these changes in subsequent releases. But
we wish to add protection in this release such that
future release migrations are protected.

Revert "fs: Migration should handle bucketConfigs as regular objects. (#4482)"
This reverts commit 976870a391.

Revert "fs: Migrate object metadata to objects directory. (#4195)"
This reverts commit 76f4f20609.
2017-06-12 17:40:28 -07:00
Harshavardhana 155a90403a fs/erasure: Rename meta 'md5Sum' as 'etag'. (#4319)
This PR also does backend format change to 1.0.1
from 1.0.0.  Backward compatible changes are still
kept to read the 'md5Sum' key. But all new objects
will be stored with the same details under 'etag'.

Fixes #4312
2017-05-14 12:05:51 -07:00
Harshavardhana 76f4f20609 fs: Migrate object metadata to objects directory. (#4195)
Fixes #3352
2017-05-05 08:49:09 -07:00
Harshavardhana 4747adfcb4 fs: Enable returning ETag along with ListObjects() (#4042)
This is to comply with S3 behavior, we previously removed
reading `fs.json` for optimization reasons but we have a
reason to believe that providing ETag and using gjson
provides needed benefit of not having to deal with
unmarshalling overhead of golang stdlib.

Fixes #4028
2017-04-04 09:14:03 -07:00
Harshavardhana 47ac410ab0 Code cleanup - simplify server side code. (#3870)
Fix all the issues reported by `gosimple` tool.
2017-03-08 10:00:47 -08:00
Krishna Srinivas 82373e3d50 fs: cleanup - do not cache size of metafiles (#3630)
* Remove Size() method and size field from lock.LockedFile
* WriteTo method of fsMeta and uploadsV1 now takes concrete type *lock.LockedFile
2017-01-25 12:29:06 -08:00
Harshavardhana 62f8343879 Add constants for commonly used values. (#3588)
This is a consolidation effort, avoiding usage
of naked strings in codebase. Whenever possible
use constants which can be repurposed elsewhere.

This also fixes `goconst ./...` reported issues.
2017-01-18 12:24:34 -08:00
Harshavardhana 1c699d8d3f fs: Re-implement object layer to remember the fd (#3509)
This patch re-writes FS backend to support shared backend sharing locks for safe concurrent access across multiple servers.
2017-01-16 17:05:00 -08:00
Bala FA 1d4ac4b084 Rename getUUID() into mustGetUUID() (#3320)
In case of UUID generation failure mustGetUUID() will panic than
infinitely trying in for loop.
2016-11-22 16:52:37 -08:00
Anis Elleuch ffbee70e04 Avoid removing 'tmp' directory inside '.minio.sys' (#3294) 2016-11-20 14:25:43 -08:00
Harshavardhana 716316f711 Reduce number of envs and options from command line. (#3230)
Ref #3229

After review with @abperiasamy we decided to remove all the unnecessary options

- MINIO_BROWSER (Implemented as a security feature but now deemed obsolete
  since even if blocking access to MINIO_BROWSER, s3 API port is open)
- MINIO_CACHE_EXPIRY (Defaults to 72h)
- MINIO_MAXCONN (No one used this option and we don't test this)
- MINIO_ENABLE_FSMETA (Enable FSMETA all the time)

Remove --ignore-disks option - this option was implemented when XL layer
 would initialize the backend disks and heal them automatically to disallow
 XL accidentally using the root partition itself this option was introduced.

This behavior has been changed XL no longer automatically initializes
`format.json`  a HEAL is controlled activity, so ignore-disks is not
useful anymore. This change also addresses the problems of our documentation
going forward and keeps things simple. This patch brings in reduction of
options and defaulting them to a valid known inputs.  This patch also
serves as a guideline of limiting many ways to do the same thing.
2016-11-11 16:40:55 -08:00
Harshavardhana 6494b77d41 server: Add more elaborate startup messages. (#2731)
These messages based on our prep stage during XL
and prints more informative message regarding
drive information.

This change also does a much needed refactoring.
2016-10-05 12:48:07 -07:00
Krishna Srinivas 9358ee011b logging: Print stack trace in case of errors.
fixes #1827
2016-09-13 21:18:30 -07:00
Krishna Srinivas 3aa0574c6b FS/multipart: Append the parts to a file as and when the parts arrive. (#2513) 2016-08-31 13:42:57 -07:00
Harshavardhana bccf549463 server: Move all the top level files into cmd folder. (#2490)
This change brings a change which was done for the 'mc'
package to allow for clean repo and have a cleaner
github drop in experience.
2016-08-18 16:23:42 -07:00