3 Commits

Author	SHA1	Message	Date
Harshavardhana	97376f6e8f	improve performance for inlined data (#15603) ... inlined data often is bigger than the allowed O_DIRECT alignment, so potentially we can write 'xl.meta' without O_DSYNC instead we can rely on O_DIRECT + fdatasync() instead. This PR allows O_DIRECT on inlined data that would gain the benefits of performing O_DIRECT, eventually performing an fdatasync() at the end. Performance boost can be observed here for small objects < 128KiB. The performance boost is mainly seen on HDD, and marginal on NVMe setups.	2022-08-29 11:19:29 -07:00
Klaus Post	974073a2e5	directio: Check if buffers are set. (#13440) ... Check if directio buffers have actually been fetched and prevent errors on double Close. Return error on Read after Close. Fixes ``` panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0xf8582f] goroutine 210 [running]: github.com/minio/minio/internal/ioutil.(*ODirectReader).Read(0xc0054f8320, {0xc0014560b0, 0xa8, 0x44d012}) github.com/minio/minio/internal/ioutil/odirect_reader.go:88 +0x10f io.ReadAtLeast({0x428c5c0, 0xc0054f8320}, {0xc0014560b0, 0xa8, 0xa8}, 0xa8) io/io.go:328 +0x9a io.ReadFull(...) io/io.go:347 github.com/minio/minio/internal/ioutil.ReadFile({0xc001bf60e0, 0x6}) github.com/minio/minio/internal/ioutil/read_file.go:48 +0x19b github.com/minio/minio/cmd.(*FSObjects).scanBucket.func1({{0xc00444e1e0, 0x4d}, 0x0, {0xc0040cf240, 0xe}, {0xc0040cf24f, 0x18}, {0xc0040cf268, 0x18}, 0x0, ...}) github.com/minio/minio/cmd/fs-v1.go:366 +0x1ea github.com/minio/minio/cmd.(*folderScanner).scanFolder.func1({0xc00474a6a8, 0xc0065d6793}, 0x0) github.com/minio/minio/cmd/data-scanner.go:494 +0xb15 github.com/minio/minio/cmd.readDirFn({0xc002803e80, 0x34}, 0xc000670270) github.com/minio/minio/cmd/os-readdir_unix.go:172 +0x638 github.com/minio/minio/cmd.(*folderScanner).scanFolder(0xc002deeb40, {0x42dc9d0, 0xc00068cbc0}, {{0xc001c6e2d0, 0x27}, 0xc0023db8e0, 0x1}, 0xc0001c7ab0) github.com/minio/minio/cmd/data-scanner.go:427 +0xa8f github.com/minio/minio/cmd.(*folderScanner).scanFolder.func2({{0xc001c6e2d0, 0x27}, 0xc0023db8e0, 0x27}) github.com/minio/minio/cmd/data-scanner.go:549 +0xd0 github.com/minio/minio/cmd.(*folderScanner).scanFolder(0xc002deeb40, {0x42dc9d0, 0xc00068cbc0}, {{0xc0013fa9e0, 0xe}, 0x0, 0x1}, 0xc000670dd8) github.com/minio/minio/cmd/data-scanner.go:623 +0x205d github.com/minio/minio/cmd.scanDataFolder({_, _}, {_, _}, {{{0xc0013fa9e0, 0xe}, 0x802, {0x210f15d2, 0xed8f903b8, 0x5bc0e80}, ...}, ...}, ...) github.com/minio/minio/cmd/data-scanner.go:333 +0xc51 github.com/minio/minio/cmd.(*FSObjects).scanBucket(_, {_, _}, {_, _}, {{{0xc0013fa9e0, 0xe}, 0x802, {0x210f15d2, 0xed8f903b8, ...}, ...}, ...}) github.com/minio/minio/cmd/fs-v1.go:364 +0x305 github.com/minio/minio/cmd.(*FSObjects).NSScanner(0x42dc9d0, {0x42dc9d0, 0xc00068cbc0}, 0x0, 0xc003bcfda0, 0x802) github.com/minio/minio/cmd/fs-v1.go:307 +0xa16 github.com/minio/minio/cmd.runDataScanner({0x42dc9d0, 0xc00068cbc0}, {0x436a6c0, 0xc000bfcf50}) github.com/minio/minio/cmd/data-scanner.go:150 +0x749 created by github.com/minio/minio/cmd.initDataScanner github.com/minio/minio/cmd/data-scanner.go:73 +0xb0 ```	2021-10-14 10:19:17 -07:00
Harshavardhana	d00ff3c453	use O_DIRECT for all ReadFileStream (#13324) ... This PR also removes #13312 to ensure that we can use a better mechanism to handle page-cache, using O_DIRECT even for Range GETs.	2021-09-29 16:40:28 -07:00