minio/pkg/madmin
Harshavardhana 3184205519
fix: config to support keys with special values (#9304)
This PR adds context-based `k=v` splits based
on the sub-system which was obtained, if the
keys are not provided an error will be thrown
during parsing, if keys are provided with wrong
values an error will be thrown. Keys can now
have values which are of a much more complex
form such as `k="v=v"` or `k=" v = v"`
and other variations.

additionally, deprecate unnecessary postgres/mysql
configuration styles, support only

- connection_string for Postgres
- dsn_string for MySQL

All other parameters are removed.
2020-04-09 21:45:17 -07:00
..
examples Add madmin package context support (#9172) 2020-03-20 15:00:44 -07:00
README.md fix: change policies API to return and take struct (#9181) 2020-04-07 19:30:59 -07:00
api-error-response.go Add API retry functionality in mc admin (#7602) 2019-05-09 17:41:54 -07:00
api-log-entry.go Add madmin package context support (#9172) 2020-03-20 15:00:44 -07:00
api-log.go fix: admin console logger changes to log.Info 2020-03-20 15:14:14 -07:00
api.go fix: Quit when the context is canceled in madmin (#9264) 2020-04-06 17:50:14 -07:00
api_test.go Replace Minio refs in docs with MinIO and links (#7494) 2019-04-09 11:39:42 -07:00
config-commands.go fix: change policies API to return and take struct (#9181) 2020-04-07 19:30:59 -07:00
config-help-commands.go fix: change policies API to return and take struct (#9181) 2020-04-07 19:30:59 -07:00
config-history-commands.go fix: change policies API to return and take struct (#9181) 2020-04-07 19:30:59 -07:00
config-kv-commands.go fix: config to support keys with special values (#9304) 2020-04-09 21:45:17 -07:00
encrypt.go Migrate all backend at .minio.sys/config to encrypted backend (#8474) 2019-11-01 15:53:16 -07:00
encrypt_test.go madmin: Migrate to provable secure channel construction (#8395) 2019-10-15 02:36:04 +05:30
group-commands.go fix: change policies API to return and take struct (#9181) 2020-04-07 19:30:59 -07:00
hardware-commands.go Add madmin package context support (#9172) 2020-03-20 15:00:44 -07:00
heal-commands.go fix: startup load time by reusing storageDisks (#9210) 2020-03-27 14:48:30 -07:00
heal-commands_test.go Replace Minio refs in docs with MinIO and links (#7494) 2019-04-09 11:39:42 -07:00
info-commands.go Add madmin package context support (#9172) 2020-03-20 15:00:44 -07:00
kms-commands.go fix: change policies API to return and take struct (#9181) 2020-04-07 19:30:59 -07:00
obd.go increase drive OBD blocksize to 4MB (#9258) 2020-04-08 06:04:27 -07:00
obd_freebsd.go Implement oboard diagnostics admin API (#9024) 2020-03-26 21:07:39 -07:00
obd_other.go Implement oboard diagnostics admin API (#9024) 2020-03-26 21:07:39 -07:00
parse-kv.go fix: config to support keys with special values (#9304) 2020-04-09 21:45:17 -07:00
policy-commands.go fix: change policies API to return and take struct (#9181) 2020-04-07 19:30:59 -07:00
profiling-commands.go Add madmin package context support (#9172) 2020-03-20 15:00:44 -07:00
retry.go fix: Quit when the context is canceled in madmin (#9264) 2020-04-06 17:50:14 -07:00
service-commands.go Add madmin package context support (#9172) 2020-03-20 15:00:44 -07:00
top-commands.go fix: change policies API to return and take struct (#9181) 2020-04-07 19:30:59 -07:00
transport.go Add madmin package context support (#9172) 2020-03-20 15:00:44 -07:00
update-commands.go Add madmin package context support (#9172) 2020-03-20 15:00:44 -07:00
user-commands.go fix: change policies API to return and take struct (#9181) 2020-04-07 19:30:59 -07:00
utils.go fix: change policies API to return and take struct (#9181) 2020-04-07 19:30:59 -07:00

README.md

Golang Admin Client API Reference Slack

The MinIO Admin Golang Client SDK provides APIs to manage MinIO services.

This quickstart guide will show you how to install the MinIO Admin client SDK, connect to MinIO admin service, and provide a walkthrough of a simple file uploader.

This document assumes that you have a working Golang setup.

Initialize MinIO Admin Client object.

MinIO


package main

import (
    "fmt"

    "github.com/minio/minio/pkg/madmin"
)

func main() {
    // Use a secure connection.
    ssl := true

    // Initialize minio client object.
    mdmClnt, err := madmin.New("your-minio.example.com:9000", "YOUR-ACCESSKEYID", "YOUR-SECRETKEY", ssl)
    if err != nil {
        fmt.Println(err)
        return
    }

    // Fetch service status.
    st, err := mdmClnt.ServerInfo()
    if err != nil {
        fmt.Println(err)
        return
    }
	for _, peerInfo := range serversInfo {
		log.Printf("Node: %s, Info: %v\n", peerInfo.Addr, peerInfo.Data)
	}
}

Service operations Info operations Healing operations Config operations Top operations IAM operations Misc KMS
ServiceRestart ServerInfo Heal GetConfig TopLocks AddUser GetKeyStatus
ServiceStop ServerCPULoadInfo SetConfig SetUserPolicy StartProfiling
ServerMemUsageInfo ListUsers DownloadProfilingData
ServiceTrace ServerDrivesPerfInfo AddCannedPolicy ServerUpdate
NetPerfInfo
ServerCPUHardwareInfo
ServerNetworkHardwareInfo
StorageInfo

1. Constructor

New(endpoint string, accessKeyID string, secretAccessKey string, ssl bool) (*AdminClient, error)

Initializes a new admin client object.

Parameters

Param Type Description
endpoint string MinIO endpoint.
accessKeyID string Access key for the object storage endpoint.
secretAccessKey string Secret key for the object storage endpoint.
ssl bool Set this value to 'true' to enable secure (HTTPS) access.

2. Service operations

ServiceStatus(ctx context.Context) (ServiceStatusMetadata, error)

Fetch service status, replies disk space used, backend type and total disks offline/online (applicable in distributed mode).

Param Type Description
serviceStatus ServiceStatusMetadata Represents current server status info in following format:
Param Type Description
st.ServerVersion.Version string Server version.
st.ServerVersion.CommitID string Server commit id.
st.Uptime time.Duration Server uptime duration in seconds.

Example


   st, err := madmClnt.ServiceStatus(context.Background())
   if err != nil {
   	log.Fatalln(err)
   }
   log.Printf("%#v\n", st)

ServiceRestart(ctx context.Context) error

Sends a service action restart command to MinIO server.

Example

   // To restart the service, restarts all servers in the cluster.
   err := madmClnt.ServiceRestart(context.Background())
   if err != nil {
       log.Fatalln(err)
   }
   log.Println("Success")

ServiceStop(ctx context.Context) error

Sends a service action stop command to MinIO server.

Example

   // To stop the service, stops all servers in the cluster.
   err := madmClnt.ServiceStop(context.Background())
   if err != nil {
       log.Fatalln(err)
   }
   log.Println("Success")

ServiceTrace(ctx context.Context, allTrace bool, doneCh <-chan struct{}) <-chan TraceInfo

Enable HTTP request tracing on all nodes in a MinIO cluster

Example

    doneCh := make(chan struct{})
    defer close(doneCh)
    // listen to all trace including internal API calls
    allTrace := true
    // Start listening on all trace activity.
    traceCh := madmClnt.ServiceTrace(context.Background(), allTrace, doneCh)
    for traceInfo := range traceCh {
        fmt.Println(traceInfo.String())
    }

3. Info operations

ServerInfo(ctx context.Context) ([]ServerInfo, error)

Fetches information for all cluster nodes, such as server properties, storage information, network statistics, etc.

Param Type Description
si.Addr string Address of the server the following information is retrieved from.
si.ConnStats ServerConnStats Connection statistics from the given server.
si.HTTPStats ServerHTTPStats HTTP connection statistics from the given server.
si.Properties ServerProperties Server properties such as region, notification targets.
Param Type Description
ServerProperties.Uptime time.Duration Total duration in seconds since server is running.
ServerProperties.Version string Current server version.
ServerProperties.CommitID string Current server commitID.
ServerProperties.Region string Configured server region.
ServerProperties.SQSARN []string List of notification target ARNs.
Param Type Description
ServerConnStats.TotalInputBytes uint64 Total bytes received by the server.
ServerConnStats.TotalOutputBytes uint64 Total bytes sent by the server.
Param Type Description
ServerHTTPStats.TotalHEADStats ServerHTTPMethodStats Total statistics regarding HEAD operations
ServerHTTPStats.SuccessHEADStats ServerHTTPMethodStats Total statistics regarding successful HEAD operations
ServerHTTPStats.TotalGETStats ServerHTTPMethodStats Total statistics regarding GET operations
ServerHTTPStats.SuccessGETStats ServerHTTPMethodStats Total statistics regarding successful GET operations
ServerHTTPStats.TotalPUTStats ServerHTTPMethodStats Total statistics regarding PUT operations
ServerHTTPStats.SuccessPUTStats ServerHTTPMethodStats Total statistics regarding successful PUT operations
ServerHTTPStats.TotalPOSTStats ServerHTTPMethodStats Total statistics regarding POST operations
ServerHTTPStats.SuccessPOSTStats ServerHTTPMethodStats Total statistics regarding successful POST operations
ServerHTTPStats.TotalDELETEStats ServerHTTPMethodStats Total statistics regarding DELETE operations
ServerHTTPStats.SuccessDELETEStats ServerHTTPMethodStats Total statistics regarding successful DELETE operations
Param Type Description
ServerHTTPMethodStats.Count uint64 Total number of operations.
ServerHTTPMethodStats.AvgDuration string Average duration of Count number of operations.
Param Type Description
DriveInfo.UUID string Unique ID for each disk provisioned by server format.
DriveInfo.Endpoint string Endpoint location of the remote/local disk.
DriveInfo.State string Current state of the disk at endpoint.

Example


   serversInfo, err := madmClnt.ServerInfo(context.Background())
   if err != nil {
   	log.Fatalln(err)
   }

   for _, peerInfo := range serversInfo {
   	log.Printf("Node: %s, Info: %v\n", peerInfo.Addr, peerInfo.Data)
   }

ServerDrivesPerfInfo(ctx context.Context) ([]ServerDrivesPerfInfo, error)

Fetches drive performance information for all cluster nodes.

Param Type Description
di.Addr string Address of the server the following information is retrieved from.
di.Error string Errors (if any) encountered while reaching this node
di.DrivesPerf disk.Performance Path of the drive mount on above server and read, write speed.
Param Type Description
disk.Performance.Path string Path of drive mount.
disk.Performance.Error string Error (if any) encountered while accessing this drive.
disk.Performance.WriteSpeed float64 Write speed on above path in Bytes/s.
disk.Performance.ReadSpeed float64 Read speed on above path in Bytes/s.

ServerCPULoadInfo(ctx context.Context) ([]ServerCPULoadInfo, error)

Fetches CPU utilization for all cluster nodes.

Param Type Description
cpui.Addr string Address of the server the following information is retrieved from.
cpui.Error string Errors (if any) encountered while reaching this node
cpui.CPULoad cpu.Load The load on the CPU.
Param Type Description
cpu.Load.Avg float64 The average utilization of the CPU measured in a 200ms interval
cpu.Load.Min float64 The minimum utilization of the CPU measured in a 200ms interval
cpu.Load.Max float64 The maximum utilization of the CPU measured in a 200ms interval
cpu.Load.Error string Error (if any) encountered while accessing the CPU info

ServerMemUsageInfo(ctx context.Context) ([]ServerMemUsageInfo, error)

Fetches Mem utilization for all cluster nodes.

Param Type Description
memi.Addr string Address of the server the following information is retrieved from.
memi.Error string Errors (if any) encountered while reaching this node
memi.MemUsage mem.Usage The utilitzation of Memory
Param Type Description
mem.Usage.Mem uint64 The total number of bytes obtained from the OS
mem.Usage.Error string Error (if any) encountered while accessing the CPU info

NetPerfInfo(ctx context.Context, int size) (map[string][]NetPerfInfo, error)

Fetches network performance of all cluster nodes using given sized payload. Returned value is a map containing each node indexed list of performance of other nodes.

Param Type Description
Addr string Address of the server the following information is retrieved from.
Error string Errors (if any) encountered while reaching this node
ReadThroughput uint64 Network read throughput of the server in bytes per second

ServerCPUHardwareInfo(ctx context.Context) ([]ServerCPUHardwareInfo, error)

Fetches hardware information of CPU.

Param Type Description
hwi.Addr string Address of the server the following information is retrieved from.
hwi.Error string Errors (if any) encountered while reaching this node
hwi.CPUInfo []cpu.InfoStat The CPU hardware info.
Param Type Description
CPUInfo.CPU int32 CPU
CPUInfo.VendorID string Vendor Id
CPUInfo.Family string CPU Family
CPUInfo.Model string Model
CPUInfo.Stepping int32 Stepping
CPUInfo.PhysicalId string Physical Id
CPUInfo.CoreId string Core Id
CPUInfo.Cores int32 Cores
CPUInfo.ModelName string Model Name
CPUInfo.Mhz float64 Mhz
CPUInfo.CacheZSize int32 cache sizes
CPUInfo.Flags []string Flags
CPUInfo.Microcode string Micro codes

ServerNetworkHardwareInfo(ctx context.Context) ([]ServerNetworkHardwareInfo, error)

Fetches hardware information of CPU.

Param Type Description
hwi.Addr string Address of the server the following information is retrieved from.
hwi.Error string Errors (if any) encountered while reaching this node
hwi.NetworkInfo []net.Interface The network hardware info
Param Type Description
NetworkInfo.Index int32 positive integer that starts at one, zero is never used.
NetworkInfo.MTU int32 maximum transmission unit
NetworkInfo.Name string e.g., "en0", "lo0", "eth0.100"
NetworkInfo.HardwareAddr []byte IEEE MAC-48, EUI-48 and EUI-64 form
NetworkInfo.Flags uint32 e.g., FlagUp, FlagLoopback, FlagMulticast

StorageInfo(ctx context.Context) (StorageInfo, error)

Fetches Storage information for all cluster nodes.

Param Type Description
storageInfo.Used []int64 Used disk spaces.
storageInfo.Total []int64 Total disk spaces.
storageInfo.Available []int64 Available disk spaces.
StorageInfo.Backend struct{} Represents backend type embedded structure.
Param Type Description
Backend.Type BackendType Type of backend used by the server currently only FS or Erasure.
Backend.OnlineDisks BackendDisks Total number of disks online per node (only applies to Erasure backend) represented in map[string]int, is empty for FS.
Backend.OfflineDisks BackendDisks Total number of disks offline per node (only applies to Erasure backend) represented in map[string]int, is empty for FS.
Backend.StandardSCData int Data disks set for standard storage class, is empty for FS.
Backend.StandardSCParity int Parity disks set for standard storage class, is empty for FS.
Backend.RRSCData int Data disks set for reduced redundancy storage class, is empty for FS.
Backend.RRSCParity int Parity disks set for reduced redundancy storage class, is empty for FS.
Backend.Sets [][]DriveInfo Represents topology of drives in erasure coded sets.

Example


   storageInfo, err := madmClnt.StorageInfo(context.Background())
   if err != nil {
   	log.Fatalln(err)
   }

   log.Println(storageInfo)

5. Heal operations

Heal(ctx context.Context, bucket, prefix string, healOpts HealOpts, clientToken string, forceStart bool, forceStop bool) (start HealStartSuccess, status HealTaskStatus, err error)

Start a heal sequence that scans data under given (possible empty) bucket and prefix. The recursive bool turns on recursive traversal under the given path. dryRun does not mutate on-disk data, but performs data validation.

Two heal sequences on overlapping paths may not be initiated.

The progress of a heal should be followed using the same API Heal by providing the clientToken previously obtained from a Heal API. The server accumulates results of the heal traversal and waits for the client to receive and acknowledge them using the status request by providing clientToken.

Example


    opts := madmin.HealOpts{
            Recursive: true,
            DryRun:    false,
    }
    forceStart := false
    forceStop := false
    healPath, err := madmClnt.Heal(context.Background(), "", "", opts, "", forceStart, forceStop)
    if err != nil {
        log.Fatalln(err)
    }
    log.Printf("Heal sequence started at %s", healPath)

HealStartSuccess structure

Param Type Description
s.ClientToken string A unique token for a successfully started heal operation, this token is used to request realtime progress of the heal operation.
s.ClientAddress string Address of the client which initiated the heal operation, the client address has the form "host:port".
s.StartTime time.Time Time when heal was initially started.

HealTaskStatus structure

Param Type Description
s.Summary string Short status of heal sequence
s.FailureDetail string Error message in case of heal sequence failure
s.HealSettings HealOpts Contains the booleans set in the HealStart call
s.Items []HealResultItem Heal records for actions performed by server

HealResultItem structure

Param Type Description
ResultIndex int64 Index of the heal-result record
Type HealItemType Represents kind of heal operation in the heal record
Bucket string Bucket name
Object string Object name
Detail string Details about heal operation
DiskInfo.AvailableOn []int List of disks on which the healed entity is present and healthy
DiskInfo.HealedOn []int List of disks on which the healed entity was restored

6. Config operations

GetConfig(ctx context.Context) ([]byte, error)

Get current config.json of a MinIO server.

Example

    configBytes, err := madmClnt.GetConfig(context.Background())
    if err != nil {
        log.Fatalf("failed due to: %v", err)
    }

    // Pretty-print config received as json.
    var buf bytes.Buffer
    err = json.Indent(buf, configBytes, "", "\t")
    if err != nil {
        log.Fatalf("failed due to: %v", err)
    }

    log.Println("config received successfully: ", string(buf.Bytes()))

SetConfig(ctx context.Context, config io.Reader) error

Set a new config.json for a MinIO server.

Example

    config := bytes.NewReader([]byte(`config.json contents go here`))
    if err := madmClnt.SetConfig(context.Background(), config); err != nil {
        log.Fatalf("failed due to: %v", err)
    }
    log.Println("SetConfig was successful")

7. Top operations

TopLocks(ctx context.Context) (LockEntries, error)

Get the oldest locks from MinIO server.

Example

    locks, err := madmClnt.TopLocks(context.Background())
    if err != nil {
        log.Fatalf("failed due to: %v", err)
    }

    out, err := json.Marshal(locks)
    if err != nil {
        log.Fatalf("Marshal failed due to: %v", err)
    }

    log.Println("TopLocks received successfully: ", string(out))

8. IAM operations

AddCannedPolicy(ctx context.Context, policyName string, policy *iampolicy.Policy) error

Create a new canned policy on MinIO server.

Example

	policy, err := iampolicy.ParseConfig(strings.NewReader(`{"Version": "2012-10-17","Statement": [{"Action": ["s3:GetObject"],"Effect": "Allow","Resource": ["arn:aws:s3:::my-bucketname/*"],"Sid": ""}]}`))
    if err != nil {
        log.Fatalln(err)
    }

    if err = madmClnt.AddCannedPolicy(context.Background(), "get-only", policy); err != nil {
		log.Fatalln(err)
	}

AddUser(ctx context.Context, user string, secret string) error

Add a new user on a MinIO server.

Example

	if err = madmClnt.AddUser(context.Background(), "newuser", "newstrongpassword"); err != nil {
		log.Fatalln(err)
	}

SetUserPolicy(ctx context.Context, user string, policyName string) error

Enable a canned policy get-only for a given user on MinIO server.

Example

	if err = madmClnt.SetUserPolicy(context.Background(), "newuser", "get-only"); err != nil {
		log.Fatalln(err)
	}

ListUsers(ctx context.Context) (map[string]UserInfo, error)

Lists all users on MinIO server.

Example

	users, err := madmClnt.ListUsers(context.Background());
    if err != nil {
		log.Fatalln(err)
	}
    for k, v := range users {
        fmt.Printf("User %s Status %s\n", k, v.Status)
    }

9. Misc operations

ServerUpdate(ctx context.Context, updateURL string) (ServerUpdateStatus, error)

Sends a update command to MinIO server, to update MinIO server to latest release. In distributed setup it updates all servers atomically.

Example

   // Updates all servers and restarts all the servers in the cluster.
   // optionally takes an updateURL, which is used to update the binary.
   us, err := madmClnt.ServerUpdate(context.Background(), updateURL)
   if err != nil {
       log.Fatalln(err)
   }
   if us.CurrentVersion != us.UpdatedVersion {
       log.Printf("Updated server version from %s to %s successfully", us.CurrentVersion, us.UpdatedVersion)
   }

StartProfiling(ctx context.Context, profiler string) error

Ask all nodes to start profiling using the specified profiler mode

Example

    startProfilingResults, err = madmClnt.StartProfiling(context.Background(), "cpu")
    if err != nil {
            log.Fatalln(err)
    }
    for _, result := range startProfilingResults {
        if !result.Success {
            log.Printf("Unable to start profiling on node `%s`, reason = `%s`\n", result.NodeName, result.Error)
        } else {
            log.Printf("Profiling successfully started on node `%s`\n", result.NodeName)
        }
    }

DownloadProfilingData(ctx context.Context) ([]byte, error)

Download profiling data of all nodes in a zip format.

Example

    profilingData, err := madmClnt.DownloadProfilingData(context.Background())
    if err != nil {
            log.Fatalln(err)
    }

    profilingFile, err := os.Create("/tmp/profiling-data.zip")
    if err != nil {
            log.Fatal(err)
    }

    if _, err := io.Copy(profilingFile, profilingData); err != nil {
            log.Fatal(err)
    }

    if err := profilingFile.Close(); err != nil {
            log.Fatal(err)
    }

    if err := profilingData.Close(); err != nil {
            log.Fatal(err)
    }

    log.Println("Profiling data successfully downloaded.")

11. KMS

GetKeyStatus(ctx context.Context, keyID string) (*KMSKeyStatus, error)

Requests status information about one particular KMS master key from a MinIO server. The keyID is optional and the server will use the default master key (configured via MINIO_KMS_VAULT_KEY_NAME or MINIO_KMS_MASTER_KEY) if the keyID is empty.

Example

    keyInfo, err := madmClnt.GetKeyStatus(context.Background(), "my-minio-key")
    if err != nil {
       log.Fatalln(err)
    }
    if keyInfo.EncryptionErr != "" {
       log.Fatalf("Failed to perform encryption operation using '%s': %v\n", keyInfo.KeyID, keyInfo.EncryptionErr)
    }
    if keyInfo.UpdateErr != "" {
       log.Fatalf("Failed to perform key re-wrap operation using '%s': %v\n", keyInfo.KeyID, keyInfo.UpdateErr)
    }
    if keyInfo.DecryptionErr != "" {
       log.Fatalf("Failed to perform decryption operation using '%s': %v\n", keyInfo.KeyID, keyInfo.DecryptionErr)
    }