minio/pkg/madmin
Klaus Post c7178d2066 Profiling: Add base, fix memory profiling (#8850)
For 'snapshot' type profiles, record a 'before' profile that can be used 
as `go tool pprof -base=before ...` to compare before and after.

"Before" profiles are included in the zipped package.

[`runtime.MemProfileRate`](https://golang.org/pkg/runtime/#pkg-variables) 
should not be updated while the application is running, so we set it at startup.

Co-authored-by: Harshavardhana <harsha@minio.io>
2020-01-21 15:49:25 -08:00
..
examples Add data usage collect with its new admin API (#8553) 2019-12-12 06:02:37 -08:00
README.md Migrate config to KV data format (#8392) 2019-10-22 22:59:13 -07:00
api-error-response.go Add API retry functionality in mc admin (#7602) 2019-05-09 17:41:54 -07:00
api-log.go The prometheus metrics refractoring (#8003) 2019-10-22 21:01:14 -07:00
api.go fix: close and drain the response body always (#8847) 2020-01-21 02:46:58 -08:00
api_test.go Replace Minio refs in docs with MinIO and links (#7494) 2019-04-09 11:39:42 -07:00
config-commands.go Migrate config to KV data format (#8392) 2019-10-22 22:59:13 -07:00
config-help-commands.go Add help with order of keys (#8535) 2019-11-19 13:48:13 -08:00
config-history-commands.go Enhance config restore to carry previous set content as well (#8483) 2019-11-05 06:18:26 -08:00
config-kv-commands.go Export command prints turned-off sub-sys as comments (#8594) 2019-12-03 10:50:20 -08:00
encrypt.go Migrate all backend at .minio.sys/config to encrypted backend (#8474) 2019-11-01 15:53:16 -07:00
encrypt_test.go madmin: Migrate to provable secure channel construction (#8395) 2019-10-15 02:36:04 +05:30
group-commands.go The prometheus metrics refractoring (#8003) 2019-10-22 21:01:14 -07:00
hardware-commands.go The prometheus metrics refractoring (#8003) 2019-10-22 21:01:14 -07:00
heal-commands.go The prometheus metrics refractoring (#8003) 2019-10-22 21:01:14 -07:00
heal-commands_test.go Replace Minio refs in docs with MinIO and links (#7494) 2019-04-09 11:39:42 -07:00
info-commands.go Add bucket and object count along with total object size (#8639) 2019-12-12 09:58:59 -08:00
kms-commands.go The prometheus metrics refractoring (#8003) 2019-10-22 21:01:14 -07:00
parse-kv.go Ensure comment is always a valid key (#8604) 2019-12-05 18:17:42 +05:30
policy-commands.go The prometheus metrics refractoring (#8003) 2019-10-22 21:01:14 -07:00
profiling-commands.go Profiling: Add base, fix memory profiling (#8850) 2020-01-21 15:49:25 -08:00
retry.go Add API retry functionality in mc admin (#7602) 2019-05-09 17:41:54 -07:00
service-commands.go The prometheus metrics refractoring (#8003) 2019-10-22 21:01:14 -07:00
top-commands.go The prometheus metrics refractoring (#8003) 2019-10-22 21:01:14 -07:00
update-commands.go The prometheus metrics refractoring (#8003) 2019-10-22 21:01:14 -07:00
user-commands.go The prometheus metrics refractoring (#8003) 2019-10-22 21:01:14 -07:00
utils.go The prometheus metrics refractoring (#8003) 2019-10-22 21:01:14 -07:00

README.md

Golang Admin Client API Reference Slack

The MinIO Admin Golang Client SDK provides APIs to manage MinIO services.

This quickstart guide will show you how to install the MinIO Admin client SDK, connect to MinIO admin service, and provide a walkthrough of a simple file uploader.

This document assumes that you have a working Golang setup.

Initialize MinIO Admin Client object.

MinIO


package main

import (
    "fmt"

    "github.com/minio/minio/pkg/madmin"
)

func main() {
    // Use a secure connection.
    ssl := true

    // Initialize minio client object.
    mdmClnt, err := madmin.New("your-minio.example.com:9000", "YOUR-ACCESSKEYID", "YOUR-SECRETKEY", ssl)
    if err != nil {
        fmt.Println(err)
        return
    }

    // Fetch service status.
    st, err := mdmClnt.ServerInfo()
    if err != nil {
        fmt.Println(err)
        return
    }
	for _, peerInfo := range serversInfo {
		log.Printf("Node: %s, Info: %v\n", peerInfo.Addr, peerInfo.Data)
	}
}

Service operations Info operations Healing operations Config operations Top operations IAM operations Misc KMS
ServiceRestart ServerInfo Heal GetConfig TopLocks AddUser GetKeyStatus
ServiceStop ServerCPULoadInfo SetConfig SetUserPolicy StartProfiling
ServerMemUsageInfo ListUsers DownloadProfilingData
ServiceTrace ServerDrivesPerfInfo AddCannedPolicy ServerUpdate
NetPerfInfo
ServerCPUHardwareInfo
ServerNetworkHardwareInfo
StorageInfo

1. Constructor

New(endpoint string, accessKeyID string, secretAccessKey string, ssl bool) (*AdminClient, error)

Initializes a new admin client object.

Parameters

Param Type Description
endpoint string MinIO endpoint.
accessKeyID string Access key for the object storage endpoint.
secretAccessKey string Secret key for the object storage endpoint.
ssl bool Set this value to 'true' to enable secure (HTTPS) access.

2. Service operations

ServiceStatus() (ServiceStatusMetadata, error)

Fetch service status, replies disk space used, backend type and total disks offline/online (applicable in distributed mode).

Param Type Description
serviceStatus ServiceStatusMetadata Represents current server status info in following format:
Param Type Description
st.ServerVersion.Version string Server version.
st.ServerVersion.CommitID string Server commit id.
st.Uptime time.Duration Server uptime duration in seconds.

Example


   st, err := madmClnt.ServiceStatus()
   if err != nil {
   	log.Fatalln(err)
   }
   log.Printf("%#v\n", st)

ServiceRestart() error

Sends a service action restart command to MinIO server.

Example

   // To restart the service, restarts all servers in the cluster.
   err := madmClnt.ServiceRestart()
   if err != nil {
       log.Fatalln(err)
   }
   log.Println("Success")

ServiceStop() error

Sends a service action stop command to MinIO server.

Example

   // To stop the service, stops all servers in the cluster.
   err := madmClnt.ServiceStop()
   if err != nil {
       log.Fatalln(err)
   }
   log.Println("Success")

ServiceTrace(allTrace bool, doneCh <-chan struct{}) <-chan TraceInfo

Enable HTTP request tracing on all nodes in a MinIO cluster

Example

    doneCh := make(chan struct{})
    defer close(doneCh)
    // listen to all trace including internal API calls
    allTrace := true
    // Start listening on all trace activity.
    traceCh := madmClnt.ServiceTrace(allTrace, doneCh)
    for traceInfo := range traceCh {
        fmt.Println(traceInfo.String())
    }

3. Info operations

ServerInfo() ([]ServerInfo, error)

Fetches information for all cluster nodes, such as server properties, storage information, network statistics, etc.

Param Type Description
si.Addr string Address of the server the following information is retrieved from.
si.ConnStats ServerConnStats Connection statistics from the given server.
si.HTTPStats ServerHTTPStats HTTP connection statistics from the given server.
si.Properties ServerProperties Server properties such as region, notification targets.
Param Type Description
ServerProperties.Uptime time.Duration Total duration in seconds since server is running.
ServerProperties.Version string Current server version.
ServerProperties.CommitID string Current server commitID.
ServerProperties.Region string Configured server region.
ServerProperties.SQSARN []string List of notification target ARNs.
Param Type Description
ServerConnStats.TotalInputBytes uint64 Total bytes received by the server.
ServerConnStats.TotalOutputBytes uint64 Total bytes sent by the server.
Param Type Description
ServerHTTPStats.TotalHEADStats ServerHTTPMethodStats Total statistics regarding HEAD operations
ServerHTTPStats.SuccessHEADStats ServerHTTPMethodStats Total statistics regarding successful HEAD operations
ServerHTTPStats.TotalGETStats ServerHTTPMethodStats Total statistics regarding GET operations
ServerHTTPStats.SuccessGETStats ServerHTTPMethodStats Total statistics regarding successful GET operations
ServerHTTPStats.TotalPUTStats ServerHTTPMethodStats Total statistics regarding PUT operations
ServerHTTPStats.SuccessPUTStats ServerHTTPMethodStats Total statistics regarding successful PUT operations
ServerHTTPStats.TotalPOSTStats ServerHTTPMethodStats Total statistics regarding POST operations
ServerHTTPStats.SuccessPOSTStats ServerHTTPMethodStats Total statistics regarding successful POST operations
ServerHTTPStats.TotalDELETEStats ServerHTTPMethodStats Total statistics regarding DELETE operations
ServerHTTPStats.SuccessDELETEStats ServerHTTPMethodStats Total statistics regarding successful DELETE operations
Param Type Description
ServerHTTPMethodStats.Count uint64 Total number of operations.
ServerHTTPMethodStats.AvgDuration string Average duration of Count number of operations.
Param Type Description
DriveInfo.UUID string Unique ID for each disk provisioned by server format.
DriveInfo.Endpoint string Endpoint location of the remote/local disk.
DriveInfo.State string Current state of the disk at endpoint.

Example


   serversInfo, err := madmClnt.ServerInfo()
   if err != nil {
   	log.Fatalln(err)
   }

   for _, peerInfo := range serversInfo {
   	log.Printf("Node: %s, Info: %v\n", peerInfo.Addr, peerInfo.Data)
   }

ServerDrivesPerfInfo() ([]ServerDrivesPerfInfo, error)

Fetches drive performance information for all cluster nodes.

Param Type Description
di.Addr string Address of the server the following information is retrieved from.
di.Error string Errors (if any) encountered while reaching this node
di.DrivesPerf disk.Performance Path of the drive mount on above server and read, write speed.
Param Type Description
disk.Performance.Path string Path of drive mount.
disk.Performance.Error string Error (if any) encountered while accessing this drive.
disk.Performance.WriteSpeed float64 Write speed on above path in Bytes/s.
disk.Performance.ReadSpeed float64 Read speed on above path in Bytes/s.

ServerCPULoadInfo() ([]ServerCPULoadInfo, error)

Fetches CPU utilization for all cluster nodes.

Param Type Description
cpui.Addr string Address of the server the following information is retrieved from.
cpui.Error string Errors (if any) encountered while reaching this node
cpui.CPULoad cpu.Load The load on the CPU.
Param Type Description
cpu.Load.Avg float64 The average utilization of the CPU measured in a 200ms interval
cpu.Load.Min float64 The minimum utilization of the CPU measured in a 200ms interval
cpu.Load.Max float64 The maximum utilization of the CPU measured in a 200ms interval
cpu.Load.Error string Error (if any) encountered while accessing the CPU info

ServerMemUsageInfo() ([]ServerMemUsageInfo, error)

Fetches Mem utilization for all cluster nodes.

Param Type Description
memi.Addr string Address of the server the following information is retrieved from.
memi.Error string Errors (if any) encountered while reaching this node
memi.MemUsage mem.Usage The utilitzation of Memory
Param Type Description
mem.Usage.Mem uint64 The total number of bytes obtained from the OS
mem.Usage.Error string Error (if any) encountered while accessing the CPU info

NetPerfInfo(int size) (map[string][]NetPerfInfo, error)

Fetches network performance of all cluster nodes using given sized payload. Returned value is a map containing each node indexed list of performance of other nodes.

Param Type Description
Addr string Address of the server the following information is retrieved from.
Error string Errors (if any) encountered while reaching this node
ReadThroughput uint64 Network read throughput of the server in bytes per second

ServerCPUHardwareInfo() ([]ServerCPUHardwareInfo, error)

Fetches hardware information of CPU.

Param Type Description
hwi.Addr string Address of the server the following information is retrieved from.
hwi.Error string Errors (if any) encountered while reaching this node
hwi.CPUInfo []cpu.InfoStat The CPU hardware info.
Param Type Description
CPUInfo.CPU int32 CPU
CPUInfo.VendorID string Vendor Id
CPUInfo.Family string CPU Family
CPUInfo.Model string Model
CPUInfo.Stepping int32 Stepping
CPUInfo.PhysicalId string Physical Id
CPUInfo.CoreId string Core Id
CPUInfo.Cores int32 Cores
CPUInfo.ModelName string Model Name
CPUInfo.Mhz float64 Mhz
CPUInfo.CacheZSize int32 cache sizes
CPUInfo.Flags []string Flags
CPUInfo.Microcode string Micro codes

ServerNetworkHardwareInfo() ([]ServerNetworkHardwareInfo, error)

Fetches hardware information of CPU.

Param Type Description
hwi.Addr string Address of the server the following information is retrieved from.
hwi.Error string Errors (if any) encountered while reaching this node
hwi.NetworkInfo []net.Interface The network hardware info
Param Type Description
NetworkInfo.Index int32 positive integer that starts at one, zero is never used.
NetworkInfo.MTU int32 maximum transmission unit
NetworkInfo.Name string e.g., "en0", "lo0", "eth0.100"
NetworkInfo.HardwareAddr []byte IEEE MAC-48, EUI-48 and EUI-64 form
NetworkInfo.Flags uint32 e.g., FlagUp, FlagLoopback, FlagMulticast

StorageInfo() (StorageInfo, error)

Fetches Storage information for all cluster nodes.

Param Type Description
storageInfo.Used []int64 Used disk spaces.
storageInfo.Total []int64 Total disk spaces.
storageInfo.Available []int64 Available disk spaces.
StorageInfo.Backend struct{} Represents backend type embedded structure.
Param Type Description
Backend.Type BackendType Type of backend used by the server currently only FS or Erasure.
Backend.OnlineDisks BackendDisks Total number of disks online per node (only applies to Erasure backend) represented in map[string]int, is empty for FS.
Backend.OfflineDisks BackendDisks Total number of disks offline per node (only applies to Erasure backend) represented in map[string]int, is empty for FS.
Backend.StandardSCData int Data disks set for standard storage class, is empty for FS.
Backend.StandardSCParity int Parity disks set for standard storage class, is empty for FS.
Backend.RRSCData int Data disks set for reduced redundancy storage class, is empty for FS.
Backend.RRSCParity int Parity disks set for reduced redundancy storage class, is empty for FS.
Backend.Sets [][]DriveInfo Represents topology of drives in erasure coded sets.

Example


   storageInfo, err := madmClnt.StorageInfo()
   if err != nil {
   	log.Fatalln(err)
   }

   log.Println(storageInfo)

5. Heal operations

Heal(bucket, prefix string, healOpts HealOpts, clientToken string, forceStart bool, forceStop bool) (start HealStartSuccess, status HealTaskStatus, err error)

Start a heal sequence that scans data under given (possible empty) bucket and prefix. The recursive bool turns on recursive traversal under the given path. dryRun does not mutate on-disk data, but performs data validation.

Two heal sequences on overlapping paths may not be initiated.

The progress of a heal should be followed using the same API Heal by providing the clientToken previously obtained from a Heal API. The server accumulates results of the heal traversal and waits for the client to receive and acknowledge them using the status request by providing clientToken.

Example


    opts := madmin.HealOpts{
            Recursive: true,
            DryRun:    false,
    }
    forceStart := false
    forceStop := false
    healPath, err := madmClnt.Heal("", "", opts, "", forceStart, forceStop)
    if err != nil {
        log.Fatalln(err)
    }
    log.Printf("Heal sequence started at %s", healPath)

HealStartSuccess structure

Param Type Description
s.ClientToken string A unique token for a successfully started heal operation, this token is used to request realtime progress of the heal operation.
s.ClientAddress string Address of the client which initiated the heal operation, the client address has the form "host:port".
s.StartTime time.Time Time when heal was initially started.

HealTaskStatus structure

Param Type Description
s.Summary string Short status of heal sequence
s.FailureDetail string Error message in case of heal sequence failure
s.HealSettings HealOpts Contains the booleans set in the HealStart call
s.Items []HealResultItem Heal records for actions performed by server

HealResultItem structure

Param Type Description
ResultIndex int64 Index of the heal-result record
Type HealItemType Represents kind of heal operation in the heal record
Bucket string Bucket name
Object string Object name
Detail string Details about heal operation
DiskInfo.AvailableOn []int List of disks on which the healed entity is present and healthy
DiskInfo.HealedOn []int List of disks on which the healed entity was restored

6. Config operations

GetConfig() ([]byte, error)

Get current config.json of a MinIO server.

Example

    configBytes, err := madmClnt.GetConfig()
    if err != nil {
        log.Fatalf("failed due to: %v", err)
    }

    // Pretty-print config received as json.
    var buf bytes.Buffer
    err = json.Indent(buf, configBytes, "", "\t")
    if err != nil {
        log.Fatalf("failed due to: %v", err)
    }

    log.Println("config received successfully: ", string(buf.Bytes()))

SetConfig(config io.Reader) error

Set a new config.json for a MinIO server.

Example

    config := bytes.NewReader([]byte(`config.json contents go here`))
    if err := madmClnt.SetConfig(config); err != nil {
        log.Fatalf("failed due to: %v", err)
    }
    log.Println("SetConfig was successful")

7. Top operations

TopLocks() (LockEntries, error)

Get the oldest locks from MinIO server.

Example

    locks, err := madmClnt.TopLocks()
    if err != nil {
        log.Fatalf("failed due to: %v", err)
    }

    out, err := json.Marshal(locks)
    if err != nil {
        log.Fatalf("Marshal failed due to: %v", err)
    }

    log.Println("TopLocks received successfully: ", string(out))

8. IAM operations

AddCannedPolicy(policyName string, policy string) error

Create a new canned policy on MinIO server.

Example

	policy := `{"Version": "2012-10-17","Statement": [{"Action": ["s3:GetObject"],"Effect": "Allow","Resource": ["arn:aws:s3:::my-bucketname/*"],"Sid": ""}]}`

    if err = madmClnt.AddCannedPolicy("get-only", policy); err != nil {
		log.Fatalln(err)
	}

AddUser(user string, secret string) error

Add a new user on a MinIO server.

Example

	if err = madmClnt.AddUser("newuser", "newstrongpassword"); err != nil {
		log.Fatalln(err)
	}

SetUserPolicy(user string, policyName string) error

Enable a canned policy get-only for a given user on MinIO server.

Example

	if err = madmClnt.SetUserPolicy("newuser", "get-only"); err != nil {
		log.Fatalln(err)
	}

ListUsers() (map[string]UserInfo, error)

Lists all users on MinIO server.

Example

	users, err := madmClnt.ListUsers();
    if err != nil {
		log.Fatalln(err)
	}
    for k, v := range users {
        fmt.Printf("User %s Status %s\n", k, v.Status)
    }

9. Misc operations

ServerUpdate(updateURL string) (ServerUpdateStatus, error)

Sends a update command to MinIO server, to update MinIO server to latest release. In distributed setup it updates all servers atomically.

Example

   // Updates all servers and restarts all the servers in the cluster.
   // optionally takes an updateURL, which is used to update the binary.
   us, err := madmClnt.ServerUpdate(updateURL)
   if err != nil {
       log.Fatalln(err)
   }
   if us.CurrentVersion != us.UpdatedVersion {
       log.Printf("Updated server version from %s to %s successfully", us.CurrentVersion, us.UpdatedVersion)
   }

StartProfiling(profiler string) error

Ask all nodes to start profiling using the specified profiler mode

Example

    startProfilingResults, err = madmClnt.StartProfiling("cpu")
    if err != nil {
            log.Fatalln(err)
    }
    for _, result := range startProfilingResults {
        if !result.Success {
            log.Printf("Unable to start profiling on node `%s`, reason = `%s`\n", result.NodeName, result.Error)
        } else {
            log.Printf("Profiling successfully started on node `%s`\n", result.NodeName)
        }
    }

DownloadProfilingData() ([]byte, error)

Download profiling data of all nodes in a zip format.

Example

    profilingData, err := madmClnt.DownloadProfilingData()
    if err != nil {
            log.Fatalln(err)
    }

    profilingFile, err := os.Create("/tmp/profiling-data.zip")
    if err != nil {
            log.Fatal(err)
    }

    if _, err := io.Copy(profilingFile, profilingData); err != nil {
            log.Fatal(err)
    }

    if err := profilingFile.Close(); err != nil {
            log.Fatal(err)
    }

    if err := profilingData.Close(); err != nil {
            log.Fatal(err)
    }

    log.Println("Profiling data successfully downloaded.")

11. KMS

GetKeyStatus(keyID string) (*KMSKeyStatus, error)

Requests status information about one particular KMS master key from a MinIO server. The keyID is optional and the server will use the default master key (configured via MINIO_KMS_VAULT_KEY_NAME or MINIO_KMS_MASTER_KEY) if the keyID is empty.

Example

    keyInfo, err := madmClnt.GetKeyStatus("my-minio-key")
    if err != nil {
       log.Fatalln(err)
    }
    if keyInfo.EncryptionErr != "" {
       log.Fatalf("Failed to perform encryption operation using '%s': %v\n", keyInfo.KeyID, keyInfo.EncryptionErr)
    }
    if keyInfo.UpdateErr != "" {
       log.Fatalf("Failed to perform key re-wrap operation using '%s': %v\n", keyInfo.KeyID, keyInfo.UpdateErr)
    }
    if keyInfo.DecryptionErr != "" {
       log.Fatalf("Failed to perform decryption operation using '%s': %v\n", keyInfo.KeyID, keyInfo.DecryptionErr)
    }