Golang Notes

This will encompass things that I find useful and end up looking up later when I haven't written go in a while.

Installation

Install on linux

wget https://golang.org/dl/go1.15.2.linux-amd64.tar.gz
tar -C /usr/local -xzf go1.15.2.linux-amd64.tar.gz
echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc
source ~/.bashrc
go version

Resource: https://golang.org/doc/install

Start New Project

This will create the project and then get any dependencies you've defined in any go code you've put in place:

go mod init github.com/<git account name>/<project name>
go get -v

Resource: https://golang.org/doc/code.html

Concepts

functionName vs FunctionName

A function that starts with a lowercase letter is only available within the package in which it's defined.

A function that start with an uppercase letter is available to any packages in your program.

Resource:
https://stackoverflow.com/questions/38616687/which-way-to-name-a-function-in-go-camelcase-or-semi-camelcase

Compile binary

go build

String Interpolation

var a string = "Hello"
var b string = "World"
c := fmt.Sprintf("%s %s!", a, b)
fmt.Println(c)

Debugging

Start debugger:
dlv debug

Set breakpoint on line 36 in main.go:
break main.go:36

Set breakpoint on line 8 in anotherGoFile.go:
break anotherGoFile.go:8

Debug program with command line arguments (after compiling)
dlv debug -- -a argument -b another_argument

Print contents of the b variable
print b

Restart the debugging process
r

Resources:
https://github.com/derekparker/delve/issues/178
https://www.youtube.com/watch?v=zgLjVD5ZSOc

Useful Functions

Remove empty strings trailing in a slice

func RemoveTrailingEmptyStringsInStringArray(sa []string) []string {
  lastNonEmptyStringIndex := len(sa) - 1
  for i := lastNonEmptyStringIndex; i >= 0; i-- {
    if sa[i] == "" {
      lastNonEmptyStringIndex--
    } else {
      break
    }
  }
  return sa[0 : lastNonEmptyStringIndex+1]
}

Read an input file from a specified path

func readLines(filePath string) ([]string, error) {
  b, err := ioutil.ReadFile(filePath)
  if err != nil {
    return nil, err
  }

  return RemoveTrailingEmptyStringsInStringArray(strings.Split(string(b), "\n")), nil
}

Compiling binaries

OSX 64-bit:

env GOOS=darwin GOARCH=amd64 go build -o osx

Linux 64-bit:

env GOOS=linux GOARCH=amd64 go build -o linux

Resources:
https://www.digitalocean.com/community/tutorials/how-to-build-go-executables-for-multiple-platforms-on-ubuntu-16-04
https://stackoverflow.com/questions/42706246/how-to-build-executable-with-name-other-than-golang-package

Return multiple values from function

You can natively return multiple values in go. Here's a basic function example (specifcally the declaration and return):

func greatFunc(param1 string, param2 string, param3 string) (bool, string) {
// body omitted
return true, returnString

Resource:
https://gobyexample.com/multiple-return-values

Multiline string

    var payload = `  include myfile
}`

Resource:
https://play.golang.org/p/kcupXFwzgrZ

Comments

Normal

// comment

Block

/*
comment
*/

Convert byte array to string

s := string(byteArray)

Resource:
https://stackoverflow.com/questions/14230145/what-is-the-best-way-to-convert-byte-array-to-string

Nested directory creation

err := os.MkdirAll("/tmp/foo/bar", os.ModePerm)

Resource:
https://stackoverflow.com/questions/28448543/how-to-create-nested-directories-using-mkdir-in-golang

Convert int to str

import "strconv"
strconv.Itoa(123) // "123"

Resource: https://stackoverflow.com/questions/10105935/how-to-convert-an-int-value-to-string-in-go

Check if string contains another string

import (
    "strings"
)

var var2 string = "bla"
output := strings.Contains("blablabla", var2) 
fmt.Println(output) // true

Resource: https://stackoverflow.com/questions/45266784/go-test-string-contains-substring

Filepath operations

import "fmt"
import "path/filepath"

func main() {
	t := "/etc/init.d/somefile"
	fmt.Printf("base is %q dir is %q is it absolute? %v\n",
		filepath.Base(t),
		filepath.Dir(t),
		filepath.IsAbs(t))
	d, f := filepath.Split(t)

	fmt.Printf("split returns file %q dir %q\n", f, d)
        fmt.Printf("clean it up %q", filepath.Clean("a/b/c/../d//////../../f"))
}

If you need to return a filepath:

d := "/etc/bla/bla2/bla3/"
filepath.Clean(filepath.Join(d, "../")) // /etc/bla/bla2

Get the directory of a path

fmt.Println(filepath.Dir("/root/.ansible/site.yml"))

Playground: https://play.golang.org/p/nlcH1G3x5Vs

Get the last element of a path

base := filepath.Base("/home/dennis/IdeaProjects/Playground/hello.go")
fmt.Println("Base:", base)
# returns hello.go

Resource: https://ispycode.com/GO/Path-and-Filepath/Last-element-of-path

Read template file

The person that put this together is a hero (Gustavo Bittencourt).

package main

import (
  "log"
  "os"
  "path"
  "text/template"
)

type Command struct {
  ClassName string
  CmdName   string
  Cmd       string
}

func main() {
  command := Command{
    ClassName: "my_cmd",
    CmdName:   "cmd",
    Cmd:       "curl target.com:8001/asdf",
  }

  t := template.New("puppetModule.tmpl")

  t, err := t.ParseFiles(path.Join("templates", "puppetModule.tmpl"))

  if err != nil {
    log.Fatal("Parse: ", err)
  }

  err = t.Execute(os.Stdout, command)
  if err != nil {
    log.Fatal("Execute: ", err)
  }
}

puppetModule.tmpl

class {{.ClassName}} {
  exec { '{{.CmdName}}':
    path => '/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin',
    command => '{{.Cmd}}'
  }
}

Resource:
https://stackoverflow.com/questions/32388777/error-when-creating-a-template-and-then-parsing-from-files

Go get cert errors fix

Obviously, this is not ideal (observe the -insecure flag):

go get -u -v -insecure <target project>

From go help get:
-u: Look for updates to existing packages
-v: Verbose progress and debug output

Resource:
https://github.com/golang/go/issues/18519

Generate random int in a range

import (
  "fmt"
  "math/rand"
  "time"
)

func randInt(min int, max int) int {
  return min + rand.Intn(max-min)
}

func main() {
  start := 1
  finish := 6
  rand.Seed(time.Now().UTC().UnixNano())
  fmt.Printf("%v", rand.Intn(finish-start)+start)
}

Resources:
https://stackoverflow.com/questions/44659343/how-to-choose-a-random-number-from-the-range
https://golang.cafe/blog/golang-random-number-generator.html

Get lowercase version of string

import "strings"
strings.ToLower("Gopher")

Resource:
https://stackoverflow.com/questions/10411538/how-do-i-convert-a-string-to-a-lower-case-representation

Copy file

import (
  "io/ioutil"
  "log"
)

 func CpFile(src string, dst string) {
   input, err := ioutil.ReadFile(src)
   if err != nil {
     log.Println(err)
     return
   }

   err = ioutil.WriteFile(dst, input, 0644)
   if err != nil {
     log.Println("Error creating", dst)
     log.Println(err)
     return
   }
 }

Resource:
https://opensource.com/article/18/6/copying-files-go

Determine OS being used

package main

import (
	"fmt"
	"runtime"
)

func main() {
	if runtime.GOOS == "linux" {
		fmt.Println("You are running on Linux")
	} else if runtime.GOOS == "darwin" {
		fmt.Println("You are running on OSX")
	} else if runtime.GOOS == "windows" {
		fmt.Println("You are running on Windows")
	} else {
		fmt.Println("Unsupported OS detected")
	}
}

Playground:
https://play.golang.org/p/ULd8sMpy0dt
Resources:
https://golangcode.com/detect-if-code-is-running-on-windows-at-runtime/

Go get private repo

git config --global url."git@github.com:".insteadOf "https://github.com/"

Resource:
https://bialon.net/post/how-to-go-get-private-repos/

Get Working Directory

func Gwd() string {
	dir, err := os.Getwd()
	if err != nil {
		log.Fatalln(err)
	}
	return dir
}

Resource:
https://gist.github.com/arxdsilva/4f73d6b89c9eac93d4ac887521121120

String to String Array

cmd := "puppetserver ca list --all"
strArr := strings.Fields(cmd)
fmt.Println(strArr[0])
// Will print: puppetserver

Resource:
https://stackoverflow.com/questions/13737745/split-a-string-on-whitespace-in-go

Get everything but first element of array

range Arr[1:]

Resource:
https://stackoverflow.com/questions/37429248/skip-first-element-of-an-array-in-golang

Convert string array to string

strings.Join(arr []string, seperator string) string

Resource:
https://stackoverflow.com/questions/41756412/golang-convert-type-string-to-string

Install a specific binary version

go get github.com/some/tool@v1.0.1

Resource:
https://stackoverflow.com/questions/53368187/go-modules-installing-go-tools

Unit Testing

Run tests without caching

As of 1.12, you can no longer use GOCACHE=off, i.e.

GOCACHE=off go test -v -race ./...

However, you can use -count=1. For example:

go test -count=1 -v -race ./...

Resources:
https://github.com/golang/go/issues/26809
https://github.com/golang/go/issues/22758

Unit test unexported functions

An unexported function aka a private function requires a little hack to write unit tests for. The steps are as follows:

  1. Create a export_test.go file in which you will create an exported variable that points to the unexported function. For example:
package <package in which the unexported function exists>
var Backup = backup
  1. Next, create your normal _test file and be sure to reference the exported variable you created in export_test.go

Resource:
https://medium.com/@robiplus/golang-trick-export-for-test-aa16cbd7b8cd

Check if string is in a slice

func stringInSlice(s string, slice []string) bool {
	for _, v := range slice {
		if s == v {
			return true
		}
	}
	return false
}

Example usage:

if stringInSlice("bla", blaSlice) {
   fmt.Println("Gotem!")
}

Playground: https://play.golang.org/p/NoucaeldZoO

Go Mod

Create new go.mod file:

go mod init

Add module requirements for all packages

go mod tidy

Resources:
https://blog.golang.org/migrating-to-go-modules
https://medium.com/@fonseka.live/getting-started-with-go-modules-b3dac652066d

Slice

A slice is essentially an array in Golang.

Convert bytes to a string

s := string([]byte{65, 66, 67, 226, 130, 172})
fmt.Println(s) // ABC€

Resource: https://yourbasic.org/golang/convert-string-to-byte-slice/

Output to file

m := "stuff to write in file"
err = ioutil.WriteFile("./file.txt", m, 0644)
if err != nil {
	log.Fatalf("error: %v", err)
}

Resource: https://gobyexample.com/writing-files

Unmarshal yaml

Begin by using https://mengzhuo.github.io/yaml-to-go/ to create a struct for the yaml file, for example:

type ansible []struct {
	Name   string   `yaml:"name"`
	Hosts  string   `yaml:"hosts"`
	Become bool     `yaml:"become"`
	Roles  []string `yaml:"roles"`
}

Once you've done that, read the yaml file and unmarshal it into the struct:

b, err := ioutil.ReadFile("./ansible.yml")
if err != nil {
	log.Fatal(err)
}

d := ansible{}
err = yaml.Unmarshal(b, &d)
if err != nil {
	log.Fatal(err)
}
log.Printf("%v", d)

If you make modifications to the yaml, marshal it and write it back to the file:

d[0].Roles = append(d[0].Roles, "hello2")
log.Printf("%v", d)

m, err := yaml.Marshal(&d)
if err != nil {
	log.Fatalf("error: %v", err)
}

err = ioutil.WriteFile("./ansible_modified.yml", m, 0644)
if err != nil {
	log.Fatalf("error: %v", err)
}

Resources:
https://github.com/go-yaml/yaml
https://gist.github.com/ka2n/4457eacdb6c986624eb29cc02fe8d31c
https://play.golang.org/p/nYZCWIi2hxa

Multi line variable assignment

var data2 = `
---
- hosts: all
  remote_user: root
  roles:
  - common

- hosts: lb_servers
  remote_user: root
  roles:
  - lb-nginx

- hosts: backend_servers
  remote_user: root
  roles:
  - tomcat

- hosts: memcached_servers
  remote_user: root
  roles:
  - memcached
`

Resource: https://github.com/ansible/ansible-examples/blob/master/tomcat-memcached-failover/site.yml

Set

// Create new empty set
set := make(map[string]bool)

// Add new element to the set
set["bla"] = true

// Print the elements of the set
for k := range set {
    fmt.Println(k)
}

// Add item to set if it's not already there
if !set["bla"] {
    set["bla"] = true
}

Resources:
https://yourbasic.org/golang/implement-set/
https://play.golang.org/p/T2TR_Aib-H7

Regular expressions

Check if string matches regex

file := "ansible.cfg.old"
matched, _ := regexp.MatchString(`ansible.cfg$`, file)
// Return false
fmt.Println(matched)

Resources:
https://yourbasic.org/golang/regexp-cheat-sheet/
https://play.golang.org/p/UUqSQHCqMsZ

Check if string is valid hostname

This will match valid hostnames as per RFC 1123.

line := "hostname.example.com"
validHostname, _ := regexp.MatchString(`^(([a-zA-Z0-9]|[a-zA-Z0-9][a-zA-Z0-9\-]*[a-zA-Z0-9])\.)*([A-Za-z0-9]|[A-Za-z0-9][A-Za-z0-9\-]*[A-Za-z0-9])$`, line)
if validHostname {
    fmt.Println(line)
}

Resources:
https://stackoverflow.com/questions/106179/regular-expression-to-match-dns-hostname-or-ip-address
https://play.golang.org/p/Mhckhn4jj3r

Print go representation of a value

This is very helpful if you're trying to do something like copying a big slice into the The Go Playground, or want to get a sense of how something looks in go.

fmt.Printf("%#v", bigSlice)

Resource: https://stackoverflow.com/questions/24489384/how-to-print-the-values-of-slices

Convert relative to absolute path

// Operating from /home/user/dir
absFilePath = ""
path, err := filepath.Abs(Cli.FilePath)
if err != nil {
    log.Printf("Error getting absolute path from %s", path)
}

absFilePath = path

Run goscorecard locally

Get the CLI:

go get github.com/gojp/goreportcard/cmd/goreportcard-cli

Navigate to the directory you want to run it on, and then run:

goreportcard-cli -v

Resource: https://github.com/gojp/goreportcard

Validate input emails

package validators

import (
	"fmt"
	"net"
	"regexp"
	"strings"
)

// IsEmailValid checks an input email to determine
// if it is a valid email address.
// Code was found at: https://golangcode.com/validate-an-email-address/
func IsEmailValid(email string) bool {
	var emailRegex = regexp.MustCompile("^[a-zA-Z0-9.!#$%&'*+\\/=?^_`{|}~-]+@[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?(?:\\.[a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?)*$")

	fmt.Printf("Validating email: %s\n", email)

	if len(email) < 3 && len(email) > 254 {
		return false
	}
	if !emailRegex.MatchString(email) {
		return false
	}
	parts := strings.Split(email, "@")
	mx, err := net.LookupMX(parts[1])
	if err != nil || len(mx) == 0 {
    	fmt.Printf("Error, invalid email: %s", email)
		return false
	}
	return true
}

Resource: https://golangcode.com/validate-an-email-address/

Get the type of an object

var x interface{} = r.FormValue("email")
xType := fmt.Sprintf("%T", x)
fmt.Println(xType) // "string"

Resource: https://yourbasic.org/golang/find-type-of-object/

Response Codes

Can be found here: https://golang.org/src/net/http/status.go

Show supported functions

fooType := reflect.TypeOf(result)
for i := 0; i < fooType.NumMethod(); i++ {
    method := fooType.Method(i)
    fmt.Println(method.Name)
}

Resource: https://stackoverflow.com/questions/21397653/how-to-dump-methods-of-structs-in-golang