Linux Cheatsheet

Highly recommended: tldr-pages/tldr (website)

collection of community-maintained help pages for command-line tools, that aims to be a simpler, more approachable complement to traditional man pages.

Linux Cheatsheet

OS-specific

CentOS

Yum Command Cheat Sheet

Ubuntu

Unattended reboot

sudo editor /etc/gdm3/custom.conf

# Uncomment the following lines and change user1 to the value of $USER:
#   AutomaticLoginEnable = true
#   AutomaticLogin = user1

Debian Unattended Upgrades

Reinstall Unity on Ubuntu

sudo apt-get update
sudo apt-get install --reinstall ubuntu-desktop
sudo apt-get install --reinstall unity

Fix mesg: ttyname failed: Inappropriate ioctl for device

Comment out mesg n || true in /root/.profile:
```
# mesg n || true
test -t 0 && mesg n
```
Source

System-wide configuration

System Services

Print service definition
```
systemctl cat SERVICE_NAME.service
```

Logs

Recover logs from failed /etc/fstab during boot

Search by date
```
journalctl --since today
```

Search by keyword

grep --ignore-case --regexp=KEYWORD --files-with-matches --dereference-recursive /var/log 2> /dev/null

Search by file

journalctl --file /var/log/FILENAME.journal

Networking

Connect to network on interface eth0 on boot

sudo systemctl status network
sudo editor /etc/sysconfig/network-scripts/ifcfg-eth0
# Set ONBOOT=yes for desired network

Set DNS server

editor /etc/resolv.conf
# Add DNS server
#   Example:
#     nameserver 8.8.8.8

Source

List kernel routing tables
```
route --numeric
```

SSH

Generate SSH key

# -t specifies the type of key to generate
# -C provides a comment for the key
ssh-keygen -t ed25519 -C "[email protected]"

Source

Test passphrase for SSH key

# -y "This option will read a private OpenSSH format file and print an OpenSSH public key to stdout."
ssh-keygen -y -f <your_private_ssh_key>

Regenerate remote host identification entry in known_hosts file
```
ssh-keygen -f "~/.ssh/known_hosts" -R <IP_ADDRESS>
```
SSH Message Numbers

Examples:

send packet: type 50 receive packet: type 51

Env vars

Given a file named example.env with contents:

a='jane'
b='john'
c='doe'

Source all variables:
```
source example.env
```

Export all variables in file

# After sourcing all variables, we can export them
export $(cut --delimiter= --fields=1 example.env)

Source

Filesystem

Filesystem Hierarchy Standard (FHS)

The Filesystem Hierarchy Standard (FHS) defines the directory structure and directory contents in Linux distributions. Source

List 5 largest files in folder and sort by size

du --human-readable --summarize * | sort --reverse --human-numeric-sort | head --lines 5

Report file system disk space usage
```
df --human-readable
```

Create filesystem on external disk

# Create filesystem
mkfs.ext4 /dev/vdb

# Create mountpoint directory
mkdir --parents /data

# Mount external disk to /data
mount /dev/vdb /data

Mount from /etc/fstab
```
mount --all
```
List all partitions and their Universally Unique Identifier (UUID)
```
sudo blkid
```

Create sequence of numbered directories

# Creates directories 1/ ... 10/
mkdir -p $(seq 1 10)

Peripherals

Cheatsheet from opensource.com: Linux commands to display your hardware information

As a general rule device shown in /dev/sd* are storage devices as opposed to the ones shown in /dev/bus.

List connected system devices
```
find /dev/sd*
```
List available USB ports
- Simple
```
find /dev/bus/
```
- Advanced
```
ls /sys/bus/usb/devices/*
```
Display info for specific USB device
```
sudo lsusb -D /dev/bus/usb/001/005
```
List block devices
```
lsblk
```
List the partition tables for the specified devices
```
sudo fdisk --list
```
List hardware
```
sudo lshw
```

List hardware (only input devices)

sudo lshw -class disk -class input -short

List USB devices
- usb-devices
  
  usb-devices is a (bash) shell script that can be used to display details of USB buses in the system and the devices connected to them.
```
usb-devices
```
  Source
- lsusb
  
  lsusb is a utility for displaying information about USB buses in the system and the devices connected to them
```
lsusb
```
  Source
- usbview
  
  usbview provides a graphical summary of USB devices connected to the system. Detailed information may be displayed by selecting individual devices in the tree display.
```
usbview
```
  Source

Default editor

Set default editor for sudo systemctl edit --full SERVICE_NAME
```
sudo update-alternatives --config editor
```

Disk Space

Check available disk space

df --human-readable / | awk 'END{print $4}' | cut --delimiter='G' --fields=1

RAM

Check available RAM

free --total --gibi | awk '/Total:/ { print $2 }'

Swap Memory

Configure swap memory (run as root user)

configure_swap_memory() {
  local -r swap_file_path="$1"
  local -r swap_memory_size_in_gb="$2"

  set -o errexit
  set -o pipefail
  set -o nounset

  echo 'Disable all swap memory temporarily'
  swapoff --all

  # ATTENTION: fallocate is faster than dd but some systems require dd usage.
  # If the fallocate command below fails, use the following instead:
  #
  # size=$((swap_memory_size_in_gb * 1024))
  # sudo dd if=/dev/zero of="$swap_file_path" count=$size bs=1MiB
  #
  echo "Create $swap_file_path of size ${swap_memory_size_in_gb}GB"
  fallocate --length "${swap_memory_size_in_gb}G" "$swap_file_path"

  echo "Set up a Linux swap area in file $swap_file_path"
  mkswap "$swap_file_path"

  echo "Enable file $swap_file_path for paging and swapping"
  swapon "$swap_file_path"

  echo "Setting file permissions to 0600 of $swap_file_path"
  chmod 600 "$swap_file_path"

  echo 'Verify increased size of swap memory'
  grep SwapTotal /proc/meminfo
}

configure_swap_memory '/swapfile' 8

Recommended size of swap space

User-specific configuration

Set locale

Follow the instructions below to fix the following warnings:

perl: warning: Setting locale failed.

perl: warning: Please check that your locale settings:

LANGUAGE = (unset),

LC_ALL = (unset),

LC_CTYPE = "UTF-8",

LANG = "en_US.UTF-8"

are supported and installed on your system.

perl: warning: Falling back to the standard locale ("C").

Append the following lines to ~/.bashrc:
```
export LANGUAGE=en_US.UTF-8
export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8
export LC_CTYPE=en_US.UTF-8
```
Preserve env when executing sudo
```
sudo --preserve-env <command>
```

Print non-root username

# Returns name of user with UID 1000 (defaults to non-root user)
uid=1000
getent passwd "$uid" | cut -d: -f1

Source

Alternatively:

# "SUDO_USER Set to the login name of the user who invoked sudo."
echo $SUDO_USER

Source

Git

Mark all subdirectories as safe in git

# ls options
# -1 list one file per line
#
# xargs options
# -L max. lines
\ls -1 --directory $PWD/. | xargs -L 1 --delimiter='\n' git config --global --add safe.directory

Delete local and remote tag

# Delete local tag
git tag --delete <TAG_NAME>
# Delete remote tag
git push --delete origin <TAG_NAME>

Rename local and remote branch

# Placeholders: old_name new_name

# If old_name not checked out already
git checkout old_name

# Rename local branch
git branch -m old_name new_name

# Delete old_name remote branch and push new_name local branch
git push origin :old_name new_name

# Reset upstream branch fo new_name local branch
git push origin --set-upstream new_name

Source

List changed files on branch with regard to main branch

git --no-pager diff --name-only FETCH_HEAD $(git merge-base FETCH_HEAD main)

Source

Improve performance of git clone

Read GitHub's performance comparison for git clone and introduction to shallow and partial clone
Solve merge conflict
```
git mergetool
```
Check performance metrics and recommendations for git repo using git-sizer
```
git-sizer --verbose
```

List biggest files in .git folder

git rev-list --objects --all \
| git cat-file --batch-check='%(objecttype) %(objectname) %(objectsize) %(rest)' \
| sed --quiet 's/^blob //p' \
| sort --numeric-sort --key=2 \
| tail --lines 10 \
| cut --characters=1-12,41- \
| $(command -v gnumfmt || echo numfmt) --field=2 --to=iec-i --suffix=B --padding=7 --round=nearest

Source

Git pull all repos in a directory

find . -type d -depth 1 -exec git --git-dir={}/.git --work-tree=$PWD/{} pull \;

Source

GPG

List all keys in long-format

gpg --list-secret-keys --keyid-format LONG

Generate a new GPG key pair

gpg --full-generate-key
# Follow instructions from link below

Source

Delete GPG key pair

# Delete secret key
gpg --delete-secret-key <key-ID>
# Delete public key
gpg --delete-key <key-ID>

Renew expired yarn GPG key

curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -

Source

Docker

Watch status of Docker containers

You may need to install the watch command depending on your Linux distribution.
```
# Interval unit is seconds
watch --interval 1 'docker ps -a'
```

Enter a crashing Docker container

docker commit <container_id> my-broken-container
docker run --interactive --tty my-broken-container /bin/bash

Check cause of container crash

dmesg --ctime | grep --extended-regexp --ignore-case --before-context=100 'killed process'

Find absolute path to Docker volume in file system

docker volume inspect --format '{{ .Mountpoint }}' VOLUME_NAME

Search local Docker images by name

# Replace placeholder IMAGENAME by Docker image name you are searching for
docker images '*IMAGENAME*'

Docker in Docker (DinD)

The trick is to mount /var/run/docker.sock as a volume. The Docker container can then access Docker on the host.

Source

Update a Docker image

The following image uses the image my_image:my_tag as a placeholder.

# Replace placeholder image_name with the name of the image you want to update
image_name=my_image:my_tag
container_name=update-image-container

# Start a container with the image to update
docker run --rm --tty --detach --name "$container_name" "$image_name"

# Log into the image to update it
docker exec --interactive --tty "$container_name" bash
# Update the image (the following command is executed inside the image). Example:
touch /tmp/new_file.txt
# Exit the image
exit

# Commit the image, i.e., create a new image from the container’s changes
docker commit "$container_name" "$image_name"

docker stop "$container_name"

Image Manipulation

Source

Horizontally append images

  convert *.png +append horizontal-image.png

Vertically append images

  convert *.png -append vertical-image.png

Create image with white background
```
convert -size 32x32 xc:white empty.jpg
```

Replace alpha channel with white background in all PNG images in nested directories

for i in `ls -1 **/*.png`; do convert $i -background white -alpha remove -alpha off $i; done

Source

Export multiple PNG images to PDF using img2pdf

# Enter directory containing PNGs
cd directory-with-images/

# Remove alpha from images and convert colorspace to RGB (img2pdf does not support ICC)
for i in *.png; convert $i -colorspace rgb -alpha off $i; end

# Merge images into PDF
# (Optional) Add --pagesize A4 to normalize page size
img2pdf --output merged.pdf *.png

# (Optional) To add OCR text layer, execute the following line
# ocrmypdf --force-ocr merged.pdf merged.pdf

PDF manipulation

Add OCR text layer to PDF using ocrmypdf

ocrmypdf --force-ocr input.pdf output.pdf

Merge PDFs
```
pdfunite in-1.pdf in-2.pdf out.pdf
```
Reduce PDF file size

Requires gs and shrinkpdf installation
```
shrinkpdf in.pdf out.pdf
```
Search for text in PDF using pdfgrep
```
pdfgrep pattern file.pdf
```

Optimization

Parallel

Run command in parallel using GNU parallel

parallel --halt-on-error now,fail=1 'set -o errexit; set -o pipefail; set -o nounset; echo {}' ::: 1 2 3

Parallel manual

Time

Schedule command execution at specific time and date
```
at hh:mm
```

Store output of time command in variable without output of argument

# Format output of time command
# Source: https://www.gnu.org/savannah-checkouts/gnu/bash/manual/bash.html#index-TIMEFORMAT
TIMEFORMAT='Elapsed time in seconds: %lR'

# Quoting from http://mywiki.wooledge.org/BashFAQ/032:
#   Keep stdout untouched.
#   The shell's original file descriptor (FD) 1 is saved in FD 3, which is inherited by the subshell.
#   Inside the innermost block, we send the time command's stdout to FD 3.
exec 3>&1
# Captures stderr and time.
elapsed_time=$( { time ls 1>&3; } 2>&1 )
exec 3>&-
echo "$elapsed_time"

Source

Copy dirs and files

Base rsync options

rsync --archive --compress --verbose --human-readable --progress [PATH] [PATH]

Display overall progress in rsync

# Use rsync options --info=progress2 and --no-inc-recursive
# Example:
cd /tmp/bar && find . -type -f | parallel --halt-on-error now,fail=1 -X rsync --relative--no-inc-recursive --info=progress2 --human-readable './{}' /tmp/bar/ ; }

Transfer large files efficiently

# Options
# WARNING: Add --compress to RSYNC_OPTS only if transferring over a slow connection.
# For local transfers --compress actually slows down the operation
readonly RSYNC_OPTS=(--hard-links --archive --relative --partial '--info=progress2' --human-readable)
readonly RSYNC_EXCLUDES=(--exclude=archive
  --exclude=.git
  --exclude=.gitignore
  --exclude=.idea
  --exclude=.vagrant
  --exclude=__pycache__
  --exclude=*.swp
  --exclude=.vscode)
  # 0 jobs in parallel translates to as many as possible
  readonly NUMBER_OF_JOBS=0

cd src-dir && find . -type f | parallel --jobs "$NUMBER_OF_JOBS" --halt-on-error now,fail=1 -X rsync "${RSYNC_OPTS[@]}" "${RSYNC_EXCLUDES[@]}" ./{} dest-dir/

Source

String manipulation

Replace characters in filenames found in current working directory

# Replace all occurrences of underscores in filenames of files in current working directory with a hyphen
for f in *; do mv $f ${f//_/-}; done

Shell Scripting

Script template:

Robust and portable shebang
Recommended bash/shell options for scripts
Inherit shell options in subshells

Example SHELLOPTS docs

#!/usr/bin/env bash
#
# Script description

# Bash options
# Inherit errexit in subshells (requires Bash 4.4+)
shopt -s inherit_errexit

# Shell options
set -o errexit
set -o pipefail
set -o nounset

# Export enabled shell options to subshells
# Documentation:
# https://www.gnu.org/software/bash/manual/html_node/Bash-Variables.html
export SHELLOPTS

Bash FAQ
Bash Study Guide
Bash variables

Conditionally pass arguments to command

declare conditional_rename=()
if [ true ]; then
  # Including value of variable in case it's not empty
  conditional_rename=("${conditional_rename[@]}" --before-context=3)
fi

# Converts to grep --before-context=3 'sh' readme.md
grep "${conditional_rename[@]}" 'sh' readme.md

rodrigobdz / linux-cheatsheet Goto Github PK