Marcus Kazmierczak

Home mkaz.blog

Using the Command-line

I live on the terminal. The two windows I always have open, a terminal and a browser. This is a guide to how I use the command-line for development, productivity, and daily computing tasks. This covers everything from basic navigation to using the terminal as a complete development environment. Whether you're new to the command-line or looking to level up your skills, this guide provides practical tips and real-world examples.

Setup

First up, use source control to help manage all of your configs so its easier to setup new systems and to track changes. I have a dotfiles github repo where I keep my dotfiles. I then create symlinks from the repo to the proper spot, so the repo version is always current.

For example, my .zshrc in my home directory is a symlink to the repo: ln -s ~/dotfiles/zshrc ~/.zshrc

There are a couple of programs you can use to manage dotfiles, GNU Stow and Chezmoi are two such programs, but I find it much easier to handle on my own. I have an init script which I run when setting up a system that just links them up.

Use Package Managers

For linux system this is obivous, for Macs use Homebrew. Save your self the headache, it just works and is awesome. See site for installation and details.

Use Aliases and Functions

I recommend using aliases to save time for common commands and options, save yourself typing and remembering them. For example, I would often tail a server log file, instead of remembering location (which may change per server) and typing it all out setup alog aliases to tail apache logs.

alias alog="tail -f /var/log/apache2/error.log"

If you need permission, youc an include sudo in the alias

alias alog="sudo tail -f /var/log/apache2/error.log"
alias apt='sudo aptitude'

Be observant of common tasks and commands that you do, particularly ones that you might make mistakes and create aliases for them. For example, different systems use different flags for ps command, create a common alias that works across all or combine grep within the command.

alias xps='ps -ax '
alias xpsg='ps -ax | grep -i'

If you have an alias, but want to run the normal command, put a \ in front, for example: \ls

You can use functions to use arguments in different spots or do a little more than an alias. Here's an alias which gives me the last 10 items in the current directory.

alias lsr="ls -lrt | tail -n 10"

However, that only works in the current directory, using a function you can set it up to work by passing in any directory, the $@ is the arguments passed into the function:

function lsr() { ls -lrt $@ | tail -n 10 ;}
$ lsr /etc/

So, where do you put all this good stuff? I have a main profile file which I put common pieces and since I bounce between Macs and Linux, I also have system specific profiles. Additionally, I create host specific files for each server, your mileage may vary. Here's how I source the other files from the main profile:

SYS_OS=`uname -a` # linux or mac
SHORT_HOSTNAME=`hostname -s`

# system aliases
if [[ "$SYS_OS" == 'Linux' ]]; then
    PATH="..LINUX PATH..."
    source ~/dotfiles/aliases.lx
else
    PATH="... MAC PATH..."
    source ~/dotfiles/aliases.mac
fi

# run host specific profile
if [[ -e ~/dotfiles/profile.$SHORT_HOSTNAME ]]; then
    source ~/dotfiles/profile.$SHORT_HOSTNAME
fi

A couple of quick tips on navigating around using the command-line.

cd : goes straight to home directory cd - : cd with a dash returns to previous directory

I also create the following aliases to save time moving up directories:

alias cd..: cd ..
alias cd...: cd ../../
alias cd....: cd ../../../

Zoxide is a great utility that tracks your directories and allows you to jump to any by just typing fragments. For example, you can type z dirname and it will take you directly there. For example, if you are editing files in /Users/mkaz/src/mkaz/PROJ just typing z PROJ will take you there.

You can install Zoxide using most package managers, for example brew install zoxide

There are mysterious functions called pushd, popd and dirs which use a stack to manage directories pushing them on and off the stack and changing directories as you go. These commands have been around a long time, but I've never fully groked and incorporated into my flows.

Learn Vim

Vim is a powerful text editor, there is way too much to go into in this article. Becoming efficient with vim saved me the most time over anything, especially if you do a lot of work text editing or work on remote systems.

I wrote an extensive guide Working with Vim, from introducing the basics, to screencasts and tips to make you more productive using vim.

Vim as Your Editor (Not IDE)

I use vim as a text editor, not as an IDE. The command-line is my IDE. Vim is an editor. It opens so quickly, it's no hassle to open and close it frequently. Just like the editor text pane in an IDE, if I'm editing a file it's open, if not it's closed.

For plugin management, I recommend vim-plug. Here's a section from my .vimrc to setup plugins:

call plug#begin('~/.vim/plugged')
Plug 'fatih/vim-go'
Plug 'junegunn/fzf',  { 'dir': '~/.fzf' }
Plug 'junegunn/fzf.vim'
Plug 'scrooloose/nerdcommenter'
Plug 'SirVer/ultisnips'
Plug 'tpope/vim-surround'
call plug#end()

To install plugins run :PlugInstall or to update :PlugUpdate.

Common Commands

Use ** to recurse through all sub-directories, for example to find all jpg files recursing all directories

ls -l **/*.jpg

Redirect output to a file, the > operator will redirect the output of a command to a file, creating file if needed, overwriting file if exists.

ls -1 > file-list.txt

Append output to a file, the >> operator will redirect the output of a command and append to an existing file, or create if needed

ls -lrt >> file-list.txt

I use this to create scratch files to keep random notes:

echo "quick notes" >> ~/jots/notes.md
svn commit -m "Awesome commit message" >> ~/jots/commits.log

You can also redirect to /dev/null to ignore output:

python chatty-script.py > /dev/null       # standard errors show
python chatty-script.py > /dev/null 2>&1  # errors and output to /dev/null

The pipe operator, | takes output of one command and passes it as STDIN to the next command, the pipe is the magic glue of unix.

ls -l | grep myfile

The xargs commands takes a piped in list and reverses it to be command line arguments to the next command

ls -1 | xargs touch

Now you can start combining in various ways. For example, list all files installed, grab only the ones marked as “deinstall" and pass that list into aptitude to purge

dpkg —get-selections | grep deinstall | xargs aptitude purge

This will delete all subversion files and directories, I never remember the syntax for the find command, so often pipe it into grep.

find . | grep .svn | xargs rm -rf

Changing permissions using chmod can be tedious at times, but there are a few short cuts to use besides the octal numbers. Short cuts for user, group, other and all exist. Other is not user, not group. All is all three. So you can do the following

chmod -R a+r *    # recursively give read rights to everyone
chmod -R o-r *    # recursively remove read rights from other
chmod u+rw *      # give user read + write access
chmod -R a+rX *   # sets all directories as read-executable

Text Processing

Don't open a file just to see the first or last few lines

head file.txt   # first 10 lines
tail file.txt   # last 10 lines

Or if the file is short you, cat will output it all:

cat file.txt

If I want to peek in a file I use less, because it will paginate if too big, is searchable, and easy to quit, plus no chance of editing if I'm just looking.

less file.txt

Count lines in a file: wc -l file.txt

Count words in a file: wc -w files.txt

Text Search

ripgrep (rg) has become my go-to search tool on the command line. It's a blazing fast recursive searcher, smarter than grep and more convenient than ack or ag. It automatically respects your .gitignore and is optimized for searching codebases.

Here are some of my most-used rg options and patterns:

  • Case-insensitive search: rg -i pattern (Add -i for case-insensitive matches.)

  • Invert match (show lines that do NOT match): rg -v pattern file.txt

  • Count matches per file: rg -c pattern (Shows the number of matches in each file.)

  • List only filenames with matches: rg -l pattern (Use -l to list files, or -L to list files without matches.)

  • Show lines before/after match: rg -B 2 -A 2 pattern file.txt (Shows 2 lines before and after each match.)

  • Search recursively in a specific filetype: rg pattern --type py (Only search Python files. See rg --type-list for all types.)

  • Exclude files or directories: rg pattern --glob '!*.min.js' (Exclude minified JS files.)

  • Search for a word, not a substring: rg -w word (Matches only whole words.)

  • Show line numbers (default): rg pattern (Line numbers are shown by default.)

If you used to rely on ack or ag, rg is a drop-in replacement and even faster. It ignores version control and build directories by default, so you get clean results.

Examples:

  • Find all uses of "jquery" except in JavaScript files: rg jquery --glob '!*.js'

  • Search for a function definition in PHP files: rg 'function my_func' --type php

  • List all files containing "TODO": rg -l TODO

The amazing awk, sed, cut, sort and more can be used to process text in every which way. However, I mostly find their commands cryptic and tend to use on limited basis. A couple of easier ones that I can keep track of:

Remove duplicate lines: sort -u file.txt > FILE.new

cut displays columns using -d for delimiter, and -f the column numbers to grab.

cut -d ":" -f 1 /etc/passwd    # column 1
cut -d ":" -f 1,3 /etc/passwd  # columns 1 and 4
cut -d ":" -f 4- /etc/passwd   # columns 4 to end

Substitute in file, outputting results: sed s/foo/bar/ file.txt

Substitute in file but saving to file: sed -ie s/foo/bar/ file.txt

Print lines 10 thru 20 of file: sed -n '10,20p' file.txt

However, that p argument in sed is just enough that I can't remember and I'm more likely to use head-tail by grabbing first 20 lines using head and pipe into tail which only shows last 10, not as efficient but I can remember how:

head -n 20 file.txt | tail

A few pointers on learning more about these commands:

Automate

Use loops to run repetitive tasks

You can use a loop on the command-line to do some simple tasks, an example creating a backup copy of all txt files: $ for file in *.txt ; do cp $file $file.bak; done

Also, I find myself using small bash scripts to run repetitive tasks. It is often quickest to copy-paste-edit commands in a text file, you can also add loops and logic to the script. Bash programming is pretty powerful, a little cryptic but learning a few basics can really help. See this Bash Guide for Beginners

A loop example, create a file called loop.sh with the following:

for VARIABLE in file1 file2 file3
do
    echo $VARIABLE
done

Run using: $ bash loop.sh

Use cron to schedule tasks Use cron to schedule routine tasks, such as backups, updates or other scripts you may want to run on a set basis. Cron is pretty powerful and can run things on just about an schedule, for example hourly, every other day, first monday of month, etc… I have a tutorial on how to use unix crontab

Time an Event If you want to see how long a script takes to run, you can use time. You simply type it before your command or script. time sleep 5

Delay an Event There is a sleep command, which I used above which allows you to delay something to be run, for example a screenshot, reminder, or something else you may want to run shortly, number in seconds. sleep 5; echo "Foo"

Unix Utilities

Built-in Utilities

Use rename to batch rename a set of files, it uses a regex for syntax rename "s/JPG/jpg/" *.JPG

Rename files with spaces to dashes: rename "y/ /-/" *

The utility shuf can grab random lines from a file. This example will grab a single random line shuf -n 1 quotes.txt

The cal utility prints out a calendar, specify a year to print a full year. For example: cal 2013, cal 1970, cal 1776

$ cal
   December 2013
Su Mo Tu We Th Fr Sa
 1  2  3  4  5  6  7
 8  9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31

Use curl for all things HTTP, testing, debugging sites. The flag I probably use most is -I to dump the headers from a call. For example: curl -I https://wordpress.com/ See the Art of HTTP Scripting and Fun with Curl for more great curl tips.

You can use pass as a good password mangaer, or use pwgen as a nice utility to generate passwords, for example pwgen 32

Use pandoc to convert text files In the past year or so I discovered the pandoc utility which can convert numerous text file formats, it works great. For example if you want to convert Markdown to HTML. I also use it often to convert Markdown to slides. It also supports PDF, ebooks and many other, see pandoc site for supported formats.

Rsync for Copying Files Use rsync as a deployment tool by copying files to an external server. For example: rsync -avzC -e ssh ./htdocs/ mkaz@mkaz.com:/sites/mkaz.com/htdocs will copy the htdocs directory to the remote server mkaz.com. Rsync is pretty powerful tool, it is also commonly used as backup tool. See these rsync examples for more.

Modern Utilities

Use starship to configure your prompt Starship is a powerful tool that allows you to customize your prompt with numerous integrations, all far easier than setting PS1 vars.

Use fd as a better find The fd utility is a simple tool to find files, it is fast and much easier to use than the classic find.

Use fuzzy finders for better ux The fzf and fzy utilities are powerful tools that allows fuzzy finding to search large lists easily. It is useful on the command-line and integrates with numerous tools. Fzy is a slightly newer version, worth

Use htop instead of top Use htop for a better interface into top processes running and cpu usage, probably not installed by default but in most package repositories. Setup an alias and you won't have to remember to type it. alias top=htop

Use eza as a better ls Use eza as a better way to list files, easier color coding and more power. Add alias ls=eza so your muscle memory doesn't have to change.

Development Workflows: Unix as IDE

The command-line provides all the features of an integrated development environment (IDE) such as PHPStorm, Eclipse, or Netbeans. Common IDE features include text editor, project search/replace, automatic building/testing/linting, and source control integration - the command-line handles all of these.

Build, Test, Lint

Building, testing, and linting are straightforward on the command-line since these are typically command-line tools that IDEs abstract away. Working directly with the tools removes abstraction layers and prevents errors.

While Makefiles are the traditional standard, I recommend using a justfile for project automation. just is a modern command runner with better defaults, easier syntax, and a more pleasant experience than Make. It works cross-platform, is easy to install, and is rapidly becoming the tool of choice for scripting project tasks.

build:
    npm run build

test:
    npm test

lint:
    eslint src/

Then you can run just build, just test, or just lint consistently across projects.

Command-line Options and Tips

Create directories all the way down, use -p and it will create all the in-between directories as needed. mkdir -p ~/Documents/dir1/dir2/dir3

If you want this to be always the case: alias mkdir="mkdir -p"

Unzip and Extract in one command:

tar xvfz tarball.tar.gz
tar xvfj tarball.tar.bz2

The !! command will run the previous command, by itself it isn't that useful since an up arrow will show previous command, but combined with sudo you can run previous command with sudo permission like so: sudo !!

For vim, add this to your vimrc, allow you to use :w!! to save using sudo

" sudo write
ca w!! w !sudo tee >/dev/null "%"

Use ctrl-r on the command-line and start typing to search back for previous commands

A clever idea with ctrl-r is to use comments to tag a command, making it easier to search in the future. So when I type the original command like so: dd if=debian.img of=/dev/disk2 bs=1m # bootable-usb

I can use ctrl-r and search on bootable-usb and it will bring it back, takes a little foresight that you'll want to search for the command again.

You can use { and } to specify a range of items for example

$ echo {one,two,three}-alligator
one-alligator two-alligator three-alligator

This is useful in various ways to save running the same command with a minor difference, I use this often when creating a set of directories, the following creates five directories: mkdir -p app/storage/{cache,logs,meta,sessions,views}

You can use !(pattern) to do something with everything except, so for example if you want to delete all files except .txt files: rm !(*.txt)

Use SSH Config and SSH Keys If you log in to various servers often, use ssh config for those servers, you can configure user and shortcuts to make it easier. For example, ssh -l marcus.kaz remote.server.location.com can turn into just ssh remote. See this article Simplify Your Life with an SSH Config file

Here's an article on how to setup ssh keys and use ssh-agent. Using ssh keys can be more secure than basic passwords and using ssh-agent makes using ssh keys easier by caching your password, so you don't have to retype frequently.

Task Management

Todo.txt: A simple system developed by Gina Trapani using plain text files to track todo lists with a todo.sh script to manage it all. Available for numerous platforms at todotxt.com.

Task Warrior: A more feature-rich command-line todo manager at taskwarrior.org with many add-ons and third-party apps. The vit curses-based front-end is particularly nice.

My own tasks management tool that creates a TUI kanban board.

Send Email from Command-line

Something I don't use often enough is sending e-mail from the command-line, I think because I never know if an environment is configured to send mail. It's always a bit tricky, here's a guide setting up msmtp to send email using using GMail as the transport. Another alternate mail agent is ssmtp which can also use Gmail as a transport.

Once mail is configured you can use mail or mailx command to send email, depending on your system, both work the same. The commands take the body of the message as STDIN (piped in), which makes it real easy to send results of a command via email:

For example:

command  | mail -s "subject" user@domain.com

To send the contents of a file as the body message use:

mail -s "subject" user@domain.com < file.txt

To send a file as an attachment:

echo "Attached" | mail -s "subject" -a file.txt user@domain.com

Presentations

Use MARP: Markdown Presentations to create slide decks using just Markdown. You write your slides in a single Markdown file, using --- to separate slides, and MARP converts it into HTML, PDF, or PowerPoint presentations.

High-level workflow:

Step 1: Install MARP CLI:

You can install MARP globally with npm: npm install -g @marp-team/marp-cli

Step 2: Write your slides:

Create a Markdown file (e.g., slides.md). Use --- to separate slides:

``` # My Presentation

## Slide 2

  • Bullet
  • Points ```

Step 3: Convert to presentation:

Generate an HTML or PDF with: marp slides.md --html marp slides.md --pdf

Step 4: Themes and customization:

MARP supports themes, custom CSS, and speaker notes.

For a step-by-step guide and advanced features, see my Markdown Slides with MARP article.