The Unix Commandline

Pardon the dust. Work in progress.

This page is intended to give you a primer on using the Unix command line to make doing science efficient and productive.

SSH

Why use Unix tools? Modularity and the Unix philosophy

Heading
  • markdown list 1
    • nested list 1
    • nested list 2
  • markdown list 2

Tale of a shell

shell ? 

shell environment

text editor

shortcuts

Navigating the file system

listing files

paths

finding files

A pipe dream

what?

redirection

screen/tmux

shell scripting, automating and parallelising tasks

bash basics

Make

xargs

GNUParallel

Working with files

Working with databases

you can pipe anything to anything else, and usually it'll do something. with most of the standard linux tools, it'll even do what you expect. --scott simpson

References:

Linux Productivity tools : https://www.usenix.org/sites/default/files/conference/protected-files/lisa19_maheshwari.pdf

https://www.learnenough.com/command-line-tutorial

Conquering the commandline: http://conqueringthecommandline.com/book

https://cfenollosa.com/misc/workshop_unix_cheat_sheet.pdf

https://cfenollosa.com/misc/workshop_unix.pdf

Command-line text processing: https://github.com/learnbyexample/Command-line-text-processing

Book: Bioinformatics data skills

Free Book: The Linux Commandline - http://linuxcommand.org/tlcl.php

Commandline tools can be 236X faster than MapReduce : https://adamdrake.com/command-line-tools-can-be-235x-faster-than-your-hadoop-cluster.html

GNU Parallel: https://opensource.com/article/18/5/gnu-parallel