name: inverse layout: true class: center, middle, inverse --- # UNIX for Beginners --- layout: false # Getting Started There are lots of UNIX tutorials on the web. This one is not better than any of those, but it is happening right now, so it should be easy to pay attention and learn the basics. To get started, open a terminal/shell window * Shell on other unix systems * Cygwin (Terminal Emulator on Windows) * WSL (Linux system on Windows 10) * Or go [here](https://www.tutorialspoint.com/unix_terminal_online.php) --- # What is UNIX? UNIX is a kind of computer operating system invented in the 1970s. You may have heard of many variants of UNIX: > *BSD, Arch Linux, Gentoo, NixOS, Alpine, Debian, Ubuntu, RHEL etc.. UNIX is popular because it is simple, modular, secure, and extremely flexible. Most UNIX tools are open-source. UNIX is the preferred operating system for basically all difficult or important tasks in scientific computing. --- # The UNIX terminal The usual (and old-fashioned) way to interact with a UNIX computer is via the terminal, which is a text-based interface that allows you to give the computer commands and ask it questions. * Open a program * Analyze the data in this file * What files do I have in this directory? * Go to another directory * Rename a file * Get a file from somewhere else and bring it here. The terminal is a text display. When you type commands into it, you are giving instructions to the computer, and it gives you information back. Usually, commands have the following form: ```bash command -option argument1 argument2 ... ``` --- # Manpages UNIX provides a utility for learning more about commands. Every command that is intrinsic to the UNIX distribution (`cd`, `ls`, `mkdir`, `etc`.) has a manual page. These can be accessed with the `man` command. To see the manual page for the interesting command `grep`, type: ```bash man grep ``` You will see a manual page that describes the functionality of the command and its various options. Scroll through the manual (which can be very long) one screenful at a time with the space bar, one line at a time with the Enter key, and quit by typing `q`. --- # Paths and the directory structure Using the UNIX terminal, you must frequently tell the computer where in its memory to find a certain file, directory, or program. In UNIX, a folder is called a "directory". A directory is a place in the computer's memory where you can store files and other directories. Every directory is contained within another directory except for one, the top-level directory, which is known as the "root" directory. This structure is called the file hierarchy. The root directory contains many directories, which also contain many directories. Each directory has a name, so you can say which directory you are interested in by putting a `/` (slash) between the names of directories. name is your username. --- # Home directory For example, my root directory contains a directory called `home`, which contains a directory whose name is the same as my username. If your username is `vr0`, then the path to your home directory might be ```bash /home/vr0 ``` The leading `/` means the root directory. Inside the root directory, there is a directory called `home`. Inside that directory, there is another named `vr0`. Likewise, if you have a directory inside your home directory called `stuff`, then the path to this directory is ```bash /home/vr0/stuff ``` --- # Case sensitivity You should also know that UNIX is case-sensitive, which means that it knows the difference between lower-case and upper-case letters, and it cares about the difference. The name `vr0` is different from all of these: ```bash vr0 VR0 Vr0 ``` --- # Navigation When you are navigating through the directory structure (finding a file, processing data, etc.), you must know where you are, and you must know where you want to go. If you are ever unsure of where in the directory tree you are, then you can use the ```bash pwd ``` command, which stands for "print working directory". To go to a different directory, use the `cd` command. `cd` takes one argument, which is a path. `cd` stands for "change directory". To go to your home directory, type: ```bash cd /home/vr0 ``` No matter where you were when you typed this, you are now in `/home/vr0`. More on navigation soon. --- # Listing the contents of a directory To see the contents of your current directory, you can list its contents ```bash ls ``` When you type the `ls` command without any arguments, it simply lists all the files and directories that are in your _current_ directory. If you use `ls` with a path like `/home/vr0`, you will list the contents of `/home/vr0` without actually going there: ```bash ls /home/vr0 ``` --- # Hidden files Some files are hidden by default by the `ls` command. These are files that you don't want to see every day, but are essential configuration files for various programs you use. These files hold information about your preferences when using different programs, and the first character is a `.` (dot). They usually reside in your home directory. Type ```bash ls -a ``` (a stands for "all") to show all files in a directory. This is what it will look like, with the "hidden" files first, then the other files. ```bash .git .vimrc .emacs documents src training ``` You usually won't need to worry about these files, but it's important to know they are there. --- # Relative paths I Now you can go different places in your computer's memory and see what's there. But you don't have to type in a full path all the time. You can use a "relative path", which means that you type the path relative to the directory you are in. When you type ```bash cd /home/vr0/stuff/images ``` you are instructing the computer to go to the root directory `/`, then the home directory, then the `vr0` directory, and so on. If you were in `/home/vr0` already, then you could just type ```bash cd stuff/images ``` Since `stuff` is a subdirectory of `/home/vr0`, you can just put it in your path directly without referencing it through `/home/vr0`. Note that there is no leading `/` in the relative path because `stuff` is not a subdirectory of the root directory - it's relative to your current directory. --- # Relative paths II If you were in `/ home/vr0/stuff/images` and you wanted to go to `/home/vr0/stuff/documents`, then you could type: ```bash cd /home/vr0/stuff/documents ``` but that is long and hard to type. You can use a relative path here too, with the `..` symbol, which stands for the "parent" directory, that is, the directory which contains the current directory. Only the root directory has no parent directory. To get to `/home/vr0/stuff/documents` from `/home/vr0/stuff/images`, just type: ```bash cd ../documents ``` ### Shortcuts: Type `cd ~` to go to your home directory. Type `cd -` to go to the previous directory. --- # Where you are There is one more symbol to know about, which will come in handy later, which allows you to make reference to the current directory. This is the `.` (dot) symbol, which means "the current directory". For example, ```bash cp /home/vr0/stuff/document.txt . ``` Which means, in English, "copy `/home/vr0/stuff/document.txt` to my current directory". This is much easier than explicitly stating the full path to wherever you are. --- # Commands I Now it's time for a digression before we discuss some other commands - it's important to know what a command really is. When you type a command into the terminal and press [Enter], your computer looks around in certain places for a file that has the same name as the command you typed in. A command is just a file with computer instructions in it. When your computer finds the right file, it reads those instructions, and then does whatever the command tells it to do. Commands like `cd` and `ls` and `pwd` are just _executable files_ that contain computer instructions to do certain things, and to show you certain things on your screen. --- # Commands II For example, the command to list the contents of a directory, `ls`, is actually an executable file. Use the `which` command to find out where the executable file is: ```bash which ls ``` On my system this gives `/bin/ls`. Since commands are just files with instructions in them, you can make your own commands. This is one of the most powerful features of UNIX - it is very customizable. A UNIX system automatically looks in certain directories for executable files/commands. Changing these variables is somewhat beyond the scope of this presentation, but you can see your path by doing: ```bash echo $PATH ``` --- # Making files and directories The `mkdir` command makes a new directory, and you can use it like this: ```bash mkdir name ``` where `name` is what you want to name the new directory. If you list several names separated by a space, `mkdir` will create directories with all the names you specify. The `cp` command copies a file or directory from one place to another: ```bash cp thing somewhere ``` where `thing` is the file or directory you want to copy, and `somewhere` is the place where you want to put it. If `thing` is a directory, and you want to copy the directory and all its contents, then you can use the `-r` flag, as in: ```bash cp -r myfiles /home/vr0 ``` --- # Moving things You can also use `cp` to rename a file as you copy it: ```bash cp thing somewhere/newname ``` Now `thing` has been copied, and the new copy renamed `newname` and put in `somewhere`. The `mv` command moves a file or directory from one place to another, or renames files or directories. Its usage is similar to that of cp: ```bash mv thing somewhere ``` or `mv name newname`. You can use mv to move files or directories. `mv` does not need a `-r` flag to move a directory and all its contents. The `mv` command can also be used to rename files or directories similarly to `cp`: ```bash mv thing somewhere/newname ``` --- # Deleting things The `rm` command removes (deletes) a file or directory. It is used like this: ```bash rm thing ``` where thing is a file. Empty directories can be removed with the `rmdir` command. You can remove a non-empty directory with the `-r` (recursive) flag. *Be careful* with the `rm` command, especially this: ```bash rm -r * ``` --- # Permissions I UNIX has a special way of keeping track of who is allowed to do things with different files. Typing `ls -l` gives ```bash drwxr-xr-x 5 vr0 vr0 512 Apr 16 09:49 projects drwxr-xr-x 8 vr0 vr0 1024 Jul 7 13:41 scratch drwxr-xr-x 3 vr0 vr0 512 Jun 26 15:13 src drwxr-xr-x 2 vr0 vr0 512 May 28 14:45 stuff drwxr-xr-x 4 vr0 vr0 512 Jun 26 15:02 testing -rwx------ 1 vr0 vr0 23476 May 11 05:43 beastie.png ``` The first thing on each line that looks like this: `drwxr-xr-x` is the permissions field. The `d` at the beginning means that this object is a directory. The following characters are divided into 3 fields of permissions: user, group, and other. The first three characters `rwx` are the user's (owner's) permissions. The second three characters `r-x` are the permissions for other people in the user's group. The third three characters are permissions for anyone else. --- # Permissions II Each set of permissions (user, group, and other) has three subfields that indicate the permissions for three different types of actions you can take on a file. The permissions for the user look something like this: `rwx` which means that the user has "read", "write", and "execute" permission for this file. * "Read" `r` means that you can look at the contents of the file. * "Write" `w` permission means you can modify the contents, change the filename, or delete it altogether if you wish. * "Execute" permission means you can invoke the file as a program. There is a special way of telling UNIX how you want the permissions for a certain file to be, using the `chmod` command, which stands for "change mode". --- # Permissions III You can specify how the permissions should be for each of the three types of users. To change the group permission for a file to make it "group writable" with these permissions: ```bash rwxr-x--- ``` you type: ```bash chmod g+w filename ``` where `filename` is the name of the file whose permissions you wish to change. The `g+w` part means "group add write", which means "Make this file writable by users in my group". Similarly, `g+r` and `g+x` mean "make group readable" and "make group executable", respectively. You can do this for any of the three types of users too: `u+x` makes a file executable for you, and `o+r` makes a file readable by everyone else. You can specify all types of users with a like this: `a-x`, which makes the file non-executable by all users. --- # Viewing and editing text files ### Viewing files * `cat` (concatenate) dumps the contents of a text file on the screen * `more` shows the contents of a text file, one screenfull at a time. * `less` is a slightly better version of `more` * `head` shows the first few lines of a file. * `tail` shows the last few lines of a file ### Editing files Your UNIX system likely comes with a basic text editor. In addition, nearly all UNIX systems have at least one of these: * `vi` Default text editor on all UNIX systems. Do not use this unless you know what you are doing. * `emacs` Slightly easier to use * `nano/pico` Easy to use. --- # More commands A standard UNIX system has many hundreds of commands. Most are not needed for everyday work. * `ps` lists the active processes (programs) on your system * `top` lists the processes that are using the most processor time * `kill` and `pkill` can be used to end runaway processes * `find` finds files that match certain critera * `grep` searches text files for patterns or words, e.g. ```bash grep -i vr0 documents_that_may_contain_my_name/* ``` * `alias` make new names for complicated commands --- # Creating a new UNIX executable Here is an example. On my system, the `python` executable is located here: ```bash which python /usr/local/bin/python ``` Let's create a new text file called `test.py` ```python #!/usr/local/bin/python print("Hello, World!") ``` Make the file executable ```bash chmod +x test.py ``` Now run it ```bash ./test.py ``` We need the `./` to tell the system to look in the current directory for the `test.py` executable. --- # Using executables If you want to use this often, I suggest putting it in a special directory, and adding it to your `$PATH`: ```bash cd ~ mkdir bin mv test.py bin export PATH=~/bin:$PATH ``` A better solution is to add `~/bin` to your path in a config (`.profile`, `.bashrc`, etc) file. --- # Shortcuts and tips Copy all the `.txt` files in this directory into another directory: ```bash cp *.txt my_text_files ``` Other examples ```bash mv ../*.csv ../../data_files ``` Enumerate parts of file names ```bash rm thing{1,2,3}.pdf ``` --- # Interacting with the command line Most shells are set up to make it easy to run commands * `[up arrow]` previous command * `[down arrow]` next command * `!$` previous argument, as in ```bash pdflatex foo bibtex !$ ``` * `[tab]` completion of program and file names, up to uniqueness, e.g ```bash whic[tab] ``` completes the command `which`. ```bash cd Ap[tab] ``` completes the directory name `Applications`. --- # All your favorite programs on the command line ### Version control and collaboration > git, hg, svn, cvs A typical git session: ```bash mkdir myproject cd !$ git init Initialized empty Git repository in /home/vr0/work/src/myproject/.git/ git remote add origin git@github.com:vr0/myproject.git [make changes] git commit -am "first commit" git push -u origin master ``` --- # Remote access Use `curl` to scrape a web page ```bash curl https://www.unix.lu ``` Use `ssh` to log in to a remote host ```bash ssh vr0@remotehost ``` --- # Further resources Google your UNIX problems. UNIX has been around a lot longer than Google, and you are not the first person to have trouble. If that fails, ask someone!