Warming up with Unix commands

Getting Stearted

In this lesson, you will learn some basic commands of the Unix operative system. The operating system (OS) is the system software that manages computer hardware, software resources, and provides common services for computer programs, e.g. allowing for the communication between hardware and software. In order to use a computer, you need an OS. At this moment you may be using a Virtual Machine containing the Ubuntu OS, a Linux distribution, a Unix-like OS. The basic commands used in Linux and Unix are essentially the same.

If you open a terminal window a prompt will appear where you can type command lines. A snapshot of the prompt is shown here below:

Example of prompt of bash shell

You can inspect the environment by typing the following commands:

 $> whoami             #  (my username)
 $> hostname [-A]      #  (machine name)
 $> pwd                #  (current directory)
 $> ls [-ltr]          #  list files and dirs
 $> ps                 #  running processes
 $> top                #  more on running procs
 $> date               #  show date and time
 $> cal 2020           #  show 2020 calendar

Need help?

 $> man ls             #  manpage for ls
 $> info ls 
 $> gnuplot -h         #help for larger programs (e.g. gnuplot)

Shell history:

 $> history           # show last commands typed
 $> !s                # last command starting with 's'
 $> !5                # run again command #5 

Command-line tricks:

<tab>                #autocomplete
$ ls D<tab>
$ ls Downloads/

<ctrl>-a #move cursor to beginning of the line
<ctrl>-e #move to end
<ctrl>-c #cancel, quit, kill
<ctrl>-d #logout
<ctrl>-z #exit without quit (suspend)

At a glance, during this lecture we'll cover the following specific topics:

• Filesystem
• Users and Groups
• Processes
• Environment
• Interpreters / Scripting

Filesystem

Navigating in the filesystem

As seen above, the command $pwd (present working directory) shows the directory you are presently working on. Now we want to see how to navigate to other directories. This is done by using the command cd (change directory) followed by a path.

If we want to move for instance to the folder LabQSM we need to provide its path. A path is the address of a file or a folder in the filesystem.

We can either use:

  $> cd /home/max/LabQSM         # absolute path  or
  $> cd ./LabQSM/                # relative path,
  

provided that you are already in /home/max. This avoids typing complete paths.

You can then navigate up on level by typing:

  $> cd ..

and you will be back to the HOME directory. You can also navigate to multiple directories as:

  $> cd ./LabQSM/LAB_1/test_diamond 

More examples:

  $> cd /etc/init.d/          # absolute
  $> cd ../../usr/bin         # relative

Finally the commands

  $> cd     or    $> cd ~    or    $> cd $HOME

take you to the HOME directory, independently of the present working directory. In order to inspect the content of a folder, or the existence of a file, you can use the ls (list) command. Examples follow:

  $>  ls ./LabQSM     or     ls -a      or    ls -ltr

Exercise:

Using the ls command, look at the content of the root folder (the folder labelled / ).

Creating, removing, copying and moving files and directories

A way to create a new file is the following:

• $> cat > filename
• Enter the content e.g. your name
• Press ctrl + d to return to command prompt.

A file named filename containing a text with your name has been created. You can find it in the list using the ls command, and you can inspect it by typing:

 $> cat filename

We will see later other ways to create and edit files using the editor named vi.

To create a new directory use the command:

 $>  mkdir dirname

Files and directory are removed using the command rm, as

 $> rm filename
 $> rm -r dirname

To move a file from a directory to another, use the command mv:

 $> mv filename ./LabQSM

The same syntax applies also for moving directories. The same command is also used to rename (or overwrite) files:

 $> mv filename newfilename

File and directory can be also copied (duplicated) in the same directory or in other directories:

 $> cp filename ./LabQSM/         #copy the file in the LabQSM directory with the same name
 $> cp filename ./LabQSM/newname  #copy the file in the LabQSM directory with a new name
 $> cp -r dirname ./LabQSM/       #copy the directory dirname and all its content in the LabQSM directory

In all cases, when the possibility of losing information exists, a confirmation is asked to the user (who has to provide a y/n answer). This behaviour can be switched off by using the flag -f (force) on each of the commands above (rm, mv, cp). Instead the option -i (interactive), makes the command require the user interaction.

Users, Groups & Permissions

Now let's inspect the content of a directory in details, by typing

 $>  ls -al 

You can see the list of the files, containing information for each file including the owner and the group, the permissions (who can read, write, or execute that file), the dates it was modified, the size etc. Here is an example:

Permissions can be changed using the chmod command:

 $>  chmod a+r  file        # makes file readable by everyone
 $>  chmod u+x  file        # makes file executable by the user
 $>  chmod go-rw file       # makes file not readable nor writable by group and others

In the example above, the community of users is described by means of u,g,o,a (user, group, others, all). As an example, users belonging to the same group of the file will have permissions specified for g, etc.

As an example, you can try to create a file in the root directory. You can go to the root directory by typing $cd ../.. from your home (relative path) or directly by typing $cd / (absolute path). The command $cat > filename will not take effect as you do not have permission to write in the root directory.

Exercise:

Create a file in your HOME directory, check out the permissions granted by default, and change them to make the file writable by all users.

Processes

Linux (Unix) is a native multi-tasking and multi-user systems. So, it allows for multiple processes to operate simultaneously without interfering with each other. A process is an executing instance of a program and carry out different tasks within the operating system.

You can monitor the active process by using the command ps

$>  ps      #shows the processes for the current shell
$>  ps -a   #shows all processes not associated with a terminal
$>  ps -x   #shows all process owned by you

The ps command also shows the unique process id (PID), the terminal type that the user is logged into (TTY), the amount of CPU in minutes and seconds that the process has been running for (TIME), and the name of the command that launched the process (CMD).

In order to see all the options available for a specific command, e.g. ps, you can use the man command (manual) e.g.

$>  man ps   #shows all the option for the ps command
$>  man ls   #shows all the option for the ls command

It is possible to kill a process by typing:

$>  kill [-9] <pid>       # <pid> is the process id as given by ps

Processes that are launched interactively can be stopped by typing

$>  ctrl+z

and can be sent in background or foreground by using bg and fg

Exercise

• Launch the process bc -l (This is an arbitrary precision calculator language with interactive execution of statements) and perform an arbitrary operation.
• Stop the process
• Send it in foreground (resume the process)
• Stop the process again
• Kill the process
• Verify that the process is not running anymore

Environment

Each shell (bash interpreter, here) comes with environment variables. Variables related to your environment that can be modified and customized.

$> env                        # shows all the defined environment variables
$> echo $<var-name>       # write the content of the variable 

Relevant variables are:

$> echo $HOME       # This variable contains the absolute path or your Home directory 
$> echo $USER       # This variable contains your uername
$> echo $PATH       # This variable specify a set of directories where executable programs are located. 

Executables located in the directories can be run simply by typing the name of the executable.

In general, if you want to execute an executable (binary or script), simply by typing its name you can either copy it in a directory contained in $PATH, or add the directory (e.g. mydir) containing your executable in the $PATH variable. This can be done by using the export command.

$> export PATH=$PATH:mydir

Many environment variables are defined in a file present in your home directory, named .bashrc, that is read ad each login, so it is possible to modify or define new variables permanently by editing this file. In order for the changes to take effect, you need to read and execute the content of the file by using the source command

$> source ~/.bashrc 

Variables can be modified or created from scratch:

$> myvar="This is my new var"
$> echo $myvar

When assigning variables pay attention to quotes:

$> var="$HOME"
$> echo $var   ->   /HOME/labcqm
$> var='$HOME'
$> echo $var   ->   "$HOME"    
$> var=`hostname`
$> echo $var   ->   qmobile

The last quote pair (back quotes, ``) expands a command before the assignment is performed (very useful for scripting).

It is also possible to define new commands: here, for instance, we modify the rm command to make it safer:

$> alias rm='rm -i'

now the rm command will ask for confirmation before removing a file:

$> touch pippo.txt         #create an empty file 
$> rm pippo.txt            #will ask: remove pippo.txt? (y/n)

Another example:

$> alias hello="echo hello $USER"
$> hello  -> hello max

Exercise

In this course, we will make use of the executable pw.x from the Quantum ESPRESSO distribution.

• Verify that the file pw.x is contained in one of the directories listed in $PATH
• Verify also that the file pw.x is an executable file

Creating and editing text files

There are several ways to create files, let's explore some of them:

 $> touch myfile.txt  #create an empty file

A way to create a file with some content was already see before:

• $> cat > filename.txt
• Enter the content e.g. your name
• Press ctrl + d to return to command prompt.

or alternatively:

$> cat > filename.txt <<EOF
   >Enter your content
   >EOF

or we can use and editor:

• vi myfile.txt #Using the editor vi
• edit myfile.txt #Using the getid editor (Ubuntu)

While gedit is more intuitive, here we suggest using the vi editor. A bit more involuted, but very useful once you learn how to use it. A short guide of the vi editor can be found in ./LabQSM/docs/vi_cheat_sheet.pdf and can be open with the pdf reader "evince".

evince ./LabQSM/docs/vi_cheat_sheet.pdf

Let's create a text file named atp.txt with the following content:

9850 Nadal
6630 Federer
3075 Berrettini
12030 Djokovic

To view the file content you can use the following commands:

$> cat atp.txt
$> more atp.txt  # <space> for next page
$> less atp.txt  # <arrows> to move and <q> to exit

or, of course, you can open it wit an editor vi/gedit.

To view the start/end of file:

$> head -1 atp.txt
$> tail -2 atp.txt

Sort, check contents:

$> wc -l atp.txt # provides the number of lines
$> sort atp.txt  # sort the file according to its first digit
$> sort -n atp.txt  # sort the file according to its numerical value
$> sort -nr atp.txt  # sort the file according to its numerical value in reverse order
$> sort -k2 atp.txt. #sort column 2 

Search contents: grep: The grep utility searches any given input files, selecting lines that match one or more patterns.

$> grep er atp.txt # Select lines matching with the string 
$> grep -v er atp.txt # Selected lines are those not matching 

More commands

""Compressing files""