Linux shell
Intro
The shell has been the major interface for the Unix/Linux operating system since it was first conceived. The shell allows interaction with the operating system through a text based interface. The shell provides the user with the following features:
An easy to use command line interface (CLI)
Combine existing tools to create new tools
Input/output redirection
Wildcard characters for filename abbreviation
Variables and options for environment customisation
Ability to write shell functions and programs
Command-line editing
Command history (quick access to previous commands)
Arithmetic
Command abreviations
The user starts the shell by logging into the computer with a userid and password:
******************************************************************************
*** THE UNIVERSITY OF YORK IT SERVICES ***
*** ***
*** THIS IS A PRIVATE COMPUTER ***
*** UNAUTHORISED ACCESS STRICTLY PROHIBITED ***
******************************************************************************
login: user001
password:
Last login: Mon Sep 8 14:12:44 2014 from gallifrey.york.ac.uk
-bash-4.1$
The last line is a command prompt and it is the means by which the computer is telling you that it is ready to accept a command from you. If you do not see the prompt, the computer is probability still executing the last command you have typed. The user types commands which take the form:
command [ options ] [ arguments ]
Options to a command are usually proceeded by a - or -- and are optional, this differentiates them from the arguments which are mandatory. The following example shows the echo command which prints the arguments, and the ls command which displays the users files. There will be more explanation of files and the ls command later.
-bash-4.1$ echo Hello World
Hello World
-bash-4.1$ ls
bin Chemistry Desktop examples Experiments intel jobs logs tmp
-bash-4.1$ ls -l
total 296
drwxr-xr-x 2 abs4 csrv 4096 Jun 24 09:39 bin
drwxr-xr-x 3 abs4 csrv 4096 Jun 6 09:23 Chemistry
drwxr-sr-x 2 abs4 elecclust 4096 Mar 11 10:53 Desktop
drwxr-xr-x 3 abs4 csrv 4096 Jun 30 12:21 examples
drwxr-xr-x 5 abs4 csrv 4096 May 23 11:34 Experiments
drwxr-xr-x 3 abs4 csrv 4096 Aug 14 12:26 intel
drwxr-sr-x 3 abs4 elecclust 4096 Aug 15 12:49 jobs
drwxr-xr-x 2 abs4 csrv 266240 Aug 15 13:48 logs
drwxr-xr-x 3 abs4 csrv 4096 Aug 14 14:50 tmp
-bash-4.1$
To logout of the shell type logout, exit or press Ctrl + d.
-bash-4.1$ exit
logout
Connection to ecgberht closed.
Note
The are a number of shells available to the user. In this tutorial we will be using Bash, the most widely used Linux shell.
Files and directories
Filesystem organisation
The file system is the component of the operating system that organises data into files. These files are organised into directories.
When you have logged in you will be placed in a directory which is called your home directory. To find the name of the directory use the pwd command (print working directory). Use the cd command to change directory.
-bash-4.1$ pwd
/usr/researchcomp/elecclust/abs4
-bash-4.1$ cd /usr
-bash-4.1$ pwd
/usr
-bash-4.1$ cd
-bash-4.1$ pwd
/usr/researchcomp/elecclust/abs4
-bash-4.1$ cd ..
-bash-4.1$ pwd
/usr/researchcomp/elecclust
-bash-4.1$ cd .
-bash-4.1$ pwd
/usr/researchcomp/elecclust
-bash-4.1$
The output of the pwd command, /usr/researchcomp/elecclust/abs4, is called a pathname, and this specifies the location of the users home directory. The first / in the pathname is the root directory. Names following the / are directory names. Directories within directories are called sub-directories. Pathanmes can also specify the location within the filesystem of files. Only the last name of a pathaname can be a file or directory.
The cd command lets you change your working directory to another location in the file system. cd with no arguments places you back in your home directory. The special directory .. references the directory above your current directory (known as the parent directory). The is another special direcory . which references the current directory. These two directories can be viewed as links.
Listing files and directories
To list the files in a directory use the ls (list) command.
-bash-4.1$ ls
afile bin Chemistry Desktop examples Experiments intel jobs logs tmp
-bash-4.1$ ls -l
total 296
-rw-r--r-- 1 abs4 csrv 0 Sep 8 16:26 afile
drwxr-xr-x 2 abs4 csrv 4096 Jun 24 09:39 bin
drwxr-xr-x 3 abs4 csrv 4096 Jun 6 09:23 Chemistry
drwxr-sr-x 2 abs4 elecclust 4096 Mar 11 10:53 Desktop
drwxr-xr-x 3 abs4 csrv 4096 Jun 30 12:21 examples
drwxr-xr-x 5 abs4 csrv 4096 May 23 11:34 Experiments
drwxr-xr-x 3 abs4 csrv 4096 Aug 14 12:26 intel
drwxr-sr-x 3 abs4 elecclust 4096 Aug 15 12:49 jobs
drwxr-xr-x 2 abs4 csrv 266240 Aug 15 13:48 logs
drwxr-xr-x 3 abs4 csrv 4096 Aug 14 14:50 tmp
ls without any options or arguments lists the names of the files and directories in the current working directory. In this example above it can be hard to see which names refer to files or directories. We will show you how to do this later on.
The -a option shows all files, filenames starting with . are usually hidden from display.
-bash-4.1$ ls -a
. Experiments intel .profile
.. .felix jobs .pulse
afile .gconf .lesshst .pulse-cookie
.bash_history .gconfd .local .python_history
bin .gnome2 logs .qmon_preferences
Chemistry .gnote .matlab .Rhistory
.config .gnupg .mcrCache8.3 .ssh
.dbus .gstreamer-0.10 .modulerc .subversion
Desktop .gvfs .nautilus tmp
.emacs.d .history .ngspice_history .Xauthority
examples .ICEauthority .nx
We can combine options to give more detail.
-bash-4.1$ ls -al
total 440
drwx------ 30 abs4 elecclust 4096 Sep 8 16:26 .
drwxrws--- 14 jaw500 elecclust 4096 Sep 8 16:25 ..
-rw-r--r-- 1 abs4 csrv 0 Sep 8 16:26 afile
-rw------- 1 abs4 elecclust 16495 Sep 8 15:40 .bash_history
drwxr-xr-x 2 abs4 csrv 4096 Jun 24 09:39 bin
drwxr-xr-x 3 abs4 csrv 4096 Jun 6 09:23 Chemistry
drwxr-sr-x 4 abs4 elecclust 4096 Mar 11 10:53 .config
drwx--S--- 3 abs4 elecclust 4096 Mar 11 10:51 .dbus
drwxr-sr-x 2 abs4 elecclust 4096 Mar 11 10:53 Desktop
drwxr-xr-x 3 abs4 csrv 4096 May 23 14:52 .emacs.d
drwxr-xr-x 3 abs4 csrv 4096 Jun 30 12:21 examples
drwxr-xr-x 5 abs4 csrv 4096 May 23 11:34 Experiments
drwxr-xr-x 2 abs4 csrv 4096 Jul 1 12:00 .felix
drwx--S--- 4 abs4 elecclust 4096 May 2 16:09 .gconf
drwx--S--- 2 abs4 elecclust 4096 May 2 16:34 .gconfd
drwx--S--- 4 abs4 elecclust 4096 Mar 11 10:53 .gnome2
drwxr-sr-x 3 abs4 elecclust 4096 Mar 11 10:53 .gnote
drwx--S--- 2 abs4 elecclust 4096 Mar 11 10:52 .gnupg
drwxr-sr-x 2 abs4 elecclust 4096 Mar 11 10:53 .gstreamer-0.10
drwx--S--- 2 abs4 elecclust 4096 Mar 11 10:52 .gvfs
-rw------- 1 abs4 csrv 978 Jun 6 09:32 .history
-rw------- 1 abs4 elecclust 314 Mar 11 10:52 .ICEauthority
drwxr-xr-x 3 abs4 csrv 4096 Aug 14 12:26 intel
drwxr-sr-x 3 abs4 elecclust 4096 Aug 15 12:49 jobs
-rw------- 1 abs4 csrv 46 Jun 6 09:31 .lesshst
drwxr-sr-x 3 abs4 elecclust 4096 Mar 11 10:52 .local
drwxr-xr-x 2 abs4 csrv 266240 Aug 15 13:48 logs
drwxr-xr-x 3 abs4 csrv 4096 May 2 16:06 .matlab
drwxr-xr-x 9 abs4 csrv 4096 Jul 3 11:54 .mcrCache8.3
-rw-r--r-- 1 abs4 csrv 32 Sep 5 08:05 .modulerc
drwxr-sr-x 2 abs4 elecclust 4096 Mar 11 10:53 .nautilus
-rw------- 1 abs4 elecclust 0 Jan 13 2014 .ngspice_history
drwx--S--- 6 abs4 elecclust 4096 Apr 25 13:36 .nx
-rw-r--r-- 1 abs4 elecclust 145 May 19 11:59 .profile
drwx------ 2 abs4 csrv 4096 Mar 11 10:54 .pulse
-rw------- 1 abs4 elecclust 256 Mar 11 10:54 .pulse-cookie
-rw------- 1 abs4 csrv 49 Jun 3 13:42 .python_history
-rw-r--r-- 1 abs4 csrv 342 Jun 16 12:57 .qmon_preferences
-rw------- 1 abs4 csrv 40 May 23 11:09 .Rhistory
drwxr-sr-x 2 abs4 elecclust 4096 Jun 5 12:53 .ssh
drwxr-xr-x 2 abs4 csrv 4096 May 2 16:06 .subversion
drwxr-xr-x 3 abs4 csrv 4096 Aug 14 14:50 tmp
-rw------- 1 abs4 csrv 488 Sep 8 15:48 .Xauthority
-bash-4.1$
The next example displays the directory in the long format using the -l option, much more information is displayed about the directories and files.
ls can take arguments as well. When specifying an argument ls displays the information for that file or directory.
-bash-4.1$ ls Experiments
architest.dtr OLD simple_verbs.dtr~
architest.dtr~ OLD CART simple verbs_to_Dunstan.txt
exploded.csv ordered simple verbs_to_Dunstan.xlsx
Latest CART phon.csv simple_verbs_to.txt
mian.rp phonsorted simple_verbs.txt
NotesAboutInfixPredictions.pdf simple_verbs.dtr simpverbsort
-bash-4.1$ ls -l Experiments
total 368
-rw-r--r-- 1 abs4 csrv 919 May 23 11:08 architest.dtr
-rw-r--r-- 1 abs4 csrv 909 May 23 11:08 architest.dtr~
-rw-r--r-- 1 abs4 csrv 3613 May 23 11:08 exploded.csv
drwxr-xr-x 2 abs4 csrv 4096 May 23 11:34 Latest CART
-rw-r--r-- 1 abs4 csrv 4019 May 23 11:28 mian.rp
-rw-r--r-- 1 abs4 csrv 193602 May 23 11:08 NotesAboutInfixPredictions.pdf
drwxr-xr-x 2 abs4 csrv 4096 May 23 11:08 OLD
drwxr-xr-x 2 abs4 csrv 4096 May 23 11:08 OLD CART
-rw-r--r-- 1 abs4 csrv 3613 May 23 11:08 ordered
-rw-r--r-- 1 abs4 csrv 6217 May 23 11:08 phon.csv
-rw-r--r-- 1 abs4 csrv 6217 May 23 11:08 phonsorted
-rw-r--r-- 1 abs4 csrv 17663 May 23 11:08 simple_verbs.dtr
-rw-r--r-- 1 abs4 csrv 17647 May 23 11:08 simple_verbs.dtr~
-rw-r--r-- 1 abs4 csrv 8058 May 23 11:08 simple verbs_to_Dunstan.txt
-rw-r--r-- 1 abs4 csrv 30416 May 23 11:08 simple verbs_to_Dunstan.xlsx
-rw-r--r-- 1 abs4 csrv 4696 May 23 11:08 simple_verbs_to.txt
-rw-r--r-- 1 abs4 csrv 17525 May 23 11:08 simple_verbs.txt
-rw-r--r-- 1 abs4 csrv 17647 May 23 11:08 simpverbsort
-bash-4.1$ ls -ld Experiments
drwxr-xr-x 5 abs4 csrv 4096 May 23 11:34 Experiments
-bash-4.1$ ls -l afile
-rw-r--r-- 1 abs4 csrv 0 Sep 8 16:26 afile
-bash-4.1$ ls /usr
appl cmsmigratetest lfa phpweb src vleexam
archive cvs lib puppet systems vle-sysadmin
backups datasets lib64 puppetdev tmp webmisc
bin etc libexec researchcomp transfer work
central games local rlink userfs yorkroot
cert idm logfiles sbin UserFS yorkweb
cmsmedia include mirror scratch vle yorkwebtest
cmsmediatest java misc secbuffer vle-arch
cmsmigrate kerberos opapp share vle-eldt
-bash-4.1$
Using a directory name as an option causes ls to list the contents of the directory. To list the attributes of the directory use the -d option. You can use a pathname as the argument.
Creating, moving and copying files and directories
You can create directories, move or copy files or directories to other locations in the filesystem using the mkdir (make directory) mv (move) and cp (copy) commands.
-bash-4.1$ ls
afile bin Desktop Experiments jobs new-dir
bfile Chemistry examples intel logs tmp
-bash-4.1$ mv afile new-dir
-bash-4.1$ cp bfile new-dir
-bash-4.1$ ls
bfile Chemistry examples intel logs tmp
bin Desktop Experiments jobs new-dir
-bash-4.1$ ls new-dir
afile bfile
-bash-4.1$ mv new-dir/afile .
-bash-4.1$ ls
afile bin Desktop Experiments jobs new-dir
bfile Chemistry examples intel logs tmp
-bash-4.1$
This example creates a new directory, new-dir, We then move the file afile to it and create a copy of bfile. We then move the file afile back to our current working directory. Note the use of the . file to reference the current working directory. We can use full or partial pathnames to reference other parts of the file system.
Copying a directory is a little more complicated and the directory may contain files and directories. We use the -r command to cp to do this.
-bash-4.1$ ls
afile bin Desktop Experiments jobs tmp
bfile Chemistry examples intel logs
-bash-4.1$ ls tmp
icc-start ifort-start logs mpi-stop start test
icc-stop ifort-stop mpi-start new-dir stop
-bash-4.1$ cp tmp/test .
cp: omitting directory `tmp/test'
-bash-4.1$ cp -r tmp/test .
-bash-4.1$ ls
afile bin Desktop Experiments jobs test
bfile Chemistry examples intel logs tmp
-bash-4.1$ ls test
test.c test.cpp test.f test.f90 test.x
-bash-4.1$
In this example we wish to copy the contents of the directory tmp/test into the current directory. cp will not copy a directory by default. We have to use the -r (recursive) option to tell cp to copy all files and directories within the directory.
Deleting files and directories
The rm (remove) command is used to delete files.
-bash-4.1$ ls
afile bin Desktop Experiments jobs test
bfile Chemistry examples intel logs tmp
-bash-4.1$ rm bfile
-bash-4.1$ ls
afile Chemistry examples intel logs tmp
bin Desktop Experiments jobs test
-bash-4.1$
To delete directories use the rmdir (remove directory) command.
-bash-4.1$ rmdir dira
rmdir: failed to remove `dira': Directory not empty
-bash-4.1$ rm -r dira
-bash-4.1$ ls
afile Chemistry dirb Experiments jobs test
bin Desktop examples intel logs tmp
-bash-4.1$ rm -ri dirb
rm: descend into directory `dirb'? y
rm: descend into directory `dirb/dirb'? y
rm: remove regular empty file `dirb/dirb/afile'? y
rm: remove directory `dirb/dirb'? y
rm: remove regular empty file `dirb/afile'? y
rm: remove directory `dirb'? y
-bash-4.1$
rmdir will only remove empty directories. To remove a directory and all it’s contents use the rm -r (recursive) option to the rm command. To be safe and check the files before you remove them use rm -ri (recursive and interactive) options.
Editing and displaying the contents of files
Text Editors
Below is a list of some of the most popular text editors, nano is probably the simpliest however, the added capabilities of vim and emacs make them well worth learning. vim comes with the program vimtutor, which you can run and follow along with. emacs also has a tutorial and you can start emacs and type C-h t , that is, Ctrl-h followed by t. Both these programs are fantastic editors but their capabilities are beyond this beginners tutorial.
Displaying the contents of files
The commands cat (concatenate files) and more displays the contents of file.
cat and more-bash-4.1$ cat snark2
The Hunting of the Snark
By Lewis Carroll
Fit the First
The Landing
"Just the place for a Snark!" the Bellman cried,
As he landed his crew with care;
Supporting each man on the top of the tide
By a finger entwined in his hair.
"Just the place for a Snark! I have said it twice:
That alone should encourage the crew.
Just the place for a Snark! I have said it thrice:
What I tell you three times is true."
-bash-4.1$ more snark
The Hunting of the Snark
By Lewis Carroll
Fit the First
The Landing
"Just the place for a Snark!" the Bellman cried,
As he landed his crew with care;
Supporting each man on the top of the tide
By a finger entwined in his hair.
"Just the place for a Snark! I have said it twice:
That alone should encourage the crew.
Just the place for a Snark! I have said it thrice:
What I tell you three times is true."
The crew was complete: it included a Boots—
A maker of Bonnets and Hoods—
A Barrister, brought to arrange their disputes—
And a Broker, to value their goods.
A Billiard-marker, whose skill was immense,
--More--(2%)
The cat command displays all the text in the users file on the screen. This can prove difficult to read if there are large amounts of text. The more command paginates the text and displays portions of it on the screen. The user can use character command to move through the file:
SPACE- display the next screen of textq- quit displaying the fileb- skip backwards through he file/pattern- search for text in the file
Files and directory permissions
Groups are provided to manage sets of users and control access to files and directories. All users belong to a default group and may be a member of other groups.
-bash-4.1$ groups
csrv pfs17 pfs34 csys cvssys itsilg rhpcs sshfix git5 git6 git7 elecclust yc-gauss yc-install yc-colum
-bash-4.1$ ls -l
total 332
-rw-r--r-- 1 abs4 csrv 0 Sep 8 16:26 afile
drwxr-xr-x 2 abs4 csrv 4096 Jun 24 09:39 bin
drwxr-xr-x 3 abs4 csrv 4096 Jun 6 09:23 Chemistry
drwxr-sr-x 2 abs4 elecclust 4096 Mar 11 10:53 Desktop
drwxr-xr-x 3 abs4 csrv 4096 Jun 30 12:21 examples
drwxr-xr-x 5 abs4 csrv 4096 May 23 11:34 Experiments
drwxr-xr-x 3 abs4 csrv 4096 Aug 14 12:26 intel
drwxr-sr-x 3 abs4 elecclust 4096 Aug 15 12:49 jobs
drwxr-xr-x 2 abs4 csrv 266240 Aug 15 13:48 logs
-rw-r--r-- 1 abs4 csrv 25678 Sep 9 10:24 snark
-rw-r--r-- 1 abs4 csrv 433 Sep 9 10:27 snark2
drwxr-xr-x 2 abs4 csrv 4096 Sep 8 17:08 test
drwxr-x--- 5 abs4 csrv 4096 Sep 8 17:01 tmp
-bash-4.1$
The groups command displays which groups you are a member of. Each file and directory you create will be owned by you and be potentially accessible to a group. In the above example the file afile is owned by abs4 and is accessible to the csrv group.
There is a special group sometimes called world, or other, which contains all users of the system.
In the above example the first column of the directory listing shows the permissions of the files. These permissions control who is allowed access the files and directories. There are three categories of user who can have potential rights to access the files - owner, group, world. The access rights to the files are displayed in the form of a sequence of letters like drwxr-xr-x. The meaning is:
directory
|
drwxr-xr-x
|_||_||_|
| | |
| | other permissions
| group permissions
user permissions
d - if present, this is a directory, otherwise it is a file. The following three
letters are in three groups and state the access permissions for the owner,
group, world users
w - the file can be written to
r - the file can be read
x - if a file: it can be executed, if a directory: it can be accessed
drwxr-x--- 5 abs4 csrv 4096 Sep 8 17:01 tmp
This is a directory, the owner abs4 can read, write and access the directory. Members of the group, csrv, can read and access the directory, they can not create or write to files in the directory, all other users do not have any access to the directory.
Changing permissions
To change file permissions use the chmod command.
-bash-4.1$ ls -l snark
-rw-r--r-- 1 abs4 csrv 25678 Sep 9 10:24 snark
-bash-4.1$ chmod go-rwx snark
-bash-4.1$ ls -l snark
-rw------- 1 abs4 csrv 25678 Sep 9 10:24 snark
The chmod command has the form:
chmod <mode> <file>
mode takes the form of:
[ugoa...][[+-=][perms...]...]
u = user (owner)
g = group
o = other (world)
a = all (user, group, other)
+ add permission
- remove permission
= explicitly set permission
w = can be written to
r = can be read
x = can be executed if a file, if a directory it can be accessed
-bash-4.1$ ls -l snark
-rw-r--r-- 1 abs4 csrv 25678 Sep 9 10:24 snark
-bash-4.1$ chmod a+rx snark
-bash-4.1$ ls -l snark
-rwxr-xr-x 1 abs4 csrv 25678 Sep 9 10:24 snark
-bash-4.1$ chmod go-rwx snark
-bash-4.1$ ls -l snark
-rwx------ 1 abs4 csrv 25678 Sep 9 10:24 snark
-bash-4.1$ chmod u-w snark
-bash-4.1$ ls -l snark
-r-x------ 1 abs4 csrv 25678 Sep 9 10:24 snark
-bash-4.1$ chmod a=r snark
-bash-4.1$ ls -l snark
-r--r--r-- 1 abs4 csrv 25678 Sep 9 10:24 snark
-bash-4.1$ chmod u=w snark
-bash-4.1$ ls -l snark
--w-r--r-- 1 abs4 csrv 25678 Sep 9 10:24 snark
-bash-4.1$ chmod u+r snark
-bash-4.1$ ls -l snark
-rw-r--r-- 1 abs4 csrv 25678 Sep 9 10:24 snark
-bash-4.1$
To change the group of a file use the command chgrp <groupname> <filename>.
-bash-4.1$ ls -l snark
-rw-r--r-- 1 abs4 csrv 25678 Sep 9 10:24 snark
-bash-4.1$ groups
csrv pfs17 pfs34 csys cvssys itsilg rhpcs sshfix git5 git6 git7 elecclust yc-gauss yc-install yc-colum
-bash-4.1$ chgrp rhpcs snark
-bash-4.1$ ls -l snark
-rw-r--r-- 1 abs4 rhpcs 25678 Sep 9 10:24 snark
-bash-4.1$
History, command line editing and job control
History
The history command lists the last commands you typed.
-bash-4.1$ history
1 cd
2 ls -l
3 who
4 ps
5 sleep 200 &
6 ps
7 fg
8 history
-bash-4.1$
Command line editing
You can select past commands using the up and down arrow keys. You can edit the command line using the left and right arrow keys and any of the following commands:
Keystroke |
Result |
|---|---|
Ctrl-P |
previous command |
Ctrl-N |
next command |
Ctrl-R string |
previous command containing string |
Ctrl-B |
move back one character |
Ctrl-F |
move forward one character |
DEL |
delete previous character |
Ctrl-D |
delete character under cursor |
ESC-D |
delete word forward |
ESC-H |
delete word backward |
Ctrl-T |
transpose two characters |
ESC-F |
move forward one word |
ESC-B |
move back one word |
Job control
Job control deals with managing your programs whilst they are running. Linux uses the name process for a running program. The ps command list all the processes you have running.
-bash-4.1$ ps
PID TTY TIME CMD
14521 pts/3 00:00:00 bash
16165 pts/3 00:00:00 ps
-bash-4.1$
From this we can see that we have two processes running, the bash shell and the ps command. Associated with each process is a unique identifier - PID (process ID).
We can manage processes, especially commands that take a long time to run, by making them a background process adding an & to the end of the line. The shell then becomes free for us to execute more commands.
-bash-4.1$ ps
PID TTY TIME CMD
14521 pts/3 00:00:00 bash
17005 pts/3 00:00:00 ps
-bash-4.1$ sleep 120
^C
-bash-4.1$ sleep 120 &
[1] 17026
-bash-4.1$ ps
PID TTY TIME CMD
14521 pts/3 00:00:00 bash
17026 pts/3 00:00:00 sleep
17027 pts/3 00:00:00 ps
-bash-4.1$ echo I am doing other work
I am doing other work
-bash-4.1$ echo my work is complete
my work is complete
[1]+ Done sleep 120
-bash-4.1$
The sleep command does nothing for the number of seconds specified in the argument. The first invocation of sleep is terminated (killed) by the impatient user typing Ctrl-c. The second invocation places the command in the background, we are then able to do other work. The Done statement informs us that the command has terminated.
-bash-4.1$ sleep 360 &
[1] 17761
-bash-4.1$ sleep 1000 &
[2] 17766
-bash-4.1$ jobs
[1]- Running sleep 360 &
[2]+ Running sleep 1000 &
-bash-4.1$ fg
sleep 1000
^Z
[2]+ Stopped sleep 1000
-bash-4.1$ jobs
[1]- Running sleep 360 &
[2]+ Stopped sleep 1000
-bash-4.1$ bg
[2]+ sleep 1000 &
-bash-4.1$ jobs
[1]- Running sleep 360 &
[2]+ Running sleep 1000 &
x-bash-4.1$ fg %1
sleep 360
^C
-bash-4.1$ jobs
[2]+ Running sleep 1000 &
-bash-4.1$
In this example we put two jobs into the background. The fg command moves the last job placed in the background into the foreground. Ctrl-z stops (pauses, not kills) the job and returns to the command line. The bg command places the paused job in the background. fg can bring specific jobs to the foreground by specifying the job number.
Environment variables and shell scripts
Environment variables
In the Linux shell a variable is a named object that contains data and which can be used by programs and commands. Environment variables provides a simple way to share configuration settings between multiple applications and processes in Linux. For example the value of an environmental variable can be the default editor that should be used, which can then be used by command to invoke the correct editor when necessary.
Predefined environment variables
Variable |
Value |
|---|---|
HOME |
path to the home directory of the current user. |
PWD |
path to your working directory. |
OLDPWD |
path to your previous working directory. |
SHELL |
name of the running, interactive shell, e.g. |
TERM |
name of the running terminal, e.g. |
PAGER |
path to the program used to list the contents of files, e.g. |
EDITOR |
path to the program used for editing files, e.g. |
To use an environment variable precede its name with a $ character. We can display all defined environment variables with printenv, and set values with export.
-bash-4.1$ echo $PWD
/usr/researchcomp/elecclust/abs4
-bash-4.1$ export MYVAR="My variable"
-bash-4.1$ echo $MYVAR
My variable
-bash-4.1$ export MYVAR="My current directory is ${PWD}"
-bash-4.1$ echo $MYVAR
My current directory is /usr/researchcomp/elecclust/abs4
-bash-4.1$
Shell scripts
Shell scripts are files which contain shell commands. You run the script by typing its filename. Things to note:
The first line of the file should contain the string
#!/usr/bin/env bash. This informs the shell which program to run the scriptConsider adding execute permission to the file to allow easy execution
Create a directory in your home directory to place all your scripts and add the directory path to your
PATHvariableYou will see other
bashscripts begin with#!/bin/bash, and there is a good explanation on the differences and why we suggest#!/usr/bin/env bashon Ask Ubuntu.
-bash-4.1$ cat simple
#!/usr/bin/env bash
echo "I am a very simple script"
-bash-4.1$ sh simple
I am a very simple script
-bash-4.1$ chmod u+x simple
-bash-4.1$ ls -l simple
-rwxr--r-- 1 abs4 csrv 46 Sep 11 12:28 simple
-bash-4.1$ ./simple
I am a very simple script
-bash-4.1$ chmod u+x simple
-bash-4.1$ ls -l simple
-rwxr--r-- 1 abs4 csrv 46 Sep 11 12:28 simple
-bash-4.1$ ./simple
I am a very simple script
-bash-4.1$ pwd
/usr/researchcomp/elecclust/abs4/scripts
-bash-4.1$ simple
-bash: simple: command not found
-bash-4.1$ export PATH="${PATH}:${HOME}/scripts"
-bash-4.1$ simple
I am a very simple script
-bash-4.1$
Input and output redirection, pipes, and filters
Input and redirection
We can change the behaviour of programs to redirect input from a file instead of the keyboard and write to a file instead of the screen. The > character is used to redirect output to a file and < to redirect input.
-bash-4.1$ echo Hello World > afile
-bash-4.1$ cat afile
Hello World
-bash-4.1$ wc -l < wordsworth
25
-bash-4.1$
The wc -l command counts the number of lines typed. In this example we have redirected the input from a text file called wordsworth.
Pipes
Pipes allow the output of one program to be fed into the input of another. The | is the pipe symbol (Shift + on a GB ISO keyboard).
This example counts the number of lines in a set of files. We write the output to a file. The file is then sorted, using the sort command, into ascending order to give us the file order by number of lines.
-bash-4.1$ ls
carroll keats milton tennyson thomas wordsworth
-bash-4.1$ wc -l * > linecount
-bash-4.1$ cat linecount
733 carroll
423 keats
156 milton
11 tennyson
28 thomas
25 wordsworth
1376 total
-bash-4.1$ sort -n -k 1 linecount
11 tennyson
25 wordsworth
28 thomas
156 milton
423 keats
733 carroll
1376 total
-bash-4.1$
First we count the lines in each of the six file and redirect the output to a file. We then sort the file by the first column in numerical order.
A quicker and more efficient way without using the intermediary file is to use a pipe.
wc into sort-bash-4.1$ wc -l * | sort -n -k 1
11 tennyson
25 wordsworth
28 thomas
156 milton
423 keats
733 carroll
1376 total
-bash-4.1$
Filters
A filter is a program that transforms an input stream into an output stream. Almost all Linux programs do this. The pipe is used to connect the filters. Here is an example of finding all the user-names of people logged into the computer.
-bash-4.1$ last |more
abs4 pts/14 gallifrey.york.a Thu Sep 11 08:47 still logged in
jg757 pts/3 elecpc111.ohm.yo Thu Sep 11 01:51 still logged in
dl792 pts/3 host-172-18-1-89 Wed Sep 10 23:10 - 23:34 (00:24)
yx664 pts/9 :1001.0 Wed Sep 10 17:55 still logged in
yx664 pts/7 :1001.0 Wed Sep 10 17:54 still logged in
yx664 pts/5 :1001.0 Wed Sep 10 17:40 still logged in
yx664 pts/13 :1001.0 Wed Sep 10 16:07 still logged in
yx664 pts/10 :1001.0 Wed Sep 10 16:05 still logged in
rm591 pts/14 mandle.york.ac.u Wed Sep 10 16:01 - 16:01 (00:00)
yx664 pts/13 :1002.0 Wed Sep 10 15:56 - 16:03 (00:07)
yx664 :1002 :1002 Wed Sep 10 15:54 - 16:04 (00:09)
jdr500 pts/9 10.240.171.184 Wed Sep 10 15:53 - 17:14 (01:20)
yx664 pts/7 :1001.0 Wed Sep 10 15:51 - 17:38 (01:47)
yx664 pts/6 :1001.0 Wed Sep 10 15:47 - 16:05 (00:17)
yx664 :1001 :1001 Wed Sep 10 15:45 still logged in
rm591 pts/14 mandle.york.ac.u Wed Sep 10 15:36 - 15:38 (00:01)
rm591 pts/14 mandle.york.ac.u Wed Sep 10 14:48 - 15:34 (00:45)
rm591 pts/14 mandle.york.ac.u Wed Sep 10 14:32 - 14:35 (00:02)
yx664 pts/13 :1001.0 Wed Sep 10 13:41 - 15:42 (02:00)
yx664 pts/10 :1001.0 Wed Sep 10 13:17 - 15:44 (02:26)
yx664 pts/9 :1001.0 Wed Sep 10 12:51 - 15:42 (02:50)
yx664 pts/6 :1001.0 Wed Sep 10 12:21 - 15:42 (03:20)
yx664 pts/9 :1001.0 Wed Sep 10 11:57 - 12:23 (00:26)
--More--
-bash-4.1$ last | sort | uniq -w 9 | cut -c1-9
abs4
at568
dl792
ff555
fjg504
jdr500
jg757
kb1024
klcm500
ma725
msr514
pbc500
pbk1
rfle500
rm591
root
sjb508
sl561
sy757
tao500
tm588
wtmp begi
yw679
yx664
-bash-4.1$
The last command displays all users and the dates and times they have logged in. We then sort this, and pass it through uniq, which removes duplicate lines by comparing only the first 9 characters. We then remove the remainder of the line after the username with the cut command.
-bash-4.1$ last | cut -c1-9 | sort | uniq
abs4
at568
dl792
ff555
fjg504
jdr500
jg757
kb1024
klcm500
ma725
msr514
pbc500
pbk1
rfle500
rm591
root
sjb508
sl561
sy757
tao500
tm588
wtmp begi
yw679
yx664
-bash-4.1$
Useful commands
To find more information on any command below, type man <command> which will open up the built-in manual page for that command.
a
alias Create an alias
apropos Search Help manual pages (man -k)
apt-get Search for and install software packages (Debian/Ubuntu)
aptitude Search for and install software packages (Debian/Ubuntu)
aspell Spell Checker
awk Find and Replace text, database sort/validate/index
b
basename Strip directory and suffix from filenames
bash GNU Bourne-Again SHell
bc Arbitrary precision calculator language
bg Send to background
break Exit from a loop
builtin Run a shell builtin
bzip2 Compress or decompress named file(s)
c
cal Display a calendar
case Conditionally perform a command
cat Concatenate and print (display) the content of files
cd Change Directory
cfdisk Partition table manipulator for Linux
chgrp Change group ownership
chmod Change access permissions
chown Change file owner and group
chroot Run a command with a different root directory
chkconfig System services (runlevel)
cksum Print CRC checksum and byte counts
clear Clear terminal screen
cmp Compare two files
comm Compare two sorted files line by line
command Run a command - ignoring shell functions
continue Resume the next iteration of a loop
cp Copy one or more files to another location
cron Daemon to execute scheduled commands
crontab Schedule a command to run at a later time
csplit Split a file into context-determined pieces
cut Divide a file into several parts
d
date Display or change the date & time
dc Desk Calculator
dd Convert and copy a file, write disk headers, boot records
ddrescue Data recovery tool
declare Declare variables and give them attributes
df Display free disk space
diff Display the differences between two files
diff3 Show differences among three files
dig DNS lookup
dir Briefly list directory contents
dircolors Colour setup for `ls`
dirname Convert a full pathname to just a path
dirs Display list of remembered directories
dmesg Print kernel & driver messages
du Estimate file space usage
e
echo Display message on screen
egrep Search file(s) for lines that match an extended expression
eject Eject removable media
enable Enable and disable builtin shell commands
env Environment variables
ethtool Ethernet card settings
eval Evaluate several commands/arguments
exec Execute a command
exit Exit the shell
expect Automate arbitrary applications accessed over a terminal
expand Convert tabs to spaces
export Set an environment variable
expr Evaluate expressions
f
false Do nothing, unsuccessfully
fdformat Low-level format a floppy disk
fdisk Partition table manipulator for Linux
fg Send job to foreground
fgrep Search file(s) for lines that match a fixed string
file Determine file type
find Search for files that meet a desired criteria
fmt Reformat paragraph text
fold Wrap text to fit a specified width.
for Expand words, and execute commands
format Format disks or tapes
free Display memory usage
fsck File system consistency check and repair
ftp File Transfer Protocol
function Define Function Macros
fuser Identify/kill the process that is accessing a file
g
gawk Find and Replace text within file(s)
getopts Parse positional parameters
grep Search file(s) for lines that match a given pattern
groupadd Add a user security group
groupdel Delete a group
groupmod Modify a group
groups Print group names a user is in
gzip Compress or decompress named file(s)
h
hash Remember the full pathname of a name argument
head Output the first part of file(s)
help Display help for a built-in command
history Command History
hostname Print or set system name
i
iconv Convert the character set of a file
id Print user and group id's
if Conditionally perform a command
ifconfig Configure a network interface
ifdown Stop a network interface
ifup Start a network interface up
import Capture an X server screen and save the image to file
install Copy files and set attributes
j
jobs List active jobs
join Join lines on a common field
k
kill Stop a process from running
killall Kill processes by name
l
less Display output one screen at a time
let Perform arithmetic on shell variables
link Create a link to a file
ln Create a symbolic link to a file
local Create variables
locate Find files
logname Print current login name
logout Exit a login shell
look Display lines beginning with a given string
lpc Line printer control program
lpr Off line print
lprint Print a file
lprintd Abort a print job
lprintq List the print queue
lprm Remove jobs from the print queue
ls List information about file(s)
lsof List open files
m
make Recompile a group of programs
man Help manual
mkdir Create new folder(s)
mkfifo Make FIFOs (named pipes)
mkisofs Create an hybrid ISO9660/JOLIET/HFS filesystem
mknod Make block or character special files
more Display output one screen at a time
mount Mount a file system
mtools Manipulate MS-DOS files
mtr Network diagnostics (traceroute/ping)
mv Move or rename files or directories
mmv Mass Move and rename (files)
n
netstat Networking information
nice Set the priority of a command or job
nl Number lines and write files
nohup Run a command immune to hangups
notify-send Send desktop notifications
nslookup Query Internet name servers interactively
o
open Open a file in its default application
op Operator access
p
passwd Modify a user password
paste Merge lines of files
pathchk Check file name portability
ping Test a network connection
pkill Stop processes from running
popd Restore the previous value of the current directory
pr Prepare files for printing
printcap Printer capability database
printenv Print environment variables
printf Format and print data
ps Process status
pushd Save and then change the current directory
pv Monitor the progress of data through a pipe
pwd Print Working Directory
q
quota Display disk usage and limits
quotacheck Scan a file system for disk usage
quotactl Set disk quotas
r
ram ram disk device
rcp Copy files between two machines
read Read a line from standard input
readarray Read from stdin into an array variable
readonly Mark variables/functions as readonly
reboot Reboot the system
rename Rename files
renice Alter priority of running processes
remsync Synchronize remote files via email
return Exit a shell function
rev Reverse lines of a file
rm Remove files
rmdir Remove folder(s)
rsync Remote file copy (Synchronize file trees)
s
screen Multiplex terminal, run remote shells via ssh
scp Secure copy (remote file copy)
sdiff Merge two files interactively
sed Stream Editor
select Accept keyboard input
seq Print numeric sequences
set Manipulate shell variables and functions
sftp Secure File Transfer Program
shift Shift positional parameters
shopt Shell Options
shutdown Shutdown or restart linux
sleep Delay for a specified time
slocate Find files
sort Sort text files
source Run commands from a file `.`
split Split a file into fixed-size pieces
ssh Secure Shell client (remote login program)
strace Trace system calls and signals
su Substitute user identity
sudo Execute a command as another user
sum Print a checksum for a file
suspend Suspend execution of this shell
sync Synchronize data on disk with memory
t
tail Output the last part of file
tar Store, list or extract files in an archive
tee Redirect output to multiple files
test Evaluate a conditional expression
time Measure Program running time
timeout Run a command with a time limit
times User and system times
touch Change file timestamps
top List processes running on the system
traceroute Trace Route to Host
trap Run a command when a signal is set(bourne)
tr Translate, squeeze, and/or delete characters
true Do nothing, successfully
tsort Topological sort
tty Print filename of terminal on stdin
type Describe a command
u
ulimit Limit user resources
umask Users file creation mask
umount Unmount a device
unalias Remove an alias
uname Print system information
unexpand Convert spaces to tabs
uniq Uniquify files
units Convert units from one scale to another
unset Remove variable or function names
unshar Unpack shell archive scripts
until Execute commands (until error)
uptime Show uptime
useradd Create new user account
userdel Delete a user account
usermod Modify user account
users List users currently logged in
uuencode Encode a binary file
uudecode Decode a file created by uuencode
v
vdir Verbosely list directory contents (`ls -l -b`)
vi Text Editor
vmstat Report virtual memory statistics
w
wait Wait for a process to complete
watch Execute/display a program periodically
wc Print byte, word, and line counts
whereis Search the user's $path, man pages and source files for a program
which Search the user's $path for a program file
while Execute commands
who Print all usernames currently logged in
whoami Print the current user id and name (`id -un`)
wget Retrieve web pages or files via HTTP, HTTPS or FTP
write Send a message to another user
x
xargs Execute utility, passing constructed argument list(s)
xdg-open Open a file or URL in the user's preferred application.
yes Print a string until interrupted
zip Package and compress (archive) files.
. Run a command script in the current shell
!! Run the last command again
### Comment / Remark