Console Tricks Seminar¶
1. Introduction: What Is a Shell?¶
Confused?
Maybe this looks more like what you expected:
But is that a shell?
Yes and no. Let's have a look at the definition of a shell:
In computing, a shell is a user interface for access to an operating system's services. In general, operating system shells use either a command-line interface (CLI) or graphical user interface (GUI), depending on a computer's role and particular operation. Wikipedia
Maybe you wouldn't expect it, but the Windows Desktop, including the Windows Explorer, is a shell (and Microsoft actually calls it so).
On Linux systems we usually mean CLIs (command-line interfaces) when we're talking about shells. That doesn't mean that there aren't graphical shells. The X window system + your desktop environment is also a shell.
Back to our Linux “shell”…
What we see here are actually two things:
- A graphical terminal emulator
- A shell command prompt
The terminal window is basically a software version of this little fellow here: a very modern VT100 terminal (therefore the name “terminal emulator”).
And the shell?¶
- The shell itself interprets text commands and provides feedback as text
- Therefore any Linux/UNIX shell can also be operated via serial COM ports
- Anything display-related is responsibility of the terminal
- Colors, formatting, cursor movement etc. are represented as certain (ANSI-standardized) “escape sequences” and interpreted by the displaying terminal
2. Getting Started: The Basics¶
Where do I find stuff?¶
Linux systems (and many other UNIX-like systems) follow a filesystem structure standard called the “Filesystem Hierarchy Standard” (FHS).
Basic principle: All folders fall into these two orthogonal categories: shareable vs. unshareable – static vs. variable
“Shareable” files are those that can be stored on one host and used on others. “Unshareable” files are those that are not shareable. For example, the files in user home directories are shareable whereas device lock files are not.
“Static” files include binaries, libraries, documentation files and other files that do not change without system administrator intervention. “Variable” files are files that are not static.
Furthermore, the standard defines a basic folder tree.
The most important directories:
/– The root filesystem/bin– Essential user command binaries/boot– Static files for the boot loader/dev– Device files/etc- Host-specific system configuration/home– User home directories/lib(/lib64) – Essential shared libraries and kernel modules/media– Mount point for removable media/mnt– Mount point for temporarily mounted filesystems/opt– Add-on application software packages/root– Home directory of the root user/run– Run-time variable data/sbin– System binaries/srv– Data for services provided by this system/tmp– Temporary files
Besides these top-level directories FHS defines two secondary filesystem trees:
/usr– Contains shareable, read-only data used by all users of the system/var– Contains variable (shareable and unshareable) data files (e.g. mail spools, log files, temporary files)
Both directories contain a subtree similar to the top-level root filesystem tree.
Most notable sub-directories of /usr:¶
/usr/bin– Most user command binaries/usr/lib– Libraries and object files for user programs/usr/share– Static, architecture-independent resource files (definitions, icons, media files, *.desktop files, static game data etc.)
Important configuration files¶
Besides the preceding list of directories, there are also some very important configuration files which you should know about.
/etc/passwd– usernames and home directories, readable by everyone (usually no passwords included, despite the name)/etc/shadow– passwords for users, only readable by root/etc/hosts– mapping from 127.0.0.1 and [::1] to localhost, additional hostname configurations/etc/fstab– system-wide mount points for storage devices/etc/profile,/etc/profile.d/*– global shell settings
And Finally the Good Stuff…¶
Now that we have a basic understanding of the system we're working on, we can start with actual command-line fu. \o/
Once logged into a system, you'll be placed in an initial directory, usually your home directory. The folder you're currently residing in is your current working directory (CWD).
To find out where we are, type
pwd
Moving Around¶
Great! we know where we are. I guess you're curious whether there are files in this directory…
ls
Is that all? Probably not.
ls -a
Files that start with a dot are hidden files. Use the -a flag to list hidden files, too.
If you want more information than just the file names, use -l. This will also tell you the owner of a file, its size, permissions and last modification date. -h makes the filesize more human-readable (instead of printing bytes).
ls -lh
You can of course also combine it with the -a flag to show hidden files.
The CWD can be changed with the cd command.
cd exampledirectory1
Verify that we're in a different directory now:
pwd
The shortcuts . and .. represent the current and the parent directory. It's easy to go one level back up:
cd ..
pwd
File Permissions¶
As already seen before, each file and directory has a basic set of permissions assigned to it.
Permissions are either represented as characters (e.g. -rw-rw-r--) or octal numbers (e.g. 644).
Recall our file listing from before:
ls -l
The first character simply shows the type of the file (d for directory, - for normal file).
The other nine (3x3) characters represent the permissions for the owning user, the owning group and others (i.e. everyone else). These are readable (r), writable (w) and executable ( x ).
User and group of a file are shown in the third and fourth column of the file listing.
Manipulating File Permissions¶
File permissions can be changed by the chmod command. The command takes the new permissions and the filename as parameter:
chmod a=rwx examplefile
ls -l
This will set all (a) permissions to rwx (full permissions). Permissions can also be set for only the user (u), the group (g) or everyone else (o). Multiple definitions are separated by commas.
chmod u=rwx,g=rw,o=r examplefile
ls -l
Instead of setting the full permission bitmask with =, individual bits can be added or subtracted using + and -.
The easiest way to add the executable bit for user, group and others is
chmod +x examplefile
ls -l
(note the implied a in front of the +)
In the same way we can remove write permissions for just the group:
chmod g-w examplefile
ls -l
But what about directories?¶
Read and write permissions for folders work basically the same as for files. They define whether you can list a folder's contents (using the ls command) and whether you can add or delete files in this directory.
But what do execute permissions mean on a folder?
Execution rights on a folder allow the user to
- change to it using the
cdcommand - access its files and subfolders
chmod -x exampledirectory1
cd exampledirectory1
Calculating Numeric Permissions¶
A shorter (yet slightly harder to understand) notation is using octal numbers.
Octal permissions are defined as follows:
r := 4
w := 2
x := 1
To get to the desired permission set, add these numbers for each of user, group and others.
The resulting three-digit octal number represents the full set of permissions.
Example:¶
Express the permission set -rwxrw-r-- as octal number.
- Add up permissions for user:
r+w+x = 4+2+1 = 7 - Add up permissions for group:
r+w = 4+2 = 6 - Add up permissions for others:
r = 4
Building a 3-digit number from it, we end up with 764.
chmod 764 examplefile
ls -l
Common numeric permissions¶
Although numeric permissions seem to be complicated, in almost all cases you only need to remember five numbers.
Files:¶
- 644: Normal file permissions, everyone can read, only user can write
- 755: Executable file, everyone can read and execute, only user can write
- 600: Private file, only user can read and write (e.g. SSH private key)
Folders:¶
- 755: Normal folder permissions, everyone can change to it and list contents, only user can add new files
- 700: Private folder, only user can change to it, list contents and add files (e.g. home directory)
Basic File Operations¶
Besides changing file permissions and listing directories you can of course also create, copy, rename, delete and create files and folders.
Copying files¶
Copying files can be done with the cp command:
cp [OPTION]... SOURCE DEST
cp [OPTION]... SOURCE... DIRECTORYCommon options are:
-R, -r, --recursive copy directories recursively
-a, --archive preserve most attributes if possible
(such as owner, permissions), implies `-R`
-v, --verbose explain what is being doneRenaming (moving) files¶
Renaming files can be done with the mv command:
mv [OPTION]... SOURCE DEST
mv [OPTION]... SOURCE... DIRECTORYCommon options are:
-f, --force do not prompt before overwriting
-i, --interactive prompt before overwriting
-v, --verbose explain what is being doneRemoving files¶
Removing/deleting files can be done with the rm command:
rm [OPTION]... FILE...Common options are:
-f, --force ignore nonexistent files and arguments, never prompt
-i prompt before every removal
-r, -R, --recursive remove directories and their contents recursively
(DANGEROUS!!!)
-v, --verbose explain what is being doneRemoving empty directories¶
Removing/deleting empty directories can be done with the rmdir command:
rmdir [OPTION]... DIRECTORYCommon options are:
-v, --verbose output a diagnostic for every directory processedMoney question: why would you need rmdir, when you can use rm -r?
Counter question: Why do you cut your hair with scissors when you could use the lawn mower?
rmdir will fail when the directory is not empty. Similarly, rm (without -r) will fail if you try to delete a directory instead of a file. This is simply a sane precaution. Use rm -r sparsely and only if needed and…
Creating directories¶
Creating new directories can be done with the mkdir command:
mkdir [OPTION]... DIRECTORYCommon options are:
-p, --parents no error if existing, make parent directories as needed
-v, --verbose print a message for each created directoryWorking with Text Files¶
Working with and editing (text) files on the command line is worth a full seminar. There are lots of tools you can use to work very effectively with text files on the command line.
We will cover some of these tools in future lessons, but some are also out of scope of this seminar.
In this lesson I will only present three basic tools: cat, tail and head.
Printing or concatenating files¶
The cat command reads one or multiple files (or standard input), concatenate them and print them on the standard output.
cat [OPTION]... [FILE]...Common options are:
-n, --number number all output lines
-s, --squeeze-blank suppress repeated empty output linescat examplefile
Printing only the last lines of a file¶
Printing only the last n (default: 10) lines of one or multiple files can be accomplished with the tail command.
tail [OPTION]... [FILE]..Common options are:
-n, --lines=K output the last K lines, instead of the last 10;
or use -n +K to output starting with the Kthtail -n2 examplefile
tail -n+2 examplefile
Printing only the first lines of a file¶
Similar to tail, printing the first n (default: 10) lines of one or multiple files can be accomplished with the head command.
head [OPTION]... [FILE]..Common options are:
-n, --lines=[-]K print the first K lines instead of the
first 10; with the leading '-', print
all but the last K lines of each filehead -n2 examplefile
head -n-1 examplefile
Editing Text Files¶
Printing text files is neat, but most of the time we want to edit them.
Using input and output redirection (which we'll discuss in a later chapter) you can actually use cat, head and tail to edit files. But most of the time it is much more convenient to use a text editor.
There are a number of text editors for the command line. The most common are
nano(very easy to use, but not very powerful)vim(very powerful, but takes a long time to learn)emacs(as powerful asvimwith higher RMS factor)
nano¶
nano is very simple to use, but also doesn't provide many editing features.
All keyboard shortcuts are shown at the bottom. ^ means Ctrl. So ^X (the exit command) means Ctrl+X.
If you try to exit the program but changed your file, you will be asked if you want to save.
Correctly configured, vim provides all features of a modern text editor and most of certain development IDEs. However, it takes a lot of time to learn.
There are four modes in vim:
- normal mode (default mode)
- insert mode (the real editing mode)
- visual mode (for selecting text)
- command (ex) mode
When you start vim, you are in normal mode. Tap i to go into edit mode. <ESC> to go back to normal mode.
Type :w<ENTER> in normal mode to go temporarily to command mode and save your file.
Type :q<ENTER> to exit vim. If you changed your file but want to exit without saving, type :q!<ENTER>
There are some great web ressources about learning vim. A small selection:
Additionally, there is also the command vimtutor which comes with vim. It will interactively run you through all the basic editing features, vim provides.
Help! I Can't Remember Everything!¶
Keep calm, none of us can.
If you ever forget how a command works, try running it with --help or, if that doesn't work, with -h (but be careful, in certain cases -h may also mean something else than help). Most commands print basic usage instructions when provided with these flags.
For example:
mkdir --help
If that is not enough, there is almost always a Manual page (short: Manpage), accessible through the man command:
man passwd
Note the (1) behind the title. This tells you that you are currently looking at the manpage for the command passwd. If you want information about the /etc/passwd file, type:
man 5 passwd
There is a total of 9 possible categories for Manpages. An extract from the man Manpage (man man):
1 Executable programs or shell commands
2 System calls (functions provided by the kernel)
3 Library calls (functions within program libraries)
4 Special files (usually found in /dev)
5 File formats and conventions eg /etc/passwd
6 Games
7 Miscellaneous (including macro packages and conventions), e.g. man(7), groff(7)
8 System administration commands (usually only for root)
9 Kernel routines [Non standard]Don't worry, if you can't remember them all. I cannot remember anything except for 1, 5 and 8 either.
Most of the time you'll get the right Manpage without explicitly specifying the number anyway. And if not, you know where too look it up.
End of Lesson 1¶
That marks the end of lesson 1. Next time we'll focus on some more advanced command line techniques. We'll cover:
- Globbing
- Environment variables
- Basic scripting
- Input/Output redirection and Piping
- Background jobs
Thanks for listening.