This document is intended to provide a foundation of skills for using Linux in console or shell mode. The reader is not expected to have any prior knowledge of how Linux works, although in this case the reader should at least look through Linux Concepts and Running Commands before moving on to the Command Reference. The Command Reference introduces the core set of Linux commands categorized by the type of functionality they provide. The final section on Advanced Features illustrates several techniques that will help you get the most out of Linux.
Linux Concepts
Users:
In order to use a Linux machine you must have an account on that machine. The account information specifies your user name, and password, which together control your access to the machine. There is also a unique integer associated with your user account called your user ID or UID.
Groups:
Users working for the same department or on a joint project can be put into a named group. This is simply a method of organizing users. Like with individual users, there is a group id or GID associated with each group.
Files:
All information stored on a Linux machine is stored in files. In addition to containing user or program data a file has information associated with it regarding which user owns it, which users can access it, when it was created, when it was last modified, and how large it is.
Directories:
Directories are like folders. They can contain files as well as other directories, and give structure to the file system. Under Linux there is only one directory tree and it's root is /. Like files, each directory has associated with it information as to it's size and which users are allowed to access it, etc...
Paths:
A path is a string of directories possibly ending with a file name. The directory and file names are separated by / (front slash) characters. An example is: /dir1/dir2/file which is a path to file contained in dir2, which is in turn contained in dir1, which is contained in the root directory, /.
Permissions:
Permissions are an important feature of Linux. They provide security by limiting what actions a user can perform on a given file or directory. Permissions control read, write, and execute access to a file for the file's owner, group and other users. If you try to manipulate a file in a manner for which you do not have sufficient privileges, an error occurs.
Processes:
When a user runs a command Linux creates a process to contain the command's instructions (code). A process also contains control information such as the UID of the user who started the command, and a unique process id or PID. All management of processes is done using this unique PID.
The Shell:
In console mode a user interacts with the machine through a shell. A shell is a program used to start other programs. A shell is configured by setting it's environment (configuration) variables. When you log on to a Linux machine it will automatically create a shell process for you, and set the shell's environment variables to default values. Note that this document assumes the reader is using bash (the Bourne Again SHell), which is the default for most current linux distributions.
Running Commands
Entering Commands:
To enter a command you simply type it's name after the shell prompt and press enter. The shell prompt is usually of the form [user@host directory]$, but is configurable, and can vary from machine to machine. Most commands usually accept several arguments or options (sometimes called flags). Normally arguments are set off using one or two -'s (dashes). If a command requires arguments and none are supplied, the command will usually display a brief usage message describing the arguments it requires. A typical command with arguments might look like:
command -a1 -a2
command --long_argument_name
The PATH:
The PATH is a shell environment variable that determines which directories Linux will look in for a command's program file if a full path to that file is not specified. The PATH variable consists of a string of directory names separated by :'s (colons). Most if not all of the commands listed in the reference reside in directories that are specified in the PATH and can be run simply by entering their names at the prompt. For security reasons the current directory is not in the PATH so to run a program contained in the current directory you must prefix ./ (dot front slash) to the programs name as in:
./command
Command Reference
Getting Help:
Most console Linux commands contain small routines that print out usage information when a -h or --help flag is passed to them. In addition, most Linux machines have literally thousands of pages of documentation stored on them. One of the most convenient sources of information is the man page system. You need only supply the man program with the name of a command as an argument and it will retrieve and display the complete manual for that command.
command -h: Display a brief help message about command.
command --help: Display a brief help message about command.
man command: Show the complete manual page for command.
Listing Files:
One of the most basic operations you can perform is to list the files in a directory with ls. This lets you examine the contents of the directory and find the files you want to work with. If the directory listing is too long to fit on one screen you may want to pipe the out put of ls to a text viewing program like less (see section 4.6).
ls: Lists the content of the current directory.
ls -a: List all files including hidden files.
ls -l: Use long listing format (displays permissions, owner, size, date, etc...).
ls | less: Pipe the directory listing into less for easy viewing.
Changing Directories:
When you log on to a linux machine you are automatically placed into your home directory. To change to another directory you use the cd command. cd accepts a full path to a directory, a path relative to the current directory, or one of a number of special arguments displayed below.
cd path: Change to the directory specified by path.
cd ~ : change to your home directory.
cd - : change to your previous directory.
cd .. : change to the current directory's parent.
Managing Files and Directories
The cp command is useful for making a backup copy of a file or a directory. The mkdir command creates a new, empty directory at the specified location. mv can be used to move a file from one directory or another, or to change it's name by moving it to a different file name in the same directory. The rm lets you delete old files and free up storage space on the system.You can use rm -R to recursively delete the contents of a directory, and then delete the directory itself. rmdir will remove a directory, but only if it is already open.
cp source_path destination_path: Copy files from source to destination.
cp -R source_path destination_path: Recursive copy.
mkdir directory: Make a new directory.
mv source_path destination_path: Move or rename files.
rm files: Remove files.
rm -R directory: Recursive delete.
rmdir directory: Remove an empty directory.
touch file: Create an empty file.
Locating Files
When your files are spread over many directories, or you are looking for a specific system file, find and locate can save you manually searching directories. find will start from the specified directory and search the whole subtree from that point on. locate, on the other hand, maintains a database of all the files on the system. It simply looks in the database to see if any files match the supplied name. locate is much faster than find, but it's database is only updated once a day so it will not be able to find recently created files.
find path -name filename: Find the file called filename starting the search from path.
locate filename Find files in the database whose names contain filename.
Working with Text Files
To view the contents of a short text file use cat to print it to the screen. less allows you to scroll through a long text file using the regular arrow keys as well as the page up, and page down keys. grep is a tool for searching text files for a specific string. When grep finds an instance of the string it will print the whole line it occurs on. sort, as the name implies, will sort a text file and display the results on the screen. Lastly, pico is a small text editor with built in search and spell check features.
cat filename: Print the contents of a text file called filename to the screen.
grep string filename: Search for and display lines of filename that contain occurrences of string.
less filename: Scroll through the content of a text file. Press q to quit.
pico filename: Open filename in a simple text editor.
sort filename: Sort the lines in filename by alphabetical order.
Uncompressing Archives
When retrieving files from the Internet they are usually compressed in some way to save space. Most often a large number of files will be compressed into a single archive file for easier downloading. To get at the files in the archive you need to determine how the archive was created. Normally the file extension of the archive will indicate which tool is required to uncompress it.
bunzip2 filename.bz2: Uncompress a bzip2 file (*.bz2). Used for big files.
gunzip filename.gz: Uncompress a gzipped (.gz) archive.
unzip filename.zip: Uncompress a PkZip or WinZip file (*.zip)
tar -xvf filename.tar: Untar a tar (.tar) archive (tarball).
tar -xvzf filename.tar.gz: Uncompress and untar a compressed tar archive (*.tar.gz or *.tgz).
Using Network Services
Network services allow you to access resources on other machines (servers) in the network from your machine. The ftp program allows you to connect to an FTP (file transfer protocol) server. An FTP server will let you upload (put) an download (get) files. FTP is not a secure protocol and all communications to the FTP server are sent as plain text, including your password. The ping command will issue a ping request to the specified server. Most machines will answer a ping request if they are connected to the network and working correctly. The ssh program will let you connect to a server running the Secure Shell protocol. Using ssh you can log on to another machine and use it in console mode from your own machine. SSH is a secure protocol so all communications with the remote server are encrypted. telnet serves the same purpose as ssh, but is an older, unsecured protocol. Some older machines only run TELNET servers, but whenever possible ssh should be used instead.
ftp server: Connect to an FTP server.
ping server: Send a ping request to server.
ssh -l user server: Connect to a remote machine using the Secure Shell protocol.
telnet server: Connect to another machine using the TELNET protocol.
Printing
Access to printing from the Linux console is provided by the Lpr (Line PRinter) set of commands. Using the lpr command you can print a text or PostScript (.ps) document to any attached printer. To see what printers are available on the system you can use the lpstat command. The lpq command lets you examine the contents of the print queue for a given printer. The display shows the numeric job ID, owner, file name of each job in the queue. Lastly, the lprm command will remove a job from a print queue by it's job ID. Of course, you may only remove a job from a queue if you placed it there initially.
lpq -Pprinter: Show jobs in the print queue for printer.
lpr -Pprinter document: Print document to printer.
lprm -Pprinter n: Remove job with ID n from printer.
lpstat -a: Show status information for all available printers.
Viewing System Information
The following commands display miscellaneous information about the system.
date: Print the operating system time and date.
df -h: Print disk usage information.
free: Print memory usage information.
history: Show commands executed on the current account.
hostname: Print the name of the local machine (host).
pwd: Print working (current) directory.
rwho -a: List all users logged into the network.
uptime: Print the amount of time since the last reboot.
who: List the users logged into the machine.
whoami: Print your login name.
Controlling Processes
ps: List the active processes you own and their PID's.
ps -aux: List all the active processes, together with the name of the user that owns each process.
top: Show a continually updated listing of active processes.
command &: Run command in the background.
fg: Bring a background or stopped process to the foreground.
bg: Send the process to the background. The same can be accomplished with Ctrl-z.
kill pid: Force a process to shutdown. First determine the pid of the process to kill using ps.
killall -9 name: Kill all processes with the specified name.
nice program level: Run program with niceness level. Specifying a high nice level will make the program run with a lower priority.
Changing File Permissions
chown user.group filename: Change the user which owns filename to user, and the group which owns filename to group. Of course, you can only chown a file you already own.
chmod [augo][+-][rwx] filename: Modify the access permissions on filename by granting or revoking (+-) read, write, or execute (rwx) access to all, owner, group, or other users (augo). Use with care. You can deny yourself access to your own files by setting incorrect permissions on them.
Advanced Features
Redirection:
The shell gives all programs access to three pointers called standard input (STDIN), standard output (STDOUT), and standard error (STDERR) when the program is started. Normally STDIN is connected to the keyboard, while STDOUT and STDERR are connected to the screen. However, with redirection you can change where a program gets it's input from, and where it sends it's output to. The redirection operator > lets you specify a file name to receive the STDOUT (output) of the command, while the < operator lets you connect STDIN to a file containing input. Some examples are:
command1 > output.txt send output to output.txt
command1 < input.txt get input from input.txt
Piping:
The main purpose of piping is to redirect STDOUT from one process to STDIN of another process. The pipe operator is |, the pipe character (found above the enter key on most keyboards). To pipe the output of command1 to the input of command2 you would enter the following:
command1 | command2
https://www.fosslinux.com/45587/linux-command-cheat-sheet.htm
------------------------------------------------------------------------------------------
Most Linux systems use the same default keyboard shortcut to start terminal: Ctrl-Alt-T.
pwd
You should see a directory path printed out (probably something like /home/YOUR_USERNAME), then another copy of that odd bit of text.
Be extra careful with case when typing in the command line. Typing PWD instead of pwd will produce an error, but sometimes the wrong case can result in a command appearing to run, but not doing what you expected. We’ll look at case a little more on the next page but, for now, just make sure to type all the following lines in exactly the case that’s shown.
Now to the command itself. pwd is an abbreviation of ‘print working directory’. All it does is print out the shell’s current working directory.
cd /
pwd
Note that the directory separator is a forward slash (”/”), not the backslash that you may be used to from Windows or DOS systems
Beware: although the “/” directory is sometimes referred to as the root directory, the word “root” has another meaning. root is also the name that has been used for the superuser since the early days of Unix. The superuser, as the name suggests, has more powers than a normal user, so can easily wreak havoc with a badly typed command. We’ll look at the superuser account more in section 7. For now you only have to know that the word “root” has multiple meanings in the Linux world, so context is important.
cd home
pwd
To go up to the parent directory, in this case back to “/”, use the special syntax of two dots (..) when changing directory (note the space between cd and .., unlike in DOS you can’t just type cd.. as one command):
cd ..
pwd
Typing cd on its own is a quick shortcut to get back to your home directory:
cd
pwd
You can also use .. more than once if you have to move up through multiple levels of parent directories:
cd ../..
pwd
Notice that in the previous example we described a route to take through the directories. The path we used means “starting from the working directory, move to the parent / from that new location move to the parent again”. So if we wanted to go straight from our home directory to the “etc” directory (which is directly inside the root of the file system), we could use this approach:
cd
pwd
cd ../../etc
pwd
Most of the examples we’ve looked at so far use relative paths. That is, the place you end up at depends on your current working directory. Consider trying to cd into the “etc” folder. If you’re already in the root directory that will work fine:
cd /
pwd
cd etc
pwd
But what if you’re in your home directory?
cd
pwd
cd etc
pwd
You’ll see an error saying “No such file or directory” before you even get to run the last pwd. Changing directory by specifying the directory name, or using .. will have different effects depending on where you start from. The path only makes sense relative to your working directory.
But we have seen two commands that are absolute. No matter what your current working directory is, they’ll have the same effect. The first is when you run cd on its own to go straight to your home directory. The second is when you used cd / to switch to the root directory. In fact any path that starts with a forward slash is an absolute path. You can think of it as saying “switch to the root directory, then follow the route from there”. That gives us a much easier way to switch to the etc directory, no matter where we currently are in the file system:
cd
pwd
cd /etc
pwd
It also gives us another way to get back to your home directory, and even to the folders within it. Suppose you want to go straight to your “Desktop” folder from anywhere on the disk (note the upper-case “D”). In the following command you’ll need to replace USERNAME with your own username, the whoami command will remind you of your username, in case you’re not sure:
whoami
cd /home/USERNAME/Desktop
pwd
There’s one other handy shortcut which works as an absolute path. As you’ve seen, using “/” at the start of your path means “starting from the root directory”. Using the tilde character (”~”) at the start of your path similarly means “starting from my home directory”.
cd ~
pwd
cd ~/Desktop
pwd
Now that odd text in the prompt might make a bit of sense. Have you noticed it changing as you move around the file system? On a Ubuntu system it shows your username, your computer’s network name and the current working directory. But if you’re somewhere inside your home directory, it will use “~” as an abbreviation. Let’s wander around the file system a little, and keep an eye on the prompt as you do so:
cd
cd /
cd ~/Desktop
cd /etc
cd /var/log
cd ..
cd
mkdir /tmp/tutorial
cd /tmp/tutorial
Notice the use of an absolute path, to make sure that we create the tutorial directory inside /tmp. Without the forward slash at the start the mkdir command would try to find a tmp directory inside the current working directory, then try to create a tutorial directory inside that. If it couldn’t find a tmp directory the command would fail.
In case you hadn’t guessed, mkdir is short for ‘make directory’. Now that we’re safely inside our test area (double check with pwd if you’re not certain), let’s create a few subdirectories:
mkdir dir1 dir2 dir3
There’s something a little different about that command. So far we’ve only seen commands that work on their own (cd, pwd) or that have a single item afterwards (cd /, cd ~/Desktop). But this time we’ve added three things after the mkdir command. Those things are referred to as parameters or arguments, and different commands can accept different numbers of arguments. The mkdir command expects at least one argument, whereas the cd command can work with zero or one, but no more. See what happens when you try to pass the wrong number of parameters to a command:
mkdir
cd /etc ~/Desktop
Back to our new directories. The command above will have created three new subdirectories inside our folder. Let’s take a look at them with the ls (list) command:
ls
If you’ve followed the last few commands, your terminal should be looking something like this:
Notice that mkdir created all the folders in one directory. It didn’t create dir3 inside dir2 inside dir1, or any other nested structure. But sometimes it’s handy to be able to do exactly that, and mkdir does have a way:
mkdir -p dir4/dir5/dir6
ls
This time you’ll see that only dir4 has been added to the list, because dir5 is inside it, and dir6 is inside that. Later we’ll install a useful tool to visualise the structure, but you’ve already got enough knowledge to confirm it:
cd dir4
ls
cd dir5
ls
cd ../..
The “-p” that we used is called an option or a switch (in this case it means “create the parent directories, too”). Options are used to modify the way in which a command operates, allowing a single command to behave in a variety of different ways. Unfortunately, due to quirks of history and human nature, options can take different forms in different commands. You’ll often see them as single characters preceded by a hyphen (as in this case), or as longer words preceded by two hyphens. The single character form allows for multiple options to be combined, though not all commands will accept that. And to confuse matters further, some commands don’t clearly identify their options at all, whether or not something is an option is dictated purely by the order of the arguments! You don’t need to worry about all the possibilities, just know that options exist and they can take several different forms. For example the following all mean exactly the same thing:
# Don't type these in, they're just here for demonstrative purposes
mkdir --parents --verbose dir4/dir5
mkdir -p --verbose dir4/dir5
mkdir -p -v dir4/dir5
mkdir -pv dir4/dir5
Now we know how to create multiple directories just by passing them as separare arguments to the mkdir command. But suppose we want to create a directory with a space in the name? Let’s give it a go:
mkdir another folder
ls
You probably didn’t even need to type that one in to guess what would happen: two new folders, one called another and the other called folder. If you want to work with spaces in directory or file names, you need to escape them. Don’t worry, nobody’s breaking out of prison; escaping is a computing term that refers to using special codes to tell the computer to treat particular characters differently to normal. Enter the following commands to try out different ways to create folders with spaces in the name:
mkdir "folder 1"
mkdir 'folder 2'
mkdir folder\ 3
mkdir "folder 4" "folder 5"
mkdir -p "folder 6"/"folder 7"
ls
Although the command line can be used to work with files and folders with spaces in their names, the need to escape them with quote marks or backslashes makes things a little more difficult. You can often tell a person who uses the command line a lot just from their file names: they’ll tend to stick to letters and numbers, and use underscores (“_”) or hyphens (”-”) instead of spaces.
Creating files using redirection
Our demonstration folder is starting to look rather full of directories, but is somewhat lacking in files. Let’s remedy that by redirecting the output from a command so that, instead of being printed to the screen, it ends up in a new file. First, remind yourself what the ls command is currently showing:
ls
Suppose we wanted to capture the output of that command as a text file that we can look at or manipulate further. All we need to do is to add the greater-than character (”>”) to the end of our command line, followed by the name of the file to write to:
ls > output.txt
This time there’s nothing printed to the screen, because the output is being redirected to our file instead. If you just run ls on its own you should see that the output.txt file has been created. We can use the cat command to look at its content:
cat output.txt
echo "This is a test"
Yes, echo just prints its arguments back out again (hence the name). But combine it with a redirect, and you’ve got a way to easily create small test files:
echo "This is a test" > test_1.txt
echo "This is a second test" > test_2.txt
echo "This is a third test" > test_3.txt
ls
You should cat each of these files to theck their contents. But cat is more than just a file viewer - its name comes from ‘concatenate’, meaning “to link together”. If you pass more than one filename to cat it will output each of them, one after the other, as a single block of text:
cat test_1.txt test_2.txt test_3.txt
Where you want to pass multiple file names to a single command, there are some useful shortcuts that can save you a lot of typing if the files have similar names. A question mark (”?”) can be used to indicate “any single character” within the file name. An asterisk (”*”) can be used to indicate “zero or more characters”. These are sometimes referred to as “wildcard” characters. A couple of examples might help, the following commands all do the same thing:
cat test_1.txt test_2.txt test_3.txt
cat test_?.txt
cat test_*
If you look at the output of ls you’ll notice that the only files or folders that start with “t” are the three test files we’ve just created, so you could even simplify that last command even further to cat t*, meaning “concatenate all the files whose names start with a t and are followed by zero or more other characters”. Let’s use this capability to join all our files together into a single new file, then view it:
cat t* > combined.txt
cat combined.txt
What do you think will happen if we run those two commands a second time? Will the computer complain, because the file already exists? Will it append the text to the file, so it contains two copies? Or will it replace it entirely? Give it a try to see what happens, but to avoid typing the commands again you can use the Up Arrow and Down Arrow keys to move back and forth through the history of commands you’ve used. Press the Up Arrow a couple of times to get to the first cat and press Enter to run it, then do the same again to get to the second.
As you can see, the file looks the same. That’s not because it’s been left untouched, but because the shell clears out all the content of the file before it writes the output of your cat command into it. Because of this, you should be extra careful when using redirection to make sure that you don’t accidentally overwrite a file you need. If you do want to append to, rather than replace, the content of the files, double up on the greater-than character:
cat t* >> combined.txt
echo "I've appended a line!" >> combined.txt
cat combined.txt
Repeat the first cat a few more times, using the Up Arrow for convenience, and perhaps add a few more arbitrary echo commands, until your text document is so large that it won’t all fit in the terminal at once when you use cat to display it. In order to see the whole file we now need to use a different program, called a pager (because it displays your file one “page” at a time). The standard pager of old was called more, because it puts a line of text at the bottom of each page that says “–More–” to indicate that you haven’t read everything yet. These days there’s a far better pager that you should use instead: because it replaces more, the programmers decided to call it less.
less combined.txt
When viewing a file through less you can use the Up Arrow, Down Arrow, Page Up, Page Down, Home and End keys to move through your file. Give them a try to see the difference between them. When you’ve finished viewing your file, press q to quit less and return to the command line.
Unix systems are case-sensitive, that is, they consider “A.txt” and “a.txt” to be two different files. If you were to run the following lines you would end up with three files:
echo "Lower case" > a.txt
echo "Upper case" > A.TXT
echo "Mixed case" > A.txt
Generally you should try to avoid creating files and folders whose name only varies by case. Not only will it help to avoid confusion, but it will also prevent problems when working with different operating systems. Windows, for example, is case-insensitive, so it would treat all three of the file names above as being a single file, potentially causing data loss or other problems.
You might be tempted to just hit the Caps Lock key and use upper case for all your file names. But the vast majority of shell commands are lower case, so you would end up frequently having to turn it on and off as you type. Most seasoned command line users tend to stick primarily to lower case names for their files and directories so that they rarely have to worry about file name clashes, or which case to use for each letter in the name.
When you consider both case sensitivity and escaping, a good rule of thumb is to keep your file names all lower case, with only letters, numbers, underscores and hyphens. For files there’s usually also a dot and a few characters on the end to indicate the type of file it is (referred to as the “file extension”). This guideline may seem restrictive, but if you end up using the command line with any frequency you’ll be glad you stuck to this pattern.
mv combined.txt dir1
You can confirm that the job has been done by using ls to see that it’s missing from the working directory, then cd dir1 to change into dir1, ls to see that it’s in there, then cd .. to move the working directory back again. Or you could save a lot of typing by passing a path directly to the ls command to get straight to the confirmation you’re looking for:
ls dir1
Now suppose it turns out that file shouldn’t be in dir1 after all. Let’s move it back to the working directory. We could cd into dir1 then use mv combined.txt .. to say “move combined.txt into the parent directory”. But we can use another path shortcut to avoid changing directory at all. In the same way that two dots (..) represents the parent directory, so a single dot (.) can be used to represent the current working directory. Because we know there’s only one file in dir1 we can also just use “*” to match any filename in that directory, saving ourselves a few more keystrokes. Our command to move the file back into the working directory therefore becomes this (note the space before the dot, there are two parameters being passed to mv):
mv dir1/* .
The mv command also lets us move more than one file at a time. If you pass more than two arguments, the last one is taken to be the destination directory and the others are considered to be files (or directories) to move. Let’s use a single command to move combined.txt, all our test_n.txt files and dir3 into dir2. There’s a bit more going on here, but if you look at each argument at a time you should be able to work out what’s happening:
mv combined.txt test_* dir3 dir2
ls
ls dir2
With combined.txt now moved into dir2, what happens if we decide it’s in the wrong place again? Instead of dir2 it should have been put in dir6, which is the one that’s inside dir5, which is in dir4. With what we now know about paths, that’s no problem either:
mv dir2/combined.txt dir4/dir5/dir6
ls dir2
ls dir4/dir5/dir6
Notice how our mv command let us move the file from one directory into another, even though our working directory is something completely different. This is a powerful property of the command line: no matter where in the file system you are, it’s still possible to operate on files and folders in totally different locations.
Since we seem to be using (and moving) that file a lot, perhaps we should keep a copy of it in our working directory. Much as the mv command moves files, so the cp command copies them (again, note the space before the dot):
cp dir4/dir5/dir6/combined.txt .
ls dir4/dir5/dir6
ls
We can use the cp command again, but instead of giving it a directory path as the last argument, we’ll give it a new file name instead:
cp combined.txt backup_combined.txt
ls
That’s good, but perhaps the choice of backup name could be better. Why not rename it so that it will always appear next to the original file in a sorted list. The traditional Unix command line handles a rename as though you’re moving the file from one name to another, so our old friend mv is the command to use. In this case you just specify two arguments: the file you want to rename, and the new name you wish to use.
mv backup_combined.txt combined_backup.txt
ls
This also works on directories, giving us a way to sort out those difficult ones with spaces in the name that we created earlier. To avoid re-typing each command after the first, use the Up Arrow to pull up the previous command in the history. You can then edit the command before you run it by moving the cursor left and right with the arrow keys, and removing the character to the left with Backspace or the one the cursor is on with Delete. Finally, type the new character in place, and press Enter or Return to run the command once you’re finished. Make sure you change both appearances of the number in each of these lines.
mv "folder 1" folder_1
mv "folder 2" folder_2
mv "folder 3" folder_3
mv "folder 4" folder_4
mv "folder 5" folder_5
mv "folder 6" folder_6
ls
Now we know how to move, copy and rename files and directories. Given that these are just test files, however, perhaps we don’t really need three different copies of combined.txt after all. Let’s tidy up a bit, using the rm (remove) command:
rm dir4/dir5/dir6/combined.txt combined_backup.txt
Perhaps we should remove some of those excess directories as well:
rm folder_*
What happened there? Well, it turns out that rm does have one little safety net. Sure, you can use it to delete every single file in a directory with a single command, accidentally wiping out thousands of files in an instant, with no means to recover them. But it won’t let you delete a directory. I suppose that does help prevent you accidentally deleting thousands more files, but it does seem a little petty for such a destructive command to balk at removing an empty directory. Luckily there’s an rmdir (remove directory) command that will do the job for us instead:
rmdir folder_*
Well that’s a little better, but there’s still an error. If you run ls you’ll see that most of the folders have gone, but folder_6 is still hanging around. As you may recall, folder_6 still has a folder 7 inside it, and rmdir will only delete empty folders. Again, it’s a small safety net to prevent you from accidentally deleting a folder full of files when you didn’t mean to.
In this case, however, we do mean to. The addition of options to our rm or rmdir commands will let us perform dangerous actions without the aid of a safety net! In the case of rmdir we can add a -p switch to tell it to also remove the parent directories. Think of it as the counterpoint to mkdir -p. So if you were to run rmdir -p dir1/dir2/dir3 it would first delete dir3, then dir2, then finally delete dir1. It still follows the normal rmdir rules of only deleting empty directories though, so if there was also a file in dir1, for example, only dir3 and dir2 would get removed.
A more common approach, when you’re really, really, really sure you want to delete a whole directory and anything within it, is to tell rm to work recursively by using the -r switch, in which case it will happily delete folders as well as files. With that in mind, here’s the command to get rid of that pesky folder_6 and the subdirectory within it:
rm -r folder_6
ls
Remember: although rm -r is quick and convenient, it’s also dangerous. It’s safest to explicitly delete files to clear out a directory, then cd .. to the parent before using rmdir to remove it.
Unlike graphical interfaces, rm doesn’t move files to a folder called “trash” or similar. Instead it deletes them totally, utterly and irrevocably. You need to be ultra careful with the parameters you use with rm to make sure you’re only deleting the file(s) you intend to. You should take particular care when using wildcards, as it’s easy to accidentally delete more files than you intended. An errant space character in your command can change it completely: rm t* means “delete all the files starting with t”, whereas rm t * means “delete the file t as well as any file whose name consists of zero or more characters, which would be everything in the directory! If you’re at all uncertain use the -i (interactive) option to rm, which will prompt you to confirm the deletion of each file; enter Y to delete it, N to keep it, and press Ctrl-C to stop the operation entirely.
Let’s start with a simple question. How many lines are there in your combined.txt file? The wc (word count) command can tell us that, using the -l switch to tell it we only want the line count (it can also do character counts and, as the name suggests, word counts):
wc -l combined.txt
Similarly, if you wanted to know how many files and folders are in your home directory, and then tidy up after yourself, you could do this:
ls ~ > file_list.txt
wc -l file_list.txt
rm file_list.txt
That method works, but creating a temporary file to hold the output from ls only to delete it two lines later seems a little excessive. Fortunately the Unix command line provides a shortcut that avoids you having to create a temporary file, by taking the output from one command (referred to as standard output or STDOUT) and feeding it directly in as the input to another command (standard input or STDIN). It’s as though you’ve connected a pipe between one command’s output and the next command’s input, so much so that this process is actually referred to as piping the data from one command to another. Here’s how to pipe the output of our ls command into wc:
ls ~ | wc -l
Notice that there’s no temporary file created, and no file name needed. Pipes operate entirely in memory, and most Unix command line tools will expect to receive input from a pipe if you don’t specify a file for them to work on. Looking at the line above, you can see that it’s two commands, ls ~ (list the contents of the home directory) and wc -l (count the lines), separated by a vertical bar character (”|”). This process of piping one command into another is so commonly used that the character itself is often referred to as the pipe character, so if you see that term you now know it just means the vertical bar.
Note that the spaces around the pipe character aren’t important, we’ve used them for clarity, but the following command works just as well, this time for telling us how many items are in the /etc directory:
ls /etc|wc -l
Phew! That’s quite a few files. If we wanted to list them all it would clearly fill up more than a single screen. As we discovered earlier, when a command produces a lot of output, it’s better to use less to view it, and that advice still applies when using a pipe (remember, press q to quit):
ls /etc | less
Going back to our own files, we know how to get the number of lines in combined.txt, but given that it was created by concatenating the same files multiple times, I wonder how many unique lines there are? Unix has a command, uniq, that will only output unique lines in the file. So we need to cat the file out and pipe it through uniq. But all we want is a line count, so we need to use wc as well. Fortunately the command line doesn’t limit you to a single pipe at a time, so we can continue to chain as many commands as we need:
cat combined.txt | uniq | wc -l
That line probably resulted in a count that’s pretty close to the total number of lines in the file, if not exactly the same. Surely that can’t be right? Lop off the last pipe to see the output of the command for a better idea of what’s happening. If your file is very long, you might want to pipe it through less to make it easier to inspect:
cat combined.txt | uniq | less
It appears that very few, if any, of our duplicate lines are being removed. To understand why, we need to look at the documentation for the uniq command. Most command line tools come with a brief (and sometimes not-so-brief) instruction manual, accessed through the man (manual) command. The output is automatically piped through your pager, which will typically be less, so you can move back and forth through the output, then press q when you’re finished:
man uniq
Because this type of documentation is accessed via the man command, you’ll hear it referred to as a “man page”, as in “check the man page for more details”. The format of man pages is often terse, think of them more as a quick overview of a command than a full tutorial. They’re often highly technical, but you can usually skip most of the content and just look for the details of the option or argument you’re using.
The uniq man page is a typical example in that it starts with a brief one-line description of the command, moves on to a synopsis of how to use it, then has a detailed description of each option or parameter. But whilst man pages are invaluable, they can also be inpenetrable. They’re best used when you need a reminder of a particular switch or parameter, rather than as a general resource for learning how to use the command line. Nevertheless, the first line of the DESCRIPTION section for man uniq does answer the question as to why duplicate lines haven’t been removed: it only works on adjacent matching lines.
The question, then, is how to rearrange the lines in our file so that duplicate entries are on adjacent lines. If we were to sort the contents of the file alphabetically, that would do the trick. Unix offers a sort command to do exactly that. A quick check of man sort shows that we can pass a file name directly to the command, so let’s see what it does to our file:
sort combined.txt | less
You should be able to see that the lines have been reordered, and it’s now suitable for piping straight into uniq. We can finally complete our task of counting the unique lines in the file:
sort combined.txt | uniq | wc -l
By minimising the amount of time spent logged in as root, the use of su reduces the window of opportunity in which to make a catastrophic mistake. Despite that, human nature being what it is, many administrators have been guilty of leaving long-running terminals open in which they’ve used su to switch to the root account. In that respect su was only a small step forward for security.
When using su your entire terminal session is switched to the other user. Commands that don’t need root access, something as mundane as pwd or ls, would be run under the auspices of the superuser, increasing the risk of a bug in the program causing major problems. Worse still, if you lose track of which user you’re currently operating as, you might issue a command that is fairly benign when run as a user, but which could destroy the entire system if run as root.
Better to disable the root account entirely and then, instead of allowing long-lived terminal sessions with dangerous powers, require the user to specifically request superuser rights on a per-command basis. The key to this approach is a command called sudo (as in “switch user and do this command”).
sudo is used to prefix a command that has to be run with superuser privileges. A configuration file is used to define which users can use sudo, and which commands they can run. When running a command like this, the user is prompted for their own password, which is then cached for a period of time (defaulting to 15 minutes), so if they need to run multiple superuser-level commands they don’t keep getting continually asked to type it in.
Assuming you’re on a Linux system that uses sudo, and your account is configured as an administrator, try the following to see what happens when you try to access a file that is considered sensitive (it contains encrypted passwords):
cat /etc/shadow
sudo cat /etc/shadow
If you enter your password when prompted you should see the contents of the /etc/shadow file. Now clear the terminal by running the reset command, and run sudo cat /etc/shadow again. This time the file will be displayed without prompting you for a password, as it’s still in the cache.
For instructions targeting Ubuntu, a common appearance of sudo is to install new software onto your system using the apt or apt-get commands. If the instructions require you to first add a new software repository to your system, using the apt-add-repository command, by editing files in /etc/apt, or by using a “PPA” (Personal Package Archive), you should be careful as these sources are not curated by Canonical. But often the instructions just require you to install software from the standard repositories, which should be safe.
One trick with sudo is to use it to run the su command. This will give you a root shell even if the root account is disabled. It can be useful when you need to run a series of commands as the superuser, to avoid having to prefix them all with sudo, but it opens you up to exactly the same kind of problems that were described for su above. If you follow any instructions that tell you to run sudo su, be aware that every command after that will be running as the root user.
Before we conclude this tutorial it’s worth mentioning hidden files (and folders). These are commonly used on Linux systems to store settings and configuration data, and are typically hidden simply so that they don’t clutter the view of your own files. There’s nothing special about a hidden file or folder, other than it’s name: simply starting a name with a dot (”.”) is enough to make it disappear.
cd /tmp/tutorial
ls
mv combined.txt .combined.txt
ls
You can still work with the hidden file by making sure you include the dot when you specify its file name:
cat .combined.txt
mkdir .hidden
mv .combined.txt .hidden
less .hidden/.combined.txt
If you run ls you’ll see that the .hidden directory is, as you might expect, hidden. You can still list its contents using ls .hidden, but as it only contains a single file which is, itself, hidden you won’t get much output. But you can use the -a (show all) switch to ls to make it show everything in a directory, including the hidden files and folders:
ls
ls -a
ls .hidden
ls -a .hidden
Notice that the shortcuts we used earlier, . and .., also appear as though they’re real directories.
As for our recently installed tree command, that works in a similar way (except without an appearance by . and ..):
tree
tree -a
Switch back to your home directory (cd) and try running ls without and then with the -a switch. Pipe the output through wc -l to give you a clearer idea of how many hidden files and folders have been right under your nose all this time. These files typically store your personal configuration, and is how Unix systems have always offered the capability to have system-level settings (usually in /etc) that can be overridden by individual users (courtesy of hidden files in their home directory).
rm -r /tmp/tutorial
ls /tmp
As a last step, let’s close the terminal. You can just close the window, but it’s better practice to log out of the shell. You can either use the logout command, or the Ctrl-D keyboard shortcut. If you plan to use the terminal a lot, memorising Ctrl-Alt-T to launch the terminal and Ctrl-D to close it will soon make it feel like a handy assistant that you can call on instantly, and dismiss just as easily.
Source: https://ubuntu.com/tutorials/command-line-for-beginners#1-overview
No comments:
Post a Comment