Bash
- The correct title of this article is bash. The initial letter is capitalized due to technical restrictions.
Bash is a Linux command shell written for the GNU project. Its name is an acronym for Bourne-again shell—a pun on the Bourne shell (sh), which was an early, important Unix shell.
Bash is the default shell on most Linux systems and was previously my favourite shell. After nearly 14 years of using the bash shell as my primary shell, I switched to Z shell (zsh). It is awesome and extremely powerful!
see: Bash/scripts for examples
Contents
Bash builtins
A shell builtin is a command or a function, called from a shell, that is executed directly in the shell itself, instead of an external executable program which the shell would load and execute.
Shell builtins work significantly faster than external programs, because there is no program loading overhead. However, their code is inherently present in the shell, and thus modifying or updating them requires modifications to the shell. Therefore shell builtins are usually used for simple, almost trivial, functions, such as text output. Because of the nature of Linux, some functions of the operating system have to be implemented as shell builtins. The most notable example is cd, which changes the working directory of the shell. Because each executable program runs in a separate process, and working directories are specific to each process, loading cd as an external program would not affect the working directory of the shell that loaded it.
bash, :, ., [, alias, bg, bind, break, builtin, cd, command, compgen, complete, continue, declare, dirs, disown, echo, enable, eval, exec, exit, export, fc, fg, getopts, hash, help, history, jobs, kill, let, local, logout, popd, printf, pushd, pwd, read, readonly, return, set, shift, shopt, source, suspend, test, times, trap, type, typeset, ulimit, umask, unalias, unset, wait
Bash shell shortcuts
CTRL Key Bound
Basic commands | |||
---|---|---|---|
Command | Description | ||
Ctrl + a | Jump to the start of the line | ||
Ctrl + b | Move back a char | ||
Ctrl + c | Terminate the command | ||
Ctrl + d | Delete from under the cursor | ||
Ctrl + e | Jump to the end of the line | ||
Ctrl + f | Move forward a char | ||
Ctrl + h | Backspace | ||
Ctrl + k | Delete to EOL | ||
Ctrl + l | Clear the screen | ||
Ctrl + n | Next command line (useful for "scrolling" with Ctrl + p) | ||
Ctrl + p | Previous command line | ||
Ctrl + r | Search the history backwards | ||
Ctrl + R | Search the history backwards with multi occurrence | ||
Ctrl + u | Delete backward from cursor | ||
Ctrl + xx | Move between EOL and current cursor position | ||
Ctrl + x @ | Show possible hostname completions | ||
Ctrl + w | deletes the token left of the cursor | ||
Ctrl + z | Suspend / Stop the command | ||
Ctrl + / | Undo last command-line edit |
ALT Key Bound
Basic commands | |||
---|---|---|---|
Command | Description | ||
Alt + < | Move to the first line in the history | ||
Alt + > | Move to the last line in the history | ||
Alt + ? | Show current completion list | ||
Alt + * | Insert all possible completions | ||
Alt + / | Attempt to complete filename | ||
Alt + . | Yank last argument to previous command | ||
Alt + b | Move backward | ||
Alt + c | Capitalize the word | ||
Alt + d | Delete word | ||
Alt + f | Move forward | ||
Alt + l | Make word lowercase | ||
Alt + n | Search the history forwards non-incremental | ||
Alt + p | Search the history backwards non-incremental | ||
Alt + r | Recall command | ||
Alt + t | Move words around | ||
Alt + u | Make word uppercase | ||
Alt + back-space | Delete backward from cursor |
More Special Keybindings
Basic commands | |||
---|---|---|---|
Command | Description | ||
$ 2T | All available commands(common) | ||
$ (string)2T | All available commands starting with (string) | ||
$ /2T | Entire directory structure including Hidden one | ||
$ 2T | Only Sub Dirs inside including Hidden one | ||
$ *2T | Only Sub Dirs inside without Hidden one | ||
$ ~2T | All Present Users on system from "/etc/passwd" | ||
$ $2T | All Sys variables | ||
$ @2T | Entries from "/etc/hosts" | ||
$ =2T | Output like ls or dir |
Bash Bang (!) commands
Re-run all or part of a previous command.
!! Run the last command again !foo Run the most recent command that starts with 'foo' (e.g. !ls) !foo:p Print out the command that !foo would run also add it to the command history !$ Run the last word of the previous command (same as Alt + .) !$:p Print out the word that !$ would substitute !* Run the previous command except for the last word !*:p Print out the previous command except for the last word ^foo^bar Run the previous command replacing foo with bar
Bash syntax highlights
Bash's command syntax is a superset of the Bourne shell's command syntax. The definitive specification of Bash's command syntax is the Bash Reference Manual distributed by the GNU project. This section highlights some of Bash's unique syntax features.
The vast majority of Bourne shell scripts can be executed without alteration by Bash, with the exception of those Bourne shell scripts that happen to reference a Bourne special variable or to use a Bourne builtin command. The Bash command syntax includes ideas drawn from the Korn shell (ksh) and the C shell (csh), such as command-line editing, command history, the directory stack, the $RANDOM and $PPID variables, and POSIX command substitution syntax: $(...). When being used as an interactive command shell, Bash supports completion of partly typed-in program names, filenames, variable names, etc. when the user presses the TAB key.
Bash syntax has many extensions that the Bourne shell lacks. Several of those extensions are enumerated here.
Integer mathematics
A major limitation of the Bourne shell is that it cannot perform integer calculations without spawning an external process. Bash can perform in-process integer calculations using the ((...)) command and the $[...] variable syntax, as follows:
VAR=55 # Assign integer 55 to variable VAR. ((VAR = VAR + 1)) # Add one to variable VAR. Note the absence of the '$' character. ((++VAR)) # Another way to add one to VAR. Performs C-style pre-increment. ((VAR++)) # Another way to add one to VAR. Performs C-style post-increment. echo $[VAR * 22] # Multiply VAR by 22 and substitute the result into the command. echo $((VAR * 22)) # Another way to do the above.
The ((...)) command can also be used in conditional statements, because its exit status is 0 or 1 depending on whether the condition is true or false:
if ((VAR == Y * 3 + X * 2)); then echo Yes fi ((Z > 23)) && echo Yes
The ((...)) command supports the following relational operators: '==', '!=', '>', '<', '>=', and '<='.
Bash cannot perform in-process floating point calculations. The only Unix command shells capable of this are Korn Shell (1993 version) and zsh (starting at version 4.0).
I/O redirection
Bash has several I/O redirection syntaxes that the traditional Bourne shell lacks. Bash can redirect standard output and standard error at the same time using this syntax:
command &> file
which is simpler to type than the equivalent Bourne shell syntax, "command > file 2>&1". Bash, since version 2.05b, can redirect standard input from a string using the following syntax (sometimes called "here strings"):
command <<< "string to be read as standard input"
If the string contains whitespace, it must be quoted.
Example: Redirect standard output to a file, write data, close file, reset stdout
# make Filedescriptor(FD) 6 a copy of stdout (FD 1) exec 6>&1 # open file "test.data" for writing exec 1>test.data # produce some content echo "data:data:data" # close file "test.data" exec 1>&- # make stdout a copy of FD 6 (reset stdout) exec 1>&6 # close FD6 exec 6>&-
Open and close files
# open file test.data for reading exec 6<test.data # read until end of file while read -u 6 dta do echo "$dta" done # close file test.data exec 6<&-
Catch output of external commands
# execute 'find' and store results in VAR # search for filenames which end with the letter "h" VAR=$(find . -name "*h")
EOF
The `cat <<EOF`
Bash syntax is very useful when one needs to work with multi-line strings in Bash (e.g., when passing a multi-line string to a variable, file, or a piped command).
- Pass a multiline string to a variable:
$ sql=$(cat <<EOF SELECT foo, bar FROM db WHERE foo='baz' EOF )
The $sql
variable now holds newlines as well, you can check it with `echo -e "$sql"`
:
SELECT foo, bar FROM db WHERE foo='baz'
- Pass a multiline string to a file:
$ cat <<EOF > print.sh #!/bin/bash echo \$PWD echo $PWD EOF
The print.sh file now contains:
#!/bin/bash echo $PWD echo /home/user
- Pass a multiline string to a command/pipe:
$ cat <<EOF | grep 'b' | tee b.txt | grep 'r' foo bar baz EOF
This creates b.txt
file with both bar
and baz
lines but prints only the bar
.
In-process regular expressions
Bash 3.0 supports in-process regular expression matching using the following syntax, reminiscent of Perl:
[[ string =~ regex ]]
The regular expression syntax is the same as that documented by the regex(3) man page. The exit status of the above command is 0 if the regex matches the string, 1 if it does not match. Parenthesized subexpressions in the regular expression can be accessed using the shell variable BASH_REMATCH, as follows:
if [[ abcfoobarbletch =~ 'foo(bar)bl(.*)' ]]; then echo The regex matches! echo $BASH_REMATCH -- outputs: foobarbletch echo ${BASH_REMATCH[1]} -- outputs: bar echo ${BASH_REMATCH[2]} -- outputs: etch fi
This syntax gives performance superior to spawning a separate process to run a grep command, because the regular expression matching takes place within the Bash process. If the regular expression or the string contain whitespace or shell metacharacters (such as '*' or '?'), they should be quoted.
Backslash escapes
Words of the form $'string' are treated specially. The word expands to string, with backslash-escaped characters replaced as specified by the C programming language. Backslash escape sequences, if present, are decoded as follows:
Backslash Escape |
Expands To ... |
---|---|
\a | An alert (bell) character |
\b | A backspace character |
\e | An escape character |
\f | A form feed character |
\n | A new line character |
\r | A carriage return character |
\t | A horizontal tab character |
\v | A vertical tab character |
\\ | A backslash character |
\' | A single quote character |
\nnn | The eight-bit character whose value is the octal value nnn (one to three digits) |
\xHH | The eight-bit character whose value is the hexadecimal value HH (one or two hex digits) |
\cx | A control-X character |
The expanded result is single-quoted, as if the dollar sign had not been present.
A double-quoted string preceded by a dollar sign ($"...") will cause the string to be translated according to the current locale. If the current locale is C or POSIX, the dollar sign is ignored. If the string is translated and replaced, the replacement is double-quoted.
The full list is as follows:
# \a an ASCII bell character (07) # \d the date in "Weekday Month Date" format (e.g., "Tue May 26") # \D{format} the format is passed to strftime(3) and the result is inserted into the prompt string; # \e an ASCII escape character (033) # \h the hostname up to the first `.' # \H the hostname # \j the number of jobs currently managed by the shell # \l the basename of the shell's terminal device name # \n newline # \r carriage return # \s the name of the shell, the basename of $0 (the portion following the final slash) # \t the current time in 24-hour HH:MM:SS format # \T the current time in 12-hour HH:MM:SS format # \@ the current time in 12-hour am/pm format # \A the current time in 24-hour HH:MM format # \u the username of the current user # \v the version of bash (e.g., 2.00) # \V the release of bash, version + patchelvel (e.g., 2.00.0) # \w the current working directory # \W the basename of the current working directory # \! the history number of this command # \# the command number of this command # \$ if the effective UID is 0, a #, otherwise a $ # \nnn the character corresponding to the octal number nnn # \\ a backslash # \[ begin a sequence of non-printing characters, which could be used to embed a terminal control sequence into the prompt # \] end a sequence of non-printing characters
Variables
When variables are used they are referred to with the $
symbol in front of them. There are several useful variables available in the shell program. Here are a few:
$$
= The PID number of the process executing the shell.$?
= Exit status variable.$0
= The name of the command you used to call a program.$1
= The first argument on the command line.$2
= The second argument on the command line.$n
= The nth argument on the command line.$*
= All the arguments on the command line.$#
= The number of command line arguments.
The "shift" command can be used to shift command line arguments to the left, i.e., $1
becomes the value of $2
, $3
shifts into $2
, etc. The command, "shift 2" will shift 2 places meaning the new value of $1
will be the old value of $3
and so forth.
- Print specific characters stored in a variable:
# The syntax is ${variable:start:length}. Omitting "length" value gives rest of string. $ val="hello" $ echo ${val:0:1} # Print out the first character of $val ("h" in this example)
Tests
There is a function provided by bash called test which returns a true or false value depending on the result of the tested expression (see: wikipedia:test (Unix) for more details). Its syntax is:
test expression
It can also be implied as follows:
[ expression ]
The tests below are test conditions provided by the shell:
-b file
= True if the file exists and is block special file.-c file
= True if the file exists and is character special file.-d file
= True if the file exists and is a directory.-e file
= True if the file exists.-f file
= True if the file exists and is a regular file-g file
= True if the file exists and the set-group-id bit is set.-k file
= True if the files' "sticky" bit is set.-L file
= True if the file exists and is a symbolic link.-p file
= True if the file exists and is a named pipe.-r file
= True if the file exists and is readable.-s file
= True if the file exists and its size is greater than zero.-s file
= True if the file exists and is a socket.-t fd
= True if the file descriptor is opened on a terminal.-u file
= True if the file exists and its set-user-id bit is set.-w file
= True if the file exists and is writable.-x file
= True if the file exists and is executable.-O file
= True if the file exists and is owned by the effective user id.-G file
= True if the file exists and is owned by the effective group id.file1 -nt file2
= True if file1 is newer, by modification date, than file2.file1 -ot file2
= True if file1 is older than file2.file1 -ef file2
= True if file1 and file2 have the same device and inode numbers.-z string
= True if the length of the string is 0.-n string
= True if the length of the string is non-zero.string1 = string2
= True if the strings are equal.string1 != string2
= True if the strings are not equal.!expr
= True if the expr evaluates to false.expr1 -a expr2
= True if both expr1 and expr2 are true.expr1 -o expr2
= True if either expr1 or expr2 is true.
The syntax is:
arg1 OP arg2
where OP is one of -eq, -ne, -lt, -le, -gt, or -ge
. Arg1 and arg2 may be positive or negative integers or the special expression "-l string
" which evaluates to the length of string.
- Examples:
if [ ! -e foo ]; then echo "NO FILE"; else cat foo; fi if [ -d "/home/bob" -a ! -d "/home/alice" ]; then echo "Bob exists, but not Alice"; fi
Colours in bash
Black 0;30 Dark Gray 1;30 Blue 0;34 Light Blue 1;34 Green 0;32 Light Green 1;32 Cyan 0;36 Light Cyan 1;36 Red 0;31 Light Red 1;31 Purple 0;35 Light Purple 1;35 Brown 0;33 Yellow 1;33 Light Gray 0;37 White 1;37
Here is an example borrowed from the Bash-Prompt-HOWTO:
PS1="\[\033[1;34m\][\$(date +%H%M)][\u@\h:\w]$\[\033[0m\] "
This turns the text blue, displays the time in brackets (very useful for not losing track of time while working), and displays the user name, host, and current directory enclosed in brackets. The "\[\033[0m\]
" following the $ returns the colour to the previous foreground colour.
- Another example:
PS1="\[\033[1;30m\][\[\033[1;34m\]\u\[\033[1;30m\]@\[\033[0;35m\]\h\[\033[1;30m\]] \[\033[0;37m\]\W \[\033[1;30m\]\$\[\033[0m\] "
yields:
[user@host] directory $
Break down:
\[\033[1;30m\] - Sets the color for the characters that follow it. Here 1;30 will set them to Dark Gray. \u \h \W \$ - From the table above \[\033[0m\] - Sets the colours back to how they were originally.
Bash startup scripts
When Bash starts, it executes the commands in a variety of different scripts.
When Bash is invoked as an interactive login shell, or as a non-interactive shell with the --login option, it first reads and executes commands from the file /etc/profile, if that file exists. After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readable. The --noprofile option may be used when the shell is started to inhibit this behavior.
When a login shell exits, Bash reads and executes commands from the file ~/.bash_logout, if it exists.
When an interactive shell that is not a login shell is started, Bash reads and executes commands from ~/.bashrc, if that file exists. This may be inhibited by using the --norc option. The --rcfile file option will force Bash to read and execute commands from file instead of ~/.bashrc.
When Bash is started non-interactively, to run a shell script, for example, it looks for the variable BASH_ENV in the environment, expands its value if it appears there, and uses the expanded value as the name of a file to read and execute. Bash behaves as if the following command were executed:
if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi
but the value of the PATH variable is not used to search for the file name.
If Bash is invoked with the name sh, it tries to mimic the startup behavior of historical versions of sh as closely as possible, while conforming to the POSIX standard as well. When invoked as an interactive login shell, or a non-interactive shell with the --login option, it first attempts to read and execute commands from /etc/profile and ~/.profile, in that order. The --noprofile option may be used to inhibit this behavior. When invoked as an interactive shell with the name sh, Bash looks for the variable ENV, expands its value if it is defined, and uses the expanded value as the name of a file to read and execute. Since a shell invoked as sh does not attempt to read and execute commands from any other startup files, the --rcfile option has no effect. A non-interactive shell invoked with the name sh does not attempt to read any other startup files. When invoked as sh, Bash enters posix mode after the startup files are read.
When Bash is started in posix mode, as with the --posix command line option, it follows the POSIX standard for startup files. In this mode, interactive shells expand the ENV variable and commands are read and executed from the file whose name is the expanded value. No other startup files are read.
Bash attempts to determine when it is being run by the remote shell daemon, usually rshd. If Bash determines it is being run by rshd, it reads and executes commands from ~/.bashrc, if that file exists and is readable. It will not do this if invoked as sh. The --norc option may be used to inhibit this behavior, and the --rcfile option may be used to force another file to be read, but rshd does not generally invoke the shell with those options or allow them to be specified.
Environment variables
$CDPATH
- does for the cd built-in what PATH does for executables. By setting this wisely, you can cut down on the number of key-strokes you enter per day.
- Example
$ export CDPATH=.:~:~/docs:~/src:~/src/ops/docs:/mnt:/usr/src/redhat:/usr/src/redhat/RPMS:/usr/src:/usr/lib:/usr/local:/software:/software/redhat
- Using this, cd i386 would likely take you to /usr/src/redhat/RPMS/i386 on a Red Hat Linux system. Make sure that you do include . in the list or you'll find that you can't change to directories relative to your current one without prefixing them with ./
$HISTIGNORE
- Set this to to avoid having consecutive duplicate commands and other not so useful information appended to the history list. This will cut down on hitting the up arrow endlessly to get to the command before the one you just entered twenty times. It will also avoid filling a large percentage of your history list with useless commands.
- Example
$ export HISTIGNORE="&:ls:ls *:mutt:[bf]g:exit"
- Using this, consecutive duplicate commands, invocations of ls, executions of the mutt mail client without any additional parameters, plus calls to the bg, fg and exit built-ins will not be appended to the history list.
$MAILPATH
- bash will warn you of new mail in any folder appended to MAILPATH. This is very handy if you use a tool like procmail to presort your e-mail into folders.
- Try adding the following to your ~/.bash_profile to be notified when any new mail is deposited in any mailbox under ~/Mail.
MAILPATH=/var/spool/mail/$USER for i in ~/Mail/[^.]* do MAILPATH=$MAILPATH:$i'?You have new mail in your ${_##*/} folder' done export MAILPATH unset i
$TMOUT
- If you set this to a value greater than zero, bash will terminate after this number of seconds have elapsed if no input arrives.
shopt
Check your shell options by issuing the following command:
shopt # see: help shopt
A typical output should look something like the following:
cdable_vars off cdspell off checkhash off checkwinsize off cmdhist on dotglob off execfail off expand_aliases on extglob off histreedit off histappend off histverify off hostcomplete on huponexit off interactive_comments on lithist off login_shell on mailwarn off no_empty_cmd_completion off nocaseglob off nullglob off progcomp on promptvars on restricted_shell off shift_verbose off sourcepath on xpg_echo off
where each of the above mean (see man bash
or help set
for more info.):
- cdable_vars
- an argument to the cd builtin command that is not a directory is assumed to be the name of a variable dir to change to.
- cdspell
- minor errors in the spelling of a directory component in a cd command will be corrected.
- checkhash
- bash checks that a command found in the hash table exists before execute it. If no longer exists, a path search is performed.
- checkwinsize
- bash checks the window size after each command and, if necessary, updates the values of LINES and COLUMNS.
- cmdhist
- bash attempts to save all lines of a multiple-line command in the same history entry. Allows re-editing of multi-line commands.
- dotglob
- bash includes filenames beginning with a `.' in the results of pathname expansion.
- execfail
- a non-int shell will not exit if it cannot execute the file specified as an argument to the exec builtin command, like int sh.
- expand_aliases
- aliases are expanded as described above under ALIASES. This option is enabled by default for interactive shells.
- extglob
- the extended pattern matching features described above under Pathname Expansion are enabled.
- histappend
- the history list is appended to the file named by the value of the HISTFILE variable when shell exits, no overwriting the file.
- hostcomplete
- and readline is being used, bash will attempt to perform hostname completion when a word containing a @ is being completed
- huponexit
- bash will send SIGHUP to all jobs when an interactive login shell exits.
- interactive_comments
- allow a word beginning with # to cause that word and all remaining characters on that line to be ignored in an interactive shell
- lithist
- if cmdhist option is enabled, multi-line commands are saved to the history with embedded newlines rather than using semicolon
- login_shell
- shell sets this option if it is started as a login shell (see INVOCATION above). The value may not be changed.
- mailwarn
- file that bash is checking for mail has been accessed since the last checked, ``The mail in mailfile has been read is displayed.
- no_empty_cmd_completion
- bash will not attempt to search the PATH for possible completions when completion is attempted on an empty line.
- nocaseglob
- bash matches filenames in a case-insensitive fashion when performing pathname expansion (see Pathname Expansion above).
- nullglob
- bash allows patterns which match no files (see Pathname Expansion above) to expand to a null string, rather than themselves.
- progcomp
- the programmable completion facilities (see Programmable Completion above) are enabled. This option is enabled by default.
- promptvars
- prompt strings undergo variable and parameter expansion after being expanded as described in PROMPTING above.
- shift_verbose
- the shift builtin prints an error message when the shift count exceeds the number of positional parameters.
- sourcepath
- the source (.) builtin uses the value of PATH to find the directory containing the file supplied as an argument.
- xpg_echo
- the echo builtin expands backslash-escape sequences by default.
Bash (Shellshock) vulnerability
- Run the following command from a Bash shell:
$ env 'x=() { :;}; echo vulnerable' 'BASH_FUNC_x()=() { :;}; echo vulnerable' bash -c "echo test"
If you see the following output, you are VULNERABLE:
vulnerable bash: BASH_FUNC_x(): line 0: syntax error near unexpected token `)' bash: BASH_FUNC_x(): line 0: `BASH_FUNC_x() () { :;}; echo vulnerable' bash: error importing function definition for `BASH_FUNC_x' test
If you see the following output, you are NOT VULNERABLE:
bash: warning: x: ignoring function definition attempt bash: error importing function definition for `BASH_FUNC_x' test
If you are vulnerable, make sure you update Bash to the latest version your Linux distribution has to offer. If you still see the same vulnerability after updating from a repository, you should probably down the the latest source code of Bash and compile it on your own. Do not take this bug lightly!
Christoph's Additions
PS1='\[\033[0;31m\][\u@christophchamp]\[\033[00m\]:\[\033[0;33m\]`pwd`\[\033[00m\]> ' [christoph@christophchamp]:/home/christoph>
References
External links
- Bash home page
- Bash FAQ
- Bash 3.0 Announcement
- The GNU Bash Reference Manual, (HTML version) by Chet Ramey and Brian Fox, ISBN 0954161777
Bash guides from the Linux Documentation Project
- Bash Guide for Beginners
- BASH Programming - Introduction HOW-TO
- Advanced Bash-Scripting Guide — An in-depth exploration of the art of shell scripting
- Text manipulation tools — from GNU/Linux Command-Line Tools Summary
Other guides and tutorials
- Bash Pitfalls / BashFAQ
- Linux Shell Scripting Tutorial - A Beginner's handbook
- About Shells
- Beginners Bash Tutorial
- Advancing in the Bash Shell tutorial
- Linux Know-How including the Bash Guide for Beginners
- BashScripts.org
- Common features
- Get the most out of bash