< GNU/Linux
This is about how to work efficiently on the command line in GNU/Linux (and probably other POSIX OS’s, although I haven’t tested that theory).
Contents
|
Reference links / Cheat sheets / Tutorials
Osamu Aoki. Debian Reference Chapter 8 – Debian tips (http://www.debian.org/doc/manuals/reference/ch-tips.en.html). Retrieved on 2007-05-11 11:18. {{4 stars}
The Linux Cookbook: Tips and Techniques for Everyday Use – Viewing Text (http://www.dsl.org/cookbook/cookbook_13.html#SEC183). Retrieved on 2007-05-11 11:18.
The Ultimate Linux Reference Guide for Newbies (http://blog.lxpages.com/ultimate_linux.html). Retrieved on 2007-05-11 11:18.
Bash Hackers Wiki (http://bash-hackers.org/wiki/doku.php?id=). Retrieved on 2007-05-11 11:18.
This wiki is intended to hold documentations of any kind about the GNU Bash. It’s one of more applications of bash-hackers.org site. See also the forum.
!: Please direct any questions about Bash, and also discussions about this wiki to the forum ❗ The main motivation was to provide human-readable documentation and information to not force users to read every bit of the Bash manpage – which is PITA sometimes. The docs here are not meant as newbie tutorial, more as educational summary. Stranger! Feel free to register and edit the contents. The registration is only there to prevent SPAM.
http://wooledge.org/mywiki. Retrieved on 2007-05-11 11:18.
The ‘official’ channel FAQ for freenode’s #bash channel is BashFAQ. For common mistakes made by Bash programmers, see BashPitfalls. For general Unix issues, see DotFiles, Permissions, ProcessManagement, or UsingFind. Miscellaneous pages: FtpMustDie, XyProblem.
http://www.johnstowers.co.nz/wiki/index.php/Useful_Commands
http://www.pixelbeat.org/cmdline.html
Important key combinations to know
Ctrl-c Stop current foreground process Ctrl-z Suspend current foreground process Ctrl-d Send end-of-file character (needed for mail command, etc.)
Wildcards and globbing
How to exclude a folder from your glob
[^a] — The crude, not-very-powerful way
Let’s say you had these files in your directory:
> ls css favicon.ico img index.html picture_library plesk-stat test
(those are the default files that are created by Plesk when you use it to set up a new domain with physical hosting, in case you were wondering…)
So if you wanted to refer to everything except plesk-stat and picture_library, you could do…
> ls [^p]* -d css favicon.ico img index.html test
…which is the opposite of this:
> ls [p]* -d picture_library plesk-stat
If you wanted to refer to everything except plesk-stat (but including picture_library), it’s a bit harder…
You can’t just do this:
> ls [^pl]* -d css favicon.ico img index.html test
or
> ls [^p][^l]* -d css favicon.ico img index.html test
because then picture_library would be excluded.
You could do this:
> ls ?[^l]* -d
But only by luck. That wouldn’t work if you had something like flags (with a l in position 2) that you wanted to include… So we got lucky.
What we need is something more powerful, something that will work in any situation (for any arbitrary combination of includes/excludes)…
!(pattern-list)
Prerequisite:
> shopt -s extglob > shopt extglob extglob on
> ls !(plesk-stat) -d css favicon.ico img index.html picture_library test
Very cool. Now we can easily do a “delete everything except” command!:
> rm -r !(plesk-stat)
GLOBIGNORE
> GLOBIGNORE=plesk-stat > ls * -d css favicon.ico img index.html picture_library test
Also very cool! And this one (apparently) doesn’t require you to explicitly enable any shopt’s…
Readlines / ~/.inputrc / stty
man readline:
readline will read a line from the terminal and return it, using prompt as a prompt. If prompt is NULL or the empty string, no prompt issued. The line returned is allocated with malloc(3); the caller must free it when finished. The line returned has the final newline removed, so only the text of the line remains. readline offers editing capabilities while the user is entering the line. By default, the line editing commands are similar to those of emacs. A vi-style line editing interface is also available. … An emacs-style notation is used to denote keystrokes. Control keys are denoted by C-key, e.g., C-n means Control-N. Similarly, meta keys are denoted by M-key, so M-x means Meta-X. (On keyboards without a meta key, M-x means ESC x, i.e., press the Escape key then the x key. This makes ESC the meta prefix. The combination M-C-x means ESC-Control-x, or press the Escape key then hold the Control key while pressing the x key.)
Commands for Moving beginning-of-line (C-a) Move to the start of the current line. end-of-line (C-e) Move to the end of the line. forward-char (C-f) Move forward a character. backward-char (C-b) Move back a character. forward-word (M-f) Move forward to the end of the next word. Words are composed of alphanumeric characters (letters and digits). backward-word (M-b) Move back to the start of the current or previous word. Words are composed of alphanumeric characters (letters and digits). clear-screen (C-l) Clear the screen leaving the current line at the top of the screen. With an argument, refresh the current line without clearing the screen. redraw-current-line Refresh the current line. Commands for Manipulating the History accept-line (Newline, Return) Accept the line regardless of where the cursor is. If this line is non-empty, it may be added to the history list for future recall with add_history(). If the line is a modified history line, the history line is restored to its original state. previous-history (C-p) Fetch the previous command from the history list, moving back in the list. next-history (C-n) Fetch the next command from the history list, moving forward in the list. beginning-of-history (M-) Move to the end of the input history, i.e., the line currently being entered. reverse-search-history (C-r) Search backward starting at the current line and moving ‘up’ through the history as necessary. This is an incremental search. ... Commands for Changing Text delete-char (C-d) Delete the character at point. If point is at the beginning of the line, there are no characters in the line, and the last character typed was not bound to delete-char, then return EOF. backward-delete-char (Rubout) Delete the character behind the cursor. When given a numeric argument, save the deleted text on the kill ring. forward-backward-delete-char Delete the character under the cursor, unless the cursor is at the end of the line, in which case the character behind the cursor is deleted. quoted-insert (C-q, C-v) Add the next character that you type to the line verbatim. This is how to insert characters like C-q, for example. tab-insert (M-TAB) Insert a tab character. self-insert (a, b, A, 1, !, ...) Insert the character typed. transpose-chars (C-t) Drag the character before point forward over the character at point, moving point forward as well. If point is at the end of the line, then this transposes the two characters before point. Negative arguments have no effect. transpose-words (M-t) Drag the word before point past the word after point, moving point over that word as well. If point is at the end of the line, this transposes the last two words on the line. upcase-word (M-u) Uppercase the current (or following) word. With a negative argument, uppercase the previous word, but do not move point. downcase-word (M-l) Lowercase the current (or following) word. With a negative argument, lowercase the previous word, but do not move point. capitalize-word (M-c) Capitalize the current (or following) word. With a negative argument, capitalize the previous word, but do not move point. ... Killing and Yanking kill-line (C-k) Kill the text from point to the end of the line. backward-kill-line (C-x Rubout) Kill backward to the beginning of the line. unix-line-discard (C-u) Kill backward from point to the beginning of the line. The killed text is saved on the kill-ring. kill-whole-line Kill all characters on the current line, no matter where point is. kill-word (M-d) Kill from point the end of the current word, or if between words, to the end of the next word. Word boundaries are the same as those used by forward-word. backward-kill-word (M-Rubout) Kill the word behind point. Word boundaries are the same as those used by backward-word. unix-word-rubout (C-w) Kill the word behind point, using white space as a word boundary. The killed text is saved on the kill-ring. unix-filename-rubout Kill the word behind point, using white space and the slash character as the word boundaries. The killed text is saved on the kill- ring. delete-horizontal-space (M-\) Delete all spaces and tabs around point. kill-region Kill the text between the point and mark (saved cursor position). This text is referred to as the region. copy-region-as-kill Copy the text in the region to the kill buffer. copy-backward-word Copy the word before point to the kill buffer. The word boundaries are the same as backward-word. copy-forward-word Copy the word following point to the kill buffer. The word boundaries are the same as forward-word. yank (C-y) Yank the top of the kill ring into the buffer at point. yank-pop (M-y) Rotate the kill ring, and yank the new top. Only works following yank or yank-pop. ... Completing complete (TAB) Attempt to perform completion on the text before point. The actual completion performed is application-specific. Bash, for instance, attempts completion treating the text as a variable (if the text begins with $), username (if the text begins with ~), hostname (if the text begins with @), or command (including aliases and functions) in turn. If none of these produces a match, filename completion is attempted. Gdb, on the other hand, allows completion of program functions and variables, and only attempts filename completion under certain circumstances. possible-completions (M-?) List the possible completions of the text before point. insert-completions (M-*) Insert all completions of the text before point that would have been generated by possible-completions. menu-complete Similar to complete, but replaces the word to be completed with a single match from the list of possible completions. Repeated execu‐ tion of menu-complete steps through the list of possible completions, inserting each match in turn. At the end of the list of comple‐ tions, the bell is rung (subject to the setting of bell-style) and the original text is restored. An argument of n moves n positions forward in the list of matches; a negative argument may be used to move backward through the list. This command is intended to be bound to TAB, but is unbound by default. delete-char-or-list Deletes the character under the cursor if not at the beginning or end of the line (like delete-char). If at the end of the line, behaves identically to possible-completions. ... Miscellaneous ... prefix-meta (ESC) Metafy the next character typed. ESC f is equivalent to Meta-f. undo (C-_, C-x C-u) Incremental undo, separately remembered for each line. revert-line (M-r) Undo all changes made to this line. This is like executing the undo command enough times to return the line to its initial state. tilde-expand (M-&) Perform tilde expansion on the current word. ... character-search (C-]) A character is read and point is moved to the next occurrence of that character. A negative count searches for previous occurrences. character-search-backward (M-C-]) A character is read and point is moved to the previous occurrence of that character. A negative count searches for subsequent occur‐ rences. insert-comment (M-#) Without a numeric argument, the value of the readline comment-begin variable is inserted at the beginning of the current line. If a numeric argument is supplied, this command acts as a toggle: if the characters at the beginning of the line do not match the value of comment-begin, the value is inserted, otherwise the characters in comment-begin are deleted from the beginning of the line. In either case, the line is accepted as if a newline had been typed. The default value of comment-begin makes the current line a shell comment. If a numeric argument causes the comment character to be removed, the line will be executed by the shell.
Here I have been using readline for years whenever I used the bash shell and I didn’t even realize it. Nor did I know about many hidden features of readline. It’s like a fully-featured text-editor built-in to a single line — crazy — Tyler (2007-08-06 20:01)
Troubleshooting
sudo: __: command not found, but works if you give it the full path
[1] suggests looking at sudo /usr/bin/env | grep PATH and seeing if that is different.
For me right now it is different:
> sudo /usr/bin/env | grep PATH PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin > /usr/bin/env | grep PATH PATH=/var/lib/gems/1.8/gems/subwrap-0.3.10/bin:/home/tyler/bin:/home/tyler/public/shell/bin:/var/lib/gems/1.8/bin:/home/tyler/bin:/home/tyler/bin:/home/tyler/public/shell/bin:/home/tyler/public/shell/devscripts_bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games/var/lib/gems/1.8/bin/:/var/lib/gems/1.8/bin/:/var/lib/gems/1.8/bin/:/var/lib/gems/1.8/bin/:/var/lib/gems/1.8/bin/:/var/lib/gems/1.8/bin/:/var/lib/gems/1.8/bin/
(Note that sudo echo $PATH and echo $PATH are identical — so apparently sudo echo $PATH is lying and only sudo /usr/bin/env | grep PATH can be trusted…?)
Why is it different?
On another machine I have access to, sudo /usr/bin/env | grep PATH and /usr/bin/env | grep PATH are identical…. as it seems they should be.
http://forums.macosxhints.com/showthread.php?t=20781 suggestion is to “create a new user”. I don’t want to create a new user!
I read somewhere that if you have an error or anything wrong in your .profile/.bash_profile/.bashrc that sudo will get “confused” and fall back to its default (truncated) value for $PATH. I tried commenting out a bunch of stuff in my .* files and it didn’t seem to fix anything.
Troubleshooting: Fixing broken terminals
Not echoing back / not letting you type
If the terminal isn’t echoing back what you type to it, type reset.
If it locks up (in vim usually), press Ctrl+q.
There’s junk characters on the screen!
clear or Ctrl+l will redraw the screen for you.
Not echoing anything you type (but still responding to what you type)
Cure:
stty echo
How to cause this problem: The terminal can get hosed if you cat some binary files that have certain characters that mess up the terminal.
Troubleshooting: Fixing keys that don’t seem to be working
Home key doesn’t work?
You can probably use Ctrl+a to do the same thing.
A more permanent solution:
export TERM=linux
On other systems, the fix is to change it from linux to something else:
export TERM=xterm
Vim outputs H when you press Home and F when you press End?
:!uname -a [No write since last change] Linux ... 2.6.16-1.2069_FC4smp #1 SMP Tue Mar 28 12:47:32 EST 2006 i686 i686 i386 GNU/Linux :!echo $TERM linux
Solution?
export TERM=xterm
and restart vim.
Delete key doesn’t work?
This has happened to me on OpenBSD (at work) and FreeBSD (on nearlyfreespeech.net). —Tyler)
Edit your ~/.inputrc. Add this line:
"\e[3~": delete-char
Problems: You can’t move a file into a directory that doesn’t exist
For crying out loud. Why can’t it just make the directory if it doesn’t exist?? It doesn’t complain that the target file doesn’t exist. Why can’t it be equally complacent about the target directory?.
As a result of this annoyance, one often has to make extra calls to mkdir (or mkdir -p, since mkdir suffers from the same problem otherwise).
Redirection / Input/output streams / File descriptors
Introduction
BASH Programming – Introduction HOW-TO: All about redirection (http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-3.html). Retrieved on 2007-05-11 11:18.
There are 3 file descriptors: stdin, stdout and stderr (std=standard). …
Why there are 2 standard output streams (stdout, stderr)
This lets you do things like:
some_important_command | grep something
and still see error output from the command (even if the errors don’t match the grep search term).
What if I want everything to be output to stdout (a single stream that I can pipe somewhere)?
If you’d rather treat stderr as stdout, just redirect stderr to stdout:
some_important_command 2>&1 | grep something
What if I want all output streams (file descriptors) redirected to a file?
Then &> is your best bet.
> svn mkdir lib svn: Try 'svn add' or 'svn add --non-recursive' instead? svn: Can't create directory 'lib': File exists > svn mkdir lib &>/dev/null
What if I don’t want to see any output?
Then redirect to /dev/null
> svn mkdir lib svn: Try 'svn add' or 'svn add --non-recursive' instead? svn: Can't create directory 'lib': File exists > svn mkdir lib 1>/dev/null 2>&1 # Or: > svn mkdir lib &>/dev/null
Note that the order you put >/dev/null and 2>&1 in does matter — you have to redirect FD1 (stdout) to >/dev/null before you redirect FD2 (stderr) to FD1 (stdout)…
> svn mkdir lib 2>&1 >/dev/null svn: Try 'svn add' or 'svn add --non-recursive' instead? svn: Can't create directory 'lib': File exists
Question: Redirection to a file (>) gives “Permission denied”, even if you run it as sudo
> svn diff > out -bash: out: Permission denied
Okay, that makes sense so far. Since . has the following permissions the current user is not root:
drwxr-xr-x 19 root root 4096 Feb 12 14:49 ./
But why, if I do, sudo would it still not work!?
> sudo svn diff > out -bash: out: Permission denied
Workaround:
> sudo touch out > sudo svn diff > out -bash: out: Permission denied > sudo chmod a+w out > sudo svn diff > out # Works
How do I output to a log file and to standard output
Obviously, if you redirect standard output to a file, like this:
some_long_running_or_verbose_script > something.log
then you won’t see any standard output!
What you want is a sort of splitter, a T fitting that you can put on the pipe so it goes to two destinations. Fortunately, GNU provides just that: tee!
some_long_running_or_verbose_script | tee something.log
Before I learned about tee, I used to do it like this:
some_long_running_or_verbose_script > something.log & ; tail -F something.log
but that’s not as elegant or concise. And I wonder if it’s possible that it will miss something at the beginning of the log, before tail is started.
tee --append
How do I redirect the output of the bash keyword ‘time‘?
Harri Järvi. Redirecting output of the bash keyword time (http://www.cs.tut.fi/~jarvi/tips/bash.html). Retrieved on 2007-05-11 11:18.
In a shell script I wanted to take the execution time of a command and redirect it to a file. First I tried the following:
time command > time.txtIt didn’t work. Then I found out that the time command prints its output to stderr. I changed the command to:
time command 2> time.txtIt didn’t work either. The time’s output was still printed on the console.
It turned out that time is a reserved word in bash. It’s not like most of the builtin commands in bash but it’s a true part of the command line syntax like ‘if’ or ‘while’.
Bash 3.1.0 manual says the following about pipelines and time reserved word:
… If the time reserved word precedes a pipeline, the elapsed as well as user and system time consumed by its execution are reported when the pipeline terminates. […] Each command in a pipeline is executed as a separate process (i.e., in a subshell).
The builtin help command of bash says the following:
$ help time time: time [-p] PIPELINE Execute PIPELINE and print a summary of the real time, user CPU time, and system CPU time spent executing PIPELINE when it terminates. The return status is the return status of PIPELINE. The `-p' option prints the timing summary in a slightly different format. This uses the value of the TIMEFORMAT variable as the output format. times: times Print the accumulated user and system times for processes run from the shell.…
Also from the syntax description it becomes clear that the output of the timing is not appended to the stderr of the command, but it’s actually output by the shell itself on an upper level. The Time keyword activates a flag and after the whole pipeline command is executed, the timing information is printed to stderr. The time keyword is above the whole command and all pipes and redirections in it. This is why trying naively to redirect the output of time didn’t work. It didn’t work specifically because how bash syntax is defined: Adding redirection at the end of the command will be interpreted as part of the command to time.
Redirecting the output of the bash time can be achieved by executing the whole command (including the time part) in a subshell as follows:
(time command) 2> time.txtLaunching a subshell is not necessary. Redirecting output of a code block works as well.
{ time ls; } 2> time.txtThis will probably be more efficient that executing the external /usr/bin/time command . Also the bash time command may have some features you need. Or you want to rely on bash time because you don’t know if the system installed time utility will have the features you need or not.
In redirection and pipe behaviour this is actually the equivalent of executing the external time command /usr/bin/time .
Suppressing the output from the command itself can be made by redirecting its standard output and standard error to /dev/null
{ time command > /dev/null 2>&1 ; } 2> time.txt
Useful commands/utilities
GNU Core Utilities
http://www.gnu.org/software/fileutils/doc/faq/core-utils-faq.html. Retrieved on 2007-05-11 11:18.
Together the three packages combined implement a core set of GNU utilities: […]
- fileutils
- sh-utils [shellutils]
- textutils.
Has lots of answers to common questions; may be useful for troubleshooting your particular problem!
textutils
http://www.ibm.com/developerworks/library/l-tiptex1.html. Retrieved on 2007-05-11 11:18.
… In all, throughout this series, we will get to know cat and tac; head and tail; sort and uniq — and will discuss the dumping, folding, splitting, indexing, and other capabilities of some of the most common UNIX and Linux text utilities. These are some of the most useful bits of code that you have on your computer; alas, they are often also the least used — perhaps because man pages can be so hard to follow if you don’t already know what you’re doing. That is why, rather than be just another copy of man page options, this series will be a guided tour with scenarios that put common commands through their paces so you get a hands-on feel for how and when to use them. Once you have the basics down, you will find the (often arcane) man pages much easier to follow. …
http://www-128.ibm.com/developerworks/linux/library/l-textutils.html. Retrieved on 2007-05-11 11:18.
Covers: Regular expressions grep fgrep egrep The grep utilities: A real-world example cut cut: Two real-world examples paste join join: A real-world example awk head tail
head / tail / [middle]
Reading text streams in chunks with head and tail (http://www.ibm.com/developerworks/linux/library/l-tiptex3.html). Retrieved on 2007-05-11 11:18.
tail
> cat filename | tail 3 # last 3 lines > cat filename | tail -n+3 # all lines starting at line 3 (inclusive)
tail --help:
If the first character of N (the number of bytes or lines) is a `+', print beginning with the Nth item from the start of each file, otherwise, print the last N items in the file.
[middle?]
I could have sworn I’ve seen a command like this used somewhere — a perfect compliment to head and tail –, but now I can’t seem to find it…
What was it called? middle? body?
cut — cut out selected fields of each line of a file
- -c select a range of characters
- -f select fields, used with comma-, tab-, etc.- delimited fields (set delimiter with -d delimiter)
Example usage:
> ls --full-time | cut -c 34-52 2006-07-03 15:26:11 2006-08-24 14:24:17 2006-08-14 22:58:38 2006-08-14 22:58:38 2006-08-24 14:24:17
How do I reverse the lines in a file?
My first guesses were sort and rev.
Nope, tac is what you want.
echo
echo is pretty much the most basic command available.
/bin/echo
How to avoid printing a newline character
> echo "1234" | wc -c 5 > echo -n "1234" | wc -c 4
man
expr (Math/arithmetic on the command line)
[tyler: ~/svn/code/plugins/database_log4r]> expr 2+3 2+3 [tyler: ~/svn/code/plugins/database_log4r]> expr 2 + 3 5 [tyler: ~/svn/code/plugins/database_log4r]> expr 8 * 3 expr: syntax error [tyler: ~/svn/code/plugins/database_log4r]> expr 8 \* 3 24
patch (apply a diff)
See Patches
watch (repeat a command continuously)
http://old.pupeno.com/blog/poor-man-s-continuous-unit-testing-with-ruby-on-rails/. Retrieved on 2007-02-16 17:14.
watch ls -l big-file-being-copied.txt
Gives output that updates every 2 (adjustable) seconds. Sort of like top except it can be used to watch the output of any command-line program.
Examples:
watch vmstat watch who
See also: my repeat.rb / repeat.sh
shopt (“shell options”?)
> shopt cdable_vars off cdspell off checkhash off checkwinsize on cmdhist on dotglob off execfail off expand_aliases on extdebug off ...
Set an option like this: shopt -s expand_aliases
Unset an option like this: shopt -u expand_aliases
I’m not even sure what any of these options are for… but they’re there.
Leslie P. Polzer (2007-02-15). My sysadmin toolbox (http://www.linux.com/feature/60179). Retrieved on 2007-05-11 11:18. 
My working day includes a variety of tasks, and most of them take place on the command line, because that approach enables me to do things in the most efficient way. But you can also waste a lot of time on the command line if you don’t know what utilities will give you what you need quickly. Here’s an introduction to the most important tools I use every day.
zsh
The GNU Bourne Again Shell, bash, is the command line interpreter traditionally associated with Linux systems, and most GNU/Linux systems ship it as default. While it has considerably improved in terms of comfort, it stands behind the powerful Z Shell, which you can use as a superset of the Bourne Again Shell.
…
fmt/par
Outlook Express users send you messages with extra-long line lengths? Need to have a text file formatted with an 80-character boundary? Easy — use fmt, the traditional Unix paragraph formatter, or a modern version called par :
fmt -80 report.txt par w80 report.txtI especially like to reformat mail quotes from within Vim (c) by selecting the relevant paragraphs in visual mode (Ctrl-V) and then running the command !par w72 in ex mode. par will preserve any quotation marks and paragraph divisions.
cat
cat simply concatenates files or standard I/O. In combination with a here document, cat can create entire files from scratch:
cat > /etc/resolv.conf full.txt
If everything in Unix really was a file, as it is supposed to be in Plan 9, you could even use it on sockets, thereby using cat to send and receive data through a network connection; but since this isn’t the case, you have to resort to its specialized companion netcat.
However, please take care to avoid the “useless uses of cat,” lest you win an award for your folly.
muttng
Why prefer mutt (or its patched cousin muttng) to the many GUI messaging clients out there? For a lot of reasons: painless integration with your favorite text editor (vi and Emacs users will like that), sensible key bindings, high configurability, pretty standard-compliant support for the major mail protocols, and easy usage over network connections among them.
In my experience it’s not worth adapting to mutt if you’re only getting a few messages every other day, but once your mail volume rises above a certain number, you will be way more effective with mutt.
dillo
The Dillo project has created an X Window Web browser with a very small footprint. Dillo is an excellent tool with which to quickly view HTML documents from the command line, as it starts up in an instant and has a very clean interface. I like to use it for HTML manuals and to browse TexInfo documentation in HTML format.
OpenSSH
Designed as a secure replacement for telnet, rologin, and ftp, OpenSSH offers substantially more comfort and features than the tools it replaces. You can create encrypted tunnels with it and forward X windows. Some people even like to use it for mass command execution in cluster environments or large installations [sounds like Capistrano!].
xbindkeys/actkbd
These two programs are daemons that will execute a specific command when they receive the corresponding mouse or keyboard shortcut.
xbindkeys needs X running and reads input using Xlib. You can associate certain inputs with actions by defining them in rules in the file .xbindkeysrc in your home directory. For example, you could bind your fourth mouse button to start your favorite terminal emulator, and two function keys could be set up to control the volume.
actkbd works both under X and on the console. You could, for example, use it to eject your CD-ROM tray with a keypress both on the console and in X. It uses the evdev interface to receive events, and might therefore also work with exotic input devices (think of buttons on webcams).
For maximum efficiency, use both together.
vim
Here comes the religious part. What I like about Vim is its speed, its ubiquity (you can find a plain vi editor on every Unix box) and its modal interface, which enables me to modify my files with high speed.
grep, sed, cut, awk, perl
Here are five omnipresent and flexible tools to work on text. Use grep to filter out (un-)important stuff or search for something, sed to do quick patching or on-the-fly text rewriting, and cut to select columns.
If these don’t suffice, throw in awk as a Turing-complete replacement for all three. If you’re still not content or have a taste for it, use Perl, which scales well for larger projects, frees you from quoting hassles, and is faster.
I use the GNU versions of sed, cut, and awk, since they offer the most features, but you can find these utilities in a variety of flavors on every Unix system.
mlterm
Mlterm is my favorite X terminal emulator. Both fast and powerful, it also sports excellent support for non-English scripts. It’s less cluttered than Konsole and has a smaller footprint, but it still offers a palette of features such as background fading (change brightness depending on focus), background images, and a GUI configurator. I use it with a black background, white text color, and a font that doesn’t hurt my eyes (try ISO10646_UCS4_1_BOLD=-misc-fixed-bold-r-normal–15-140-75-75-c-90-iso10646-1:40;15 or something similar in your ~/.mlterm/font file).
pgrep/pkill
These two tools let you quickly list (pgrep) and signal (pkill) processes, as you can see here:
pgrep fox # roughly equivalent to ‘ps -ax | grep fox’ pkill fox # kill all processes matching ‘*fox*’
Simulating the second example would likely take a combination of ps and awk — not nice for stuff that you use often! The two utilities come with the procps package.
Conclusion
Since I need to mix the CLI with GUI applications, like browsers, I need a link to tie those two worlds together — best with a minimum of mouse usage. The usual WIMP-style interfaces (KDE, GNOME, and most window managers and alternative desktop environments) cannot do this right, so I’m using a tiling window manager (currently Ion3. Throw in a browser with configurable shortcuts, and you’ll hardly ever have to reach for the mouse again!
Controlling processes
ps
ps edit
ps command=
man ps
PROCESS STATE CODES
Here are the different values that the s, stat and state output specifiers (header "STAT" or "S") will display to describe the state of a
process.
D Uninterruptible sleep (usually IO)
R Running or runnable (on run queue)
S Interruptible sleep (waiting for an event to complete)
T Stopped, either by a job control signal or because it is being traced.
W paging (not valid since the 2.6.xx kernel)
X dead (should never be seen)
Z Defunct ("zombie") process, terminated but not reaped by its parent.
For BSD formats and when the stat keyword is used, additional characters may be displayed:
< high-priority (not nice to other users)
N low-priority (nice to other users)
L has pages locked into memory (for real-time and custom IO)
s is a session leader
l is multi-threaded (using CLONE_THREAD, like NPTL pthreads do)
+ is in the foreground process group
http://nixcraft.com/linux-software/431-what-i-o-wait-under-ps-command.html. Retrieved on 2007-05-11 11:18.
Generally, a process is put into D state when waiting for some sort of I/O to complete. The process has made a call into the kernel and is waiting for the result. During this period, it is unable to be interrupted, as doing so might jeopardize the state of the driver and hardware.
http://studentweb.tulane.edu/~jchrist1/Linux%20Unleashed,%20Third%20Edition/ch34/609-612.html. Retrieved on 2007-05-11 11:18.
ps Command Output The output of the ps command is always organized in columns. The first column is labeled PID, which means “Process ID” number. Every process on the system has to have a unique identifier so Linux can tell which processes it is working with. Linux handles processes by assigning a unique number to each process, called the process ID number (or PID). PIDs start at zero when the system is booted and increment by one for each process run, up to some system-determined number (such as 65,564) at which point it starts numbering from zero again, ignoring those that are still active. Usually, the lowest number processes are the system kernel and daemons, which start when Linux boots and remain active as long as Linux is running. When you are working with processes (such as terminating them), you must use the PID. The TTY column in the ps command output shows you which terminal the process was started from. If you are logged in as a user, this will usually be your terminal or console window. If you are running on multiple console windows, you will see all the processes you started in every window displayed. The STAT column in the ps command output shows you the current status of the process. The two most common entries in the status column are S for “sleeping” and R for “running.” A running process is one that is currently executing on the CPU. A sleeping process is one that is not currently active. Processes may switch between sleeping and running many times every second. The TIME column shows the total amount of system (CPU) time used by the process so far. These numbers tend to be very small for most processes because they require only a short time to complete. The numbers under the TIME column are a total of the CPU time, not the amount of time the process has been alive. Finally, the COMMAND column contains the name of the command line you are running. This is usually the command line you used, although some commands start up other processes. These are called “child” processes, and they show up in the ps output as if you had entered them as commands.
http://www.comfsm.fm/computing/UNIX/ps.html. Retrieved on 2007-05-11 11:18.
There are a few pieces of information about each process that need some explanation, expecially the cyptic ones like the STAT column. USER: The owner of the process, typically the user who started it PID: The processes unique ID number. These are assigned sequentially as processes start. When they reach 30,000 or so, the number starts over again at 0. 0-5 are usually low-level operating system processes which never exit, however. %CPU: Percentage of the CPU’s time spent running this process. %MEM: Percentage of total memory in use by this process VSZ: Total virtual memory size, in 1K blocks. RSS: Real Set Size, the actual amount of physical memory allocated to this process. TTY: Terminal associated with this process. A ? indicates the process is not connected to a terminal. STAT: Process state codes. Common states are S – Sleeping, R – Runnable (on run queue), N – Low priority task, Z – Zombie process START: When the process was started, in hours and minutes, or a day if the process has been running for a while. TIME: CPU time used by process since it started. COMMAND: The command name. This can be modified by processes as they run, so don’t rely on it abolutely!
http://www.slackbook.org/html/process-control-ps.html. Retrieved on 2007-05-11 11:18.
Second, there is a new column: STAT. It shows the status of the process. S stands for sleeping: the process is waiting for something to happen. Z stands for a zombied process. A zombied processes is one whose parent has died, leaving the child processes behind. This is not a good thing. D stands for a process that has entered an uninterruptible sleep. Often, these processes refuse to die even when passed a SIGKILL. You can read more about SIGKILL later in the next section on kill . W stands for paging. A dead process is marked with an X. A process marked T is traced, or stopped. R means that the process is runable.
pstree
… is a really interesting, more visual alternative to ps.
Kill
http://www.linuxquestions.org/questions/programming-9/killing-a-running-script-on-remote-computer-633539/. Retrieved on 2007-05-11 11:18.
Log on to the remote shell and use “ps” to find the process ID (pid) of the script you want to kill. Perhaps “ps -ef | grep scriptname”. Then use “kill -1 pid” to kill the process. If signal 1 doesn’t kill it, move through these signal values in this order – 12, 15 and 9.
man: manual
http://www.onlamp.com/pub/a/bsd/2000/10/04/FreeBSD_Basics.html
http://www.onlamp.com/pub/a/bsd/2000/10/11/FreeBSD_Basics.html
See also: whatis
apropos: search the manual page names and descriptions
> apropos "system log" gnome-system-log (1) - the GNOME System Log Viewer logger (1) - a shell command interface to the syslog(3) system log module logrotate (8) - rotates, compresses, and mails system logs sysklogd (8) - Linux system logging utilities. syslogd (8) - Linux system logging utilities. syslogd-listfiles (8) - list system logfiles xlogo (1) - X Window System logo
What does the number in parentheses mean? CRONTAB(5) ?
They’re section numbers. Usually you don’t have to specify one.
You can specify sections like this:
usage: man [section] name man 5 crontab
How to fetch a web page / make an HTTP request
wget
wget edit
Save as a different filename with -O option.
Don’t forget to enclose the URL in if it has >1 parameters (& mark)!
# Not gonna work: wget -O nt.gif http://www.neuroticweb.com/recursos/css-rounded-box/rounded.php cn=nt&co=e4ecec&ci=A9B8CF # Gonna work: wget -O nt.gif 'http://www.neuroticweb.com/recursos/css-rounded-box/rounded.php cn=nt&co=e4ecec&ci=A9B8CF'
Warning about & characters
Many URLs contain & characters. That’s fine as far as your web browser is concerned, but on the command line, & takes on a special meaning: it causes the command before the & to be started and backgrounded and then the part after the & (which in the case of a URL, wouldn’t be a valid command but it would be treated as a command nonetheless) to be started and foregrounded.
> echo 1 & echo 2 [2] 11524 2 1 [2]- Done echo 1
> wget http://sourceforge.net/project/downloading.php?groupname=wikipedia&filename=mediawiki-1.6.5.tar.gz&use_mirror=easynews [2] 11320 [3] 11321 ... [2] Exit 1 wget http://sourceforge.net/project/downloading.php?groupname=wikipedia [3]- Done filename=mediawiki-1.6.5.tar.gz
To get around this, simply and enclose the URL in quotes:
> wget "http://sourceforge.net/project/downloading.php?groupname=wikipedia&filename=mediawiki-1.6.5.tar.gz&use_mirror=easynews"
Retrying a troublesome download
http://playingwithsid.blogspot.com/2007/09/wget-re-tries.html. Retrieved on 2007-05-11 11:18.
So, the wget program grabbing the file from the Internet running in background sputters after 20 or so tries and gives up. This calls for extreme measures.
$ wget -t 0 -c http://examplesite.com/pub/xxxx.bz2The
-t(--tries) set to 0 (zero) which means it keeps on trying infinitely until the hell freezes over.And
-c(--continue) is to continue grabbing the half-downloaded file.
Downloading from sourceforge
Use the direct link it gives you (“Your download should begin shortly. If you are experiencing problems with the download please use this direct link.”).
So
http://easynews.dl.sourceforge.net/sourceforge/wikipedia/mediawiki-1.6.5.tar.gz
instead of
http://sourceforge.net/project/downloading.php?groupname=wikipedia&filename=mediawiki-1.6.5.tar.gz&use_mirror=easynews
curl
…
links/lynx: Text-only Web browser
…
How do I get the current date/time in ISO8601 format?
> date -Idate 2006-08-21 > date --iso-8601=date 2006-08-21
> date -Iminutes 2006-08-21T11:15-0700
Can I get rid of the annoying -0700 timezone indicator?
Not with the –iso-8601 option. But you can specify custom formats…
> date +%Y%m%dT%H%M%S 20060821T112047 > date +%Y%m%dT%H%M 20060821T1120
How do I do some operation on the most recently update file (which may change frequently) in the working directory?
ls -t | head -n 1 | vimopen
This is faster than doing ls -t and then typing out whichever filename is listed first…
How do I get the output to be on one line instead of many?
The old variable=`…`; echo $variable trick
# Either one of these: vimsub "`(output=\`cgrep Smith . il\` ; echo $output;)`" smith '"my_name"' vimsub "`echo \`cgrep Smith . il\``" smith '"my_name"' # Has this effect: in all files containing the word Smith (case insensitive), replace the word Smith with "my_name"
The old echo `…` trick
$ echo `cgrep Smith . il` ./faq.html ./contact.html ./index.html # But: $ cgrep Smith . il ./faq.html ./contact.html ./index.html
The old “… | cat -” trick
ls may list things in columns
To force it to list just one file name per line (1 column), you can do this:
ls | cat - or ls | grep . (some kind of text filter that doesn't alter anything)
How do I combine/merge/concatenate the output streams of two commands into one so that I can pipe it (as a single stream) to another command?
A common use for this technique would be if you need to prefix the output from command A with some literal string before piping it on to command B.
You want: {prefix string literal} + {output from command A} | {command B}
The easiest way that I know of to produce a string literal on standard out is with the echo command:
echo "hello there"
or
echo -n "hello there without a trailing newline"
Since echo is a command, though, I may as well generalize to find a way to combine the output of any two commands…
cat is no help to us; it can concatenate two files, but that doesn’t help us concatenate two “input streams”.
cat - is no help either: it simply takes the single input stream it receives and passes it along to its standard output stream.
Currently the best solution I’ve found is this, combining echo and ``, but I hope someone will show me a better way:
$ echo `echo -n 'prefix '``echo -e "main\noutput\nstream"` | cat - prefix main output stream
Here’s another (this one a real-world) example:
echo -n 'String '`xclip -o` | xmacroplay :0 >/dev/null 2>/dev/null
The main problem with that solution is that it strips out all newline characters (\n) and puts the entire output stream on a single line!
This is fine sometimes, but some of the time you will have multiple lines of input that you will want to stay intact!
…
Found it! #Grouping commands!
> { echo -n 'prefix '; echo -e "main\noutput\nstream"; } | cat -
prefix main
output
stream
Grouping commands
http://www.network-theory.co.uk/docs/bashref/CommandGrouping.html. Retrieved on 2007-05-11 11:18.
{ list; }Placing a list of commands between curly braces causes the list to be executed in the current shell context. No subshell is created. The semicolon (or newline) following list is required.
http://bash-hackers.org/wiki/doku.php?id=syntax:ccmd:grouping_plain. Retrieved on 2007-05-11 11:18.
The input and output file descriptors are cumulative:
{ echo "PASSWD follows" cat /etc/passwd echo echo "GROUPS follows" cat /etc/group } >output.txtThis compound command also usually is the body of a function definition, though not the only compound command that’s valid there:
print_help() { echo "Options:" echo "-h This help text" echo "-f FILE Use config file FILE" echo "-u USER Run as user USER" }
Full-screen applications: When you close them, why does that screen stay on the screen with some terminals but for others not
Examples of what I call a “class A” full-screen app:
Here’s “class B” full-screen apps, which seem to behave a little bit differently:
I’ve seen one of two behaviors:
Screen restores When you close a class-A full-screen app, the screen is restored to exactly how it was before you started the full-screen app. No output whatsoever from the class-A app is visible anymore (except with aptitude: after installing some packages, that output apparently outputs straight to standard output, and that output is residual after exiting aptitude (the actual GUI part of aptitude, however, is not visible after exiting)). Screen stays When you close a class-A full-screen app, everything that was on the screen when you closed that full-screen app (the “last full-screen”) stays visible, but is shifted upward one line to make room for a command prompt at the bottom of the screen. This is very useful if, for example, you open a man page for reference, and then want to suspend it temporarily in order to try out something that you just learned; since the last full-screen is still visible, you can refer back to it as you continue to type on the command line!. This is my preferred behavior!
When I say “close”, I mean either exit/quit or simply suspend with Ctrl+Z; both seem to have the identical effect in my tests.
[To do: put in some screenshots that show each of these two behaviors]
Where do I see each of these behaviors:
- Ubuntu 7.10 + GNOME + {gnome-terminal, xterm, etc.}: Screen restores for class-A apps (for class-B apps, it stays)
- Windows using PuTTY: Screen stays
How do I fix this in Ubuntu??
How does time work?
[Under the hood (category)]
It’s almost like it runs a separate process or something…
Bash
Bash edit
This is for stuff pertaining to Bash, but probably a lot of it pertains to other POSIX shells too. I only call it “Bash” because that’s what I use and I’m not sure if things work in other shells or not because I don’t use other shells.
http://www.ibm.com/developerworks/library/l-bash-parameters.html?ca=drs-. Retrieved on 2007-05-11 11:18.
The bash shell is available on many Linux® and UNIX® systems today, and is a common default shell on Linux.
GNU/Linux command line edit (Category edit) .
GNU/Linux edit (Category edit) .
GNU/Linux edit (Category edit) .
Reference links
- gnu.org reference
- tdlp.org: Bash Guide for Beginners
Be efficient: reuse commands you’ve already typed.
You can simply press Up to get back to your last command. Keep pressing up or down to browse through your [readline] history.
If you know what the command you want to re-use starts with or if it contains some unique string of text, then check out “event designators”.
If you don’t remember exactly what it looked like or how long ago you last used it or you just want to see a list of all the commands you’ve recently used, try the history command. It’s neat!
history command
Typing history alone, will show you your entire history … which may not be what you want, particularly if it’s 1000s of lines long!
Instead, get in the habit of specifying a number as the argument, which will specify how many lines it returns. So if you do history 10, you’ll get a list of the 10 most recent commands you’ve typed.
> history 10 1377 hg color 1378 hg unroller 1379 vim ~/.bashrc 1380 . ~/.bashrc 1381 h 1382 hg unroller 1383 history --help 1384 history -d 1293 1385 man history 1386 history 10
This is basically equivalent to just doing history | tail -n10
To execute one of the commands listed in the history, just type !{line_number}. For example, assumin the history above !1380 would execute the . ~/.bashrc command.
I’ve added these aliases to my ~/.bashrc to make it more convenient to work with the history command:
alias hist='history' # Slightly shorter to type. alias his='history' alias hi='history' alias h='history 50' # Show about a page worth of history. This is what I'll want to do most of the time. alias hgrep='history|grep ' # List all command lines containing the search term. alias hg='history|grep '
man history will take you to BASH_BUILTINS(1); from there, search for /^ *history and you’ll jump right to the section.
history [n]
history -c
history -d offset
history -anrw [filename]
history -p arg [arg ...]
history -s arg [arg ...]
With no options, display the command history list with line numbers. Lines listed with a * have been modified. An argument of
n lists only the last n lines. If the shell variable HISTTIMEFORMAT is set and not null, it is used as a format string for
strftime(3) to display the time stamp associated with each displayed history entry. No intervening blank is printed between
the formatted time stamp and the history line. If filename is supplied, it is used as the name of the history file; if not,
the value of HISTFILE is used. Options, if supplied, have the following meanings:
-c Clear the history list by deleting all the entries.
-d offset
Delete the history entry at position offset.
-a Append the âânewââ history lines (history lines entered since the beginning of the current bash session) to the history
file.
-n Read the history lines not already read from the history file into the current history list. These are lines appended
to the history file since the beginning of the current bash session.
-r Read the contents of the history file and use them as the current history.
-w Write the current history to the history file, overwriting the history fileâs contents.
-p Perform history substitution on the following args and display the result on the standard output. Does not store the
results in the history list. Each arg must be quoted to disable normal history expansion.
-s Store the args in the history list as a single entry. The last command in the history list is removed before the args
Options
HISTCONTROL
A colon-separated list of values controlling how commands are saved on the history list. If the list of values includes ignorespace, lines
which begin with a space character are not saved in the history list. A value of ignoredups causes lines matching the previous history
entry to not be saved. A value of ignoreboth is shorthand for ignorespace and ignoredups. A value of erasedups causes all previous lines
matching the current line to be removed from the history list before that line is saved. Any value not in the above list is ignored. If
HISTCONTROL is unset, or does not include a valid value, all lines read by the shell parser are saved on the history list, subject to the
value of HISTIGNORE. The second and subsequent lines of a multi-line compound command are not tested, and are added to the history regard‐
less of the value of HISTCONTROL.
HISTFILE
The name of the file in which command history is saved (see HISTORY below). The default value is ~/.bash_history. If unset, the command
history is not saved when an interactive shell exits.
HISTFILESIZE
The maximum number of lines contained in the history file. When this variable is assigned a value, the history file is truncated, if nec‐
essary, by removing the oldest entries, to contain no more than that number of lines. The default value is 500. The history file is also
truncated to this size after writing it when an interactive shell exits.
HISTIGNORE
A colon-separated list of patterns used to decide which command lines should be saved on the history list. Each pattern is anchored at the
beginning of the line and must match the complete line (no implicit ‘*’ is appended). Each pattern is tested against the line after the
checks specified by HISTCONTROL are applied. In addition to the normal shell pattern matching characters, ‘&’ matches the previous history
line. ‘&’ may be escaped using a backslash; the backslash is removed before attempting a match. The second and subsequent lines of a
multi-line compound command are not tested, and are added to the history regardless of the value of HISTIGNORE.
HISTSIZE
The number of commands to remember in the command history (see HISTORY below). The default value is 500.
HISTTIMEFORMAT
If this variable is set and not null, its value is used as a format string for strftime(3) to print the time stamp associated with each
history entry displayed by the history builtin. If this variable is set, time stamps are written to the history file so they may be pre‐
served across shell sessions.
Event designators (! commands)
Event Designators
An event designator is a reference to a command line entry in the his-
tory list.
! Start a history substitution, except when followed by a blank,
newline, = or (.
!n Refer to command line n.
!-n Refer to the current command line minus n.
!! Refer to the previous command. This is a synonym for â!-1â.
!string
Refer to the most recent command starting with string.
!?string[?]
Refer to the most recent command containing string. The trail-
ing ? may be omitted if string is followed immediately by a new-
line.
^string1^string2^
Quick substitution. Repeat the last command, replacing string1
with string2. Equivalent to ââ!!:s/string1/string2/ââ (see Mod-
ifiers below).
!# The entire command line typed so far.
Especially useful, I find, is the !? variety:
!?svn export !?svn export?:s/home/moo/ !?generate contr ./script/generate controller user
Problem: Quoted single argument containing spaces is treated as multiple arguments
action="svn ci -m 'My commit message'" cd /home/services/httpd/shared/include/ $action svn: Commit failed (details follow): svn: '/home/services/httpd/shared/include/commit' is not under version control
cp /some/long/path /some/other/long/path vim $_
is easier to type than
cp /some/long/path /some/other/long/path vim /some/other/long/path
Also:
cp a b c d e f /some/long/path/to/dest/folder/ cd $_
rather than
cp a b c d e f /some/long/path/to/dest/folder/ cd $_
Also, if you ever discover that a directory that you thought existed (or should exist) doesn’t exist…as in the following situation, this is a handy trick…
> cp dev/shell/config/irbrc dev/public/ruby/ cp: cannot create regular file `dev/public/ruby/irbrc': No such file or directory > mkdir $_ (try again)
When last argument has spaces
You can’t just do this, because for some reason, even though you escaped the space, $_ still treats those two words as separate arguments.
~ > mkdir Two\ Words ~ > cd $_ bash: cd: Two: No such file or directory ~ > mkdir 'Two Words' ~ > cd $_ bash: cd: Two: No such file or directory # It's exactly as if you'd done this: ~ > cd Two Words bash: cd: Two: No such file or directory ~ > echo $_ Words # See?
This is how you gotta do it (wrap $_ in double quotes):
~ > cd "$_" ~/Two Words >
Positional parameters ($*)
* Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, it expands to a single word with
the value of each parameter separated by the first character of the IFS special variable. That is, "$*" is equivalent to "$1c$2c...",
where c is the first character of the value of the IFS variable. If IFS is unset, the parameters are separated by spaces. If IFS is
null, the parameters are joined without intervening separators.
@ Expands to the positional parameters, starting from one. When the expansion occurs within double quotes, each parameter expands to a sep‐
arate word. That is, "$@" is equivalent to "$1" "$2" ... If the double-quoted expansion occurs within a word, the expansion of the first
parameter is joined with the beginning part of the original word, and the expansion of the last parameter is joined with the last part of
the original word. When there are no positional parameters, "$@" and $@ expand to nothing (i.e., they are removed).
"$*" is equivalent to "$1 $2 ..." (assuming $IFS is ‘ ‘).
"$@" is equivalent to "$1" "$2" ....
> echo "'"$IFS"'" ' '
See http://svn.tylerrick.com/public/examples/bash/CommandLineArguments-* for some good examples of when you would need to use “$@”.
You can copy your own variables into the positional parameters
Before you do, though, be sure to make a backup:
original_params=("$@")
> set -- "a b" c > print_args $@ There are 3 args: $1=a $2=b $3=c
How to get full path of script
cd `pwd`/`dirname $0`/
Does this always work?
Not when you have symlinks apparently…
Lance came up with this… I’m not sure if he still believes it’s necessary…
[sed (category)]
DIR=`ls -l \`pwd\`/\`dirname $0\`/ | grep "\`basename $0\` -> " | sed -e 's/^.* -> \(.*\)$/\1/' | xargs dirname`
How can I start and immediately background 2 scripts (script1 & ; script2 &)?
Doesn't work: $ ./script/server & ; tail -f log/development.log & -bash: syntax error near unexpected token `;' Works: $ ./script/server & tail -f log/development.log &
Here’s the thought process I use to keep things straight:
~/.bash_profile:- This is only sourced when you first log in with your user.
- So put things there that you basically want to only run once per “login”…
- For instance, additions to the $PATH environment variable. It’s better to put that here than in
~/.bashrcbecause if you put it in~/.bashrc, you will keep adding to $PATH every time yousu yourself, I think. (Not that you probably do that very often.) So that can cause it to append the same thing to $PATH repeatedly (and unnecessarily), making it really long.
~/.bashrc:- Things that it’s okay to do repeatedly / more often with no side-effects.
- Don’t put $PATH additions here.
- It’s okay to put aliases, bash functions, etc. here
- It’s okay to set variables here, like $EDITOR
When bash is invoked as an interactive login shell, it reads and executes commands from:
- /etc/profile
- ~/.bash_profile
- ~/.bash_login
- ~/.profile
When an interactive (but non-login) shell is started, bash reads and executes commands from:
- /etc/bash.bashrc
- ~/.bashrc
Usually, I see a line in ~/.bash_profile that also sources ~/.bashrc (so put things there that should happen no matter what) !
interactive login shell:
> sudo su - tyler ~/.bash_profile ~/.bashrc ~/.bashrc
interactive (but non-login) shell
> sudo su tyler ~/.bashrc
Alex Zarutin at Unix Shell – Difference between .bashrc and .bash_profile? (http://www.webservertalk.com/archive109-2005-1-898875.html). Retrieved on 2007-03-13 15:42.
contains a list of commands to be executed when you log in and contains a list of commands to be executed every time you open a new shell. There is a slight difference between them: is read once at the beginning of a session, whereas is read every time you open a new terminal (e.g. a new xterm window). In a traditional setup you would define variables like PATH in , and things like aliases and functions in . But since usually is pre-configured to read the content of anyway, you might as well save some effort and put all your configuration into .
http://www.linuxforums.org/forum/linux-newbie/1182-difference-between-bashrc-bash_profile.html :
.bashrc is only used when you start bash as a non-login shell. There are some other files and environment variables used, too. A login shell is the shell you get when you log in, and that means that it sets up some extra things, like aliases, extra PATH elements and completion stuff, that you only use in interactive mode, while an ordinary shell is a shell that interprets shell scripts and such, where some aliases and completion stuff won’t be needed. A login shell might also print a greeting or so, and you probably won’t want that in a script interpreter. Note, however, that a shell can be an interactive shell without being a login shell. That commonly includes the situations where the shell is invoked “under” a login shell, such as in X. All PATH elements and stuff have already been set up when the X session script runs, and therefore the shells started in xterms need not do it again.
From `man bash` in the Invocation section:
A login shell is one whose first character of argument zero is a -, or one started with the –login option.
An interactive shell is one started without non-option arguments and without the -c option whose standard input and error are both connected to terminals (as determined by isatty(3)), or one started with the -i option. PS1 is set and $- includes i if bash is interactive, allowing a shell script or a startup file to test this state.
The following paragraphs describe how bash executes its startup files. If any of the files exist but cannot be read, bash reports an error. Tildes are expanded in file names as described below under Tilde Expansion in the EXPANSION section.
When bash is invoked as an interactive login shell, or as a non-interactive shell with the –login option, it first reads and executes commands from the file /etc/profile, if that file exists. After reading that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile, in that order, and reads and executes commands from the first one that exists and is readable. The –noprofile option may be used when the shell is started to inhibit this behavior.
When a login shell exits, bash reads and executes commands from the file ~/.bash_logout, if it exists.
When an interactive shell that is not a login shell is started, bash reads and executes commands from ~/.bashrc, if that file exists. This may be inhibited by using the –norc option. The –rcfile file option will force bash to read and execute commands from file instead of ~/.bashrc.
When bash is started non-interactively, to run a shell script, for example, it looks for the variable BASH_ENV in the environment, expands its value if it appears there, and uses the expanded value as the name of a file to read and execute. Bash behaves as if the following command were executed:
if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fibut the value of the PATH variable is not used to search for the file name.
Floyd L. Davidson at Unix Shell – Difference between .bashrc and .bash_profile? (http://www.webservertalk.com/archive109-2005-1-898875.html). Retrieved on 2007-03-13 15:46.
1) The profile file […] is read only by login shells. It should have things that initialize the terminal or otherwise need be done one time only when a user does a login. Typically, initializing a terminal is slow. Hence one common example might be “stty ^H erase”. Another example is “PATH=~/bin:$PATH”, which is quick and easy, but if done for every subshell adds unnecessary search elements to the PATH variable if subshells are layers deep. 2) The rc file is read by every interactive subshell (and not by login shells). It should have things that are needed by for interactive use, but are excess baggage for non-interactive subshells. Typically that would be the definition of aliases. … The basic idea, which made obvious sense back when the functionality was first added to /bash/, is to speed up shell initialization by compartmentalizing it. When a 10Mb disk was the norm, and it was *SLOW*, the time spent slowly reading in a large _~/.profile_ for every single shell invocation, the way /sh/ did, was significant and a very noticeable bottleneck. Today of course not only is the disk file many many times faster itself, but we have the kernel cashing oft used disk files and it can be assumed that for a typical /bash/ invocation all of the init files have already been cached in RAM, and no disk activity is even done.
Thorsten Kampe at Unix Shell – Difference between .bashrc and .bash_profile? (http://www.webservertalk.com/archive109-2005-1-898875.html). Retrieved on 2007-03-13 15:46.
This is less efficient, as environment that is inherited is redefined. and expressions like PATH=/new/path:${PATH} get a /new/path added with every instance of nested bash’es.
That’s correct (but again a bash nuisance). Put all your stuff in ..bashrc and only those things only needed in a login shell in to ..bash_profile. My .bash_profile contains only “source ~/.bashrc”.
> echo $PATH | wc -c 425 [tyler: ~]> sudo su tyler [tyler: ~]> echo $PATH | wc -c 505 # it grows every time! [tyler: ~]> sudo su - tyler [tyler: ~]> echo $PATH | wc -c 265 # here it apparently gets reset!
Syntax
> echo "reverberate" \ > [prompts for input, allowing you to continue your command on the next line] reverberate
But this doesn’t work:
> echo "reverberate" # a comment \ reverberate [prints immediately]
Nor does this:
> echo "reverberate" \ # a comment reverberate [prints immediately]
It would be nice to be able to document each line in a multi-line command. For example:
sudo gem install --local pkg/OurExtensions-0.0.1.gem \ --rdoc # So you can test that the rdoc installs properly \ --test # So you can ensure that all the tests pass still \ --force # In case it's already installed
Is there anyway to do that? Make a nice well-documented multi-line comment?
[Escaping (category)] things
man bash
!, must be quoted to prevent history expansion.
There are three quoting mechanisms: the escape character, single quotes, and double quotes.
A non-quoted backslash (\) is the escape character. It preserves the literal value of the next character that follows, with the
exception of . If a \ pair appears, and the backslash is not itself quoted, the \ is treated as a line
continuation (that is, it is removed from the input stream and effectively ignored).
Enclosing characters in single quotes preserves the literal value of each character within the quotes. A single quote may not occur
between single quotes, even when preceded by a backslash.
Enclosing characters in double quotes preserves the literal value of all characters within the quotes, with the exception of $, â, \,
and, when history expansion is enabled, !. The characters $ and â retain their special meaning within double quotes. The backslash
retains its special meaning only when followed by one of the following characters: $, â, ", \, or . A double quote may be
quoted within double quotes by preceding it with a backslash. If enabled, history expansion will be performed unless an ! appearing
in double quotes is escaped using a backslash. The backslash preceding the ! is not removed.
The special parameters * and @ have special meaning when in double quotes (see PARAMETERS below).
[Problems (category)] ! treated as event designator, even within double quotes (“)
Example of problem:
[tyler: ~]> echo "!" -bash: !: event not found [tyler: ~]> echo -e "#\!/usr/bin/ruby\nputs 'hi'" #\!/usr/bin/ruby puts 'hi' [tyler: ~]> echo -e "#!/usr/bin/ruby\nputs 'hi'" -bash: !/usr/bin/ruby\nputs: event not found
Solution/workaround:
[tyler: ~]> echo -e '#!'"/usr/bin/ruby\nputs 'hi'" #!/usr/bin/ruby puts 'hi'
Expressed even more concisely, here is the same solution again:
[tyler: ~/dev/tyler]> echo "Hi!" -bash: !": event not found [tyler: ~/dev/tyler]> echo "Hi\!" Hi\! [tyler: ~/dev/tyler]> echo "Hi"'!' Hi! [tyler: ~/dev/tyler]> echo 'Hi!' Hi!
[Here document strings (category)] (heredoc strings)
Lets you create a multi-line string of input to be given as standard input to a script or program.
> cat - line 2 > End line 1 line 2
Useful for quickly creating a new file and putting some contents in it… [I guess this doesn’t work after all. Is something like this not possible??]
> out line 2 > End > cat out line 1 line 2
Unfortunately, this can’t be used to pass multi-line command-line arguments to a program. I still haven’t found a way to do that.
You might try this:
> echo 'line 1'"\n"'line 2' line 1\nline 2
but unfortunately the “\n” is not treated as a newline character. Same with when I tried that trick for svn:
> svn commit -m 'line 1'"\n"'line 2'
Echo does let you do stuff like that, with its -e option — but most programs don’t have an option like that:
> echo -e 'line 1'"\n"'line 2' line 1 line 2
if/then/fi statements
http://svn.tylerrick.com/public/bash/examples/if_then-cant_have_empty_block.sh
#!/bin/bash
if false; then
# Do nothing
else
echo 'Hello world!'
fi
Produces a syntax error:
./test.sh: line 5: syntax error near unexpected token `else' ./test.sh: line 5: `else'
Workaround:
if false; then
# Do nothing
/bin/true
else
echo 'Hello world!'
fi
# outputs "Hello world!"
for Loops
http://www.google.com/search?q=bash+for+loops
To list all subdirectories of the current directory:
for dir in *; do if [ -d $dir ]; then echo $dir; fi; done
(Am I just missing something? I didn’t see any options for ls that would make it list only directories…)
To execute a command inside each subdirectory of the current directory:
for dir in *; do if [ -d $dir ]; then cd $dir; do_something; cd ..; fi; done for dir in *; do if [ -d $dir ]; then cd $dir; pwd; cd ..; fi; done
For every Rails application in your ‘projects’ directory, run the tyler_svn_configure balloon:
~/projects/ > for dir in *; do if [ -d $dir ]; then echo; echo $dir; cd $dir; ruby -ropen-uri -e 'eval(open("http://balloon.hobix.com/tyler_svn_configure").read)'; cd ..; fi; done
http://tldp.org/LDP/abs/html/loops1.html
Numerical
> for i in `seq 1 3`; do \ > echo $i > done 1 2 3
> i=1; for x in a b c d; do \ > echo "$i. $x" > let i+=1 > done
Command completion
Just press tab.
You can set up custom completion with the complete command.
FIGNORE
A colon-separated list of suffixes to ignore when performing filename completion (see READLINE below). A filename whose suffix matches one
of the entries in FIGNORE is excluded from the list of matched filenames. A sample value is ".o:~" (Quoting is needed when assigning a
value to this variable, which contains tildes).
if / ifneq / fi / tests / conditions
…
Article metadata
Keywords: terminal, console, command line
GNU/Linux / Command line edit (Category edit)