From WhyNotWiki

This is about how to work efficiently on the command line in GNU/Linux (and probably other POSIX OS's, although I haven't tested that theory).

[edit] Reference links / Cheat sheets / Tutorials

Osamu Aoki. Debian Reference Chapter 8 - Debian tips (http://www.debian.org/doc/manuals/reference/ch-tips.en.html). Retrieved on 2007-05-11 11:18. {{4 stars}

The Linux Cookbook: Tips and Techniques for Everyday Use - Viewing Text (http://www.dsl.org/cookbook/cookbook_13.html#SEC183). Retrieved on 2007-05-11 11:18.

The Ultimate Linux Reference Guide for Newbies (http://blog.lxpages.com/ultimate_linux.html). Retrieved on 2007-05-11 11:18.

Bash Hackers Wiki (http://bash-hackers.org/wiki/doku.php?id=). Retrieved on 2007-05-11 11:18.

This wiki is intended to hold documentations of any kind about the GNU Bash. It’s one of more applications of bash-hackers.org site. See also the forum.

!: Please direct any questions about Bash, and also discussions about this wiki to the forum :!:

The main motivation was to provide human-readable documentation and information to not force users to read every bit of the Bash manpage - which is PITA sometimes. The docs here are not meant as newbie tutorial, more as educational summary.

Stranger! Feel free to register and edit the contents. The registration is only there to prevent SPAM.

http://wooledge.org/mywiki. Retrieved on 2007-05-11 11:18.

The 'official' channel FAQ for freenode's #bash channel is BashFAQ. For common mistakes made by Bash programmers, see BashPitfalls. For general Unix issues, see DotFiles, Permissions, ProcessManagement, or UsingFind.

Miscellaneous pages: FtpMustDie, XyProblem.

http://www.johnstowers.co.nz/wiki/index.php/Useful_Commands

http://www.pixelbeat.org/cmdline.html

[edit] Important key combinations to know

Ctrl-c  Stop current foreground process
Ctrl-z  Suspend current foreground process
Ctrl-d  Send end-of-file character (needed for mail command, etc.)

[edit] Readlines / ~/.inputrc / stty

man readline:

readline will read a line from the terminal and return it, using prompt as a prompt. If prompt is NULL or the empty string, no prompt issued. The line returned is allocated with malloc(3); the caller must free it when finished. The line returned has the final newline removed, so only the text of the line remains.

readline offers editing capabilities while the user is entering the line. By default, the line editing commands are similar to those of emacs. A vi-style line editing interface is also available.

...

An emacs-style notation is used to denote keystrokes. Control keys are denoted by C-key, e.g., C-n means Control-N. Similarly, meta keys are denoted by M-key, so M-x means Meta-X. (On keyboards without a meta key, M-x means ESC x, i.e., press the Escape key then the x key. This makes ESC the meta prefix. The combination M-C-x means ESC-Control-x, or press the Escape key then hold the Control key while pressing the x key.)

   Commands for Moving
       beginning-of-line (C-a)
              Move to the start of the current line.
       end-of-line (C-e)
              Move to the end of the line.
       forward-char (C-f)
              Move forward a character.
       backward-char (C-b)
              Move back a character.
       forward-word (M-f)
              Move forward to the end of the next word.  Words are composed of alphanumeric characters (letters and digits).
       backward-word (M-b)
              Move back to the start of the current or previous word.  Words are composed of alphanumeric characters (letters and digits).
       clear-screen (C-l)
              Clear  the  screen leaving the current line at the top of the screen.  With an argument, refresh the current line without clearing the
              screen.
       redraw-current-line
              Refresh the current line.

   Commands for Manipulating the History
       accept-line (Newline, Return)
              Accept the line regardless of where the cursor is.  If this line is non-empty, it may be added to the history list for  future  recall
              with add_history().  If the line is a modified history line, the history line is restored to its original state.
       previous-history (C-p)
              Fetch the previous command from the history list, moving back in the list.
       next-history (C-n)
              Fetch the next command from the history list, moving forward in the list.
       beginning-of-history (M-<)
              Move to the first line in the history.
       end-of-history (M->)
              Move to the end of the input history, i.e., the line currently being entered.
       reverse-search-history (C-r)
              Search backward starting at the current line and moving ‘up’ through the history as necessary.  This is an incremental search.
       ...

   Commands for Changing Text
       delete-char (C-d)
              Delete the character at point.  If point is at the beginning of the line, there are no characters in the line, and the last  character
              typed was not bound to delete-char, then return EOF.
       backward-delete-char (Rubout)
              Delete the character behind the cursor.  When given a numeric argument, save the deleted text on the kill ring.
       forward-backward-delete-char
              Delete  the  character under the cursor, unless the cursor is at the end of the line, in which case the character behind the cursor is
              deleted.
       quoted-insert (C-q, C-v)
              Add the next character that you type to the line verbatim.  This is how to insert characters like C-q, for example.
       tab-insert (M-TAB)
              Insert a tab character.
       self-insert (a, b, A, 1, !, ...)
              Insert the character typed.
       transpose-chars (C-t)
              Drag the character before point forward over the character at point, moving point forward as well.  If point is  at  the  end  of  the
              line, then this transposes the two characters before point.  Negative arguments have no effect.
       transpose-words (M-t)
              Drag  the  word before point past the word after point, moving point over that word as well.  If point is at the end of the line, this
              transposes the last two words on the line.
       upcase-word (M-u)
              Uppercase the current (or following) word.  With a negative argument, uppercase the previous word, but do not move point.
       downcase-word (M-l)
              Lowercase the current (or following) word.  With a negative argument, lowercase the previous word, but do not move point.
       capitalize-word (M-c)
              Capitalize the current (or following) word.  With a negative argument, capitalize the previous word, but do not move point.
       ...

   Killing and Yanking
       kill-line (C-k)
              Kill the text from point to the end of the line.
       backward-kill-line (C-x Rubout)
              Kill backward to the beginning of the line.
       unix-line-discard (C-u)
              Kill backward from point to the beginning of the line.  The killed text is saved on the kill-ring.
       kill-whole-line
              Kill all characters on the current line, no matter where point is.
       kill-word (M-d)
              Kill from point the end of the current word, or if between words, to the end of the next word.  Word boundaries are the same as  those
              used by forward-word.
       backward-kill-word (M-Rubout)
              Kill the word behind point.  Word boundaries are the same as those used by backward-word.
       unix-word-rubout (C-w)
              Kill the word behind point, using white space as a word boundary.  The killed text is saved on the kill-ring.
       unix-filename-rubout
              Kill  the  word behind point, using white space and the slash character as the word boundaries.  The killed text is saved on the kill-
              ring.
       delete-horizontal-space (M-\)
              Delete all spaces and tabs around point.
       kill-region
              Kill the text between the point and mark (saved cursor position).  This text is referred to as the region.
       copy-region-as-kill
              Copy the text in the region to the kill buffer.
       copy-backward-word
              Copy the word before point to the kill buffer.  The word boundaries are the same as backward-word.
       copy-forward-word
              Copy the word following point to the kill buffer.  The word boundaries are the same as forward-word.
       yank (C-y)
              Yank the top of the kill ring into the buffer at point.
       yank-pop (M-y)
              Rotate the kill ring, and yank the new top.  Only works following yank or yank-pop.

   ...

   Completing
       complete (TAB)
              Attempt to perform completion on the text before point.  The actual completion performed is application-specific.  Bash, for instance,
              attempts  completion  treating  the text as a variable (if the text begins with $), username (if the text begins with ~), hostname (if
              the text begins with @), or command (including aliases and functions) in turn.  If none of these produces a match, filename completion
              is  attempted.   Gdb,  on  the other hand, allows completion of program functions and variables, and only attempts filename completion
              under certain circumstances.
       possible-completions (M-?)
              List the possible completions of the text before point.
       insert-completions (M-*)
              Insert all completions of the text before point that would have been generated by possible-completions.
       menu-complete
              Similar to complete, but replaces the word to be completed with a single match from the list of possible completions.  Repeated execu‐
              tion of menu-complete steps through the list of possible completions, inserting each match in turn.  At the end of the list of comple‐
              tions, the bell is rung (subject to the setting of bell-style) and the original text is restored.  An argument of n moves n  positions
              forward  in  the  list  of matches; a negative argument may be used to move backward through the list.  This command is intended to be
              bound to TAB, but is unbound by default.
       delete-char-or-list
              Deletes the character under the cursor if not at the beginning or end of the line (like delete-char).  If at  the  end  of  the  line,
              behaves identically to possible-completions.

   ...

   Miscellaneous
       ...
       prefix-meta (ESC)
              Metafy the next character typed.  ESC f is equivalent to Meta-f.
       undo (C-_, C-x C-u)
              Incremental undo, separately remembered for each line.
       revert-line (M-r)
              Undo all changes made to this line.  This is like executing the undo command enough times to return the line to its initial state.
       tilde-expand (M-&)
              Perform tilde expansion on the current word.
       ...
       character-search (C-])
              A character is read and point is moved to the next occurrence of that character.  A negative count searches for previous  occurrences.
       character-search-backward (M-C-])
              A  character is read and point is moved to the previous occurrence of that character.  A negative count searches for subsequent occur‐
              rences.
       insert-comment (M-#)
              Without a numeric argument, the value of the readline comment-begin variable is inserted at the beginning of the current line.   If  a
              numeric argument is supplied, this command acts as a toggle:  if the characters at the beginning of the line do not match the value of
              comment-begin, the value is inserted, otherwise the characters in comment-begin are deleted from the beginning of the line.  In either
              case, the line is accepted as if a newline had been typed.  The default value of comment-begin makes the current line a shell comment.
              If a numeric argument causes the comment character to be removed, the line will be executed by the shell.

Here I have been using readline for years whenever I used the bash shell and I didn't even realize it. Nor did I know about many hidden features of readline. It's like a fully-featured text-editor built-in to a single line -- crazy — Tyler (2007-08-06 20:01)

[edit] Troubleshooting

[edit] sudo: __: command not found, but works if you give it the full path

[1] suggests looking at sudo /usr/bin/env | grep PATH and seeing if that is different.

For me right now it is different:

> sudo /usr/bin/env | grep PATH
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/X11R6/bin
> /usr/bin/env | grep PATH
PATH=/var/lib/gems/1.8/gems/subwrap-0.3.10/bin:/home/tyler/bin:/home/tyler/public/shell/bin:/var/lib/gems/1.8/bin:/home/tyler/bin:/home/tyler/bin:/home/tyler/public/shell/bin:/home/tyler/public/shell/devscripts_bin/:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games/var/lib/gems/1.8/bin/:/var/lib/gems/1.8/bin/:/var/lib/gems/1.8/bin/:/var/lib/gems/1.8/bin/:/var/lib/gems/1.8/bin/:/var/lib/gems/1.8/bin/:/var/lib/gems/1.8/bin/

(Note that sudo echo $PATH and echo $PATH are identical -- so apparently sudo echo $PATH is lying and only sudo /usr/bin/env | grep PATH can be trusted...?)

Why is it different?

On another machine I have access to, sudo /usr/bin/env | grep PATH and /usr/bin/env | grep PATH are identical.... as it seems they should be.

http://forums.macosxhints.com/showthread.php?t=20781 suggestion is to "create a new user". I don't want to create a new user!

I read somewhere that if you have an error or anything wrong in your .profile/.bash_profile/.bashrc that sudo will get "confused" and fall back to its default (truncated) value for $PATH. I tried commenting out a bunch of stuff in my .* files and it didn't seem to fix anything.

[edit] Troubleshooting: Fixing broken terminals

[edit] Not echoing back / not letting you type

If the terminal isn't echoing back what you type to it, type reset.

If it locks up (in vim usually), press Ctrl+q.

[edit] There's junk characters on the screen!

clear or Ctrl+l will redraw the screen for you.

[edit] Not echoing anything you type (but still responding to what you type)

Cure:

stty echo

How to cause this problem: The terminal can get hosed if you cat some binary files that have certain characters that mess up the terminal.

[edit] Troubleshooting: Fixing keys that don't seem to be working

[edit] Home key doesn't work?

You can probably use Ctrl+a to do the same thing.

A more permanent solution:

export TERM=linux

On other systems, the fix is to change it from linux to something else:

export TERM=xterm

[edit] Vim outputs H when you press Home and F when you press End?

:!uname -a
[No write since last change]
Linux ... 2.6.16-1.2069_FC4smp #1 SMP Tue Mar 28 12:47:32 EST 2006 i686 i686 i386 GNU/Linux

:!echo $TERM
linux

Solution?

export TERM=xterm

and restart vim.

[edit] Delete key doesn't work?

This has happened to me on OpenBSD (at work) and FreeBSD (on nearlyfreespeech.net). —Tyler)

Edit your ~/.inputrc. Add this line:

"\e[3~": delete-char

[edit] Problems: You can't move a file into a directory that doesn't exist

For crying out loud. Why can't it just make the directory if it doesn't exist?? It doesn't complain that the target file doesn't exist. Why can't it be equally complacent about the target directory?.

As a result of this annoyance, one often has to make extra calls to mkdir (or mkdir -p, since mkdir suffers from the same problem otherwise).

[edit] Redirection / Input/output streams / File descriptors

[edit] Introduction

BASH Programming - Introduction HOW-TO: All about redirection (http://tldp.org/HOWTO/Bash-Prog-Intro-HOWTO-3.html). Retrieved on 2007-05-11 11:18.

There are 3 file descriptors: stdin, stdout and stderr (std=standard).

...

[edit] Why there are 2 standard output streams (stdout, stderr)

This lets you do things like:

some_important_command | grep something

and still see error output from the command (even if the errors don't match the grep search term).

[edit] What if I want everything to be output to stdout (a single stream that I can pipe somewhere)?

If you'd rather treat stderr as stdout, just redirect stderr to stdout:

some_important_command 2>&1 | grep something

[edit] What if I want all output streams (file descriptors) redirected to a file?

Then &> is your best bet.

> svn mkdir lib
svn: Try 'svn add' or 'svn add --non-recursive' instead?
svn: Can't create directory 'lib': File exists

> svn mkdir lib &>/dev/null

[edit] What if I don't want to see any output?

Then redirect to /dev/null

> svn mkdir lib
svn: Try 'svn add' or 'svn add --non-recursive' instead?
svn: Can't create directory 'lib': File exists

> svn mkdir lib 1>/dev/null 2>&1

# Or:

> svn mkdir lib &>/dev/null

Note that the order you put >/dev/null and 2>&1 in does matter -- you have to redirect FD1 (stdout) to >/dev/null before you redirect FD2 (stderr) to FD1 (stdout)...

> svn mkdir lib 2>&1 >/dev/null
svn: Try 'svn add' or 'svn add --non-recursive' instead?
svn: Can't create directory 'lib': File exists

[edit] Question: Redirection to a file (>) gives "Permission denied", even if you run it as sudo

> svn diff > out
-bash: out: Permission denied

Okay, that makes sense so far. Since . has the following permissions the current user is not root:

drwxr-xr-x 19 root  root    4096 Feb 12 14:49 ./

But why, if I do, sudo would it still not work!?

> sudo svn diff > out
-bash: out: Permission denied

Workaround:

> sudo touch out
> sudo svn diff > out
-bash: out: Permission denied
> sudo chmod a+w out
> sudo svn diff > out
# Works

[edit] How do I output to a log file and to standard output

Obviously, if you redirect standard output to a file, like this:

some_long_running_or_verbose_script > something.log

then you won't see any standard output!

What you want is a sort of splitter, a T fitting that you can put on the pipe so it goes to two destinations. Fortunately, GNU provides just that: tee!

some_long_running_or_verbose_script | tee something.log

Before I learned about tee, I used to do it like this:

some_long_running_or_verbose_script > something.log & ; tail -F something.log

but that's not as elegant or concise. And I wonder if it's possible that it will miss something at the beginning of the log, before tail is started.

tee --append

[edit] How do I redirect the output of the bash keyword 'time'?

Harri Järvi. Redirecting output of the bash keyword time (http://www.cs.tut.fi/~jarvi/tips/bash.html). Retrieved on 2007-05-11 11:18.

In a shell script I wanted to take the execution time of a command and redirect it to a file. First I tried the following:

time command > time.txt

It didn't work. Then I found out that the time command prints its output to stderr. I changed the command to:

time command 2> time.txt

It didn't work either. The time's output was still printed on the console.

It turned out that time is a reserved word in bash. It's not like most of the builtin commands in bash but it's a true part of the command line syntax like 'if' or 'while'.

Bash 3.1.0 manual says the following about pipelines and time reserved word:

...

If the time reserved word precedes a pipeline, the elapsed as well as user and system time consumed by its execution are reported when the pipeline terminates. [...]

Each command in a pipeline is executed as a separate process (i.e., in a subshell).

The builtin help command of bash says the following:

$ help time
time: time [-p] PIPELINE
    Execute PIPELINE and print a summary of the real time, user CPU time,
    and system CPU time spent executing PIPELINE when it terminates.
    The return status is the return status of PIPELINE.  The `-p' option
    prints the timing summary in a slightly different format.  This uses
    the value of the TIMEFORMAT variable as the output format.
times: times
    Print the accumulated user and system times for processes run from
    the shell.

...

Also from the syntax description it becomes clear that the output of the timing is not appended to the stderr of the command, but it's actually output by the shell itself on an upper level. The Time keyword activates a flag and after the whole pipeline command is executed, the timing information is printed to stderr. The time keyword is above the whole command and all pipes and redirections in it. This is why trying naively to redirect the output of time didn't work. It didn't work specifically because how bash syntax is defined: Adding redirection at the end of the command will be interpreted as part of the command to time.

Redirecting the output of the bash time can be achieved by executing the whole command (including the time part) in a subshell as follows:

(time command) 2> time.txt

Launching a subshell is not necessary. Redirecting output of a code block works as well.

{ time ls; } 2> time.txt

This will probably be more efficient that executing the external /usr/bin/time command . Also the bash time command may have some features you need. Or you want to rely on bash time because you don't know if the system installed time utility will have the features you need or not.

In redirection and pipe behaviour this is actually the equivalent of executing the external time command /usr/bin/time .

Suppressing the output from the command itself can be made by redirecting its standard output and standard error to /dev/null

{ time command > /dev/null 2>&1 ; } 2> time.txt

[edit] Useful commands/utilities

[edit] GNU Core Utilities

http://www.gnu.org/software/fileutils/doc/faq/core-utils-faq.html. Retrieved on 2007-05-11 11:18.

Together the three packages combined implement a core set of GNU utilities: [...]

• fileutils
• sh-utils [shellutils]
• textutils.

Has lots of answers to common questions; may be useful for troubleshooting your particular problem!

[edit] textutils

http://www.ibm.com/developerworks/library/l-tiptex1.html. Retrieved on 2007-05-11 11:18.

...

In all, throughout this series, we will get to know cat and tac; head and tail; sort and uniq -- and will discuss the dumping, folding, splitting, indexing, and other capabilities of some of the most common UNIX and Linux text utilities. These are some of the most useful bits of code that you have on your computer; alas, they are often also the least used -- perhaps because man pages can be so hard to follow if you don't already know what you're doing. That is why, rather than be just another copy of man page options, this series will be a guided tour with scenarios that put common commands through their paces so you get a hands-on feel for how and when to use them. Once you have the basics down, you will find the (often arcane) man pages much easier to follow.

...

http://www-128.ibm.com/developerworks/linux/library/l-textutils.html. Retrieved on 2007-05-11 11:18.

Covers: Regular expressions grep fgrep egrep The grep utilities: A real-world example cut cut: Two real-world examples paste join join: A real-world example awk head tail

[edit] head / tail / [middle]

Reading text streams in chunks with head and tail (http://www.ibm.com/developerworks/linux/library/l-tiptex3.html). Retrieved on 2007-05-11 11:18.

[edit] tail

> cat filename | tail 3      # last 3 lines
> cat filename | tail -n+3   # all lines starting at line 3 (inclusive)

tail --help:

If the first character of N (the number of bytes or lines) is a `+',
print beginning with the Nth item from the start of each file, otherwise,
print the last N items in the file.

[edit] [middle?]

I could have sworn I've seen a command like this used somewhere -- a perfect compliment to head and tail --, but now I can't seem to find it...

What was it called? middle? body?

[edit] cut — cut out selected fields of each line of a file

• -c select a range of characters
• -f select fields, used with comma-, tab-, etc.- delimited fields (set delimiter with -d delimiter)

Example usage:

> ls --full-time | cut -c 34-52
2006-07-03 15:26:11
2006-08-24 14:24:17
2006-08-14 22:58:38
2006-08-14 22:58:38
2006-08-24 14:24:17

[edit] echo

echo is pretty much the most basic command available.

/bin/echo

How to avoid printing a newline character

> echo "1234" | wc -c
5
> echo -n "1234" | wc -c
4

[edit] man

[edit] How to fetch a web page / make an HTTP request

[edit] wget

Save as a different filename with -O option.

Don't forget to enclose the URL in if it has parameters (? mark)!

# Not gonna work:
wget -O nt.gif http://www.neuroticweb.com/recursos/css-rounded-box/rounded.php
cn=nt&co=e4ecec&ci=A9B8CF

# Gonna work:
wget -O nt.gif 'http://www.neuroticweb.com/recursos/css-rounded-box/rounded.php
cn=nt&co=e4ecec&ci=A9B8CF'

[edit] curl

[edit] links/lynx: Text-only Web browser

[edit] expr (Math/arithmetic on the command line)

[tyler: ~/svn/code/plugins/database_log4r]> expr 2+3
2+3
[tyler: ~/svn/code/plugins/database_log4r]> expr 2 + 3
5

[tyler: ~/svn/code/plugins/database_log4r]> expr 8 * 3
expr: syntax error
[tyler: ~/svn/code/plugins/database_log4r]> expr 8 \* 3
24

[edit] patch (apply a diff)

See Patches

[edit] watch (repeat a command continuously)

http://old.pupeno.com/blog/poor-man-s-continuous-unit-testing-with-ruby-on-rails/. Retrieved on 2007-02-16 17:14.

watch ls -l big-file-being-copied.txt

Gives output that updates every 2 (adjustable) seconds. Sort of like top except it can be used to watch the output of any command-line program.

Examples:

watch vmstat
watch who

See also: my repeat.rb / repeat.sh

[edit] shopt ("shell options"?)

> shopt
cdable_vars     off
cdspell         off
checkhash       off
checkwinsize    on
cmdhist         on
dotglob         off
execfail        off
expand_aliases  on
extdebug        off
...

Set an option like this: shopt -s expand_aliases

Unset an option like this: shopt -u expand_aliases

I'm not even sure what any of these options are for... but they're there.

[edit]

Leslie P. Polzer (2007-02-15). My sysadmin toolbox (http://www.linux.com/feature/60179). Retrieved on 2007-05-11 11:18.

My working day includes a variety of tasks, and most of them take place on the command line, because that approach enables me to do things in the most efficient way. But you can also waste a lot of time on the command line if you don't know what utilities will give you what you need quickly. Here's an introduction to the most important tools I use every day.

[edit] zsh

The GNU Bourne Again Shell, bash, is the command line interpreter traditionally associated with Linux systems, and most GNU/Linux systems ship it as default. While it has considerably improved in terms of comfort, it stands behind the powerful Z Shell, which you can use as a superset of the Bourne Again Shell.

...

[edit] fmt/par

Outlook Express users send you messages with extra-long line lengths? Need to have a text file formatted with an 80-character boundary? Easy -- use fmt, the traditional Unix paragraph formatter, or a modern version called par :

 fmt -80 report.txt
 par w80 report.txt

I especially like to reformat mail quotes from within Vim (c) by selecting the relevant paragraphs in visual mode (Ctrl-V) and then running the command !par w72 in ex mode. par will preserve any quotation marks and paragraph divisions.

[edit] cat

cat simply concatenates files or standard I/O. In combination with a here document, cat can create entire files from scratch:

cat > /etc/resolv.conf <<EOD search farpoint.local nameserver longbow EOD

Or combine files:

cat part1.txt part2.txt >> full.txt

If everything in Unix really was a file, as it is supposed to be in Plan 9, you could even use it on sockets, thereby using cat to send and receive data through a network connection; but since this isn't the case, you have to resort to its specialized companion netcat.

However, please take care to avoid the "useless uses of cat," lest you win an award for your folly.

[edit] muttng

Why prefer mutt (or its patched cousin muttng) to the many GUI messaging clients out there? For a lot of reasons: painless integration with your favorite text editor (vi and Emacs users will like that), sensible key bindings, high configurability, pretty standard-compliant support for the major mail protocols, and easy usage over network connections among them.

In my experience it's not worth adapting to mutt if you're only getting a few messages every other day, but once your mail volume rises above a certain number, you will be way more effective with mutt.

[edit] dillo

The Dillo project has created an X Window Web browser with a very small footprint. Dillo is an excellent tool with which to quickly view HTML documents from the command line, as it starts up in an instant and has a very clean interface. I like to use it for HTML manuals and to browse TexInfo documentation in HTML format.

[edit] OpenSSH

Designed as a secure replacement for telnet, rologin, and ftp, OpenSSH offers substantially more comfort and features than the tools it replaces. You can create encrypted tunnels with it and forward X windows. Some people even like to use it for mass command execution in cluster environments or large installations [sounds like Capistrano!].

[edit] xbindkeys/actkbd

These two programs are daemons that will execute a specific command when they receive the corresponding mouse or keyboard shortcut.

xbindkeys needs X running and reads input using Xlib. You can associate certain inputs with actions by defining them in rules in the file .xbindkeysrc in your home directory. For example, you could bind your fourth mouse button to start your favorite terminal emulator, and two function keys could be set up to control the volume.

actkbd works both under X and on the console. You could, for example, use it to eject your CD-ROM tray with a keypress both on the console and in X. It uses the evdev interface to receive events, and might therefore also work with exotic input devices (think of buttons on webcams).

For maximum efficiency, use both together.

[edit] vim

Here comes the religious part. What I like about Vim is its speed, its ubiquity (you can find a plain vi editor on every Unix box) and its modal interface, which enables me to modify my files with high speed.

[edit] grep, sed, cut, awk, perl

Here are five omnipresent and flexible tools to work on text. Use grep to filter out (un-)important stuff or search for something, sed to do quick patching or on-the-fly text rewriting, and cut to select columns.

If these don't suffice, throw in awk as a Turing-complete replacement for all three. If you're still not content or have a taste for it, use Perl, which scales well for larger projects, frees you from quoting hassles, and is faster.

I use the GNU versions of sed, cut, and awk, since they offer the most features, but you can find these utilities in a variety of flavors on every Unix system.

[edit] mlterm

Mlterm is my favorite X terminal emulator. Both fast and powerful, it also sports excellent support for non-English scripts. It's less cluttered than Konsole and has a smaller footprint, but it still offers a palette of features such as background fading (change brightness depending on focus), background images, and a GUI configurator. I use it with a black background, white text color, and a font that doesn't hurt my eyes (try ISO10646_UCS4_1_BOLD=-misc-fixed-bold-r-normal--15-140-75-75-c-90-iso10646-1:40;15 or something similar in your ~/.mlterm/font file).

[edit] pgrep/pkill

These two tools let you quickly list (pgrep) and signal (pkill) processes, as you can see here:

pgrep fox # roughly equivalent to 'ps -ax | grep fox' pkill fox # kill all processes matching '*fox*'

Simulating the second example would likely take a combination of ps and awk -- not nice for stuff that you use often! The two utilities come with the procps package.

[edit] Conclusion

Since I need to mix the CLI with GUI applications, like browsers, I need a link to tie those two worlds together -- best with a minimum of mouse usage. The usual WIMP-style interfaces (KDE, GNOME, and most window managers and alternative desktop environments) cannot do this right, so I'm using a tiling window manager (currently Ion3. Throw in a browser with configurable shortcuts, and you'll hardly ever have to reach for the mouse again!

[edit] man: What does the number in parentheses mean? CRONTAB(5) ?

I guess they're section numbers

You can specify sections like this:

usage: man [section] name
man 5 crontab

[edit] wget

[edit] Warning about & characters

Many URLs contain & characters. That's fine as far as your web browser is concerned, but on the command line, & takes on a special meaning: it causes the command before the & to be started and backgrounded and then the part after the & (which in the case of a URL, wouldn't be a valid command but it would be treated as a command nonetheless) to be started and foregrounded.

> echo 1 & echo 2
[2] 11524
2
1
[2]-  Done                    echo 1

> wget http://sourceforge.net/project/downloading.php?groupname=wikipedia&filename=mediawiki-1.6.5.tar.gz&use_mirror=easynews
[2] 11320
[3] 11321
...
[2]   Exit 1                  wget http://sourceforge.net/project/downloading.php?groupname=wikipedia
[3]-  Done                    filename=mediawiki-1.6.5.tar.gz

To get around this, simply and enclose the URL in quotes:

> wget "http://sourceforge.net/project/downloading.php?groupname=wikipedia&filename=mediawiki-1.6.5.tar.gz&use_mirror=easynews"

[edit] Downloading from sourceforge

Use the direct link it gives you ("Your download should begin shortly. If you are experiencing problems with the download please use this direct link.").

So

http://easynews.dl.sourceforge.net/sourceforge/wikipedia/mediawiki-1.6.5.tar.gz

instead of

http://sourceforge.net/project/downloading.php?groupname=wikipedia&filename=mediawiki-1.6.5.tar.gz&use_mirror=easynews

[edit] How do I get the current date/time in ISO8601 format?

> date -Idate
2006-08-21
> date --iso-8601=date
2006-08-21

> date -Iminutes
2006-08-21T11:15-0700

Can I get rid of the annoying -0700 timezone indicator?

Not with the --iso-8601 option. But you can specify custom formats...

> date +%Y%m%dT%H%M%S
20060821T112047
> date +%Y%m%dT%H%M
20060821T1120

[edit] How do I do some operation on the most recently update file (which may change frequently) in the working directory?

ls -t | head -n 1 | vimopen

This is faster than doing ls -t and then typing out whichever filename is listed first...

[edit] How do I get the output to be on one line instead of many?

[edit] The old variable=`...`; echo $variable trick

# Either one of these:
vimsub "`(output=\`cgrep Smith . il\` ; echo $output;)`" smith '"my_name"'
vimsub "`echo \`cgrep Smith . il\``" smith '"my_name"'

# Has this effect: in all files containing the word Smith (case insensitive), replace the word Smith with "my_name"

[edit] The old echo `...` trick

$ echo `cgrep Smith . il`
./faq.html ./contact.html ./index.html

# But:
$ cgrep Smith . il
./faq.html
./contact.html
./index.html

[edit] The old "... | cat -" trick

ls may list things in columns

To force it to list just one file name per line (1 column), you can do this:

ls | cat -
or
ls | grep .
(some kind of text filter that doesn't alter anything)

[edit] How do I combine/merge/concatenate the output streams of two commands into one so that I can pipe it (as a single stream) to another command?

A common use for this technique would be if you need to prefix the output from command A with some literal string before piping it on to command B.

You want: {prefix string literal} + {output from command A} | {command B}

The easiest way that I know of to produce a string literal on standard out is with the echo command:

echo "hello there"

or

echo -n "hello there without a trailing newline"

Since echo is a command, though, I may as well generalize to find a way to combine the output of any two commands...

cat is no help to us; it can concatenate two files, but that doesn't help us concatenate two "input streams".

cat - is no help either: it simply takes the single input stream it receives and passes it along to its standard output stream.

Currently the best solution I've found is this, combining echo and ``, but I hope someone will show me a better way:

$ echo `echo -n 'prefix '``echo -e "main\noutput\nstream"` | cat -
prefix main output stream

Here's another (this one a real-world) example:

echo -n 'String '`xclip -o` | xmacroplay :0 >/dev/null 2>/dev/null

The main problem with that solution is that it strips out all newline characters (\n) and puts the entire output stream on a single line!

This is fine sometimes, but some of the time you will have multiple lines of input that you will want to stay intact!

...

Found it! #Grouping commands!

> { echo -n 'prefix '; echo -e "main\noutput\nstream"; } | cat -
prefix main
output
stream

[edit] Grouping commands

http://www.network-theory.co.uk/docs/bashref/CommandGrouping.html. Retrieved on 2007-05-11 11:18.

{ list; }

Placing a list of commands between curly braces causes the list to be executed in the current shell context. No subshell is created. The semicolon (or newline) following list is required.

http://bash-hackers.org/wiki/doku.php?id=syntax:ccmd:grouping_plain. Retrieved on 2007-05-11 11:18.

The input and output file descriptors are cumulative:

{
  echo "PASSWD follows"
  cat /etc/passwd
  echo
  echo "GROUPS follows"
  cat /etc/group
} >output.txt

This compound command also usually is the body of a function definition, though not the only compound command that’s valid there:

print_help() {
  echo "Options:"
  echo "-h           This help text"
  echo "-f FILE      Use config file FILE"
  echo "-u USER      Run as user USER"
}

[edit] Full-screen applications: When you close them, why does that screen stay on the screen with some terminals but for others not

Examples of what I call a "class A" full-screen app:

• man
• vim
• aptitude

Here's "class B" full-screen apps, which seem to behave a little bit differently:

• top

I've seen one of two behaviors:

Screen restores

When you close a class-A full-screen app, the screen is restored to exactly how it was before you started the full-screen app. No output whatsoever from the class-A app is visible anymore (except with aptitude: after installing some packages, that output apparently outputs straight to standard output, and that output is residual after exiting aptitude (the actual GUI part of aptitude, however, is not visible after exiting)).

Screen stays

When you close a class-A full-screen app, everything that was on the screen when you closed that full-screen app (the "last full-screen") stays visible, but is shifted upward one line to make room for a command prompt at the bottom of the screen. This is very useful if, for example, you open a man page for reference, and then want to suspend it temporarily in order to try out something that you just learned; since the last full-screen is still visible, you can refer back to it as you continue to type on the command line!. This is my preferred behavior!

When I say "close", I mean either exit/quit or simply suspend with Ctrl+Z; both seem to have the identical effect in my tests.

[To do: put in some screenshots that show each of these two behaviors]

Where do I see each of these behaviors:

• Ubuntu 7.10 + GNOME + {gnome-terminal, xterm, etc.}: Screen restores for class-A apps (for class-B apps, it stays)
• Windows using PuTTY: Screen stays

How do I fix this in Ubuntu??

[edit] How does time work?

[Under the hood (category)]

It's almost like it runs a separate process or something...

[edit] Bash-specific

Bash edit

> history 10
 1377  hg color
 1378  hg unroller
 1379  vim ~/.bashrc
 1380  . ~/.bashrc
 1381  h
 1382  hg unroller
 1383  history --help
 1384  history -d 1293
 1385  man history
 1386  history 10

alias hist='history'    # Slightly shorter to type.
alias his='history'
alias hi='history'
alias h='history 50'    # Show about a page worth of history. This is what I'll want to do most of the time.
alias hgrep='history|grep ' # List all command lines containing the search term.
alias hg='history|grep '

   Event Designators
       An  event designator is a reference to a command line entry in the his-
       tory list.

       !      Start a history substitution, except when followed by  a  blank,
              newline, = or (.
       !n     Refer to command line n.
       !-n    Refer to the current command line minus n.
       !!     Refer to the previous command.  This is a synonym for â!-1â.
       !string
              Refer to the most recent command starting with string.
       !?string[?]
              Refer  to the most recent command containing string.  The trail-
              ing ? may be omitted if string is followed immediately by a new-
              line.
       ^string1^string2^
              Quick  substitution.  Repeat the last command, replacing string1
              with string2.  Equivalent to ââ!!:s/string1/string2/ââ (see Mod-
              ifiers below).
       !#     The entire command line typed so far.

!?svn export

!?svn export?:s/home/moo/

!?generate contr
./script/generate controller user

action="svn ci -m 'My commit message'"
cd /home/services/httpd/shared/include/
$action

svn: Commit failed (details follow):
svn: '/home/services/httpd/shared/include/commit' is not under version control

cp /some/long/path /some/other/long/path
vim $_

cp /some/long/path /some/other/long/path
vim /some/other/long/path

cp a b c d e f /some/long/path/to/dest/folder/
cd $_

cp a b c d e f /some/long/path/to/dest/folder/
cd $_

> cp dev/shell/config/irbrc dev/public/ruby/
  cp: cannot create regular file `dev/public/ruby/irbrc': No such file or directory
> mkdir $_
<up><enter>(try again)

echo "All the command-line parameters are: "$*""
echo "All the command-line parameters are: "$@""

cd `pwd`/`dirname $0`/

DIR=`ls -l \`pwd\`/\`dirname $0\`/ | grep "\`basename $0\` -> " | sed -e 's/^.* -> \(.*\)$/\1/' | xargs dirname`

Doesn't work:
$ ./script/server & ; tail -f log/development.log &
-bash: syntax error near unexpected token `;'

Works:
$ ./script/server & tail -f log/development.log &

> sudo su - tyler
~/.bash_profile
~/.bashrc
~/.bashrc

> sudo su tyler
~/.bashrc