Favorite Unix Commands
clippy.inIt took me a moment to figure out what was going on there (with the whole clippy thing). The bulk of this list is the top commands from commandlinefu which has been on HN before [0], [1].
You can even go meta and install a command to let you search commandlinefu [2].
[0] http://news.ycombinator.com/item?id=527486
I have a suspicion that it's actually a kind of targeted example-advertisement for the clippy thingo ;) and did somewhat work for me actually, I do now feel interested in learning more about the site
On my mobile device, it was nearly unusable...
Maximized on a 1080p monitor it was irritating with far too many internal scroll bars.
Great list of commands though.
Many years ago, before I ever used Linux in any serious fashion, I often used a password that ended with '!!'. One day I was playing around with setting up MySQL and was having a very hard time with something that seemed super simple. I don't remember exactly how I figured it out, I think I accidentally type my password at the wrong time in the command line and observed some odd behavior and, after hours, finally tracked it to the '!!' command. I decided to try a different root password for MySQL and it was smooth sailing from there. A few months later I was transferring a domain away from a small DNS hosting provider. After a week or so of waiting support told me they were having a hard time with my account and couldn't do what they needed to do to initiate the transfer. I don't have the support emails, must have been my old hotmail account, but something they said suggested they might be trying to run commands on my account with my password (I knew they stored it in plain text since it was in several emails from them and I knew they were Linux servers) and it reminded me about the '!!' issue I had with MySQL. I changed my password and they were able to move forward. I wish I had dug deeper into the issues at the time they occurred.
Check fasd, it will blow your mind, be prepared! :) Fasd (pronounced similar to "fast") is a command-line productivity booster.
I've been using this for a while, and trust me, its changed my command line workflow, and I wish, this should come as inbuilt for all POSIX shells!
Here is how it works -
If you use your shell to navigate and launch applications, fasd can help you do it more efficiently. With fasd, you can open files regardless of which directory you are in. Just with a few key strings, fasd can find a "frecent" file or directory and open it with command you specify. Below are some hypothetical situations, where you can type in the command on the left and fasd will "expand" your command into the right side. Pretty magic, huh?
v def conf => vim /some/awkward/path/to/type/default.conf
j abc => cd /hell/of/a/awkward/path/to/get/to/abcdef
m movie => mplayer /whatever/whatever/whatever/awesome_movie.mp4
o eng paper => xdg-open /you/dont/remember/where/english_paper.pdf
vim `f rc lo` => vim /etc/rc.local
vim `f rc conf` => vim /etc/rc.conf
The name fasd comes from the default suggested aliases f(files), a(files/directories), s(show/search/select), d(directories).
Fasd ranks files and directories by "frecency," that is, by both "frequency" and "recency." The term "frecency" was first coined by Mozilla and used in Firefox.
Here is the Link - https://github.com/clvv/fasd
I forgot about the DNS querying of wikipedia. Probably could've used that to entertain myself on my flights over the holidays since I'm pretty sure gogo inflight passes DNS through.
Anyway, I'm a big fan of piping things into xargs. xxd is a terminal-based hex editor. It can convert both to and from hex to binary. I also use 'pgrep -lf 'partial_program_name' a lot in place of 'ps aux | grep partial_program_name'. 'pkill -9 partial_program_name' searches for and kills all processes matching the string.
If they allow DNS out you can use their Internet for free via iodine: http://code.kryo.se/iodine/
I run an iodine server for myself, you can get free Internet at quite a few loginpage WiFi routers with it in airports, cafes etc.
ssssssh ! When proxying over DNS becomes mainstream, it'll be shutdown !!
interesting that there's an android package.
Could it turn your old android device into an HTTP tunnel?
+1 for xxd, great for debugging purposes e.g. analysing serial output from socat (also mentioned by another commentor below).
Handy addition to the "mount | column -t" trick:
You can use this within Vim to pretty-format text tables, initialization of variables, etc. Just highlight the lines in question in visual line mode (V) and type
!column -t<CR>
Ctrl+l is my most used command. I need a clean terminal
Huh. I've always just called "clear".
Same here; and love that it doesn't clear the command line you're currently typing.
Does anyone know if there's a key command for clearing the scroll buffer?
Does anybody know if there is a command for creating an infinite stream of the character "y" + carriage return ?
yes(1).
I second this, I also find myself impulsively smashing enter multiple times sometimes between commands, if I am typing in a series of commands, its just so my brain can catch up with what I want to do.
Nice
Ctrl+D for quick logout too.
Cmd + K on the OS X Terminal.
That's gold. I've been using "clear" this whole time and hated it, particularly since I'm used to "cls" from MS-DOS. So, for me, the way to clear the screen was:
cls <enter> clear<enter>
lol.
I see Cmd + K on the menu now, but I guess I wasn't curious enough to poke around. Thanks a bunch.
didn't know this hotkey existed.. nice
a few nice ones here too, http://www.pixelbeat.org/cmdline.html, http://www.pixelbeat.org/docs/linux_commands.html
always worth looking at these types of lists imo, whilst some of the commands mightn't really fit into your workflow or seem useful immediately, they're often exactly the snippets of information that can save hours at a later stage..
I have this in my .bashrc:
function lc() { if [[ "$#" -gt 1 ]]; then for DIR in "$@"; do echo -n "$DIR - " ; ls -AU1 $DIR | wc -l ; done ; else ls -AU1 "$@" | wc -l ; fi; }
So, "lc /dir" will count the number of files in /dir and "lc /dir/*" will count the files in subdirectories of /dir. This is useful if you're working in an environment where you may have thousands of files in a directory, and a regular "ls -l" will lock your terminal while it eats your entire scrollback buffer.
Here's an idea: always pipe ls throug a pager! If it's less than a page, it would just print the results (you have to set some flags on less that I don't remember). If it's more, then press d or /term-to-search :)
Won't page if less than one screenful. zsh users can also do:less -F
(less than character)< filename
This might be quicker than firing up a shell script:
> lc /dir" will count the number of files in /dir
> and "lc /dir/*" will count the files in subdirectories of /dir.find /dir -type f -maxdepth 1 | wc -lfind /dir -type f -mindepth 2 | wc -l
grep -r . "some random debug message" (searches for the passage recursively in all files from the directory it was executed, the main reason I like developing in linux more than windows)
If you are searching inside code specifically, you may like 'ack'. It does the same thing grep does, except it has preset 'excludes', is recursive by default, and the output is a bit prettier.
Or even 'ag' (the silver searcher). Like ack, but "better". https://github.com/ggreer/the_silver_searcher
"better" or just faster since its written in C ?
It is not a drop-in replacement for ack. e.g. there are some basic usages it does not support -- iirc counts (ag -c pattern).
Eh, faster is faster, does it matter why?
The part about written in C was just added on. My point was "better or faster". If you say "faster is better" I'll have to agree :)
Oh thanks! I will give it a whirl
or (more commonly for me):-grep -rs
(The -s supresses errors about unreadable files that may clutter the output. -i for case insensitivity.)grep -rsiOn windows
findstr /SPINC:"some random debug message" *
Will do the same. FINDSTR is no grep, but for very simple cases like this is just fine
SPINC and SPIMC are my favorite options...
I have never understood the love for `sudo !!`. I think `ctrl+pa sudo ` is faster, explicit, and more versatile.
I disagree. I think typing one more command "sudo !!" is much easier and straight-forward than doing
1. Ctrl+p. Go to the previous line in bash history.
2. Ctrl+a. Go to the beginning of the new line.
3. Type "sudo ". While you're typing your whole commandline shifts around as well.
The latter mixes history commands, navigation commands and also modifies a previous command instead of just typing a new 7-letter command. It's also more in terms of keystrokes (though just barely).
You know writing out their function does not make them longer. ;) I agree with the parent because my left pinky is always on capsLock (my ctrl; I swapped caps with ctrl) so a ctrol ap is far easier than to go to shift + 1 twice(to type !). Again, this can be different in other keyboards/setups.
I just wanted to illustrate the increased complexity and different domains of the commands.
My favorite thing about `sudo !!` is that when you IM it to your friend who's learning Unix they'll take that literally rather than realize they should type it into their terminal, leading to a hilarious teachable moment months later when they actually watch you using the command line.
You know, I never knew ctrl-p - that's a much nicer way of doing it.
Alt Sys Rq o - useful if you need to shut down a system and your kvm is not working.
It's probably a good idea to tErminate, kIll, Sync, and remoUnt-ro before hitting Off or reBoot.
I'm a fan of using cut or sed or awk and ending it with:
| sort | uniq -c | sort -n -r | head
Please comment if you have a favorite not on this list.
If you use tmux (or screen) you want to use byobu!
https://www.google.de/search?q=byobu+tmux&tbm=isch
It's a fancy statusbar that shows CPU speed, IP-address, hostname, ...
You can get a fancy status bar with screen, you just need a screenrc file: http://www.mbeckler.org/blog/2012/11/28/my-screenrc-file-cir...
Another option is tmux-powerline[1]
I have something similar in my .tmux.conf file
Screenshot: http://i.imgur.com/u6JY0.png
Config file: https://github.com/tylerkahn/dotfiles/blob/master/.tmux.conf...
Works with OSX and Linux.
'tac'. It is 'cat', but backwards.
Are you some kind of genius?
If you enjoy 'tac', you might also enjoy 'sponge'.
Spong 'soaks up' input an releases it all on one chunk so that you can do things like:
(`cat foo > foo` does not work)cat foo | sponge fooThat's neat and all, but what if you don't have sponge? Well, just use tac twice!
(Obviously this is rather wasteful ;))cat foo | tac | tac > fooI cannot understand how this works.
"cat foo > foo" breaks, because the shell truncates the file "foo" while it's setting up the redirection, before it launches "cat foo", and so cat has nothing to read.
"cat foo | sponge foo" works because the shell is not responsible for writing to foo; it launches cat (which opens the file for reading) then launches sponge (which eventually opens the file for writing).
I would thus expect "cat foo | tac | tac > foo" to fail in the same way as "cat foo > foo" because the shell is still going to open the file foo and truncate it, but experimentation shows that it does actually work. Is it because the shell launches each pipeline one at a time, so cat has read the file before "tac > foo" truncates it? Is it a race-condition or a corner-case?
That's a good question. I had though it worked because the truncate would not actually happen until tac actually wrote something, but now that I look at it more I don't think that is actually the case. `cat > foo` immediately truncates foo before anything is written. Also `tac foo > foo` doesn't work, which discredits my hypothesis that tac's buffering was causing it.
Furthermore `cat foo | cat > foo` does not work, but `cat foo | cat | cat > foo` does work.
I suspect this behavior is actually a race condition of some sort.
Edit: definetly a race condition, shown by larger files:
% wc -l foo 1310720 foo % cat foo | cat | cat > foo % wc -l foo 16384 1
I always get confused between "tac" and "rev", which is also "cat, but backwards" for a certain definition of "backwards".
When rewriting large amounts of ASCII text (or comments in Code), I really find "par" helpful.
http://www.nicemice.net/par/ (or apt-get install par): Just use it in : : vi with your cursor in front of a badly wrapped block of text. : : Press !}par<enter> and the text will be : : piped to par : : which fixes up : : the : : formatting and keeps aligned frame markers : : intact magically. : : Just use it in vi with your cursor in front of a badly wrapped : : block of text. Press !}par<enter> and the text will be piped : : to par which fixes up the formatting and keeps aligned frame : : markers intact magically. :Or you can just select the block of text and type gq. This will do what par does, and can be adjusted with vim's 'textwidth' and 'formatoptions' settings.
More of a procedure, always list file and edit the command before deleting it:export CDPATH=.:~/projects export PS1="$(uname -n)$ " alias ltr='ls -ltr'
So this is a sanity check to avoid this mistake:$ ll file.txt -rw-r--r-- 1 zera staff 0 Jan 7 20:28 file.txt Ctrl-p, Ctrl-a, Ctrl-d, Ctrl-d, r,m $ rm file.txt
*Edit, one more:$ rm *.txtEnsure you overwrite a file (or aren't):
$ cp -i ~/file.config /etc/
Helps me quickly list only directories. Any better alternatives ?alias lsd="ls -ltrF | grep ^d"Using `find` is nice to get just directories:
$ find . -type d -maxdepth 1
I use:
% ls -d */
ls -al | grep ^d
(Fails if a directory has a linefeed in its name followed by a 'd'. :-)
edit: for the same output as your command you can use:find . -type ddir -lDunno what "better" means, but this also works:
find -maxdepth 1 -type d
seconded. what I have always used
Depends on the shell. In zsh (with EXTENDED_GLOB): *(/) matches only directories.
I usually
alias lsd='ls -d */'ls -l | grep drw
I know, it's a bad habit
echo */ (perfect when your unable to fork)
slalias emacs="nano"
export EDITOR=/usr/bin/nano
nooooooo!
tsort. It's pointless until you need it, but when you need it, it's totally awesome.
less. Overkill, but SO MUCH BETTER THAN tail -f. Seriously. You can toggle "follow mode" without losing your place in the incoming stream.
> less. Overkill, but SO MUCH BETTER THAN tail -f. Seriously.
If you do this on my servers, I am going to find you, and then I am going to kill... um... your process.
less with SHIFT-F takes a whole CPU core to run on a busy log file. tail -f takes almost nothing to run. Now combine that information with the reality of a whole bunch of devs who don't know it or don't care about it. :(
I prefer:-
to:tail --follow=filename
The former will spot inode changes and reopen the filename if the file wraps (or has something else done to it to change its inode). The latter will just sit there seeing no new input in such cases.tail -f filenamemultitail[1] is also really nice for following multiple log files. It can do highlighting of lines based on regex's, and has built-in highlighting of a wide variety of log formats
Does anybody know if there is a good standalone regex highlighter? I usually use `ack --passthru`, but that has always seemed a tad.. overkillish to me.
Hm, can you start at the end of file without reading the entire file with less? Use case is tailing large logs.
Yes, either type G when presented with the top of the file or pass +G on the command line. It will then read all the intervening file to calculate the line numbers, and tells you it's doing this and to interrupt it to stop. To prevent it doing this in the first place use the -n option.
"less -n huge_file" and then G only reads the start and end of the file. Type G again to move to the latest end of the file if it's growing.
Yes. It is CPU intensive (see auntie post above).
dtrace :P
On zsh, "print -rC2" for printing some listing in 2 column format. The number of columns (C) can be specified. (print is a builtin).
zsh's file globbing is awesome. Some simple ones:
*(.) - only files
*(/) - only dirs
*(.m0) - files modified today
*(.om[1,15]) - 15 recently modified filesfaucet 80 --in cat
This is useful for creating a connection for receiving the input of what you get...
e.g. running
faucet 80 --in cat
and later
curl http://127.0.0.1:80
will deliver to you console :
GET / HTTP/1.1 User-Agent: curl/7.21.4 (i686-pc-linux-gnu) libcurl/7.21.4 OpenSSL/0.9.8n zlib/1.2.5 libidn/1.19 Host: 127.0.0.1:900 Accept: /
Just some additional information:
Faucet is part of "netpipes", which is available for most Unixoid operating systems. I guess it was inspiration to build "netcat". But netcat was a absolute terrible hack of a program, so "socat" (socket-cat?) was born.
http://www.dest-unreach.org/socat/
In case you need the functionality of bash's /dev/tcp [1] and you are pondering to install netcat or netpipes, do yourself a favor and just go for socat, it's insanely more powerful, read the manpage, read the examples!
http://www.dest-unreach.org/socat/doc/socat.html#EXAMPLES
--
[1] but debian and ubuntu have decided to compile without it... https://bugs.launchpad.net/ubuntu/+source/bash/+bug/215034
Alternatively you could do
netcat -l -p 80
One of my favorite aliases that wasn't listed.
alias easy_search="curl -s http://pypi.python.org/simple/ | perl -ne 'print if s/<[^>]+>//g && $.>1'|grep"
I like the wikipedia DNS text lookup - I work in a "firewalled" environment where the unices are only allowed DNS queries. If only wikipedia could provide central switchboard numbers in their text fields as well ...
Single user mode on an HPUX machine was, I think, the most useful to me.
(http://www.unixhub.com/docs/hpux/hpux_boot.html)
I wish I still had that machine, but I do not.
I love playing with OSes and HPUX was one of the ones I didn't get to play with (not to mention Irix and AIX).
What a ridiculous fixed header. It's taking up 2/3 of my phone's screen.
Try turning your phone into the portrait orientation.
Here's my list, a very compact plaintext file: http://mmb.pcb.ub.es/~carlesfe/unix/tricks.txt
Not really a unix command, but a vi command that has been really useful, specially if you do lots of editing protected files and want to keep your custom vim configuration:
!sudo tee %
Need to send the contents of the file to that command. Otherwise it effectively empties the file. (typo?)
:%!sudo tee %
What vi mode is that entered in?
Normal mode (:help :! for more information on this command).
What benefit does
have over doing it at the command line other than filling in the current filename for you? I suspect the original poster meant something like "gg!Gsudo tee %".:!sudo tee %:w !sudo tee %
I find this site handy for clever commands: http://www.commandlinefu.com/commands/browse
I like those un- commands for easier archive extracting, e.g.:
alias unbz2='tar xvjf' alias untar='tar xvfz'
I use atool (http://www.nongnu.org/atool/), which is packaged in Debian and Ubuntu (and I am sure in other distributions as well). It supports all the formats I have encountered so far (tar, tar.gz, zip, rar, bz2 and others).
It provides a few commands, two of which I use the most:
The best part is aunpack extracts files into a subdirectory (I hate polluting the current directory with files from a new archive which wasn't properly packaged to include a directory), and it does it only when necessary. Before I found it, I always used to create a temp directory, cd to it, unpack into it, and possibly move files around etc.als - list files in an archive aunpack - exatract files from an archiveFortunately GNU tar has gotten smarter. You can do just "tar xvf" now and it will do the right thing whether the archive is compressed or not.
When creating the archive, you still have to use your favorite compression method (XZ aka LZMA, activated with J is even better, but slightly slower, than bzip).
I always use dtrx (do the right extraction). Check out its features:
mount -t overlayfs -o lowerdir=$HOME,upperdir=/tmp/tmpHOME overlayfs $HOME
What is the purpose of this?
Very useful, if you want to make a checkpoint (possibly for /), break things and then get back to the original state.
If you'd do that for / , your own system would start behaving, like a system booted from LiveCD. And, after reboot, would return to the origanal state.
Overlayfs is pretty cool actually. Allows you to overlay a lower directory with some other upper directory and mount the result somewhere. All the changes that you are making in the mounted directory would be visible to you, and would be stored in the upper directory.
In the example above, after running this command, content of your $HOME would stop changing. But you still will be able to change it, with all the changed content going into tmpHome.
Ah, very clever. I figured it would make changes to $HOME temporary but couldn't immediately think of why you would do that.
Awesome! I didn't know what overlayfs was, but recently saw it while playing with OpenWRT. Thanks for sharing :)
And you can also use it on EC2, to reduce costs of EBS I/O.
Great post! I think "lsof" should be there along with "netstat".
There's a typo: The command is spelled "apropos" (not apropo).
Corrected this typo. Thanks.
I think I might've just saved days of my life....thank you!
Glad it helped you! :)
that's the original
esc-#, comments the current line
ctrl-\, kills current process even xtail
xtail, tails dirs
tweaks in .inputrc to have up and down arrow do backward history search, like in ipython.
sudo !!
Instant favorite
I often alias `sudo` to `fucking` to help vent my frustrations
$ rm -rf somedir rm: somedir/: Permission denied $ fucking !!I have been known to use sudo when it is not appropriate to use the f-word ;)
Along the lines of magic shell variables, there's also '!$', which selects the last argument of the previous command. Often I'm looking at different aspects of the same file:
$ file foo $ ls -lh !$ $ vim !$I don't find this one useful. [up-arrow] automatically inserts the last command, which gives a generic mechanism to correct the command and it requires no more typing than !!: [up-arrow][control-a]sudo [enter]
moving your hand over to arrow keys is why I would use the sudo !! version.
[control-p] then ;)
Remove trash Emacs leaves lying around:
$ find . -name \*~ -delete
alias cd5='pushd .; cd ../../../../..'
This lets me cd five directories at a time, and if I want to go back, I do a "popd"
in ksh set -o vi so I can use the vi editor to navigate the command line history.
grep -r . "Some random debug msg"