Designing command-line interfaces
antoarts.comHere's another one:
Provide explicit flags for default behavior.
For example, if your lines-of-code counting utility excludes preprocessor directives by default, and includes them when you pass "-i", provide an "-x" switch that signals to use the default behavior. This way, when someone wants to write a bash script that uses your utility, they can do this:
case $include_directives in
y)
LOC_FLAGS='-i';;
n)
LOC_FLAGS='-x';;
esac
myloc $LOC_FLAGS
1.) It's explicit, and thus more obvious and clearer.
2.) It's more consistent, so there's no risk of empty/unset variables, whitespace, or other edge conditions screwing up a delicate munging operation.
3.) It's change-tolerant, so if future versions of your utility change the default behavior, scripts will continue functioning as expected.
Other benefit: if you aliased myownloc="myloc -i --complicated_option", you can run myownloc -x to selectively disable the -i switch.
What about the disadvantages?
- It doubles the number of flags of the program.
- It allows invalid or ambiguous combinations: is `myloc -i -x` valid? If yes which flag takes precedence?
It seems these must outweigh the benefits or it wouldn't be so uncommon.
Don't call it -i and -x, call it -i and --no-i or better, -i, --include and --no-include. Some getopts can provide this automatically.
Last flag wins is a common choice because it allows some script to contain yourcommand --include $@ and then you can override with --no-include.
I've seen two conventions for conflicting flags: complain and bail out, or "last one wins" so in your example, -x.
"- It doubles the number of flags of the program."
I don't agree with this. One of the advantages of command line programs is that you can have a large amount of possible arguments, but it doesn't really matter for the user, as long as there is some way to give the most common (like --help and --long-help, or listing the important options in the man page, then have a full list) . This is where GUI's fall down, you must process all options to get to ones you want; where as command line programs can have just the options you want set mentioned. (have you seen mplayer's man page lately?)
As well as having negating arguments, you also want long versions. This is so, when used in scripts, the arguments passed to a program make some kind of sense.
With your precedence argument, just look at some of the standard tools. For rm, -f will override -i. I want this behaviour, personally. It seems 'obvious' that it should be that way.
It doesn't necessarily double the number of flags, it doubles the number of boolean flags. And if you use a convention like joeyh mentioned then the length of your help text won't double, nor will the cognitive burden on users who have to remember the options.
A disadvantage of this system is that --no-x might not always make sense or read well so if you make it consistent it may be weird. On the other hand if you use --no-this and --without-that then you are increasing the documentation and cognitive burden on yourself and your users. For most geeks this is admittedly a pretty minor disadvantage but it's the only one I have off the top of my head.
Oh, and another:
Use your language's command-line option processing libraries.
OptionParser in ruby and argparse in python. There is no reason to eschew these libraries: they're part of the standard library, they require zero coupling to your app logic, and they handle all of the edge cases for free.
"But I can just shift the arguments", you say! Yeah? Great! What if the user pipes input through STDOUT? What if the user passes a flag, a required argument, and then another flag? Will your script read the last flag correctly, or has the naive logic already entered "required argument processing mode"? Just use the library. It's a solved problem!
Just use the library. It's a solved problem!
Indeed! In fact it is so solved that the Python standard library has solved it differently 3-4 (I've lost count) times already, let alone the dozen or two extra packages at PyPI ;)
Yeah most of my scripts use OptParse and I'm procrastinating the rewrite for newer versions of python. I think I may just use getopt, which isn't going anywhere, and roll my own extensions if I need them.
I choose not to discuss implementation in that article, but you should never reinvent the wheel (poorly).
I think a lot of people don't even realize such libraries are out there, or don't understand that there's a wheel being reinvented. ("It's just command line options. Why would you need a whole library to do that?")
Some of the wheels are also very big and crufty--- I would only link GNU Getopt to a C program if it were a Real Program, either with complex option processing or need for industrial-strength polish. For simple one-offs it's a lot easier to have 3 lines of strcmp(), and Getopt feels like overkill.
And if you use C++ and are willing to use Boost, program_options: http://www.boost.org/doc/libs/1_47_0/doc/html/program_option...
I recognize that many people are not willing to use C++, and among those that are, many are still unwilling to use Boost, but I find the program_options library to be great. An example use that I think is reasonable and a big win over doing it myself: https://github.com/scotts/cellgen/blob/master/src/main.cpp#L...
> many are still* unwilling to use Boost*
I can appreciate the subtlety of wording, but in reality it is more of
> many will never be touching Boost even with a long pole and for a large sum of money
:)
Yeah, I was thinking of the people who are violently opposed to it. And, man, I have to admit I really don't get that attitude. There are some well-designed libraries in there, and you can pick-and-choose which libraries you want to use.
I respect people's decision, I guess I just don't understand it.
When I have to compile your program which uses that oh-so-useful Boost library, I curse you to the skies.
I haven't used C++ since college so I don't understand why this would be an issue.
If you don't have an easy way of installing boost, it becomes complicated - just to compile one program.
That said, boost is a dependency that pays off in terms of programmer time if what you're doing is complicated enough (in other words, if you think you might reimplement some part of boost, you're better off using it as a dependency instead.)
Boost is installed standard with most Linux distributions now.
What's the deal with not liking Boost? I've used several of its components off and on over the years.
I'm working on a CLI right now that's rather "complex" in Ruby and i'm seeing the other extreme: CLI frameworks and DSLs. I don't know about other languages but in Ruby your library options range from the minimal (DIY with OptsParser, Trollop) to the gigantic (Thor, Rubicon) and everything in between...
For Java: JCommander http://jcommander.org/
This might not be shared by others, but here it goes.
If your CLI program is a file-format conversion utility, please include a way to dump meta-data, header formats, etc. Don't just silently convert from A to B, allow me to get at the info you have gathered from the input.
For example, a binary disassembler SHOULD dump the executable header format. An spreadsheet converter utility SHOULD display how many sheets there are, if there are macros, how many rows, etc.
One of my pet peeves is pdftotext, a very nifty utility that I use to convert PDF reports to ASCII for subsequent awking. pdftotext has an option to specify start and end pages to extract, but it doesn't have an -i or --info option that tells me how many pages a PDF file has. So, my scripts have a very high upper-limit, like 1000, and it converts the file page by page, until the output text page has a size of zero.
Which reminds me, I should probably fork the fucker this weekend, now that I have some free time.
"[...] but it doesn't have an -i or --info option that tells me how many pages a PDF file has"
You could use pdfinfo - in Debian / Ubuntu, it is in the same package as pdftotext (poppler-utils).
To extract (only) the number of pages of a PDF:
$ pdfinfo FILE.pdf | grep '^Pages' | tr -s ' ' | cut -d ' ' -f 2Thank you. Now I don't have to hate people who leave blank pages without the customary "This page intentionally left blank".
One thing that always infuriates me:
If I go to --help or the man page for your command, and don't see a real example of how to use it immediately, you've failed me as a user.
Seeing your syntax tree and a list of every option and its description doesn't help me when I'm first trying to use your program. I just want to see one or two quick examples of real commands with a short sentence explaining each. After that I'll dive into the mess that is the dozens of flags and inputs to decipher exactly what I want.
It always struck me as odd that man pages tend to have the EXAMPLES section as the last section of the document. I really would like it the other way around but this kinda fits with a bottom-up approach to learning. Look at the small details then get the full picture and how it's supposed to be used.
I've seen a few discussions of this around.
I agree with you, but there are some who believe that examples do not belong in manages (not sure what their rationale is).
In my experience, it's from some GNU folk who like their god-forsaken abomination that is info docs.
man docs have a specific format, a followed by b followed by c, etc. Examples are towards the end, near bugs and author.
It may not be how you want it, but at least it's consistent. shift-G will take you straight to the end, with hopefully some examples.
Do you really want to do this (y/n)?
Do you really want to do this (y/N)?
I absolutely agree - and not only does the article barley scratch what is Unix standard anyways, and the more tricky questions such as piping, exit and error states, or the environment are left out. Kind of disappointing, in my opinion.
I'm also used to seeing Do you want to do this? [n]:
One more:
Keep your "usage" blurb succinct and clear.
Don't clobber your users' terminals with two pages of output when they're not expecting it. If "yourapp", "yourapp -h" or "yourapp --invalid-flag" results in two screenfuls of information containing your app's license, installation instructions, contribution notes, an exhaustive listing of every single one of the 100 available subcommands, and a verbose representation of a configuration setting that 75% of your users won't care about, you're doing it wrong. (I'm looking at you, rvm).
This is similar to the idea of "You don't really understand something unless you can explain it so your grandmother gets it." Your app doesn't really have a sensible interface unless the top layer of its abstraction, or the most common commands, can be summarized in less than scrillions of lines of text. If you simply can't trim it down far enough, it's because you're breaking from the Unix philosophy and your utility is doing too much.
Also tedious:
It's succinct but that second line is almost completely useless.$ ls -Q ls: illegal option -- Q usage: ls -aAbBcCdeEfFghHiklLmnopqrRsStuUwxvV1@/[c | v]%[atime | crtime | ctime | mtime | all] [files]One of my favorites is tar:
Oh, right! `-Acdtrux'! How could I have forgotten. I deal with ACD Trucks all the time. (?!?!)$ tar -m foo tar: You must specify one of the `-Acdtrux' options Try `tar --help' or `tar --usage' for more information.
Speaking of "usage" blurbs, for $DIETY's sake, please assume 80-character wide terminals.
Also I'm looking at you, rails.
The name should be short. A long name will be tedious to type, so don’t call you version control program my-awesome-version-control-program. Call it something short, such as avc (Awesome Version Control).
$ my-aw<TAB> # yay!
Also, please don't make me hit the - character when I call your program.$ may-aw<TAB> my-awe-inspiring-tool my-awwwww-how-cute-program my-awesome-version-control-prog my-aww-yeah-tool my-awchoo-i-sneezed-program my-aw-long-your-names-are-grandmaI prefer - over _ over smashingwordstogether. There are the benefits of readability and not needing to intermittently press the shift key. Also, you might prefer
or to assign that to a different key. This allows you to quickly cycle through valid options so you are not playing "what's the next letter?" games.bind 'tab:menu-complete'
"GUIs also have the advantage when it comes to presenting and editing information that is by nature graphical. This includes photo manipulation and watching movies (I have always wanted a program that shows a movie in my terminal by converting it to ASCII art in real-time, that would be sweet)."
This is an interesting discussion - what is the CLI, exactly? My personal PDF viewer is Zathura[1], which is controlled like VIM. On the other hand, programs like Kismet run on a terminal, but have mouse controlled menus.
Personally, I feel that Zathura is a CLI application, even though it depends on X, because the interaction is done in a keyboard driven way with no graphical widgets.
I have always wanted a program that shows a movie in my terminal by converting it to ASCII art in real-time, that would be sweet
For the benefit of the original author, mplayer and xine both have ASCII art output modes (mplayer -quiet -vo aa, aaxine).
True. I often use text-based applications because I can usually expect them to have a sensible and efficient interface which does not require the mouse. There are graphical applications which follow the same spirit, but they are pretty well hidden.
(edit) There is a bit of terminology substitution going on in linked article. Command line interface is a shell. That's where one types the commands. Calling command options and arguments an interface may be technically correct, but it is not what is conventionally understood under a term of CLI.
---
Speaking from an experience writing CLIs for configuration-heavy embedded devices, the key design element of a functional CLI is a context-aware TAB expansion. This is what makes a CLI truly convenient for routine use.
For example, if a mysql shell was smarter, it would've been allowing this:
> use p<tab><enter> expands to "use production"
> show c<tab> s<tab><tab><enter> expands to "show columns from secondary"
> select * f<tab> s<tab><tab> ... expands to "select * from secondary ..."
# ip addr <tab> shows "ip addr add"
# <tab> shows "ip addr del"
# <tab> shows "ip addr"
I think the use of "interface" is sensible, if somewhat ambiguous on the surface. In this context, options & arguments are the interface to the utility, not the "CLI", which is the interface to the command line itself.
Context-aware tab completion is very doable in the unix world, actually; it's just a hassle on the part of the utility-writer. Have you ever noticed that if you add a new file "foo/bar/baz.py" to a mercurial repository and type "hg add fo<TAB>", it will autocomplete all the way to "foo/bar/baz.py", without stopping at each directory? This is because hg has overridden the default tab completion provided by bash and defined its own set of possibilities.
I wish it were easier to set up; I think if that were the case, we'd see a lot more utilities with spot-on tab completion.
If you switch to Postgres, psql (the commandline client) supports the features you're asking for. :)
Command-line usability is at least 50% about tab completion.
zsh has an extensible command/expansion system much like what you are asking for, allowing users to implement plugins documenting how to autocomplete commands, options, and data types passed to options (files, yes/no, etc.). It is fantastical.
ESR's "The Art of Unix Programming" has a good section on consistent default of command line options
You might want to fix this typo.
> Maybe it¿s just me, but I prefer to remotely control computers via SSH over VNC
If you're really running "SSH over VNC" then you're doing it wrong. ;)
> I have always wanted a program that shows a movie in my terminal by converting it to ASCII art in real-time, that would be sweet
man mplayer and look for the -vo flag which controls the video output mode/driver. Two common options for video output (-vo) are the 'aa' (ASCII Art) and 'caca' (Color Coded ASCII Art). There is a third, 'bl' ("blinkenlights") but it's hardware dependent.
I agree with most of the article, except the yes/no part. In fact I think it's better to do :
"Do you want to do this (Y/n)?"
Where the most common option is uppercase so I can just hit enter.
The important point here is to at least require that <Enter> key, rather than just using getchar() or whatever else and continuing on unexpectedly.
I'm glad he mentioned the "Silence trumps noise" point. I like to call the idiom "no news is good news". Tell me only when I need to know something (like a failure); "-v" is always there if I need it. It feels uncomfortable to get no output at first, but once you get used to it there's much less to read.
It's rather like the Plan 9 convention of programs returning strings instead of ints when then terminate; an empty string means success, a non-empty string contains the error message. I wish other OSes had adopted that.
There is no silence vs noise problem. Send output to STDOUT and "noise" to STDERR. Use -q and -v when appropriate. I thought it was weird that the author created a false dichotomy between being informational and giving only output that can be easily processed.
Problem with "silence" and "no news" is they are indistinguishable from many failure modes.
I vastly prefer confirmation of action(s) as default and -q flag. At very lest there needs to be -v flag that provides confirmation of action(s)
Configure your shell to report nonzero exit statuses.
In zsh this is done by "setopt print_exit_value". It's a pity shells don't do it by default.joey@gnu:~>true joey@gnu:~>false zsh: exit 1Good hint, but it's value is very niche. Waiting until exit for status is fail. Many commands take seconds if not minutes/hours to complete. I wanna see some indication that they are working and not hung/etc.
Additionally, for interactive CLI programs or suites of programs, consistency is key. Offer the same way to select options, confirm or deny information, input data etc.
I'm also working on a project where by entering "?" at any interactive section you're taken to an interactive help menu. From here you can query (amongst other things) the state of the program, something that isn't always clear when you're using command line software, as you can't have extra info somewhere in the corner or whatever.
One cool feature of the juniper junos CLI is that all the CLI commands have an option for XML output, making parsing of the CLI output in scripts a little more sane.
There was a project mentioned on HN fairly recently (~weeks ago) about an extended pipe system, using JSON and a bunch of common schemas, as I recall.
Both http://acko.net/blog/on-termkit and http://blogs.perl.org/users/rocco_caputo/2011/05/apppipefilt... seem to fit what I was thinking of, but I'm sure there was something else.
Nice ideas, but the inertia of existing unix tools is going to be hard to overcome, and a system isn't much use until all your common tools support it (or at least don't break it).
I've started adding an option to output a complete Galaxy tool config file for some CLI apps.
-- http://wiki.g2.bx.psu.edu/Admin/Tools/Tool%20Config%20Syntax
I talked some folks into using JSON output for a tool, and am now in the midst of fixing both ends so that the output is valid JSON, instead of a newline delimited file with JSON on each line.......
Non-interactive programs get the most attention in this article, while text-based user interfaces are barely covered at all.
That's disappointing, I was hoping I'd learn how vi, etc. worked from this, since I know nothing about writing command line interfaces other than input and output to the last column of the last line of the terminal. Does anyone know of a good article/introduction to this?
Besides the technical details, there are good practices which should be followed, but I don't know of any document which lists them. This is a pity, because though text-based interfaces are often designed in a much more efficient way than GUI interfaces, they are seldom consistent between themselves, and often get something wrong. A few common ones:
- Don't make the user reach for distant keys like escape or pagedown/pageup (also support ^N/^P or ^B/^F) or the arrow keys (also support hjkl), unless you really need to.
- For one-line text entry, support readline bindings (^W, ^U, ^Y, ^B, ^F, etc.)
- If you show a list, provide a way to search for an item rather than moving through the list item by item or page by page.
- Unless there is a good reason not to, spawn $PAGER to show text and $EDITOR to edit text.
- If there is a finite set of actions to choose from, provide one-key hotkeys for each one. Don't require unnecessary use of the control key. Optionally show the list of possible or common actions, but have an option to hide it and save screen space for users who don't need it anymore (like mutt does). Likewise, if there are several items that can get focus, provide hotkeys, don't require the user to Tab their way through all of them.
Obviously, there are more.
> For one-line text entry, support readline bindings (^W, ^U, ^Y, ^B, ^F, etc.)
Better yet, don't try to emulate readline--just use readline itself.
> spawn $PAGER to show text and $EDITOR to edit text.
If you're old school enough, honor $VISUAL and fall back to $EDITOR if it's not set.
linenoise[1] is a fairly small readline alternative which seems to be gaining some traction. I know that one of the more common objections I've seen to using readline in some smallish utility is that it's a pretty big library, so that might help out.
If you want to know how to create programs like vi, you should check out ncurses (http://www.gnu.org/s/ncurses/, tutorial: http://tldp.org/HOWTO/NCURSES-Programming-HOWTO/)
Examples include cd, calling it with no arguments returns you to the home directory.
How did I never know that before?
Make sure you also know about cd - (go to previous directory) and cd foo bar in zsh (go to ${PWD/foo/bar}).
> Silence trumps noise
How about, ec2-server-start -n 4 -t medium -i img-xxxg Which expands to start 4 medium instances using image img-xxxg. Will you prefer a long wait and then silently back on prompt or an indication of something happening?
I agree with what he is saying, but I think there is a hint of unfair generalization here.
> Naming your utility
Again small unix commands are like precious three letter domain names. Not always viable. Also, although most of single letter commands are free, one should avoid to name a CLI binary in single letter, because 1. users often type single letter stuff by mistake. 2. users use single letter aliases.
> Will you prefer a long wait and then silently back on prompt or an indication of something happening?
This is what the -v, --verbose option is for. Without this flag, you should assume everything is operating as expected until you receive an error message or an exit status >0.
The article is interesting up to a point, but what really struck me is that this blog claims to be written by a 15 year old. I think the topics and depth of the other articles on the blog spell great things for this guy's future!
> Multi-letter options start with two hyphens, and each such argument must be separated with spaces.
That's something that's always annoyed me about screen; commands such as "wipe" are "screen -wipe"
there's "find ./ -type d", also (find directories only.)
'find' is an interesting animal: it* really only has a handful of traditional option parameters, and these are of the single-dash-single-letter variety. The single-dash-word parameters are all part of its expression language for selecting filesystem objects.
* checked against /usr/bin/find on OS X 10.6.8 (which seems to lack an option parameter for printing its own version number) and GNU find 4.2.27
'find' is one of the oldest Unix programs still in common use. Its standardized arguments predate modern conventions for command line arguments.
After -h --help, I think the most essential option is -n --dry-run
See the paper by Rob Pike and Brian Kernighan 'Program Design in the UNIX Environment' (aka 'cat -v considered harmful'):