Settings

Theme

Shell pipes and in-place insanity

fakedrake.github.io

15 points by fakedrake 11 years ago · 15 comments

Reader

jamesdutc 11 years ago

I think this behaviour is supposed to be a feature of shell. Pipelines are executed in parallel.

Also, see `sponge` from `moreutils` (http://kitenet.net/~joey/code/moreutils/):

    $ man sponge
    ...
    sponge  reads standard input and writes it out to the specified file. 
    Un‐like a shell redirect, sponge soaks up all its input before opening the output file. 
    This allows constructing pipelines that read from and write to the same file.
sikhnerd 11 years ago

This is the expected behavior of pipes[0]. A quick look at Lists [1] clears up how you can control this a little better (+Job Control). jamesdutc also linked to sponge, which may suit your usecase as well.

  [0] https://www.gnu.org/software/bash/manual/bashref.html#Pipelines
  [1] https://www.gnu.org/software/bash/manual/bashref.html#Lists
mtdewcmu 11 years ago

That was a contrived example.

The shell sets up the pipe and file redirections for each process in the pipeline, then lets them go. It has no control over when the child processes read or write. It could be a race condition, but it's written in such a way that it appears that the file will get truncated before it can be read virtually 100% of the time.

  • fakedrakeOP 11 years ago

    You are right I should have clarify that. thank you

    • mtdewcmu 11 years ago

      I'm not sure if it's actually a race; it probably depends on how bash is implemented. But this prints what is expected:

      echo "1\n2\n3\n4\n" > numbers && cat numbers | { sleep 1 && cat > numbers && cat numbers ; }

      That's not exactly the same (because the redirection for cat #2 now happens in a subshell), but it shows that this is basically racy. :)

dredmorbius 11 years ago

NB: If you're going to use CSS styling for code examples, choose foreground and background colors, as well as text sizes, which are actually legible.

Monospace fonts, black on white, at default sizes, are your best bets.

  • simoncion 11 years ago

    Counteranecdatum for the author: I find your choice of foreground and background colors, syntax highlighting colors, font style and size, and line length inside your code blocks to be quite legible and pleasant to read.

    Please make these sorts of choices again in the future!

  • LukeShu 11 years ago

    Um... I see almost-black Courier New on off-white, at 13px and a slightly increased line-height. That seems fine to me.

Keyboard Shortcuts

j
Next item
k
Previous item
o / Enter
Open selected item
?
Show this help
Esc
Close modal / clear selection