HTTP/1.1 pipelining example: retrieving 100s of small files

2 points by textmode 4 years ago · 0 comments · 1 min read

A simple demonstration of HTTP/1.1 pipelining with netcat.

Retrieving 100s of small JPEG files with a single TCP connection. Here, 2 TCP connections for 141 files.

Splitting the output into image files and making a PDF from the images.

yy025 can be found here:

https://news.ycombinator.com/item?id=27178484

https://news.ycombinator.com/item?id=27180125

yy056 can be found here:

https://news.ycombinator.com/item?id=27769305

    # std utils: split, seq, od, tr, sed, flex
    # common utils: nc, xxd, imagemagick
    # simple utils made with flex: yy025, yy056

    (export Connection=keep-alive
    x1=http://www.minimizedistraction.com/img/vrg_google_doc_final_vrs03-
    x2(){ seq -f "$x1%g.jpg" $1 $2;};
    x3(){ yy025|nc -vvn 173.236.175.199 80;};
    x2   1 100|x3;
    x2 101 200|x3;
    )|exec yy056|exec od -An -tx1 -vw99999|exec tr -d '\40'|exec sed 's/ffd9ffd8/ffd9\
    ffd8/g'|exec sed -n /ffd8/p|exec split -l1;
    for x in x??;do xxd -p -r < $x > $x.jpg;rm $x;done;
    convert x??.jpg 1.pdf 2>/dev/null;rm x??.jpg

    firefox ./1.pdf

No comments yet.