Show HN: A Swiss army knife for testing HTTP from the terminal
Hiya HN!
Just released Artillery Probe - a Swiss army knife for testing HTTP from the CLI. Think mini-curl with better UX for common use-cases, plus a couple of extra features.
Would love for you to try it and give some feedback!
https://www.artillery.io/blog/swiss-army-knife-for-http-testing
and:
npm install artillery@latest
- First of all, it's a HTTP client! It does all the usual stuff you'd expect from a HTTP client... HTTP methods, request bodies, custom headers, forms, Basic Auth etc.
- Got some JSON or XML back? It'll pretty-print it, and syntax highlight it for you.
- It'll show you request waterfalls like this one: (inspired by httpstat)
DNS Lookup | TCP Connection | SSL Handshake | Time to First Byte | Content Transfer
56ms | 14ms | 19ms | 181ms | 88ms |
| | | | |
56ms | | | |
70ms | | |
89ms | |
270ms |
total:358ms
- You can set expectations on the response, e.g. have the CLI check that the response is a 200, or that a certain header is set, and exit with non-zero code if not. Super handy for quick acceptance testing.
We've got lots of ideas for improvements, but would love to hear what you think! Nice idea. NodeJS and thousands of NPM dependencies are a big no-go with a red flag on top, though. One of the things something like Go probably would have been a better choice for. See "bombardier" for example. Bombardier is cool, but serves a very different use case. I mentioned elsewhere in the comments that we have a Docker image, and are working on other methods of installing the CLI to alleviate some of these dependency-related concerns. Getting side tracked here, but there seems to be a common sentiment when it comes to Node.js that it's uniquely insecure. Node.js has indeed had some unfortunate press when it comes to supply-chain security, but every other runtime is susceptible to those attacks (PiPy, Gems, Maven, Rust Crates). Ultimately of course, if you choose to avoid using any software built on top of those stacks, that's your choice. Artillery specifically is no different to any other Node.js-based project in how large the dependency tree is. VSCode for instance is used by millions of developers has 1.6k dependencies [1]. > Bombardier is cool, but serves a very different use case. Yes, of course. I was just taking an arbitrary example from my common HTTP toolbox. > VSCode for instance is used by millions of developers has 1.6k dependencies [1]. VSCode is maintained by Microsoft plus a huge community and they are transparent about their process in regards of auditing/freezing dependencies. Nice project. How are the request waterfalls done? thank you! we just grab the timings info for the request from the underlying HTTP library, and sprinkle some ASCII art on top. That part was inspired by httpstat [1] We want to extend those with support for Server-Timing next, and also Core Web Vitals [3] (via Playwright) for web pages. 1. https://github.com/reorx/httpstat 2. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Se... Other than pretty printing the output, what does this offer that curl doesn't already do? A few things :) Syntax highlighting Request performance waterfalls Built-in JMESPath queries for JSON (same syntax as in AWS CLI). XML and HTML can be queried too. Ability to set checks on the response. This one is super handy for quick acceptance tests. does this require Node.JS? edit: it looks like you need to install 1276 npm dependencies to use this tool https://github.com/artilleryio/artillery/network/dependencie... i'll pass, the risk is unacceptable for value it offers, especially in corporate environments > Think mini-curl with better UX for common use-cases, plus a couple of extra features. > it looks like you need to install 1276 npm dependencies to use this tool Sincere question (probably highlights my lack of understanding Node package management): how can this be a "mini" curl with that many dependencies? “mini” in the sense that it doesn’t do everything that curl does. curl does a whole lot. this tool focuses on more common use cases and makes them friendlier. plus with request waterfalls and assertions it does things curl can’t do. the number of packages is a bit of a misnomer anyway. Artillery Probe is part of Artillery which does load testing, with multiple protocols, support for multi-step scenarios, publishing to a variety of monitoring systems (Datadog, Prometheus etc) and more. That’s what most of those packages enable. I see. Thanks for the info and thanks for creating/sharing a great tool! 1276 deps... how is that even possible for a simple HTTP fetcher. Does node have no standard library? actually amazing. This is essentially what I was referring to with my comment below. If you can improve the installation experience such that I don't have to care that you developed with node.js, so much the better. If I were a node.js developer I surely wouldn't want my global package name space polluted like that. npm is not for end users. the solution is not write CLIs in JavaScript, unless you have a JS-library that you also want to make available as a CLI packaging Node in a binary won't solve the problem of JavaScript's plagued ecosystem and in fact i'd rather prefer to be able to mitigate the issues/update packages myself rather than waiting for a new binary to release I agree with your conclusion to not write cli tools in Javascript. However, packaging node in a binary is pretty much what electron does today, albeit with a GUI. Regardless, it's not a problem I wish to spend any effort to solve since it's not my problem and I've (we've) already dismissed the premise (writing CLI tools in js) lol. Another option is using Deno! sure, that’s a valid concern in some environments. fwiw we use Snyk.io for dependency scanning no, this is a valid concern in all environments, but in some environments this is more of a concern > Snyk.io for dependency scanning so what? does the CLI update itself automatically? Yes, those dependency trees can be large. Yes, supply chain attacks are a real threat. But Node isn’t that different than Python or Ruby in that regard. How far down the stack do you personally choose to go? I trust you’re familiar with that famous paper published by a certain mr Thompson in the mid-80s? The world is a big place. There’s a lot of software out there written in Node.js, used happily and productively by millions of developers, many of them in corporate environments. Given the opinions you expressed elsewhere in the thread here I think it’s clear that this tool is not for you. I hope no one is forcing you to use it. honest question to the authors: why cannot this tool not be written in JavaScript? what technically prevents you from designing the tool independent of the Node.JS and JavaScript ecosystem? Maybe it's what they know. Common advice for startups is to not get mired in language/framework debates: use what you already know. > npm install Sorry, lost me at this part. Have a better installation story like a single executable. Beyond that, what does this do that curl, wget, et al do not? How is your interface better than those? Is it robust in the face of errors? What http protocol versions does it support? > Have a better installation story like a single executable. Holy guacamole this is an entitled sentiment. It’s an open source project and it’s free. Feedback was explicitly requested. That was my feedback on installation. It's not entitled and has nothing to do with the project being free or open source. Where'd that come from? Probably from the way it was worded. if you’re on a Mac, Artillery has a Homebrew formula too. We have an official Docker image too. Other ways of installing are on the roadmap too (self contained tarballs and binaries). As to what it does better than curl/wget. Probe is geared towards interactive use with HTTP APIs. So you get syntax highlighting and pretty printing for JSON, built-in querying, request waterfall visualizations, and ability to set expectations on every response. HTTP/2 is supported. https://www.artillery.io/blog/swiss-army-knife-for-http-test... Does artillery properly escape the environment variables into json strings? I don't see how it could if you're using bash expansion in double quotes. This feature would only work with string values without spaces or special characters. Don't really see any improvement here. https://www.artillery.io/blog/swiss-army-knife-for-http-test... So by embedding a json query evaluator in your http client, this means that every time I iterate on my JMESPath query I'm making a new request? Is there an offline mode? What if my Json file is 200MiB? Or if the request was not idempotent? Just saying there's good reasons why json query features are not typically built into one-shot http request tools. However, if your tool was not one-shotand you had an interactive session and cached/stored the entire result and then allow the user to interactively query it, then I could see some better value here. Yeah so the JSON quoting part is something I’m pretty pleased with. We use a YAML parser (JSON is a subset of YAML) to parse those values, which is what allows for double quotes to be omitted. Good point on iterating on the query! We already save the body into a temp file, so we can make Probe be able to run queries on a file. Adding it to the todo list. :) (In my own workflow I use gron a lot for getting an overview of the shape of unfamiliar JSON, super handy tool) As to an interactive shell… yes, 100%. Kicking ideas around something like that as well! > We use a YAML parser (JSON is a subset of YAML) I believe that has more sharp edges than one might suspect Also, I appreciate that every service should strive to respond in 10s, but there wasn't an obvious way to get it to be more patient thanks for trying it out! adding “make timeouts configurable” to the todo list. you’re right on that YAML-as-JSON thing. If everything is quoted as JSON, those type conversions shouldn’t kick in. Otherwise there’s room for surprises - perhaps we can do something to make those cases more obvious. > what does this do that curl, wget, et al do not they don’t get VC-funded by YCombinator, do they? /s What makes this a worse "installation story" than e.g. apt install, pip install, git clone && ./configure && make && make install? It would take me something extraordinary to move away from curl Sure, if you've got muscle memory for all the flags (plus it lets you reason about all the other programs that pipe stuff to curl). I personally find myself using httpie [1] at least as much. I think curlie is the middle ground and the best (IMO) in the space. A nicer CLI for interactive use and can use and output curl flags for scripting. As a maintainer, I feel obligated to mention Hurl [1], a tool for testing HTTP with plain text and curl. It’s a wrapper around libcurl, in a single binary, with syntactic sugar for asserts. You can use it also like curl to give you an HTTP output. [1] https://hurl.dev I haven’t come across Hurl before. It looks cool! The big win with curl is the growing adoption of "copy as curl" in browser dev tools as well as "example of API call in curl" for third party API documentation. I don't think I could move to anything else for that reason alone. curl is amazing! I love curl. This is not meant to replace it completely. Probe just makes those most common use-cases friendlier, like looking at headers, or inspecting JSON retuned by an API with syntax highlighting, or querying without needing to reach for jq etc. In Emacs, I found the restclient plugin quite useful. https://github.com/pashky/restclient.el I imagine other editors have similar plugins. Awesome! This comment made reading this thread worthwhile.
The user may be less surprised if the flag were named "--yaml-as-json" or whatever DX friendly variant one wishes, in order to more clearly mark that string as yaml. The user is actually told about the "JSON or YAML", but only if they try to include some scalar value that isn't legal yaml --json "{username: testuser, password: testpassword, birthdate: 2022-05-10}"
...,"birthdate":"2022-05-10T00:00:00.000Z"}
followed by a ginormous stacktrace and a console.error dump of some event object RequestError: Timeout awaiting 'request' for 10000ms
at ClientRequest.<anonymous> (/home/node/node_modules/got/dist/source/core/index.js:962:65)