Show HN: Usetrace – Virtual users doing your repetitive web testing
usetrace.comGreat start to a great product.
We have an Angular app w/ OAuth flows and connections to outside apps. We started with Protractor running locally and on SauceLabs, but it hasn't seen much love because adding new tests takes time, especially for complicated stuff like oauth flows.
Your product should allow our "Product guys" and QA guy write the smoke tests in much less time and maintain them without having to write code or take up developer time, which is huge in a team of 2 guys who write front-end code and would be maintaining the tests.
The only real issues I have right now are 1. Errors can be hard to track down since there isn't any console output. That's something I miss from SauceLabs which was a huge help. 2. Smaller, but annoying, pasting isn't allowed when creating a test. This is painful for username/password entry. It would also be nice to use some kind of variable to ease this. 3. I'm unsure of pricing. When I'm logged in, I see that I have the free plan, but I see no limits or details about the plan.
Other than that, it was a great feeling to get a basic smoke test, plus jenkins integration, plus flowdock integration within an hour of discovering the app. Again, great job!
Hi, thanks for the awesome feedback!
About pricing, you can test all features for free. If you like it and continue to use it, you can subscribe to one of our plans.
Copy pasting is a feature which we haven't solved yet. It would definitely be a huge productivity boost when making traces.
Console output is in our backlog but currently we focus on more important issues.
Pretty cool. Signed up and figured out how to start using it right away, which is in and of itself a great thing.
Questions:
1. When creating a trace, how do you specify when you're finished?
2. Is there a way to set a single trace to run on all browsers?
3. How do you specify what constitutes a "success" vs. a "fail" for any given trace?
1. Just click "Run Trace" to verify the automation works.
2. Yes, from Dashboard it's currently FF&Chrome only, but via API you can include IEs. I can help you to get started (mailto arto@usetrace.com)
3. If a Trace can be walked through it passes. A Trace includes steps like clicks on buttons and checks (explicit things that must be visible on the page, e.g. text or other elements).
I really like the Visual Studio "load testing" [1] for this purpose. My team uses it. It doesn't just include a trace of paths through the application, it performs important load testing and analysis. Well worth the price.
How different is this from https://www.rainforestqa.com/ ?
I haven't used either extensively, but they look similar in a few ways, yet different. I am curious if anyone with knowledge of the two could elaborate on the differences.
In Usetrace you define modular tests by using your own site. Tests (or traces) are visually represented like cartoon strips. You can follow test execution from a live view in your browser. Tests are executed automatically and in parallel. An average test run completes in 10-20s from request. Error screenshots and reports can be emailed or send to your team chat e.g. HipChat.
In Rainforest you define tests by writing instructions for human testers. Reports only tell you if a test was successful or which steps succeeded / failed. It takes close to half an hour for one two step test to complete in five browsers.
Cheers, Eero CEO of Usetrace http://twitter.com/EeroHeroHeino
For me, I'm more interested in how this differs from Selenium. Is it just making this sort of testing process easier?
We've build on top of selenium. Traces are modular in a similar fashion than programming parametrized modules and they can be updated separately - Mixing the benefits of programmable tests with the easiness of record&playback.
It's more of a pain in the ass but I can write my own tests with Selenium in Cucumber or RSPEC or what have you, and I can use all kinds of wrappers for it(Watir, TelluriumDriver, etc). I understand the ease of just doing a "trace" graphical test instead of writing the code, but why would I pay you 90$/month for the privilege? Not only that, I can get much more granular with my RSPEC-Selenium tests. Oh yes, I can also re-use methods that encapsulate sequences of decisions like sign_up_for_website instead of having to resign up every time I make a new trace.
I can go through a sign up flow and then grab the last user with Ruby and do actual checks on it's methods and members, and then go back to browser navigation. This seems like a less free way to do fewer checks with a prettier interface.
We're currently providing a way to reuse the beginning of trace in other traces. Most traces start with logging in to the app, so you can create the login trace, and share it to all other traces. So you end up defining signing up once.
I'm surprised a GUI for selenium test creation hasn't been done sooner. It looks like you guys have done a good job, but the feature I would need to see before using this seriously is a way to get the selenium code output and fine tune it. That way most tests could be created by less technical people but if there are issues an engineer could step in and fix it.
You can add coded steps when you need advanced features. You'll have javascript in your site's context available.
When you can, please open source as much as you can back to the Selenium project -- especially if you fix and find bugs! :-)
Very nice, dashboard and test working as expected, intuitive etc. Only comment is that from the front page I wasn't sure what I was signing up for (saw no demo)
Thanks! We're going to improve the landingpage. Meanwhile, check out this video: https://www.youtube.com/watch?v=ZQqAr4jJ9Xw - signing up is also free.
It also wasn't obvious that you had to scroll down to read more stuff even though it was yet another Twitter Bootstrap thing that we should all know and understand in this 'sign up to our Saas startup' context.
I'm trained because it's been a year or so since apparently all startup web sites are like that, an option I've seen is to add an arrow to indicate there's more.
Thanks, a good point!
On Ubuntu 12.04 with latest Firefox, clicking signup locks up Firefox and subsequently locks up the system. Eventually I am able to launch a terminal and kill the Firefox process. Using latest Chrome, everything works fine.
How much time does it take to set this up? I ask because it would be nice to set up the test for free then start paying the $90-$700 to actually run unless this is something that a busy engineer can set up in a few hours.
You can create a trace for your web app and integrate it with Jenkins in 5 minutes.
One customer, that didn't have any automated testing, was able to create 40 cross-browser traces and integrate runs with Jenkins in two days. The average running time for their whole set is about 3 minutes.
When you register account, you can test for free. CI and reporting are under settings.
Why wouldn't you just use Cucumber or RSPEC+Selenium(or some other variant library for a different language) to do it?
Most of the teams do not have required skills for scripted test automation. Even the teams that do have the skills struggle to have time for testing.
Personally after doing programmed Selenium tests for 9 years, I have learned that when releasing software daily, scripted tests lag all the time behind development. Providing a simplified testing tool that allows easy trace creation and maintenance without programming is needed.
However traditional record & playback tools are not working for agile teams due to maintenance burden. Usetrace brings concepts from the scripted testing available to non-programmers.
Because you can pay these guys to do that for you. Plus, they'll support it and provide fake user data.
You really need to work on your sign-up form validation. At least add client side validation.
Some improvements made. Going to be better!