Rainforest Launches On Demand Service That Uses API To Spin Up QA Testers For Web Sites And Apps

Rainforest, a Y Combinator company, has developed an on-demand service that runs functional tests against a crowd of people through Mechanical Turk, Amazon’s crowdsourcing platform. The service is similar to how a customer can use Amazon Web Services to spin up and down instances. But in this case a customer makes an API request for people instead of machines.

The company is also working toward developing ways to seek out workers through private groups to suit companies that have NDAs and bound by regulatory issues.

With Rainforest, the customer can scale the QA work up or down, depending on what they want tested. Customers write tests in plain English, the requirements they have, and then the service takes people through a series of steps to run the tests that the customer wants performed. For example, a customer can ask a simple question about the layout of a site and get it tested in real-time.

Testing the sign up for a service on a website is another example of a task that can be done on the Rainforest service. Stevens-Smith said the task is non-trivial with traditional testing as it requires the developer to write and update test scripts using tools such as Selenium or Capybara. With Rainforest, a user creates a sign up test and then configures it. The sign up is only deployed if it passes in Rainforest.

Zapier, the app connector service, uses Rainforest to test its core flows such as sign ups to its website, said Co-Founder Fred Stevens-Smith in an email interview. “ It’s unacceptable for them not to work so they test programatically on every deployment,” he said.

Rainforest plays on the increasingly important requirement for companies to keep their sites continually optimized, Stevens-Smith said. A few years ago it did not matter that QA took 48 hours. In today’s SaaS and app world, updates can be as often as five times per day. To uncover hard to discover bugs, companies often hire human QA testers to manually check. Or, companies do it in an ad-hoc way, which he said slows down development speed and frustrates highly paid team members. Alternatively, developers write automated tests, which is time-consuming and frustrating for highly skilled developers. Here’s how he sums it up:

Why did we build Rainforest? QA sucks. But we all have to do it. Like payments pre-Stripe, QA is a process that every developer hates. Yet for some reason nobody is solving this problem. Every part of our development workflow has been totally reimagined in the past few years. Startups that have taken a design-driven approach and introduced a 10x faster / simpler / cheaper product have dominated. There’s tons of innovation. Except in testing

The data collected could over time add some new dimensions to the service. The structured information from the code changes could help define the cause of certain problems and from what parts of the app are surfacing issues. Tracking the code changes could help predict bugs. “This means we could start to integrate with your IDE to tell you lines of code we think might cause bugs and a list of bugs we’ve seen related to this code, or as a post-commit hook on Github that would ping you when we think that you’ve committed a breaking change,” Stevens-Smith said.

uTest is a potential competitor to Rainforest. It has developed a strong reputation in the market and recently turned its attention to a deeper focus on mobile apps.

(Feature image via James Cridland on Flickr via Creative Commons)