Context-Driven Testing, Bonusbox Style
Overview
 
For those unaware, "Context-Driven Testing" is a testing concept conceived by James Bach and friends that essentially bundles basic common sense pragmatism into a set of seven general principles. This list of principles is more or less how we function at Bonusbox:
 
  1. The value of any practice depends on its context.
  1. There are good practices in context, but there are no best practices.
  1. People, working together, are the most important part of any project’s context.
  1. Projects unfold over time in ways that are often not predictable.
  1. The product is a solution. If the problem isn’t solved, the product doesn’t work.
  1. Good software testing is a challenging intellectual process.
  1. Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.
 
How do we apply these principles at Bonusbox? For Bonusbox, the first two principles liberate us from the constraints and costs of a lot of traditional QA behaviors. But principles 3 and 7 force us to regularly re-evaluate our practices (or perhaps, "anti-practices"), as the company, and the product continues to grow and mature. Now, on to the specifics!
 
Breakin' The Law
 
There are a number of testing practices at Bonusbox, that at first glance, might make us appear to be what used to be called "cowboy coders". Nothing could be further from the truth. There is a method to this madness. However, it's quite true that this method runs directly counter to a number of ancient "industry standards" when it comes to QA as a role, and testing as a practice. Here are the 4 most obvious ones:
 
Think Outside The Black Box -  To test at Bonusbox, you need to understand Rails and object oriented concepts the way a dev understands them. You need to be able to do everything a dev can do, with the possible exception of actually writing his code. The days of requirements checklists, and hand-testing an opaque user interface are long gone. To add value to the team, you need to be able to identify the source of problems, including employing whatever technical tools are necessary to do that, even to the point of being able to fix them yourself.
 
Staging Is For Sissies - The motto at Bonusbox is "break it fast, fix it fast". In order to execute successfully on continuous deployment, the tester does his work from the same basic environment the developer does: his desktop. Anything that requires external resources that cannot be duplicated or properly mocked, will get tested directly in production. For such cases, breaks will not be backed out unless they're completely irreparable. Instead, they are fixed immediately. 
 
The Dog Ate My Test Plan - Test plans are traditionally drawn up at the time requirements are defined, during feature planning. This is simply untenable in an environment where features change shape and size almost hourly, as they make their way through the development process of a lean startup. Testers must be able to stay on top of this changing landscape, by staying connected to the code, and to design and development discussions in Trello.
 
Open Borders - In legacy development environments, the tester's value was defined by how much he slowed the progress to production. Bug counts, feature rejections, and defects passed into production, were all metrics that gauged his competency. He was something of a gatekeeper, standing guard over the business owner's anxiety. Today, the situation is almost entirely the opposite. The QA is a free agent, a special forces commando. He is far less interested in completely sealing off borders, than he is in tracking down and pacifying threats on their own turf. He moves from team to team, working to help them get to deployment as quickly as possible, but also as cleanly as possible. 
 
Talk To Me
As you've probably intuited already from the above, good communications and good relationships are key to the success of the modern QA Engineer. Though this is listed at the bottom, it is probably the most important feature of QA at Bonusbox. The QA must be capable of negotiating difficult conflicts and analyzing complex problems dispassionately, and he must be willing to communicate his findings with both developers and product managers, in their own language. 
 
 
FAQ