Friday, 09 February 2018
In this article, we explore how Behaviour Driven Development standardises requirements capture and what tools can be used to help us.
For anyone struggling with using Behaviour Driven Development (BDD) tools for testing, we would recommend reading Aslak Hellesøy's article on the misuse of Cucumber. As he says:
"I'm happy to have created such a popular tool, but saddened to see how it's misused and misunderstood. If you think Cucumber is a testing tool...you are wrong."
Whilst his frustration with the misuse of BDD tools like Cucumber is understandable, not all issues stem from misunderstanding.
As Aslak points out. "Cucumber was born out of the frustration with ambiguous requirements and misunderstandings between the people who order the software and those who deliver it." So perhaps some of the criticisms are borne out of a frustration with not knowing how to approach the problems that tools like Cucumber are not designed to meet.
Behaviour Driven Development
First of all, let’s recap on what BDD is. It’s a process by which requirements are turned into tested, production-ready code by clearly specifying the requirement through the use of a story where there is a title, a narrative and examples.
Requirements are written in the form of:
- Feature (or title): a one line description
- Narrative: a short explanation of who is doing what and why
- Scenarios: specific examples of the requirement in action
The genius was then to turn these stories into executable tests. The scenarios are mapped line by line to executable code, which will pass or fail. For this to be effective, the language used for these stories has to be clear and unambiguous. This language is called Gherkin.
BDD example
Let’s say there is a requirement that a change should be made to Prolifics Testing’s website, to make the Contact Us form include a field for the type of enquiry: Training, Consultancy or Other.
When following a BDD process, the business analyst will, in collaboration with development and test, write the requirement in Gherkin as follows:
Feature: Dropdown list of enquiry types
In order that enquiries are handled by the right department
As a user
I should be able to fill in the Contact Us page and choose the type of enquiry
Scenario: Make a training enquiry
Given that I am on the page https://prolifics-testing.com/contact-us
When I choose ‘Training’ from the ‘Select an Interest’ field
And fill in the First Name field with Bob
And the Last Name field with Jones
And the Email Field with bj@test.com
And the Phone Field with 02089052766
Then I should see ‘Thankyou for your enquiry’
And the training department will be notified.
It still looks like English, but this can become an executable test or series of tests using tools like Cucumber, JBehave or Behat. Each line of the scenario is mapped to executable code that will start a browser and perform each of the steps.
We now have a reusable automated acceptance test that can be run and re-run for subsequent releases.
But before we even think about the benefits of having an automated acceptance test, let’s just consider the implications of having written the requirement in this format. On the one hand, the business analyst hasn’t been able to write the requirement freely, which is a common grumble about this approach. However, in the example above, just writing out the requirement has already thrown up some new questions:
- Where should enquiries marked as Other go?
- What does ‘And the training department will be notified’ actually mean? An email, IM message, SMS? If so where to?
Without the discipline of having to specify the system with practical examples that must drive an automated script, it’s easy for details to be glossed over or misunderstood. This is the main benefit of using this approach and the driver behind Cucumber being downloaded more than 5 million times in the first three years.
Investing in BDD - does it cover everything?
However, getting up to speed can take a significant investment of time and resources and it’s perhaps inevitable that sponsors ask – so how far can we go with this BDD stuff?
For a start, it looks like we now have an automated regression pack. Is all our regression testing covered?
On the face of it, yes. However, let’s consider what needs to be tested for the example above.
The new feature has a pretty clear requirement, which is all about the movement of text – the enquiry from a form to each destination. However, in addition to the explicit new requirement, there are many implicit requirements.
The form with the new field on should not just be functional, it needs to look good and it needs to look good on a broad range of devices. What would you think if you couldn’t read the text in the drop-down or it was covered by another item on the page?
In theory, you could define up front all the things that could go wrong but you would need to know what those things are, which you don’t! As a result, some tests will need to be manual or use automation to get to a particular point before recording results for later analysis by a human.
The developer may have needed to perform several distinct tasks in capturing and storing the new information, looking up routing rules for the enquiry type and finally sending the notification. The development for each of these tasks will have testing around it. Creating artificial features and scenarios and executable feature code is time consuming with little benefit.
If there are concerns about the stability of some parts of the solution, then some automated tests may need to be written which tax the system. So what would a scenario look like?
We could write out each scenario in long hand but if we are planning on performing hundreds or thousands scenarios with varying numbers of steps, who would like to volunteer?
Alternatively, our feature will have to look like this:
Given I am on the Contact Us page
And I perform various random acts
Then I can still see the Contact Us page
…which hardly adds much value.
It’s hardly surprising that attempting to use Cucumber or any other pure BDD tool for purposes other than it was intended can be frustrating.
So what options are are available?
First, and perhaps obviously, you can just use different tools for different jobs.
However, there are some more integrated alternatives. Fitnesse and RobotFramework both have mechanisms for writing tests in a style suitable for BDD whilst retaining their broader capabilities. However, other tools are still usable, even if not written with BDD in mind.
Given that we will be performing manual tests, can BDD tools help? Of course they can.
Performing the scenarios manually does not make them less valid and manually performing a step can also mean implicitly testing all the things which have not been defined but which a human with their innate intelligence can pick up.
Finding out more
If you would like to learn more about how to make BDD work for you, then contact us for a no-obligation chat. You can even have a go at using the form...
