Showing posts with label Cucumber. Show all posts
Showing posts with label Cucumber. Show all posts

Wednesday, August 14, 2013

Cucumber test automation

These past few weeks I have been involved in updating my Visual Editor test plan, as well as working as writing a test case to test the visual editor headings.  I have also started to train one of our QA volunteers by running a pair programing session with her. This time I had the role of the teacher rather than the trainee, and I am pleased that I have gained enough knowledge that I was able to do an adequate job explaining our tools and processes to her.  The way I ran this session with her is that I  worked on a test case I needed to get done, and as I worked I showed her what the steps were. and how to debug it.  That seemed to work well for her, though in the next session, we are planning on her working on her own test case.

In this blog entry, I wanted to explain more about the FOSS automation tools we are using to do browser automation testing here at Wikimedia.  These include Cucumber, Ruby, and Selenium with Watir web driver.  We also use Page Object Design Pattern, which is a testing design pattern used to improve the maintenance of  automation tests.  I am excited to learn these tools not only because are they easy to learn and use, but also because they are up-and-coming browser automation testings tools starting to be seen more and more in the software testing world.  Another plus is that when you mention you are working with Cucumber, you are sure to get a good reaction from others.  Usually the reaction is something like this: "Huh?  What is that? Never heard of it".  In my daughter's case, her reaction was to insist to me that Cucumber is a vegetable and cannot be anything else but something you can eat!
 
One of Cucumber's strengths is that it is a communication tool as well as an automated testing tool.
Cucumber lets software development teams describe how software should behave in plain text.  The text is written in a business-readable domain-specific language and thus can be used for written specifications, automated tests and as a development-aid - all rolled into one.  Cucumber works with Ruby, Java, .NET, Flex or web applications written in any language.  In our case, we are running it with Ruby.



Writing a test case in Cucumber is really as simple as writing a test case in plain english, except for the fact that it must be put into Cucumber's Given, When, and Then syntax.  Here is an explanation and example of Cucumber taken directly from our Quality Assurance/browser testing document:

Cucumber demands specifications in a particular form, Given/When/Then:
  • A Given statement should only ever mention starting conditions.
  • A When statement should always be some sort of verb, or action.
  • A Then statement always has the words "should" or "should not", or equivalent.

Feature: My Feature
  Scenario: Testing some aspect of My Feature
    Given <some initial condition>
    When <I take some action>
    Then <I should observe some result in the browser>
  
Any Given/When/Then statement may also have an arbitrary number of "And" clauses:
 
Scenario: Testing some complicated aspect of My Feature
  Given <some initial condition>
    And <some other condition>
  When <I take some action>
    And <I take some other action>
    And <I take yet another action>
  Then <I should observe some result in the browser>
    And <I should not observe some other result>
 
In order to write a cucumber test for the feature Search which should work in any given wiki, you should first write down how it is expected to behave in "plain english". It would like like this:
 
On any random page in any wiki, type "main" into the search box.  
A list of results should appear, and "Main Page" should be the top result.  
On another random page in any wiki, type "ma" into the search box and click the Search icon.  
I should then be on a page with a set of search results, and "Main Page" should be one of the results.
 
Now, here it is after being cucumberized:
 
Feature: Search

Scenario: Search suggestions
    Given I am at random page
    When I search for: main
    Then a list of suggested pages should appear
       And Main Page should be the first result

 Scenario: Fill in search term and click search
    Given I am at random page
    When I search for: ma
        And I click the Search button
    Then I should land on Search Results page 

See how simple that is?  Easy Peasy :)

The next step would be to run this cucumber script from the command line, where it will detail the steps you need to take to put the test case into Ruby.   More on this in my next entry.

Tuesday, July 16, 2013

How I got to my first finished test

   Sorry, I am afraid that I have neglected my blog as of late.  Life has a way of getting in the way!  I have had my plate full to the brim doing my OPW internship, and at the same time I am doing a lot of juggling coordinating my kids summer schedules and mine.  It has been an extremely hectic few weeks, but I hope at this point things are settling down, and I will have more opportunity to blog.

   As I mentioned in the last entry, my project this summer is to provide a battery of tests for the new WYSIWYG editor feature of Wikipedia, called Visual Editor.  Visual Editor is an important new feature that was launched in July to all logged-in and anonymous users as the default editor for the English-language Wikipedia. It is expected to be deployed to almost all Wikipedias by the end of July, making it available for all editors.  There are a huge number of things to be tested within Visual Editor, so I am planning to do a battery of tests on some parts of it, rather than attempt to cover it all.  One of the first things I decided to test is to make sure that links are working properly within the Visual Editor.  As I wrote the test case for the links feature, I was pleased that I also made progress towards my individual goal of learning Cucumber, Ruby and Page Object Design.  Finally, last week after 19 commits over several weeks,  I was able to reach a milestone and commit the test as finished.  After this last commit, my links test was merged into our master repository.  The link test is now part of the automated browser-level tests that run continuously on our Jenkins instance, doing compatibility and regression testing.

  It is a good feeling to know that I have completed a test that is already being run by WMF to check that links are functioning properly in the newly released Visual Editor feature.  So what did it take to get to this final commit for my Visual Editor links test?  Well, a lot of elbow grease, more than anything else.  This is what I've always found to be the main ingredient when coding.  A lot of trial and error has to happen before you get to the point where you are able to get your final code committed.  In  this case, there were 19 commits along the way to getting the test case finished.  There was also a lot of help I received along the way from my mentor Chris McMahon, who pair programmed with me several times as we developed the tests.  I have found pair programming is an extremely effective technique, and I'm pleased that Chris and Zeljko have introduced me to the concept.  I have found that you can learn much more efficiently and quickly when you pair program, especially at the earlier stage of coding in new languages.

  I ran into several other issues as I developed the code for the link test.  One early issue was that Chris realized some of my Cucumber code duplicated code we had already written for other tests, so he had to refactor the Cucumber links script I had written.  This meant that I had to redo some of the coding on my Ruby scripts that corresponded to changes he made to the Cucumber script after refactoring.  I believe that due to the fact we were pair programming, this was caught before too much time went by, and it was a relatively easy change for me.  If I had handed the test case over after weeks of working on it by myself, it probably would have taken me much longer to rework the code.  This is example of how pair programming can work well, and can end up saving time in the end.

  Another problem I ran into as I developed my Cucumber feature script was that the sequence of steps I had created in Cucumber did not create the proper link in the Link UI.  Upon looking at the Link UI, I realized I was missing a step.  I then had to add the missing step to my Cucumber script to make it work properly.  Another issue I encountered towards the end of my coding was the need to reformat some of my scripts to get rid of the extra white-space and new lines.  It was also necessary to rename the script from ve_links.feature to visual_editor.links.feature to correspond to the naming conventions we were using.  In a nutshell, these were some of the challenges I faced when creating my links test.

Now for some public service announcements concerning Wikimedia automation testing..

Coming this Thursday, July 18th...
 QA automation workshop at  16:00 UTC !  Workshop is being held at this time especially for our Asia/European volunteers to get onboard!  Please sign up at the link if you are interested in learning or working with Cucumber and Ruby and contributing to our test case coverage.



Can't make it but are interested in learning more about current and future WMF QA activities?  Please sign up at our Wikimedia QA mailing list.

Further questions or interested in volunteering to write tests?  Feel free to email me at rachelqa99 [at] gmail [dot] com .

Thank you for your interest :)