Wrap up of my Outreach Program for Women Internship

My internship is rapidly coming to a close, so the time has come for me to do a wrap up of my time here.


Overall, +2 for the internship.  The internship was a really positive experience for me on many levels.  I met amazing people and got an inside view of how an Open Source project works.  Chris and Zeljko were awesome mentors, and I learned a great deal from them about Quality Assurance and automation.  I also accomplished my goal of learning to code QA automation tests in Cucumber and Ruby.  It was truly inspiring to work in an environment that held itself to the high standards it promotes of collaboration, transparency and sense of community.  By participating in the Engineering Community Meetings I got to see the ways in which Wikimedia works to involve volunteers at many levels.  Very impressive was the quality of the QA automation workshops we held, and the number of volunteers participating.  Also impressive was the QA mailing list, and the high level of discourse there.  I always wondered how open source projects like Wikipedia are able to be so successful.  Now I know; it is the dedication and passion of the people involved that makes these projects happen.  It  was cool to see this up close and personal, as I saw how projects evolved and people collaborated together to make things happen.

On a personal note, I am grateful that programs like the Outreach Program for Women exist to give people like myself an opportunity to learn new skills.  I have been trying to get into the QA automation field for some time, but unfortunately I have found that the corporate world is not so eager to take a chance on you if you don't already have the skill set they are looking for.  This becomes the classic chicken-and egg problem.  You can't get the job till you have the skill, and you can't get the skill until someone gives you an opportunity to work in that field.  Kudos to OPW and Open Source for giving people a chance to advance themselves and learn new things, and to break out of this cycle!

On to the list of my major accomplishments:

• Wrote a test plan for VisualEditor, detailing the types of tests needed to test VisualEditor.
• My test cases were the first automated tests to be written to check VE functionality.  They included tests for logins, links, headers, references and bullets.  To see my coding contributions, please go here.
• Learned cutting edge technology to write my tests: Cucumber, Ruby, Page Object Design and Selenium/Watir.
• Did exploratory testing of VisualEditor resulting in several bugs being found, and filed in Bugzilla.
• Make use of Git and Gerrit to check in my code and get code reviewed.
• Contributed to the QA community outreach at Wikimedia by helping to run the QA automation workshop for volunteers, attending Engineering Community Meetings, and pair programming to  train a new QA volunteer.

For the future, I am planning to still contribute to QA automation efforts at Wikimedia, and will also be job hunting, looking for a QA automation job in the Boston area.  By the way, if anyone out there has any job leads for this sort of job, please let me know!  (Email: rachelqa99 (at) gmail.com)
 
Thanks to the Wikimedia Foundation for giving me this wonderful opportunity, and thanks to the Outreach Program for Women for making this program happen.  

See you all around the web. :)

Cucumber test automation

These past few weeks I have been involved in updating my Visual Editor test plan, as well as working as writing a test case to test the visual editor headings.  I have also started to train one of our QA volunteers by running a pair programing session with her. This time I had the role of the teacher rather than the trainee, and I am pleased that I have gained enough knowledge that I was able to do an adequate job explaining our tools and processes to her.  The way I ran this session with her is that I  worked on a test case I needed to get done, and as I worked I showed her what the steps were. and how to debug it.  That seemed to work well for her, though in the next session, we are planning on her working on her own test case.

In this blog entry, I wanted to explain more about the FOSS automation tools we are using to do browser automation testing here at Wikimedia.  These include Cucumber, Ruby, and Selenium with Watir web driver.  We also use Page Object Design Pattern, which is a testing design pattern used to improve the maintenance of  automation tests.  I am excited to learn these tools not only because are they easy to learn and use, but also because they are up-and-coming browser automation testings tools starting to be seen more and more in the software testing world.  Another plus is that when you mention you are working with Cucumber, you are sure to get a good reaction from others.  Usually the reaction is something like this: "Huh?  What is that? Never heard of it".  In my daughter's case, her reaction was to insist to me that Cucumber is a vegetable and cannot be anything else but something you can eat!
 
One of Cucumber's strengths is that it is a communication tool as well as an automated testing tool.
Cucumber lets software development teams describe how software should behave in plain text.  The text is written in a business-readable domain-specific language and thus can be used for written specifications, automated tests and as a development-aid - all rolled into one.  Cucumber works with Ruby, Java, .NET, Flex or web applications written in any language.  In our case, we are running it with Ruby.



Writing a test case in Cucumber is really as simple as writing a test case in plain english, except for the fact that it must be put into Cucumber's Given, When, and Then syntax.  Here is an explanation and example of Cucumber taken directly from our Quality Assurance/browser testing document:

Cucumber demands specifications in a particular form, Given/When/Then:

• A Given statement should only ever mention starting conditions.
• A When statement should always be some sort of verb, or action.
• A Then statement always has the words "should" or "should not", or equivalent.

Feature: My Feature
  Scenario: Testing some aspect of My Feature
    Given <some initial condition>
    When <I take some action>
    Then <I should observe some result in the browser>

Any Given/When/Then statement may also have an arbitrary number of "And" clauses:

Scenario: Testing some complicated aspect of My Feature
  Given <some initial condition>
    And <some other condition>
  When <I take some action>
    And <I take some other action>
    And <I take yet another action>
  Then <I should observe some result in the browser>
    And <I should not observe some other result>

In order to write a cucumber test for the feature Search which should work in any given wiki, you should first write down how it is expected to behave in "plain english". It would like like this:

On any random page in any wiki, type "main" into the search box.  
A list of results should appear, and "Main Page" should be the top result.  
On another random page in any wiki, type "ma" into the search box and click the Search icon.  
I should then be on a page with a set of search results, and "Main Page" should be one of the results.

Now, here it is after being cucumberized:

Feature: Search

Scenario: Search suggestions
    Given I am at random page
    When I search for: main
    Then a list of suggested pages should appear
       And Main Page should be the first result

 Scenario: Fill in search term and click search
    Given I am at random page
    When I search for: ma
        And I click the Search button
    Then I should land on Search Results page 

See how simple that is?  Easy Peasy :)

The next step would be to run this cucumber script from the command line, where it will detail the steps you need to take to put the test case into Ruby.  More on this in my next entry.

A fruitful week

  I thought I would write up my prior week activities as it was an unusually varied week, where I got to try a lot of new things.

  Starting off last Tuesday, we had an IRC meeting for the Outreach Program for Women, where I got to hear about the projects all the interns are working on.  Here is a description of how the meeting went from an email that I sent to my mentor Zeljko:

  "All 37 of the interns introduced ourselves and talked about our projects. I mentioned that we are looking for volunteers to help us do Cucumber/Ruby test cases, and one of the OPW interns messaged me that she was interested in helping us!  She is familiar with Gherkin, which I believe is the language for Cucumber. (editors note: Yes, I confirmed Gherkin is the language for Cucumber).  Anyhow, I sent her the link for our QA mailing list, and she signed up while we were chatting!   Her name is Shivani Poddar and here is her blog: http://shivanipoddar.wordpress.com blog. She is working on the Gnome Music project for her internship.   I also learned there is a $500 travel allowance for each OPW intern...I'm not sure of the criteria but I can send the complete transcript of the session when I receive it from Marina who runs OPW.
Also I discovered Marina, who runs the OPW program at Gnome, works nearby which is quite amazing.  We are going to try to meet up on Thursday for coffee . So, I found it to be an enjoyable and productive meeting."

  The next day, Zeljko requested that I test the new Mediawiki-Vagrant software that has recently been developed at Wikimedia.  MediaWiki-Vagrant is a nifty piece of software that creates a portable MediaWiki development environment.  It consists of a set of configuration scripts for Vagrant and VirtualBox that automate the creation of a virtual machine that runs MediaWiki software.  Once installed, there are a number of configurations or "roles" that can be set up for this software.  The one we are using is the Selenium automated testing role.  I was asked to document the steps it took to get this working on Windows, so that volunteers could use Mediawiki_Vagrant to automatically set up their own QA environments for the upcoming workshop.  Rather than having to install Ruby and the other QA tools, they can just run Mediawiki-Vagrant to get their test environments set up automatically.  The software had already been tested on Ubuntu and Mac, and my job was to test it on my Windows machine.  Well, this turned out to be more than I had bargained for.  All three pieces of software (VirtualBox, Vagrant and MediaWiki-Vagrant)  are needed to make MediaWiki-Vagrant work, and there was a lot to learn as I installed and tested the software.  One problem I discovered is that  Mediawiki-Vagrant won't work if it is installed in the  C:\Program Files directory.  It doesn't like the spaces in the name, Program Files, and the command "vagrant up" that starts the config process will bomb out as a result.  This wasn't obvious at first, but after some in-depth troubleshooting I was able to figure it out.  I then reinstalled MediaWiki-Vagrant at the C:\ level, and the "vagrant up" command worked, and config completed successfully.  I then changed the roles.yaml file to point to selenium testing, and ran one of our tests.  This time I got another error message: "Unable to pick a platform for the provided browser (RuntimeError)", at which point Zeljko pointed me to a section about Adding a Gui  that he thought would solve the problem.  After installing the Nomachine client as suggested, I am still unable to connect to mediawiki-vagrant.dev through NoMachine.  If anyone has gotten NoMachine working with Vagrant on Windows, please drop me a line in either the comments or in email to let me know how you did it.


On Thursday, we ran a great QA Automation Workshop for volunteers.  It was Zeljko's first time running a workshop, and my first time helping out with one.  He did a great job explaining the material and keeping everyone engaged.  During the workshop, Zeljko used our Cucumber/Ruby/Selenium tools to demonstrate how to write and run an actual test for the Wikilove feature of Wikipedia..  I would encourage you all to watch the YouTube video of his presentation, or at least part of it, to get an idea of how we run our workshops at Wikimedia.  In this case, the workshop was not onsite at the Wikimedia offices, but was run completely remote through Google Hangout.  (You may want to skip the first 3-4 minutes of the video as we were we sorting out technical difficulties getting the video feed working ).  I have been impressed by the number of volunteers that have turned out for our QA workshops.  For this workshop, we had 11 volunteers attending!  Some of them have already contributed to our QA efforts by writing new tests for us, or by fixing easy bugs.  We have just set up a link to our easy bugs in Bugzilla, so volunteers are able to browse for bugs that are easy to fix, assign a bug to themselves, and then work on fixing it.

After the workshop, I was able to run out and meet Marina Zhurakhinskaya for coffee, which was totally awesome.  I learned a lot from her about the history of the OPW program and more about the FOSS movement.

Of course, in between all of these events I was hard at work on another Visual Editor test that checks if  Visual Editor is present when user logs in.

 So, all in all, a very productive and interesting week.  Now it looks like I've caught up a bit with my writeups, so I will try to keep it up and catch up on the current week's activities in my next blog.

 
Anyone interested in helping us write QA tests?
Please sign up at our Wikimedia QA mailing list.

Further questions or queries?  Feel free to email me at rachelqa99 [at] gmail [dot] com .

Thank you for your interest :)

How I got to my first finished test

   Sorry, I am afraid that I have neglected my blog as of late.  Life has a way of getting in the way!  I have had my plate full to the brim doing my OPW internship, and at the same time I am doing a lot of juggling coordinating my kids summer schedules and mine.  It has been an extremely hectic few weeks, but I hope at this point things are settling down, and I will have more opportunity to blog.

   As I mentioned in the last entry, my project this summer is to provide a battery of tests for the new WYSIWYG editor feature of Wikipedia, called Visual Editor.  Visual Editor is an important new feature that was launched in July to all logged-in and anonymous users as the default editor for the English-language Wikipedia. It is expected to be deployed to almost all Wikipedias by the end of July, making it available for all editors.  There are a huge number of things to be tested within Visual Editor, so I am planning to do a battery of tests on some parts of it, rather than attempt to cover it all.  One of the first things I decided to test is to make sure that links are working properly within the Visual Editor.  As I wrote the test case for the links feature, I was pleased that I also made progress towards my individual goal of learning Cucumber, Ruby and Page Object Design.  Finally, last week after 19 commits over several weeks,  I was able to reach a milestone and commit the test as finished.  After this last commit, my links test was merged into our master repository.  The link test is now part of the automated browser-level tests that run continuously on our Jenkins instance, doing compatibility and regression testing.

  It is a good feeling to know that I have completed a test that is already being run by WMF to check that links are functioning properly in the newly released Visual Editor feature.  So what did it take to get to this final commit for my Visual Editor links test?  Well, a lot of elbow grease, more than anything else.  This is what I've always found to be the main ingredient when coding.  A lot of trial and error has to happen before you get to the point where you are able to get your final code committed.  In  this case, there were 19 commits along the way to getting the test case finished.  There was also a lot of help I received along the way from my mentor Chris McMahon, who pair programmed with me several times as we developed the tests.  I have found pair programming is an extremely effective technique, and I'm pleased that Chris and Zeljko have introduced me to the concept.  I have found that you can learn much more efficiently and quickly when you pair program, especially at the earlier stage of coding in new languages.

  I ran into several other issues as I developed the code for the link test.  One early issue was that Chris realized some of my Cucumber code duplicated code we had already written for other tests, so he had to refactor the Cucumber links script I had written.  This meant that I had to redo some of the coding on my Ruby scripts that corresponded to changes he made to the Cucumber script after refactoring.  I believe that due to the fact we were pair programming, this was caught before too much time went by, and it was a relatively easy change for me.  If I had handed the test case over after weeks of working on it by myself, it probably would have taken me much longer to rework the code.  This is example of how pair programming can work well, and can end up saving time in the end.

  Another problem I ran into as I developed my Cucumber feature script was that the sequence of steps I had created in Cucumber did not create the proper link in the Link UI.  Upon looking at the Link UI, I realized I was missing a step.  I then had to add the missing step to my Cucumber script to make it work properly.  Another issue I encountered towards the end of my coding was the need to reformat some of my scripts to get rid of the extra white-space and new lines.  It was also necessary to rename the script from ve_links.feature to visual_editor.links.feature to correspond to the naming conventions we were using.  In a nutshell, these were some of the challenges I faced when creating my links test.

Now for some public service announcements concerning Wikimedia automation testing..

Coming this Thursday, July 18th...
 QA automation workshop at  16:00 UTC !  Workshop is being held at this time especially for our Asia/European volunteers to get onboard!  Please sign up at the link if you are interested in learning or working with Cucumber and Ruby and contributing to our test case coverage.



Can't make it but are interested in learning more about current and future WMF QA activities?  Please sign up at our Wikimedia QA mailing list.

Further questions or interested in volunteering to write tests?  Feel free to email me at rachelqa99 [at] gmail [dot] com .

Thank you for your interest :)

Introduction

Hello and welcome to my little corner of the Internet!  I am really excited to be starting my first blog.
I've always been interested in blogging, but have never got around to it due to time constraints and fear that I'd run out of things to say.  Now that blogging is a requirement for my Gnome FOSS Outreach Program for Women internship, I no longer have those excuses.  In fact, I have a perfect subject to write about as the Outreach Program for Women internship provides me with a lot of built-in material to focus on.  I will be blogging mainly about the project I will be doing for the the  internship which is Visual Editor Browser Test Automation for the Wikimedia Foundation.  Here is my proposal for the project.

A little bit about me: My name is Rachel, and I  live outside of Boston, MA with my husband and our two children.  I have a BS in Mathematics, and Minor in Computer Science from SUNY Albany, and many years experience working in the IT field at various positions. These include Linux System Administration, Release Engineer, QA tester and Technical Support Engineer.  I have been a volunteer for the Wikimedia Foundation Quality Assurance automation group since January helping them with Browser Test Automation in an effort to gain more Quality Assurance experience.  When I was told about the OPW internship, I was quite interested as it seemed to be right up my alley.  I applied, and am thrilled that I was one of the candidates chosen.  Here is a little more about me.

I will be updating here at least every two weeks, but hopefully more often that that.  I would like to discuss the technical aspects of the project, but hopefully in a way that everyone can follow, even without much knowledge of test automation or coding.  We are using some interesting, cutting edge tools such as Cucumber, Ruby, and Page Objects that I hope to blog more about.  As part of my internship I may be doing volunteer coordination for the Quality Assurance automation group at the Wikimedia Foundation, so I hope to talk about that as well.  I also would like to blog about other topics that this project and internship touch on, such as quality assurance, Open Source, and women in technology.

 Did I mention  how excited I am to be part of  the Outreach Program for Women, working on such a great project?  This internship that will give me the opportunity to learn QA test automation in depth, and to help out with the open source cause at the same time.  I want to give a big "thank you" to all the amazing people that have gotten me to this point today.  A big thanks to Karen Sandler and Marina Zhurakhinskaya for running OPW,  and to Quim Gill for coordinating the OPW at the Wikimedia Foundation.  Many thanks to Chris McMahon and Zeljko Filipin for being such incredible mentors, and going above and beyond to get me started in QA automation, and for helping me during the application process for the OPW internship.  I am looking forward to having a great summer as an OPW intern!