“Whole-team” Test Automation

At my company, we practice the art of “whole-team test automation.”  Very simply, this means that test automation is not a specialized skill on our test team.  For us, the ability to develop automated tests is a required skill for all members of our test team.  At our company, highly skilled automated test engineers are common as dirt.

If you thinking that this seems like a lot of work, you are right.  It takes an effort.  So, why would we invest in this idea?

• In agile development, rapid development of automated tests is critical.  While we still struggle with keeping up with automation during sprint work, the option is always there for the testers and the scrum teams to do what is best for the project.  This may mean aggressively developing regression tests throughout the project.  It is not acceptable for one project to have good coverage just because they happen to have the good automator – all projects deserve technically skilled testers.
• At many companies, automated tests are developed, executed, and owned by a separate group within the test team. This creates a two-tiered test team.  It creates a situation where the most technical members become more technical because they have the most practice and the least technical members have no opportunities to grow their skills.  That sucks if you not technical and have no opportunity to grow.
• The more technical a tester is, the more tools the tester has in his toolkit, and the better the tester will adapt to new situations.  The new and changing situations range from new technologies such as services testing to a fundamental change in how software testing is done.  Adapt or die.
• Agile testers are developers.  As members of a software development team, everyone should know how to code.  It is good for your self esteem, and it helps you gain and keep the respect of the other developers on your team.

If you are convinced that we are on to something, how do you go about it?

• Build or select a suitable test framework and implementation methodology, and develop a transparent approach to testing.  For this, we chose HP’s BPT framework with QTP as the underlying automation engine.  Using BPT and our “Archetype” implementation technique, all members work together and take advantage previous work and each other’s skills.  With a good framework, new team members can build tests, and experienced automators have opportunities to work on challenging work.
• Good recruiting.  Regular, run-of-the-mill “QAs” are not right people for the team.  Be prepared for this to take a long time.  Because of the history of testing (a combination of “QA” process types and non-technical manual script runners), many good people stay away from testing jobs.  My only advice here is to start with high standards, don’t let your standards drop, and keep hope.  Great people are out there.  And in my opinion, most recruiters will not be your friend.  They will pressure you to take conventional candidates.  Be aware of recruiters.
• Be prepared to train people .  Some of the most obviously experienced candidates will take the most time.  Many experienced automators do not work in a whole-team automation environment.  As a result, they tend to build tests for themselves and not for the team.  There is often “unlearning” that has to happen.  
• Promote technical skills. Learn new programming languages and new technologies.  This year when we studied Ruby together, we spent some time looking at Rails projects.  By looking at and studying Rails, we accidentally learned about MVC frameworks.  
• Devote enough time. In addition to the sprint work that we do (with or without other testers), we devote an hour each day to work together on automation.  This way, we share ideas, learn from each other, set high expectations for each other, and continue to practice our art.  There is a bonus with this approach when you are working on projects that are not interesting.  If your project stinks, you can work with other team members on their automation needs.  Everyone wins.

As you live with the “whole-team” automated test philosophy, be prepared for things you do not predict.  Train your team, give them the tools and support, give up control, and let them surprise you.

Programming Challenges for the Test Team

This is a follow up to a earlier post on Ruby programming.  This is part of the continuing effort to raise the technical bar on the test team (and to keep it high).  Here is the latest Ruby Challenge:

A farmer has a 40-pound rock that he uses with a balance-type scale to measure grains and feed.  He lends that rock to a neighbor, and the neighbor accidently breaks the rock.  It is broken into four pieces.  He returns the rocks to the farmer and is very apologetic.   The farmer, unexpectedly, is pleased.  He says with these four pieces of rock and the balance-scale that he can now measure everything from 1 to 40 pounds.  What is the weight of each rock?

I know that some of the team know the answer (or at least remember the riddle).  But the answer doesn’t really matter.  Here is the challenge:  Using Ruby (or some other programming language), write a script that tests your guess. 

As a bonus, write an algorithm that derives all the possible combinations of stone weights.

Bonus points for elegant code.

Bragger’s rights for the first good script.  

Learning Ruby (or Python), What Does That Get Us?

In a recent team meeting, we discussed ways to increase our technological capacity.  Currently, we have a fairly technical test team.  Many of us are very good (or getting very good) at VB script, we have passable SQL skills, and possess a variety of backgrounds that include technical training and programming.  As far as most test teams go, we are very technical, and we have the interest to take it further.  The question is “where?”

One idea that came up was for us to learn another scripting language, such as Ruby or Python.  The next thought, though, was what do we do with that knowledge? Hmm. 

Using a scripting language with a testing library such as WATIR or Selenium, we can interact with web objects.  Having the ability to interact with web objects and script test logic is a powerful thing.  With this, you can write many utilities to assist with testing.  These utilities do not have to be full-fledged, stand alone automation tools, but they are great levers for hybrid testing, tool-assisted manual testing.

But we have QTP (and presently, enough licenses).  So what would Ruby or Python get us that we don’t already have?  Here a couple thoughts in no particular order:

* Technical prestige.  VB Script is a great tool, but it doesn’t get as much respect as other languages.  Jerks.

* License extender.  If you use Ruby or Python for day-to-day tasks, you do not tie up expensive QTP licenses.

* Professional growth.  Learning and using open-source tools enables you to evaluate different tools and to participate in more industry discussions.

* An open mind.  Learning how to perform a task using different tools opens you to different approaches to other things.

* Interaction with the open-source community.  Becoming proficient with these tools may give you an opportunity to give back.

Now, we need to find the time to learn Ruby (or Python)