Test Automation and the Art of Poetry

You never know where life will take you.  In college, I studied literature – and mostly stayed away from the math/computer science buildings on campus.  For some reason, I was interested in the early English Renaissance and Sir Philip Sidney.  In one of Sidney’s works, A Defence of Poesie (also known as The Defence of Poetry and An Apology for Poetry) , he writes about the purpose of poetry and his interpretation of art.

College was a long time ago, and I don’t remember a lot of my study of Sidney.  One thing I do remember is that Sidney wrote that the finest art hides its art.  The best art or poetry looks easy.  The finest art, the art that takes the greatest talent and the hardest work, looks simple when done well.  I like to think about this when I think about test automation.

A good test automation framework should look simple.  In fact, it should look stupid simple. It should be something that you can explain to someone without a lot of big words or long manuals.  It should be easy to read without having to think too hard.

With this as a guiding principle, I think a good automation program should rely on abstraction.  You can present complex ideas by breaking the ideas into simple, generalized parts.  The simpler parts are often easy to describe so that people can understand them.  Good test automation should be abstracted to the business level.  By abstracting to the business level, you increase readability and open your work to others. You don’t have to be specialist to understand what is going on --  just the same as you shouldn’t need to be a PhD literary critic to appreciate good poetry.

You may have some clever code and some complex code, but when it is implemented, it should look simple.  In an automation framework, the implementation should look simple.  In the actual code libraries that support your framework, the code should look simple.  If not, refactor.  We expect the application developers to do this with their code; we should do the same.

If it looks complex, it is probably either or bad automation or bad poetry. The next time you are looking for a  case for simplicity, look no further than Sir Philip Sidney.  Simplify, simplify, simplify!

Great Day for Ruby Programming (and Our Test Team)

While reviewing some Ruby blogs and forums, I found

http://www.beginnerruby.com/ruby-executables/package-ruby-scripts-as-executables-with-ocra/

This information on the ocra Ruby gem is breathing new life into our scripting program.  Using Ocra, we can package our Ruby scripts into executables that we can distribute easily.  Ocra creates .exe files that include references to all the related files and Ruby gems used in a script.  These executables can be run on machines without worrying about any dependencies -- you don't even need to install ruby on a machine.  This is one of the most liberating things I have seen in a while.

Since we started with our Ruby study on the test team, we have developed utilities ranging from a services testing framework to a utility for updating expected results for automated tests to another utility for releasing locks in our QC/QPT tests.  These utilities are great, but they suffered because it is a bother to install ruby, all the associated gems, and referenced files on each machine.  Because it is a bother (we are sort of lazy), we often didn't do it, and the great work we did gathered dust.

Using Ocra today, we created executables, and this great work from the past year came back to life.  After we verified that this gem works (thanks Josh), it is hard not to get excited about the possibilities.  We developed libraries in our bigger projects (the services test framework, most noteably) that we have referenced in other projects.  Now, I am thinking of other places we can use these resources.

It is all coming together.  The test team with the biggest tool kit wins.  Death to boring work.

“Whole-team” Test Automation

At my company, we practice the art of “whole-team test automation.”  Very simply, this means that test automation is not a specialized skill on our test team.  For us, the ability to develop automated tests is a required skill for all members of our test team.  At our company, highly skilled automated test engineers are common as dirt.

If you thinking that this seems like a lot of work, you are right.  It takes an effort.  So, why would we invest in this idea?

• In agile development, rapid development of automated tests is critical.  While we still struggle with keeping up with automation during sprint work, the option is always there for the testers and the scrum teams to do what is best for the project.  This may mean aggressively developing regression tests throughout the project.  It is not acceptable for one project to have good coverage just because they happen to have the good automator – all projects deserve technically skilled testers.
• At many companies, automated tests are developed, executed, and owned by a separate group within the test team. This creates a two-tiered test team.  It creates a situation where the most technical members become more technical because they have the most practice and the least technical members have no opportunities to grow their skills.  That sucks if you not technical and have no opportunity to grow.
• The more technical a tester is, the more tools the tester has in his toolkit, and the better the tester will adapt to new situations.  The new and changing situations range from new technologies such as services testing to a fundamental change in how software testing is done.  Adapt or die.
• Agile testers are developers.  As members of a software development team, everyone should know how to code.  It is good for your self esteem, and it helps you gain and keep the respect of the other developers on your team.

If you are convinced that we are on to something, how do you go about it?

• Build or select a suitable test framework and implementation methodology, and develop a transparent approach to testing.  For this, we chose HP’s BPT framework with QTP as the underlying automation engine.  Using BPT and our “Archetype” implementation technique, all members work together and take advantage previous work and each other’s skills.  With a good framework, new team members can build tests, and experienced automators have opportunities to work on challenging work.
• Good recruiting.  Regular, run-of-the-mill “QAs” are not right people for the team.  Be prepared for this to take a long time.  Because of the history of testing (a combination of “QA” process types and non-technical manual script runners), many good people stay away from testing jobs.  My only advice here is to start with high standards, don’t let your standards drop, and keep hope.  Great people are out there.  And in my opinion, most recruiters will not be your friend.  They will pressure you to take conventional candidates.  Be aware of recruiters.
• Be prepared to train people .  Some of the most obviously experienced candidates will take the most time.  Many experienced automators do not work in a whole-team automation environment.  As a result, they tend to build tests for themselves and not for the team.  There is often “unlearning” that has to happen.  
• Promote technical skills. Learn new programming languages and new technologies.  This year when we studied Ruby together, we spent some time looking at Rails projects.  By looking at and studying Rails, we accidentally learned about MVC frameworks.  
• Devote enough time. In addition to the sprint work that we do (with or without other testers), we devote an hour each day to work together on automation.  This way, we share ideas, learn from each other, set high expectations for each other, and continue to practice our art.  There is a bonus with this approach when you are working on projects that are not interesting.  If your project stinks, you can work with other team members on their automation needs.  Everyone wins.

As you live with the “whole-team” automated test philosophy, be prepared for things you do not predict.  Train your team, give them the tools and support, give up control, and let them surprise you.

Programming Challenges for the Test Team

This is a follow up to a earlier post on Ruby programming.  This is part of the continuing effort to raise the technical bar on the test team (and to keep it high).  Here is the latest Ruby Challenge:

A farmer has a 40-pound rock that he uses with a balance-type scale to measure grains and feed.  He lends that rock to a neighbor, and the neighbor accidently breaks the rock.  It is broken into four pieces.  He returns the rocks to the farmer and is very apologetic.   The farmer, unexpectedly, is pleased.  He says with these four pieces of rock and the balance-scale that he can now measure everything from 1 to 40 pounds.  What is the weight of each rock?

I know that some of the team know the answer (or at least remember the riddle).  But the answer doesn’t really matter.  Here is the challenge:  Using Ruby (or some other programming language), write a script that tests your guess. 

As a bonus, write an algorithm that derives all the possible combinations of stone weights.

Bonus points for elegant code.

Bragger’s rights for the first good script.