Tuesday, December 20, 2011

Let’s Write an Automated Test Framework


I am a huge one-time fan of HP’s BPT framework.  Test execution may be slow, but it is the best tool I have found for a growing test team to use in a whole-team test automation environment.  BPT has powerful abstraction of test scripts to make it easy for business-focused people and new test team members to build tests.  It enables automators to build tests using simple keyword-driven techniques and more powerful and sophisticated test scripts when the situation demands it.  As soon as I learned to ignore the misguided documentation from HP on how to use BPT, it became a very powerful tool.

Over the past year, I have been struggling with the new (and supposedly improved) version of QC/BPT.  The past year has been a mess.  What was a promising enterprise solution for long-term use has, in my mind, crumbled into a pile of what-could-have-been.  As it looks now, it is time to write an open-source automated test framework.

There are many others who have gone down this path.  Most have mixed results (we have looked for other frameworks that meet our needs and haven’t been satisfied with anything yet).  I am still convinced that BPT (as we implemented it) is the best test automation framework out there.  And since so few other people have found BPT (or found it to be a suitable solution for them), I think my team is uniquely (I am sure we are not alone) suited to drive the development of this project.  We have seen what works well.  We have solved the key problems of automation Fire Swamp -- Remember the movie the Princess Bride (1987) when Westley and Buttercup escaped from the Lightning Quicksand?

Westley: No, no. We have already succeeded. I mean, what are the three terrors of the Fire Swamp? One, the flame spurt - no problem. There's a popping sound preceding each; we can avoid that. Two, the lightning sand, which you were clever enough to discover what that looks like, so in the future we can avoid that too.

Of course, in the next moment, he learns about the Rodents of Unusual Size (ROUS) -- the third of the problems.

In the case of test automation, the three terrors are lack of readability, lack of maintainability, and lack of flexibility.  By using our implementation of BPT, we know what problems to avoid.

Look out HP, I think we might write a framework.

5 comments:

  1. I am new user on HP BPT framework but I think It is a powerful framework to make efficent further tests. Nice blog Bob , I just became a follower of it.

    ReplyDelete
  2. Thanks for sharing, I will bookmark and be back again

    Software Testing Services Company

    ReplyDelete
  3. interesting article. Lot of parallels between your experiences with BPT and agile and what we have been doing in our company

    I really liked BPT 9.2 and i thought the concept was great. We also implemented a BPT methodology within an agile team as a way of allowing us to do earlier test design and to develop a more maintainable/ "agile" automation framework. I also agree that test execution isn't great from a speed perspective. Though (like you) we developed some workarounds.

    I'm curious to know at what level you modeled your components? Per screen? Per section of the screen? Per "business process"? .... is this something you can elaborate on?

    ReplyDelete
    Replies
    1. Hi R

      When I first started using BPT, there was not much guidance from HP on what should make a component. The "business process" components typically don't give you the flexibility you need. We use a combination of 'archetype' components and functional components. The archetype components are low level components like Enter Data ... Window or Verify Data ... Section of ... Window. The scope of the component depends on the complicity of window. Functional components are more business focused. They perform a task such as open an application or register a user -- they are high level and usually make the test easy to read. Most of our decisions come down to readability. Let me know if you want more details. I could share some detailed notes with you...Bob

      Delete
    2. Hi Bob,
      I really like the trend you were taking in explaining BPT. I am stepping into BPT now and would be glad to learn every possible thing anyone has that could help me.

      Thanks Vic

      Delete