Migration from BPT 10 to 11: Changes to Data Tables

In standard QTP, data tables are the work horses for managing test data.  They are used for data driving tests, passing data from one action to another, storing large volumes of test results, and for iterating tests.  In BPT, data tables take a back seat to other means of managing test data, most notably, test parameter data.  But even in BPT, there is a place for data tables.  For example, test data read from associated Excel files can be read in at run time and table data from the application under test can be stored in the component data table and reviewed in the test logs.

In BPT 10, each component has a local datasheet.  Because each component runs independently (remember the slow execution speed of BPT 10 – one component is loaded, it runs, the component is closed, the next component is loaded, it runs, …), there is no global data sheet.

There is a key benefit in BPT 10 to the way tests are constructed.  In a component, you can create data tables named “Actual” and “Expected.”  If this component is used more than once in a test, you don’t have to worry about name contention – the scope of data table names is in a single instance of a component.  In standard QTP, all data tables are exposed to the entire test and you have to make sure that no data tables share the same name. In BPT 11, tests share much more in common with standard QTP tests, and data tables must now be unique.

In my migration from BPT 10 to 11, I had to rethink some of my key assumptions about how I use data tables.

1. Because of the appearance of the global data sheet in the business component, I changed from using the DataTable.Import method to DataTable.ImportSheet.  The Import method loads the Excel sheet into the first data table. In BPT 10, the first data table is a local sheet; in BPT 11, the first data table is the global sheet. 

A surprise change for me when I ran my components migrated from 10 in 11 is that the imported data was not where I expected it to be.  This caused verifications to fail.  I was further surprised when some of my tests ran extra iterations.  Tests ran extra iterations because the import method added rows to the global datasheet, and the global datasheet determines the number of iterations a test runs.

ImportSheet enables you to specify which sheet to import and where to place it.  When using ImportSheet, you have to know the name of the data sheet you want to overwrite – if your test iterates, this must be a unique name.

2. Because the scope of data sheet names is not longer local to business components, you must manage the names of the sheets and make sure the names are unique.  As you do this, consider analysis of test results.  If your test has 10 iterations, there may be 10 instances of the “Expected_Result” sheet.  Getting the iteration value of component at run time is difficult so I chose to make the sheets unique by adding part of a time stamp to the name.  Ten instances of a sheet with similar names is difficult to analyze.  To give clarity, I added a line to the log file identifying which data sheet goes with which iteration.

Here is a summary of the changes I made:

BPT 10:

DataTable.Import GetCurrentTestPath() & "\" & strQCExcelSheet

BPT 11:

strUniqueSheetName = “Reference_” & Minute(Now) & Second(Now)

Reporter.ReportEvent micDone, strUniqueSheetName, “Look for reference data at “ & strUniqueSheetName

DataTable.AddSheets(strUniqueSheetName)

DataTable.ImportSheet GetCurrentTestPath() & "\" & strQCExcelSheet, 1, strUniqueSheetName

 

Generally, the migration from BPT 10 to 11 did not require changes to the automation code.  Underlying BPT is QTP, and QTP code did not change.  The change in how BPT handles data tables may be unnoticed in many implementations, depending on which features of BPT are used.  For those who push the limits of how BPT handles data, this undocumented change to data tables will be a stumbling block.  I suppose there is a price to pay to get BPT to perform at an acceptable speed.  I’ll pay this one.

Checklist for Agile Testing

During a recent test team discussion, we reviewed the key features of agile software testing.  As a team, we all have a good idea of what is important with agile testing and how it is different from conventional testing.  We worked on the checklist to give us a tool for keeping our agile test mindset fresh.

Use the checklist to question your own agile test performance.  It is good for a self examination or a discussion with your project team.  For example, you can use it to share your vision of agile testing with a team that is new to agile development.

For most of the questions, "yes" is the best answer.  It is doubtful that you will answer "yes" (or whatever the best answer is ) for all the questions -- use the questions and answers to provoke thought.

Leadership

o       Am I taking items off the mental shelf space for the lead programmer, scrum master, and/or business user?

o       Am I the scrum master for the project? If not, could or should I have been?

o       Did I help the scrum team understand risks?

o       Do I organize work for the project?

o       Have I made decisions in the best interest of the project?  Did I give up ownership and did I take on ownership of right things to make the project successful?

o       Did I help new team members learn the project?

Analysis

o       Am I documenting the project, opening tickets and describing requirements?

o       Am I writing user stories?

o       Did I come up with any of the requirements for the project?  This is an indication that you are actively participating in the project.

o       Did I define and/or develop tests at the beginning of the sprint? Early work is good work.

Collaboration

o       Are other members of the scrum team helping with testing?  In agile, any member of the team should be able to test.

o       Do I feel like I am up to date with programmers and business users throughout the sprint?  A catch-up period later in the sprint is a bad sign.

o       During meetings, do I write on the white board?  This is an indication that you are actively thinking, communicating, and  participating in the project.

o       Have I sat with a programmer while he or she wrote code for the project?

o       Have I sat with the business users? 

o       Have I tested on the programmer’s workstation recently?  This is key to providing really immediate feedback.

o       Is there a lot of scrum team chatter?  Teams that talk to each other are more agile than those that don’t.

Testing

o       Am I giving really fast feedback to programmers on the most important things?

o       Am I known as “Fast Feedback [your name]” or “Quick Test [your name]?”

o       Are the programmers comfortable coding for the full sprint? Good agile testing can directly increase the productivity of the programmers.

o       Did I add value as a testing expert?  For example, did I help business users perform good test analysis?

o       Did I build any tools to help verify the project?

o       Did I update existing automated tests that changed because of my project work?

o       Did I use the simplest solutions for the problems I worked on?

o       Did we get user acceptance feedback very early and very often in the sprint?

o       Have I built regression tests for new functionality before or while it is being built?

o       How long are issues sitting with me before I turn them around?  Issues should sit no longer than a day or two.

o       How old are the bugs we are finding on the project?  The only good bug is a dead bug, and the best dead bug is one that didn't live long.

o       Is a lot of testing taking place at the end of the sprint?  Best answer is "very little."  Make an effort to test early and regularly throughout the sprint.  If you are testing a lot at the end, you are probably not being very agile.

o       Was more than one person involved in user acceptance sessions?  Do the users I worked with represent the user community well?

Distributed Agile Test Teams -- Making It Work

I just returned from a trip to Krakow, Poland where I visited with members of my test team.  Our test team is split between two offices.  In one office, we have programmers and testers, and in the other office, we have testers and no programmers.  Because of this, it is not possible for all the scrum teams to be colocated.  This creates some challenges, but so far, everything has been a solveable problem.  In some ways, using distributed agile test teams is not idea, and in other ways, it is a big strategic advantage.

Here are some things we do to make distributed agile testing work:

• Make the best use of the overlap hours.  In our case, we have at least three overlap hours between Poland and the eastern US.  These hours are precious.  During this time, we have stand-up scrum meetings and we work on test team projects.  
• Invest in the right tools.  This means buying or building tools that enable everyone on all teams to have the same advantages.  For example, we migrated to Quality Center when we opened the Poland office.  QC is just about as responsive in Poland as it is in the US.
• Communicate!  We use video conferencing, collaboration software, and instant messaging everyday.  Cheap tools can erase the miles and bring your teams together.  We have started using Adobe Connect for video conference and collaboration.  One cool feature of this is the ability to record meetings.  This gives the abilty for programmers to pass along a quick recorded demo of the day's work to the tester on the other side.  User acceptance test sessions can be recorded -- in fact, a UAT performed during off hours can be reviewed later in another office.
• Travel between sites.  New hires in our Poland office spend several months in the US working the development and testing teams.  A big success factor is building strong relationships.  It is difficult to really get to know and to trust someone if you only talk on the phone or communicate through email.  The benefits to people traveling and to the people being visited outweight the costs of traveling.
• Find the right projects.  Regression test analysis and many test automation-related projects are easily done in Poland.  Most new functionality projects are more easily done in the other office where there is better access to programmers and business users.  
• Hire the right people.  This is always important, but it is really important when you have to rely on trust and communication with people that you don't see on a regular basis.

What I have found so far is that the issues related to distributed agile testing are common with distributed agile development.  If you are looking for more information on distributed agile development, here is a great resource:
http://download.microsoft.com/download/4/4/a/44a2cebd-63fb-4379-898d-9cf24822c6cc/distributed_agile_development_at_microsoft_patterns_and_practices.pdf

Taking a Step With Ruby

Last month, I questioned what we on a test team would gain from learning Ruby and how we could apply it in our daily work.  We already use QuickTest Pro to automate much of our testing.  QTP is a powerful tool that allows us to interact with our applications and gives us a platform for VB script utilities.  With this, what can we expect from Ruby?

A few days ago, we took the first step as a team to see what there is to see.  We bought copies of Brian Marick’s “Everyday Scripting with Ruby for Team, Testers, and You.”

One of the things I like about Marick’s approach is that he stays away from GUI testing.  He mentions it and brings up WATIR, the web GUI automated testing framework for Ruby, but he focuses the book on smaller, very practical goals.  For example, instead of teaching how to automate a web page, he uses the example of building a utility to send text message to yourself when a long-running process completes.  He shows Ruby as a bionic arm for a tester, not a full-fledged robot tester.  From the perspective of learning (no one on our team has much experience with Ruby or similar object-orientated languages such as Python), this approach seems like a great idea.  It results in small projects that are easy to complete, practical to use, and valuable for building skills.

Our goal as a team is work through the book and exercises over the course of a few months.  This will be one of those voyages where we don't know exactly where we are going or when we will get there.  Wish us luck.

Give up the “Need” for Testing

We currently staff our development projects according to a standard ratio of programmers to testers – each project has a dedicated test resource who is expected to perform most or all of the testing.  With the prospect of rapid growth, we may not be able to keep our same ratios, and testing may be stressed.  While thinking about this challenge, I played with the idea changing our test model so that testing is built around a core test team, not around a required ratio on each project team.

Doing this would require the test team to give some things up.  For example, we would have to give up the notion of testing or QA being an independent verification entity.  This concept is a left over from traditional, waterfall development.  In agile, teams hold to a “whole team” philosophy where all members of a team work together to complete a job, no matter their formal roles.  Testing does not always have to be done by testers.  And testers may take on other roles, such as requirements analysis.

You would also have to give up the notion of the guaranteed need for a test resource.  This is an uncomfortable spot to put yourself in, but it is probably helpful to consider yourself expendable from time to time. It can be a great motivator to prove your value.

So, when you give up the “need” for testing resources, what do you have left?  You have to sell yourself and your team on your own merits.

What should the role of the test team be?  Simply, you should do what you do best: 

 ·  Develop, execute, and analyze regression tests.  In a continuous integration environment, good regression tests and great analysis are an essential safety net and the best way to provide immediate feedback on quality.

 ·  Develop test tools and find the most efficient ways to solve testing problems.

 ·  Provide insight into test analysis and help teams find weak spots in their code.

 ·  Ask questions.  Good testers are great question askers.  Keep asking these questions; they are one of the best tools to building quality into products.

 ·  Be a trusted advisor and provide trusted assessments. 

 ·  Finally, know your craft really, really well -- this cannot be understated.  If testing resources are not “needed,” they will be used only when they are respected and valued.

This is one of those times when you have to give up something in order to gain it.  Give up the guaranteed need for testing and give up the limits of conventional testing.  It is an existential leap with some risk, but the potential reward is a lean, efficient test team.

Learning Ruby (or Python), What Does That Get Us?

In a recent team meeting, we discussed ways to increase our technological capacity.  Currently, we have a fairly technical test team.  Many of us are very good (or getting very good) at VB script, we have passable SQL skills, and possess a variety of backgrounds that include technical training and programming.  As far as most test teams go, we are very technical, and we have the interest to take it further.  The question is “where?”

One idea that came up was for us to learn another scripting language, such as Ruby or Python.  The next thought, though, was what do we do with that knowledge? Hmm. 

Using a scripting language with a testing library such as WATIR or Selenium, we can interact with web objects.  Having the ability to interact with web objects and script test logic is a powerful thing.  With this, you can write many utilities to assist with testing.  These utilities do not have to be full-fledged, stand alone automation tools, but they are great levers for hybrid testing, tool-assisted manual testing.

But we have QTP (and presently, enough licenses).  So what would Ruby or Python get us that we don’t already have?  Here a couple thoughts in no particular order:

* Technical prestige.  VB Script is a great tool, but it doesn’t get as much respect as other languages.  Jerks.

* License extender.  If you use Ruby or Python for day-to-day tasks, you do not tie up expensive QTP licenses.

* Professional growth.  Learning and using open-source tools enables you to evaluate different tools and to participate in more industry discussions.

* An open mind.  Learning how to perform a task using different tools opens you to different approaches to other things.

* Interaction with the open-source community.  Becoming proficient with these tools may give you an opportunity to give back.

Now, we need to find the time to learn Ruby (or Python)

Am I Agile or Mini Waterfall? Do I care?

After our development team migrated to agile, we assumed that we are an agile test team.  After some time and belly button staring, we came to the conclusion that we were a mini-waterfall test team and not very agile.

We tested and delivered code every four weeks, and we went to scrum meetings.  It looked agile, but our mini waterfall approach was very traditional.  Programmers and product owners designed and implemented the product enhancements, and testers stood against the wall waiting for code to test at the end of the sprint.  We were second-class members of the development team.

The move to agile testing started by dealing with the denial that we were not already there and by trying to articulate what exactly agile testing is.  While not everything changed (we still analyze and test software), we changed our testing philosophy.  We found ways to change our role from tester to “developer.”  We took steps to own the early days of a sprint be taking on more of a business analyst role.  We engaged the product owners earlier and more often, changing from software watchdog to advocate for functionality and usability.  And we actively sought ways to provide immediate feedback throughout the sprint, reducing the end-of-sprint rush.

So, should you care if you are mini waterfall or agile?  Agile testing gave us something that we lacked with mini waterfall, a way to become full share members of the development team.