Category Archives: Coding

Goldilocks and Test Automation

I’ve struggled with the right level of automation for a long time and alluded to a level of salvation here.  Since that post, I’ve moved on to engage automation using Given-When-Then (and its collateral ideas) much like a sailor engages a lighthouse at night.  In my most recent experience, it was more like a sailor is on the bridge watching the light and I’m below deck providing interpretation and direction.

My testing team created an excellent suite of automated regression tests for the stored procedures used in our application.  But I’m struggling with the number of tests created for this suite.

It might be that my definition of “regression tests” is more economical than that of my teammates and I own that communication error: I probably was not clear on what I thought this task should produce.  But it also goes to the heart of what I believe about automation tests.

In 2007, I, like many others, was enamored with automated tests against the UI of my applications.  A steady diet of automation tools over the succeeding years (starting with HP QTP) were both valuable and frustrating.  I eventually gave into the frustration because the tests I, and many others, wrote were fragile and numerous.  My response was to limit what I automated because inflicting such a level  of maintenance on a product team was just wrong.

I watched as other teams continued to indulge in not only creating these tests but to develop frameworks around frameworks that provided some theoretical value and, in my opinion, dubious claims of a positive ROI.  I was unimpressed with what appeared to be the emperor (in their “new clothes”) dining happily on over-heated porridge.

With training and some practice in Given-When-Then and its description of business outcomes, I started to see practicality again in test automation.  A test that evaluates business outcome is, in my opinion, more stable, valuable, and maintainable than several poorly designed automated UI tests.  Then, there are two questions.  Always.  Two.  Questions.

Can I automate this test?  Of course, I can automate just about any test.  The avalanche of software utilities that pop up around tools for test automation is evidence of over-indulgence in coding.  Not testing.  The second question is most key:  Should I automate this test?

I advocate for tests designed for simplicity and to exercise isolated business outcomes.  The tests should be motivated by risk, value, and maintenance.  Upon a review of these criteria, I stand the best chance of balancing tests (the most value) and the code (that which must be maintained to deliver the test).

My team and I will meet to review our regression suite.  I look forward to exploring differing definitions of “regression test”.  I welcome their perspectives that could help all of us discover the combination of test and code that is “just right”.


Technology Stacks (yawn)

I was assigned a Test Lead role for a project that, among other goals, is investigating SPA.  I thought I’d jargon-up my opening line because it looks all cool but it has a distinct “Emperor’s New Clothes” feel.  Single Page Application (SPA) is all the rage as of this writing (I may be a few months behind – “all the rage” is hard to maintain in software development).

Plainly said, I ain’t impressed.  In fact, I think we’re coming full circle from (former “all the rage”) desktop computing applications.  You may recall the shrink-wrapped, shiny boxes containing the coolest new application, requiring twenty minutes of disk swapping to install.  I grant you today’s installation is far simpler and faster but in the end, it’ll be a desktop application hosted in your browser.  Yup – full circle.

What really frosts me is developers talking about new frameworks and utilities they are using.  Not having familiarity with the names (which for my money are nowhere nearly as cool as Lotus 1-2-3, Wordstar, Paradox, or Sidekick), I did a bunch of research and experimenting.  I was able to install < insert popular framework>, its recommended testing tool <insert popular testing tool for popular framework>, and run a few tests.  As Biff exclaimed in “Back to the Future II”, there’s something very familiar about this.

It doesn’t stop there.  Since there are soooo many files used to create an SPA, you need a utility to bundle them.  These also have unique names but that hasn’t fool me.  Lastly, in a penultimate salute to developer vanity, there is even a utility to convert (sorry – they call it transcompile but I think that’s a pile of sh-t) source code to javascript.

If I have this straight, there is a “new” IDE for coding, a “new” utility for “compiling”, and a utility for bundling all to support SPA.  Maybe it’s just me but I’m pretty sure I can do all that in Visual Studio today.  With the Community version, I can even do it for the same price.

This journey started because we wanted to pursue SPA.  Its shiny and new.  As we experimented with it, we discovered the technology stack.  It’s as if we were walking along in the woods, and found a neat stick we could use.  When we picked it up, the other end disturbed a bee’s nest.  When you pick up a stick, you always pick up both ends.


Learning to Read Code? Here’s my Short Primer!

I support the idea of testers learning to read code. It’s also a great opportunity to build rapport with developers. If you want to learn how to write programs, I support that also.
However, I think reading code will benefit you more when you know more about it. For example, when you read a mystery book, you look for certain type of characters (a detective), and certain types of activity (discovering a clue). Reading code is much the same. Here’s my primer.

Core Constructs
There are four basic code constructs supported by just about every language. These are important to understand because each has implications to your testing. The constructs are assignment, If statements, loops, and math. Programming is an act of combining these constructs to accomplish business goals.

Code Construct How to Identify It (in most languages)
Assignment Look for an equals sign (=) between words and numbers
dateOfBirth = 6/30/2001
If Statements Search for the word ‘if’
Loops Loops contain with the word ‘for’ or the word ‘while’
Math Most languages use common symbols to perform arithmetic. The exception is multiplication which is often represented by the asterisk (*).

In addition to the core constructs, code supports collecting and reporting of information (input/output), and storage of information.

Assignment sets the value of a variable. For example, when you enter a date into a program, the program probably assigns it to a variable (temporary storage – see below) so that it might act on it. This allows the program to verify the date is valid and other tests before it is used elsewhere.
Testing Implication: Information collected from a user by the program is always suspect. This includes data collected from a web page, an application form, a file, or a web service. A healthy dose of Domain testing is in order.

The If statement provides a program the ability to evaluate data, and to direct workflow. For example, you want to open a credit card account and the company has a business rule requiring applicants be at least 25 years old. If you provide your birthday as today, the program evaluates that date and rejects it.
Testing Implication: Business rules may have multiple outcomes. In the example above, there are at least three. Your testing may need to functionally evaluate the program for each outcome: (1)the applicant is 25 years old, and the applicant is (2)more than or (3)less than 25 years old. Additionally, the program’s workflow may be different for each outcome.

Loops allow the program perform the same code sequence many times. For example, when your music program displays a list of songs, the same code sequence executes to display each line but the data it uses, the song name, changes each time through the code sequence.
Testing Implication: The number of times a code sequence executes may depend on the data, it may execute a fixed number of times, it may change program behavior if there is an error, and in some cases it may never stop. Review programs for loops and evaluate what controls the execution of loop.

Programs perform basic math to trigonometry. For example, a spreadsheet performs many arithmetic functions.
Testing Implication: The arithmetic equations themselves should be evaluated in isolation. Once you have confidence it the equations, the inputs to those equations may require Domain analysis and testing.

We are probably most familiar with the input and output of programs. We enter data into a screen, and we see data displayed on a screen or printed report. Programs accept data from other programs, web services, files, or cameras. Conversely, programs deliver information as files or strings of characters.
Testing Implication: There are many forms of input and output and they vary based on source (input) and destination (output).
How to Recognize It: Theses are largely code sequences specific to the requirement of the program. This is a great opportunity to review the functionality with your developers, collaborate on the testability of inputs and outputs, and learn.

Programs store data in two forms: temporarily and permanently. Temporary storage (known as variables) allows the program to work with data to meet the business needs. Permanent storage provides for persistence of information over a long period of time (disk drives and databases are examples of permanent storage).
Testing Implication: Temporary storage may be an issue in embedded applications where memory is scarce. Permanent storage can improve program testability when used to create logs or journals of transactions.
How to Recognize It: Temporary storage is declared explicitly in many languages. The syntax includes a data type identifier and a variable name. Permanent storage usually requires code libraries to read and write information to a file or database – another opportunity to talk to your developer.

Now, go find some code and practice!  Leave some feedback on the examples you found!