A technical tip all testers should know

A Ministry of Testing Club post suggested we pen some thoughts around the topic title above.  In addition, it encourages us to blog for the first time or to start again.

 

Technical Tip: Never Believe What You See on the Screen

Your user interface will, occasionally, lie to you.  You can’t trust it.  The information you see on your screen will suffer from staleness, subtle misspellings, misplacement, absence, and other challenges that infect the journey of data from where it originates to where it’s displayed.

Staleness
You enter data, run the transaction, and the result is 14.02.  You enter different data, run the transaction again, and the result is 14.02.  It’s probably a defect.  Verify the result at its source!  Maybe the database was not updated, or maybe the code that moves the result from memory to the screen didn’t execute or executed incorrectly.  Collaborate with your developer to find the answer!

Subtle Missppellings
Many times the text that appears on the screen was copied from requirements.  Who cares if the requirements are spelled wrong?  You can’t beleive how often this happens but often enough that I check the spelling of text destined for screen when it appears in the requirements.  When you find spelling errors in requirements, request to have them corrected and prevent this very annoying defect!  By the way, did you find both spelling errors in this paragraph?

Misplacement
When a paragraph or image appears in an odd place on the page, it’s easy to see.  When the placement is off by a few pixels, it’s much more challenging.  While a small difference might impact the page by moving text or other images, page construction can be complex.  I recommend having both a sample of the page from your User Experience designers and the page constructed by the program available on your machine.  Move back and forth between the two using Alt-Tab to look for those subtle differences.

Absence
You complete the transaction and expect to see your result on the page.  Instead, you see nothing.  The developer checks the code and verifies that it should appear.  It’s possible that your result has been placed in a position located off the screen.  The program is only too happy to place your text past the edge of your screen – after all the position is just a number to the computer.  It knows nothing of screen size or boundaries.  This is especially true when positions are calculated and absolute.  You might use the developer tools to verify the text is a part of the page, and then collaborate with your developer to experiment with positioning.

Advertisements

No Limits #1

I made a wiring error that left this power supply useless.

I started from scratch to build this replacement.

Cool!  Now, build another one ’cause I need two.  That are almost alike.

This one needs a wicked heat sink.

How will all those components fit on this board?  Perhaps a different placement of the heat sink?  It seems I need more space.

I don’t have to use the original board.

When you think, believe, act, or talk with limits to methods, behaviors, or actions, you also limit your possible solutions.  See your world as if it always has fuzzy borders, as if the edges of your vision fades to a cloudy background full of possibilities, as if every challenge has multiple solutions.

As a tester, I am challenged frequently to scrutinize gherkin that describe what to build, software that was built using both knowledge, experience, and interpretation, and, most challenging of all, suggestions on what and how to test.

You start by stating you see things differently.  Engage your peers in a conversation to nourish your gherkin into robust examples, to explore new and changed code frequently, to understand alternative viewpoints on what others want to learn about product behaviors.

Then, challenge (respectfully but with assertiveness).  I say challenge because sometimes products are not as complex or interrelated as they seem, because a simpler test may lead to adventurous exploring, because you need to use your time wisely, and because the succeeding conversation may uncover the assumptions that have become hidden with familiarity or painful with production issues.

Uncover the perceived limits and unlock many potential solutions.

It’s almost bullying

We’ve heard it many times and we’ve been told many times.  Too many times.  Too many times testers believe when they are told “…there’s nothing to test.”  It’s that line used to brush a Tester aside or to subtly invite the Tester to leave.  I believe we reconsider the phrase and take back control of what we, as Testers, believe the testing opportunities are.

When you’re told there is nothing to test, close the card.  Stop the execution.  Find another card for that developer.  When they say there is nothing to test, they are telling you they have no work to perform on the story card.  They’re probably gold plating existing code and don’t want to be discovered.  It’s a no-work Iteration for them.

It might be worse.  When I hear that phrase, it’s as if they are suggesting I’m too, shall we say, inexperienced to understand how their work might be tested.  Why aren’t we insulted?  I think it is borderline bullying.  Who are they to determine what can or cannot be tested?  You’re the tester!

Lastly, let’s turns the tables on them.  When you hear that phrase, you might suggest that the complexity of the story card might be beyond their capabilities.  Something like “…you know, the work defined for the story card is pretty complex.  Maybe YOU aren’t the right person for the card.  There is nothing for YOU to code.”

You are an equal member of the project team and deserve respect for asking about the testing opportunities for any work defined in your project.  Asking questions is part of your job and part of your responsibility.  Question the testability.  Question the design.  Question the implementation.  Challenge their opinion of testing opportunities and change their opinion of you.

 

It’s About Information

What the hell?  I’m getting pressure to make a date?  To be clear, this is the date we discussed about six weeks ago, made assumptions about some tasks, made assumptions about availability, and made assumptions on what we knew.  Since then, tasks have required more duration because people are providing end-of-year feedback, we decided to have some people take training in preparation for the next release, and the vendor’s system has some behaviors we didn’t expect.

I should point out, because as a tester it is my responsibility to provide information, that our performance tests are incomplete (we have not executed the most important one), our production verification utility won’t be designed until the week before the date, and part our solution has some inconsistent behavior.

I’m satisfied if the former is acknowledged and the latter is understood because, as a tester, it is not my responsibility to decide the next step.

 

Perry Mason and Assumptions

That’s twice!  Twice this week my assumptions caused a bit of trouble.  Last Saturday, I placed an order for parts.  They are not expensive but I’m careful about my selections and want to verify multiple times that I have the right parts.  On Saturday, I was confident in my selections and I placed the order.

On Sunday, I performed a test with a voltage regulator.  By design, voltage goes in and exactly 3.3 volts comes out.   I connected my 3.7 volt battery to the input and found 2.9 volts at the output.  My assumption was the battery voltage was greater than the output (that is, 3.7 > 3.3) so the regulator should work.  I didn’t have the right parts.  Fortunately, I was able to remedy this assumption by amending my order.

Today, I was updating a Java program.  I needed a string representation of date and time so I used the SimpleDateFormat object to create the representation.  I handed the format() method a Calendar object and started the program.  I was not getting the expected result.  Ignoring sarcastic little voices, I reviewed my changes and added some logging.  The voices were correct; it was the format() method.  My assumption was I needed only to provide the method a Calendar object.  It needed a DateTime object.  Fortunately, I was able to remedy this assumption.

Assumptions are sneaky.  Sometimes we know about them and discuss them, other times they drift in like a slow forming fog.  You don’t know its there but you discern some challenge.  It’s too subtle to change a direction but you’re cognizant of it.

I have many example of this experience through my recent Earl Stanley Gardner binge reading.  Gardner brought us Perry Mason.  If you are aware of Perry Mason only as a TV series, I strongly recommend the books.  They are engaging, humorous, and full of clues.  There are strong analogs to story cards.  When a story card goes awry, check assumptions and check the story.  One or two facts get misaligned and a defect is created.  Gardner weaves multiple facts and multiple explanations for facts.  It the end, Perry Mason, in the role of tester, fights through the fog of assumption, illuminates small details, and finds the one true story to exonerate his client.

As a tester, you will benefit greatly by having your assumptions dashed, and attention to detail challenged.  This week, two assumptions caused some delay in executing my hobby.  What assumptions might we be making that could cause a delay in our projects?

 

Goldilocks and Test Automation

I’ve struggled with the right level of automation for a long time and alluded to a level of salvation here.  Since that post, I’ve moved on to engage automation using Given-When-Then (and its collateral ideas) much like a sailor engages a lighthouse at night.  In my most recent experience, it was more like a sailor is on the bridge watching the light and I’m below deck providing interpretation and direction.

My testing team created an excellent suite of automated regression tests for the stored procedures used in our application.  But I’m struggling with the number of tests created for this suite.

It might be that my definition of “regression tests” is more economical than that of my teammates and I own that communication error: I probably was not clear on what I thought this task should produce.  But it also goes to the heart of what I believe about automation tests.

In 2007, I, like many others, was enamored with automated tests against the UI of my applications.  A steady diet of automation tools over the succeeding years (starting with HP QTP) were both valuable and frustrating.  I eventually gave into the frustration because the tests I, and many others, wrote were fragile and numerous.  My response was to limit what I automated because inflicting such a level  of maintenance on a product team was just wrong.

I watched as other teams continued to indulge in not only creating these tests but to develop frameworks around frameworks that provided some theoretical value and, in my opinion, dubious claims of a positive ROI.  I was unimpressed with what appeared to be the emperor (in their “new clothes”) dining happily on over-heated porridge.

With training and some practice in Given-When-Then and its description of business outcomes, I started to see practicality again in test automation.  A test that evaluates business outcome is, in my opinion, more stable, valuable, and maintainable than several poorly designed automated UI tests.  Then, there are two questions.  Always.  Two.  Questions.

Can I automate this test?  Of course, I can automate just about any test.  The avalanche of software utilities that pop up around tools for test automation is evidence of over-indulgence in coding.  Not testing.  The second question is most key:  Should I automate this test?

I advocate for tests designed for simplicity and to exercise isolated business outcomes.  The tests should be motivated by risk, value, and maintenance.  Upon a review of these criteria, I stand the best chance of balancing tests (the most value) and the code (that which must be maintained to deliver the test).

My team and I will meet to review our regression suite.  I look forward to exploring differing definitions of “regression test”.  I welcome their perspectives that could help all of us discover the combination of test and code that is “just right”.

My GWT Identity Crisis

There are a few things I’ve not tried yet.  Sadly, possibly, I’ve not tried karaoke, and I’ve not tried kite boarding.  I am however getting my first experience in applying Given-When-Then in describing product behaviors on a real project.  I’m very excited at the possibility of merging business behavior with automation.

I engaged Specification By Example, I attended hands-on training, and I have participated in periodic drive-by creations of GWT scenarios.  I felt very ready.  Hurtling into some of our project’s first story cards, I was shocked to review GWT that contained behavior AND, take a deep breath, implementation details.

As an example, I offer a scenario from an API I wrote for a website.

GIVEN active flights exist,

WHEN the website requests a list of active flights,

THEN the reply includes all active flights

AND their unique identifier,

AND their altitude,

AND their last update.

This GWT represents my interpretation of what I have learned and, as noted above, my limited experience.  Here is how the same scenario might appear if I violate one of the most fundamental rules.

GIVEN the active_flight field indicates true,

WHEN the website executes GET against the activelist API,

THEN the response contains an HTML table detailing the flight’s GUID,

AND the flight’s altitude as an integer,

AND the last update in a UTC format.

In my opinion, this is more than a violation of the “no implementation details” rule.  This description locks the behavior to a specific column of a database, defines the verb and API name, prescribes a specific rendering method, and calls out individual field data types.  Its as if a developer provided the information they needed to implement the scenario to an analyst.

While this is clearly an extreme example, I recently read such a scenario.  When I asked, the reason implementation details were included was because this is a “technical story card” (different from a “business story card”), and because it is easier to provide all the information in one place.

(ahem) Bull.  Shit.

In my opinion, this method of practicing and segregating GWT ignores one of its most quintessential and valuable aspects: collaboration.  Yes, I read Specification By Example and attended training.  I learned the mechanics but in both cases I left with something more.

I perceived an overwhelming feeling that during a conversation about product details between an analyst, a tester, and a developer, these few people begin to share not only their ideas but their understanding.  It IS NOT the GWT itself that encourages automation or reduces defects rather it is those precious minutes where an honest, introspective, and respectful dialog clarifies the behavior of a simple product.

Just writing GWT provides a small window into a product.  Collaborating over the value of a specific business behavior encourages teamwork and quality where everyone can take pride and feel their contribution matters.