Testing as an Advocacy

When you work in testing long enough, you develop or adopt positions on many aspects of testing.  Some positions may be influenced by your environment, some by your peers (respected and otherwise), and some by practice.  Work in testing longer and you may want to share your positions and occasionally encourage others to adopt them.  We see this with testing models or techniques, testing schools of thought, and testing automation.  I believe it is testing positions and the varied interpretations, discussions, and debates that makes the craft a satisfying career choice.

I believe it is becoming something more.

With the introduction and practice of agile methodologies (Scrum, DevOps, et al), the practice of testing inside those methodologies has grown in importance.  Testing must not “wait until the end”, or plan large testing events “after the code is deployed to the test server”.  Waiting impedes project pace by time spent waiting and by discovering defects that were in the code while we were waiting.
The demands of business are driving the adopting of CI/CD and testing can not only help with that, it can drive it.  Testing must drive project pace and to drive project pace, we must become advocates.

Testers advocate for testability, shift left, and buying more than building.

Testability
Testers must advocate for testability in requirements and designs.  Testing can no longer afford to wait for information.  Rather, it must influence requirements for a single clarity and collaborate with team members to share that clarity.
Additionally, testers must actively participate is product designs (high level and detailed) to influence them for testability.  Request the design be transparent with key information and behaviors by using some form of logging, and request the design be controllable by having the ability to mock product objects.  When a product has good testability, it easier to test and can be tested earlier and quicker.

Shift Left
Testers must advocate for Shift Left.  Shift Left encourages new and changed products be evaluated closer to construction.  In many cases, this means more unit tests and deeper unit tests.  In some cases, the unit tests become the regression suite that can execute at any time.  Regression no longer need occur at the end of development!  We need to know as soon as possible if a recent change has impacted the application.
If much of the testing is shifted left, what do testers do?  They are reviewing the unit tests and suggesting more, they are exploring risks, and they are exploring environmental dependencies such as security, configurations, and connectivity.

Buy More Than You Build
Testers must advocate for buying tools and utilities rather than requesting them be built.  There are certainly many cases where building a tool or utility makes sense.  For several other cases, buying tools and utilities can get testers testing quickly.

I believe by advocating some or all of these ideas, testers become project drivers and CI/CD supporters.

Lead on, Testers!

Advertisements

An RPA Journey

Last Summer, I was assigned to a Robotic Process Automation (RPA) project as the Lead Developer. RPA is a growing field because the automation can improve work throughput at a greater accuracy.

We recently completed the development and introduced our automated process.  A process that once require tens of hours was completed in minutes.  While it was very gratifying to deliver this product, I look forward to returning to test engineering soon.  As I take leave of this RPA world, I thought about the development experience and wanted to share some thoughts.

From Test Engineer to RPA Developer?
I was looking for a different gig within my company.  One platform was experimenting with RPA and thought a Test Engineer (that is, a person with automation experience) would be the best fit.  When I investigated the job a little more, there was some chance that I could explore AI.  I was offered the position and started learning about RPA and an RPA tool.

Emperor’s New Clothes
RPA is marketed as tools that can save labor, improve accuracy, and increase productivity at a lower cost all through automation.  In my opinion, all of that is true.  I also believe that RPA tools are marketed more towards the business side of an enterprise.  The sirenic user interface of an RPA tool may have you believe that anyone can create process automation.  The reality is that these tools are used to write programs, and writing programs – even RPA programs – is challenging.

The RPA tool is actually just another Integrated Development Environment (IDE).  Make no mistake – it is used to create scripts that interact with your applications in the same way as your employees.  In that sense, the creation of a script is very much a software development effort and must be treated as such.  There are no robots here and the use of the word “robot” is very misleading and should be considered window dressing.

If you believe that someone with Excel macro experience can use RPA tools to deliver high quality, error free automation, you will be, in many cases, profoundly disappointed.  Excel macros and RPA programs are on opposite ends of a development complexity spectrum.  Without an understanding of software development methodologies, basic programming concepts, data quality, or detailed process definitions, it is possible to realize significantly less benefit than expected.  In my opinion, the “best practices” recommended by RPA tool companies are a sad substitute for software development experience.

Early in the project, I found many similarities between what I could do in Visual Studio (also an IDE) in C# and the RPA tool.  With my experience in software development, I was able to learn the RPA tool quickly.  The RPA user interface is but an abstraction of basic programming concepts, and it reminded me of the Lego Mindstorms IDE used by many children to build and program small mechanical machines (these machines can perform some sophisticated activities as demonstrated annually by many First Lego League teams).
In my opinion, placed into the hands of a professional developer (or developers), the RPA tools can deliver great products that can benefit your enterprise.  However, that benefit can be realized only with a strong working partnership between the development team and your business team.

Tests and Testing Tell the Story
The first process we automated was complex.  It was as complex as a new API and warranted the normal approach of decomposing it into components.  The components were described in multiple story cards and we started construction.

At the end of a story card, we found it valuable to create unit tests for the components.  Even the tests were built in the RPA tool.  In this manner, we could evaluate multiple scenarios easily.  When we had a critical mass of components, the development team met in what we called an “Integration Session” to assemble the components into the automated process.

We evaluated the automated process with sample input data, and we were able to provide diverse scenarios that exercised the process.  We found defects and corrected them.  We were also able to demonstrate the automated process to our business team members frequently.

Discoveries Along the Way
The description of our development approach should sound familiar to agile practitioners.  Agile provided us the flexibility we needed to learn and adapt.

For example, we wrote code to collect a set information for a person from an application.  We discovered that, sometimes, there is more than one person.  We refactored to collect a set of information per person.

We wrote code to place numbers into a spreadsheet and retrieve formula results.  We discovered that, sometimes, the spreadsheet provides feedback and the formulas require a macro to make adjustments.  We refactored to detect the feedback and run the macro.

We continued in this way until we deployed the process and started using production data.  While we still made discoveries and refactored, we also realized that the cost of some changes may not have enough benefit to justify the change.  We discovered “done” and, as hard as it was, we needed to say “enough”.  That doesn’t mean our discoveries are not addressed rather it means the project met its goals.  The discoveries and enhancements will be addressed over time.

Right Size Your Team to the Process
When considering processes for RPA, the complexity and effort should help set the level of project management.
Our team is a handful of developers and business consultants.  To resolve details for our first process, we required information from our business consultants frequently.  Additionally, we relied on a separate team for test data.  Since RPA is a software development effort and the process complexity was high, I appreciated the benefits our project manager brought to our project.
However, the second process was far simpler.  In my opinion, it was simple enough that a two developers could complete it without the need for project management.

Celebrate The Team
Over the course of my project, the developers and I worked with some wonderfully gifted and passionate business people.  They knew their business processes, they knew some of the challenges, and they believed in the promise of process automation to improve their productivity.  Together, we defined, built, tested, and celebrated the products we created.

Deploy small, deploy often.

I recently a part of a production deployment.  It is not like a deployment of the early 2000s where everything went up and you expected it to work.  No.  It was deployed in parts because, well, we forgot stuff and we discovered stuff.

We deployed the product and started it up.  We discovered something we didn’t expect.  In the early 2000s, we would have executed the back out plan and deployed another day (a delay in business value).  However, this is 2019.  We made a correction in production and tried again.  We did this several times until we finally had the entire product operating as expected.  Early reviews of its functions say that the product was even operating successfully (read: providing business value now).  I love testing in production!

Could we have had a better plan for deployment?  Yes!  Based on this experience, our next deployment will be better.  Ain’t that being agile?

Did we deploy the product that was approved by multiple interested parties?  Hell no!  We deployed something that demonstrated excellent functionality in a non-production environment.  When we deployed to production, we solved issues like forgotten configurations, forgotten files, and an unexpected data format.  That is, we LEARNED a lot about our product in another environment, corrected our mistakes and misses, and got it working.  When it started working, then the real learning started!

We discovered some things our product could not do because of data quality (by the way, our error handling caught these!).  These will be addressed in a future deployment.  We discovered some functionality that needs tweaking.  This will be addressed in a future deployment.

Going forward, we deploy small and deploy often.  We will learn as we did above because it is soooo valuable!  Testing in production is not a crime, it is a learning experience!  If you can, go forth and learn!

Here’s a New Idea You’ll Love!

Think about the last time you heard an idea or suggestion that was extremely, violently, and orthogonal to how you think, believe, or live.  Of course your reaction to it was welcoming, engagingly curious, and you had a deep, deep desire to try it.

Probably not.

I see this occasionally, as many testers might, when I suggest alternatives to existing or established testing practices.  In my experience, the reaction is usually negative.  Occasionally, there is so much push back that surmounting just the negativity would be a day’s work.

My approach has been to keep trying because I hope, perhaps a quixotic hope, that one day they may ask the key question.  I listen for this question every time I make my wild suggestions.  I hope, just once, they would ask how.

“Joe, how would that crazy idea be possible?”

For now, I wait until they are ready.  Forcing the idea and my notions of implementing it would waste time and drive them away.

How have you approach the suggestion of your crazy ideas?

The New QA

Our applications are growing larger and more complex in business functionality, and they are available on multiple platforms.  Our business partners expect more functionality with better quality delivered in a timely fashion while our technical leaders attempt to simplify designs and make it all possible.

The ability to deliver value in an application AND making it simple to use challenges experience designers, developers, analysts, and especially testers.  An expectation to evaluate application behaviors in depth through just the user interface has not only become a fallacy, it has become insufficient and risky.

High profile projects in your organization must be delivered quickly and as a tester, you cannot wait for completed code to begin testing, and you cannot assure quality merely by testing through a UI.  The definition of testing must extend beyond exercising code to evaluate requirements and risks.  To expect high quality products at a good pace, you must foster and nourish quality as a team sport.  Your time is now and project teams are ready to hear you!

Testing now begins when work is assigned.  You are the Question Asker and you scrutinize acceptance criteria.
Soon after the assignment of a story card (or your method of work conveyance upon your team), the Three Amigos is your first opportunity to foster team quality, your first opportunity to “test”.  Your tests include

• A check for acceptance criteria clarity
  • Does everyone have the same, deep understanding of WHAT is being built?
  • Have those understandings been verbally explored and vetted?
  • Have you vocalized your assumptions and had them clarified?
• A check for validity
  • Is this product being constructed at the right time?
  • Are there dependencies that could prevent its timely completion?
• A check for value
  • Some products are defined at some time in the past – does the present context of the project still support the work defined by this card?
  • Will the completed work still be valuable to the project and the product?

Testing continues as products are designed.  You are the Quality Advocate for testability in designs and implementations.
Most products require some thought before their construction.  The project team explores possibilities for implementation during design meetings.  Your testing continues at these meetings.  While you could also attend design reviews, a design review may imply that the design has been decided – this is too late.
When the design of a product is discussed, your primary goal is the testability of the product.  If the design addresses just the business intent of the product, then it is not testable.  As a tester, advocate for testability.  Make a request for error logging, for transaction transparency, and for data transparency.  An introspective product is a testable product.

During implementation, paired programming becomes paired collaboration.  You are the Quality Accomplice with simple tests at the ready and encouraging reflection on how the product in front of you will interact with the system.
Quality must be built into a product: the product is defined in a manner that is clear to everyone, the product is designed so that inspection is simple, and the product is constructed and checked multiple times during its implementation.  As a tester, you are the Quality Accomplice by having tests ready for execution at any time, and by working with the developer to help them create a quality product that meets the business intent.  To work effectively with a developer, you must understand the system and its components, and how the components interact to provide value.  With that understanding, you can ask questions how the code in component A will interact with the code in component B under multiple scenarios.  This collaboration helps everyone reflect on the integration of small parts defined in story card and their contribution to a product that operates correctly in multiple scenarios.

I encourage you to go forth and experiment with these ideas.

 

 

A technical tip all testers should know

A Ministry of Testing Club post suggested we pen some thoughts around the topic title above.  In addition, it encourages us to blog for the first time or to start again.

 

Technical Tip: Never Believe What You See on the Screen

Your user interface will, occasionally, lie to you.  You can’t trust it.  The information you see on your screen will suffer from staleness, subtle misspellings, misplacement, absence, and other challenges that infect the journey of data from where it originates to where it’s displayed.

Staleness
You enter data, run the transaction, and the result is 14.02.  You enter different data, run the transaction again, and the result is 14.02.  It’s probably a defect.  Verify the result at its source!  Maybe the database was not updated, or maybe the code that moves the result from memory to the screen didn’t execute or executed incorrectly.  Collaborate with your developer to find the answer!

Subtle Missppellings
Many times the text that appears on the screen was copied from requirements.  Who cares if the requirements are spelled wrong?  You can’t beleive how often this happens but often enough that I check the spelling of text destined for screen when it appears in the requirements.  When you find spelling errors in requirements, request to have them corrected and prevent this very annoying defect!  By the way, did you find both spelling errors in this paragraph?

Misplacement
When a paragraph or image appears in an odd place on the page, it’s easy to see.  When the placement is off by a few pixels, it’s much more challenging.  While a small difference might impact the page by moving text or other images, page construction can be complex.  I recommend having both a sample of the page from your User Experience designers and the page constructed by the program available on your machine.  Move back and forth between the two using Alt-Tab to look for those subtle differences.

Absence
You complete the transaction and expect to see your result on the page.  Instead, you see nothing.  The developer checks the code and verifies that it should appear.  It’s possible that your result has been placed in a position located off the screen.  The program is only too happy to place your text past the edge of your screen – after all the position is just a number to the computer.  It knows nothing of screen size or boundaries.  This is especially true when positions are calculated and absolute.  You might use the developer tools to verify the text is a part of the page, and then collaborate with your developer to experiment with positioning.

No Limits #1

I made a wiring error that left this power supply useless.

I started from scratch to build this replacement.

Cool!  Now, build another one ’cause I need two.  That are almost alike.

This one needs a wicked heat sink.

How will all those components fit on this board?  Perhaps a different placement of the heat sink?  It seems I need more space.

I don’t have to use the original board.

When you think, believe, act, or talk with limits to methods, behaviors, or actions, you also limit your possible solutions.  See your world as if it always has fuzzy borders, as if the edges of your vision fades to a cloudy background full of possibilities, as if every challenge has multiple solutions.

As a tester, I am challenged frequently to scrutinize gherkin that describe what to build, software that was built using both knowledge, experience, and interpretation, and, most challenging of all, suggestions on what and how to test.

You start by stating you see things differently.  Engage your peers in a conversation to nourish your gherkin into robust examples, to explore new and changed code frequently, to understand alternative viewpoints on what others want to learn about product behaviors.

Then, challenge (respectfully but with assertiveness).  I say challenge because sometimes products are not as complex or interrelated as they seem, because a simpler test may lead to adventurous exploring, because you need to use your time wisely, and because the succeeding conversation may uncover the assumptions that have become hidden with familiarity or painful with production issues.

Uncover the perceived limits and unlock many potential solutions.