Switching two screens feature and 13 bugs

The feature was supposed to switch between two screens, let’s call them “SCI” and “SCH”. I found 13 bugs not looking just for SCI / SCH, but going what I imagined as user flows, spontaneous ways and applying some of my experience.

  1. Icon for SCI screen was not updated
  2. Instruction text was not updated
  3. Invisible control: Interactions with input in bottom of SCI screen were not possible or possible only partially, because of overlapping invisible control. To catch this bug one had to add special settings for SCI.
  4. Too long time to the next screen from SCI for clients (because of merge error app was requesting all the type S data from organization instance)
  5. Empty SCI screen appeared after sending app to background and returning back
  6. Lock screen appeared if app got put to sleep after pressing power button (app should not be locked in SCI mode)
  7. After timeout exit on passage app returns to screen SL, not SCI
  8. (minor) After updating SCH data order of data with the same time may change in SCH.
  9. (crash) SCI with multi-source data -> SCH -> choosing the same client again -> cancelling -> crash
  10. Multi-source SCI does not result in successful passage if first source chosen has no SCI option set
  11. (minor) Timeout action cleaning data from SCI screen does not close dialog “Whoops!”, what may create confusion for the next client.
  12. After attempt to fix issue 10 app started to crash on Multi-source SCI screen for type of client
  13. Old issue for SCH screen display with specific test data

and there was 14th not found by me (found later by other tester):
14. If only one source selected in Choose multiple sources, the next screen does not start in the chosen mode

Nothing “cool” is meant in this story. The story is about how non-standard, non-“factory” methods reveal a lot of things “factory” methods don’t.

Advertisements

Was it manual testing? (PHA)

“It isn’t what we don’t know that gives us trouble, it’s what we know that ain’t so.” – Will Rogers. Citation taken from “An Introduction to General Systems Thinking” by Gerald M. Weinberg

This time our brave tester gets a bunch of new stories, codename PHA. And…

  1. Terms for the stories were not agreed
  2. Data types the stories cover were not categorized and documented
  3. Developers didn’t get right data types scope
  4. Requirements were incomplete (like preservation of old functionality, new default states)
  5. Story included incorrect generalizing statements (not all data types were and could be covered, not all the same way)
  6. There were miscommunications between stakeholders
  7. For some unknown reason initial implementation ignored graphics created by designers

 

So our protagonist…

  1. Suggested new terms and argued for their usage
  2. Documented categorization, talked about it, created a document
  3. Doubted story description (too bad the miscommunication was found too late)
  4. Pointed at what was missing, so that went to later better descriptions
  5. Pointed at incorrect statements
  6. Talked to every stakeholder
  7. Insisted that initial implementation was wrong and brought developers to talk with designers and others
  8. And reported 9 more problems with implementation, partially fixed.

Was it “manual testing”?

On Rapid Software Testing class

Rapid Software Testing (RST), even in scope of the class, is a huge topic. My account is a glance of it, not even outline. If you want to know more, you definitely want to read open course description and materials.

RST class delivers on what it promises. That’s important. Again, you want to check the course materials on what is promised. To show a bit of what is promised, I’m using two slides from the course public slides.

RST slide 01

RST slide 02

That all gets demonstrated in the class.

RST lives up to its principles. Important part of the class is asking questions, not only about testing exercises, but about what you want to know of class topics and other topics of testing. Don’t hesitate to ask about what testing problems you may need help with. Important point is that you have high chance to get in the class explanations and examples you ask for.

RST does not have “finish point”. Learning must not stop on final class day. RST brings in many topics and it does not have to end. For unprepared people course may be a starting point, for people who followed RST before taking class it is like a waypoint. Both types get a lot to continue to learn and to develop.

As it is promised, every category of participants can benefit from RST class. Unprepared participants get something new to think about, see more ways to do testing. Prepared see how this works in action and practice it. Even if you listened to several talks / podcasts of RST, “it’s one thing to read about riding a bicycle and completely different thing to ride a bicycle yourself”.

Unprepared participants can see that testing is way more than “test cases”, get opportunity to break free from factory and test case thinking. They also test in exercises for 3 days without writing single test case. Prepared get tricky exercises and “food for thinking”, practice testing in sessions and in cooperation with other participants in class.

Some topics we learned and discussed: what’s wrong with test cases, how to start testing something instantly without following lengthy procedures, where to get information for your testing, what your oracles (entities determining what is a problem and what is not) could be, when to stop testing, how to put coverage without counting test cases, what heuristics may aid your test ideas, how to think about product-associated risks, how to explain to others why testing takes so long…

Although you get certificate in the end of the class, IMO it is important to realize that RST is not about certification. You learn, you practice, you get leads to learn more, you get “entry ticket” to community where you can ask more. Thoughts, heuristics and connections you get with the course are much more important than any certificate.

Keep in mind that RST is a lot about time, “How use the time I get better”. While it may be not emphasized every time, the questions brought up in the Kiev class, like “Why not test cases?” and “How to learn newcomers?” or “Can we get rid of testing role?” are about use of time too.

Back in 1990s guys like me were fascinated a lot with action movies, since then we use meme “Show me your kung-fu!”. In some way RST is like kung-fu we saw in movies: there are no “kung-fu” cases, you learn from other people, you practice a lot, you have to be always prepared to show what you are capable of. Also, like martial arts have connection to thinking and philosophy, RST does too.

If you wonder “What the course content is about” and you are interested to learn more about RST, once more, please read open course materials and documents (section “How Students Should Prepare”) .

Reasons why linking automated checks to test management system could be not good idea

My answer to question “Do you know test case management tool which can create & execute both manual and automated tests?”

I saw such, let’s call them integrations, and I strongly advise not to connect automated checks to test case / test run / test management systems.

First I’ll name the integrations I saw by test management software participating in the integrations

Quality Center (by the time — HP Quality Center)

In my experience it

  • was laggy (slow)
  • was Internet Exlorer-dependent
  • was cumbersome
  • had issues with Unicode support

No one on team liked it, other teams disliked it too. Everyone wanted to avoid it, because composing test runs in it was a pain.

TestRail

TestRail is much better software than QC in terms of browsers support, ease of use, responsiveness and speed, but has its limitations (like any other software). If you have or going to have a lot of test cases (for whatever reason), you going to need good structure for them, search on big databases is slow, and editing / changing test runs takes time and effort.


Of the two integrations I saw every was eventually abandoned. 

Before you start any integration, you better consider what do you want it to deliver and what disadvantages it may bring. The main source of disadvantage are that

  • there are no perfect APIs with instant response and delivery

    Even if you perceive system response as instant, it takes hundreds of milliseconds, and in the course of run they stack. That may be good enough for tests you do / mark / report yourself, but if your automated checks run contains hundreds of them, this takes a toll.

  • Web interfaces to work with “test runs” leave a lot to be desired in terms of ease and responsiveness.

The integrations I saw had issues:

  • Composing “test runs” in system may be slow unless it is trivial
  • Editing “test runs” may be slow and inconvenient
  • If your automated checks run is bound to obtain information from test management system, every run will be slower, just because of what I said above: no instant response, every request adds time to the run.
  • For the same reason getting report from test management system may slow for you getting results of your runs (run results have to be messaged back via APIs from whatever runs automated checks, and processed by test management system software)
  • The test management system have limited number of reports with limited formats, which may not suit for your automation checking needs
  • Integration connectors (the software between automated checks run software and test management software) has issues too. Software which is not supposed to look for problems does not solve problems in other software, it may add problems.

Please ask yourself “What do I want to achieve? What do I want my team to have and to do?” and think it over for a while.

  • Do you want people who will do check automation run and debug tests faster and in more convenient way? They need independent run management means for that, nothing like any test management system existing.
  • Do you want checks run faster on some CI system? Those runs better be independent of any test management systems.
  • What reports do you need? What reports does your team need? Are you sure that the system you want will deliver what you and them need? Maybe some report tool will do that faster and in more convenient way, without overloading some test management system database?

Look for getting possibilities and avoiding issues. In my experience that way led out of integrations with test management systems.

And I ask you to consider if test cases are the right tools for the jobs you want to be done.

 

“Stack of books” heuristic

Facing some problem like mathematical problems, I take stack of books on subject / problem. I open the top one, and look if it applies to my problem, if it is written clear enough for me, if it maybe contains solved problems like mine. If it does not, I put that book away and take the next from the stack. For more or less standard problems it should be somewhere. That worked for my mathematical problems in school, later for my mathematical problems in university, and it works for my IT / programming learning tasks.

For IT / programming tasks it may be not stack of books, but a number of e-guides, tutorials, articles, posts. For instance, it worked when I needed to learn something about SOAP protocol and services. I looked into one article, than one more, third, forth and finally found good enough explanation to build some concept of it in my head.

I often recall it when I see someone asking advice on “best” course or book on some topic. There could be not one book best for all. You better use a stack to find some which best for you.

Why grumpy?

Again and again people ask me why I always say that there is no such thing as good morning and why I’m grumpy in general.

“You know”, I say, “with my job I (am) …

  • Look for problems
  • Find problems
  • Talk about problems
  • Supposed to find as many problems as I can
  • Supposed to find most serious problems (like crashes) and talk about them
  • Afraid I miss important problem
  • Afraid that problem will be found not by me, but by someone else
  • Expected to win a “race” and find more problems with feature or product than everyone else
  • Nervous if I find too few problems
  • Think where more problems can be found
  • Motivate / instruct / learn how to find even more problems
  • Better remember every problem I found in years of my experience, as some problem somewhere can be the same or found with the same means / ways
  • I should not believe anyone, unless I really see only minor change with my own eyes. Even if one of the programmers says that he talked to the designer and checked with her, I should not believe it, I should check with designer myself (and find some minor but problems).

“If here were no problems, you would not see me here”, I add, “Because where there are no problems, people don’t need to hire me”.

If they persist, I ask “If you were given $1,000,000 yesterday, would we talk now?”. Most just laugh. “Good morning is where would you be if you were given $1,000,000 yesterday. But you are here. That means that something is not good”.

My first page at Wordpress

Hello there! My name is Roman Podolyan I work in Software Testing and this blog is dedicated to testing and mostly computer stuff I come in touch and want to write in blog about.

My work in Software Testing lasts for 10 years so far, during that journey I found some interesting things to share and write about, and gathered some experience, which, I hope, will be useful for some readers.

fractal04