6

I'm working at a startup company and feel like they get into a deadlock by testing UI in wrong ways. They are creating a brand new project from scratch with a new UI and this is how they perform testing:

  • They run constant ad hoc testing each sprint and open bugs if a specific color is off, even-though we are still in development and fully aware of some colors and shapes being off

  • They do regression testing to make sure previous functions are still working. But anyone who develops UI knows, one small css change somewhere might effect other elements on the page (temporarily). So we fix one thing but it creates a temporary bug on some other elements and we get 5 bugs opened because of them right away.

I feel like development spends more time to close these random bugs, which they are fully aware of, just to keep the QA team satisfied. If QA just waits until the project is close to completion and then test the fine tuning, then development could spend more time doing actual work, rather than close these bugs constantly.

What are the best ways of testing UI? do you test constantly, or do you wait until you have a proper product to test? Isn't testing UI different form testing a backend service?

Improve this question
3
  • It feels like the expected layout is not properly defined. Do have a wireframe HTML/CSS template? If not, I suggest that you start by creating one. You'll save a lot of back and forth between QA and Dev.
    – Florent B.
    Mar 30, 2018 at 15:13
  • 1
    we do , but this has to do with pixel perfect testing. If section 'A' of the ui is developed, they want it to remain pixel perfect (color, size, behaviour) all the time. but this is not how UI works (this is how backend works)
    – Daniel
    Mar 30, 2018 at 15:37
  • Do you have a separate QA environment, where code is pushed periodically?
    – Vishal Aggarwal
    Apr 3, 2018 at 11:10

6 Answers 6

Reset to default
1

There's a couple of issues here I see:

  • color. This stuff is worked out just like other design elements with the UI folks or the folks doing the UI if not separate roles.

  • brittle tests. This sounds like you have HTML element identifiers that are based on layout. So when the layout breaks (say for a form someone is filling in) the automation breaks, even though the form and the fields are essentially the same. So switch from layout identifiers and if necessary add a bunch of identifiers (usually id) to the critical form and action elements (buttons, links, etc) to address this.

The best way to test the UI is to make sure that you have UI tests that

  • Can run continuously in branches (post push to remote) using a CI system
  • Use good element identifiers that are robust and unique and not brittle
  • Use Page Objects to extract and name element identifiers
  • Use a custom DSL as appropriate to extract common functionality into named routines
  • Differentiate and tag test types such as smoke, happy, sad and optional
  • Are easy available and often in the same code repository as application code
  • Do not perform data combinatoric testing that should be done through unit and integrated tests
  • Do not duplicate unit and integrated tests
  • Do not otherwise perform the task of missing unit and integrated tests

Test constantly and don't wait. Test early and often. Shift Testing left. These are good guidelines to go by but there are subtle parts to the implementation of these practices.

The difference between this and testing a back end service is primary two things

  • speed. browsers tests take minutes and hours. unit/integrated take milliseconds and seconds
  • control. browsers are foreign asynchronous devices over a network that can drop packets.
Improve this answer
1

First of all, there is no "they" , it's always "We".

Having said that, in these situations usually I recommend "n-1" automation.

In this , QA manual is done in same sprint focusing more on functional testing and test automation is done for the previous completed sprint assuming they met DoD and UI is stable.

Improve this answer
0

Well, testers will report whatever discrepancy they observe in the UI but the question is about the process of handling it.

Whatever is pushed in testing environment should be testable. You guys are doing it the right way by reporting everything. But, it seems like an issue with development process that is resulting in side effects and in agile they will just focus on current sprint.

And yes, UI testing is different from backend service testing, but with UI, the scope of testing of impacting areas is huge. So, regression testing plays an important role here.

First let’s discuss the problem:

Ideally, the side effects of the UI changes should be minimum.

Following the UI style guide, developers write common code. For example; code for website banner is common. Font site, hexadecimal code and other related UI aspects of a section are setup in a way that every developer can use it whenever working on the banner part. Such code does not result in side-effects. So, if you make any change to this code, it will only affect every place wherever the banner appears but nothing else.

So, I am somehow concluding that the development process has issues.

If a developer is working on a UI piece for which he cannot use already defined style then he will not directly write the custom code but will request the UI designer for it. Designer will do some research on already defined styles and suggest which one the developer should use or will create a new style that adheres to developer needs and will also specify which other parts of application can use this new style. In our project, there was a dedicated designer responsible for managing the UI style as the project was UI heavy. And whenever, he changed a particular style, he exactly knew which parts of the application would be affected because of this change.

Now talking about the solution, if your UI code is frequently impacting multiple areas of application then that’s absolutely is an issue with development process. If developers don’t know about these side effects then that’s an issue and even if they know but park the side effects to handle in future then that is also not a good approach. Your team definitely need to sit and and come up with some suggestions how this can be handled.

Apart from this, what I would suggest is to let the testers log defects for the impacted areas (it is actually good that they are doing it). But fix only defects that are in scope of the sprint. Then have a stabilizing sprint after every 3 to 4 sprints where developers don’t pic any new work but fix the issues that are open (which were created by testers as side effect of particular piece of code). They should log every issue that they see but should focus on working on the ones that affect the current sprint. Any issue affecting the current sprint should get developer’s attention.

Improve this answer
5
  • Thank you for your answer. I have another suggestion, let me know what you think about it? I suggest to only test "functional" aspects of the sprint, for example, if the sprint says "but a red textbox on the UI", testers should test "if there is a textbox on the UI", color shouldn't matter because it's not functional. Leave all the coloring and pixel verification for the very end of the project when all the elements are put together. problem is, if management sees a defects they want it fixed. They don't care if it's regression, adhoc or anting else.
    – Daniel
    Mar 31, 2018 at 3:02
  • Okay. So, I have few questions here ( in order to understand your situation better). Is there a UI styleguide already in place? How are your user stories written? Do they include UI as well? In your sprint demo when you demonstrate your work to PO how much emphasis he/she puts on UI? What is the likelihood of your suggestion to be accepted? See, as per agile, the output of a sprint should be a shipable functionality related to the application under development. And moving UI to the end of the project will have its own adverse impacts.
    – Aalok
    Mar 31, 2018 at 12:39
  • No no i'm not suggesting to move the UI testing to the end. UI dev will take "color, and shapes" into account when working each sprint. Demos are very important for the management, but what i'm saying is, IF sometimes the colors are off, or the items are out of place (but still functional), don't open a defect and test these as a final test at the end. no point in pixel perfecting when there is more dev work to do on other elements
    – Daniel
    Mar 31, 2018 at 13:40
  • I have understood your point. Another point that I observed is the pressure from QA team because they will definitely keep on raising bugs. So, my suggestion is to talk to all stakeholders. Inform them that since dev guys are working on UI related items, so QA team might see some discrepancies in the UI. We would recommend that all such observations if raised as bugs, should be lined to XYZ story (create a user story for UI discrepancies). And say that “ Most of the issues would be resolved as the development progresses, but in case anything is left we will make sure that issue is addressed”
    – Aalok
    Mar 31, 2018 at 14:17
  • And also say that once we mark this new story as “Ready for QA”, then testing team can test all the associated defects. And it might take a few sprints before it is marked as “Ready for QA”Present the facts related to slowing down of development speed because of these distractions. I hope that stakeholders will understand. But the main point here is, discuss with Scrum master,developers and testers together. Don’t try to do this with only the testers. All the best!
    – Aalok
    Mar 31, 2018 at 14:20
0

1)Get someone to properly prioritize these bugs and look at the bigger picture and decide what is important, if these UI issues are not important then they should not even reach the dev team till necessary.

2) The other way around this would be to amend requirements (so they are more accurate because it would seem there is a misunderstanding between QA and the Devs) or get the dev team to create their own bugs for these issues that way QA know the devs are aware and its in the backlog.

Improve this answer
-1

The first point you have specified is likely caused by the fact you're either missing or have your DOD (Definition of Done) incomplete. Hence QA cannot get a proper signal if the functionality (the Story or whatever issue types you are using) is ready for testing. This is not the QA issue in particular but rather an issue of your processes which integrate different phases of SDLC.

The second issue seems fine to me. You cannot keep in mind all the places which have been affected by your CSS change. It is worth having all such the places documented. After all this is what Regression Testing is required for (the earlier you find a bug the better).

So my advice would be to fix the way how you describe the added functionality and specify the concrete DoD for each task. Is would also be useful to revise the workflow of your tasks in general because as it looks to me (from your description), you have the task which is incomplete from dev side but the QA alredy tests it.

Another option is to dedicate some person to review and filter the defects (QA Manager for example) so that Dev would get more relevant set of issues.

By the way if your QA works in a "manual" mode by the moment, then once they will have moved to Auto Tests the number of the issues will ony inrease and you'll be recalling the current state of the things as the old good times.

Improve this answer
7
  • We do have DOD, however, I feel like QA team wants to keep busy and just do random testing after DOD i s met. Does it make sense to do regression on a half finished UI? I mean regression on backend makes sense, but UI elements are all connected to each other. Shouldn't they wait until we have something proper put together first?
    – Daniel
    Mar 30, 2018 at 14:44
  • Probably your DoD is not detailed enough. Did you talk to your QA? What was their justification?
    – Alexey R.
    Mar 30, 2018 at 14:46
  • They basically say whatever that was working in the previous sprint, should also work by the end of this sprint. So "Stop whatever you're doing and fix this new bug", But, the new bug will be fixed automatically if normal development just continues and new features get added. This way we fix a bug but nothing else gets done.
    – Daniel
    Mar 30, 2018 at 14:48
  • Feel like they are just slowing Dev down
    – Daniel
    Mar 30, 2018 at 14:49
  • 1
    You could take the bug in Dev just to make your qa happy and proceed with your regular development. And close the bug once the regular development is done.
    – Alexey R.
    Mar 30, 2018 at 15:00
-1

Seems that your QA team has too much time on their hands, and not enough focus/experience.

Maybe they should focus on developing automation pipeline, mastering CI and reporting stuff like nagios, grafana/kibana, jenkins. Do they write automated regression tests?

Also, focusing on requirement analysis is perfect time during first few sprints. In every project, after reading requirements few times, I detected missing or contradictory items.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.