For the first time my team is going to take over a project from another company. Our goal is to improve quality - as I heard the project has a lot of bugs, UX issues and its performance is very low... Would you give me some ideas/best practices how to start the improvement strategy? IMO we should start with legacy bugs first but what processes/tools would be the best?
Add a comment
|
5 Answers
Start with understanding what works:
- What data is valid ?
- What data is used ?
- What is successful ?
- What are the variations ?
- What are the users like ?
- What are the long term business goals ?
- What are the short term business goals ?
Goals can be very different across industry, company stage, location, profit vs non-profit, etc. Learn what quality means to your company and then learn what the software does and then look at what are considered to be bugs. You'll approach them with the right mindset that way. Maybe losing 10% of users at signup isn't a bug 'cos competitors only reach 74% (lose 26%), etc.
After some analysis, work with the current team to prioritize and tackle bugs. Be methodological about them and be sure to also do root cause analysis so more are not piling up. Remember that tests show quality but do not improve it, work on app code is needed for that. Also do postmortems for bugs going forward and use it to identify longer-term issues. Also consider technical retro's for design improvement idea generation.
For any improvement strategy, measure it.
Just don't use number of bugs please.
Use project tools such as Jira, Pivotal Tracker, Trello, VSTS, etc. to not only handle tickets but to look at trends over time. Two long term ones to watch are effect on customers (more than just 'bug count' though, include severity and impact measures) and backlog count. It takes a lot of hard work, but over the long term these should stay 'flat' and not be always increasing. We'll add it to the backlog is one of the most comforting lies we make. I HAVE DONE THIS (avoided increasing counts - well, ok, lied about the backlog too), but only in about 10% of projects that had with a mature (software development experience wise) team and/or leadership.
Improve quality, don't just add tests.
Tests measure quality but don't improve it without further actions.
Make sure your organization focuses on quality, of which testing is a small part:
answered Nov 26, 2019 at 19:34
Your project sounds like where my project was when I started. You need to maintain your discipline. Here are the mistakes I've made, the lessons learnt, and what I've improved:
- It's easy to think "I'll remember bug X", or "I'll do it later", but in reality you'll be overwhelmed and forget. Use a bug tracker, and record each bug as soon as it's discovered! We use Trello as a simple bug/task tracker with the goal to move each task from the left side of the board to the right side, with "Done" meaning released to production and fully working.
- Explain how to reproduce the bug! I've reviewed some of my old Trello cards and they're useless because I didn't explain how to trigger the bug, assuming that it would be obvious or that I'd always remember!
- Get automated testing in place. I'm going to assume you don't have any to start with. It took me four months to begin unit testing our project. Legacy code is test-resistant, so it will take a while. I recommend "Working Effectively with Legacy Code" by Michael C. Feathers. That book is full of practical tips on how to test the untestable!
- Communicate with the stakeholders. Make sure they understand that in its current state, any work is incredibly risky. They need to accept that it needs a lot of work once you have tests in place to make the code maintainable. If you don't have support from the stakeholders they will pressure you into making the code worse without realising that it what they're doing. Also explain that in its current state all work will take much longer to complete, and will only get worse over time. You need to massage the code into a minimum maintainable state before you start adding features and fixing bugs. It doesn't have to be the whole code base all at once. Hopefully your project is somewhat modular so you can fix it piecemeal as you go along. My project is up to about 3% covered by tests, although that is not the only metric we use. I realised recently that even though we were communicating and the stakeholders are on board with the process, the stakeholders didn't truly understand how much work and how long it would take.
- Talk to as many different people as you can who have knowledge about the project. Get user feedback. Talk to the engineers if you can. Understand what the project is supposed to do for everything that you work on. Understand the technology it uses. If the code works by accident (no historical inputs have broken the code, despite the methodology being wrong) that's a future bug waiting to happen! Don't ignore it! Don't rely on an upstream input knowing the exact special format your code needs. Use standard formats where possible, e.g. JSON, not "sort of JSON sometimes".
Lastly, keep your focus! The more you improve quality the faster you'll be able to work. You're doing this for yourself, so you can serve the project better.
best practices on how to start the improvement strategy?
what processes/tools would be the best?
Nobody will be able to give a precise answer for this, because the value of any practice depends on its context.
some ideas on how to start the improvement strategy?
As said above, you will need to understand your context in order to evaluate the value of your practices. In terms of the product, one way to introduce yourself to it is through touring, using some heuristics to explore the terrain.
With the understanding of your product and other important factors, you can create a strategy, maybe in the format of the Heuristic Testing Strategy Model, so you can have a vision of your scope and techniques.
And since projects unfold over time in ways that are often not predictable and the best architectures, requirements, and designs emerge from self-organizing teams, you should experiment with different techniques and approaches in order to discover what works better in your context.
answered Nov 26, 2019 at 19:40
Context: So recently I was on a project as a consultant, to help improve reliability and scalability (fix bugs and make it go fast). I turned around their product, with a 800% performance increase and 70% reduction in errors.
These are the biggest takeaways I have from that:
Add a Real User Monitoring (RUM) and Error Monitoring tool. I used raygun.io I can not recommend the product enough! You will want to do this a soon as you can. It will give you valuable information and will give you a baseline, of errors per day and mean (& p90 p99) request time. With raygun you can see which url endpoints (I am assuming this is a webapp) are the slowest and most frequently hit. A tool like this will give you direct evidence of where errors are occurring. This will help direct what to work on, to get the 'biggest bang for your buck'.
Get a continuous deployment / devops pipeline, with blue green deployment. What this means is you have two instances of your app running. When you perform merge features or bug fixes into your release/master branch, a 'release' is created. One site instance is stopped (traffic is diverted to the other live instance). The new version is deployed, the new app is started and traffic is directed to the new version. The old version is now taken down (rinse and repeat). This will let you deploy small (lower risk) changes frequently; any time of the day. I will also let you restart apps which are melting down (e.g., a have memory leaks, a problem we had and eventually fixed) with minimal disruption.
Carve out some time for fixing bugs and performance. Clients/customers almost always want to see the new features delivered ASAP. Even at the expense of quality and performance. To the detriment of the whole project/company. Which hey, from their point of view, is quite rational! So you are either, going to need to get agreement from the business/stakeholders that XX% of the development effort is going to be allocated to quality and performance improvements, which the QA and Dev team get to prioritize. Or you are going to need to get your developers to do the work on the sly. What I mean is, lets say feature XYZ is estimated, by the developer, to take 8 hrs effort. Allocate them 12 hrs and get them to fix a performance issue you have identified together, (from looking at the Raygun reports), which is close to the area they will be working on the feature. Think of it as targeted code gardening Do what you need to do, working in the clients best interest. Failure to fix bugs and performance has bankrupted many a software company.
Gather evidence of the improvements you are making (% reduction of daily errors, getting the mean request time down by 50%, getting the p99 time down to under 500ms, etc etc) This will justify the time allocated or spent on the sly working on quality.
If you only take one thing away from this please add a Real User Monitoring (RUM) and Error Monitoring tool like raygun.io it will be the single most beneficial thing to help you plan how to tackle a quality and performance problem like this.
The following list of processes/steps would definitely help to improvise the current stage of the application especially when your team is planning to take over the project from different companies.
Split the test process into two phases, First Phase consists of identity/uncover the issues which currently exist in the system. The second phase consist of the Retesting of Fixed defects & Automation Test Suite for regression. Each stage is important to build confidence with client to move on with continuous engagement model
Phase 1 -
Explore the application and understand the requirement. Estimate for the round of end to end check and get approval from the client
API Test:
Identify the various internal and external web services involved in the application Do a through API testing and identify issues and move on to functional execution
Functional Test:
Reuse the existing Test cases for Functional test case execution and Identify issues as much as possible in the estimated timeline. Increase the backlog items by posting all approved defects
Identify the gaps in the existing test cases. update and execute the same. Check the duplicate issues from the backlog and merge if possible
Exploratory Test:
Do a complete exploratory test and add issues if any. Identify the impact areas of the issues identified and do a thorough check on those areas as well and increase the number of issues at the initial stage Start to prepare the Regression automation test suite in parallel with functional testing. It will help when all the fixes have arrived for the next round of testing
Phase 2 –
Automation Test:
Start to execute the regression suite using automation tools and captured the results on a daily basis/ whenever the build deployed with fixes Retest the resolved defects on a daily basis/sprint basis and update Add new test case if any new feature is added as part of sprint activity Add test cases continuously whenever the change is happened in the existing flow and keep update the automation scripts as well periodically.
