UAT involvement

Question

This isn't really a question for me anymore, but, out of curiousity and to help a former co-worker.

Before moving to Quality Assurance, I led a UAT team. This team was made up of former users, who didn't actually use the systems that they were testing anymore, except for when they were testing it. This seems odd to me, and I always got push back from the Business on engaging actual users. Is this common?

On to my real question though, how often should UAT be utilized. Currently, within my organization, UAT is only utilized when changes are made to SOX (Sarbanes–Oxley) impacting applications, and SOX impacting changes made to other applications. Although I'd like to involve UAT more often, I receive push back from the business. I'd like to start using them for any changes that impact a customer or end user. What is considered normal for involving User Acceptance Testing?

Answer 1

One of the key areas I have seen improvements to applications is when end users are actively involved testing and development, ideally from start to finish as part of the team.

The more removed that users are from the development and implementation and testing the less likely it is that the application will meet their business need.

I have actually seen, quite commonly, both extremes:

• End users are involved too much in testing (i.e. work goes straight from developers to UAT, by-passing any normal system testing and wastes a lot of business users time). Eventually this will lead to the situation that you describe when they will eventually say "enough is enough" and stop testing. This can then lead to the other extreme

• Users are not involved at all and an application is then forced on them simply doesn't work and is abandoned

My advice in your situation would be to fight to get access to the users, or a proxy for the users, such as a good business analyst.

You should acknowledge that their time is extremely valuable and then capture from them what they want tested, and essentially perform UAT on their behalf. Then, after it all is fixed and working, show them the results, and when they actually do their own testing (if they do) it should simply be a quick rubber stamp process to accept an already working system. This is what, IMHO, UAT should be all about.

Answer 2

Disclaimer This is solely based on my experience. I'm sure there are others who will have a radically different experience with trying to get good UAT feedback.

Note I consider UAT to be more informal than formal. I'd love formal UAT, but there simply isn't budget for that where I am. :-(

Unfortunately, in a situation where SOX compliance is involved, you're likely to have (by the nature of people who are going after SOX compliance being large companies) many layers of separation between you and the users.

For example at my firm (a multi-national manufacturing firm) we have developers --> dev manager --> Business Analyst --> Manufacturing plant manager --> users (and sometimes, there's a layer between the users and the plant managers.) I can't imagine yours is any different.

You simply have to take what you can get. You won't get 1 on 1 time from the business side. That time would have to be allocated by the development time. They might be willing to spare an hour of some order-entry person's time to show you how they're actually using the application, but they're certainly not paying for a developer's time. They have a plant to run, and you software-people already cost them too much money. (That's actually how they see us.)

I think it's safe to say there is no "normal" for UAT. You have some companies that make it a big deal. They get focus groups, they get feedback, they do surveys, provide incentives. You have others where it's simply not a big deal at all. They really don't care as long as it's "doing the job." They define doing the job as "the people using the software haven't complained about it."

The smaller the company, and the smaller the userbase, and the more public the userbase, the more UAT will probably be involved. The larger the company, the easier it is to say "You want more? Well, that's too bad." Same with having large userbases - you don't care if a couple leave. It's not worth the time compared to bringing in more people with new features. If you have a public userbase, they have more of an option to switch to something else. If it's in house software, they don't have much choice, so development has more power to just shut them down.

Personally, we would love to hear from our users, even though we fall into the category (by my criteria above) that can abuse them the worst: large ($2b+) large (7k+ users) private (in house). It's actually not us, but the layers of management in between that obfuscate the true feelings of the userbase, and the results of any UAT.

Answer 3

Unfortunately, we do not have formal UAT. A handful of support personnel walking through new features is the closest thing to UAT we do have.