The first thing to be mindful of when it comes to privacy is the simple fact that you are dealing with specific legal constraints upon what is and is not allowed. Therefore unless you have dealt with regulatory issues before, you are going to need somewhat of a different mental model for deriving requirements and testing.
Your location, the type of application (web or desktop), does the app require / transmit personal data, where the servers are and many other factors; will determine which, if any, laws you need to become familiar with.
Consider the following scenario, a web based app that is created by developers in country X with users around the world, and hosted by a third party cloud provider, whose physical server location may change on the fly. Which jurisdictions laws do you need to worry about? Probably all of them.
Fundamentally therefore privacy considerations need to be baked into the architecture and design from day one. There are far too many ways for personally identifiable information to leak otherwise. It is also not possible to secure this data in isolation, there needs to be an understanding of the risks from top to bottom of the organization (large or small) on who may get access to this.
Privacy testing is really a combination of the following:
- Store only as little information as needed. You can't leak what you don't know.
- Compliance with relevant regulations for what information you do store
- Prevention of unintentional leaks e.g. security testing for info leaks, SQL injection etc.
- Methods to identify intentional leaking. Monitor for unusual data movement patterns, back doors...
- Recovery procedures in the case of a data leak. This is technically covered by item 2 but worth reiterating. There are, depending on jurisdiction, legal obligations regarding notification of data loss.