The best way to make an audit a positive experience for all individuals involved is for the auditee organization to make it as easy as possible for the auditor to draw a positive conclusion from the evidence being presented. This is best accomplished by providing high quality evidence for each and every control.
Eric Shoemaker – Advisory CISO – Genius GRC
Working with an independent audit firm that focuses on performing high-quality audits not only enhances the credibility and trustworthiness of the SOC 2 report but also provides the most value to organizations looking to strengthen their security, privacy, and operational practices.
Jesus Jimenez – Partner
I often get asked how to “pass” an audit. The reality is that most audits are not true pass/fail in the sense that we usually think of. In high school and college, there were tests with a question bank. If you get a certain number of questions correct, the professor issues a passing grade.
Auditors have a much tougher job to do. They are in a position where the test changes for each engagement. The evidence provided to them for evaluation is based upon the nuances of how the organization being audited conducts business and how their infrastructure is configured. This is true whether the auditor is auditing against SOC 2, ISO27001, NIST 800-53 or any other framework.
At the end of the day, the auditor’s job is to evaluate the evidence presented and issue an opinion about
the design and effectiveness of an organization’s information security controls and whether or not those controls effectively manage the risk an organization poses to their customers and consumers. They do this by considering the inherent risk of the service being provided by the organization along with the sensitivity of the data they process and therefore must protect. Each valid control builds assurance to the auditor that the organization is considering the risk and managing it acceptably.
Improve the Audit Experience Using High Quality Evidence
For each control being evaluated, the auditor will request evidence that (1) a control exists and (2) it is
operating effectively. One thing that irritates both organizations being audited and their auditor is when the two parties disagree about whether a given piece of evidence proves the effectiveness of the control. In these instances, the auditor may request different evidence altogether or request additional details be provided with the existing evidence.
The best way to make an audit a positive experience for all individuals involved is for the auditee organization to make it as easy as possible for the auditor to draw a positive conclusion from the evidence being presented. This is best accomplished by providing high quality evidence for each and every control. As an easy example, consider the following SOC 2 control taken from the Vanta SOC 2 framework.
The company requires passwords for in-scope system components to be configured according to the company’s policy.
One of the nice things about the Vanta platform (and others like it) is that it will constantly monitor integrated applications to ensure that the application matches the configured password policy. This is the best kind of evidence because the auditor is able to view a timeline from the entire audit period and trust that the configured password policy was in place for the duration of the audited period and meets policy expectations.
Consider applications that have no direct integration. For those, a common practice is to provide a screenshot of the configured policy. The most common application for which this approach is taken is on-premise Active Directory. The “Default Domain Policy” GPO primarily houses the password policy, and auditors are accustomed to viewing screenshots of the password policy portion of the GPO.
Example: Poor Quality Evidence
The prior screenshot shows a poor example of evidence being provided. After reviewing this evidence, the auditor may have the following questions and will likely request a new screenshot:
1. How do I know this screenshot was taken for the organization being audited?
- This is a valid question as the screenshot doesn’t provide enough detail to demonstrate that it was in fact a representation of the organization’s environment.
2. When was this screenshot taken?
- Without a timestamp in the screenshot, the auditor may wonder if it was taken recently enough to be valid evidence of the current state.
Using this screenshot as evidence is highly likely to result in the auditor asking to review the GPO directly or to request a higher quality screenshot.
Example: Better Quality Evidence
This screenshot shows everything the auditor needs to see, and the auditor is likely to accept it as valid evidence. With that said, there is still opportunity to improve the evidence quality and win some goodwill with your auditor. As your auditor uses the evidence to form an opinion, that person will need to put in the work find the relevant pieces of information to support the control. In this case, that person will probably review all of the GPO’s on the left side despite being irrelevant to the control. Then they need to find the relevant password settings, consider when the evidence was produced, and determine if it is representative of the organization’s domain (as opposed to a screenshot from the internet).
In this case, it’s a very common screenshot, and it won’t take the auditor too long to consider. Others aren’t so easy. Repeating this cycle for dozens of screenshots during the audit make it take longer for everyone. Anything you can do to save your auditor time only helps to improve your audit experience. Additionally, it helps to limit how often the auditor asks questions about irrelevant details of evidence.
Example: High Quality Evidence
This is the same screenshot as the one prior, but it has been marked up in a way that draws attention to what is important for the auditor to review. The auditor is highly likely to review it, mark the control as acceptable, and move on to the next control. Taking this approach for each piece of evidence takes a little more time for the auditee organization, but it saves time and frustration for the auditor which in turn saves frustration for those providing the evidence. The following information is highlighted for the auditor:
1. The “Default Domain Policy” GPO and where it is linked in the domain tree. This also shows the auditor that the screenshot is the infrastructure of the organization being audited.
2. The date and time the screenshot was taken shows that it was recent.
3. The password settings are highlighted and easy to compare with the security policy.
Auditors Get Audited Too!
Some may wonder why the evidence the auditor accepts is so important to them. After all, in this example, the first screenshot shows the password settings used by the organization. Why is it such a big deal? Many don’t realize that auditors sometimes undergo a peer review process. During this process, they may have to explain months or years later why the evidence they used to form an opinion was accepted. When they are performing your audit, the idea of them being audited is rolling around in the back of their mind. The higher the quality of your evidence, the more confident they will be that they can defend the quality of their own audit.
Show your auditor some love by providing them with the evidence they need to render a positive opinion about your controls.
