Today one of my auditors sent me her matrix for auditing Microsoft SQL Server. I had seen part of it and some questions about some of the technical details I saw. In the email thread I discovered that some of it was taken from a guidance document from our professional auditing organization, ISACA (technically just the letters now, though it used to stand for Information Systems Audit and Control Association). If you're not familiar with ISACA, it's sort of like PASS, but for auditors, and with more education and certification because it's not product specific (that makes a big difference).
I'm a member of ISCA as a Certified Information Systems Auditor (CISA) in good standing, so I went to the web site, logged in, and downloaded the guidance document. I saw what I have become used to seeing in the IT audit industry: advice, specifications, and investigative questions designed by auditors who were not heavily technical in the product. As a result, the guidance document mixed up versions of SQL Server, asked auditors to check for features that are not present on all supported versions of SQL Server (like password policy enforcement, which isn't present in SQL Server). More so, I saw outright errors, like:
There were also important omissions. For instance, they tell the auditor to look at db_owner role members. But they don't say anything about:
So then I checked the list of contributors and reviewers. A lot of folks with excellent security certifications. But of the folks listed, only one had an MCSE. There were no Microsoft employees. There were no MCDBAs nor any MCITP:Database Administrators. There were no MVPs in SQL Server. And this is where I saw the huge disconnect. I think if they had gone to MS and asked for a couple of folks to review, this would have turned out to be a very usable and effective piece of guidance. As it is, some of the things leave me worried it could wrongly cost a DBA his or her job.
Which leads me to my plea: if you're going to put out guidance, please have it reviewed by the right folks - experts with the technology. They don't have to know auditing cold. They don't have to know the regulation or the law or the standard perfectly. What they do need to know is the technology. Let them tech review the document. They will be able to catch the technical mistakes. That's what they are good at, whereas a general IT auditor or CISSP or even a GSEC is not. This is for the good of the industry and for the organizations we work for.