Thanks Guys.
So let me add a bit more info as to what i already have in place.
As an example...
I have these AD security groups: 
    ...

Hey guys thanks for the suggestions so far.
After the weekend, I think i might have been a bit harsh about my manager.....
I agree with the Granular AD...