At my last job, I suggested that we try testing when we decided to hire a third programmer. We had gotten a lot of resumes and were having problems weeding them out--I suggested to do a programming test. It was really simple--my solution was 9 lines of code--basic Windows Form and inserting a record into an Access database. I gave the connection string, etc. as part of the test. I was appalled at how many people couldn't get it implemented. My boss asked if it was passable and I informed him that I cranked out my solution in less than 15 minutes. As a former computer science instructor, when writing tests, the guideline that my professors had given me as a TA was to write a test, take it and based on how long it took me to complete it, multiply that by 3 (maybe 4?) and that would be the time students would need so I thought that at most, people who said that they had skill set we were looking for should have been able to easily implement it within an hour (I also factor job interview nervousness:-)

We had one programmer knock it out and she turned out to be a pretty decent programmer.

Having said that, I really wish we would have done something similar at my current job when interviewing for the person who was to be my backup--in retrospect because after hiring a particular individual, it became obvious that he didn't have as much programmer experience as what he made his resume to look and the way he talked it up during his interview--he had spent like 6 years at his previous place doing networking and programming but it was a case of transferring into programming during the last 2 years of his time there. If I had given him a test, he wouldn't have been able to pass it but in an interview, he talked like he knew what he was doing.

My lessons learned: for jobs like that (combined network and programming work), ask how many years were spent being a programmer. Also, referring to the "rebuttal" article--I need to pay more attention to the lessons learned stories that a true senior programmer will talk about. If there's none mentioned, that's going to be a huge red flag for me from here on out.

Love these articles--they get people to thinking and talking:-)