Interesting discussion, my company makes the stuff behind all this, the controllers, management software etc.
Being in the industry, we fret over this stuff all day long, and spend our time developing things like anti-passback, n-man rules, etc. But, as discussed, you can put all the physical security in place you want, but if people don't respect it and the reason it is there, they'll try to walk around it all day long.
We (and other companies) now have a credential for your smartphone, it's arguably more secure because of our intimate relationship to our phones vs. how we treat cards/fobs. You may share your card, but I doubt you'd do the same with your phone. Also, in bulk it's cheaper than cards (the app is free, you by bulk licenses for the logical credential)
An interesting debate we have around credentials is to print or not to print. If you print on the card and require it be visible you get visual confirmation that it belongs to the person wearing it. But, if they lose it in a parking lot, the card typically clearly shows what company this card is for and the identity to spoof. If you don't print on cards, they're anonymous but you don't get constant direct visual verification. However most management software can show the associated picture of a person as they badge through a door.