I found it difficult at first snapping objects together. I then realized that if the cursor is closer to where you want your object to snap then the snap works a lot better. I know you said that you need the face or vertex to be close to the area you want to snap to but it also seemed to help to have the cursor close to the snapping point as well.
I made only simple objects for my practice challenge not the more complex castle Mike made. But, here is the objects I made snapping away:
