Usually the solution is simpler than we could think of! I have solved many problems like this one in my games using empty gameobjects as reference points, and I actually think that it may be even better performance wise since some of those behaves (parent and child correlations) are already being computed by the engine anyway.
Im glad you got this problem solved, your game is looking pretty good so far, I am looking forward to play it, congrats.