I hesitate to bring this up at this late hour, but it's an easy way to speed up render times with a camera that doesn't move. Just render the static scene, and then apply it as a camera map to every object that doesn't move. Have the objects receive the mapping and shadows but not take part in any of the other rendering. Then, you just animate your moving objects among the static camera mapped ones.
Of course, I may have not read your post thoroughly enough, and may be way off track on what you want to achieve.