Calibrating your Phantom Compass is one of the most important steps to prevent Fly-Away issues. This article with videos explain the issue of fly-away along with how to prevent this from happening in the first place. The first video also discusses what to do when fly-away occurs. The Phantom has very fluid movement.
You move the sticks and the Phantom makes that movement straight away. If it is slow or unresponsive, then you have a problem. If something is weird going on, bring your Phantom in straight away and troubleshoot.