Completing Manipulation Tasks Efficiently in Complex Environments