I too would like to find or build a headless version. In the meantime, you could try changing the default target to run multi-user instead of graphical. It doesn't remove the GUI just doesn't start...