if you dont intend to use the graphics, you can try removing the graphics module (i used nouveau) using rmmod..