Hi!

You don't need a running X server for VNC - I don't have either.
Do you want to run kdm/xdm before or start a Window-Manager directly?
What's in you ~/.vnc/xstartup file?

Nicolas