511-The process exited with an error: 511 End of error message).
(added 18/8/2007)
Whilst trying to get VMware Server 1.0.3 working on a hardened Gentoo system (as of article date, at least) , I came across the below problem (after seeing various messages just like:
511-The process exited with an error: 511 End of error message).
After (eventually!) checking /var/log/vmware/vmware-serverd.log, I noticed the below problem:
Unable to connect to the remote host: 511-The process exited with an error:
Can't load '/usr/lib/vmware/perl5/site_perl/5.005/i386-linux/auto/VMware/VmPerl/VmPerl.so' for module VMware::VmPerl:
/usr/lib/vmware/perl5/site_perl/5.005/i386-linux/auto/VMware/VmPerl/VmPerl.so: cannot make segment writable for relocation:
Permission denied at /usr/lib/vmware/perl5/site_perl/5.005/DynaLoader.pm line 169.
at /usr/lib/vmware/perl5/site_perl/5.005/i386-linux/VMware/Control.pm line 14
BEGIN failed--compilation aborted at /usr/lib/vmware/perl5/site_perl/5.005/i386-linux/VMware/Control.pm line 14.
BEGIN failed--compilation aborted at /usr/lib/vmware/perl5/site_perl/5.005/i386-linux/VMware/Control/Profiler.pm line 29.
BEGIN failed--compilation aborted at /usr/lib/vmware/perl5/site_perl/5.005/VMware/VMServerd.pm line 16.
BEGIN failed--compilation aborted at /usr/lib/vmware/serverd/init.pl line 48.
VMServerd Panic: Could not initialize Perl script handler.
511 End of error message.
A method to quickly fix this is via:
chpax -m /usr/sbin/vmware-serverd
which allows the vmware-serverd binary to use mprotect() to make .text writable (and thus perform the required relocations). For more information on why restricting mprotect() is good security practice, see the PaX documentation on mprotect.
Ideally, however, VMware would change their compilation scripts to make the Perl .so files PIC / not needing relocations like the above :)
Some other things I did in the process of getting things to work (untested, may not be actually needed):
Basically copied /etc/pam.d/sshd over /etc/pam.d/vmware-authd - this uses the pam system-auth file, which allows a more centralised configuration (and reduces problems that may be because of incorrect pam.d entries).
The gentoo xinetd.d config needed an only_from line present to make it work for me. For my needs, I just added 127.0.0.1 to the list. To access it from anywhere, 0.0.0.0 is needed.