VMware with kernel 2.6.38 [updated for kernel 2.6.39]

Running Debian Squeeze with the latest kernel 2.6.38 from Debian Backports I had some trouble getting VMware Player to work. Once started it complained about not being able to compile the “vmnet” kernel module. I found the solution here and this is how you can make it work:

First copy the source tarball to a temporary directory, unpack it and edit the file “driver.c”:

# cp /usr/lib/vmware/modules/source/vmnet.tar /tmp
# cd /tmp
# tar xvf vmnet.tar
# vi vmnet-only/driver.c

Now go to lines 267, 271, 1137 and 1139, replace “lock_kernel();” with “lock_flocks();” and save the file. Next (as root) make a backup of the original tarball and generate the new one:

# mv /usr/lib/vmware/modules/source/vmnet.tar /usr/lib/vmware/modules/source/vmnet.tar.orig
# cd /tmp
# tar cvf /usr/lib/vmware/modules/source/vmnet.tar vmnet-only

Clean up:

# cd /tmp
# rm -r vmnet.tar vmnet-only

All you have do to now is to start VMware Player again and it’ll compile all kernel modules cleanly!

UPDATE: If you are running kernel 2.6.39 (for Debian Squeeze also available in Backports) the above does not work. I found a solution in Arch Linux‘ Wiki here and listed everything below. Please notice, the procedure involves patching the sources with a patch file written by a third party. If you do not trust this patch, do not use it!

If you decide to trust the patch, start by reverting to the original vmnet.tar (if you already followed instructions for kernel 2.6.38 above) and make backups of all source tarballs (as root):

# mv /usr/lib/vmware/modules/source/vmnet.tar.orig /usr/lib/vmware/modules/source/vmnet.tar
# cd /usr/lib/vmware/modules/source
# for i in `ls *.tar`; do cp -p ${i} ${i}.orig; done

Now we need to patch the sources (don’t need to be root for this!):

# mkdir /tmp/vmware && cd /tmp/vmware
# for t in /usr/lib/vmware/modules/source/*.tar; do tar xf $t; done
# wget -qO- http://kaslit.com/downloads/vmware2.6.39fixed.patch | patch -Np1

As root:

# for d in *-only; do tar cf /usr/lib/vmware/modules/source/${d/%-only/}.tar $d; done
# cd /tmp && rm -r vmware

Now VMware should play nicely with kernel 2.6.39!