network lookup and call trace in kernel log

[zebul666]

hi.


I have installed kali linux 1.0.9 in a virtualbox VM guest (ubuntu 14.10 host)
I have bought a TP-Link WN722N usb wifi card. and use the USB pass through method of virtualbox to use it natively inside kali guest.

It is working fine.
But once in a while (say every one hour), I got a complete lock-up of the network (both virtual ethernet and usb wifi)

I have found in dmesg some stack trace like this

[ 8280.748310] INFO: task vminfo:2168 blocked for more than 120 seconds.
[ 8280.748321]       Tainted: G           O 3.14-kali1-amd64 #1
[ 8280.748325] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[ 8280.748331] vminfo          D ffff880069b74868     0  2168      1 0x00000000
[ 8280.748339]  ffff880069b74450 0000000000000086 0000000000000001 ffffffff81813480
[ 8280.748346]  0000000000014340 ffff88006a45ffd8 0000000000014340 ffff880069b74450
[ 8280.748352]  0000000000000000 ffffffff81888980 ffffffff81888984 ffff880069b74450
[ 8280.748359] Call Trace:
[ 8280.748375]  [<ffffffff814ec719>] ? schedule_preempt_disabled+0x9/0x10
[ 8280.748385]  [<ffffffff814ee252>] ? __mutex_lock_slowpath+0x142/0x1c0
[ 8280.748394]  [<ffffffff814ee2e6>] ? mutex_lock+0x16/0x25
[ 8280.748405]  [<ffffffff81412b4a>] ? dev_ioctl+0x33a/0x560
[ 8280.748417]  [<ffffffff8117e966>] ? kmem_cache_alloc+0x1a6/0x1c0
[ 8280.748428]  [<ffffffff813e3ac5>] ? sock_do_ioctl+0x55/0x60
[ 8280.748438]  [<ffffffff813e3d90>] ? sock_ioctl+0x70/0x2b0
[ 8280.748449]  [<ffffffff811985c6>] ? alloc_file+0x26/0xe0
[ 8280.748459]  [<ffffffff811a8b4e>] ? do_vfs_ioctl+0x7e/0x500
[ 8280.748469]  [<ffffffff813e4a3a>] ? sock_alloc_file+0x9a/0x130
[ 8280.748479]  [<ffffffff811b2d13>] ? __fd_install+0x23/0x60
[ 8280.748488]  [<ffffffff811a9069>] ? SyS_ioctl+0x99/0xb0
[ 8280.748499]  [<ffffffff813e6d1b>] ? SyS_socket+0x3b/0x70
[ 8280.748509]  [<ffffffff814f7879>] ? system_call_fastpath+0x16/0x1b

is it a bug in the wifi driver ? a problem with the wifi card ?
but since I have also a problem with virtual ethernet, I was more thinking about some kernel bug or a VM/virtualbox bug ?

[blackMORE]

I never thought I'd see this bug in Debian/Kali but I guess I was wrong.

You need to add the following two line

vm.dirty_background_ratio = 5
vm.dirty_ratio = 10

in

/etc/sysctl.conf

and reboot.

It's a Kernel bug and the above solution works well. Also I am assuming you allocated low Memory for your Kali VM or have a **** poor HDD speed.

Full details for this fix and explanation can be found here



Edit:
Swearing

[zebul666]

thank you very much blackMore. I will try that.

other things I have found about the issue:
if i run ifconfig in a terminal when this happens, ifconfig blocks and never output anything and I found another call trace for ifconfig in the log afterwards.

what's strange is that if I unplug the wifi card, it is gone and back to normal !

I just thought about one thing: it is that i use the virtualbox guest addition for vbox 4.3.10 available in kali repository.
whereas I am running virtualbox 4.3.18 so I am gonna try with the vbox guest addition for 4.3.18 instead to see if it makes a difference

[zebul666]

weel , the value are already at the value you ask me to set them to

sysctl -a|grep dirty
vm.dirty_background_bytes = 0
vm.dirty_background_ratio = 5
vm.dirty_bytes = 0
vm.dirty_expire_centisecs = 3000
vm.dirty_ratio = 10
vm.dirty_writeback_centisecs = 500

I go and try to use the 4.3.18 vbox guest addition to see if it makes a difference

[zebul666]

I still have the problem, even with the 4.3.18 vbox guest additions, matching the virtualbox version I use. :-(

It must be related to the usb wifi key because when I unplug it , it goes back to normal ???

[blackMORE]

I still have the problem, even with the 4.3.18 vbox guest additions, matching the virtualbox version I use. :-(It must be related to the usb wifi key because when I unplug it , it goes back to normal ???

Not too sure about that. Kernel panic with hung_task_timeout_secs and blocked for more than 120 seconds problem error happens because:

1. You have a task that is rapidly filling up your Memory
2. With full Memory, the obvious choice of Linux is to start using SWAP space.
3. But because your Memory is already full the HDD is responding too slowly, at some point Disk I/O becomes 100% (because of the difference between Disk and RAM R/W latency is always too high) and you're in a position that there's not enough Free Memory left to perform write data to swap or to retrieve it.

A similar post to mine exists here

In my case, I had this error on a test CentOS server and then on a production server. Both were fixed with the workaround above but the error on Production server came back due to a Java application's high usage. I had to increase the allocated Memory to the JVM and since then I haven't had the error.

In your case, I can advise few more suggestions:

1. If this is to do with USB Wifi dongle, are you able to use a different driver to see if that makes a difference
2. Can you try to allocate bit more Memory (i.e. if it's 1024 make it 2048)?
3. The disk type you're using in VBox, does it only affects VDI storage types? VMDK, VHD are two other options to play with.
4. Bridge Mode/NAT mode networking makes any differences?

Try out the USB dongle in different computer (or different Linux Vbox i.e. Debian, Ubuntu) and see if it affects all of them likewise. If at least one Linux distro survives, then you have a benchmark to work on.

Good luck.

[zebul666]

as suggested on virtualbox forum, I installed the official virtualbox package from oracle repo.

I does continue to freeze but I don't see anymore the call (stack) trace in the log.

Also I was already using 2048M of RAM in the VM, so I doubt it was missing any RAM ?

I tried my favorite distribution, archlinux, in another VM in virtualbox, and it worked really well. no freeze

[blackMORE]

Didn't realize or meant it. Thanks for your edit @mod. Cheers.