When I try to use dsplink 1.64 it causes a general protection fault and crashes my application.
I've been filling the dsplink source code with debug info trying to isolate the crash and so far this is what I've figured out:
The driver crashes on my first call into the dsplink api with PROC_setup. PROC_setup makes a number of calls to DRV_Invoke which make ioctls into the driver which in turn call DRV_CallAPI. DRV_CallAPI is called with the following commands in order: CMD_DRV_GETCHNLMAPTABLE_ADDRESS, CMD_PROC_SETUP, CMD_IDM_INIT, CMD_IDM_CREATE. Its the call to IDM_create on drv_pmgr.c:1561 that kills it.
Something goes bizarrely wrong with the way this function is called. If I put a printk("Entered IDM_create\n") at the very begining of the function then the stack trace generated by the general protection fault sometimes shows that its the printk itself causing the fault. Other times the printk prints out 18 characters of random garbage instead. In this case, or if I don't put debugging output inside the function (and note there are no other function calls inside this function), then the stack trace shows that DRV_Ioctl is responsible for the crash.
Here is a sample call trace:
[15882.527003] Call Trace:
[15882.527003] [<f8fd301e>] ? DRV_Ioctl+0xee7/0x11e1 [dsplinkk]
[15882.527003] [<f8ff8e16>] ? DM648MAP_Config+0x49a/0x1797c [dsplinkk]
[15882.527003] [<c042574f>] ? dequeue_task+0xcf/0x120
[15882.527003] [<c0401e16>] ? __switch_to+0xb6/0x190
[15882.527003] [<c0432e41>] ? finish_task_switch.clone.142+0x31/0xb0
[15882.527003] [<c04f18d7>] ? vfs_ioctl+0x87/0xa0
[15882.527003] [<c04f21a8>] ? do_vfs_ioctl+0x68/0x780
[15882.527003] [<c045c520>] ? hrtimer_nanosleep+0xa0/0x150
[15882.527003] [<c045b320>] ? hrtimer_wakeup+0x0/0x20
[15882.527003] [<c04e45e9>] ? fget_light+0x99/0xb0
[15882.527003] [<c04f291f>] ? sys_ioctl+0x5f/0x80
[15882.527003] [<c08f5798>] ? syscall_call+0x7/0xb
[15882.527003] Code: 00 b9 15 01 00 00 ba 08 04 00 00 b8 0b 80 00 80 bf 0b 80 00 80 e8 a2 12 00 00 e9 8f 00 00 00 31 c0 81 3e ff 00 00 00 75 70 89 2e <66> 8b 03 31 c9 66 89 46 04
[15882.527003] EIP: [<f8ff5361>] IDM_create+0xa3/0x145 [dsplinkk] SS:ESP 0068:f61cfe58
Any ideas? Why would an attempt to print a constant string print random data instead? Presumably IDM_create is using some memory it shouldn't have access to rather than the memory its supposed to be using, which would explain how I get a protection fault. I don't know enough about the inner workings of C to understand what could cause an error like this though.
