yaplej
CCNA
Premium
join:2001-02-10
White City, OR
 ·Charter Pipeline
| Help to resize an array in C.
Hey all,
I am hoping someone here can help me resize an array in C. I am working on a Linux kernel module, and each time I try to vmalloc a new block of memory, copy the contents of the old memory, and reassign the global pointer to the new memory location the system crashes.
I have a global structure pointer.
Next durring module initialization I allocate a block of memory, and assign it to the global sessions pointer.
At this point I can use the sessions pointer as an array without any problems as long as I dont need more rows than SESSIONSLOTS. The problems start when trying to allocate a new block of memory, and update the global sessions pointer to that new block of memory.
First I declare a new temporary pointer that will point to the new memory location.
Then allocate a new memory block twice the size of the currently allocated memory for the global sessions pointer.
Next I copy the memory from the current global sessions pointer memory to the temporary newsessions pointer memory.
Finally I free the global sessions pointer memory, and reassign the global sessions pointer to the newsessions allocated memory.
I have been trying all kinds of different methods trying to get it to work without any sucess. I hope someone here, can spot whatever stupid noob thing I am missing.
Everything I am working on here is Open Source so if you like I upload the full source of the kernel module.
Thanks.
--
sk_buff what?
Open Source WAN Accelerator
»trafficsqueezer.sourceforge.net/
|
|
Graycode
join:2006-04-17
 ·net2phone
| I don't see the problem either. Vmalloc() is a bit of an oddball entity though. »www.makelinux.net/ldd3/chp-8-sect-4.shtml
It would help to know exactly where the blow-up is happening. Here's a couple of thoughts:
1) Is it blowing up in that memmove()? Since you are allocating new memory to copy into, the addresses given to memmove could not overlap. Perhaps then memcpy() might be a better choice than memmove. Memmove does things to account for overlapped movement, and perhaps that's interfering with the "weird" addresses that vmalloc() might give.
2) Is it blowing up later when that memory area is being referenced? Your "struct session" ends with an 8-bit value, implying there may not be any structure alignment. But maybe the sizeof(struct session) is not used like you expect depending on how the address of a particular array member will be later calculated and referenced. The structure seems to be 21 raw bytes, or maybe 25 if the two 16-bit values cause aligned storage for the adjacent 32-bit values. Does the code that uses the memory assume that each structure instance is alinged on (for example) a 32-bit memory boundary? The assumption of padded alignment could reference memory beyond the allocation as usage approaches the end of the memory block.
Maybe someone else will have better suggestions. |
|
yaplej
CCNA
Premium
join:2001-02-10
White City, OR
·Charter Pipeline
| I have used memmove() in other areas of the kernel module without any problems, but it could be that it has issues with vmalloc'ed address space.
One solution might be to loop through each "slot" in the array, and copy that data that way. It would have the same net result without using memmove().
Referencing the memory area isn't a problem as its currently declared. I can declare SESSIONSLOTS = 1024, and everything runs fine. The problem only happens when trying to make room for additional slots in the array.
--
sk_buff what?
Open Source WAN Accelerator
»trafficsqueezer.sourceforge.net/
|
|
yaplej
CCNA
Premium
join:2001-02-10
White City, OR
·Charter Pipeline
1 edit | In case it helps here is the entire kernel module. The purpose is a simple network accelerator. Focusing on general TCP traffic acceleration using compression in this first version.
The section I am working on it the ability for multiple systems running this module to detect each other, and setup TCP sessions for acceleration/compression.
It does this by inserting some custom data into the TCP options section of the IP packet TCP segment, and tracking that data for each session.
The "sessions" are stored in the sessions global array, and that memory is assigned using vmalloc during module initialization.
When the memory allocated for sessions is full its need to be expanded, and thats where I am having problems.
Its been written on a CentOS 5.3 box, but should compile on anything with kernel 2.6.11 or newer. Kernel dev headers are required also.
--
sk_buff what?
Open Source WAN Accelerator
»trafficsqueezer.sourceforge.net/
|
|
Graycode
join:2006-04-17
·net2phone
| reply to yaplej
said by yaplej  :I have used memmove() in other areas of the kernel module without any problems, but it could be that it has issues with vmalloc'ed address space. One solution might be to loop through each "slot" in the array, and copy that data that way. It would have the same net result without using memmove(). That depends on how each "slot" was copied. The replacement of memmove() with memcpy() could be beneficial, depending on where the blow up is happening, because those 2 are internally different.
If you know where in that code that the problem happens, telling that could help others to help you. Does it die within the memmove()? |
|
yaplej
CCNA
Premium
join:2001-02-10
White City, OR
·Charter Pipeline
| It still crashed with using memcpy()
Upon looking at the code for memmove() it only has some additional code to determine if it will start moving at the first or last memory address. It actually uses memcpy if the dest is lessthan the source address.
»tomoyo.sourceforge.jp/cgi-bin/lx···_32.c#L7
Actually it looks like its crashing when trying to vmalloc_user() a new block of memory. Thats very odd that vmalloc during module initialization worked fine, and then doing it in code later crashes the system.
The code is in sessionmanager.h.
--
sk_buff what?
Open Source WAN Accelerator
»trafficsqueezer.sourceforge.net/
|
|
Graycode
join:2006-04-17
·net2phone
| said by yaplej :Actually it looks like its crashing when trying to vmalloc_user() a new block of memory. Thats very odd that vmalloc during module initialization worked fine, and then doing it in code later crashes the system. It sounds like you need to look for a resolution specific to the implementation of vmalloc_user() that you are using.
For example: »forum.soft32.com/linux/PATCH-Pre···377.html
quote:
If an attempt to allocate memory with vmalloc_user() fails, the result will be an oops when it tries to tweak the flags in the (non-existent) VMA. One could argue that __find_vm_area() should not return a random pointer on failure, but vmalloc_user() requires a check regardless.
Your own code seems OK. But it may be triggering a problem elsewhere in the OS. |
|
yaplej
CCNA
Premium
join:2001-02-10
White City, OR | reply to yaplej
I am pretty sure the problem is with allocating memory outside of the module initialization function. Why it fails I dont know, but if I try to allocate any memory outside of the module initialization function it causes my system to crash. |
|
yaplej
CCNA
Premium
join:2001-02-10
White City, OR
·Charter Pipeline
| reply to Graycode
Looks like that might have something to do with the problem. I tried using kcalloc() instead of vmalloc() or vmalloc_user(), and I was able to allocate a new block of memory.
The only problem is that kcalloc(), and kmalloc() cannot assign a large enough block of memory for what I need.
Now why vmalloc() worked in the module initialization, and not after is above my pay grade. |
|
yaplej
CCNA
Premium
join:2001-02-10
White City, OR
·Charter Pipeline
| reply to yaplej
I found out that vmalloc() allocates blocks of memory using kmalloc(), but it does so using a flag that allows it to sleep. The function I am running runs in an interrupt context, and sleeping is not allowed.
I was able get around this by using a linked list rather than an array. Using an array kmalloc() could not allocate enough memory for my needs, but a linked list fixed that issue. |
|
