kernel_optimize_test/lib
Dave Hansen 534acc057b lib: flexible array implementation
Once a structure goes over PAGE_SIZE*2, we see occasional allocation
failures.  Some people have chosen to switch over to things like vmalloc()
that will let them keep array-like access to such a large structures.
But, vmalloc() has plenty of downsides.

Here's an alternative.  I think it's what Andrew was suggesting here:

	http://lkml.org/lkml/2009/7/2/518

I call it a flexible array.  It does all of its work in PAGE_SIZE bits, so
never does an order>0 allocation.  The base level has
PAGE_SIZE-2*sizeof(int) bytes of storage for pointers to the second level.
 So, with a 32-bit arch, you get about 4MB (4183112 bytes) of total
storage when the objects pack nicely into a page.  It is half that on
64-bit because the pointers are twice the size.  There's a table detailing
this in the code.

There are kerneldocs for the functions, but here's an
overview:

flex_array_alloc() - dynamically allocate a base structure
flex_array_free() - free the array and all of the
		    second-level pages
flex_array_free_parts() - free the second-level pages, but
			  not the base (for static bases)
flex_array_put() - copy into the array at the given index
flex_array_get() - copy out of the array at the given index
flex_array_prealloc() - preallocate the second-level pages
			between the given indexes to
			guarantee no allocs will occur at
			put() time.

We could also potentially just pass the "element_size" into each of the
API functions instead of storing it internally.  That would get us one
more base pointer on 32-bit.

I've been testing this by running it in userspace.  The header and patch
that I've been using are here, as well as the little script I'm using to
generate the size table which goes in the kerneldocs.

	http://sr71.net/~dave/linux/flexarray/

[akpm@linux-foundation.org: coding-style fixes]
Signed-off-by: Dave Hansen <dave@linux.vnet.ibm.com>
Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>
2009-07-29 19:10:36 -07:00
..
lzo
reed_solomon
zlib_deflate
zlib_inflate
.gitignore
argv_split.c
atomic64.c lib: export generic atomic64_t functions 2009-07-29 19:10:35 -07:00
audit.c
bcd.c
bitmap.c bitmap: fix end condition in bitmap_find_free_region 2009-03-12 19:32:51 -07:00
bitrev.c
bug.c
bust_spinlocks.c oops handling: ensure that any oops is flushed to the mtdoops console 2009-01-06 15:59:11 -08:00
check_signature.c
checksum.c lib/checksum.c: fix endianess bug 2009-06-19 14:58:13 +02:00
cmdline.c
cpumask.c x86: remove some alloc_bootmem_cpumask_var calling 2009-06-11 19:27:07 +03:00
crc-ccitt.c
crc-itu-t.c
crc-t10dif.c
crc7.c
crc16.c
crc32.c
crc32defs.h
ctype.c
debug_locks.c lockdep: warn about lockdep disabling after kernel taint 2009-04-12 16:10:51 +02:00
debugobjects.c debugobjects: delay free of internal objects 2009-03-17 12:28:30 +01:00
dec_and_lock.c atomic: only take lock when the counter drops to zero on UP as well 2009-06-16 19:47:47 -07:00
decompress_bunzip2.c kmemtrace, kbuild: fix slab.h dependency problem in lib/decompress_bunzip2.c 2009-04-03 12:22:45 +02:00
decompress_inflate.c kmemtrace, kbuild: fix slab.h dependency problem in lib/decompress_inflate.c 2009-04-03 12:21:36 +02:00
decompress_unlzma.c kmemtrace, kbuild: fix slab.h dependency problem in lib/decompress_unlzma.c 2009-04-03 12:22:59 +02:00
decompress.c bzip2/lzma: don't stop search at first unconfigured compression 2009-02-19 13:36:55 -08:00
devres.c
div64.c
dma-debug.c dma-debug: Fix the overlap() function to be correct and readable 2009-07-10 22:18:45 +02:00
dump_stack.c
dynamic_debug.c Dynamic debug: fix typo: -/-> 2009-07-28 13:45:22 -07:00
extable.c module: trim exception table on init free. 2009-06-12 21:47:04 +09:30
fault-inject.c Remove remaining unwinder code 2009-01-06 15:59:11 -08:00
find_last_bit.c
find_next_bit.c
flex_array.c lib: flexible array implementation 2009-07-29 19:10:36 -07:00
gcd.c lib: add lib/gcd.c 2009-06-18 13:04:05 -07:00
gen_crc32table.c
genalloc.c lib/genalloc.c: remove unmatched write_lock() in gen_pool_destroy 2009-06-16 19:47:53 -07:00
halfmd4.c
hexdump.c hexdump: remove the trailing space 2009-06-16 19:47:51 -07:00
hweight.c
idr.c cgroup: CSS ID support 2009-04-02 19:04:53 -07:00
inflate.c
int_sqrt.c
iomap_copy.c
iomap.c
iommu-helper.c
ioremap.c
irq_regs.c
is_single_threaded.c
kasprintf.c
Kconfig lib: Provide generic atomic64_t implementation 2009-06-15 13:27:38 +10:00
Kconfig.debug kmemleak: Allow the early log buffer to be configurable. 2009-06-25 10:16:13 +01:00
Kconfig.kgdb
Kconfig.kmemcheck kmemcheck: make kconfig accessible for other architectures 2009-06-15 15:49:17 +02:00
kernel_lock.c sched: TIF_NEED_RESCHED -> need_reshed() cleanup 2009-03-06 12:48:55 +01:00
klist.c driver core: Remove completion from struct klist_node 2009-01-06 10:44:30 -08:00
kobject_uevent.c driver core: allow non-root users to listen to uevents 2009-04-16 16:17:09 -07:00
kobject.c kobject: make kset_create check kobject_set_name return value 2009-06-15 21:30:24 -07:00
kref.c
libcrc32c.c
list_debug.c
lmb.c lmb: Rework lmb_dump_all() output 2009-02-11 13:38:00 +11:00
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c locking: rename trace_softirq_[enter|exit] => lockdep_softirq_[enter|exit] 2009-03-13 01:32:36 +01:00
Makefile lib: flexible array implementation 2009-07-29 19:10:36 -07:00
nlattr.c Merge branch 'master' of git://git.kernel.org/pub/scm/linux/kernel/git/kaber/nf-next-2.6 2009-03-26 22:45:23 -07:00
parser.c
percpu_counter.c Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-01-06 17:10:04 -08:00
plist.c
prio_heap.c lib: fix sparse shadowed variable warning 2009-01-06 15:59:11 -08:00
prio_tree.c
proportions.c Merge branch 'core-fixes-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/linux-2.6-tip 2009-01-06 17:10:04 -08:00
radix-tree.c lib: do code optimization for radix_tree_lookup() and radix_tree_lookup_slot() 2009-06-16 19:47:49 -07:00
random32.c
ratelimit.c
rational.c lib: isolate rational fractions helper function 2009-06-11 08:51:08 -07:00
rbtree.c rb_tree: remove redundant if()-condition in rb_erase() 2009-06-16 19:47:56 -07:00
reciprocal_div.c
rwsem-spinlock.c
rwsem.c
scatterlist.c scatterlist: make sure sg_miter_next() doesn't return 0 sized mappings 2009-04-22 08:35:09 +02:00
sha1.c
show_mem.c
smp_processor_id.c cpumask: convert lib/smp_processor_id to new cpumask ops 2009-01-30 15:47:34 +01:00
sort.c generic swap(): lib/sort.c: rename swap to swap_func 2009-01-08 08:31:14 -08:00
spinlock_debug.c
string_helpers.c
string.c
swiotlb.c swiotlb: change swiotlb_bus_to[phys,virt] prototypes 2009-04-08 16:18:38 +02:00
syscall.c
textsearch.c
ts_bm.c
ts_fsm.c
ts_kmp.c
vsprintf.c vsprintf: introduce %pf format specifier 2009-04-29 20:55:55 +02:00