validate_junk. from Michal Mazurek

1. Optionally add random "canaries" to the end of an allocation. This
requires increasing the internal size of the allocation slightly, which
probably results in a large effective increase with current power of two
sizing. Therefore, this option is only enabled via 'C'.
2. When writing junk (0xdf) to freed chunks (current default behavior),
check that the junk is still intact when finally freeing the delayed chunk
to catch some potential use after free. This should be pretty cheap so
there's no option to control it separately.
ok deraadt tb

C standard are all weak.
Apply __{BEGIN,END}_HIDDEN_DECLS to gdtoa{,imp}.h, hiding the
arch-specific __strtorx, __ULtox_D2A, __strtorQ, __ULtoQ_D2A symbols.

instead of simply zapping it. this can save many syscalls in a program
that repeatedly grows and shrinks a buffer, as observed in the wild.

review by millert, binary checking process with doug, concept with guenther

what's going on.

to hold the malloc lock across mmap syscalls in all cases. dropping it
allows another thread to access the existing chunk cache if necessary.
could be improved to be a bit more aggressive, but i've been testing this
simple diff for some time now with good results.

ok otto

improve the random stream itself (it doesn't), but to introduce
noise in the arc4random calling pattern. Thanks to matthew@ who
pointed out bias in a previous diff, ok deraadt@ matthew@

freed chunk is actually freeable immediately. catch more errors.
hints/ok otto

deterministic behavior. four selected because it's more than three, less
than five. i.e., no particular reason.

can avoid reinventing the wheel
ok guenther schwarze

ok crickets@

a free chunk at random and may allow to increase delayed chunk array.
ok otto

we always junk small chunks now, and the first part of pages,
but only after free. J still does the old thing. j disables everything.
Consider experimental as we evaluate performance in the real world.
ok otto

okay otto@

Should improve sparc64 and other be archs. ok matthew@ miod@

malloc can, as always, be emulated via realloc(NULL).
ok deraadt

Like calloc(), except without the cleared-memory gaurantee
ok beck guenther, discussed for more than a year...

This is merely a by-product of figuring out the amount of phk@ code
contained herein; i'm not planning to hack on this file.

ok guenther otto

expanded, but not enough due to precedence error. Spotted by Thorsten Glaser.

unmapping of freed allocations without disabling chunk randomisation
like the "Freeguard" ('F') option does. Make security 'S' option
use 'U' and not 'F'.
Rationale: guarding with no chunk randomisation is great for debugging
use-after-free, but chunk randomisation offers better defence against
"heap feng shui" style attacks that depend on carefully constructing a
particular heap layout so we should leave this enabled when requesting
security options.

Okay otto@.
Found by Michal Mazurek <akfaew at jasminek dot net>, thanks!

param.h symbol reduction.
ok guenther

scanning for free space if the hint isn't available.
also, on further inspection, this will prevent pmap_prefer from "improving"
our hint.

clear out the entire requested area, not just a perfect fit.  second,
use mquery to check for room to avoid getting an address we don't like
and having to send it back.

in order to span the the entire cache.  second, on free use the same offset
to put things in the cache instead of always starting at zero.
ok otto

overallocating and then releasing unneeded memory pages.
ok otto

into an bogus pointer error instead of a segfault.
- Document that we use the assumption that a non-MAP_FIXED mmap() with
hint 0 never returns NULL.

space for meta data by only allocating space actually needed for
the bitmap (modulo alignment requirements). ok deraadt@

use-after-free bugs; ok krw@ dlg@ pirofti@

no security, for consistency.  use of this option is discouraged. :)
ok deraadt guenther millert

cases.

position zero, skipping a random number of free slots and then
picking the next free one. This slowed things down, especially if
the number of full slots increases.
This changes the scannning to start at a random position in the
bitmap and then taking the first available free slot, wrapping if
the end of the bitmap is reached. Of course we'll still scan more
if the bitmap becomes more full, but the extra iterations skipping
free slots and then some full slots are avoided.
The random number is derived from a global, which is incremented
by a few random bits every time a chunk is needed (with a small optimization
if only one free slot is left).
Thanks to the testers!

1UL to 1U.