use KS_ERR_NOMEM instead of -1.

This reverts commit f18b6e19.

* ks_free: The 'p' argument should have type unsigned char* to match the type of the 'encoding' argument to ks_asm

do boundary check for MCContext::Instances[]. this fixes crash-01-x64-empty-tombstone-value-shouldnt-be-inserted-into-map.c

sanity check on number of bits of FP in ARM, AArch64 & X86 parsers. this fixes crash-27-x64-too-many-bits-for-uint64_t.c