Former-commit-id: 8945bb40bc258c939a13d7f04caa693e2544c073

Former-commit-id: 3aa347754b217b5873769d6b0e9a67aee4615977

PNTransformDriver will now perform shuffle validation, if appropriate. To determine if shuffle validation is appropriate, I also changed PNStackLayout.hpp to return if shuffle validation is appropriate for a layout. 

Also enforce that a.ncexe is the first file to be modified, and minor code clean up and tweaks to logging information. 



Former-commit-id: 28d2baf35a7b7e05d0e72619665be58c7b19bbda

Former-commit-id: 3b0cd479001c9c272e559772de1e588b53cb86b4

Former-commit-id: 037b8ab9a4e8992e3dd8f65172c451cfd264b75d

Tests to make sure that vetted tokens come from a single signature


Former-commit-id: e532126bf0874cca10852870ace8518b4482890d

PN Will not register modificaitons, but instead will accumulate modifications that succeed. This is a hack because zsh will not pass if I delay modifications until the end. 

Also a few additions to logging, error logging, and comments. Some minor cleanup as well. 



Former-commit-id: a3f941e8db8093467283772f1b7337e95b10e826

Former-commit-id: e5c9cf6fb5ae915bf56fb4060554ce61217d1b68

Former-commit-id: 8318542979dcced1aeb5a9a8fab29984939cdcfa

Former-commit-id: f4d606317eebd6ffb15ce7a41d809a09fc8f83b0

PNTransformDriver.cpp: important comments as to what padding is actually added for a variable. 



Former-commit-id: 6835caa6ec2a7a5aae1de19286ce7eee2424be1c

Former-commit-id: fceabead2959b653227b3358697fbd2d531d144d

PN optimization. PN will now do a binary search algorithm to find bad modifications. All functions with coverage are now validated, and validated faster than previous version that only validated if coverage was above the threshold. 

This version does not have shuffle validation, I will add that feature in a future update. It was easier to remove it for this optimization. Typically this is only used for Main. 

New logging summary output including how many functions are modified with high coverage, low coverage and no coverage. 



Former-commit-id: 468864bd92166e2eafafddd65b152c721173d294

Former-commit-id: 896e7b4b6d739f9077e7d590f5962a8d5f35dae7

Former-commit-id: 744772d87669a169a20f8ecf2c775edee84c79f6

PNTransformDriver.cpp using basename of shared objects to match file name stored in db. This is a temporary fix due to the way shared objects are currently stored in the db. 



Former-commit-id: b01bf4ca7674cfef8fe585ff327802f61870eea7

Former-commit-id: d0b753e6ae7790499b14f1bdf8b3d45dfe034449

coverage will get the basename of a shared object absolute address to compare against the stored url of a shared object. This should be corrected such that the db stores the absolute path of the shared object and comparison does not require me to take the basename. 




Former-commit-id: 76d8d5886eec5f023b0876c49a23ac8b9074f18c

Former-commit-id: a37f2277be18954c153346a11d1617b8a2bf302c

I had a change of heart, a summary report of functions exceeding the PN threshold is given by PN now, not coverage.cpp. This way the summary is based on the threshold passed on PN not a hard coded 50% like I had it before. 



Former-commit-id: b4db538e3255d72ae6e4c9cf6bf93741ff0476a2

coverage.cpp, prints out to stdout the total number of non-plt funcs (functions that don't begin with '.') that exceed 50% coverage. 



Former-commit-id: a0b76fa27864a094ab3d271f71523bc483bee40f

Former-commit-id: 466370d9eebd247831907328c49237c16457013c

Former-commit-id: 3ba17ed0594306b5fdf3ca5f4617cf84c1fdff83

Second, and last partial commit which now fully updates the toolchain for the manual testing infrastructure. 



Former-commit-id: 85081dd4402ea35a7431aa5ce1f384956a8533a2

Former-commit-id: 9adaaf5a10aa179adbc36318ca31f5e58756091b

Former-commit-id: 14384d4beb601076e2d73d52a381028c1b611be1

Former-commit-id: 277087b0ddac7b57b7fa47f3c68d715aaf143957

this was causing the wrong register values to be printed


Former-commit-id: 42634807719c3dcedb43a28226cd2f05b651334b

Former-commit-id: a8e23ff674c8df242c8ff03d95f863116aace9b7

Former-commit-id: 7da867d1a66189b9ae230a16b241c4a81ceb175e

1) Altered the way in which stack prologues are detected. I no longer use the CFG code. Apparently that code relied on Function_t knowing the entry of the function, so I have decided to rely on the same data. I now loop starting at this entry instruction through every fallthrough until a NULL fallthrough is reached or the stack prologue is detected. I also added detection of push jmp pairs added by fix calls. If this is detected in the prologue, the push is ignored as a pushed register, and I will continue analysis of the prologue starting at the pushed address. This has been observed to happen in shared objects for thunk code. Sometimes this code is inserted immediately before the stack allocation site. There are asserts in this version to detect some conditions concerning this new code, I will remove these later. These changes are primarily in OffsetInference.cpp.
2) I incorporated Duc's code for storing layout information in the DB for ground truth purposes. This is turned on with --ground_truth flag. This functionality should not affect anyone unless you use this flag. 



Former-commit-id: 1fecfd5592e216ffb9ba9ec54cbc7e8fc2e3aa33

Former-commit-id: 820ba0fc9f2230df340d69409a79be696f2abbe6

Former-commit-id: e57d602f80db860d44cdd75850f18b7e1d7dfd60

Former-commit-id: 6d1df0ad00bbcb11c80124e063e613b1c387f3a6

Former-commit-id: 1bf3f4d11b2a7a98423dff36e1066e681db4d088

When looping through shared objects in PNTransformDriver.cpp, I previously deleted FileIR's after I was done analyzing them, however I believe this causes an issue if function pointers are reused later for different functions. This version has a temporary fix where I don't delete any FileIR's. 


Former-commit-id: af1b80fcc678ef52bc2507fc00daafedebe95533

Altered the santizing filter in PNTransformDriver, a non-null target for an instruction that is not a ret, call, or jmp, must be in the same function (cannot be null).



Former-commit-id: 841895d92dc909c29f952f9dc36746ecebb909b8

The method for undoing function transformations in PN used the function name as a key to a map containing the functions to revert. I have changed this to use the function pointer as the key. 

I have also added output of the "URL" of the file for the function being altered in the log, so we can differentiate functions with the same name. This change has not yet been made for the summary reports in the log. 



Former-commit-id: 701d41952b2ecb78cf82d801196ee0f890f734e5

Fixed a big in OffsetInference.cpp. Previously a mapping of functions to layout inferences was made using the funciton name as the map key. With shared objects, it is now possible to have functions with the same name, which causes issues. Specifically, the map is always consulted first before generating a layout, and a previous entry is found for functions with duplicate names. 



Former-commit-id: 962df148c31aabf60e8156c4853e73f7158aaa33

Added a sanitizing step to PN such that functions that have instructions falling through to other non-null functions, or functions with instructions that are not ret, call or jmp that have indirect targets outside the function are excluded from any transformation. 

Also added a check for fix call push/jmp pairs when determing the prologue of a function. I will ignore a push constant followed by an unconditional jmp. 



Former-commit-id: 3d4d65f5a985284653d417f72bf8208b48a59113