Former-commit-id: f4d606317eebd6ffb15ce7a41d809a09fc8f83b0

PNTransformDriver.cpp: important comments as to what padding is actually added for a variable. 



Former-commit-id: 6835caa6ec2a7a5aae1de19286ce7eee2424be1c

Former-commit-id: fceabead2959b653227b3358697fbd2d531d144d

PN optimization. PN will now do a binary search algorithm to find bad modifications. All functions with coverage are now validated, and validated faster than previous version that only validated if coverage was above the threshold. 

This version does not have shuffle validation, I will add that feature in a future update. It was easier to remove it for this optimization. Typically this is only used for Main. 

New logging summary output including how many functions are modified with high coverage, low coverage and no coverage. 



Former-commit-id: 468864bd92166e2eafafddd65b152c721173d294

Former-commit-id: 896e7b4b6d739f9077e7d590f5962a8d5f35dae7

Former-commit-id: 744772d87669a169a20f8ecf2c775edee84c79f6

PNTransformDriver.cpp using basename of shared objects to match file name stored in db. This is a temporary fix due to the way shared objects are currently stored in the db. 



Former-commit-id: b01bf4ca7674cfef8fe585ff327802f61870eea7

Former-commit-id: d0b753e6ae7790499b14f1bdf8b3d45dfe034449

coverage will get the basename of a shared object absolute address to compare against the stored url of a shared object. This should be corrected such that the db stores the absolute path of the shared object and comparison does not require me to take the basename. 




Former-commit-id: 76d8d5886eec5f023b0876c49a23ac8b9074f18c

Former-commit-id: a37f2277be18954c153346a11d1617b8a2bf302c

I had a change of heart, a summary report of functions exceeding the PN threshold is given by PN now, not coverage.cpp. This way the summary is based on the threshold passed on PN not a hard coded 50% like I had it before. 



Former-commit-id: b4db538e3255d72ae6e4c9cf6bf93741ff0476a2

coverage.cpp, prints out to stdout the total number of non-plt funcs (functions that don't begin with '.') that exceed 50% coverage. 



Former-commit-id: a0b76fa27864a094ab3d271f71523bc483bee40f

Former-commit-id: 466370d9eebd247831907328c49237c16457013c

Former-commit-id: 3ba17ed0594306b5fdf3ca5f4617cf84c1fdff83

Second, and last partial commit which now fully updates the toolchain for the manual testing infrastructure. 



Former-commit-id: 85081dd4402ea35a7431aa5ce1f384956a8533a2

Former-commit-id: 9adaaf5a10aa179adbc36318ca31f5e58756091b

Former-commit-id: 14384d4beb601076e2d73d52a381028c1b611be1

Former-commit-id: 277087b0ddac7b57b7fa47f3c68d715aaf143957

this was causing the wrong register values to be printed


Former-commit-id: 42634807719c3dcedb43a28226cd2f05b651334b

Former-commit-id: a8e23ff674c8df242c8ff03d95f863116aace9b7

Former-commit-id: 7da867d1a66189b9ae230a16b241c4a81ceb175e

1) Altered the way in which stack prologues are detected. I no longer use the CFG code. Apparently that code relied on Function_t knowing the entry of the function, so I have decided to rely on the same data. I now loop starting at this entry instruction through every fallthrough until a NULL fallthrough is reached or the stack prologue is detected. I also added detection of push jmp pairs added by fix calls. If this is detected in the prologue, the push is ignored as a pushed register, and I will continue analysis of the prologue starting at the pushed address. This has been observed to happen in shared objects for thunk code. Sometimes this code is inserted immediately before the stack allocation site. There are asserts in this version to detect some conditions concerning this new code, I will remove these later. These changes are primarily in OffsetInference.cpp.
2) I incorporated Duc's code for storing layout information in the DB for ground truth purposes. This is turned on with --ground_truth flag. This functionality should not affect anyone unless you use this flag. 



Former-commit-id: 1fecfd5592e216ffb9ba9ec54cbc7e8fc2e3aa33

Former-commit-id: 820ba0fc9f2230df340d69409a79be696f2abbe6

Former-commit-id: e57d602f80db860d44cdd75850f18b7e1d7dfd60

Former-commit-id: 6d1df0ad00bbcb11c80124e063e613b1c387f3a6

Former-commit-id: 1bf3f4d11b2a7a98423dff36e1066e681db4d088

When looping through shared objects in PNTransformDriver.cpp, I previously deleted FileIR's after I was done analyzing them, however I believe this causes an issue if function pointers are reused later for different functions. This version has a temporary fix where I don't delete any FileIR's. 


Former-commit-id: af1b80fcc678ef52bc2507fc00daafedebe95533

Altered the santizing filter in PNTransformDriver, a non-null target for an instruction that is not a ret, call, or jmp, must be in the same function (cannot be null).



Former-commit-id: 841895d92dc909c29f952f9dc36746ecebb909b8

The method for undoing function transformations in PN used the function name as a key to a map containing the functions to revert. I have changed this to use the function pointer as the key. 

I have also added output of the "URL" of the file for the function being altered in the log, so we can differentiate functions with the same name. This change has not yet been made for the summary reports in the log. 



Former-commit-id: 701d41952b2ecb78cf82d801196ee0f890f734e5

Fixed a big in OffsetInference.cpp. Previously a mapping of functions to layout inferences was made using the funciton name as the map key. With shared objects, it is now possible to have functions with the same name, which causes issues. Specifically, the map is always consulted first before generating a layout, and a previous entry is found for functions with duplicate names. 



Former-commit-id: 962df148c31aabf60e8156c4853e73f7158aaa33

Added a sanitizing step to PN such that functions that have instructions falling through to other non-null functions, or functions with instructions that are not ret, call or jmp that have indirect targets outside the function are excluded from any transformation. 

Also added a check for fix call push/jmp pairs when determing the prologue of a function. I will ignore a push constant followed by an unconditional jmp. 



Former-commit-id: 3d4d65f5a985284653d417f72bf8208b48a59113

If spasm fails during validation in PNTransformDriver.cpp, an assert failure should occur. Consider doing something other than assert in the future. 

Also, PNTransformDriver.cpp had a logging message that should be only output for verbose logging, corrected this. 



Former-commit-id: b243cf07308e18eebe7cd1a56c5a7be11ceffd37

Former-commit-id: dc655bb413f8ad2393f7d79fb709e75db0bddc37

Former-commit-id: 07b9609bc34e80245329c7bea562c3e016b8fc83

generate_spri.cpp, added debugging log output into the spri file to indicate what the label_offset is every time it is changed to help catch a non-deterministic bug that has been seen with producing labels. 



Former-commit-id: 12e8fc23bc6f49b118b014b64f47a6be2b70f7ec

Updating OffsetInference.cpp, a previous optimization to not attempt layouts that are the same resulted in a null pointer use. I put in some null pointer checks to correct the problem.



Former-commit-id: 01b85de0d8045b5ba426ceaaf90d69cc18877093

Updated spasm such that the output file provided as input is actually used, and the usage message is now correct. Additionally, assertion failures occur if the symbol file is not specified. 



Former-commit-id: 8fac258daad4e4729df7e7201545128c71a23cdf

Fixed error with shared object p1 protection. Was double freeing a pointer in the generatetransfroms function. 



Former-commit-id: fb7d93fe968533450440006f7050381a23865e5c

Former-commit-id: b16db42784ea830d589198120d3959ee9919e5ea

OffsetInference.cpp, Iff the AOI inference has the same number of boundaries as the SOI or DOI inference, the subsequent inferences are made null. For now, no matter what, the p1 inference is still generate. DOI tends to never be used, because it is usually identical to AOI, so the idea is to not validate it twice. 

Updaing PNTransformDriver .cpp and .hpp and PNMain, both to clean up the code but also to allow for performing PN on shared objects. This can be done by specifiying a flag to PNMain. This functionality is currently minimally tested, however, without specifying shared object protection, the functionality appears unchanged. I am committing for now. If I can validate the shared object protection I will update the scripts that call the PN transform code to use the new flag. 



Former-commit-id: cfa3c0a081799174b9bf879ee7879f18c264af83