Skip to content
Snippets Groups Projects
Commit 05ad836b authored by Stefan Nagy's avatar Stefan Nagy
Browse files

Update README.md

parent 84cc5206
No related branches found
No related tags found
1 merge request!15Update README.md
Pipeline #13513 passed
Stage: clean
Stage: build
Stage: test
Hide whitespace changes
Inline Side-by-side
Showing
with 56 additions and 116 deletions
README.md
+ 56
− 116
View file @ 05ad836b
# Zafl (Zipr-based AFL)
Welcome to *Zafl*, a project to fuzz X86 64-bit binary programs.
# ZAFL: Zipr-based AFL
Welcome to **ZAFL**: a project to extend compiler-quality instrumentation speed *and* transformation support to the fuzzing of x86-64 binary programs. The key features of ZAFL include:
* Fast, space-efficient, and inlined binary fuzzing instrumentation via the Zipr binary rewriting infrastructure.
* A platform to extend and combine compiler-style code transformations (e.g., CMP unfolding) to binary-only fuzzing.
* Full compatibility with the AFL and AFLPlusPlus fuzzer ecosystem.
Key features of Zafl:
* Uses Zipr, a fast, space-efficient binary rewriter to inline AFL-style instrumentations
* 100% AFL compatible
* Platform for experimenting with other instrumentation to guide afl, e.g., add calling context to the edge-profile function (a la Angora)
<table><tr><td align=center colspan="2"><div><b>Presented in our paper</b> <a href="https://www.usenix.org/conference/usenixsecurity21/presentation/nagy"><i>Breaking-through Binaries: Compiler-quality Instrumentation for Better Binary-only Fuzzing</i></a><br>(To appear in the 2021 USENIX Security Symposium).</td </tr>
<tr><td><b>Citing this repository:</b></td>
<td><code class="rich-diff-level-one">@inproceedings{nagy:breakingthrough, title = {Breaking Through Binaries: Compiler-quality Instrumentation for Better Binary-only Fuzzing}, author = {Stefan Nagy and Anh Nguyen-Tuong and Jason D. Hiser and Jack W. Davidson and Matthew Hicks}, booktitle = {{USENIX} Security Symposium (USENIX)}, year = {2021},}</code></td></tr>
<tr><td><b>License:</b></td><td><a href="https://git.zephyr-software.com/opensrc/zafl/-/blob/master/LICENSE">BSD 3-Clause License</a></td></tr>
<tr><td><b>Disclaimer:</b></td><td><i>This software is strictly a research prototype.</i></td></tr></table>
## Installation with Docker
## Prerequisites
* **Install environment**: ZAFL's installation currently supports recent 64-bit Linux environments (i.e., Ubuntu 16.04 and up).
* **Supported binaries**: ZAFL supports instrumenting/transforming x86-64 Linux binaries of varying type (C and C++, stripped and unstripped, and position independent and position-non-independent). At this time, ZAFL cannot support binaries with DRM, obfuscation, or tamper-resistant protections.
* **Windows binaries**: ZAFL offers preliminary cross-platform instrumentation support for Windows 7 PE32+ binaries, though WinAFL-compatible fuzzing instrumentation is not yet supported at this time.
* **Recommended installation**: For first-time users we recommend the *Docker-based* installation. For developers and advanced users we recommend the *source-based* installation.
A Docker-based installation is recommended for those getting started with Zafl.
Please intending to develop new functionality or fix bugs in existing code should install from source (see next section).
## Installation from Docker (recommended)
To install the Docker-based installation, please see the [install and use directions for the Docker-based setup.](https://git.zephyr-software.com/opensrc/libzafl/-/wikis/home)
To install the docker-based installation, please see the [install and use directions for the Docker-based setup.](https://git.zephyr-software.com/opensrc/libzafl/-/wikis/home)
## Installation from Source (for developers)
Installing from source is recommended only for those intending to bug-fix or develop new features for ZAFL. **The following instructions assume you are running a compatible Linux environment** (i.e., Ubuntu 16.04 and up) **with AFL installed**.
## Installation from Source
Installing from source is not recommended for first-time Zafl users. Please see the Docker-based setup instructions.
This method is recommended only for those intending to bug-fix or develop new features for Zafl.
The instructions that follow assume that:
* you are using a recent version of Linux, e.g., Ubuntu 18.04
### Install AFL locally
```bash
git clone https://github.com/google/AFL
```
Follow directions to build and install AFL
### Install the Zipr static binary rewriting infrastructure
#### Step 0: Install the Zipr binary rewriting infrastructure
See directions for this at https://git.zephyr-software.com/opensrc/zipr
### Testing Zipr
#### Step 1: Testing Zipr
Test the binary rewriting infrastructure by rewriting /bin/ls
```bash
cd /tmp
$PSZ /bin/ls ls.zipr -c rida
```
Your terminal's output should look like this:
```
Using Zipr backend.
......@@ -53,46 +46,30 @@ Performing step fix_calls [dependencies=mandatory] ...Done. Successful.
Program not detected in signature database.
Performing step zipr [dependencies=clone,fill_in_indtargs,fill_in_cfg,pdb_register] ...Done. Successful.
```
Invoke the rewritten version of /bin/ls and make sure it runs normally:
```
./ls.zipr
```
### Download and Set environment for IRDB-SDK and IRDB-libs
Option 1:
If you built Zipr from source, you can use Zipr's set_env_vars feature to
include the right settings.
Option 2:
Download the sdk:
#### Step 2: Download and set environment for IRDB-SDK and IRDB-libs
**Option 1 (recommended)**: If you built Zipr from source, run its `set_env_vars` to prepare the environment.
**Option 2**: Manually download and configure the SDK:
```bash
git clone --recurse-submodules http://git.zephyr-software.com:opensrc/irdb-sdk.git
export IRDB_SDK=$PWD/irdb-sdk
```
Download the the built libraries for your system:
```
TBD
export IRDB_LIBS=/path/to/irdb-libs/libs
```
### Downloading and Building ZAFL
#### Step 3: Download and build ZAFL
Once Zipr has been installed, clone the repo for ZAFL and build.
```bash
git clone --recurse-submodules http://git.zephyr-software.com:opensrc/zafl.git
# or: git clone --recurse-submodules git@git.zephyr-software.com:opensrc/zafl.git
cd zafl
```
Setup your environment:
Prepare ZAFL's environment:
```bash
. set_env_vars
```
And build zafl.
Build ZAFL:
```bash
scons
# or scons debug=1
......@@ -100,60 +77,30 @@ scons
# or scons debug=1 -j3
```
## Testing Zafl
Before running Zafl, always make sure to have the proper environment variables set
## Using ZAFL for Binary Fuzzing
Before running ZAFL, always make sure to prepare both it and Zipr's environments:
```bash
cd ~/zafl_umbrella
. set_env_vars
cd ~/zipr && . set_env_vars
cd ~/zafl && . set_env_vars
```
Zafl also needs to find Zipr, which can be done by setting:
#### Step 0: Ensure ZAFL's smoke tests succeed
```bash
export PSZ=/path/to/ps_zipr.sh
# or
export PATH=$PATH:$(dirname /path/to/ps_zipr.sh)
cd ~/zafl/test/bc && ./test_bc.sh
```
#### Running Zafl smoke tests
This test will instrument and fuzz a copy of Linux's `bc` binary. If successful, the final output should contain something like:
```bash
cd $ZAFL_HOME/test/bc
./test_bc.sh
command_line : afl-fuzz -i zafl_in -o zafl_out -- ./bc.zafl
TEST PASS: ./bc.zafl: execs_per_sec : 1904.76
```
The test will run afl on bc, instrumented with the proper instrumentation inlined.
You will see several fuzzing run, each of which should take on the order of 30 seconds.
Once done, the output should end with something like:
```
unique_crashes : 0
unique_hangs : 0
last_path : 0
last_crash : 0
last_hang : 0
execs_since_crash : 30242
exec_timeout : 20
afl_banner : bc
afl_version : 2.52b
target_mode : default
command_line : afl-fuzz -i zafl_in -o zafl_out -- /usr/bin/bc
TEST PASS: /usr/bin/bc: execs_per_sec : 1904.76
~/zafl_umbrella/test/bc
```
#### Final sanity check
#### Step 1: Instrument your binary
**zafl.sh** is the primary script for instrumenting/transforming binaries for fuzzing. Below presents an example of running it on the `ls` binary
```bash
cd /tmp
zafl.sh /bin/ls ls.zafl
zafl.sh /bin/ls /tmp/ls.zafl
```
**zafl.sh** is the primary script for adding afl instrumentation to binaries.
You should see:
The output should contain the following:
```
zafl.sh /bin/ls ls.zafl
Zafl: Transforming input binary /bin/ls into ls.zafl
Zafl: Issuing command: /home/zafl_guest/zafl_umbrella/install/zipr_umbrella/peasoup_examples/tools/ps_zipr.sh /bin/ls ls.zafl -c move_globals=on -c zafl=on -o move_globals:--elftables -o zipr:--traceplacement:on -o zipr:true -o zafl:--stars
Using Zipr backend.
Detected ELF file.
Performing step gather_libraries [dependencies=mandatory] ...Done. Successful.
......@@ -169,48 +116,41 @@ Performing step zafl [dependencies=none] ...Done. Successful.
Performing step zipr [dependencies=clone,fill_in_indtargs,fill_in_cfg,pdb_register] ...Done. Successful.
```
You can run **ls.zafl** as you would **ls**: ```./ls.zafl```
Zafl'd binaries can be run normally. There is no extra output.
To make sure the binary has been instrumented properly: ```ZAFL_DEBUG=1 ./ls.zafl```
#### Step 2: Running an instrumented binary
All ZAFL'd binaries can be run normally just as their original uninstrumented versions: ```./tmp/ls.zafl```.
To ensure the binary has been instrumented properly: ```ZAFL_DEBUG=1 ./tmp/ls.zafl```
The output should start with:
```
Error getting shm environment variable - fake allocate AFL trace map
Success at mmap!
libautozafl: auto-initialize fork server
```
Let's prep for fuzzing:
#### Step 3: Fuzzing an instrumented binary
Let's prep a seed directory for fuzzing:
```bash
mkdir input_seeds
echo "hello" > input_seeds/hello.seed
mkdir in && echo "hello" > in/seed
```
Let's now run the Zafl'd binary with afl:
Let's now run the ZAFL'd binary with AFL:
```bash
afl-fuzz -i input_seeds -o out -- ./ls.zafl @@
afl-fuzz -i in -o out -- ./tmp/ls.zafl @@
```
If afl complains about `missing instrumentation`, you'll need to set the following environment variable:
If AFL complains about `missing instrumentation`, you'll need to set the following environment variable before fuzzing:
```bash
export AFL_SKIP_BIN_CHECK=1
```
You can also run the usual afl utilities, e.g:
ZAFL's instrumentation also supports AFL's other utilities, e.g.:
```bash
afl-showmap -o map.out -- ./ls.zafl
afl-cmin -i out/queue/ -o out.cmin -- ./ls.zafl @@
afl-showmap -o map.out -- ./tmp/ls.zafl
afl-cmin -i out/queue/ -o cmin.out -- ./tmp/ls.zafl @@
```
Et voila!
# TL;DR
Once everything is installed properly, you can prep a binary for fuzzing with the simple command:
Once everything is installed properly, you can prepare a binary for fuzzing with the simple command:
```bash
zafl.sh <target_binary> <zafl_output_binary>
```
To see what transformations ZAFL currently supports, simply run ```zafl.sh --help```.
**We welcome any community contributions, and ideas for improvements and new fuzzing transformations!** Feel free to contact us by opening up an `Issue`, or submitting new features through a `Merge Request`. **Happy fuzzing!**
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment