Rethinkdb Testing
Notes:
This repo is currently not working (nor has it ever worked).
RethinkDB has 2 major problems:
- Most testing is done at a unit test level. This testing is achieved
by linking components of the source code into a new binary that simply
invokes the unit tests. Zipring the original binary does nothing to gain
confidence in the Zipring process because the unit tests use a different binary.
** Zipring the unit tests binary may provide some additional confidence in Zipr.
But confidence is not proportional to the confidence gained from the original testing. The unit tests were designed to test the correctness of the source code, bit the compiler may have compiled things very differently than in the final binary which is compiled with optimizations, inlining, etc. Thus, Zipring the testing binary is not really gaining confidence that Zipr works OK on the final, optimized binary. ** The testing that could be performed on the final binary to gain more confidence in Zipr fall into the category integration tests. These are good tests that would work well for the purpose of gaining confidence in the Zipr'd binary. Unfortunately they are only part of the testing for RethinkDB and likely much lower coverage than the unit tests. Thus, they may not be sufficient to gain the confidence the original developers had in the full system. How comprehensive these tests are is unknown. - I cannot make the integration tests run.
** They run in CI on Github, but have been failing for so long that Github has discarded the logs from the last successful build. This discarding hampers progress. ** The Docker version of RethinkDB is 2.4.2. There is no specifications for how to run Gitub CI for 2.4.2. The Github version with CI is 2.3.2, so some changes in building and running tests has happened recently. There is some information about using CircleCI for CI in version 2.4.2, but critical information for building/running integration tests seems to be missing. ** So far, all attempts to at running any tests beyond the unit tests (on the original binary) have failed badly. I can run the unit tests, however.