Jest and JSVerify: Property Based Testing

Jest

Is probably a framework you’ve used if you are a nodejs head.

Property Based Testing

This is an approach where your tests pulls random numbers during it’s operation. The seed is controlled by the library and failing tests will spit the seed as a string. I use jsverify. If you want this in C++ use rapidcheck.

Jest + JSVerify

Imagine you previously wrote workingFunction() but it’s slow. Now you want to replace it with experimentalReplacementForOldWorkingFunction() but aren’t sure if all possible inputs map out correctly. This is how I would “Bake in” a randomized test to make sure my new function was working. Eventually after this test is passing, you refactor the old function name and just put the new body in there. Of course you would need now to save the old, slow version (so this test can exist). I normally put this in the test file in question. No need for a shared include or access from your runtime anymore.

// works great
function workingFunction(vin: Vec4): number {
  return 0;
}

// you just wrote this and aren't sure if it works for all inputs
function experimentalReplacementForOldWorkingFunction(vin: Vec4): number {
  return 0;
}

test('test if function replacement works for all inputs', done => {
  jest.setTimeout(1000 * 60);
  const body = () => {
    
    // generate a random vector of length 4 over the range of [-1,1]
    // note if these get changed to 1E2 there are less possible vectors generated but it will
    // take fewer test runs to hit all possible values.
    const rv4 = () => {
      return <Vec4>[
        jsc.random(-1E3, 1E3)/1E3,
        jsc.random(-1E3, 1E3)/1E3,
        jsc.random(-1E3, 1E3)/1E3,
        jsc.random(-1E3, 1E3)/1E3
      ];
    };
    
    // our ideal input
    let ideal = rv4();
    
    // our expected output
    let expected = workingFunction(ideal);
    
    // our second output, in question
    let got = experimentalReplacementForOldWorkingFunction(ideal);
    
    // do the functions return the same number?
    expect(got).toBe(expected); 
    
    return true;
  }
  
  // wrap our lambda in this jsc thing. t wraps the call to body
  const t = jsc.forall(jsc.constant(0), body);
  
  const props: any = {tests: 1E4}; // 10k tests
  // const props: any = {tests: 1E2}; // run fewer tests
  
  // this is where we can put seeds
  // props.rngState = "031acbde5804f27639";
  // props.rngState = "077fd3ab6948015ce2";
  
  // actually run the test, run t which calls body
  expect(jsc.check(t, props)).toBe(true);
  
  done();
});