Source: context/test-evidence/zset-ffi.md

On this page

Test evidence: zset-ffi

What each test guards and where a rival failed. Keyed by test name; the reasoning is in context/code/zset-ffi.md. The tests are crates/zset-ffi/tests/{abi,header,dylib}.rs.

Injected in the source, one at a time, each reverted and the suite rerun (a build that printed Running was confirmed each time). The failing tests, by name:

#overflow_is_an_error_code_and_leaves_the_set_unchanged

a holds k = MAX and j = 1; adding a set that overflows k must return the overflow code and leave both keys as they were. The key j sorts before k, so a partial in-place add changes j first; a fixture where the overflowing key came first would pass that rival.

#a_weight_overflow_in_a_step_is_a_code_and_poisons_the_circuit

A weight reaches MAX in an integrate, then one more overflows inside step. The step returns the overflow code, the next step and a new operator return the poisoned code.

#a_rebuild_resumes_across_calls_and_matches_one_step

Nine keys, chunks of two, a deadline callback that is always expired: the job needs five calls' worth of chunks (the callback is counted: five, one per chunk), and the sum of what the sink yields equals one uninterrupted step on a second circuit. The loop is bounded at 20.

#min_and_max_read_only_values_of_positive_weight

Weights of mixed sign in one group, where the smallest value has a negative weight, so > 0 and != 0 give different minima.

#a_loaded_library_adds_queries_frees_and_reports_overflow

The built libzset_ffi.{so,dylib} is opened with libloading and driven through symbol lookups: the contract a CFFI caller meets, including the exported names, not the rlib.

#the_header_declares_exactly_the_exported_functions_with_their_types

Parses src/lib.rs and the header (comments stripped), maps each Rust type to its C spelling and compares names, parameter types and return types; asserts the scanner saw more than 30 functions, so an empty parse cannot pass. The first version compared only arity and passed a retyped parameter.

#the_header_constants_are_the_rust_constants

Every ZS_ERR_*, ZS_ABI_VERSION, ZS_OK and the four ZS_AGG_* equal the Rust values; the the header's ZS_ERR_* lines are counted against Code::ALL, which the codes! macro generates, so a new variant without a line fails (it did, injected as Novel = 9).

#a_chunked_rebuild_through_delay_is_one_tick

After #345's review the rebuild is one logical tick. A snapshot of two keys, chunks of one, an always-expired deadline: the first call is done == 0, the second done == 1 (the last chunk is Done despite the deadline), and a delay sink shows the previous tick's key once. Under the old per-chunk ticking the delay showed the first chunk instead.

#Review of #347 (8a76469b)

Injected afterwards, one at a time, each failing by name: a null buffer written through (the test binary aborts, which is the failure); zs_circuit_input adding the input before checking out, and zs_circuit_take clearing the sink first: a_null_out_pointer_is_refused_before_any_state_changes; a rebuild stepped on another circuit: a_rebuild_is_stepped_only_on_its_own_circuit; the sink bit unchecked in take: stream_ids_and_sink_ids_are_not_interchangeable; a flat_map or keep failure ignored: a_callback_that_reports_failure_fails_the_step_and_kills_the_circuit; re-entry undetected, or step not setting in_step: a_callback_that_calls_back_into_its_circuit_gets_poisoned_and_changes_nothing (and the unit test below for the first); a count that drops its zero row: a_count_or_sum_that_cancels_to_zero_keeps_its_row; len32 truncating: a_length_of_4_gib_or_more_is_refused_not_truncated. The aliasing finding (zs_zset_add(a, a)) is not an assertion failure on a native build: only Miri finds it, and a_set_added_to_or_subtracted_from_itself_is_defined is the test Miri failed before the fix and passes after (recipe in zset-ffi.md).

#a_set_added_to_or_subtracted_from_itself_is_defined

Self-add doubles, self-sub empties, and a self-add that overflows leaves the set unchanged.

#a_null_buffer_is_refused_before_it_is_written

cap >= needed with a null buffer is ZS_ERR_NULL_ARGUMENT; a null buffer with too small a cap still probes the size.

#a_null_out_pointer_is_refused_before_any_state_changes

The next input id, the sink's contents and the job's progress are observed after a refused call with a null out-pointer; each is as before.

#a_rebuild_is_stepped_only_on_its_own_circuit

A job made from one circuit stepped on another is ZS_ERR_BAD_HANDLE and neither sink moves.

#stream_ids_and_sink_ids_are_not_interchangeable

A stream id given to take, and a sink id given to delay, output and push, are refused.

#a_callback_that_reports_failure_fails_the_step_and_kills_the_circuit

keep, flat_map and expired callbacks each return their failure value: ZS_ERR_CALLBACK, then ZS_ERR_POISONED.

#a_callback_that_calls_back_into_its_circuit_gets_poisoned_and_changes_nothing

A filter callback calls zs_circuit_input on its own circuit: it gets ZS_ERR_POISONED, the outer step completes, and no input was added.

#tests::a_circuit_left_mid_step_by_a_skipped_frame_answers_poisoned_to_everything

What a Lisp non-local exit through zs_circuit_step leaves behind is the in_step flag set; every later call is ZS_ERR_POISONED and writes nothing.

#a_count_or_sum_that_cancels_to_zero_keeps_its_row

Codex's finding: Some(0) was filtered away for a non-empty group.

#a_length_of_4_gib_or_more_is_refused_not_truncated

len32 refuses 1 << 32 and accepts u32::MAX.

#Delta review of #347 (9bcfa9d1)

Injected, one at a time: a fault that overwrites the first (note replaced by a plain set) fails an_index_by_callback_that_fails_reports_the_callback_code_not_a_later_fault; zs_rebuild_step borrowing the job before checking the circuit (Miri only), and a free that ignores in_step or busy, fail a_callback_cannot_step_or_free_the_job_or_circuit_it_runs_under (and under Miri, as use-after-free); an owner compared by address instead of id fails a_circuit_freed_and_another_made_does_not_inherit_its_jobs; the interrupted check removed fails a_host_step_between_rebuild_calls_is_interrupted_not_a_poison; a retyped ZsKeepFn fails the_header_callback_typedefs_match_the_rust_callback_types.

#an_index_by_callback_that_fails_reports_the_callback_code_not_a_later_fault

The callback fails (returns 3) and emits nothing, so index_by also sees zero keys. The step must report ZS_ERR_CALLBACK, the first fault, not the consequent BAD_VALUE.

#a_callback_cannot_step_or_free_the_job_or_circuit_it_runs_under

An expired callback steps its own job, frees the job and frees the circuit. The step is refused with ZS_ERR_POISONED, both frees do nothing, and the job then finishes normally.

#a_circuit_freed_and_another_made_does_not_inherit_its_jobs

A job outlives its circuit; sixty-four new circuits (some at the old address) all refuse it.

#a_host_step_between_rebuild_calls_is_interrupted_not_a_poison

ZS_ERR_INTERRUPTED, and the circuit still steps afterwards.

#the_header_callback_typedefs_match_the_rust_callback_types

Retyping ZsKeepFn's return in the header fails. The unary_op! signatures are now read from the macro body rather than written into the test.

#Transactions (improvise-bw4p.5)

Rivals were injected one at a time in engine.rs and watched fail by name; Miri passes on abi.

#a_rollback_after_a_failed_step_revives_the_circuit_in_its_old_state

An integrate held at i64::MAX; inside a transaction a step overflows (ZS_ERR_OVERFLOW, then ZS_ERR_POISONED), commit of it is refused as poisoned, and rollback revives the circuit with the integral where it was: -1 then gives MAX - 1, with neither the failed +1 nor the key added beside it. Rivals: a rollback that leaves the poison on, and one that does not roll the circuit back, fail this test and the callback one; a commit that does not check the poison fails this test.

#a_committed_transaction_keeps_its_steps_and_a_new_one_may_follow

commit keeps the step and its sink, and a later transaction can be opened and rolled back.

#a_failed_callback_step_is_rolled_back_too

A failure the callback only reports (the step returns, the fault is noted) is rolled back like a panic: the sink holds nothing after the rollback.

#transaction_misuse_is_a_code_and_changes_nothing

ZS_ERR_TRANSACTION for commit or rollback with none open, a second begin, and a take while one is open (nothing written to the out-pointer). A rollback after a poison from outside any transaction is also refused and the circuit stays poisoned. Rivals: take allowed in a transaction, and a second begin allowed, each fail this test.

#Review of #355 at 51438911

#f1_rebuild_begun_before_a_txn_continued_inside_and_rolled_back

Through the ABI, a job continued inside a rolled-back transaction returned 0 with done 1 and a partial load. It is now ZS_ERR_INTERRUPTED.

#f4_a_failed_rebuild_step_in_a_txn_then_rollback_then_resume

A job whose first chunk failed inside the transaction resumed without that chunk; now ZS_ERR_INTERRUPTED.

#f2_a_rollback_from_inside_a_callback_is_refused_and_the_txn_stays_open

ZS_ERR_POISONED from the callback's rollback, and the transaction still commits.

#f3_begin_on_a_circuit_poisoned_outside_a_txn_is_poisoned

begin is ZS_ERR_POISONED; commit and rollback are ZS_ERR_TRANSACTION.

#f5_a_chunk_whose_push_overflows_in_a_txn_then_rollback_is_interrupted

The same path through the ABI: zs_rebuild_step returns ZS_ERR_OVERFLOW from the chunk's push, zs_circuit_rollback succeeds, and the next zs_rebuild_step was a silent success; it is now ZS_ERR_INTERRUPTED. Rival: the mark before the push removed fails [f5].

#the_opaque_handles_are_send

ZsCircuit and ZsRebuild are Send (the engine's fault cell is an Arc<Mutex> and the callback context an unsafe impl Send wrapper). Rival: restoring the Rc<Cell> fault cell, or dropping unsafe impl Send for Ctx, fails engine.rs to build (*mut c_void cannot be sent).

#a_circuit_stepped_on_two_threads_runs_its_callbacks_on_the_thread_that_steps

A filter callback records its thread; the circuit is stepped on the test thread, moved, and stepped on another. The ids differ and match the stepping threads, so the callback is not pinned to the thread that built the circuit. The second thread's push is seen by its own step.