Repository navigation
[TS PBT][Research] Establish novelty and deliver the reproducible property-feedback paper #404
Description
Part of #345. Literature and claim design start immediately; final manuscript/artifact delivery depends on #357.
Goal
Deliver a source-backed paper whose contribution is demonstrated property-aware PBT/USVM cooperation, with claims limited to measured capabilities and evidence.
Early research input and final delivery
Before #405 fixes its development protocol, provide a nearest-work comparison and a provisional claim/evidence matrix for the core mechanism: assertion-focused observations, symbolic challenges, exact coverage and returned-input generation. This early artifact does not require this issue to close, nor does it assert novelty before the literature supports it.
Distinguish core functional feasibility, pilot empirical promise, extension results and final held-out claims throughout the manuscript. #405 can complete with negative or inconclusive evidence and a recorded next decision; do not present it as a successful superiority claim. Keep the full #345 obligations visible, and narrow a paper claim when evidence requires it without silently marking unimplemented extensions complete. Final manuscript/artifact delivery still depends on #357.
Work
- Review primary sources on hybrid fuzzing/concolic testing, coverage-guided PBT (JQF/Zest and related work), dynamic invariant inference (Daikon), static/dynamic combinations (DSD-Crasher), hypothesis falsification, generator-choice search, metamorphic and stateful PBT, and context-specific summaries.
- Build a comparison matrix: reused user oracle/preconditions/generator structure, value observations, inferred relations, symbolic challenge, feedback rounds, shrinking, language, and evaluation. Check nearest mechanisms rather than claiming novelty from the TypeScript language alone.
- Include the supplied 2026 Go/gopter + usvm-go thesis accurately: uncovered-line guidance, corpus transfer and behaviorKey novelty filtering. Verify author/title/year and a citable public record or an agreed unpublished-source citation; do not publish a private supplied file.
- Refine research questions before final evaluation: incremental utility beyond coverage, value of assertion-focused relations, support-preserving generator reuse, multi-execution/stateful relations, internal-call context, and tradeoffs under fixed budgets.
- Rework the historical coverage/type-feedback manuscript around actual new results. Keep the ICCQ unknown-call/replay-policy and IFDS/type-inference contributions distinct; cite shared infrastructure and avoid double-counting experiments.
- Maintain a claim-to-evidence ledger linking each table/statement to pinned source revisions, benchmark manifest, raw runs and regeneration commands. Separate functionality, empirical improvement, model soundness and any bounded proof claims.
- Publish actual adaptation/annotation cost, unsupported fractions, negative results, semantic limits, nondeterminism and threats to validity.
- Produce a reproducible package and a clean-checkout reproduction attempt by an independent execution/review of the documented commands. No venue submission, author change or external publication is authorized by this issue.
Definition of Done
- Closest prior work and novelty boundaries are explicitly compared with primary citations.
- [TS PBT] Evaluate causal contributions of property-aware feedback and bounded extensions #357 results are regenerated from preserved raw artifacts, including timeouts and unsupported cases.
- The paper's major claims are backed by measurements; increased feature count or a constructed example alone is not called research superiority.
- Manuscript, artifact README, environment pins, corpus licenses and claim/evidence ledger are ready for author review. A negative empirical result leads to a narrower truthful claim, not hidden benchmark selection.
Starting primary sources
- JQF/Zest: https://github.com/rohanpadhye/JQF
- Daikon: https://plse.cs.washington.edu/daikon/
- DSD-Crasher abstract/bibliography: https://plse.cs.washington.edu/daikon/pubs/CsallnerSX2008-abstract.html
These are starting points, not an exhaustive novelty search. Follow primary papers and subsequent work before fixing novelty claims.
Activity