Research map / Label-efficient and robust learning / Recognition and long-tail
Data-Driven Discovery of Design Specifications (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes D4SPEC, a data-driven method that mines candidate design specifications from an auxiliary random forest and checks them with the SAT-based Tree Ensemble Accreditor to trade off false-positive rate against specification adherence on breast cancer data.
Problem
Formal verification of ML models needs domain-specific design specifications, which are time-consuming to collect from subject matter experts.
Approach
Generate candidate input-output specifications automatically from leaves of a shallow auxiliary random forest (with support and purity thresholds), then verify them against the trained ensemble with TEA and select the prediction threshold via SPEC-ROC curves.
Key results
- Lowest FPR threshold for adherence: 0.336 for Spec 1, 0.001 for Spec 2, 0.026 for Spec 3.
- Spec 2 (most complex rules) is satisfied at any prediction threshold, while Spec 1 requires severely limiting discriminative capability.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the more complex Spec 2 rules instead of the simpler Spec 1 rules | Random forest with D4SPEC specifications · Breast Cancer Wisconsin Diagnostic | minimum FPR threshold for specification adherence: 0.336 -> 0.001 | Helped |
Surprising findings
- Simpler Spec 1 yields the highest FPR threshold, meaning it is hardest to satisfy when it misaligns with the model's learned logic; complexity explained the ordering.
Limitations
- Evaluated on a single small tabular dataset with three hand-inspected specifications.
- Relies on random forests, the only model class supported by the TEA formalism.
Experimental setup
Datasets: Breast Cancer Wisconsin Diagnostic
Benchmarks: Breast Cancer Wisconsin Diagnostic
Metrics: false positive rate threshold for specification adherence ROC curve
Builds on: Tree Ensemble Accreditor SPEC-ROC
Closest papers by meaning
- SMLE: Safe Machine Learning via Embedded Overapproximation (AAAI 2025 · similarity 0.82)
- Guided Data Generation for Understanding Model Behavior (NeurIPS 2026 · similarity 0.82)
- Classification with Conceptual Safeguards (ICLR 2024 · similarity 0.81)
- On the Ability of Developers' Training Data Preservation of Learnware (NeurIPS 2024 · similarity 0.81)
- Trustworthy Machine Learning through Data-Specific Indistinguishability (ICML 2025 · similarity 0.81)
- Reliable Algorithm Selection for Machine Learning-Guided Design (ICML 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.