Research map / Architectures and efficiency / RNNs and normalization
HyperLogic: Enhancing Diversity and Accuracy in Rule Learning with HyperNets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HyperLogic, which uses a HyperGAN-style hypernetwork to generate weights of a DR-Net rule-learning network, yielding diverse, accurate, and concise rule sets from one training session.
Problem
Differentiable rule learning networks are restricted in flexibility by interpretability-driven structure constraints, and typically learn only one rule set per training run.
Approach
Use a hypernetwork (HyperGAN-based) that samples Gaussian noise to generate the weights of a shallow binary rule-learning network, trained with a KL-to-high-entropy-prior diversity term and an L1 sparsity term on the OR-layer weights, so that many candidate rule sets can be sampled after training and the best one selected.
Key results
- On nested 5-fold CV test accuracy, HyperLogic reaches 84.90% on magic, 83.11% on adult, 85.22% on house, and 71.03% on heloc, versus DR-Net's 83.69%, 82.95%, 85.05%, and 70.07%.
- Compared with DR-Net, HyperLogic reduces both model complexity and rule complexity (Fig. 3).
- On large synthetic datasets (5000 dimensions), HyperLogic averages about a 6% F1 gain over DIFFNAPS in the regime with more categories, though it is worse at K=2 and K=5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add HyperLogic hypernetwork weight generation to DR-Net | DR-Net · magic | test accuracy (%): 83.69 ± 0.55 -> 84.90 ± 0.73 | Helped |
| Add HyperLogic hypernetwork weight generation to DR-Net | DR-Net · adult | test accuracy (%): 82.95 ± 0.45 -> 83.11 ± 0.55 | Helped |
| Add HyperLogic hypernetwork weight generation to DR-Net | DR-Net · house | test accuracy (%): 85.05 ± 0.51 -> 85.22 ± 0.62 | Helped |
| Add HyperLogic hypernetwork weight generation to DR-Net | DR-Net · heloc | test accuracy (%): 70.07 ± 0.83 -> 71.03 ± 1.07 | Helped |
| Add HyperLogic weight generation to DIFFNAPS | DIFFNAPS · Cardio | AUC: 0.56 -> 0.57 | Helped |
| Add HyperLogic weight generation to DIFFNAPS | DIFFNAPS · Disease | AUC: 0.84 -> 0.86 | Helped |
| Add HyperLogic weight generation to DIFFNAPS | DIFFNAPS · BRCA-N | AUC: 0.91 -> 0.95 | Helped |
| Add HyperLogic weight generation to DIFFNAPS | DIFFNAPS · BRCA-S | AUC: 0.86 -> 0.89 | Helped |
| Increase the number of sampled candidate rule sets M2 (ablation) | HyperLogic · magic, adult, house, heloc | test accuracy | Mixed |
| Increase the number of sampled candidate rule sets M2 (ablation) | HyperLogic · magic, adult, house, heloc | model complexity | Mixed |
| Increase training sample count M1 (ablation) | HyperLogic · magic | test accuracy | Mixed |
| Increase diversity regularization coefficient λ1 (ablation) | HyperLogic · magic | test accuracy | Mixed |
Surprising findings
- Selecting the best rule set on training accuracy does not guarantee the best test accuracy (Fig. 2).
- HyperLogic underperforms DIFFNAPS on the 2- and 5-category synthetic datasets (e.g., 0.726 vs 0.788 F1 at K=2).
Limitations
- Introduces additional hyperparameters (M1, M2, lambda1, lambda2) that must be tuned.
- Weight combination and ensemble learning strategies are not fully explored; ensemble accuracy peaks and then declines as L grows.
- Current main network is a two-layer DR-Net limited to binary classification.
Experimental setup
Datasets: MAGIC gamma telescope adult census FICO HELOC home price prediction (house) Cardio Disease BRCA-N BRCA-S
Benchmarks: magic adult heloc house Cardio Disease BRCA-N BRCA-S
Metrics: test accuracy F1 score AUC model complexity rule complexity Jaccard similarity unique rule set count
Baselines: DR-Net CG BRS RIPPER DIFFNAPS CLASSY
Builds on: DR-Net HyperGAN DIFFNAPS
Closest papers by meaning
- HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworks (ICLR 2025 · similarity 0.84)
- HyperNet Fields: Efficiently Training Hypernetworks without Ground Truth by Learning Weight Trajectories (CVPR 2025 · similarity 0.84)
- Neural Reasoning Networks: Efficient Interpretable Neural Networks with Automatic Textual Explanations (AAAI 2025 · similarity 0.84)
- Neuro-Symbolic Temporal Point Processes (ICML 2024 · similarity 0.83)
- Foundation Models Secretly Understand Neural Network Weights: Enhancing Hypernetwork Architectures with Foundation Models (ICLR 2025 · similarity 0.83)
- LightHGNN: Distilling Hypergraph Neural Networks into MLPs for 100x Faster Inference (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.