Research map / Probabilistic and scientific modeling / Molecules and proteins

Sequential Disentanglement by Extracting Static Information From A Single Sequence Element

ICML 2024 · Probabilistic and scientific modeling › Molecules and proteins · approach: Equivariant and disentangled latent representations · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DBSE, a sequential disentanglement VAE whose static posterior is conditioned on a single sequence element and whose dynamic encoder subtracts that static code from sequence features, reducing information leakage without mutual-information losses.

Problem

Unsupervised sequential disentanglement suffers from information leakage, where static information leaks into dynamic codes or vice versa, when static and dynamic codes are conditioned on the entire sequence.

Approach

Condition the static posterior on a single sequence element (x_1) rather than the whole sequence, and subtract the resulting static features from the encoded sequence before the LSTM dynamic encoder, giving a subtraction inductive bias with no mutual-information loss terms.

Key results

• On MUG, achieves 86.90% generation accuracy, IS 5.598, and improves the static-vs-dynamic leakage gap to 78.57% (static-sampling) and 84.01% (dynamic-sampling) over the prior best.
• On PhysioNet, reaches AUPRC 0.473 and AUROC 0.858, outperforming SPYL (0.367/0.764) and the raw-feature baseline RF (0.446/0.802).
• On TIMIT, reports a disentanglement gap of 31.11%, about 1.3 points above SPYL (29.81%).

Measured effects reported in the paper

Surprising findings

• Randomly sampling the anchor element per batch (instead of a fixed element) induces information leakage into the static code.
• Ablation removing both the static loss and subtraction degrades MUG accuracy substantially (to 10.94% static-gap-related metrics), showing both components matter.

Limitations

• Depends on the VAE backbone, which often yields low-quality reconstructions and samples.
• Assumes every sequence element contains the static factor; fails when some elements lack static information.
• Static information is extracted from a single chosen element, which may limit use on sequences where that element is uninformative.

Experimental setup

Datasets: Sprites MUG PhysioNet Air Quality ETTh1 TIMIT Jesters

Benchmarks: Sprites MUG PhysioNet ETTh1 Air Quality TIMIT Jesters

Metrics: Accuracy Inception score intra-entropy H(y|x) inter-entropy H(y) leakage gap AUPRC AUROC MAE EER classification accuracy

Baselines: MoCoGAN DSVAE R-WAE S3VAE SKD C-DSVAE SPYL FHVAE GP-VAE GLR

Builds on: DSVAE C-DSVAE FHVAE SPYL

Closest papers by meaning

• Sample and Predict Your Latent: Modality-free Sequential Disentanglement via Contrastive Estimation (ICML 2023 · similarity 0.90)
• Disentanglement Beyond Static vs. Dynamic: A Benchmark and Evaluation Framework for Multi-Factor Sequential Representations (NeurIPS 2025 · similarity 0.90)
• Disentanglement via Latent Quantization (NeurIPS 2023 · similarity 0.89)
• Multifactor Sequential Disentanglement via Structured Koopman Autoencoders (ICLR 2023 · similarity 0.89)
• $\alpha$TC-VAE: On the relationship between Disentanglement and Diversity (ICLR 2024 · similarity 0.87)
• Graph-based Unsupervised Disentangled Representation Learning via Multimodal Large Language Models (NeurIPS 2024 · similarity 0.87)