AI Paper Atlas

Research map / Architectures and efficiency / Attention and transformers

How Should Transformers Encode Numeric Values in Electronic Health Records?

Maria Elkjær Montgomery, Christian Igel, Mikkel Fruelund Odgaard, Martin Sillesen, Mads Nielsen

ICML 2026 · Architectures and efficiency › Attention and transformers · approach: Transformer expressivity and learnability · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use FiLM value-concept conditioning instead of binned token encodingsEHR transformer · synthetic addition, multiplication, polynomial tasksinformation efficiencyImproved
use combination (continuous token insertion) vs FiLMEHR transformer · precision-sensitive arithmetic tasksinformation efficiencyHurt
add binning before linear projection (combined binning)continuous token encoding · low-data synthetic classificationAUCImproved
use discretised tokens instead of FiLMEHR transformer · sharp-edge detectioninformation efficiencyImproved
add laboratory values to lab-name-only baselineEHR transformer · breast cancer, lung cancer, stroke predictionAUROCMixed
use concatenation encoding for lab valuesEHR transformer · lung cancer predictionAUROCHurt
truncate FiLM context to match two-token binned methods (ablation)FiLM · clinical prediction tasksAUROC: p > 0.05No clear effect
vary MSE loss weight w in 0.001 to 1.5 (ablation)combination method · synthetic classification n=100,000information efficiencyNo clear effect
fit power law instead of log or linear for optimal bins vs Nbin-count heuristic · synthetic classificationR^2: 0.74 -> 0.83Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy