Research map / Detection and segmentation / CNN features
Lane2Seq: Towards Unified Lane Detection via Sequence Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Lane2Seq casts lane detection in segmentation, anchor, and parameter formats as sequence generation with a plain ViT encoder-decoder trained by cross-entropy, then tunes it with REINFORCE using format-specific metric rewards (MFRL).
Problem
Lane detection methods rely on divide-and-conquer task-specific head networks and loss functions, one per output format (segmentation, anchor, parameter), which makes models complicated and adds hyperparameters.
Approach
Unify the three lane detection output formats as discrete token sequences generated token-by-token by a plain transformer encoder-decoder with a format-transcription prompt, and inject task-specific knowledge via RL (REINFORCE) with new evaluation-metric-based rewards (LIoU, mIoU, keypoint distance, minus false-positive penalties) instead of extra heads.
Key results
- Lane2Seq (segmentation) reaches 97.95% F1 on Tusimple and 97.42% F1 on LLAMAS, new state of the art on both benchmarks.
- On CULane, Lane2Seq (segmentation) reaches 79.64% F1, above AtrousFormer's 78.08%, but CLRNet (79.73%) remains slightly higher overall.
- MFRL raises F1 on CULane by 15.37 points for segmentation (64.27% to 79.64%), 13.04 for anchor, and 10.25 for parameter formats.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MFRL to segmentation-format Lane2Seq (ablation) | Lane2Seq (segmentation) · CULane | F1 (%): 64.27 -> 79.64 (+15.37) | Helped |
| add MFRL to anchor-format Lane2Seq (ablation) | Lane2Seq (anchor) · CULane | F1 (%): 66.23 -> 79.27 (+13.04) | Helped |
| add MFRL to parameter-format Lane2Seq (ablation) | Lane2Seq (parameter) · CULane | F1 (%): 68.14 -> 78.39 (+10.25) | Helped |
| optimize only the segmentation reward Rseg (ablation) | Lane2Seq (segmentation) · CULane | F1 (%): 64.27 -> 77.69 (+13.42) | Helped |
| optimize only the anchor reward Ra (ablation) | Lane2Seq (anchor) · CULane | F1 (%): 66.23 -> 76.94 (+10.71) | Helped |
| optimize only the parameter reward Rp (ablation) | Lane2Seq (parameter) · CULane | F1 (%): 68.14 -> 77.02 (+8.88) | Helped |
| use all three format-specific rewards instead of a single reward (ablation) | Lane2Seq · CULane | F1 (%): 77.69 -> 79.64; 76.94 -> 79.27; 77.02 -> 78.39 | Helped |
| set λ4, λ5, and λ6 to 0.2, 1, and 1.5 (ablation) | Lane2Seq (segmentation) · CULane | F1 (%): 73.23 -> 79.64 | Helped |
Surprising findings
- Lane2Seq does not have a significant inference speed advantage; token-by-token decoding is slow, especially with more than 5 lanes.
- Optimizing multiple format rewards jointly gives better results than optimizing any single reward, which the authors attribute to cross-format correlations.
Limitations
- Sequence decoding is expensive and slow for scenes with more than 5 lanes, limiting real-time use.
- MFRL can currently only be applied to sequence-generation-based models.
- Lane2Seq trails CLRNet on CULane and BSNet on parameter-based results, using only a single-scale plain transformer.
Experimental setup
Datasets: CULane Tusimple LLAMAS
Benchmarks: CULane Tusimple LLAMAS
Metrics: F1 score accuracy false positives false negatives precision recall FPS MACs
Baselines: SCNN RESA AtrousFormer LaneAF LaneATT CondLaneNet CLRNet BSNet GANet YOLOP LSTR BezierLaneNet
Builds on: Pix2seq Task-Reward MAE Vision Transformer
Closest papers by meaning
- LaneSegNet: Map Learning with Lane Segment Perception for Autonomous Driving (ICLR 2024 · similarity 0.88)
- Selective Focus: Investigating Semantics Sensitivity in Post-training Quantization for Lane Detection (AAAI 2024 · similarity 0.88)
- LaneGraph2Seq: Lane Topology Extraction with Language Model via Vertex-Edge Encoding and Connectivity Enhancement (AAAI 2024 · similarity 0.88)
- A Siamese Transformer with Hierarchical Refinement for Lane Detection (NeurIPS 2024 · similarity 0.87)
- A Hybrid Global-Local Perception Network for Lane Detection (AAAI 2024 · similarity 0.87)
- GSENet:Global Semantic Enhancement Network for Lane Detection (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.