Research map / Large language models / Agents, code and math
From Commands to Prompts: LLM-based Semantic File System for AIOS
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LSFS, an LLM-based semantic file system layer on top of AIOS that lets users manage files via natural language prompts, with semantic indexes, syscalls, and APIs for retrieval, change summarization, rollback, and sharing.
Problem
Traditional file systems rely on metadata and exact commands, so users must remember cryptic file names and paths and cannot retrieve or organize files by semantic content; existing LLM file tools are application-specific rather than a general OS-level service.
Approach
Adds a semantic layer over traditional file systems: files are indexed with embeddings (all-MiniLM-L6-v2 via LlamaIndex), exposed through composite syscalls and APIs, and an LLM-based parser maps natural language prompts to these API calls, with user confirmation and rollback as safety mechanisms.
Key results
- With LSFS, target-file retrieval accuracy improves by at least 14% over LLM-only retrieval across 10–120 files (e.g., 92.9% vs 63.8% at 120 files with GPT-4o-mini) and retrieval time drops by 49.9–92.2%.
- Keyword-based retrieval achieves F1 of 0.891–0.909 for LSFS, versus 0.829–0.840 for the TFS search window and 0.337–0.402 for TFS-grep; TFS-grep* scores 1.000.
- The LSFS parser averages about 90% API-call parsing accuracy across Gemini-1.5-flash, GPT-4o-mini, Qwen-2, and Gemma-2, with 100% for some APIs on GPT-4o-mini and Qwen-2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LSFS semantic retrieval instead of LLM retrieval without LSFS | Semantic file retrieval with Gemini-1.5-flash · 10, 20, 40, and 120 files | accuracy: 75.0% -> 95.0%; 77.3% -> 91.3%; 70.91% -> 93.4%; 35.2% -> 92.9% | Helped |
| Use LSFS semantic retrieval instead of LLM retrieval without LSFS | Semantic file retrieval with GPT-4o-mini · 10, 20, 40, and 120 files | accuracy: 80% -> 95.0%; 69.1% -> 91.3%; 69.2% -> 93.4%; 63.8% -> 92.9% | Helped |
| Use LSFS semantic retrieval instead of LLM retrieval without LSFS | Semantic file retrieval with Gemini-1.5-flash · 10, 20, 40, and 120 files | retrieval time: 97.40(s) -> 14.39(s); 213.69(s) -> 16.69(s); 312.39(s) -> 23.86(s); 605.59(s) -> 48.08(s) | Helped |
| Use LSFS semantic retrieval instead of LLM retrieval without LSFS | Semantic file retrieval with GPT-4o-mini · 10, 20, 40, and 120 files | retrieval time: 61.14(s) -> 30.64(s); 129.92(s) -> 40.39(s); 239.49(s) -> 57.1(s); 938.68(s) -> 88.93(s) | Helped |
| Use LSFS keyword retrieval instead of TFS-grep* | Keyword file retrieval · 10, 20, and 40 files | F1-score: 1.000 vs 0.891; 1.000 vs 0.900; 1.000 vs 0.909 | Hurt |
| Use LSFS keyword retrieval instead of TFS search window or TFS-grep | Keyword file retrieval · 10, 20, and 40 files | F1-score: 10 files: 0.829, 0.402 -> 0.891; 20 files: 0.840, 0.337 -> 0.900; 40 files: 0.817, 0.348 -> 0.909 | Helped |
| Enable user confirmation before file operations | LSFS file retrieval for operations · 10, 20, 40, and 120 files | retrieval error rate: 13% -> 0%; 16.7% -> 0%; 15.8% -> 0%; 14.8% -> 0% | Helped |
Surprising findings
- TFS-grep* achieves perfect precision and recall, outperforming LSFS on keyword retrieval, which the authors attribute to manual format correction.
- Adding a user confirmation step reduced retrieval-driven operation errors from roughly 13–17% to 0% in the reported experiment.
Limitations
- Keyword retrieval by LSFS is below the manually corrected grep baseline (TFS-grep*), with precision and recall around 0.83–0.96.
- Gemma-2 and Qwen-2 were excluded from the retrieval comparison due to unstable performance.
- Evaluation uses small custom file sets (10–120 files) and custom prompts, with limited reported detail on dataset construction.
Experimental setup
Datasets: user-constructed file folders of research papers and documents (10, 20, 40, 120 files)
Benchmarks: semantic file retrieval (custom) keyword-based file retrieval (custom) LSFS parser API-call accuracy (custom, 30 samples per API)
Metrics: accuracy retrieval time precision recall F1-score success rate
Baselines: TFS search window TFS-grep TFS-grep* LLM without LSFS (Gemini-1.5-flash, GPT-4o-mini) AutoGPT Code Interpreter
Builds on: AIOS LlamaIndex
Closest papers by meaning
- LightRetriever: A LLM-based Text Retrieval Architecture with Extremely Faster Query Inference (ICLR 2026 · similarity 0.84)
- CRAFT: Customizing LLMs by Creating and Retrieving from Specialized Toolsets (ICLR 2024 · similarity 0.83)
- Leveraging Attention to Effectively Compress Prompts for Long-Context LLMs (AAAI 2025 · similarity 0.82)
- Needle Threading: Can LLMs Follow Threads Through Near-Million-Scale Haystacks? (ICLR 2025 · similarity 0.82)
- Gorilla: Large Language Model Connected with Massive APIs (NeurIPS 2024 · similarity 0.82)
- LLM Alignment as Retriever Optimization: An Information Retrieval Perspective (ICML 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.