Towards general embodied intelligence: integrating large language models, knowledge bases, and reasoning capabilities to build the next generation of AI agents

arXiv:2608.197942026-08-21

A survey on what it takes to turn talkative AI into robots that can actually act in the world

This survey reviews how large language models (LLMs), structured knowledge bases, reasoning abilities, and physical embodiment (robots or agents that sense and act in the real world) can come together to build general embodied intelligence. The authors summarize existing research across these areas in comparison tables, point out their limitations, and propose a conceptual framework linking them. It is explicitly a guiding roadmap rather than a working engineered system.

What they did

The paper starts from the observation that LLMs like GPT-4, PaLM, Gemini, and Claude are fluent in language but remain 'disembodied'—they have no grounded understanding of the physical world
It reviews LLM architecture and training, methods for connecting LLMs to knowledge bases, reasoning frameworks such as Chain-of-Thought (step-by-step reasoning) and Tree-of-Thought (branching reasoning), and embodied intelligence research where agents learn and act in physical or simulated environments
A conceptual framework is proposed to show how these four pieces—LLMs, knowledge bases, reasoning, embodiment—could work together for perception, reasoning, and action, though the authors are clear it is a guiding model, not an implemented engineering architecture
Five key open challenges are identified for reaching general embodied intelligence: efficient/lightweight LLM deployment, closed-loop knowledge integration that updates in real time, hybrid symbolic-neural reasoning, perception-action grounding (turning language commands into physical actions), and continual learning without forgetting
Compared to prior embodied-AI reviews, the authors claim this is the first to systematically unify language models, knowledge, reasoning, and physical interaction into one architecture

Fig. 1: Schematic diagram of a unified multimodal large language model architecture.

TABLE I: Comparative Summary of Related Embodied Intelligence Review Articles
Authors	Year	Core Contributions	Limitations
Yao et al. [151]	2025	Reviews multimodal embodied intelligence tasks and models; explores LLM/MLM integration with robots; proposes PPA paradigm and evaluates MLMs.	Classical AI lacks physical interaction; DRL is data-hungry and less robust; VLN training is costly and generalization is poor; LLMs lack causal reasoning; Sim2Real transfer is weak; evaluation standards are lacking.
Dong et al. [135]	2025	Analyzes LLM-enabled Human-Robot Symbiotic Manufacturing (HRSM), including interaction, collaboration, and execution.	Privacy and data sensitivity risks; hallucination and bias in LLMs; absence of unified evaluation standards; integration difficulties in diverse environments; token length limits document processing.
Tan et al. [70]	2025	Proposes CEIM to fuse multi-source data and LLM reasoning for semantic integration and decision-making in customized manufacturing.	Data fusion issues; low semantic alignment; real-time control accuracy is lacking; high computation cost; insufficient standardization of AI-device interfaces.
Jeong et al. [59]	2024	Surveys five directions of LLM/VLM in robotics: reward design, control, planning, manipulation, and scene understanding.	Limited by embedded compute; weak sensor fusion; prompt bias and poor interpretability; lacks domain-specific knowledge; Sim2Real generalization is weak.
Wang et al. [68]	2025	Discusses LLMs in robotics tasks and evaluates GPT-4V; explores future applications in agriculture and medicine.	Plan robustness is lacking; long prompts require expert input; fixed action sets reduce flexibility; API is closed-source; RGBD-based 3D reasoning is limited.
Our Survey	2025	Systematically analyzed the synergy among LLMs, knowledge bases, reasoning capabilities, and embodied intelligence; proposed a unified framework and five future directions for GEI.	Limited empirical examples; insufficient discussion on multilingual settings and real-world deployment.

Fig. 2: Evolution and Taxonomy of Large Language Models.

TABLE II: Comparison of Research on Large Language Models (LLMs)
Author(s)	Core Improvement	Advantages	Application Scenarios
Cummins et al. [25]	Compiler optimization framework	Enhances disassembly processes	Compiler optimization
Hu et al. [174]	Prompting strategies incorporating context to enhance VQA performance	Strengthens contextual understanding	Knowledge-based applications
Hu et al. [166]	LLM-TIKG: Threat Intelligence Knowledge Graph framework	Enhances cybersecurity analysis	Threat Intelligence Analysis
Qian et al. [80]	Infrastructure for large-scale LLM training	Efficiently handles large-scale training tasks	LLM training infrastructure
Sun et al. [121]	UniCoder: Intermediate representation to improve downstream NLP task processing	Enhances the model’s ability to handle complex tasks	Model scaling
Zhang et al. [156]	BB-GeoGPT: Framework tailored for Geographic Information Science	Highly adaptable to domain-specific knowledge	Geographic Information Science (GIS)
Wang et al. [139]	VideoAgent: Vision-language integrated system for long-form video interpretation	Improves multimodal reasoning capabilities	Multimodal reasoning
Xia et al. [19]	LLMGA: LLM-based generation assistant combining image generation and editing tools	Precise control over image synthesis,	Image generation and editing
Zhou et al. [172]	Weak-to-Strong Search: Greedy search for aligning large models with smaller counterparts	Compute-efficient, enhances model generalization	Model alignment and fine-tuning
Land et al. [110]	Glitch token analysis and detection method	Improves tokenizer robustness	tokenizer optimization

Fig. 3: Schematic architecture of the Transformer model.

TABLE III: Summary of Key Contributions in Embodied Intelligence Research
Author	Year	Research Topic	Key Contribution	Application Domain
Zardini et al. [46]	2020	Co-Design Framework	Structured co-design for integration	General systems
Ma et al. [18]	2021	Soft Robotics Actuators	Integrated sensing & actuation	Soft robotics
Gupta et al. [4]	2021	DERL for Adaptation	Co-evolution of body and control	Locomotion & control
Roy et al. [96]	2021	ML in Embodied Systems	Energy-aware physical learning	Robotics & ML
Mengaldo et al. [45]	2022	Physics Modeling	Physics-based modeling	Soft robotics
Iida et al. [43]	2022	Temporal Dynamics	Timescale-based self-organization	Adaptive systems
Long et al. [158]	2023	Human-in-the-loop Learning	Interactive simulation for training	Surgical robotics
Zeng et al. [39]	2023	LLMs for Robotics	Survey of LLM-enhanced robotics	Human-robot interaction
Fan et al. [54]	2024	LLMs in Industry	LLM-based autonomous control	Industrial robots
Feng et al. [69]	2025	Spatial AI Reasoning	Spatial AI via LLMs	Multiscale AI systems

Fig. 4: Construction Process of Large Language Models.

TABLE IV: LLM and Knowledge Graph Integration
Author(s)	Innovation	Advantages	Drawbacks
Daniel et al. [30]	Verifying RDF triples using LLMs by comparing RDF statements to external documents	High precision (88%) in RDF verification	Requires human oversight, method may be slower
Feng et al. [155]	Knowledge Graph–based thought framework to improve LLM responses with verified KG information	Reduces factual errors, enhances drug–cancer associations	Limited to biomedical, requires cross-domain testing
Dai et al. [140]	Empirical study on LLMs’ comprehension of KG input formats and prompt organization	Improves LLM performance in KG-based tasks	Varying effectiveness across different models, noisy subgraphs
Erickson et al. [75]	Enhancing LLM workflows with RAG and KGs to ensure traceability and flexibility	More accurate and explainable LLM outputs	Integration with external tools can be complex
Xiang et al. [138]	LLM-based generative retrieval framework focusing on trustworthy search results	Improves relevance, responsibility, and trustworthiness	Works better with certain query types, not universally applicable
Sequeda et al. [77]	LLM-powered enterprise question answering, supported by KGs	Provides formal validation and trusted data for queries	May not scale well for large or complex queries in diverse enterprises

Fig. 5: Description of the execution process of the intelligent document processing and knowledge question answering system.

TABLE V: LLM for Domain-Specific Applications
Authors	Innovation	Advantages	Drawbacks
Wang et al. [41]	LLM-KGMQA: Multi-hop question-answering system for the medical field, addressing entity linking and reasoning	Improved accuracy (99.80%) and robust performance	Requires significant computational resources for entity linking and multi-hop reasoning
Jiang et al. [72]	Knowledge-guided agricultural LLM for domain-specific decision-making	State-of-the-art performance in agriculture, improves accuracy in agricultural queries	Data scarcity in the domain, challenges in precision retrieval
Benjira et al. [137]	LLM-augmented KG for SDG indicator mapping, integrating open data	High precision in mapping open data to SDGs	Requires access to high-quality and diverse metadata
Freire et al. [109]	LLM-powered knowledge-sharing system for manufacturing, retrieving information from factory documentation	Enhances quick information retrieval and resolution of issues	Preference for human experts still prevails, open-source models may have privacy concerns
Xie et al. [122]	Theme-based lecture summary system using LLMs and graph-based segmentation	Enhances knowledge recall and student satisfaction	Effectiveness varies with lecture structure
Sun et al. [88]	SF-GPT: A training-free method for enhancing LLM-based KG construction	Improved recall and F1 score, no training required	May not work well with all types of datasets or knowledge fusion

Fig. 6: Flowchart of knowledge base construction and use in large language model.

TABLE VI: Category 1: Research Based on Reasoning Frameworks and Methods
Authors	Year	Innovation	Advantages	Limitations
Wu et al. [51]	2025	Proposed Long-to-Short (L2S) reasoning and model merging.	Increases reasoning efficiency while maintaining performance.	Merging efficiency depends on model size, adds computational complexity.
Tung et al. [90]	2025	Introduced GreenMind, using strategy optimization for reasoning.	Outperforms previous works on Vietnamese reasoning tasks.	Limited to specific languages, struggles with low-resource languages.
Ma et al. [100]	2025	Introduced SQL-R1, a reinforcement learning-based model for NL2SQL.	Improves accuracy in complex database tasks.	Reinforcement learning training may suffer from cold start issues.
Costas [28]	2024	GNN-RAG combines GNN reasoning with LLMs in a retrieval-augmented generation style.	Optimizes graph data processing, improves knowledge graph QA.	Performance depends on graph data quality, limited in complex reasoning.
Sui et al. [162]	2024	Proposed FiDeLiS for factuality-enhanced LLM reasoning.	Improves factuality and interpretability.	Dependency on reasoning steps increases computational cost.
Luo et al. [116]	2025	Introduced RALLRec+ for enhancing recommendation system reasoning.	Enhances recommendation reasoning, handles dynamic user preferences.	Requires continuous data updates, may still be less effective in real-time applications.

Fig. 7: Summary of inference optimization techniques in large language models.

TABLE VII: Category 2: Application-Specific Reasoning Research
Authors	Year	Innovation	Advantages	Limitations
Yang et al. [57]	2025	Introduced Reasoning-based Bias Detector (RBD).	Reduces bias in evaluations, improves consistency.	Needs significant supervision for bias detection.
Zhang et al. [73]	2024	Designed step-wise multimodal models for document reasoning.	Enhances multimodal understanding, improves complex question answering.	Relies heavily on high-quality labeled data.
Beary et al. [14]	2025	Introduced hybrid fine-tuning to enhance medical domain reasoning.	Improves reasoning accuracy in radiation oncology.	Domain-specific, limited transferability to other medical fields.
Tan et al. [141]	2025	Proposed Hydra framework for cross-source enhanced reasoning.	Excels in multi-hop and multi-entity reasoning tasks.	Requires diverse data sources, computationally intensive.
Kunat et al. [81]	2025	Merged language-specific LLMs with advanced reasoning models.	Enhances reasoning in low-resource languages.	Can be computationally expensive for merging models.
Lan et al. [95]	2024	Proposed LLM4QA for graph-based reasoning with SPARQL queries.	Improves knowledge graph reasoning efficiency.	Relies on structured knowledge graphs, limited in unstructured data.
Li et al. [66]	2024	Proposed LINKED for filtering and integrating knowledge in reasoning tasks.	Improves commonsense reasoning accuracy.	Performance may degrade with noisy knowledge sources.
Inoue et al. [160]	2025	Introduced DrugAgent, a multi-agent LLM system for drug-target prediction.	Enhances reliability and transparency in drug-target predictions.	Multi-agent approach may face challenges in information integration.
Phuc et al. [127]	2025	Proposed QUERY2TREE for reasoning over knowledge graphs.	Improves logical query accuracy with knowledge graph embeddings.	Dependent on knowledge graph quality, may struggle with complex queries.

Fig. 8: The core components and development path of embodied intelligence.

TABLE VIII: LLM-driven Embodied Task Execution and Planning
Authors	Year	Innovation	Advantages	Limitations
Fan et al. [55]	2025	Proposed a three-stage framework for industrial robot task execution using LLMs	Outstanding GPT-4 performance in industrial tasks	Limited ability in 3D spatial planning
Chen et al. [152]	2025	Designed RoboGPT with long-term decomposition and feedback adjustment	Adaptive subgoal updating	Requires large datasets and complex modules
Ruaridh et al. [106]	2025	Introduced ELLMER with RAG for multi-step reasoning	Strong sensorimotor adaptation	Tested mainly in home-like tasks
Li et al. [91]	2024	Proposed multi-modal LLMs for autonomous mining driving	Applicable to realistic mining scenarios	Conceptual work, lacks implementation
Song et al. [23]	2023	Introduced LLM-Planner for few-shot planning	Strong performance with limited data	Weak environmental state modeling
Andrew et al. [11]	2025	Unified GEA model across embodied tasks via MLLM	High cross-domain generalization	Data-hungry and complex training
Vishnu et al. [128]	2023	Proposed LGX for zero-shot navigation	Large improvement in success rate	Validated mostly in simulation
Schumann et al. [102]	2024	VELMA verbalizes visual context for navigation	Greatly improves city-scale VLN	Requires explicit visual-language mapping
Wu et al. [173]	2023	TaPA plans executable actions with scene grounding	Outperforms GPT-3.5 in planning	Object recognition impacts execution
Danny et al. [31]	2023	PaLM-E integrates vision, state, and language	Versatile across modalities and tasks	Costly training and deployment
Mower et al. [26]	2024	ROS-LLM enables non-expert robot programming via chat	Supports multiple behavior modes	Relies on full ROS ecosystem

Fig. 9: The development of embodied intelligence.

TABLE IX: Multi-Modal Fusion and Perception Enhancement in Embodied Systems
Authors	Year	Innovation	Advantages	Limitations
Sun et al. [153]	2025	Multi-level LLM framework for coal mine sensor analysis	Efficient risk prediction and learning	Requires simulated and expert knowledge
Abhay et al. [1]	2024	EnvGen: LLM-generated adaptive RL training environments	Boosts training efficiency	Depends on feedback quality
Zheng et al. [118]	2023	Steve-Eye combines LLM and visual encoders	Enables end-to-end multimodal interaction	Requires large datasets
Ma et al. [169]	2025	Perspective on LLMs in autonomous driving	Offers future research outlook	Lacks technical solutions
Liu et al. [48]	2024	EAI-SIM simulation platform with ROS	Controls UAVs and arms in photo-realistic sim	Needs powerful simulation hardware
Song et al. [111]	2024	Guide-LLM aids visually impaired via text maps	Combines commonsense and path planning	Small sample evaluation
Micol et al. [93]	2025	VITA for adaptive mental well-being coaching	Personalized multi-modal robot coaching	Limited experimental scale
Kolby et al. [79]	2023	DECKARD agent with Dream/Wake exploration	Greatly improves RL efficiency	Relies on LLM hypothesized subgoals
Yang et al. [157]	2024	EMMA distilled from text-world LLM to visual world	Successfully transfers to vision tasks	Needs dual-modality synchronization

Fig. 10: Different categories of Embodied Robots: (a) Quadruped Robots, (b) Fixed-base Robots, (c) Tracked Robots, (d) Wheeled Robots, and (e) Humanoid Robots.

TABLE X: Multi-Agent Collaboration and Organizational Strategies
Authors	Year	Innovation	Advantages	Limitations
Zhang et al. [145]	2024	ReAd mechanism improves multi-agent LLM efficiency	Fewer LLM queries and agent steps	Evaluation mainly in simulation
Liu et al. [65]	2024	FaGeL uses smart fabric for implicit feedback	Learns user preference non-intrusively	High system complexity
Guo et al. [142]	2024	Organizational prompting reduces LLM agent conflict	Boosts teamwork and efficiency	Validation in real-world pending

Fig. 11: Example diagram of the integration of embodied intelligence and large language model.

TABLE XI: Security and Robustness Challenges in Embodied LLMs
Authors	Year	Innovation	Advantages	Limitations
Liu et al. [5]	2025	Contextual backdoor attacks in LLM-based agents	Dual-modal triggers for covert actions	No defense strategy yet in real use
Zhang et al. [52]	2024	Investigated LLM-based robot jailbreak threats	Systematic exposure of jailbreak scenarios	No concrete safety mechanisms
Li et al. [92]	2024	Universal benchmarking interface for LLM agents	Fine-grained error taxonomy	No coverage of real multimodal scenes
Kovalev et al. [7]	2022	Reviewed LLM-based instruction planners without training	Works in zero-shot scenarios	Lack of real task validations
Szot et al. [12]	2023	LLaRP: RL-trained LLM policy with vision	Superior multi-task performance	Needs intensive training and environment
Leon et al. [84]	2024	Studied virtual agent personality effects in VR	Extrovert agents yield better experience	No embodiment in physical robots
Liu et al. [115]	2024	EIRAD dataset and BLIP2 for adversarial attack test	Reveals attack success in LLM agents	Evaluation mainly on textual input

Fig. 12: Embodied Intelligence, Large Language Model, Knowledge Base and Reasoning Ability together constitute the core architecture of the intelligent system.

TABLE XII: Summary of Studies on LLM-driven Embodied Control
Authors	Year	Innovation	Advantages	Limitations
Ruaridh et al. [107]	2025	Proposed ELLMER framework combining GPT-4 with RAG for task planning in unpredictable environments.	Enables long-horizon task execution via visual/force feedback; adaptable planning.	Generalization scope not fully validated.
Fan et al. [56]	2024	Introduced LLM agents in manufacturing for autonomous design, planning, and control.	Achieved 81.88% task completion with GPT-4 in complex scenarios.	Challenges remain in 3D spatial task handling and real-time integration.
Liang et al. [62]	2023	Introduced “Code as Policies” using LLMs to write robot policy code from commands.	Enables reactive and trajectory-based control via few-shot prompting.	Limited physical-world validation.
Xu et al. [71]	2025	Reviewed LLM-integrated CPSS frameworks for human-centered Industry 5.0.	Connects LLMs, perception, scheduling, and swarm intelligence into unified framework.	Lacks empirical task-specific evaluations.
Zhao et al. [134]	2024	Developed MultiBotGPT system using GPT-3.5 for UAV/UGV task execution.	Outperforms BERT in assignment success rate; improves operator experience.	Focused on limited command types and single-modal input.
Shen et al. [124]	2024	Discussed LLMs as foundation for embodied agents in physical environments.	Conceptual bridge from virtual to real-world through embodiment.	Lacks concrete framework or task-based implementation details.
Jiang et al. [78]	2024	Proposed KoMA: LLM-driven multi-agent system for autonomous driving.	Achieves robust decision-making via planning, shared memory, and reflection.	Complexity of real-world deployment and scalability remains a challenge.

Why it matters

For anyone trying to move beyond chatbots toward real robots, self-driving systems, or home service robots, this survey maps out which technical pieces already exist and which gaps still block progress. It doesn't introduce a new model or codebase, but it offers a reference point for prioritizing research directions in embodied AI.

Terms in this paper

Embodied Intelligence · AI systems, like robots, that have a physical form and can perceive and act in real environments
Knowledge Base · A structured store of facts or concepts that an AI system can query and use
Chain-of-Thought / Tree-of-Thought · Reasoning methods where a model generates intermediate steps sequentially (chain) or explores multiple reasoning paths in parallel (tree)
ReAct framework · A method letting language models alternate between reasoning and taking actions, such as using external tools
Closed-loop knowledge integration · A system design where an AI updates its stored knowledge in real time based on new information or feedback

Original abstract (English)

The convergence of large language models (LLMs), structured knowledge bases (KBs), and reasoning ability (RA) presents a promising trajectory toward general embodied intelligence (GEI). This paper reviews the evolution of LLM-centered intelligent systems, emphasising their integration with knowledge representation, logical reasoning, and physical embodiment. We analyse LLM architectures, pre-training methods, and inference mechanisms, along with their interaction with external knowledge sources and structured reasoning frameworks. Furthermore, we examine embodied intelligence (EI) paradigms wherein agents learn and act in physical environments. To synthesise these dimensions, we present a conceptual framework that illustrates the synergy among LLMs, KBs, RA, and embodiment, serving as a guiding model for perception, reasoning, and action rather than an implemented engineering architecture. To advance toward GEI, we identify five key challenges: efficient LLM deployment, closed-loop knowledge integration, hybrid symbolic-neural reasoning, perception-action grounding, and continual learning. This survey provides a comprehensive roadmap for developing adaptive, multimodal agents capable of operating in complex, dynamic settings.

Authors · Fujiang Yuan, Xia Huang, Lusheng Wang, Jun Ding, Zhen Tian, Yuxin Wang, Shaojie Gu, Yuki Funabora, Yanhong Peng, Zebing

Read on arXiv

Latest papers

All papers →

Latest from METAL LAB

Figures: Fujiang Yuan et al., arXiv:2608.19794, arxiv-nonexclusive