SNU Researchers Develop AI Technology Enabling Robots to Organize Tables Independently
- Published in IEEE Robotics and Automation Letters, a Leading Journal in Robotics
- Introduces an AI Algorithm That Identifies and Tidies Objects Using a Single RGB-D Camera
- Demonstrates High Performance in Simulation and Real-World Experiments, Paving the Way for Commercial Robot Applications


▲ (From left) Professor Songhwai Oh (Department of Electrical and Computer Engineering, Seoul National University), Professor Hyemin Ahn (Department of Electrical and Computer Engineering, POSTECH), Researchers Hogun Kee, Wooseok Oh, and Minjae Kang (Department of Electrical and Computer Engineering, Seoul National University)

Seoul National University College of Engineering announced that a research team led by Professor Songhwai Oh from the Department of Electrical and Computer Engineering has developed artificial intelligence (AI) technology enabling robots to independently identify objects on a table and efficiently organize them.

The team overcame limitations of previous research by proposing a novel table tidying AI technology. They introduced the ‘TSMCTS (Tidiness Score-Guided Monte Carlo Tree Search)’ algorithm, which uses only one RGB-D camera to distinguish objects and automatically tidies them by finding the optimal robot action sequence based on a Tidiness Score.

The research was published on August 11 in IEEE Robotics and Automation Letters (RA-L) (Impact Factor: 5.3), a leading peer-reviewed journal in robotics and automation published by the IEEE Robotics and Automation Society. IEEE RA-L is an SCIE (SCI-Expanded) indexed journal and is recognized as one of the world's most authoritative academic journals in the field of robotics.

While AI commercialization is already prevalent in various aspects of our daily lives, AI-based table tidying technology has yet to be practically implemented in homes or offices. Progress in related research has been slow due to multiple constraints, a key reason being the lack of a standard benchmark to objectively evaluate tidying performance. The absence of shared evaluation criteria across existing studies has made objective comparisons between methodologies difficult, consequently hindering the advancement of this technology research.

Furthermore, the previously well-studied ‘target image’-based tidying technology focused more on arranging objects to match a predefined target image rather than on how a robot should place objects according to the specific space and situation. This proved insufficient for teaching robots the ability to ‘clean up neatly on their own’ anytime, anywhere. For service robots to become commonplace in real home and office environments, the ability to autonomously organize objects without separate user commands or guidelines is essential. Therefore, research to overcome the limitations of existing technology was urgently needed.

Since each person has different ways of organizing items and varying standards for what constitutes tidiness, tidying is a subjective concept that is difficult for AI to learn. However, this research began with Professor Oh Sung-hoe's team's idea: if AI could learn the concept of ‘neatness,’ developing AI technology that tidies up autonomously would be possible.

The research team first decided to train a model called the ‘Tidiness Discriminator’ to enable the robot to understand ‘neatness’. This model scores the level of organization in a scene after viewing only images. To achieve this, they created the ‘Tabletop Tidying Up Dataset (TTU Dataset)’—a collection of 224,225 scene images—using four environments (cafe tables, office desks, dining tables, and bathroom spaces) and 170 types of objects. Using this dataset, the robot trained with the score discriminator can now numerically evaluate a desk's tidiness without needing a target image.

Next, the research team developed an ‘MCTS-based Planner (Monte Carlo Tree Search Planner)’ by combining Offline Reinforcement Learning* and Monte Carlo Tree Search (MCTS)**. This planner explores and executes various tidying strategies. Building on this, they successfully completed the ‘TSMCTS (Tidiness Score-Guided Monte Carlo Tree Search)’ algorithm, enabling the robot to autonomously devise and execute efficient tidying plans.
* Offline Reinforcement Learning: A reinforcement learning method where the robot learns solely from stored robot data without direct experience.
** MCTS (Monte Carlo Tree Search): A search technique used in chess and Go AI, simulating possible moves to select the best outcome


▲ Figure 1. Examples of table tidying performed by robots using the TSMCTS method in various environments

The research team implemented a robot equipped with the TSMCTS algorithm in a simulation environment identical to the real world and conducted tidying experiments. Across 750 scenarios in five environments, the robot achieved an average success rate of 88.5% and an average Tidiness Score* of 0.901. Furthermore, in real-world robot experiments conducted across four environments—cafe tables, office desks, dining tables, and bathroom spaces—the robot recorded an average success rate of 85% and an average Tidiness Score of 0.897 across 20 scenarios, demonstrating excellent performance at a level suitable for commercialization. 
* Tidiness Score: A metric evaluating how neatly a scene is organized on a scale from 0 to 1. A score closer to 1 indicates a better-organized state.

In a blind test involving 17 participants, the performance of various algorithms was compared and evaluated. The results showed that TSMCTS demonstrated the most human-like organization capabilities. This test involved showing participants the results of tidying by a robot equipped with the TSMCTS algorithm and by several comparison algorithms. Participants were then asked to tidy the desk further themselves until satisfied. The experiment measured how many items were moved in each result. The results showed that the TSMCTS algorithm achieved the shortest total movement distance (57cm) and the fewest total manipulations (103 times), indicating participants moved the fewest items. It also recorded the fewest number of manipulations (103), indicating participants moved the least.

The technology developed in this study can be immediately applied to household cleaning robots and hotel room service robots, thanks to its strength in enabling robots to perform organization tasks autonomously without human instruction. Furthermore, when applied to automation in logistics and manufacturing sites, it is expected to reduce work preparation time and minimize quality variation on kitting and packing lines that require neatly arranging parts and products of various sizes.

Furthermore, the large-scale tidying dataset released by the research team through this study holds significant potential to serve as a standard benchmark for research in object rearrangement, control, and multimodal perception. Consequently, it is expected not only to form the foundation for future follow-up research but also to contribute to the expansion of the data and algorithm ecosystem.

Professor Songhwai Oh, who led the research, stated, “AI technology enabling robots to organize and tidy up autonomously can be applied across diverse fields, including service and household robots, cafe and restaurant automation, and logistics and production lines.” He added, “We plan to advance this into organizing technology that understands object functions and context by integrating it with large language models (LLMs) in the future.”

First author Hogun Kee is pursuing a combined master's-Ph.D. program in the Department of Electrical and Computer Engineering at Seoul National University, continuing research on robot foundation model learning and humanoid control. After graduation, he plans to work as an R&D engineer in AI and robotics at domestic or international research institutes or companies.

This research was supported by the Ministry of Science and ICT and the Institute of Information & Communications Technology Planning & Evaluation (IITP) through two projects: the “[SW Star Lab] Robot Learning: Efficient, Safe, and Socially-Acceptable Machine Learning”, and the “Development of Complex Task Planning Technologies for Autonomous Agents” project, which was conducted as part of the “Development of Complex Intelligent SW Technologies for Autonomous Agents.”


[Reference Materials]
- Paper/Journal : “Tidiness Score-Guided Monte Carlo Tree Search for Visual Tabletop Rearrangement”, IEEE Robotics and Automation Letters(RA-L) 2025
- Paper Link : https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=11122657
- Research Video : https://www.youtube.com/watch?v=uVFTlOq-sxga

[Contact Infornation]
Professor Songhwai Oh, Robot Learning Laboratory, Automation and Systems Research Institute, Department of Electrical and Computer Engineering, Seoul National University / +82-2-880-1511 / songhwai@snu.ac.kr