Design and Development of a Hybrid Wheel-Climbing Warehouse Robot Based on Reinforcement Learning
DOI:
https://doi.org/10.65359/dhrqkk21Keywords:
Warehouse robot, hybrid wheel-climbing, reinforcement learning, Q-Learning, robot navigationAbstract
The development of modern warehouse systems requires robots capable of autonomous navigation and obstacle traversal in operational environments. However, conventional wheeled robots still have limitations in overcoming small obstacles that can reduce their mobility. This study aims to design and develop a warehouse hybrid wheel-climbing robot based on Reinforcement Learning. The research employed the ADDIE (Analysis, Design, Development, Implementation, and Evaluation) method. The robot was developed using an Arduino Mega 2560, an HC-SR04 ultrasonic sensor, DC motors, and servo motors as the climbing mechanism. The experimental results showed a climbing success rate of 80%–90%, an average travel time of 18.80 s, an average speed of 13.87 cm/s, and an average climbing duration of 2.58 s. The implementation of Q-Learning produced a simple navigation policy capable of selecting forward and turning actions based on environmental conditions. The results indicate that the integration of the climbing mechanism and Q-Learning has the potential to improve the mobility of warehouse robots.
Downloads
References
Almazrouei, K., Kamel, I., & Rabie, T. (2023). Dynamic Obstacle Avoidance and Path Planning through Reinforcement Learning. Applied Sciences, 13(14), 8174. https://doi.org/10.3390/app13148174
Bogue, R. (2016). Growth in e-commerce boosts innovation in the warehouse robot market. Industrial Robot: An International Journal, 43(6), 583–587. https://doi.org/10.1108/IR-07-2016-0194
Brunke, L., Greeff, M., Hall, A. W., Yuan, Z., Zhou, S., Panerati, J., & Schoellig, A. P. (2022). Safe Learning in Robotics: From Learning-Based Control to Safe Reinforcement Learning. Annual Review of Control, Robotics, and Autonomous Systems, 5(1), 411–444. https://doi.org/10.1146/annurev-control-042920-020211
Cao, R., Gu, J., Yu, C., & Rosendo, A. (2022). OmniWheg: An Omnidirectional Wheel-Leg Transformable Robot. http://arxiv.org/abs/2203.02118
Cao, Y., Ni, K., Kawaguchi, T., & Hashimoto, S. (2024). Path Following for Autonomous Mobile Robots with Deep Reinforcement Learning. Sensors, 24(2), 561. https://doi.org/10.3390/s24020561
Choi, J., Lee, G., & Lee, C. (2021). Reinforcement learning-based dynamic obstacle avoidance and integration of path planning. Intelligent Service Robotics, 14(5), 663–677. https://doi.org/10.1007/s11370-021-00387-2
Fragapane, G., de Koster, R., Sgarbossa, F., & Strandhagen, J. O. (2021). Planning and control of autonomous mobile robots for intralogistics: Literature review and research agenda. European Journal of Operational Research, 294(2), 405–426. https://doi.org/10.1016/j.ejor.2021.01.019
García, J. M., & Duarte, F. G. (2024). Mobile rolling robots designed to overcome obstacles: A review. Forces in Mechanics, 16, 100283. https://doi.org/10.1016/j.finmec.2024.100283
Gu, S., Kshirsagar, A., Du, Y., Chen, G., Peters, J., & Knoll, A. (2023). A human-centered safe robot reinforcement learning framework with interactive behaviors. Frontiers in Neurorobotics, 17. https://doi.org/10.3389/fnbot.2023.1280341
Ha, V. T., & Vinh, V. Q. (2024). Experimental Research on Avoidance Obstacle Control for Mobile Robots Using Q-Learning (QL) and Deep Q-Learning (DQL) Algorithms in Dynamic Environments. Actuators, 13(1), 26. https://doi.org/10.3390/act13010026
Hanh, L. D., & Cong, V. D. (2023). Path Following and Avoiding Obstacle for Mobile Robot Under Dynamic Environments Using Reinforcement Learning. Journal of Robotics and Control (JRC), 4(2), 157–164. https://doi.org/10.18196/jrc.v4i2.17368
Ibarz, J., Tan, J., Finn, C., Kalakrishnan, M., Pastor, P., & Levine, S. (2021). How to train your robot with deep reinforcement learning: lessons we have learned. The International Journal of Robotics Research, 40(4–5), 698–721. https://doi.org/10.1177/0278364920987859
Keith, R., & La, H. M. (2024). Review of Autonomous Mobile Robots for the Warehouse Environment. http://arxiv.org/abs/2406.08333
Kormushev, P., Calinon, S., & Caldwell, D. G. (2013). Reinforcement learning in robotics: Applications and real-world challenges. Robotics, 2(3), 122–148. https://doi.org/10.3390/robotics2030122
Lackner, T., Hermann, J., Kuhn, C., & Palm, D. (2024). Review of autonomous mobile robots in intralogistics: state-of-the-art, limitations and research gaps. Procedia CIRP, 130, 930–935. https://doi.org/10.1016/j.procir.2024.10.187
Le, H., Saeedvand, S., & Hsu, C.-C. (2024). A Comprehensive Review of Mobile Robot Navigation Using Deep Reinforcement Learning Algorithms in Crowded Environments. Journal of Intelligent & Robotic Systems, 110(4), 158. https://doi.org/10.1007/s10846-024-02198-w
Li, J., Ma, J., & Nguyen, Q. (2022). Balancing Control and Pose Optimization for Wheel-legged Robots Navigating High Obstacles. http://arxiv.org/abs/2109.09934
Li, Y., Chen, K., Collignon, S., & Ivanov, D. (2021). Ripple effect in the supply chain network: Forward and backward disruption propagation, network health and firm vulnerability. European Journal of Operational Research, 291(3), 1117–1131. https://doi.org/10.1016/j.ejor.2020.09.053
Mohamed, S., Vellaiyan, V., Kim, K., Kim, Y., & Shin, B. (2025). Development of a Four Omni-Wheeled Mobile Robot Using Telescopic Legs. Machines, 13(4), 292. https://doi.org/10.3390/machines13040292
Niloy, Md. A. K., Shama, A., Chakrabortty, R. K., Ryan, M. J., Badal, F. R., Tasneem, Z., Ahamed, M. H., Moyeen, S. I., Das, S. K., Ali, M. F., Islam, M. R., & Saha, D. K. (2021). Critical Design and Control Issues of Indoor Autonomous Mobile Robots: A Review. IEEE Access, 9, 35338–35370. https://doi.org/10.1109/ACCESS.2021.3062557
Tang, C., Abbatematteo, B., Hu, J., Chandra, R., Martín-Martín, R., & Stone, P. (2025). Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes. Annual Review of Control, Robotics, and Autonomous Systems, 8(1), 153–188. https://doi.org/10.1146/annurev-control-030323-022510
Wang, C., Yang, X., & Li, H. (2022). Improved Q-Learning Applied to Dynamic Obstacle Avoidance and Path Planning. IEEE Access, 10, 92879–92888. https://doi.org/10.1109/ACCESS.2022.3203072
Wang, X., Sun, Y., Xie, Y., Bin, J., & Xiao, J. (2023). Deep reinforcement learning-aided autonomous navigation with landmark generators. Frontiers in Neurorobotics, 17. https://doi.org/10.3389/fnbot.2023.1200214
Xiao, X., Liu, B., Warnell, G., & Stone, P. (2022a). Motion planning and control for mobile robot navigation using machine learning: a survey. Autonomous Robots, 46(5), 569–597. https://doi.org/10.1007/s10514-022-10039-8
Xiao, X., Liu, B., Warnell, G., & Stone, P. (2022b). Motion planning and control for mobile robot navigation using machine learning: a survey. Autonomous Robots, 46(5), 569–597. https://doi.org/10.1007/s10514-022-10039-8
Zhao, J., Han, T., Wang, S., Liu, C., Fang, J., & Liu, S. (2021). Design and Research of All-Terrain Wheel-Legged Robot. Sensors, 21(16), 5367. https://doi.org/10.3390/s21165367
Zhen, L., Tan, Z., de Koster, R., He, X., Wang, S., & Wang, H. (2025). Optimizing Warehouse Operations with Autonomous Mobile Robots. Transportation Science, 59(5), 1130–1152. https://doi.org/10.1287/trsc.2024.0800
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Deni Wisnu, Trinovita Zuhara Jingga, Perdana Putera (Author)
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Copyright & Licensing Policy
Copyright
The copyright of all articles published in JURNAL SINTIKA (Journal of Information Systems, Informatics Engineering, and Computer Systems) remains with the respective author(s). Authors retain full copyright ownership of their work while granting JURNAL SINTIKA the right of first publication.
Upon publication, authors agree that their articles will be distributed under the Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.
Under this license, users are free to:
- Share – Copy and redistribute the material in any medium or format.
- Adapt – Remix, transform, and build upon the material for any purpose, including commercial use.
These permissions are granted provided that the following conditions are met:
- Attribution (BY) – Appropriate credit must be given to the original author(s), a link to the license must be provided, and any changes made to the work must be clearly indicated.
- ShareAlike (SA) – If the material is modified, adapted, or remixed, the resulting work must be distributed under the same CC BY-SA 4.0 license.
Licensing Policy
By submitting a manuscript to JURNAL SINTIKA, authors agree that, if accepted for publication, their article will be published under the Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) License.
Authors retain the right to:
- Archive and distribute the published version of their article through institutional repositories, personal websites, academic social networks, or other non-commercial and commercial platforms, provided that the original publication in JURNAL SINTIKA is properly acknowledged.
- Reuse their work in future books, dissertations, conference proceedings, or other scholarly publications, with appropriate citation to the original publication.
JURNAL SINTIKA supports the principles of Open Access and encourages authors to deposit their manuscripts in institutional or subject repositories before submission (preprint), during the review process (where permitted), and after publication, in order to enhance the visibility, accessibility, dissemination, and academic impact of their research.
For complete license information, please visit the Creative Commons Attribution-ShareAlike 4.0 International (CC BY-SA 4.0) website:
https://creativecommons.org/licenses/by-sa/4.0/
