Smart transaction picking in tier-to-tier SBS/RS by deep Q-learning

Abstract

By the rapid growth of e-commerce the intralogistics sector is facing new challenges. Intralogistics sector requires more flexible scalable processes with maximum reliability and availability. They are complicated and interconnected systems whose all components are required to be perfectly coordinated with each other for optimal functionality. In this work we study an intralogistics technology shuttle-based storage and retrieval system (SBS/RS) where shuttles are tier-to-tier. In this novel system design in an effort to increase shuttle utilization as well as decrease initial investment cost shuttles are designed in a more flexible travel manner so that they can change their tiers within an aisle by using a separate lifting mechanism. Due to the complexity of such system design as well as aiming to obtain fast transaction process time by the decreased number of shuttles in the system we implement a Deep Q-Learning (DQL) approach to let shuttles select the best transaction to process based on its targets. We compare the performance of the DQL by the average cycle time per transaction performance metric with the other well-known selection rules First-in-First-Out (FIFO) and Shortest Process Time (SPT). Results show that Deep Q-Learning approach produces better results than those FIFO and SPT. © 2021 Elsevier B.V. All rights reserved.