A reinforcement learning approach for transaction scheduling in a shuttle-based storage and retrieval system

Abstract

With recent Industry 4.0 developments companies tend to automate their industries. Warehousing companies also take part in this trend. A shuttle-based storage and retrieval system (SBS/RS) is an automated storage and retrieval system technology experiencing recent drastic market growth. This technology is mostly utilized in large distribution centers processing mini-loads. With the recent increase in e-commerce practices fast delivery requirements with low volume orders have increased. SBS/RS provides ultrahigh-speed load handling due to having an excess amount of shuttles in the system. However not only the physical design of an automated warehousing technology but also the design of operational system policies would help with fast handling targets. In this work in an effort to increase the performance of an SBS/RS we apply a machine learning (ML) (i.e. Q-learning) approach on a newly proposed tier-to-tier SBS/RS design redesigned from a traditional tier-captive SBS/RS. The novelty of this paper is twofold: First we propose a novel SBS/RS design where shuttles can travel between tiers in the system, second due to the complexity of operation of shuttles in that newly proposed design we implement an ML-based algorithm for transaction selection in that system. The ML-based solution is compared with traditional scheduling approaches: first-in-first-out and shortest process time (i.e. travel) scheduling rules. The results indicate that in most cases the Q-learning approach performs better than the two static scheduling approaches. © 2023 Elsevier B.V. All rights reserved.