Optimizing same-day delivery with vehicles and drones: A hierarchical deep reinforcement learning approach

Meng Li, Kaiquan Cai and Peng Zhao

Transportation Research Part E: Logistics and Transportation Review, 2025, vol. 193, issue C

Abstract: The advent of same-day delivery services has achieved immense popularity, driven by escalating customer expectations on fast shipping and the need for market competitiveness. To optimize such services, the use of heterogeneous fleets with vehicles and drones has proven effective in reducing the resource requirements needed for delivery. This paper focuses on investigating the same-day delivery dispatching and routing problem with a fleet of multiple vehicles and drones. In this problem, stochastic and dynamic requests, coupled with their stringent time constraints, require dispatchers to make real-time decisions about optimally assigning vehicles and drones, ensuring both efficiency and effectiveness in delivery operations while taking into account the routing. To tackle this complex problem, we model it with a route-based Markov decision process and develop a novel hierarchical decision approach based on deep reinforcement learning (HDDRL). The first level of the hierarchy is tasked with determining the departure times of vehicles, balancing the trade-offs between the delivery frequency and efficiency. The second level of the hierarchy is dedicated to determining the most suitable delivery mode for each request, whether by vehicles or drones. The third level is responsible for planning routes for vehicles and drones, thereby enhancing route efficiency. These three levels in the hierarchical framework collaborate to solve the problem in a synchronized manner, with the objective of maximizing the service requests within a day. Empirical results from computational experiments highlight the superiority of the HDDRL over benchmark methods, demonstrating not only its enhanced efficacy but also its robust generalization across diverse data distributions and fleet sizes. This underscores the HDDRL’s potential as a powerful tool for enhancing operational efficiency in same-day delivery services.

Keywords: Same-day delivery; Heterogeneous fleets; Drone delivery; Route-based Markov decision process; Hierarchical framework; Deep reinforcement learning (search for similar items in EconPapers)
Date: 2025
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S1366554524004691
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:transe:v:193:y:2025:i:c:s1366554524004691

Ordering information: This journal article can be ordered from
http://www.elsevier.com/wps/find/journaldescription.cws_home/600244/bibliographic
http://www.elsevier. ... 600244/bibliographic

DOI: 10.1016/j.tre.2024.103878

Access Statistics for this article

Transportation Research Part E: Logistics and Transportation Review is currently edited by W. Talley

More articles in Transportation Research Part E: Logistics and Transportation Review from Elsevier
Bibliographic data for series maintained by Catherine Liu ().