Deep Reinforcement Learning for Crowdshipping Last-Mile Delivery with Endogenous Uncertainty

Marco Silva () and João Pedro Pedroso
Additional contact information
Marco Silva: Industrial Engineering and Management, Institute for Systems and Computer Engineering, Technology and Science, 4200-465 Porto, Portugal
João Pedro Pedroso: Industrial Engineering and Management, Institute for Systems and Computer Engineering, Technology and Science, 4200-465 Porto, Portugal

Mathematics, 2022, vol. 10, issue 20, 1-23

Abstract: In this work, we study a flexible compensation scheme for last-mile delivery where a company outsources part of the activity of delivering products to its customers to occasional drivers (ODs), under a scheme named crowdshipping. All deliveries are completed at the minimum total cost incurred with their vehicles and drivers plus the compensation paid to the ODs. The company decides on the best compensation scheme to offer to the ODs at the planning stage. We model our problem based on a stochastic and dynamic environment where delivery orders and ODs volunteering to make deliveries present themselves randomly within fixed time windows. The uncertainty is endogenous in the sense that the compensation paid to ODs influences their availability. We develop a deep reinforcement learning (DRL) algorithm that can deal with large instances while focusing on the quality of the solution: we combine the combinatorial structure of the action space with the neural network of the approximated value function, involving techniques from machine learning and integer optimization. The results show the effectiveness of the DRL approach by examining out-of-sample performance and that it is suitable to process large samples of uncertain data, which induces better solutions.

Keywords: last-mile delivery; crowd shipping; deep reinforcement learning; data-driven optimization; endogenous uncertainty (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/20/3902/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/20/3902/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:20:p:3902-:d:948477

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().