Deep reinforcement learning of autonomous control actions to improve bus-service regularity

Abstract

Bus Bunching is caused by irregularities in demand across the bus route, together with other factors such as traffic. The effect of this problem is that buses operating on the same route start to catch up with each other, severely impacting the regularity and the quality of the service. Control actions such as Bus Holding and Stop Skipping can be used to regulate the service and adjust the headway between two buses. Traditionally, this phenomenon is mitigated either reactively online through simple rule-based control, or preemptively through analytical scheduling solutions, such as mathematical optimization. Over time, both approaches degrade to an irregular service. In this work, we investigate the use of Deep Reinforcement Learning algorithms to train a policy that determines which actions should take place at specific control points to regularise the bus service. While prior studies are typically restricted to one control action, we consider both Bus Holding and Stop Skipping. We replicate benchmarks found in the latest literature, and also introduce traffic to increase the realism of the simulation. Furthermore, we also consider scenarios where the service is already unstable and buses are already bunched together, a first of this kind of study. We compare the performance of the RL-based policies with a no-control policy and a rule-based policy. The learnt policies not only keep a significantly lower headway variance and mean waiting time, but also recover from unstable scenarios and restore service regularity.