Adaptive Q-learning-supported Resource Allocation Model in Vehicular Fogs

Abstract

Urban computing has become a significant driver in supporting the delivery and sharing of services, being a strong ally to intelligent transportation. Smart vehicles present computing and communication capabilities that allow them to enable many autonomous vehicular safety and infotainment applications. Vehicular Cloud Computing (VCC) has already proven to be a technology shifting paradigm harnessing the computation resources from on board units from vehicles to form clustered computing units to solve real world computing problems. However, with the rise of vehicular application use and intermittent network conditions, VCC exhibits many drawbacks. Vehicular Fog computing appears as a new paradigm in enabling and facilitating efficient service and resource sharing in urban environments. Several vehicular resource management works have attempted to deal with the highly dynamic vehicular environment following diverse approaches, e.g. MDP, SMDP, and policy-based greedy techniques. However, the high vehicular mobility causes several challenges compromising consistency, efficiency, and quality of service. RL-enabled adaptive vehicular Fogs can deal with the mobility for properly distributing load and resources over Fogs. Thus, we propose a mobility-based cloudlet dwell time estimation method for accurately estimating vehicular resources in a Fog. Leveraging the CDT estimation model, we devise an adaptive and highly dynamic resource allocation model using mathematical formula for Fog selection, and reinforcement learning for iterative review and feedback mechanism for generating optimal resource allocation policy.