Wugedele Bao,Celimuge Wu,Siri Guleng,Jiefang Zhang,Kok-Lim Alvin Yau,Yusheng Ji

1 School of computer science and information engineering,Hohhot Minzu College,Hohhot 010051,China

2 Graduate School of Informatics and Engineering,The University of Electro-Communications,1-5-1,Chofugaoka,Chofu-shi,Tokyo,182-8585 Japan

3 Institute of Intelligent Media Technology,Communication University of Zhejiang,Zhejiang 310018,China

4 School of Science and Technology,Sunway University,5,Jalan Universiti,Bandar Sunway,47500 Petaling Jaya,Selangor,Malaysia

5 Information Systems Architecture Research Division,National Institute of Informatics,2-1-2,Hitotsubashi,Chiyoda-ku,Tokyo 101-8430 Japan

Abstract:In order to support advanced vehicular Internet-of-Things(IoT)applications,information exchanges among different vehicles are required to find efficient solutions for catering to different application requirements in complex and dynamic vehicular environments.Federated learning(FL),which is a type of distributed learning technology,has been attracting great interest in recent years as it performs knowledge exchange among different network entities without a violation of user privacy.However,client selection and networking scheme for enabling FL in dynamic vehicular environments,which determines the communication delay between FL clients and the central server that aggregates the models received from the clients,is still under-explored.In this paper,we propose an edge computing-based joint client selection and networking scheme for vehicular IoT.The proposed scheme assigns some vehicles as edge vehicles by employing a distributed approach,and uses the edge vehicles as FL clients to conduct the training of local models,which learns optimal behaviors based on the interaction with environments.The clients also work as forwarder nodes in information sharing among network entities.The client selection takes into account the vehicle velocity,vehicle distribution,and the wireless link connectivity between vehicles using a fuzzy logic algorithm,resulting in an efficient learning and networking architecture.We use computer simulations to evaluate the proposed scheme in terms of the communication overhead and the information covered in learning.

Keywords:vehicular IoT;federated learning;client selection;networking scheme

I.INTRODUCTION

With the advances of vehicular Internet-of-Things(IoT)technologies,many new applications,such as collaborative autonomous driving,are expected in the near future to enable a higher level of intelligent transportation systems.Due to the mobility of vehicles,the variety of vehicle distribution,and the diversity of application requirements,decision making in vehicular environments is particularly challenging[1,2].It is important to design an intelligent vehicle agent that can make correct decisions under complex vehicular environments.Machine learning technologies,such as deep reinforcement learning,are attracting great interest in the control of vehicle agents owing to its capability of self-tuning based on interactions with environments.However,the low convergence speed of these self-learning approaches is a key concern.Since each vehicle only has limited knowledge about the environments,it takes a long time for visiting and learning all possible states or decision making factors,resulting in a long convergence time,which poses challenges in the use of these technologies.In order to solve this problem,one possible approach is to collect the sensor data of all the vehicles,and then conduct learning based on the complete data.However,this is impossible due to the privacy concerns[3]and limited networking resources.

Federated learning(FL)can solve the above mentioned problem by conducting distributed learning at multiple clients(or learning agents)without sharing raw sensor data among themselves.FL was used by Gboard[4]to train a global model for virtual keyboard by conducting distributed learning at multiple mobile phones without sharing raw user data among themselves.Recently,FL has also been attracting a great interest in vehicular IoT for its advantage of conducting efficient learning under complex environments[5].However,most existing studies only mention about the design of FL framework in vehicular environments,and do not adequately discuss the performance of learning in dynamic vehicular networks.In order to enable FL in vehicular environments,an efficient FL client selection algorithm should be designed.The client selection mechanism must address the following two issues.

First,selected client nodes should reduce spatial relevance among themselves,so vehicle distribution must be considered.This is because neighboring vehicles in close proximity could have similar sensing capability and sensor data,so it is unnecessary to select both vehicles due to limited communication resources.Second,selected clients should be able to communicate with the central server in an efficient way because their communications can affect the training quality significantly.In other words,the central server must select efficient client nodes that can download/upload global/local FL model efficiently.These two issues can be turned into two research questions:a)“how to select client vehicles to cover the network-wide operating environment?”;and b)“how to ensure selected client vehicles are able to upload local models efficiently?”

In this paper,we discuss the client selection problem in vehicular IoT environments,and propose an efficient client selection and networking scheme for FL based on edge computing[6].The proposed scheme selects FL clients based on a distributed edge computing approach.The proposed scheme uses a fuzzy logic-based approach to divide vehicles into multiple clusters and select a vehicle in each cluster as the cluster head vehicle(or edge vehicle)to serve as the edge node,which conducts the training of FL local models.Every member(vehicle)of each cluster can transmit its data to the nearest edge vehicle,which ensures the completeness of the sensor data for generating local models.By conducting the training of local models at edge nodes,the proposed scheme ensures an efficient transmission of local models without sacrificing the information used in the training.We summarize the contributions of this paper as follows:

? We propose a distributed edge computing-based FL client selection scheme that selects edge vehicles by considering the vehicle velocity,vehicle distribution,and the wireless connectivity between vehicles.By collecting sensor data at each edge vehicle from neighboring vehicles in the vicinity,the proposed scheme can achieve a high accuracy in the training of a local model while avoiding the transmission of raw sensor data to the central server.

? The proposed scheme makes a tradeoff between the accuracy of the global model and the communication overhead of FL in vehicular environments by conducting training tasks at each edge vehicle and forming efficient clusters to ensure an efficient transmission of sensor data from each cluster member to its corresponding edge node.

? The proposed scheme integrates the distributed edge computing-based technology with FL to support distributed learning in vehicular IoT,where edge computing is used to conduct both local training and aggregation of raw sensor data generated by vehicles in the vicinity.

? We launch realistic computer simulations to evaluate the performance of the proposed scheme by comparing it with existing baselines in terms of the information included in the local model,and the communication overhead for uploading the local model.

The rest of the paper is organized as follows.We first explain the related studies briefly in Section II.Section III describes the proposed scheme in detail.Simulation results and discussions are shown in Section IV.Finally,Section V concludes this paper and points out future work.We use“vehicle”and“node”interchangeably in this paper.

II.RELATED WORK

2.1 FL for Vehicular IoT

Existing studies on FL for vehicular IoT focus on the design of architectural framework and the application of FL in vehicular environments.In[5],recent research efforts on FL for vehicular IoT are discussed and open research issues are explained.Lu et al.[7]propose an FL scheme with blockchain to achieve secure data sharing in vehicular IoT.They discuss the importance of using FL to reduce communication overhead and protect the privacy as compared with the conventional centralized learning approach.In[8],Samarakoon et al.employ FL to solve the packet queue management issue in vehicular networking,specifically the tail distribution of queue length,in vehicular environments for the purpose of providing a communication with ultra-high reliability and extremely low latency to users.Ye et al.[9]propose to use FL with vehicular edge computing for the purpose of image classification.They discuss the problem of how to aggregate local models from different clients by considering the difference in image qualities.However,the communication overhead is not discussed.In[10],the applications of FL in unmanned aerial vehicles(UAVs)are discussed.Zhang and Hanzo[11]consider a scenario where multiple UAVs collaborate with each other to conduct an image classification task,and they propose a FL-based approach to reduce the communication overhead between an UAV and the control station.A FL-based privacy-aware resource sharing scheme for decentralized vehicular environments is proposed by Lu et al.[12].Chai et al.[13]propose a knowledge sharing framework based on blockchain and FL for the purpose of protecting the privacy of user data.A hierarchical approach is used to ensure the scalability of the framework.Yu et al.[14]discuss the use of FL to solve the data caching problem among connected vehicles.They use FL to predict the popularity of a content by considering the mobility of vehicles.Lu et al.[15]point out the privacy concerns in the model update process in vehicular cyber-physical systems,and introduce a random sub-gossip approach to address these concerns.Although there are some research efforts regarding the use of FL in vehicular IoT,many important issues,such as the client node selection,communication overhead,learning performance,and the design of a global model,are not discussed adequately.This paper is targeting some of the unsolved issues,namely the communication efficient client node selection and local model uploading in vehicular environments.

2.2 Communication Efficient Approaches for FL in IoT

In[16],a multichannel medium access control(MAC)scheme is proposed to support FL by defining different access probabilities of each client based on the importance of its generated local model.By providing more communication resources to a high-priority client,[16]aims to improve the learning performance of FL in a resource-limited communication scenario.However,due to its dependance on a special MAC layer protocol and the complexity of vehicular environments,[16]cannot provide a generic solution for vehicular environments.Yang et al.[17]propose a client selection approach that takes into account the wireless connection between the central server and clients.In[18],the noisy communication environment is considered in the design of the FL training process,which includes both the dissemination of the global model and the uploading of local models.Zhu et al.[19]propose a model aggregation approach that considers the characteristics of multi-access channels in wireless environments.In[20],Yang et al.discuss the scheduling of local model updates based on the performance of FL.Ahn et al.[21]discuss the problem of implementing FL in wireless environments.Sun et al.[22]propose an adaptive approach for uploading the local models.The approach employs the non-orthogonal multiple access(NOMA)to improve spectrum efficiency and reduce the communication latency;however,the interoperability of NOMA,which is a specific physical layer technology,with existing vehicular IoT systems is a concern.He et al.[23]discuss the joint client selection and communication resource allocation problem from a theoretical perspective.As mentioned above,there are some studies discussing about the design of communication efficient approach for FL.However,a comprehensive investigation for vehicular IoT scenarios is still absent.

2.3 Edge Computing for Vehicular IoT

The concept of edge computing is directly related with FL.There are many studies on the use of edge computing in vehicular IoT.While existing studies focus on reducing the system delay by executing computational tasks near the end users,this paper discusses how to use edge computing to improve the performance of client selection and local model uploading.In vehicular networks,some vehicles can be used as edge nodes to collect sensor data from neighboring vehicles and conduct analysis based on the collected data[24,25].In[24],an edge-based data forwarding scheme is discussed.In[25],edge vehicles are used to provide a gateway functionality in aggregating different types of wireless spectrums in multi-access vehicular edge computing environments.Most existing studies on edge computing focus on optimizing the computation offloading performance.Wang et al.[26]propose a game theoretical approach to improve the computation offloading performance in vehicular edge computing environments.With the similar purpose,Liu et al.[27]discuss a computation offloading approach by considering different metrics,including the distance between the task requester and task processor nodes,the quality constraint,the wireless resource,and the computing capability of vehicles.In[28],a Markov decision process-based offloading scheme involving different vehicles is proposed.In[29],an edge computing-based approach is used to facilitate the use of blockchain in vehicular environments.Some studies discuss data caching based on edge computing technologies.In[30],a deep reinforcement learningbased approach is proposed to optimize joint allocation of the storage and computing resources in vehicular networks.Ning et al.[31]propose a deep reinforcement learning-based resource allocation scheme for traffic control systems.By conducting computational tasks near the end users,edge computing is one of the important technologies for supporting FL in vehicular environments.Different from existing studies,this paper discusses how to use edge computing to improve the process of client selection,sensor data aggregation,and local model uploading for FL.

III.PROPOSED SCHEME

3.1 Design Overview

We consider a scenario where vehicles communicate with each other and with roadside units(RSUs)through a distributed communication approach.We do not assume any particular communication interface for vehicles as the proposed scheme can be applied with any communication standards,including IEEE 802.11p and cellular Vehicle-to-Everything(V2X)sidelink interface.The proposed scheme is based on the following assumptions.Each vehicle has the capability and functionality of edge computing server and is able to contribute as an edge vehicle if it is selected by the proposed scheme.An increase of data size can always achieve a more accurate FL model.Focusing on a networking scheme for FL,we do not consider the effect of not independent and identically distributed(non-IID)data on the FL performance[32].

We consider the problem of how to improve the FL efficiency in vehicular environments.The basic design principle is to make a tradeoff between the accuracy of the global model and communication overhead.If we collect all the data to the central server(cloud server or RSU),we can achieve the best possible accuracy.However,that is impractical if not impossible since collecting the data requires a huge bandwidth which is not feasible in dynamic vehicular environments.Therefore,a FL-based approach is required to achieve a low communication overhead in the exchange of knowledge among vehicles.We propose a client selection and networking scheme for FL in vehicular IoT.The importance of client selection lies in the fact that the use of all vehicles as FL clients is inefficient and unnecessary.The networking part is to facilitate collaboration among vehicles and collect the trained local models to the FL central server that aggregates the local models and updates the global model.On the other hand,there are some information losses in the global model when we use FL instead of a complete collection of raw sensor data.Therefore,we must consider this in the design of the FL architecture.In order to efficiently support FL in vehicular environments,we use an edge vehicle-based scheme where only some vehicles are used to conduct FL,and the edge vehicles collect raw sensor data from neighboring vehicles to achieve a high accuracy with reasonable overhead.

Figure 1 provides an overview of the proposed scheme.The basic working procedure is as follows.First,the edge vehicles are selected by considering the vehicle velocity,the vehicle distribution,and the wireless connectivity between vehicles.The vehicle velocity and the vehicle distribution are used to identify relatively stable edge vehicles in order to reduce the need to change edge vehicles as time goes by.The wireless connectivity is used to provide a reliable connection between an edge vehicle and ordinary vehicles(or member vehicles),which are non-edge vehicles,in the vicinity,guaranteeing the data transmissions between an ordinary vehicle node and the nearest edge vehicle.

Figure 1.The overview of the proposed scheme.

Then,RSU,which serves as the central server of FL,initializes the global model.After receiving the global model from the RSU,each edge vehicle collects sensor data from the ordinary vehicles in the vicinity,and then conducts FL based on the data collected from multiple vehicles.Here,each ordinary vehicle only needs to send its data to an edge vehicle that has the strongest connectivity in terms of wireless link quality with itself.By using the data from multiple vehicles,each edge node can train a better local model.After a predefined time of local model training,each edge vehicle(as the client)transmits its local model to the RSU.The RSU then aggregates the local models and updates the global model.The updated version of the global model is then disseminated to all the edge vehicles for further training and use.This procedure is repeated until the central server finds a global model that satisfies the requirement of the application.

The proposed scheme inherits the privacy protection capability of FL by avoiding the transmission of vehicle raw data to the cloud(RSU).The main purpose of collecting some vehicle information from neighbor vehicles to edge vehicles is to ensure the completeness of the sensor data for generating local models without sacrificing too much privacy.For example,the cooperative perception[5]is an approach to improve the perception accuracy with a consideration of the negative affect on user privacy.Note that the transmission of vehicle raw data to a neighbor edge vehicle does not incur a high risk of privacy leakage.This is because the edge vehicle and its neighbors usually sense similar information due to close geographical locations while the aggregation of these information at the edge vehicle could improve the learning accuracy significantly.The proposed scheme can efficiently solve the tradeoff between the accuracy and privacy by collecting some vehicle information at the edge node and conducting FL at the edge node.

3.2 Edge Node Selection

The proposed scheme considers the vehicle velocity,the vehicle distribution,and the wireless connectivity between vehicles by introducing the following three factors,namely the stability factor,the topology factor,and the connectivity factor.The edge nodes are selected based on a decentralized approach.Each vehicle calculates a competency value of being an edge node for each of its neighbors upon the reception of a hello message from the neighbor.The information required for calculating the competency value is exchanged with periodical hello messages sent once per second.After comparing its own value with that of its neighbors,each vehicle knows whether it is more suitable to serve as an edge node or not.The vehicle announces itself as an edge node by sending a message to notify its neighbors if it is more suitable.

Algorithm 1.Edge node selection algorithm at each vehicle.Initialize the candidate list for edge nodes as a null list.if Received a packet from a neighbor vehicle AND the distance to the neighbor is equal to or lower than 14R then Calculate the stability factor,topology factor,and connectivity factor of the neighbor.Calculate the edge competency level of the neighbor by using a fuzzy logic-based approach based on the stability factor,topology factor,and connectivity factor.Add the neighbor to the candidate list if the list does not contain the neighbor yet.end if Calculate the edge competency level for itself.Compare the edge competency levels of all neighbors included in the candidate list,and find the neighbor that has the maximal value.if The current node has a higher edge competency level than the neighbor then Announce itself as an edge node using the next hello message.end if

As shown in Algorithm 1,the edge node selection algorithm also considers the connectivity between two neighboring edge vehicles.The optimal distance between two neighboring edge nodes is dependent on the wireless spectrum used for communications.Here,we use,whereRis the average communication range,as the target distance in order to ensure that the connectivity is strong enough to support a high level modulation and coding scheme that determines the achievable throughput for data transmissions between two neighboring edge nodes.We leave the value of target distance as a system hyper parameter,and this can be changed according to the wireless transceivers used by the vehicles.Each vehicle takes action following Algorithm 1.First,all the neighbors located within a distance ofare considered as candidates for edge nodes including the current vehicle.Therefore,the distance between two neighboring edge vehicles must be smaller thanR.Assuming the vehicles are basically distributed uniformly,we can ensure that there is at least a single edge vehicle within aregion,resulting in a stable connection between any two neighboring vehicles.Then,the current vehicle employs a fuzzy logic-based approach to calculate an edge competency level for each candidate.The vehicle with the largest edge competency level declares itself as an edge vehicle.In this way,all the edge vehicles are determined by exchanging hello messages sent only to one-hop neighbors.

Each edge vehicle and its neighboring vehicles utilizing the edge vehicle constitute a vehicle cluster,where the edge vehicle is the cluster head and others are cluster members.Since each ordinary vehicle only sends its own data to the nearest edge vehicle,there is no overlap of cluster members between different vehicle clusters.This ensures an efficient network topology for data transmissions in vehicular networks while providing a low overhead for edge node selection and cluster maintenance.

3.3 Fuzzy Logic-based Edge Competency Level Calculation

The calculation procedure of the edge competency level for each candidate is as follows.

? Calculate the three factors indicating the vehicle velocity,the vehicle distribution,and the wireless connectivity,based on predefined equations.

? Convert the three factors to the corresponding fuzzy values.

? Use predefined fuzzy rules to process the fuzzy values to calculate the competency level of the candidate in a fuzzy format.

? Use a predefined output membership function to change the representation of the competency level from a fuzzy format to a numerical format.

3.3.1 Stability factor

For a nodex,the stability factor is calculated as follows.

whereυ(·)denotes vehicle velocity andNxis a set that includes nodexand all its one-hop neighbors.avgandmaxdenote the average value and the maximum value,respectively.SF indicates the stability level,where a higher value shows a more stable level.Each node attach some information to beacon messages which are sent periodically.The information includes its velocity value|υ(x)|,the average velocity of vehicles avgy∈Nx|υ(y)|in one-hop region,and the maximum velocity of vehicles maxy∈Nx|υ(y)|in onehop region.SFis updated periodically(with interval of 1 second by default)upon reception of a hello message by using a weighted exponential moving average approach with a smooth factorα(0.7 by default)as shown in Eq.(2).Basically,a higher value ofαgives a higher weight for a more recent value.While the best value forαis dependent on the vehicle mobility and road topology,according to our experimental results,the value of 0.7 always shows a good result.That is why we setαto 0.7.

3.3.2 Topology factor

Topology factorTFindicates the distribution of vehicles as follows:

wherec(x)denotes the number of vehicles moving in the same direction as nodexin the one-hop region of nodex.A vehicle with a higherc(x)value is more likely to be elected as an edge node because the vehicle can generate more stable edge-based networking architecture due to the relatively low mobility between an edge node and neighboring vehicles.Each vehicle sends the number of neighboring vehicles andc(·)to its neighbors by attaching this information to the beacon messages.Similar toSF,TFis updated periodically as shown in Eq.(4).

3.3.3 Connectivity factor

Connectivity factorCF(x)considers the wireless connectivity between nodexand nodes in its one-hop region.A node with a higherCF(x)value is more suitable to serve as an edge node.CF(x)is calculated as follows:

where“#”denotes“Number”.

3.3.4 Membership functions

The membership functions are defined in Figure 2.These membership functions are used to change the representation of the three factors from the numerical format to the fuzzy format.

Figure 2.Fuzzy membership functions(left:SF;middle:TF;right:CF).

3.3.5 Fuzzy rules and output membership function

The same fuzzy rules as[29]are used.We use the Min-Max method[33]to combine multiple rules.The output membership function for defuzzification is shown in Figure 3.With this output membership function,the fuzzy value is converted to a numerical value that shows the edge competency level of the corresponding node.The node with the largest edge competency level among the candidates is selected as the edge node.

Figure 3.Membership function for defuzzification.

3.3.6 Advantage of using fuzzy logic

The edge node selection process has to consider three factors,namely,the stability factor,the topology factor,and the connectivity factor.The rationale behind the consideration of each factor is as follows.By using the stability factor,we can use a vehicle with lower velocity as long as possible,which is important for generating a stable edge architecture.The consideration of topology factor ensures that we can always choose a vehicle that has lower relative velocities to more neighbor vehicles.The connectivity factor ensures the communication qualities between the selected edge vehicles and their neighbors.The influence of each factor on the system performance is also dependent on the values of other factors.If the value of a factor is very low,we cannot get a satisfactory result,regardless of the other two factors.Since the vehicular environment is dynamic,and the information acquired at each vehicle is incomplete or imprecise,it is difficult to use a simple mathematical model to solve the edge node selection problem.That is why we use a fuzzy logic based approach for the edge node selection.

Figure 4 shows the effect of each factor on the competency value in the proposed fuzzy logic algorithm.We can observe from the figure that the proposed fuzzy logic based approach is able to jointly consider the three factors based on the defined fuzzy membership functions and fuzzy rule.Fuzzy logic makes a good decision when the inputs are imprecise or contradictory,and thus provides a flexible solution to the problem.

Figure 4.Effect of each factor on the competency value where three axes denote the three factors,respectively,and the competency value is shown by the color at the corresponding position[(a):some planar slices on the threedimensional distribution of the competency value;(b):a non-planar(z= x2 +y2)slice on the three-dimensional distribution of the competency value].

3.4 Communication Route Selection

The proposed scheme achieves a joint establishment of edge node selection and communication route selection.The edge nodes are not only used in processing local training models,but also used in generating efficient routes for data transmissions.Figure 5 shows the communication approach using edge nodes in the proposed scheme.Each ordinary vehicle can use its nearest edge vehicle as a forwarder to deliver packets to RSU or other vehicles.We use a routing approach that is similar to[34]where the edge vehicles are given higher priorities for being forwarder nodes.As shown in the figure,both nodes S1 and S2 use the same edge node for sending data to RSU.In this way,route selection becomes easier and more efficient as compared with the conventional routing approach because routes are determined with the selection of edge nodes without incurring extra routing overhead.Using the same forwarder nodes for different data communication flows in vehicular networks is also promising due to its advantage of achieving better MAC layer contention efficiency[34].

Figure 5.Communication route generation with edge vehicles.

Figure 6.Performance about ICI(left:ICI for different vehicle velocities;middle:Edge node change rate for different vehicle velocities;right:ICI for different numbers of vehicles).

IV.SIMULATION ANALYSIS

We generate a realistic vehicular scenario based on network simulator ns-2 using the same approach as in[34].We compare our scheme with“RSU-centric”,“Random Client”,“Random Edge”,and“CDS”[35,36]approaches.In the“RSU-centric”approach,model training is conducted at RSU using a centralized way,where all vehicles send their sensor data to the RSU.In the“Random Client”approach,some randomly selected vehicles are selected as FL clients,and no raw data are exchanged between vehicles.In the“Random Edge”approach,some vehicles are randomly selected as edge vehicles,and ordinary vehicles,which are non-edge vehicles,send data to their respective nearest edge vehicles.The edge nodes are selected to satisfy the constraint that each ordinary vehicle must find an edge node from its one-hop neighbors.The edge nodes are reselected when the current edge nodes cannot satisfy the constraint.In the“CDS”approach,the edge nodes are selected based on a connected dominating set(CDS)with the consideration of vehicle velocity.“Random Edge”and“CDS”use the selected edge nodes as FL clients.We use a freeway road with 3 lanes in each direction.The simulation parameters are shown in Table 1.The wireless channel follows the Nakagami model,of which parameters are shown in Table 2.Other parameters use the default setting of IEEE 802.11p implementation of ns-2[34].

Table 1.Simulation Environment.

Table 2.Parameters of the Nakagami Model.

4.1 Information Coverage Indicator

We introduce a metric called“Information coverage indicator(ICI)”which shows the percentage of the sensor data used in the generation of the final global model.While the accuracy of FL is dependent on the application and data,without loss of generality,we useICI as the performance indicator showing the accuracy of the model based on our assumption that more data result in a more accurate FL model.A higher ICI percentage indicates that the final global model considers more data gathered from the vehicles,and therefore the trained model is expected to be better in terms of accuracy.

Figure 6(left)shows the comparison of ICI for different vehicle velocities.The number of nodes is 300.The sensor data are generated at a rate of 48 Kbps.We can observe that,while“RSU-centric”sends all the generated sensor data to the RSU for training,its ICI is the lowest because the network bandwidth is insufficient for transmitting all the raw data.This explains that it is important to conduct local training and make a tradeoff between with and without raw data exchanges.The performance of“Random Client”is also unsatisfactory due to the following two reasons.First,the selected client vehicles do not possess a complete data to represent the environment(i.e.,road situation)well because there is no raw data exchange between vehicles,and each vehicle has only limited knowledge about the environment.Second,since the client vehicles are randomly selected,the performance is affected significantly by the vehicle mobility.“Random Edge”shows a better performance as compared with“Random Client”because each edge node collects information from neighbor vehicles for local model training.However,the inefficient selection of edge nodes incurs some information loss when reselections of edge nodes happen,resulting in an unsatisfactory performance especially for a highly mobile network scenario.Another reason for the poor performance of“Random Edge”is that the randomly selected edge nodes cannot ensure the communication reliability between an edge node and its neighbor vehicles.We observe that“CDS”performs better than“Random Edge”by considering the vehicle velocity.However,“CDS”does not show a satisfactory performance since the channel condition is not considered in the client(edge)selection.The proposed scheme achieves the best performance by considering the vehicle velocity,the vehicle distribution,and the wireless link connectivity between vehicles while selecting client nodes,and conducting the training of local model at the client vehicles.Each client node collects information from its one-hop neighbors,and therefore all the information collected by nodes in the entire network can be covered in the final global model.As shown in Figure 6(middle),the edge node change rate of the proposed scheme is much lower than“Random Edge”and“CDS”,resulting in its robustness to the vehicle velocity.

Figure 6(right)shows the ICI percentage for different vehicle densities.Since“RSU-centric”sends all raw sensor data to RSU,we observe a drastic performance degradation with the increase of node densities due to its requirement for high network bandwidth.The performance of“Random Client”is not sensitive to node density because the client vehicles do not collect raw data from other vehicles.By transmitting the raw sensor data to only one-hop region,“Random Edge”,“CDS”,and the proposed protocol also show heir strengths against the node density.The advantage of the proposed scheme over“Random Edge”and“CDS”illustrates the importance of using an efficient edge vehicle selection algorithm.

4.2 Communication Overhead

We also evaluate the communication overhead of sensor data transmissions and local model uploading.Here,the overhead is represented by the number of transmissions per packet.Figure 7(a)shows the number of transmissions for each sensor data packet at different vehicle velocities.The number of nodes is 300.As“RSU-centric”sends all raw data to RSU,it shows the highest overhead in terms of the number of transmissions.The overhead of“Random Client”is the lowest as raw sensor data are not exchanged.“Random Edge”,“CDS”,and the proposed scheme overlap because these approaches collect raw sensor data at edge vehicles.The overhead of the proposed scheme is acceptable because only one-hop wireless transmissions are used.

Figure 7(b)shows the sensor data transmission overhead for different numbers of vehicles.It is clear that the overhead of“RSU-centric”increases drastically with the number of vehicles as all the sensor data are sent to the RSU.“Random Edge”,“CDS”,and the proposed scheme show a lower overhead because the sensor data are only transmitted to an edge vehicle located in the one-hop region.The proposed scheme shows a low overhead for different vehicle densities,indicating its advantage in highly dense vehicular networks.

Figure 7.Sensor data transmission overhead[(a):for different vehicle velocities;(b):for different numbers of vehicles](“Random Edge”,“CDS”,and the proposed scheme overlap).

Figure 8(a)shows the data transmission overhead of uploading local models for different vehicle velocities.Since“RSU-centric”conducts a centralized learning approach at RSU,it does not incur the overhead of uploading local models.For“Random Edge”,an edge node reselection is triggered frequently in a highly mobile network,resulting in a higher overhead when the vehicle velocity increases.In addition,“Random Edge’ results in routes with more hops as compared with the proposed scheme due to the inefficient selection of edge nodes.Since“CDS”does not consider the channel condition in the client selection,the overhead also increases with the vehicle velocity due to the weak connectivity between two neighboring edge nodes.The proposed scheme achieves more stable and better placement of edge nodes,and therefore it reduces the overhead of reselecting edge nodes.The proposed scheme incurs the lowest overhead because the connectivity between two neighbor edge nodes is improved,and therefore the route established for local model uploading is more efficient.While“Random Client”incurs a higher overhead than the proposed scheme,its performance is better than“Random Edge”.

Figure 8.Local model data transmission overhead[(a):for different vehicle velocities;(b):for different numbers of vehicles](“Random Edge”,“CDS”,and the proposed scheme overlap).

Figure 8(b)shows the data transmission overhead of uploading local models for different vehicle densities.The overhead of“Random Edge”becomes lower with the increase of vehicle densities because a high number of vehicles can cause a more congested traffic network,which results in a lower vehicle velocity.However,this does not mean that“Random Edge”works well in a highly dense scenario.When the vehicle density increases,the number of transmissions for sensor data increases,and so the contention delay increases.This is shown in the delay analysis of these schemes(see Figure 9).The performance of the proposed approach is not affected by vehicle density because the proposed scheme works well even in a highly mobile network.

4.3 Communication Delay

Figure 9(a)shows the communication delay for various vehicle velocities.Here,the delay includes the time incurred for collecting sensor data and the model uploading.The size of raw sensor data at each sensor node is 1 KB,and the model parameter size is assumed to be 1 KB as well.It is clear that“RSUcentric”shows the largest delay,which indicates the impracticability of centralized learning in vehicular environments.The delay of“Random Edge”is affected significantly by the vehicle velocity due to the large number of transmissions as shown in Figure 8(b).While“CDS”shows a better performance than“Random Edge”,its delay is higher than“Random Client”.The proposed scheme shows a slightly lower delay as compared with“Random Client”while sending raw sensor data within one-hop region.This is because,in“Random Client”,the route selection process incurs a large delay because routes are likely to change due to the movement of vehicles.However,in the proposed scheme,since the edge nodes are selected by considering vehicle mobility and distribution,the selected edge nodes create a connected network route that is used to send model data.By sending the model data to the stable edge nodes,route changes are reduced,contributing to a lower communication delay.

Figure 9.Communication delay[(a):for different vehicle velocities;(b):for different numbers of vehicles].

Figure 9(b)shows the communication delay for various numbers of vehicles.We observe that all the schemes show an increase of delay with the growth of vehicle density.This is because a larger number of nodes results in more periodical hello message exchanges that increase the contention delay.The performance of“RSU-centric”is the worst because it sends all raw sensor data to RSU.The proposed scheme can achieve the best performance by not sending a large amount of raw sensor data and by selecting efficient edge nodes that enable more stable routes for model data transmissions.

V.CONCLUSION AND FUTURE WORK

We discuss the importance of considering the tradeoff between the accuracy of global model and communication overhead of FL in vehicular IoT,and propose an edge computing-based client selection and networking scheme.The proposed scheme uses a fuzzy logicbased distributed approach to conduct efficient selection of edge nodes to train local models.The edge vehicles are also used to work as forwarder nodes to enable efficient communications between vehicles and RSUs.We evaluate the proposed scheme by comparing it with existing baselines in various network settings focusing on the accuracy of final global model and the communication overhead.The simulation results show that the proposed scheme can achieve a low communication overhead while reducing information losses incurred by the distributed learning process of the conventional FL.

This paper does not discuss the performance of the proposed scheme in a specific FL system.In future work,we will evaluate the performance of the proposed scheme under real FL applications and datasets for vehicular environments focusing on the learning accuracy and communication overhead.We will also discuss about an improvement of the proposed scheme for FL with non-IID data.

ACKNOWLEDGEMENT

This research was supported in part by the National Natural Science Foundation of China under Grant No.62062031 and 61877053,in part by Inner Mongolia natural science foundation grant number 2019MS06035,and Inner Mongolia Science and Technology Major Project,China,in part by ROIS NII Open Collaborative Research 21S0601,and in part by JSPS KAKENHI grant numbers 18KK0279,19H04093,20H00592,and 21H03424.