Constructed encoded data based coded distributed DNN training for edge computing scenario

Deep learning in unmanned aerial vehicle (UAV) deployment encounters two problems: 1, One UAV may fail to store and execute large Deep Neural Network (DNN) model. 2, One UAV fails to accomplish real time services. One possible solution is the group of UAVs (nodes) collectively generate swarm intelligence in the form of edge computing scenario, namely in the distributed computing (DC) mode with clever arrangement, say Coded Distributed Computing (CDC). In CDC systems, the redundant computation introduced by linear coding can compensate stragglers. However, since linear property cannot pass the nonlinear activation function in Deep Neural Network (DNN) training, coding/decoding for CDC need to be applied layer by layer, which slows down the training. To avoid layer-by-layer coding/decoding, we propose a novel DNN training scheme based on constructing encoded data. This construction process lies before the training process (can be done before training without any impact on training efficiency). Based on both the original data and the newly constructed encoded data, the training phase can take advantage of the $(n,k)$ property (Wait for the first $k$ returned data) and hence improve the training speed. The training process does not require encoding/decoding operations, and hence significantly improves the training speed. Experimental results show that the training scheme based on constructed encoded data can achieve prediction accuracy approximating that of the centralized one and significantly reduce the latency compared to the layer-by-layer linear encoding and decoding scheme.