Journal of Parallel and Distributed Computing最新文献

Function-as-a-Service execution model in serverless computing has been successful in running large-scale computations like MapReduce, linear algebra, and machine learning. However, little attention has been given to executing highly-dynamic parallel applications with unbalanced and irregular workloads. These algorithms are difficult to execute with good parallel efficiency due to the challenge of provisioning the required computing resources in time, leading to resource over- and under-provisioning in clusters of static size. We propose that the elasticity and fine-grained “pay-as-you-go model” of the FaaS model can be a key enabler for effectively running these algorithms in the cloud. We use a simple serverless executor pool abstraction, and evaluate it using three algorithms with unbalanced and irregular workloads. Results show that their serverless implementation can outperform a static Spark cluster of large virtual machines by up to 55% with the same cost, and can even outperform a single large virtual machine running locally.

Deep Neural Networks (DNNs) have gained widespread popularity in different domain applications due to their dominant performance. Despite the prevalence of massively parallel multi-core processor architectures, adopting large DNN models in embedded systems remains challenging, as most embedded applications are designed with single-core processors in mind. This limits DNN adoption in embedded systems due to inefficient leveraging of model parallelization and workload partitioning. Prior solutions attempt to address these challenges using data and model parallelism. However, they lack in finding optimal DNN model partitions and distributing them efficiently to achieve improved performance.

This paper proposes a DNN model parallelism framework to accelerate model training by finding the optimal number of model partitions and resource provisions. The proposed framework combines data and model parallelism techniques to optimize the parallel processing of DNNs for embedded applications. In addition, it implements the pipeline execution of the partitioned models and integrates a task controller to manage the computing resources. The experimental results for image object detection demonstrate the applicability of our proposed framework in estimating the latest execution time and reducing overall model training time by almost 44.87% compared to the baseline AlexNet convolutional neural network (CNN) model.

Collaborative Filtering (CF) is one of the most successful techniques for quality-of-service (QoS) prediction and cloud service recommendation. However, individual QoS are time-sensitive and fluctuating, resulting in the QoS predicted by CF to deviate from the actual values. In addition, existing CF approaches ignore inauthentic QoS values given by untrustworthy users. To address these problems, we develop a two-dimensional time-aware and trust-aware service recommendation approach (TaTruSR). First, considering both timeliness and fluctuation of service QoS, an integrative method incorporates time weight (time dimension) and temporal certainty (QoS dimension) are proposed to determine the contribution of co-invoked services. Time weight is computed by a personalized logistic decay function to measure QoS changes by weighting the length of the time interval, while temporal certainty is defined by entropy to acquire the degree of QoS fluctuation over a period of time. Second, a set of most similar and trusted neighbors can be identified from the view of the time-aware similarity model and trust model. In models, the direct similarity and local trust are calculated based on the QoS ratings and contribution of co-invoked services to improve the prediction accuracy and eliminate unreliable QoS. The indirect similarity and global trust are estimated based on user relationship networks to alleviate the data sparsity problem. Finally, missing QoS prediction and reliable service recommendation for the active user can be achieved based on enhanced similarity and trust. A case study and experimental evaluation on real-world datasets demonstrate the practicality and accuracy of the proposed approach.

With the development of the times, rumors spread rapidly on the Internet. Firstly, this paper establishes a reaction-diffusion system with Allee effect to describe the rumor spreading process and derives the necessary conditions for the emergence of Turing bifurcation. Next, a parameter identification approach utilizing optimal control theory is shown. Ultimately, the impact of the magnitude of the certain parameters in the objective function on parameter identification is examined through numerous parameter identifications in continuous space and various complex networks. Additionally, the convergence rates and error magnitudes of different algorithms for parameter identification are studied across different spatial structures.

Knowledge graphs can be used in many areas related to data semantics such as question-answering systems, knowledge based systems. However, the currently constructed knowledge graphs need to be complemented for better knowledge in terms of relations. It is called knowledge graph completion. To add new relations to the existing knowledge graph by using knowledge graph embedding models, we have to evaluate $N\times N\times R$ vector operations, where N is the number of entities and R is the number of relation types. It is very costly.

In this paper, we provide an efficient knowledge graph completion framework on GPUs to get new relations using knowledge graph embedding vectors. In the proposed framework, we first define transformable to a metric space and then provide a method to transform the knowledge graph completion problem into the similarity join problem for a model which is transformable to a metric space. After that, to efficiently process the similarity join problem, we derive formulas using the properties of a metric space. Based on the formulas, we develop a fast knowledge graph completion algorithm. Finally, we experimentally show that our framework can efficiently process the knowledge graph completion problem.

HPL-AI, also known as HPL-MxP, is a new benchmark program used to evaluate the upper-bound performance of AI-related tasks on a specific computing cluster. It solves a large linear equation system in FP64, preconditioned by complete LU factorization in lower precision. In this paper, we propose a new HPL-AI approach that relies on the factorization of the coefficient matrix in mixed precision: FP32 diagonals and FP16 off-diagonals. Without compromising the quality of the resultant LU preconditioner, the proposed approach only utilizes the primitive of dense matrix multiplication in FP16 on the accelerator, maximizing the FP16 throughput. Numerical analysis and experiments validate our approach, ensuring avoidance of numerical underflow or overflow during factorization. We implement the proposed approach on Kunpeng+Ascend clusters, a novel AI-specific platform with exceedingly high FP16 peak performance. By applying various optimization techniques, including 2D lookahead, HCCL-based communication pipeline, and SYCL-based tasks overlapping, we achieve 975 TFlops on a single node and nearly 100 PFlops on a cluster of 128 nodes, with a weak scalability of 79.8%.

The g-restricted edge connectivity is an important measurement to assess the reliability of networks. The g-restricted edge connectivity of a connected graph G is the minimum size of a set of edges in G, if it exists, whose deletion separates G and leaves every vertex in the remaining components with at least g neighbors. The k-ary n-cube is an extension of the hypercube network and has many desirable properties. It has been used to build the architecture of the Supercomputer Fugaku. This paper establishes that for $g\le n$, the g-restricted edge connectivity of 3-ary n-cubes is $3^{\lfloor g/2\rfloor }(1+(g\text{mod}2\left)\right)(2n-g)$, and the g-restricted edge connectivity of k-ary n-cubes with $k\ge 4$ is $2^g(2n-g)$. These results imply that in $Q_n^3$ with at most $3^{\lfloor g/2\rfloor }(1+(g\text{mod}2\left)\right)(2n-g)-1$ faulty edges, or $Q_n^k(k\ge 4)$ with at most $2^g(2n-g)-1$ faulty edges, if each vertex is incident with at least g fault-free edges, then the remaining network is connected.

The k-ary n-cube $Q_n^k$ serves as an indispensable interconnection network in the design of data center networks, network-on-chips, and parallel computing systems since it possesses numerous attractive properties. In these parallel architectures, the paired (or unpaired) many-to-many m-disjoint path cover (m-DPC) plays a significant role in message transmission. Nevertheless, the construction of m-DPC is severely obstructed by large-scale edge faults due to the rapid growth of the system scale. In this paper, we investigate the existence of paired 2-DPC in $Q_n^k$ under the partitioned edge fault (PEF) model, which is a novel fault model for enhancing the networks' fault-tolerance related to path embedding problem. We exploit this model to evaluate the edge fault-tolerance of $Q_n^k$ when a paired 2-DPC is embedded into $Q_n^k$. Compared to the other known works, our results can help $Q_n^k$ to achieve large-scale edge fault-tolerance.

Outliers and noises are unavoidable factors that cause performance of the distributed learning algorithms to be severely reduced. Developing a robust algorithm is vital in applications such as system identification and forecasting stock market, in which noise on the desired signals may intensely divert the solutions. In this paper, we propose a Robust Diffusion Stochastic Gradient Descent (RDSGD) algorithm based on the pseudo-Huber loss function which can significantly suppress the effect of Gaussian and non-Gaussian noises on estimation performances in the adaptive networks. Performance and convergence behavior of RDSGD are assessed in presence of the α-stable and Mixed-Gaussian noises in the stationary and non-stationary environments. Simulation results show that the proposed algorithm can achieve both higher convergence rate and lower steady-state misadjustment than the conventional diffusion algorithms and several robust algorithms.