A fast malware detection model based on heterogeneous graph similarity search

The Android operating system has long been vulnerable to malicious software. Existing malware detection methods often fail to identify ever-evolving malware and are slow in detection. To address this, we propose a new model for rapid Android malware detection, which constructs various Android entities and relationships into a heterogeneous graph. Firstly, to address the semantic fusion problem in high-order heterogeneous graphs that arises with the increase in the depth of the heterogeneous graph model, we introduce adaptive weights during node aggregation to absorb the local semantics of nodes. This allows more attention to be paid to the feature information of the node itself during the semantic aggregation stage, thereby avoiding semantic confusion. Secondly, to mitigate the high time costs associated with detecting unknown applications, we employ an incremental similarity search model. This model quickly measures the similarity between unknown applications and those within the sample, aggregating the weights of nodes based on similarity scores and semantic attention coefficients, thereby enabling rapid detection. Lastly, considering the high time and space complexity of calculating node similarity scores on large graphs, we design a NeuSim model based on an encoder–decoder structure. The encoder module embeds each path instance as a vector, while the decoder converts the vector into a scalar similarity score, significantly reducing the complexity of the calculation. Experiments demonstrate that this model can not only rapidly detect malware but also capture high-level semantic relationships of application software in complex malware networks by hierarchically aggregating information from neighbors and meta-paths of different orders. Moreover, this model achieved an AUC of 0.9356 and an F1 score of 0.9355, surpassing existing malware detection algorithms. Particularly in the detection of unknown application software, the NeuSim model can double the detection speed, with an average detection time of 105 ms.