MGDN: A Graph of Graphs Neural Network for Malware Detection

Apr 30, 2026·
Jianke Yu
Jianke Yu
,
Hanchen Wang
,
Ying Zhang
,
Wenjie Zhang
,
Lu Qin
,
Longbin Lai
,
Bailin Yang
· 1 min read
CCF-A CAS Zone 1 Top JCR Q1
Abstract
With the rapid evolution of software ecosystems, malware has become increasingly sophisticated, posing severe threats to cybersecurity. Modern malware often exhibits complex hierarchical behaviors that can be naturally modeled as a Graph of Graphs (GoG): a global Function Call Graph (FCG) whose function nodes can be annotated with fine-grained Control Flow Graphs (CFGs). This nested structure makes detection challenging for existing methods. In this work, we propose a novel end-to-end framework, namely Malware Graph of Graphs Detection Network (MGDN), specifically optimized for malware detection on hierarchical program graphs. MGDN integrates four modules, including Graph Feature Construction, FCG Malware Discrimination, CFG Risk Attribution, and Integrated Malware Prediction, to capture multi-level semantic patterns and detect malicious programs. Extensive experiments on large-scale real-world datasets demonstrate that MGDN consistently outperforms state-of-the-art generic GNNs, hierarchical graph learning methods, and specialized malware detection baselines across multiple metrics. Empirically, MGDN achieves average relative improvements of 16.22% in PR-AUC, 9.91% in Macro-F1, and 1.89% in Micro-F1 against the strongest competing baselines across all evaluated datasets, highlighting its effectiveness and robustness in diverse real-world scenarios.
Type
Publication
IEEE Transactions on Knowledge and Data Engineering
Status
Peer-reviewed
Awards
CCF-A
IEEE Transactions on Knowledge and Data Engineering · 2026
CAS Zone 1 Top
IEEE Transactions on Knowledge and Data Engineering · 2026
JCR Q1
IEEE Transactions on Knowledge and Data Engineering · 2026
publications

IEEE Transactions on Knowledge and Data Engineering, 2026

With the rapid evolution of software ecosystems, malware has become increasingly sophisticated, posing severe threats to cybersecurity. Modern malware often exhibits complex hierarchical behaviors that can be naturally modeled as a Graph of Graphs (GoG): a global Function Call Graph (FCG) whose function nodes can be annotated with fine-grained Control Flow Graphs (CFGs). This nested structure makes detection challenging for existing methods. In this work, we propose a novel end-to-end framework, namely Malware Graph of Graphs Detection Network (MGDN), specifically optimized for malware detection on hierarchical program graphs. MGDN integrates four modules, including Graph Feature Construction, FCG Malware Discrimination, CFG Risk Attribution, and Integrated Malware Prediction, to capture multi-level semantic patterns and detect malicious programs. Extensive experiments on large-scale real-world datasets demonstrate that MGDN consistently outperforms state-of-the-art generic GNNs, hierarchical graph learning methods, and specialized malware detection baselines across multiple metrics. Empirically, MGDN achieves average relative improvements of 16.22% in PR-AUC, 9.91% in Macro-F1, and 1.89% in Micro-F1 against the strongest competing baselines across all evaluated datasets, highlighting its effectiveness and robustness in diverse real-world scenarios.

Jianke Yu
Authors
Jianke Yu (he/him)
PhD (2026) in Graph Machine Learning & Databases
I completed my PhD at the University of Technology Sydney (UTS) in 2026, supervised by Prof. Ying Zhang, Prof. Lu Qin and Dr Hanchen Wang. My research develops machine learning algorithms for graph-structured data and database systems — graph neural network architectures, and learning-based methods that improve database and data-mining algorithms (fraud detection, graph similarity, supergraph search, malware detection). My work has appeared in KDD and IEEE TKDE. I am joining Zhejiang Gongshang University as a faculty member in 2027.