Graph Attention Networks for Automated Feature Engineering in LargeScale Data Processing

Abstract

As data volume and complexity continue to grow, automated feature engineering has become a critical aspect of large-scale data processing. Traditional feature engineering methods often rely on domain expertise and handcrafted rules, which limit scalability and adaptability. This study explores the application of Graph Attention Networks (GATs) to automate feature extraction and selection in large-scale data environments. By leveraging attention mechanisms to dynamically assign importance to different data relationships, GATs enhance feature representation while reducing dimensionality and computational overhead. The proposed approach was evaluated on multiple real-world datasets, including financial fraud detection, network security analysis, and healthcare risk assessment, demonstrating significant improvements in predictive performance, feature interpretability, and computational efficiency. The results indicate that GAT-based automated feature engineering outperforms conventional techniques by improving model accuracy by up to 15.4% and reducing feature redundancy by 30.8%. These findings highlight the potential of graph-based deep learning in transforming automated feature engineering for high-dimensional and interconnected data.