Graph-Based Machine Learning for Optimizing Distributed Computing Performance

Abstract

In modern distributed computing environments, optimizing performance while ensuring scalability and resource efficiency is a critical challenge. Traditional optimization techniques often struggle to capture the complex interdependencies between distributed nodes, tasks, and communication overhead. This paper explores the application of Graph-Based Machine Learning (GBML) for performance optimization in distributed computing, leveraging the power of graph representations to model system interactions effectively. By structuring computing nodes and resource allocations as graph-based entities, our proposed approach enhances load balancing, minimizes latency, and improves overall throughput. We employ Graph Neural Networks (GNNs) and graph-based reinforcement learning to predict and optimize task scheduling, resource allocation, and fault tolerance in large-scale distributed systems. Experimental results demonstrate that our GBML approach achieves a 23% reduction in task execution time and a 17% improvement in resource utilization efficiency compared to conventional scheduling algorithms. Moreover, the system adapts dynamically to workload variations, reducing bottlenecks and improving resilience against failures. The proposed method proves highly effective in cloud, edge, and high-performance computing (HPC) environments, providing a scalable and adaptive solution for real-time distributed computing optimization.