Scalability of Massively Data Parallel Computing Problems

The performance of distributed parallel systems is influenced by many factors, for examples, the total number of nodes in the system, node speed, I/O speed, latency and communication speed of the inter-network. Adding more nodes and/or using more powerful nodes can improve the performance, but I/O and communication could suffer from it. To determine the relationship between performance and those factors, the scalability of the massively parallel computers by using a data parallel model is analyzed. The relationship between the number of node, the speed of the nodes and the communication speed and latency of the links between nodes, the I/O speed of the system is obtained. The results shows that it is necessary to increase the I/O speed and the speed of the links (decrease the latency) by a certain ratio while increasing the number or the speed of the processors, so as to keep the scalability of the system. Only increasing the number or the speed of the processors will decrease the scalability of the system.