High Performance Computing Cluster using Raspberry Pi Node

Conventional data-center clusters are huge in size, they require more power and more cooling solutions and they are not affordable by everyone. In this project we will introduce a High-Performance Computing cluster, which will be assembled using 8 Raspberry Pi nodes interconnected with 100 Mbits Eth

Project Title

High Performance Computing Cluster using Raspberry Pi Node

Project Area of Specialization

Computer Science

Project Summary

Conventional data-center clusters are huge in size, they require more power and more cooling solutions and they are not affordable by everyone. In this project we will introduce a High-Performance Computing cluster, which will be assembled using 8 Raspberry Pi nodes interconnected with 100 Mbits Ethernet links. This cluster will have a low total cost, comparable to that of a single workstation, and will consume relatively little energy. These qualities along with its light weight, small volume and passive, ambient cooling render it eminently suitable for a number of applications that to which a conventional cluster with its high attendant cost and special infrastructure requirements is ill-suited.

The cluster that we will present in this work combines the unconventional elements of utilizing low-cost and low-power ARM processors, commodity Ethernet interconnects, and low-power flash based local storage, whilst supporting traditional technologies such as MPI upon which many supercomputing applications are built. With a very compact overall size, light weight, and passive, ambient cooling, our cluster will be ideal for demonstration and educational purposes.

Conventional data-center clusters are huge in size, they require more power and more cooling solutions and they are not affordable by everyone. In this project we will introduce a High-Performance Computing cluster, which will be assembled using 8 Raspberry Pi nodes interconnected with 100 Mbits Ethernet links. This cluster will have a low total cost, comparable to that of a single workstation, and will consume relatively little energy. These qualities along with its light weight, small volume and passive, ambient cooling render it eminently suitable for a number of applications that to which a conventional cluster with its high attendant cost and special infrastructure requirements is ill-suited.

The cluster that we will present in this work combines the unconventional elements of utilizing low-cost and low-power ARM processors, commodity Ethernet interconnects, and low-power flash based local storage, whilst supporting traditional technologies such as MPI upon which many supercomputing applications are built. With a very compact overall size, light weight, and passive, ambient cooling, our cluster will be ideal for demonstration and educational purposes.

Project Objectives

1. Hardware Acquisition i.e. Raspberry Pie 3 Model B or B+, SD Cards, Ethernet cables, Cooling fans, USB cables etc.
2. Hardware Assembly i.e. Interconnecting 8 Raspberry Pi nodes using 100 Mbits Ethernet Links.
3. Operating System Installation.
4. Simultaneous node management.
5. Present the results of benchmarking both the computational power and network performance.

Project Implementation Method

• Hadoop
• Map reduced
• SPARK

For Becnhmarks:

• Linpack
• HPL
• iPerf
• NetPIPE
• SPEC
• IOzone

Benefits of the Project

The small size, low power usage, low cost and portability of the cluster must be contrasted against its relatively low compute power and limited communications bandwidth (compared to a contemporary, traditional HPC cluster), making this architecture most appropriate as a teaching cluster. In this role, it could be used to help students to understand the building blocks of parallel and high performance computation. It is also a valuable alternative to using virtualization to demonstrate the principles of HPC, since the latter tends to hide various “real-world” aspects of HPC such as interconnects between nodes, power, cooling, file systems, etc. Due to its low cost, it may also bring cluster computing to institutions which lack the space, funds, and administrative staff to run a conventional cluster. Even in institutions which have clusters available, clusters could be made available to students as a rapid means of testing the functionality and scaling of parallel codes without the long job queues often associated with institutional systems. It would also encourage developers to work with other architectures, enhancing code portability, which may be of increasing importance as low-power ARM chips begin to enjoy a larger role in the data centre.

Technical Details of Final Deliverable

1. 8-node Raspberry Pi cluster.
2. Benchmarking results of the computational power i.e. the CPUs and RAMs, and network performance of the cluster.

We will describe how the unconventional architecture
of this cluster reduces its total cost and makes it an ideal
resource for educational use in inspiring students who are learning the fundamentals of high-performance and scientific computing. We also explored additional application areas where similar architectures might offer advantages over traditional clusters. We foresee that, although our cluster architecture is unconventional by today’s standards, many aspects of its design—the use of open source hardware and software, the adoption of low-power processors, and the wider application of flash based storage—will become increasingly mainstream into the future.

Final Deliverable of the Project

HW/SW integrated system

Core Industry

IT

Other Industries

Core Technology

Cloud Infrastructure

Other Technologies

Big Data

Sustainable Development Goals

Industry, Innovation and Infrastructure

Required Resources

If you need this project, please contact me on contact@adikhanofficial.com