Create a Raspberry Pi Super Computer

This will be a "mostly" cut and paste tutorial. Once you reboot the computer after running raspi-config, open a terminal window, open this in Midori and just cut and paste the commands (Use Shift + Insert to paste into the terminal window).

University of Southampton is where this tutorial got started. There is more information and more links there.

There is also another site by Philip Leonard that demostrates a different way to install this.

A couple of setup notes.

1) This is an update of other installation manuals. The biggest change is that this will install MPICH2 version 1.5 instead of 1.4.1. The University of Southampton page (see above) has the original instructions.

2) The router that serves my home network runs a DHCP server at 192.168.10 but it only allocates IP addresses between the range of 50 and 100 and I don't have any other devices on the network that have a static IP address. So I will use some really high numbers for these servers. They will be

  • Master: 192.168.10.200
  • Node02: 192.168.10.210
  • Node03: 192.168.10.211
  • Node04: 192.168.10.212

You will have to look at your network and adjust your IP addresses.

3) Whenever you see your-editor this means: Using the editor of your choice. Choices could be vi, vim, nano, emacs or others. Substitute what you use for your-editor.

4) I tested this procedure using:

  • Image: 2012-10-28-wheezy-raspbian.img
  • SD Card: 32GB Class 10
  • RPI: 1 Model B Rev 2.0 (512MB) and 1 Model B Rev 1.0 (256MB)

So, let's get started.

  1. Create a SD card using a Raspian image.
  2. Start up the Raspberry Pi
  3. Set timezone
  4. Set locale
  5. Set keyboard layout
  6. Expand root partition to fill SD card
Reboot the computer

Once it's back up:

sudo apt-get update
sudo apt-get dist-upgrade

My personal preference; this is optional.

Install vim to use instead of vim-tiny:

sudo apt-get install vim

Convert the network connection from DHCP to Static IP address:

sudo *your-editor* /etc/network/interfaces

Original:

{% include_code Original interfaces file mpich2/orig_interfaces lang:bash %}

Updated:

{% include_code New interfaces file mpich2/new_interfaces lang:bash %}

Restart the network (or reboot) and make sure the connection is good.

sudo /etc/init.d/networking restart

To test, Try:

ping -c 3 google.com

Installation of the MPICH2 platform

Now let's install some software, download the source code and set up some directories.

sudo apt-get install gfortran

cd
mkdir mpich2
cd mpich2

wget http://www.mcs.anl.gov/research/projects/mpich2/downloads/tarballs/1.5/mpich2-1.5.tar.gz
tar zxvf mpich2-1.5.tar.gz

cd
sudo mkdir -p /home/rpimpi/mpich2-install

mkdir mpich_build
cd mpich_build

WARNING! The next three steps will take a long time to run. Estimates:

  • configure: about 20-30 minutes
  • make: about 1.5 hours
  • install: about 5 minutes

Your times may be different.

Configure
sudo /home/pi/mpich2/mpich2-1.5/configure -prefix=/home/rpimpi/mpich2-install
Make
sudo make
Install
sudo make install
That is all the basic installation steps

Setup the environment so you can run the programs

Type this on the command line to set up the path.

cd
$ export PATH=$PATH:/home/rpimpi/mpich2-install/bin

Edit ~/.bashrc file and add the next two lines to the bottom of the file to make the path change permanent

*your-editor* .bashrc 

# Add MPI to path
PATH="$PATH:/home/rpimpi/mpich2-install/bin"

Test the path

which mpicc

Returns: /home/rpimpi/mpich2-install/bin/mpicc

which mpiexec

Returns: /home/rpimpi/mpich2-install/bin/mpiexec

Set up the testing directory, create a file named "machinefile" and add the IP address of the machine.

cd
mkdir mpi_testing
cd mpi_testing
*your-editor* machinefile

If you don't remember what the IP address is:

ifconfig

and look for the number in the eth0 section.

Here is a sample "machinefile" with one IP address:

Now, let's run some tests:

Test 1

mpiexec -f machinefile –n 1 hostname

Output is:

raspberrypi
Test 2

mpiexec -f machinefile -n 2 ~/mpich_build/examples/cpi

Output is

Process 0 of 2 is on raspberrypi

Process 1 of 2 is on raspberrypi

pi is approximately 3.1415926544231318, Error is 0.0000000008333387

If you see similar results:

Congratulations!

Label this SD card as the Master and create another SD Card with the same applications and software but with a different static IP address. I'll leave it to you to decide how to do that. You could just repeat the same instructions or you can look in to cloneing the card with the dd command or other programs.

When you have completed creating a second (or more) SD card, continue on.

Now that you have at least two, configure the Raspberry Pi's to talk to each other and change the hostname on the Node.

Create the network of computers

  • Insert the SD cards into each Raspberry Pi
  • Attach the Raspberry Pi with the Master SD card to the monitor, keyboard and mouse.
  • Connect both to a network switch and connect the network switch to your network router.
  • Plug them in

Set up the ssh connections between the computers.

I do this a little bit more manually then others but... And a reminder; these instructions use the IP address of 192.168.10.210. Remember to change that to the IP address of your node.

Create the key on Master
cd
ssh-keygen -t rsa -C mpich2-master

When asked for a passphrase, I don't put one in, I just press the Enter key twice.

Setup the node to receive the public key
ssh pi@192.168.10.210
mkdir .ssh
chmod -R 700 .ssh
exit

The exit command will close your connection to the node.

Use scp to copy the public key to the node
cd
scp .ssh/id_rsa.pub pi@192.168.10.210:.ssh/authorized_keys

Type in pi's password when prompted and then scp will copy the public key to the node.

Check to make sure it works. This command should log you straight in.

ssh pi@192.168.10.210

Since you are logged in to the node, change the hostname.

Change the hostname on the node
sudo *your-editor* /etc/hostname

Change "raspberrypi" to "node02" or whatever your naming convention is.

And

sudo *your-editor* /etc/hosts

Look at the last line.

127.0.1.1 raspberrypi

Change "raspberrypi" to "node02" or whatever your naming convention is.

Then reboot the computer

sudo reboot

This will close the connection to the node so while that is rebooting, you have one more change to make on the master. ~/mpi_testing/machinefile needs to be updated with the IP address of the new node.

cd ~/mpi_testing
*your-editor* machinefile

{% include_code machinefile file with two IP addresses mpich2/machinefile2 lang:bash %}

Now re-run the tests from earlier (Make sure the number of nodes is 2 or how ever many nodes you now have)

Test 1
mpiexec -f machinefile –n 2 hostname

Output is:

raspberrypi
node02
Test 2
mpiexec -f machinefile -n 2 ~/mpich_build/examples/cpi

Output is

Process 0 of 2 is on raspberrypi
Process 1 of 2 is on node02
pi is approximately 3.1415926544231318, Error is 0.0000000008333387

Change the password on each node

Now that you can easily ssh into each machine, do that now and change the password.

ssh pi@192.168.10.210
passwd

At the prompt, type in the new password.

NOTE: the screen will NOT echo back what you are typing and you will have to enter the same password twice.

And when you are done with that, use the passwd command on the Master to change it there also.