Tags
Since the Raspberry Pi is out I have been looking for a small project where it makes sense to use one and I finally got one! This year the British weather went abroad and left us with two complete weeks of 28+ degrees Celsius which is unheard of (more or less). This caused many problems such as melting motorways and cracked rails but above all it exposed the inadequacy of the ventilation in my personal server room (aka garden shed). Most of the services provided by those computers are not critical but there is one that is indispensable, especially in this weather. As it happens I am a member of a local sailing club that has a weather station. The weather station uploads the wind direction and speed to the website every 15 minutes which is great but does not give you any clue about the variation which is an important factor in inland waters (the club is in this beautiful lake). Without historical information it is hard to decide if there is enough wind to justify loading the car with all the boat gear and drive there.
So, a couple of years ago we solved this by creating a small python script that downloads the pictures with the speed and direction of the wind, OCRs them to extract the speed and does some poking to work out the direction and then creates an historical chart using matplotlib. Maybe I could have done something simpler using the weather station’s API (if there is one) but this was so much fun and I did not have to ask for access to the weather station. The other club members liked it so much that the charts are now part of the club’s website. So many people use them that it became a point of honour to keep the charts updated and one of the servers in the garden shed was partly dedicated to this. With the high temperatures the service became a bit unstable and that is where the Raspberry Pi comes in.
Start here is you are only interested in the Raspberry Pi
When the Raspberry Pi was launched I tried to buy one but soon gave up with all the queues and Ebay crazy prices. After all I did not have anything planned for it and it would end up in a box like (I would guess) so many have. But this time I had a good reason. I needed to replace this overheating 4 CPU AMD server with 8Gb of RAM and a couple of SSDs… just kidding it does not have any SSDs. So I bought a bundle off some company in Amazon, connected all the wires and turned it on. The bundle came with NOOBS installed in the SD card and I installed the recommended choice of OS: the Raspbian (not a very good name I should add). If you are new to the Raspberry Pi, NOOBS is some kind of application that makes installing a Raspberry Pi simpler than installing any Windows application. I was expecting some back and forth as most Linux installations require but no. It just installed all by itself, gave a few tips during the installation and there I was on a desktop. I am sorry to say this but LXDE, the desktop, could look a bit nicer but, hey, it is free and fully functional and I am not complaining.
OCR and number crunching on a Raspberry Pi
The program that downloads the images, OCRs them, creates the new charts and uploads them to the website is written in Python 2.7 and has a few dependencies. So the first task was installing them as well as updating the Raspberry Pi software.
First update the Raspberry Pi (hopefully this in the correct order, I never remember which comes first):
sudo apt-get update sudo apt-get upgrade
Then install Tesseract OCR and Imagemagick. The latter is required for the image manipulations:
sudo apt-get install tesseract-ocr sudo apt-get install imagemagick sudo apt-get install python-imaging-tk
Imagemagick is a very cool tool for image manipulation from the command line but that is a post for another day.
I also need some fonts for the charts:
sudo apt-get install ttf-mscorefonts-installer
The reason I am using these fonts is because the script was developed on Windows and requires some common Windows fonts.
Then came the heavy duty dependencies. Matplotlib requires numpy:
sudo apt-get python-numpy sudo apt-get install python-matplotlib
These took forever to install, maybe half an hour or so but they installed without a glitch. At this point I was in awe, I mean, this is a generic computer for less than £50 all included and I can install numpy!
And finally I got to my script. As you would expect there were a few rough edges caused by crossing from Windows to Linux such as paths but in a few minutes the script was up and running. I am always surprised when a Python script just runs on a completely different platform without any code changes. But then I had to automate the script to run every 15 minutes and that is where cron comes in. Three hours later (!) I got it working. This is an area where the Windows Scheduler wins hands down. It is much easier to muddle through these tasks in Windows than on Linux. On Linux you have to learn stuff and read documentation before you can produce results. I am still not sure which one is more beneficial to my life. But hey, I got it working and I even have a couple of tips: install exim4 (sudo apt-get install exim4) to receive the errors from cron by mail. They just disappear otherwise, especially if they are security errors. To read mail on the Raspberry Pi without installing a mail reader type mail in the command line. This is Outlook’s great grandfather and is a joy to use.
The other tip is to cd to the folder where the script runs before running it in cron. This is my cron job:
*/15 * * * * cd /home/pi/cvlsc && python windspeed.py
That way all the temporary files can be created without access denied errors which were the cause of 80% of the problems.
Performance
My script is not the best example as far optimization is concerned. Typically I load the same data over and again and save and read files at will which is normal since I wrote the script in a couple of sessions over a weekend after sailing all day (sailing a Laser is surprisingly tiring) and I only use Python sporadically. But still, the Raspberry Pi just goes through it without any problems even the OCR-ing is quite fast. The only place where it choked was the creation of two Wind Roses that use all the historical data (some 80k rows) for each chart. The Raspberry Pi took around 9 minutes to create them where the AMD server took less than a minute. So now I need to create a new script that updates those charts on a different schedule, maybe once a week, or the Raspberry Pi would be clocking 100% most of the time. All in all I am very happy with the performance of the Raspberry Pi but you do need to adjust your performance expectations.
In closing
So this how my Raspberry Pi first adventure ends. The service has been up for 2 days without any glitches and although the box is warm the CPU is at 55 degrees Celsius (cat /sys/class/thermal/thermal_zone0/temp shows the CPU temperature) which seems ok given the 28+ in this office and this is not going to last…