Just a week ago I returned from a visit at the Microsoft EEC, where I had one of the most interesting experiences in my life. You may not remember, but in my opening post I wrote about me and my team winning the Imagine Cup 2008 competition. You can freely skip this post. There is actually not anything interesting in it except a link to the Lego Technic set I found in Paris. Oh and form Redmond I got another one. Ok, ok, I will cut to the point, we can talk about Lego later.
So the reason for all the silence on my blog lately is that I was busy working on my Imagine Cup project “EOS.NET” (sorry, no link here, hope to have one soon).
Shameless plug: EOS.NET is a modular system (of course MEF is used for the “modular” part) that can process satellite data from multiple satellites (currently NASA MODIS) and look for interesting events, like for example fires and deforestation (and in the future oil spill in the oceans, floods and what not). And yes it can send emails when such an event is discovered. It can also do many other interesting things but I will keep these as a surprise for now.
The main functionality that we had to work on was to make the thing run on a Microsoft Windows HPC cluster. We had to to that, so that when time comes to collect our prize and go the the Microsoft Enterprise Engineering Center (EEC) we are able to use this baby:
This is a 32 compute node cluster, and even though it is not the biggest cluster on Earth (there is another much bigger cluster at the EEC, I hope that I will get to use it next time) it can help you process your data many times faster. Oh, and with the latest version of the Windows HPC stack you do not even have to write complex MPI applications. With the new SOA model, if you know WCF you will be able to relatively easy write scalable HPC applications that can run on powerful clusters.
Of course we had some issues with our system at the beginning – a completely normal thing given the fact that we were not able to test on a real cluster before. But after some work we had amazing results – the data processing was orders of magnitude faster compared to when the application processes the data locally. One can easily imagine how the system will easily process data for the whole world and from different satellites at the same time using a large enough cluster.
After things settled a bit (although work continued with fast pace until the end of the week) and EOS.NET was able to run successfully on the cluster I even had a little time to play with the Microsoft Surface they had at the EEC. It’s really fun, you should try too:) I hope that soon these devices will be cheap enough so that everyone can afford them.
Generally the time that I spent at the EEC and at the Microsoft campus in Redmond was wonderful and I will remember it for many years.That was all for now. See you soon.
Posts
