Last week has mostly been taken up with QCon London. I really had a great time and I would like to give a big thanks to Google for supporting my travel and registration costs.

QCon is a conference focusing on 19 different tracks. Some to mention: Architectures You’ve Always Wondered About, Software Craftsmanship, Functional programming,  Irresponsible Architectures and Unusual Architects,  Pragmatic Cloud Computing, Agile Evolution, How do you test that? and Browser as a Platform. I attended to one or two talks from almost every track except .Net and Java oriented ones. Keynotes from Dan Ingalls (Forty Years of Fun with Computers), Ralph Johnson (Living and working with aging software) and Robert Martin, aka Uncle Bob (Bad Code, Craftsmanship, Engineering, and Certification) were inspring.

I also got the chance to chat with Dan Ingalls (principal architect of five generations of Smalltalk). I asked him if he follows a method while working (like Pomodore that Dan North recommended in his “Simplicity – the way of the unusual architect” talk). Hopefully, we share a similar characteristic: we can’t work if we don’t like the job but when we like it, we can’t stop working from morning till night.  He recommended me to go where I think I would have the most fun. If something bothers you, it is ok: “If it’s hot, it is hot. If it’s not, it is not!” There’s always something to do when you can’t work; empty the rubbish or wash the dishes. And when you concentrate, start to work again -but know yourself very well.

Me and Dan Ingalls

We also talked about the lack of women in computing. He shared some of his observations; for example in a conference about Wikipedia, he observed there are almost same number of women and men. But when it comes to more technical and less social conferences and events, there are really very few women participating. He also added maybe there’s a genetic factor about this. He has two boys who cannot stop being “boys” –always breaking/fixing things but in fact, that’s what all about the computers!

There are lots of ideas and keys to share, here are some main ideas:

From Uncle Bob’s keynote (slides are available here):

• Follow the Boy Scout rule: Always leave things a little better than you find.
• Methods should be less than 20 lines.
• Don’t have a function that takes a boolean. It is clear that it does two things; one if its false, another if its true.
• Cut/Paste is bug replication
• Extract until you drop! Keep extracting until all functions only do one thing
• Source code represent the design -not the UML tools.

Architectures You’ve Always Wondered About was one of the tracks I wondered about =] Some gems from (Facebook: Architecture and Design) by Aditya Agarwal (Director of Engineering at Facebook):

• Services of philosophy: choose the tool for the right task. They use Thrift, a lightweight software framework for cross-language development (C++, PHP, Python, Ruby, Erlang, Haskell, etc.)
• Most important thing in their engineering team: How quickly can you move?
  

Agarwal said despite being a small team (over 1 million active users per engineer) they do great because of the Facebook culture. There are three very important things in FB:

• Move fast and break things
• Huge impact with small teams
• Be bold and innovate
  

Agarwal also gave some important tips for MySQL. They have about 6k server-years of runtime experience without data loss or corruption (can you believe it?!) Here are my notes:

• Don’t ever store non-static data in a central database
• Data driven schemas make for happy programmers and difficult operations.
• Logical migration of data is very difficult. Create a large number of logical databases, load balance them over varying number of physical nodes.

There are 1,200,000 photo requests a second in Facebook and scaling takes iteration. They serve 20 billion photos in 4 resolutions =  80 billion photos (which would wrap around the earth more then 10 times!)

• They use cachr: cache the high volume smaller images to offload the main storage systems, and only cache 300 million images in 3 resolutions. Then disribute these through a CDN to reduce network latency

There are 400 million unique home pages and 50 million operations per second in Facebook. They have a love-hate relationship with memcache; it is easy to corrupt and has a limited data model. But it is simply crucial and it does what it does, really good.

At the end of the talk, I asked to Agarwal about their operating system choice and he told me they are probably  going to use Centos.

One of the most interesting talks was Building Skype. Learnings from almost five years as a Skype Architect by Andres Kütt (architect of Skype). First, some stats:

• There are about 650 employees at Skype (which makes 800k users per employee)
• 27.2 billion minutes of Skype to Skype calling per quarter.
• 210k minutes of calls each minute (71k contains video)

Points Kütt made:

• Rules of thumb does not apply: It is always tempting to use patterns that have worked previously but they should be used as a starting point for discussion – not as a solution.
• Functional architecture is important. You neglect how the functionality of your system is organized at your own peril.
• Simply things work. The simplier things are the more intelligent they are.
• Buzz words are dangerous: They are both dangerous as carriers of meaningless chance but also as a catalyst for breaking down relationships.
• Architecture needs to fit your organization. There’s no such thing as a beautiful system design. The design either fits what your organization needs or it doesn’t.

Dan Ingalls keynote was very entertaining. He showed his early codes and he made all the demonstration in Squeak and also shared demonstrations of lively kernel. One wise quote from Ingalls talk:

We’re bad at learning the lessons from the past because:

I have a lot more notes in my Moleskine but I need to take some time to transfer them into the blog.

I also had the chance to visit the gorgeous O’Reilly stand and buy some books (I even have Erlang Programming and 97 Things Every Project Manager Should Know signed by the authors!)

I had a great time and I look forward to being back the next time. Thanks to Google, again!

In my last post, I wrote that I am going to work on Machine Learning for my master’s thesis. I am coding an interactive machine learning framework which enables users to run basic/advanced machine learning algorithms online.

In fact, component based frameworks for collecting together data input/output, pre-processing, classification, clustering, regression and visualization schemes and alike have been implemented before in various languages, for use on different platforms, and operated on a variety of data formats. But unfortunately, due to platform depended solutions, it is difficult to try out and compare different machine learning algorithms quickly and easily.

Hopefully, with ML-LAB will provide a sophisticated and easy-to-use wireable interface for creating the workflow. You can upload a dataset, and put a classification algorithm (currently supports K-NN, Naive Bayes and ID3) after it, then wire it to a dimensionality reduction algorithm (PCA, LDA or Isomap), and if you want to, you can wire the results to another algorithm, … It has no connection limits, you can create a workflow with a hundred connections for a single dataset.

The collection of machine learning algorithms are purely implemented in Python and Django is used for interface and matplotlib for the graphics. I’m sharing some screenshots of it, you’ll notice it looks like Yahoo! Pipes a lot. Hopefully, it will be online at www.ml-lab.com after the core library finished.

You can follow ML-Lab on twitter! http://twitter.com/ml_lab