The Telia Dashboard for Dreamhack started to feel a bit dated so Hannes and I started thinking of doing a new dashboard, but in a different way. We brainstormed all the possible metrics at DH that we could visualize, everything from traffic in the network to how many cans of soda had been sold and the distribution between sexes regarding shower-count.

Since we din't have a lot of time we had to cut down on what data we were to collect, we ended up with seven (expanded to eight during the last event).

The accumulated traffic amount in bytes (or rather terrabytes) from the event start.

The current throughput through the Procera PREs, updated every second to get live graphs. It peaked at around 20Gbit/s usage of the availible 40Gbit/s

The number of unique IP-adresses seen during the event, using the Procera PIC statistics.

The number of connections per game, classified by the Procera PRE. We use the Python API to connect to the PRE every second and get the number of connections.

The accumulated time for all games, this ended up near 4000 days. This is a feature of the Procera PIC.

This uses the Twitch-api and a list of official streams to periodically connect and retrieve the viewer count of each stream and summarize them.

Thanks to STKJ and Phivil we got SNMP access to the Cisco WIFI gear and could get accuarate client count every second.

After a request from the Dreamhack Game guys we created a special rule in the Procera PRE to count the number of visitors to DHGames website and publish this.

For the visuals we decided early that we wanted something flashy using WebGL, it did't take long until we settled on a GL background and cards in the foreground that flipped horizontally when switching between the content of the cards. For the background we started with falling cubes with a nice DOF falloff. But it didn't really work, so we tried out other thing and finally settled on the wavy background that is in the final version.

During the event we came up with some ideas for future improvements. One problem is that some of the numbers are interpretations of reality, like Connected devices, so for those items we'll investigate other (better?) metrics to use to represent that data. Also the most requested feature was more linegraphs, so we'll look into how we can incorporate that into the current design.

I did get the opportunity to play with data from the PacketLogic machines that was deployed at DreamHack summer 2011. Basically we parsed a log containing a subset of all connections, extracted the outside IP (independent from if it was source or destination) and ran that through MaxMinds GeoIP API to get lon/lat. The data was mangled a bit and then fed to a website with an WebGL globe that showed the amount of connections as bars on the globe. You can see the globe "in action" in this presentation made by Karl 'IcE^' Anderssson, the teamleader for the Network-team at Dreamhack.

The globe was shown to Telia who provided the bandwidth for the event and they were interested in a more advanced version for the next event. I knew that there would be PacketLogics at the next event also so i didn't have to change much code in the backend and concentrated on figure out what would be interesting for Telia and the participants at the event and also would look cool. Since this was done on my spare time it took a couple of weeks in Photoshop before i had something that I was satisfied, then the next step, creating all the shiny meters in HTML5.

Quite a few hours of adventures in the world of JavaScript i hade a working frontend and realized that i needed a couple more scripts in the back to feed it with data. Not having my own PacketLogic to code against was a bit of a problem but the documentation is quite good and I got example snippets from a friend that works at Procera. Those scripts worked fine with some small tweaks at the start of DHW11. The kind Virus84 was kind enough to lend me three blades in the blade server he had at DreamHack. The servers had enough power to handle both the geolocation decoding of about 80k IP/sec and the massive flood of visitors, after the three day event there had been more than 120,000 visitors. One problem was disk space, I had underestimated the amount of connections and overestimated how much disk there would be, but a quick update and small bash script made sure that no old cached data remained.

The dashboard contains six different panels. In the first row we have: a WebGL globe with a heat map counting connections, a VU-meter counting unique hosts on the network and a display of the total amunt of data passing out and in from the network, and to the right a speedometer displaying in- and out-speed in real time. At the left in the bottom row there's a graph showing traffic over time for the whole event, the middle panel show the same for the last 5 minutes and was updated in real time, and to the right there was a graph showing the percentage of top 8 games played.

For DreamHack Summer 2012 some small changes of the graphic was done in the frontend but the big changes was in the backend. Since i didn't know what kind of hardware the dashboard could run on I spent some time optimizing the slowest parts which was the geolocation and image rendering. For the first attempt the geo-data was stored in a MySQL database and every 15 minutes the amount of connections during that period was summed up and extracted for the heatmap-like globe. Unfortunately the database didn't really handle the data gracefully and the SELECT took longer and longer the more data that was put in, so every other hour all rows was deleted and the DB optimized.

For DHS12 I decided to use flat files instead, the script that did the geolocation work was optimized and changed to hold the data in memory until it dumped it out into a text file every 15 minutes. The text file was then used by the script generating the heat map. The changes and optimizations had quite a large impact on performance, during DHS12 I had a couple of dualcore Opteron machines and one of these machines managed to quadruple the performance of the quad Xeons I used last event.

Telia, who also sponsor a internet café in Stockholm wanted a "Telia branded" version of the dashboard to run at the café so me and IcE started work on it. Since the café don't have any of the cool Procera equipment that's available at DreamHack IcE started working on a completely new backend and i worked on teliafying the frontend.

The Telia-version of the dashboard

Telia wanted to deploy the dashboard in two vertical TVs mounted at the café, so at the last minute I had to rotate everything to fit the screens.

The final result on the opening

What changes we will see for DHW12 is a secret so far, but there will be some changes anyway so keep an eye on http://dashboard.dreamhack.se/.

Just before DreamHack Summer 2011 Hannes "Hanes" Rahm and I sat down and talked about that it would be cool to actually submit something for the Wild-competition at DH which only was two weeks away. After a couple of hours brainstorm and research on Internet we had an idea, and Illusions is the result.

This was the first Wild/Animation entry released at DreamHack by me and Hanes. Released under the group Zink-Interactive who by placing first both in 64-bit demo and combined demo previously aren't newcomers at DreamHack. After two days work we had a lot of footage to cut together and sync with the soundtrack made by Hanes. At the end we finished at second place, with Raddox on the first place with remix of Skurt, not too bad for a first try.

Unfortunately wasn't I able to be at DreamHack when the video was shown or at the prize ceremony, but thanks to the wonderful production I could sit in a hotel room in Malmö and watch it live.

View the video at Youtube

Try figure out how the effects are made before you check out the Behind The Scenes post

After having used the old dashboard a couple of events Telia contacted me if I could create a new one, and that I could. This time I got my friend from previous adventures on this blog, Mr Hannes. His master knowlege about design and UX combined with my backend expertese led us to a novel setup.

Insted of only using one screen we wanted to give the dashboard more impact and show more stuff larger. After coming up with a couple of new ideas we finally went with splitting the dashboard into several screens. Since the there was four racks for the servers, and the width of a rack was almost exacly the same as the hight of the TV we used, we rotated them 90 degrees. The dashboards we created needed to be usable in horizontal mode also, since most (all?) of the people who viewed them via the web had horizontal screens.

With all the extra space we decided that we could afford to use one of the screens for branding only. So the left-most screen contained no data, only a clear Telia branding with an image and slogan they use elsewhere. On the other three screens we show the number of devices on the network, the current throughput from the three main halls and the accumalitive data volume as a stack of CDs compared to different well known buildings from around the world.

Steer your browser towards https://dhdashboard.se/ during an event and you'll be able to see the dashboard in person.

After the success with Illusions we started thinking of a new project, we wanted something more like a short film with a script and characters. It took several lunches before we came to the conclusion that we needed small animals that were easy to control, and didn't run away as soon as we started to film. The only animal that fitted that and that we could get hold of was snails.

So, off to my girlfriends parents house and to the raspberry bushes. Under the leaves there were hundreds of snails, in all sizes. We collected some of them in a can. Then we drove down to a meadow close by to get some material for the set, we figured we wanted two different settings. We built the set at Hanes place, and started to film a documentary in Attenborough style. The story was something like that the snails was a bit like elephants, and when they were old and weak the traveled to secret grounds to die.

The two sets was first a jungle-set with a lot of moist vegetation and a desert set with sand and dry stuff. The snails started in the jungle set and started their travel towards the hole iron wheel in the desert. By minimal modifications of the set and different angles between shots we managed to get quite a few shots out of the small sets

One negative side of using real material for the sets were all the tiny bugs that lived among the dirt and grass, some ruined the shots by running by and others wandered of on the floor. At the end of a days shooting we had all the shots we needed and carried it all outside in buckets and cleaned up.

A couple of days later I got a message on IRC from hanes, the disk where all the footage was stored had crashed, in a bad way. After finding some deep level data recovery software and running it he was able to save about half of the stuff, but the disappointment of losing soo much footage dampened our lust for working on the project and it was never finished.

There're was quite a lot questions about how the effects in Illusions was created. Some guessed that we created a new flow-dynamics algorithm others had even stranger ideas. So here's a quick post about the process that finally led us to the effects in the video. First, it's all real fluids that anyone has in the fridge, into ordinary cold water.

It all started with that we (that's me and Hannes) saw some videos of ink in water (quite far from the result we finally got) on Youtube and thought that we might be able to do something cool with that. We created a small "studio" setup in my kitchen, the main parts was a 200W light in a shoebox, a cake-plate covered in aluminium foil with a 5cm hole in the center and a plastic container from IKEA (hey, we're swedes!). Later we had to add a fan to cool down everything since the lamp gets quite warm.

With the setup ready and the water tap close by we started to add stuff while filming it. We started out with different food-colour liquids (bad), soy (bad), oil (bad), mixtures of the previous (worse!), it was mostly a failure. Then we reversed colours of the setup, instead of dark on white we removed the white background and replaced it with a black cloth. Now we started with milk at it was perfect, behaved quite close to what we hoped and as time progressed we got more and more amazing shots. To get a little variety we tried different temperatures, cold in cold, hot in cold etc. and it changed how fast and how much the milk dissolved in the water. To dispence the milk we ended up using syringes. We had three different sizes and for some of the clips we used no more then 1-2ml of milk was used, and despite that we used up quite a lot milk.

With all kind of hot and cold milk tested we ran across the street and bought milk with different fat contents and also cream. The more fat it contain the more 'thready' the result was, also it took longer for cream to dissolve. So after an hour or two of failures we continued pouring lactose-products into water for several hours, before we had enough clips to be satisfied.

Hannes wrote some music and we sat at his place to cut it into something presentable. It took far longer than we expected before the result was presentable, and sometime after midnight we decided to cut it from the planned 3 minutes down to 2:30 minutes. All and all we're very happy with the result, and it got quite a lot of attention around the world. It might even get us into a music video, more about that in the future.

You can find more and other information about this at the official Zink-Interactive blog.