Project day with Silverpond

Last Thursday I spent a day with the very talented folks at Silverpond (@SilverpondDev). It was a great opportunity to find out more about the consultancy and to learn more about the projects they work on. Since we have overlapping interests, the idea was to have a 'project day' where we would work together on some sort of demo project or tackle a small part of a much larger project. 

I was definitely thrown in the deep end of the deep-learning pool (pardon the pun) and while we didn't really get stuck into the project like we'd hoped, we had some really good conversations. I definitely gave me a much better sense of what they do and how they operate. These guys really know their stuff, and in terms of putting deep learning theory into practice, I was in way over my head,  but in a good way. Needless to say my 'to learn' list just got longer.

Approaching deep learning when you come from a solid data analysis background is really challenging, especially when you normally approach a problem with a clear scientific question to answer, and where the problem is solved by fitting models derived [sort of... ] from theoretical or empirical observations or correlations that have a real physical basis. Research that is driven mainly by the data rather than the model used to infer what's going on.

With deep learning, you almost have to unlearn the way you approach a problem and just accept the black box nature of things and that you can't necessarily see under the hood. You also need to think conceptually. Deep learning processes are surprisingly abstract and there are a lot of things you need to get your head around, for example, why batch size is important, how to determine appropriate weights, gradient descent optimisation, how to implement loss functions. At times, it feels like there are various "fudge factors" that exist just because they work... or perhaps that's my lack of deep learning knowledge creeping in. 

Noon's idea was to implement CPPN (Compositional Pattern-Producing Networks) using deeplearn.js and Silverpond's team photos, to create transitions between people based on their CIFAR-10 vectors. His idea came from a series of blog posts where a CPPN network trained on CIFAR-10's frog class in order to generate images of ANY resolution. For reasons I won't go into, this is a really useful thing to be able to do. The images from the blog post were also really striking so that was a good enough reason for me. It seems computer generated abstract art is a thing. 

   Ground Terrain (2016)

Ground Terrain (2016)

   Amphibian (2016)

Amphibian (2016)

In astronomy a similar motivation would be performing high resolution analyses on lower resolution spectra, e.g. if you're trying to extract velocity dispersions at or below the resolution limit of your spectrograph. 

Anyway, it was a really great idea, but I think we totally underestimated the time it would take for me to get up to speed with what had previously been done. To complicate things just a little more (heck, why not!?), the idea was to implement the guts of the process –  the training/optimisation part that was yet to be tackled – using deeplearn.js (despite my lack of knowledge of javascript).  I was keen to see how this might work. deeplearn.js is an open-source library that brings performant machine learning building blocks to the web, allowing you to train neural networks in a browser or run pre-trained models in inference mode. It looks really useful and there a whole swag of reasons why you might want to use it.  Needless to say my enthusiasm for tackling something challenging was thwarted by my ability to tackle said problem, or rather my ability to tackle it in one day. In all fairness, I was also thwarted by my Python3 Anaconda installation, which also meant wasting a bit of time creating a separate python environment and installing various packages – something that although useful, I am loathe to do. At some point over the next few weeks, I'll have another stab at it. Just for fun. 

The rest of the time was spent talking about various things; Planet Labs imaging, Bayesian clustering, other Silverpond projects, and the potential for an AI Accelerator. To me this is one of their most exciting ideas. From what I've seen they are in a really good position to be the go to people for deep learning internships in Melbourne. They have the expertise, they appear to be at the centre of a fast growing community and if they were to establish the right connections and networks, this could really take off.

I also had a good chat with another Silverponder who I recognised, but couldn't place, or put a name to. Turns out he was one of the students from last year's Swinburne Space Apps Hack team, and he had also taken Virginia's undergraduate astronomy course. After graduating and finishing a three month internship with Silverpond, he is now is working on a Unity application that simulates drone imaging and allows you to create labelled data in a #D gaming environment. 

In between meetings and other work, Lyndon also gave me a short demo of, a tool Silverpond is developing that allows you to upload data and tag or label objects of interest. Behind Silverbrane is a deep learning algorithm that learns to identify objects as more data is provided. I also had a good chat with Jonathan Chang, their managing director. 

All in all it was a good day, even if the project side of things didn't progress as well as I'd hoped.