Subject Follow-up: PhD positions this fallBody Hi Nando,I would like to follow up on the first contact my friend Christoph made today in your Machine Learning lecture at Oxford. I am very interested in doing a PhD with you starting this fall, do you have any PhD positions available?My background is a Master and Bachelor in Physics from Cambridge that included three research internships in computational and experimental neuroscience (MIT, Bernstein Center for Computational Neuroscience, Weizmann Institute).Since graduation I have worked 14 months as a quant at Goldman Sachs and have now spent 2.5 years as a Product Manager at Google.Working on the Product side of our speech technology (eg. "Ok Google") I have been exposed to the power of neural networks on a daily basis. However, it has also become clear to me that our knowledge about why the neural nets work is still very vague. A lot of the hyperparameter tweaking that I see being carried out by the researchers in my team seems like it could be greatly accelerated by having a more solid understanding of the mechanisms underlying the operation of deep neural networks.My interest has always been to understand the basis of intelligence. While in the past I pursued this through the study of the brain, I now believe that building better machine learning systems (and understanding them) offers the most likely route to success for the next 20 - 30 years.To a great extent this belief is based on the progress that has been made over the last decade in a number of human recognition tasks (speech, image, handwriting) using deep neural networks.In particular I am interested in finding ways to accelerate the training of neural nets based on mechanisms and constraints found in biological and physical systems. One question that fascinates me is the role of local learning and self-organization in neural networks: Due to the finite amount of DNA coding for the brain, the information content that is ‘hard wired’ into it is fundamentally limited. Yet at the age of 27, I am able to do all sorts of interesting tasks, including the typing of this email.The interesting part is that the brain manages to train a network many (6-8) orders of magnitude larger than anything that has been trained using computers. Due to the biological constraints, this learning needs to happen based on local interactions between the neurons. I conjecture that such biological constraints could improve artificial neural nets in at least two ways.First, I hope that by bridging the gap between biological and artificial networks we can find locally computable properties that can improve the trainability of neural nets. This is similar in spirit to the results from your recent paper ("Predicting Parameters in Deep Learning") that uses the local smoothness of weights in order to find representations of the weights that can be trained more efficiently.Second, it offers an alternative approach to hyperparameter selection. Your paper on Bayesian optimization (“An Entropy Search Portfolio for Bayesian Optimization”) is an example of a systematic approach to hyperparameter selection. There have been evolution inspired approaches toward this problem (eg: “Algorithms for Hyper-Parameter Optimization”) as well. I believe there are other opportunities for taking inspiration from biology:For example, biological neural network architectures are constrained by the fact that they have to be ‘buildable’ through the ‘self-assembly’ toolset available to developmental biology and operate within the biological constraints, eg. long range connections are less likely and fully connected layers typically impossible.I believe that increasing the number of neurons while constraining the search space based on biological considerations can lead to improved performance.I have attached my updated CV for reference, please let me know if you would be available for a quick chat.Thanks,Jakob