Category
16 posts
Since the arrival of Chat GPT, I’ve felt that progress in AI has been largely incremental. Of course, even Chat GPT was/is mostly just a scaled up Transformer model, which was introduced by Vaswani et al in 2017. The belief that AI is advancing relatively slowly is a radical and unpopular opinion these days, and it’s been…
Over the last few years, there have been several breakthroughs and exciting new research directions in Reinforcement Learning, Hippocampus Inspired Architectures, Attention and Few-Shot Learning. There has been a move towards multi-component, heterogeneous, stateful architectures, many guided by ideas from cognitive sciences. Google DeepMind and Google Brain are leading the way making progress on several…
Reproduced left to right from [8,10,1] Within a 5 day span in October, 4 papers came out that take a significantly different approach to AI hierarchical networks. They are all inspired by biological principles to varying degrees. It’s exciting to see different ways of thinking. Particularly at a time when there is growing skepticism…
This month’s reading list has two parts: a non-Reinforcement Learning list, and a Reinforcement Learning list. Since our next blog post will be on Reinforcement Learning, readers might like to refer to our RL reading list separately. Non-Reinforcement Learning reading list A Framework for searching for General Artificial Intelligence Authors: Marek Rosa, Jan Feyereisl, The…
1. Neuroscience-inspired Artificial Intelligence Authors: Demis Hassabis, Dharshan Kumaran, Christopher Summerfield, and Matthew Botvinick Type: Review article in Neuron Publication date: 19 July 2017 This paper outlined the contribution of neuroscience to the most recent advances in AI and argued that the study of neural computation in humans and other animals could provide useful (albeit…
Our brain is a source of great inspiration for the development of Artificial General Intelligence. In fact, one of the common views is that any effort in developing human-level AI is almost destined to fail without an intimate understanding of how the brain works. However, we do not understand our brain that well yet. But…
Digit classification error over time in our experiments. The image isn’t very helpful but it’s a hint as to why we’re excited :) Project AGI A few weeks ago we paused the “How to build a General Intelligence” series (part 1, part 2, part 3, part 4). We paused it because the next article in…
A Minecraft API is now available to train your AGIs Our News We are working hard on experiments, and software to run experiments. So this week there is no normal blog post. Instead, here’s an eclectic mix of links we’ve noticed recently. First, AlphaGo continues to make headlines. Of interest to Project AGI is Yann…
What’s AlphaGo? AlphaGo is a system that can play Go at least as well as the best humans. Go was widely cited as the hardest (and only remaining) game at which humans could beat machines, so this is a big deal. AlphaGo has just defeated a top-ranked human expert. AlphaGo Nature paper (Silver et al 2016)…
This week we have a couple of interesting links to share. From our experiments with generative hierarchical models, we claimed that the model produced by feed-forward processing should not have loops. Now we have discovered a paper by Bengio et al titled “Towards biologically plausible deep learning” [1] that supports this claim. The paper looks for…
The artificial neuron model used by Jeff Hawkins and Subutai Ahmad in their new paper (image reproduced from their paper, and cropped). Their neuron model is inspired by the pyramidal cells found in neocortex layers 2/3 and 5. It has been several years since Jeff Hawkins and Numenta published the Cortical Learning Algorithm (CLA) whitepaper.…
An interesting article by Gerard Rinkus comparing the qualities of sparse distributed representation and quantum computing. In effect, he argues that because distributed representations can simultaneously represent multiple states, you get the same effect as a quantum superposition. The article was originally titled “sparse distributed coding via quantum computing” but I think that gets the…
This month’s reading list continues with a subtheme on recurrent neural networks, and in particular Long Short Term Memory (LSTM). First here’s an interesting report on a panel discussion about the future of Deep Learning at the International Conference on Machine Learning (ICML), 2015: http://deeplearning.net/2015/07/13/a-brief-summary-of-the-panel-discussion-at-dl-workshop-icml-2015/ Participants included Yoshua Bengio (University of Montreal), Neil Lawrence (University…
John Lisman, “The Challenge of Understanding the Brain: Where We Stand in 2015“, Neuron, 2015 For many in ML and AI, biological knowledge is focussed on cortex. This paper gives an excellent broad overview of current biological understanding of intelligence. Sebastian Billaudelle and Subutai Ahmad, “Porting HTM Models to the Heidelberg Neuromorphic Computing Platform“, arXiv.org, 2015 Progress in…
If you’re looking for a formal and precise description of the current HTM/CLA algorithm, then this blog [1, 2] by Fergal Byrne may be of interest. Our experience is that while pseudocode is a good way of describing an algorithm, equations can often be more succinct and specific. The CLA whitepaper [3] describes the algorithm…
This post is about a fantastic new paper by Michael R. Ferrier, titled: Toward a Universal Cortical Algorithm: Examining Hierarchical Temporal Memory in Light of Frontal Cortical Function The paper was posted to the NUPIC mailing list and can be found via: http://numenta.org/community-content.html The paper itself is currently hosted at:…