Author
27 posts
Posts by Gideon Kowadlo.
We are very happy to announce that we’ve been awarded an Artificial Intelligence for Decision Making Initiative grant from Defence Science Institute on Effective updating of deep learning models with limited new data. More specifically, the grant is for the Continual Few-Shot Learning project that we blogged about recently.
We were really excited to present a couple of papers recently at IJCNN, the International Joint Conference on Neural Networks. One of those papers was an extension to the work that we previously published on AHA, Artificial Hippocampal Algorithm based on Complementary Learning Systems by O’Reilly and McClelland (see that paper summary blog post here).…
Executive Control is core to what most people recognise as true intelligence. For example, the ability to attend to relevant cues and maintain task dependent information whilst ignoring distracting details and taking appropriate actions. Working Memory (WM) is a core component of Executive Control. “Working memory is a short-term repository for task-relevant information that is…
We’re very happy to report that we recently published a preprint on AHA, an ‘Artificial Hippocampal Algorithm’ for Episodic Machine Learning. It’s the culmination of a multi-year research project and is a starting point for the next wave of developments. This article describes the motivation for developing AHA and a brief description of the research.…
The standard definitions of Episodic and Semantic Memory hide some important subtleties that impact the development of Machine Learning algorithms for AI (particularly those building Episodic Learning capability). This article explores this issue and provides a context for upcoming articles work to take us a step closer to ‘animal-like’ machine learning
We recently published 2 new ML/neuroscience research projects as part of the Request for Research (RFRs) projects, with WBAI. They’re fascinating topics that have arisen through the relationship with our advisor Elkhonon Goldberg from the Luria Neuroscience Institute.
Over the last few years, there have been several breakthroughs and exciting new research directions in Reinforcement Learning, Hippocampus Inspired Architectures, Attention and Few-Shot Learning. There has been a move towards multi-component, heterogeneous, stateful architectures, many guided by ideas from cognitive sciences. Google DeepMind and Google Brain are leading the way making progress on several…
The dataset is an integral part of an ML engineer’s toolkit. We recently compiled useful information about a range of these well known datasets. It’s all in one place, and hopefully useful to others as well.
ML Today Today’s Machine Learning has demonstrated unprecedented performance in what seems like every application thrown at it. Almost all the success has been based on advanced memory systems that can learn to recognise an input based on a large number of training examples. This is the equivalent to memory for facts. The biological terminology…
2018 is a fresh new year and an exciting milestone for Project AGI. Dave and I have been discussing, dreaming, playing around with and striving towards general purpose AI for over 6 years. It started with musings on the algorithmic underpinnings of consciousness and the nature of intelligence. We quickly moved into the more practical…
Reproduced left to right from [8,10,1] Within a 5 day span in October, 4 papers came out that take a significantly different approach to AI hierarchical networks. They are all inspired by biological principles to varying degrees. It’s exciting to see different ways of thinking. Particularly at a time when there is growing skepticism…
The standard machine learning approach is to learn to accomplish a specific task with an associated dataset. A model is trained using the dataset and is only able to perform that one task. This is in stark contrast to animals which continue to learn throughout life and accumulate and re-purpose knowledge and skills. The…
We’re very excited to launch AGI Experimental Framework, AGIEF, our open source framework. We first introduced it a while back, at the end of 2015 here, and it has certainly come a long way. AGIEF was created to make running rigorous AI experiments convenient, reproducible and scalable. The goals are: Repeatability: ability to save/load, stop/start…
In our most recent post we discussed the current set of experiments that we are conducting, using the MNIST dataset. We’ve also been looking at the NIST dataset which is similar, but extends to handwritten letters (as well as digits). These are extremely popular datasets and freely available, so make a great choice for testing…
By Gideon Kowadlo I’m reading a fascinating book called The Righteous Mind, by Jonathan Haidt. It’s one of those reads that can fundamentally shift the way that you see the world. In this case, the human world, everyone around you, and yourself. A central idea of the book is that our behaviour is mainly dictated…
By Gideon Kowadlo and David Rawlinson Introduction We’ve been building and testing AGI algorithms for the last few years. As the systems become more complex, we have found it ever more difficult to run meaningful experiments. To summarise, the main challenges are: testing a version of the algorithm repeatedly and over some range of parameters…
By Gideon Kowadlo and David Rawlinson In our last blog post, we discussed the repeating functional columnar structure of the neocortex, and the inconsistent terminology used to discuss it throughout the literature. As mentioned in that post, the function of the column is an important concept for understanding the function of the neocortex, and as a consequence,…
Cell-type-specific 3D reconstruction of five neighboring barrel columns in rat vibrissal cortex (credit: Marcel Oberlaender et al.) It is well established that there is a functional columnar structure repeated throughout the cortex across species (the concept received significant attention after a study by Mountcastle in 1957 [1]; see [2] and [3] for reviews). This suggests…
I found this video interview with Eric Horvitz, the head of Microsoft Research, to be a really interesting window into their work and vision – a large part of which is AI. As Eric states, there is a “growing consensus that the next, if not last, enduring competitive battlefield, among major IT companies will be…
Just a few days ago, Jay posted here about the recent public attention to AI and the potential future dangers that it presents. Discussion on the topic continues, but it’s not all talk. There are at least two institutes focussed on practical steps to mitigate the risks. There is an institute called MIRI that exists…
I recently published a post on the question of classifying ‘no input’ or missing data. There was a comprehensive reply by Fergal Byrne in a thread on the Nupic Theory mailing list. I am re-posting it here (below) as it provides a valuable perspective of the discussion in the context of human cortical function. Jan 24th 2015 – by Fergal Byrne…
I really enjoyed reading this article Why can’t the world’s greatest minds solve the mystery of consciousness? published recently in the Guardian. It’s an engaging re-exploration of the contemporary discourse on the classic mind-body dilemma. One view is that the ‘hard problem’ of consciousness is an illusion. There is no separation between mind and body, and that…
By Gideon Kowadlo, David Rawlinson and Alan Zhang Can you hear silence or see pitch black? Should we classify no input as a valid state or ignore it? To my knowledge, the machine learning and statistics literature mainly regards an absence of input as missing data. There are several ways that it’s handled. It can…
TL;DR: An SDR is a Sparse Distributed Representation, described below SDRs are biologically plausible data structures SDRs have powerful properties SDRs have received a lot of attention recently There are a few really great new resources on the topic: Presentation by Subutai Ahmad of Numenta Older introductory presentation by Jeff Hawkins Excellent summary of their characteristics in…
by Gideon Kowadlo and David Rawlinson Introduction One of the keys to understanding the neocortex as a whole, and the emergence of intelligence, is to understand how the cortical hierarchical levels interconnect. This includes: the physical connections, the meaning of the signals being transmitted, and possibly also the way the signal is encoded. Physical connections: Physical…
by Gideon Kowadlo and David Rawlinson Introduction This article contains some musings on the factors that limit the increase of intelligence as a species. We speculate that ultimately, our level of intelligence is limited by at least two factors, and possibly a third: our own cultural development, physical constraints, and an intelligence threshold. We’ll now…
by Gideon Kowadlo and David Rawlinson In a previous post (section ‘Regions’), we discussed the distinction between cortical layers and levels. The literature often describes connections between cortical layers in one level to cortical layers in another level. We are not aware of a written convention for this purpose and propose one here. The basic form describes…