Sensorimotor

One thing I initially missed about Thousand Brains Project is, when I heard it’s a “sensorimotor” framework, I started thinking about robots, but it really is talking more generally about how “locomotion”/movement is crucial to learning and intelligence.

Current AI agrees with TBP on:
1) Features: stable patterns that are shared between many inputs

Then there is:
2) Features at location (pose):
This is not considered. Current AI combines features to recreate the input but it doesn’t consider location of features in space.. at least not explicitly.

This makes it difficult to truly detect features actually! Location, including distance to sensor, is crucial for example to detect them invariantly.

3) Morphology (aka object):
The stable spatial and temporal relationship of “feature at pose” pairs to each other. This is the foundation of “objects”.

Current AI has no equivalent for this. It can’t begin to model this without 2.

Btw, objects could be physical or abstract!

And then there is
4) how object changes given various contexts (on its own, when pushed, dropped etc)

This is the movement/locomotion part. It is not enough to feed static images to AI. Video/movement/change is critical and an inseparable part of the training. Not only that but ideally it needs to be able to interact with the objects it’s being trained on as well!

Andrej Karpathy was recently talking about how we should perhaps feed text as images and combine the two modalities!

Now you can begin to see why the current AI architecture is nowhere near AGI even though it does have some elements of it.