I think it works like this. Your mind watches the world. Through input stimuli. Including sight, sound, touch, and your unattended observation device called the third eye or intuition.
The brain, based on fight or flight, always aware of possible danger, instructs the body to perform actions, by sending signals to ligaments to move arms and legs.
As the action occurs, the observational aspect watches and determines next course of action, based on results of prior action.
Because of the overload of stimuli, the mind has the ability to condense information packets, by storing past events and how objects relate to things. Basically like a fast retrieval mechanism where it doesn't have to process every bit of information, it sees things, determines if threat, then performs action, waits for feedback and then begins again.
So in teaching computers how to exist in modern world, it has to learn the same process. In other words, how does it learn "common sense".
Some very smart people are working on possible solutions. One I read about yesterday, the advanced AI team at Facebook, is combining dual neural networks to mimic our method of learning.
One Neural net will be given a task, and it will predict the outcome. The other Neural net is aware of the event to happen, it knows what the other Neural net thinks will happen and then the action occurs.
The opposing net see the results, compares to what it think should have happened and updates its model appropriately. It learns what should happen and what actually happened.
It seems like a good approach, sort of like mirrors. I wonder what would happened if there were 10 Neural Nets watching the event. Like a community learning session. Like a deep learning observational network.
That way, you could throw in more variable instead of simple tasks. Because a limited single domain expert can not survive in the world. You need a network of Neural Nets with multiple domains integrated and working in real time, always observing, predicting, watching and updating.
Seems like a good hypothesis to test. I don't do this stuff for a living, but its interesting to keep up with the latest developments in the hottest field going today.
And there you have it~!