system 2 theories #21
Comments
|
one suggestion is that system 2 is repeated intuitive responses that are evaluated for fit and modified, until something good comes up. |
|
note it has also been suggested there might be a system 3. it's not mainstream. https://www.moneyonthemind.org/post/on-the-hunt-for-system-3-is-it-real |
|
with regard to executive control, it's in general thought of as frontal parts of the brain biasing activity to be goal directed. the casscade theory is prominent and has been around since quite a while. this works via prompt chains with LLMs, too. |
|
The functional anatomy of cognitive control: A domain-general brain network for uncertainty processing https://sci-hub.usualwant.com/10.1002/cne.24804 Cognitive Control as a Multivariate Optimization Problem https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8939373/ The role of PFC networks in cognitive control and executive function https://www.nature.com/articles/s41386-021-01152-w A middle ground where executive control meets semantics: the neural substrates of semantic control are topographically sandwiched between the multiple-demand and default-mode systems https://academic.oup.com/cercor/article/33/8/4512/6706757?login=false Neurokognitive Grundlagen des kreativen Denkens
https://www.youtube.com/watch?v=dM_hambWQCk
https://www.psychologytoday.com/us/blog/experimentations/201802/your-brain-creativity There is also that concept called semantic control, seems important to me for logical reasoning. Semantic cognition uses executive semantic control and hub-and-spoke semantic representation systems https://www.sciencedirect.com/science/article/pii/S001094521830073X Creativity in verbal associations is linked to semantic control https://academic.oup.com/cercor/article/33/9/5135/6759328?login=false A Tri-network Model of Human Semantic Processing
https://www.researchgate.net/publication/319645677_A_Tri-network_Model_of_Human_Semantic_Processing Another model on attention and executive control is described in this research, using dynamical systems theory https://www.youtube.com/watch?v=19ZqeQzXVV4 https://direct.mit.edu/netn/article/6/4/960/109066/It-s-about-time-Linking-dynamical-systems-with |
|
another thing is that reason is likely dynamically learning new things, which is not just what an iteratively prompted LLM does. I think system 2 is actively learning new things on the fly. the active inference paper I've tagged you in yesterday adds to this picture that it might be the process of that higher representational system to predict the contents of the lower representational systems which might give rise to the experience of access consciousness, in contrast to the learning to be conscious account that just sees it as learning a representation about those. An active inference model of conscious access https://www.sciencedirect.com/science/article/pii/S2665945X22000092
|
|
I think LLMs have the fundamental limitation that they don't optimize for a 'new cognition' - necessary for system 2 and logical thinking. Human logical reasoning is not infallable either, but I'd guess a good chunk of our higher accuracy as compared to LLMs trained on logic is this active modulation, 'making thought fit to the rule of logic, for new things'. also this research here: Bridging Machine Learning and Logical Reasoning by Abductive Learning |
|
this is a bit confusing, because in GOFAI, logical reasoners are literally template matchers, and they work reliably. |
|
well that's my dynamical cognitivism speaking. it's all a flow. but in computer science, originally one thought there is 'stuff' in the mind. Cognitive Dynamics: Conceptual and Representational Change in Humans and Machines https://annas-archive.org/md5/39fdd62ff2c171c10a5a78aee755ae3c I think it gives a reasonable account on what representations are. I've always felt during my whole psy study that the way the mainstream theory talked about those things was kinda off. |
|
now it's becoming mainstream in cognitive neuroscience, but I remember when they talked very differently about the brain, as if it did literally explicit computations. it works as if it did it (as we see with ANNs), but there are no numbers in our heads. |
|
the simples view on system 2 I get from all those theories is the iterative updating model of working memory, but used in a 'certain modus' to operate as reason. so I think in a way, you could 'abuse' LLMs as true thinking machines, that's what I feel. might not be the most elegant thing to do, but I think if you just want it to think logically as good as we do, one can do 'stuff' to LLMs to achieve that goal, is my hunch. |
|
this paper here is also very important Toward a Control Theory of LLMs https://aman-bhargava.com/ai/2023/12/17/towards-a-control-theory-of-LLMs.html |
|
while LLMs have flaws, I think they do simulate human linguistic thinking good enough to 'get something going' - which you then could call a thinking machine. |
|
it's what my hf post was about, too. there I suggest to use differentiation to improve the accuracy of reasoning. https://huggingface.co/blog/KnutJaegersberg/active-reasoning |
|
Rational rationalization and System 2
https://hal.science/hal-03025339/file/commentary%20cushman%20postprint.pdf |
|
The Elusive Notion of “Argument Quality” https://www.wellesu.com/10.1007/s10503-017-9442-x
|
|
Towards a metacognitive dual process theory of conditional reasoning (important chapter) https://annas-archive.org/md5/db1cc71fe7ddd30c2abb8a6c93bc0339 |
|
metacognition effects on network structure (if integrated in training) it's part of system 2, apparently an impactful part
|
|
old but prominent paper on working memory Making Working Memory Work: A Computational Model of
|
|
hmm that paper by Schmidhuber is of cause also relevant: learning to think On Learning to Think: Algorithmic Information Theory for Novel Combinations of Reinforcement Learning Controllers and Recurrent Neural World Models in the video he calls it a kind of prompt writing model that is learned, to retrieve data that is useful for solving the kinda problems at hand |
|
so you have a learned way to 'populate' working memory with useful contents for whatever system 2 is doing / pursuing |
|
I think that was about retrieving the right abstractions according to Schmidhuber in the video - brilliant! |
|
instead of simulating every small step, such approach would help to learn to grab the right chunks for the task at hand, it's for a faster way of achieving a goal as long as the world model is somehow useful |
|
this is not accepted theory, but there are also different suggestions of a system 3. of cause that is also just used for self promotion, but there is something to it.
While I doubt if that necessiates naming a system on it's own, the account from wise decision making that mentions a balance is something which I think is interesting. |
|
wise decision making sounds like something the whole should optimize for, using whatever helps. |
|
I forgot one:
|
|
what I was looking for was something to balance or tweak "mind" like a metacognitive objective. |
|
of cause one can also select other things to optimize for. like intelligence itself, or say, wit. but those targets that make the agent operate in harmony with its environment will likely all converge to wisdom. |
|
better give it some extra wit, Gandalf was not dumb either. To compensate for not wise smart agents. |
|
|
|
Multi-Task Brain Network Reconfiguration is Inversely Associated with Human Intelligence https://www.biorxiv.org/content/10.1101/2021.07.31.454563v2.full
|
|
Integrated Intelligence from Distributed Brain Activity https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(20)30169-8
|
|
this notion of mixed selectivity and efficient, orthogonal task representations working together / filtering simultaneously mattering for gf again reminds me of Resers model. it's as if you only have to populate working memory in a smart way and then let it do it's thing. |
|
referring to active working memory here, using effort and executive control to populate it. but then again that other paper showed that in intelligent people there is not a big difference between task and resting state for the brain activity involved. it's a kinda zen on focussed attention. letting it happen, not stiffing the focus in a cramping way. |
|
Reser is of cause kinda short term, so one needs constant long term working memory utilization to stay in touch with 'bigger' goals - as intelligence is not lost in the immediate https://www.jneurosci.org/content/42/38/7276
|
|
not the same as a task list, more like aspirations, because it's not possible for intelligent people to plan everything happening to them |
|
|
|
this reinforces my view that all reasoning can be understood as a kind of categorization, as a kind of pigeonholing. when done right, the right pigeon goes into the right hole. which is also why GPT is not entirely off. it's lacking recursion, though. |
|
it's important to note that the category can be made up, on the fly. that's where gpt lacks. you need to place the attractor in semantics, not in a fixed distribution. there are different neural network architectures for this, but I bet one can tinker something for LLMs, which, with a little RAG help, will work at human level. one ought to endow the emerging thought with a kind of p-consciousness. that's not achieved with automated prompt engineering. each p-consciousness, each thought is original. |
|
Hot and cold executive functions in the brain: A prefrontal-cingular network it's mixed, as I said.
https://journals.sagepub.com/doi/pdf/10.1177/23982128211007769 |
there is interesting literature from psy/cognitive science how system 2 might work. it's not describing thorough cognitive architectures, but is relevant nonetheless. I'll grab some and drop them here.
I think it is in general a good perspective to think of system 2 as build on top of system 1, though involving it's own neurocircuitry. I'd presume symbolic reasoning and logic emerge from association and biased / steered usage of system 1 components. Executive functions and attention are used to control neural activity, giving rise to reason. but it's not an entirely independent system. Reason is intuition doing things to itself.
The Pros and Cons of Identifying Critical Thinking with System 2 Processing
https://philpapers.org/rec/BONTPA-3
Three stages of system 2
Analytic Thinking (Type 2 or “System 2”) for Large Language Models: using Psychology to address hallucination and reliability issues
https://osf.io/preprints/psyarxiv/n7pa4
gotta dig more
The text was updated successfully, but these errors were encountered: