Artificial intelligence can exhibit intelligent behavior through the technology of machine learning.
While this learning process follows procedures developed by humans, it is not yet fully explained why intelligence emerges from these procedures and the structure of artificial intelligence.
In this article, by contemplating the essence of learning, I aim to explore the reasons for the emergence of intelligence.
As we delve deeper into the concept of learning, we arrive at the idea that both artificial intelligence and our brains possess an inherent nature to learn how to learn.
This suggests the existence of a mechanism that can be called a Natural Born Frameworker.
Learning through the Body and Learning through Language
We come to understand the world around us and expand our capabilities by observing objects with our eyes and moving our bodies.
This is also a form of learning, which can be called learning through the body.
On the other hand, when we generally think of learning, we might imagine increasing our knowledge by reading textbooks or listening to a teacher's explanations.
Beyond learning based on such educational curricula, we also acquire diverse knowledge from conversations with friends, online news, and other sources.
This type of learning is not about visually memorizing images or learning through physical movement, but rather learning through language.
Metacognitive Learning and Metaphysical Learning
Within language-based learning, there are cases where knowledge requires repeated iteration to be memorized, and cases where it can be learned after just one or a few exposures.
Alternatively, some knowledge can be utilized by retrieving its details from a bookshelf or the internet when needed, even if not fully memorized.
In the sense of acquiring and appropriately utilizing knowledge when required, both of these patterns can be considered learning.
Among these, knowledge that cannot be memorized without repeated iteration can be called metacognitive knowledge. The process of learning the concept itself is metacognitive learning.
This is similar to physical learning, where repetition is involved in seeing objects with our eyes or moving our bodies. These can also be classified as metacognitive learning.
Conversely, acquiring knowledge that can be memorized with few trials or used by looking it up on the spot can be called metaphysical learning.
In this case, pre-learned concepts acquired through metacognitive learning can be utilized to learn new knowledge as types of those concepts or as combinations of concepts.
Since concepts already mastered through metacognitive learning can be used, metaphysical learning does not require repetition.
Natural Language Machine Learning
Let's apply this to machine learning in artificial intelligence.
Generally, neural networks used in machine learning perform metacognitive learning, which involves repetitive learning of concepts.
On the other hand, large language models capable of natural language processing similar to humans can perform learning through language.
During the pre-training and fine-tuning of large language models, language-based metacognitive learning takes place.
A trained large language model can then answer by utilizing knowledge contained in the input sentence, which means it is performing immediate metaphysical learning.
This ability for language-based metaphysical learning allows large language models to utilize new knowledge without repetitive learning.
This can be contrasted with traditional numerical machine learning, which iteratively adjusts model parameters, and can be termed natural language machine learning.
Natural Language as the Metaphysical Interface
Natural language is situated at the interface distinguishing metacognitive learning from metaphysical learning.
The interesting aspect of natural language is that it can be acquired through metacognitive learning, and upon it, metaphysical learning becomes possible.
Metaphysical Interfaces Other Than Natural Language
In fact, metacognitive learning and metaphysical learning also exist in physical learning. For example, someone skilled in sports can quickly adapt to a new sport they've never encountered before.
Similarly, a person knowledgeable in biology can immediately understand the characteristics of a new species when they see it.
Thus, even in physical learning, there exists a metaphysical interface that holds a similar position to natural language.
Framework
What lies at these interfaces is a framework distinct from elemental concepts or knowledge; it defines their relationships and structures, and enables new structuring.
As diverse metacognitive knowledge is acquired through metacognitive learning, it is sometimes possible to learn the framework at the metaphysical interface from the connections between these pieces of metacognitive knowledge.
A framework derived from physical learning enables the immediate acquisition of new knowledge through metaphysical learning after its mastery. However, the knowledge gained through such metaphysical learning is not easily communicated to others.
On the other hand, the framework derived from learning through language is natural language itself.
Therefore, knowledge acquired through metaphysical learning by learning the natural language framework can be directly input into another person's language acquisition.
This applies not only to knowledge primarily based on language acquisition, such as textbooks or online news.
An experienced soccer player trying baseball for the first time might be able to articulate the metaphysical knowledge of baseball they acquired, and convey it to other experienced soccer players. This means that if people share the same metacognitive knowledge, they can communicate what are known as "tips" or "tricks" using words.
Furthermore, one could verbally convey knowledge about a new species they observed to other biologists, thereby sharing that knowledge.
Thus, natural language is revealed to be a very powerful framework located at the metaphysical interface.
Virtual Framework
Above natural language, another framework can be acquired.
These include domain-specific frameworks or metaphysical frameworks.
In various academic disciplines, business sectors, and daily life, there are diverse domain-specific frameworks.
Scholars can make new discoveries within their specialized frameworks and easily convey these discoveries as knowledge to other scholars who possess the same framework.
The framework itself can sometimes be expressed in natural language, in which case, individuals or large language models possessing a natural language framework can acquire and understand it.
Business models and cooking recipes are also examples of such domain-specific frameworks that can be expressed in natural language.
Furthermore, mathematical formulas, programming languages, and business analysis frameworks are formal frameworks.
These can also be expressed or explained in natural language.
Such domain-specific frameworks and formal frameworks built upon natural language can be called virtual frameworks.
This can be easily understood by imagining a virtual machine running another OS on a physical computer. Another framework is functioning on top of natural language, which serves as the foundational framework.
Native Framework
Initially, this virtual framework must be understood through natural language, but with practice, it bypasses explanation and understanding via natural language and begins to function directly as a metaphysical interface framework built upon metacognitive knowledge.
This can be called a native framework.
Natural language is, in a sense, a native framework, but only in the case of one's mother tongue. Generally, languages other than one's mother tongue are acquired as virtual frameworks. As proficiency increases, they approach the status of a native framework.
The same applies to domain-specific frameworks and formal frameworks. Mathematicians can communicate natively with each other using mathematical formulas, and programmers can understand each other's intentions solely through source code without comments.
This suggests that the transition from a virtual framework to a native framework can also be applied to large language models.
The idea of detecting frequently used virtual frameworks, generating a large amount of example data using those virtual frameworks, and then fine-tuning them to become native frameworks would be worth trying immediately.
Natural Born Frameworker
Considering this, we realize that large language models might be learning these specialized and formal frameworks not only during fine-tuning but also during pre-training.
Furthermore, in that process, it's plausible that they don't learn specialized or formal frameworks natively from the outset. Instead, they first learn the natural language framework, and then, during or after achieving proficiency in it, they learn specialized or formal frameworks and assimilate them into native frameworks.
Deepening this idea of incremental framework learning, it's also conceivable that natural language learning itself is a parallel pipeline of highly granular, incremental framework learning.
That is, from the vast amount of text provided as learning data during pre-training, large language models might be learning not just individual concepts, but also some very simple rules of natural language as frameworks. Then, using these simple frameworks as a foundation, they might repeatedly learn slightly more complex rules.
In this way, starting from the stage of learning individual word concepts, they should be able to acquire compound words and basic grammar, then understand sentences, and eventually learn complex elements such as literary techniques and expressive styles.
This can be understood as a model of layered and composite framework learning, where one framework serves as the foundation for learning the next.
This highlights the image of large language models as Natural Born Frameworkers, inherently possessing the mechanism to learn frameworks from the beginning.
Attention Mechanism
The technology that actualizes the Natural Born Frameworker is the attention mechanism.
The attention mechanism is akin to selecting tokens that should be focused on within a context. It clarifies the relationships between tokens. This is precisely the nature of a framework itself: abstracting by retaining important concepts while clarifying the relationships between those concepts.
By switching this selection for each token, it becomes possible to dynamically switch frameworks as well.
This allows us to explain why the attention mechanism is a decisive technology for the evolution of large language models, using the model of the Natural Born Frameworker.
Conclusion
If this mechanism is indeed occurring during the pre-training process of large language models, then the previously enigmatic mechanism of these models becomes explainable.
This explanation encompasses the metacognitive and metaphysical learning we've discussed, the framework as a metaphysical interface, natural language enabling language acquisition and virtual frameworks, and the attention mechanism that realizes the Natural Born Frameworker.
Furthermore, two additional implications arise from this.
First, natural language possesses a highly suitable structure for incrementally developing complex frameworks from simple ones into native frameworks.
If natural language initially emerged in a simple form within human societies and gradually evolved to possess a more complex and rich structure, then this is a natural consequence.
Moreover, a structure that allows for rapid learning would be advantageous. Assuming multiple societies with various natural languages competed, the hypothesis that the natural language most suited for learning has survived to the present day is easily established.
Reflecting on the nature of natural language leads to the second implication: that we humans are also Natural Born Frameworkers.
Even if the specific foundations and mechanisms differ, our brains must also be equipped with a system, similar to the attention mechanism, that incrementally learns and flexibly modifies frameworks.