For here, I wan arrange wetin di Artificial Learning Intelligence System (ALIS) be, by breakin' down im concept, principles, main design, and how dem go develop am.
Concept
Generative AI wey dey now, especially big language models, dem train am with supervised learning wey dey based on neural network.
As a way of learning, we go call dis neural network learning "innate learning".
ALIS go join "acquired learning" process, wey dey different from innate learning, to make am fit do reasoning wey dey combine both learning processes.
For dis acquired learning, dem go gather di learned knowledge outside di neural network and use am when dem dey reason.
So, di main technical part of ALIS na to extract, store, and select and use knowledge wey dem fit reuse when dem dey reason.
Plus, ALIS no be just one single technology, na system technology wey dey combine innate learning and acquired learning.
Parts of a Learning Intelligence System
ALIS dey work with de idea say both de innate learning wey don dey and de acquired learning wey dem go consider for future, dey follow de same framework of learning and reasoning.
To explain how learning dey work for ALIS, we go define five parts of a learning intelligence system.
De first one na de intelligent processor. Dis one mean a processing system wey dey do reasoning using knowledge and dey extract knowledge for learning.
Large Language Models (LLMs) and some parts of de human brain na good examples of intelligent processors.
De second one na de knowledge store. Dis one mean a place wey dem dey save extracted knowledge and fit comot am when dem need am.
For LLMs, de knowledge store na de parameters of de neural network. For humans, e be like long-term memory for de brain.
De third one na de world. Dis one mean de outside environment as a learning intelligence system, like humans or ALIS, dey see am.
For humans, de world na reality itself. For LLMs, a system wey dey receive output from de LLM and dey give feedback to it fit be like de world.
De fourth one na state memory. Dis one mean an internal temporary memory-like part wey a learning intelligence system dey use when e dey reason.
For LLMs, dis na de memory space wey dem dey use during reasoning, dem dey call am hidden states. For humans, e be like short-term memory.
De fifth one na de framework. Dis one na wetin dem dey call thinking structure. For de language of learning intelligence systems, e mean de rules for choosing de knowledge wey dem need when dem dey reason and a logical state space structure for arranging state memory.
For LLMs, e be de semantic structure of de hidden states, and wetin e get inside na usually not clear and humans no fit understand am. Also, how dem dey choose knowledge dey inside de attention mechanism, wey dey choose which existing tokens to refer to for each token wey dem dey process.
For humans, as we talk before, e na a thinking structure. When dem dey think using a particular framework, dem dey recall a specific set of know-how from long-term memory and load am into short-term memory. Then, de information wey dem dey perceive currently dey arranged according to de thinking framework to understand de situation.
How Learning Intelligence System Dey Work
One learning intelligence system dey operate like dis:
An intelligent processor dey act on di world. Di world, based on dis action, go give results.
Di intelligent processor dey extract knowledge wey dem fit reuse from dis results and save am for di knowledge store.
When e dey act on di world many times (iteratively), di intelligent processor go choose knowledge from di knowledge store and use am to change how e dey act.
Dis na di main way e dey work.
But, for fundamental level, di ways wey dem dey extract, store, select, and use knowledge na im go decide if di system fit learn something meaningful or not.
Humans get ways wey dem dey effectively handle dis knowledge extraction, storage, selection, and utilization, wey make dem fit learn.
Neural networks, including LLMs, dem get external teachers wey dey handle how dem extract, but dem get ways for storage, selection, and utilization. Dis na im make dem fit learn as long as dem get a teacher.
Also, a learning intelligence system fit also learn how to extract, store, and select frameworks, and how to use dem inside state memory, as knowledge. Dis na im go make am fit do more complex learning.
Kinds of Knowledge
Based on these principles, when we dey design acquired learning, we must make it clear wetin kind of form di acquired knowledge go take.
One fit consider a way where acquired knowledge go also dey learned separately as neural network parameters.
But, acquired knowledge no must just dey limited to neural network parameters alone. A practical option na knowledge wey dem don write out as text in natural language.
Knowledge wey dem don write out as text in natural language fit dey extracted and used by using di natural language processing power of LLMs. Plus, since dem fit handle am as data in normal IT systems, storing and selecting am also easy.
Wetin pass dat, knowledge wey dem don write out as text in natural language easy for humans and other LLMs to check, understand, and, for some cases, even edit im content.
Dem fit also share, merge, or split am with other learning intelligence systems.
For these reasons, di acquired knowledge for di ALIS concept go first be designed to target knowledge wey dem don write out as text in natural language.
Acquired State Memory and Frameworks
We don explain de benefits of choosing natural language text as di way to present acquired knowledge.
Na so too, natural language text fit also dey used for di state memory and frameworks for reasoning.
Frameworks, as conceptual structures, fit dey stored and used in de knowledge store as knowledge wey dem don write out in natural language.
Even when dem dey start or update states based on de structure wey a framework define, text-format state memory fit dey used.
By designing not only acquired knowledge but also frameworks and state memory to dey in text format, ALIS fit use de natural language processing powers of LLMs for acquired learning and reasoning generally.
Formal Knowledge
Acquired knowledge, frameworks, and state memory fit dey expressed, no just in natural language text, but also in more strict formal languages or formal models.
Even though I write "select," di main aim for ALIS na to put plenty different acquired knowledge learning ways to make am fit use both innate and acquired learning.
Knowledge wey dem represent with formal languages or formal models fit dey made more precise and clear.
Wetin pass dat, if one framework dey expressed using a formal language or model and one initial state dey opened up in state memory, then an intelligent processor wey fit process formal models, instead of an LLM, fit do a simulation or logical development with a strict model.
A good example of such formal languages or formal models na programming languages.
As de system dey learn about de world, if e fit express de laws and concepts wey e find there as a program in a framework, then e fit simulate dem on a computer.
Column 1: Kinds of Knowledge
When dem arrange di knowledge inside one learning intelligence system, e go clear say dem fit divide am into three main types of knowledge systems and two types of state.
Di three knowledge systems na: network parameter knowledge, wey neural networks dey handle; natural knowledge, wey dem express in natural language; and formal knowledge, wey dem express in formal languages.
Di two types of state na: stateless and stateful.
Stateless network parameter knowledge na knowledge wey dey come to mind quick quick, like wetin you go see for deep learning AI. De features of cats and dogs, wey you no fit really think about or talk about well well, dem fit learn am as stateless network parameter knowledge.
Stateful network parameter knowledge na knowledge wey dey come out from fuzzy, repeated processes, like wetin dey happen for generative AI.
Stateless natural knowledge na knowledge like meanings wey dey tied to individual words.
Stateful natural knowledge na knowledge wey get context inside sentences.
Some natural knowledge don dey inside stateful network parameter knowledge from beginning, but some knowledge also dey wey dem fit get from natural language text.
Stateless formal knowledge na knowledge wey dem fit express in mathematical formulas without repetition. Stateful formal knowledge na knowledge wey dem fit express as a program.
Person fit also use im own short-term memory as state memory for natural knowledge and formal knowledge.
However, because e be short-term memory, e get problem say e hard to keep one state steady. Wetin pass dat, e no too sabi keep formalized, clear states.
On de other hand, paper, computers, and smartphones fit dey used as state memory to write down or edit natural language text, formal languages, or formal models.
Generally, pipo dey often see data on paper or computers as a knowledge store for memorizing knowledge, but dem fit also use am as state memory for arranging thoughts.
So, e clear say humans dey do intellectual activities by using all these three knowledge systems and two types of state well well.
ALIS, too, get de potential to make im capabilities better well well by enabling and strengthening intellectual activities wey dey use these same three knowledge systems and two types of state.
Specifically, ALIS get de power to use plenty knowledge stores and state memory. Wetin pass dat, e fit easily do intellectual tasks by preparing plenty of each and switching or combining dem.
Column 2: Intelligent Orchestration
Even though e good to fit gather plenty knowledge for one knowledge store, di amount of knowledge no just mean say e go automatically be better for intellectual work. Dis na because generative AI get limit to how many tokens e fit process at once, and irrelevant knowledge fit cause confusion.
Instead, if dem divide di knowledge store well and turn am into specialized knowledge stores wey get plenty knowledge, with each one holding wetin e need for a specific intellectual task, dem fit reduce de problem of token limits and noise.
But, for dat to happen, each specialized knowledge store go only fit dey used for its own assigned intellectual task.
Plenty intellectual activities na complex combinations of different intellectual tasks. So, by dividing knowledge into specialized knowledge stores based on de type of intellectual task and breaking down intellectual activity into individual tasks, ALIS fit do de whole intellectual activity by properly switching between these specialized knowledge stores.
Dis one be like an orchestra, wey get professional musicians wey dey play different instruments and a conductor wey dey lead everybody.
Through dis system technology, intelligent orchestration, ALIS go fit arrange its intellectual activities.
ALIS Main Design and How To Develop Am
From here, we go arrange how to develop ALIS.
As we don already talk for di principles and columns, ALIS naturally dey designed to easily expand im functions and resources. Dis na because di main thing about ALIS no be for specific functions, but for di processes of knowledge extraction, storage, selection, and utilization.
For example, dem fit provide plenty different types of knowledge extraction systems, and de system design go allow free choice to select from dem or use dem at di same time.
Wetin pass dat, ALIS itself fit dey made to do dis selection.
Na so too, storage, selection, and utilization fit also dey freely chosen or parallelized.
So, dem fit develop ALIS small small and fast, without needing to design all de functions like waterfall.
How ALIS Go Start
Now, make we design a very simple ALIS.
Di main interface go be a chat AI wey everybody sabi. For start, wetin di user type in go straight to di LLM. Di LLM answer go show on di screen, and di system go wait for di next thing di user go type.
When di next input come, dem go give di LLM not just di new input, but also di whole chat history between di user and di LLM.
Behind di interface of dis chat AI, dem go prepare a system to comot knowledge wey dem fit reuse from di chat history.
Dem fit add dis system to di chat AI as a process wey go run when a conversation finish or at regular times. Of course, an LLM go dey used for knowledge extraction.
Dem go give dis LLM di ALIS concept and principles, plus how to comot knowledge, as a system prompt. If di knowledge no comot as dem plan, dem go need to try and error to make di system prompt better.
Di knowledge wey dem comot from di chat history go straight into a knowledge lake. A knowledge lake na just a system for storing knowledge as e be, before dem arrange am.
Next, dem go prepare a system to arrange di knowledge lake so e go be easier to select knowledge from am.
Dis one go involve providing an embedding vector store for semantic search, like wetin dem dey use for normal RAG, and keyword indexes.
Other possibilities include making more advanced knowledge graphs or doing category classification.
Dis collection of arranged information for di knowledge lake go dey called a knowledge base. Dis whole knowledge base and knowledge lake go be di knowledge store.
Next, dem go join di knowledge store into how di chat UI dey process things.
Dis one na basically di same as a general RAG system. For user input, dem go select relevant knowledge from di knowledge store and pass am to di LLM along with di user input.
Dis go allow di LLM to automatically use knowledge when e dey process user input.
Dis way, knowledge go dey increase with each conversation with di user, making a simple ALIS possible wey dey use knowledge wey don gather from past conversations.
Simple Story
For example, imagine say one person dey build web application using dis simple ALIS.
Di person go report say di code wey di LLM give bring error. Then, di person and di LLM go work together to find solution to de problem. Make we say dem find out say de external API specification wey di LLM sabi don old, and say as dem use di latest API specification, di problem clear.
For dis case, di knowledge say di LLM API specification don old and wetin de latest API specification be, fit dey gathered for de knowledge store from dis chat.
Then, when dem want create program wey go use de same API next time, ALIS fit use dis knowledge to generate program based on de latest API specification from de beginning.
Making Di First ALIS Better
But, for dis one to happen, dem need to select dis knowledge based on wetin di user input. E fit be say dis knowledge no directly link to di user input, because di name of di API wey cause problem no go likely show for di first thing di user type.
For such case, di API name go only show for di first time for di LLM answer.
So, we go extend di simple ALIS small small by adding a system for pre-check comments and post-check comments.
Pre-check comments na like di "thought mode" wey LLMs dey use recently. We go prepare a memory wey fit hold text as state memory, and tell di LLM through a system prompt to do pre-check comments when e receive user input.
Then dem go put di LLM pre-check comment result inside state memory, and based on dis result, dem go select knowledge from di knowledge store.
Then, di chat history, pre-check comment result, knowledge wey match di user input, and knowledge wey match di pre-check comment result go dey passed to di LLM to get im answer.
Wetin pass dat, for di result wey di LLM give, dem go search for knowledge inside di knowledge store. Including any knowledge dem find there, dem go then ask di LLM to do a post-check.
If dem find any issues, dem go pass am back to di chat LLM along with di problem points and di reasons for de comments/feedback (指摘).
By giving chances to select knowledge during both pre-check comments and post-check comments, we fit increase di chances of using di knowledge wey don gather.
Wetin Go Happen Next (Outlook)
Di way wey dem dey create di first ALIS and dey add improvements to fix im problems na exactly wetin agile development be, e dey show say dem fit make ALIS better small small.
Wetin pass dat, as we don show example, di first ALIS good well well for software development. Dis na because e be one area wey get high demand and e be area wey dem fit gather knowledge clearly.
Na one area where results dey clear, still e need and e benefit well well from trial-and-error, and gathering knowledge step by step.
Also, since ALIS development itself na software development, de fact say ALIS developers fit also be ALIS users na one attractive thing.
Wetin pass dat, along with di ALIS system, dem fit openly share di knowledge lake for platforms like GitHub.
Dis go allow many pipo to contribute to making di ALIS system better and gathering knowledge, with everybody enjoying de benefits and making ALIS development faster and more efficient.
Of course, sharing knowledge no just be for ALIS developers; dem fit gather am from all software developers wey dey use ALIS.
Di natural language nature of knowledge get two extra benefits.
Di first benefit na say dem fit still use knowledge even when LLM models change or dem update am.
Di second benefit na say dem fit use di plenty knowledge lake wey don gather as a pre-training dataset for LLMs. There are two ways to use dis: as fine-tuning, or for LLM pre-training itself.
No matter wetin, if dem fit use an LLM wey don naturally learn from di knowledge wey gather for di knowledge lake, software development go become even more efficient.
Wetin pass dat, software development get different processes like requirements analysis, design, implementation, testing, operation, and maintenance. Specialized knowledge also dey for each software domain and platform. By creating a system to divide di plenty knowledge wey gather from these perspectives, an ALIS orchestra fit dey formed.
So, di main technologies for ALIS don dey. Di remaining important step na to practically try different methods — like how to extract knowledge, choosing di right knowledge, dividing specialized knowledge, and how to use state memory — to find effective ways. As e dey get more complex, processing time and LLM usage costs go also increase, so dem need to optimize am.
These trial-and-error processes and optimizations fit dey advanced in a learning-oriented way through how dem dey develop and refine frameworks.
For start, developers, as users, go likely integrate frameworks into ALIS through trial and error. But even then, dem fit give di LLM itself de task of generating framework ideas.
Then, by putting a framework for improving and discovering frameworks into ALIS, based on results wey dem get from di world and knowledge wey dem extract, ALIS itself go do trial-and-error and optimization in a learning-driven way.
ALIS For Real World
Once ALIS don reach dis level, e suppose fit learn knowledge for many different areas, no just for software development world.
Like software development, ALIS go expand im use to different intellectual works wey humans dey do with computers.
Even for such purely intellectual works, ALIS go get a quality like an embodied AI for di world e dey target.
Dis na because e dey recognize de boundary between itself and di world, e dey act on de world through dat boundary, and e fit perceive information wey e get from de world.
When dis boundary with de world dey visible physically and e dey for one place, we normally dey call am body.
But, even if de boundary no dey visible and e dey scattered for different places, de way perception and action dey happen through a boundary still dey de same like when person get physical body.
For dis sense, an ALIS wey dey do intellectual activities fit be considered to virtually get de characteristics of an embodied AI.
And, if ALIS dey refined to a stage where e fit properly learn even for new, unknown worlds, e get possibility say ALIS fit dey included as part of a real embodied AI wey get actual physical body.
Dis way, ALIS go eventually dey applied to de real world and go start to learn from it.