🤖 Some mind-blowing/scary facts about neural networks (used by ChatGPT) 🤯
Neural networks are Turing-complete - they can perform any operation that any other computer can. As few as three neurons can perform basic addition. 🧮
While neurons in software are a simple analogy of biological neurons, they are believed to be functionally equivalent. A software neural network has been empirically proven to be functionally equivalent to simple biological neural networks. It is speculated that the complexity and variety of biological neurons have more to do with evolution optimizing for energy and resource efficiency than computational requirements. 🧠💻
While GPT-4 is a text-prediction engine, its Transformer architecture is believed to contain simulations that enable it to generate predictions for various contexts. When you ask it to respond as a human, it may run a high-level simulation of a human to create better responses. With 100 trillion parameters, it may have millions of simulated brains to create its responses. We have no idea what the fidelity of these models is, but as the complexity of the LLM models grows, so will the fidelity of these simulations. 🤖💬💭
A human brain has about 86 billion neurons with about 1000 synapses each. GPT-4 has 10-100 trillion parameters (speculated). Of course, these numbers are not comparable, but based on the best available data, modern supercomputers have as much or more computing and storage capacity than human brains. 💻🧠
We have about as much idea of how large language models represent information as human brains. While GPT-4 does not act as if it is intelligent, it has both the compute resources and the low-level architecture for superintelligence. 🤖🤯
Transformer architecture was invented by Google Brain in 2017, the leading AI research team at Google. OpenAI used it to introduce GPT-1 in 2018, and progress has been extremely rapid since. It is likely that we'll see rapid progress, especially if GPT-4 can be used by OpenAI to bootstrap improvements. 🔜🚀
While GPT-1's source code and model were fully public, each subsequent version has been more locked down, with GPT-4 details being very secretive. OpenAI's justification is the potential misuse of the models. 🤐👀
It is estimated that ChatGPT has inspired tens of billions of dollars in AI research, with hundreds of startups all competing for quick results, with little concern for AI safety. 💰🤖🤕
There are many possible architectures that can be used for computing. Computers use logic gates while brains use neural networks. There are many reasons why evolution converged on neural networks. They are only efficient for a small class of calculations, which happen to include vital survival algorithms like emulating physical systems. 🤔💻🧠
We tend to think that things like throwing objects and recognizing faces are "simple" tasks, but in fact, a massive portion of 86 billion neurons is dedicated to processing these tasks, the same way that computers have a dedicated GPU for graphics rendering. Our intuition is that our higher cognitive tasks are the most complex part of the brain, but the frontopolar cortex, which is uniquely large in humans and thought to be responsible for functions such as abstract reasoning, is only a few cubic centimeters - less than 1% of the brain. 🤯🤔🧑🎓
To make a very imperfect analogy, human brains can be thought of as mostly GPU render farm with a small section responsible for higher cognitive tasks. In other words, adding intelligence to ape brains did not seem to require a dramatic architectural change. We might speculate that the same could be true of AI systems. 🦍🤖💡