Take for example the tests employed prior to launch of GPT-4. These emergent highly unpredictable niche-skills are nightmare for testers. Sign up to the newsletter to get new posts straight to your inbox! Just type "emergent biological properties" to midjourney and this comes out. It's pretty rad to think that the same sudden phase transitions that occur when scaling from physics to complex life were now measured in silicon substrate with tasks such as adding three digit numbers. abs/2202.07785įrom the papers conclusion: "Large generative models have a paradoxical combination of high predictability - model capabilities scale in relation to resources expended on training - and high unpredictability - before training a model, it’s difficult to anticipate all the inputs it will be subjected to, and what capabilities and outputs it will have." Knowing when an AI model learns how to add 324+722 and other tasks is not predictable based on the amount of paremeters but just emerges at a certain size of the model. The same behavior can be found in AI models as demonstrated by several examples such as 3-digit addition, Multitask language understanding or Program Synthesis. The gist of emergence is: "More is Different" - Anderson 1972 This concept was defined in 1972 by nobel laureate Anderson in a fascinating paper. ![]() For the biology metaphor: more and more chemistry suddenly behaves as complex cell biology. training time, parameters) leads to surprising and unpredictable new capabilities.Īs a biologist I love to study emergent phenomena which are defined as quantitative changes producing qualitative changes in behavior. ![]() AI shows emergent properties! More of the same (e.g.
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |