Anthropic, a research company, is making strides in understanding the inner workings of artificial intelligence (AI). The company is focusing on large language models (LLMs) like ChatGPT, which have been particularly challenging to decipher due to their size and complexity.
Anthropic is striving to reverse-engineer AI systems and scan the ‘brains’ of LLMs to see what they are doing, how, and why. The company has identified combinations of artificial neurons within these models that signify specific concepts or “features”. These features range from everyday objects like burritos to more complex elements like semicolons in programming code, and even potentially dangerous concepts like biological weapons.
This work has significant implications for AI safety. If researchers can figure out where danger lurks inside an LLM, they are presumably better equipped to prevent it. The field of explainable AI (XAI), which aims to make AI’s decision-making process more transparent, has grown rapidly in recent years, especially with the emergence of LLMs.
Read more: www.wired.com