News on Medial

Why AI can’t spell ‘strawberry’

TechCrunch · 9m

Large language models (LLMs) like GPT-4 can excel at complex tasks but often fail at simple ones, such as counting the number of 'r's in 'strawberry.' This is due to their reliance on tokenization, which breaks text into pieces rather than understanding individual characters. The problem is rooted in the architecture of transformers, which can't efficiently handle actual text but instead work with numerical representations. While image generators have improved, the challenges with LLMs are deeply embedded and hard to fix.