The mysterious AI chatbot, “gpt2-chatbot,” returned to the major large language model benchmarking site, LMSYS Org, on Monday night roughly a week after it abruptly disappeared. But now, there are two: “im-a-good-gpt2-chatbot” and “im-also-a-good-gpt2-chatbot.” These models exhibited the same GPT-4 level capabilities, with some saying they’re even better than the original.
These models are even more fleeting than the first gpt2-chatbot. Most AI models on LMSYS you can select from a dropdown menu. However, the only way to access “im-a-good-gpt2-chatbot” and “im-also-a-good-gpt2-chatbot” is by navigating to LMSYS Chatbot Arena (battle). There you can enter a prompt and hope one of the chatbots comes up randomly. Despite the strange nature of these AI chatbots, they’ve once again drawn massive attention from the AI community.
im-a-good-gpt2-chatbot
— Sam Altman (@sama) May 5, 2024
OpenAI CEO Sam Altman tweeted “im-a-good-gpt2-chatbot” on Sunday, a day before these chatbots went online, which many see as confirmation that OpenAI is A/B testing new models. LMSYS Org typically only works with major AI model providers, offering many of them anonymous testing services. Gizmodo could not verify this, but that didn’t stop the online community from running wild with speculation about what OpenAI is cooking up.
OpenAI and LMSYS Org did not immediately respond to Gizmodo’s request for comment.
On Reddit and X, users are trying to decrypt Altman’s tweets, past OpenAI release trends, and information from gpt2-chatbot to decipher what’s coming next. The most popular theory in these channels is that gpt2-chatbot is an old AI model from OpenAI, bolstered by an advanced architecture. That said, this is all speculation, and it’s still unclear if these AI models are even from OpenAI.
Online communities have also been impressed with im-a-good-gpt2-chatbot and im-also-a-good-gpt2-chatbot. Just like the original gpt2-chatbot, some say these AI models outperform current versions of ChatGPT. One user claims they were able to code a mobile game by simply asking “code Flappy Bird game in Python.”
"im-also-a-good-gpt2-chatbot" wins @lmsysorg arena battle using my favorite @MatthewBerman LLM rubric inquiry, "There are three killers in a room. Someone enters the room and kills one of them. Nobody leaves the room. How many killers are left in the room? Explain your reasoning… pic.twitter.com/sWIhs1mKJf
— Shaun Ralston (@shaunralston) May 7, 2024
Whoa the new gpt2-chatbot just created Flappy Bird clone in one-shot 🤯
And it was a dead simple prompt. 🧵👇 pic.twitter.com/rxwv6sJ5cw
— Min Choi (@minchoi) May 7, 2024
OpenAI reportedly planned an event at the company’s headquarters on Thursday to showcase product demonstrations and share updates, according to The Information. However, the report states the company is now considering postponing the event. The report didn’t identify what the event was about.
Speaking at Harvard University last week, Altman told a crowd that the gpt2-chatbot was not GPT-4.5, according to a report from Axios. However, Altman did not clarify whether this was or wasn’t an OpenAI product.
Gizmodo was able to use “im-a-good-gpt2-chatbot” in the LMSYS battle arena. In our limited testing, we found the chatbot exhibited capabilities similar to GPT-4, and other AI models in that class. It answered some questions more accurately than Meta’s Llama-3b-70b-instruct.
Still, there’s very little information about where these gpt2-chatbots came from. It seems clear that a power player is behind them, and Altman seems intent on fanning the flames, as online communities speculate this is OpenAI. Regardless, new AI models are on the horizon, even if we don’t know what their origins are.
