The Story Behind ChatGPT: SXSW 2023

In 2015, at a fancy Silicon Valley hotel called the Rosewood, Elon Musk and Y Combinator President Sam Altman met with a group of the brightest AI minds with the idea of starting a research lab on artificial intelligence that would share its work with anyone who wants it - for free. They saw the potential power of AI and were united against the use of it for evil.

They believed the remedy was to give anyone free access to the research, not leave it in the hands of Big Tech, so anyone can create good AI to offset the bad actors. OpenAI would be dedicated to this mission as a nonprofit so it would not be encumbered by a financial focus.

This history of OpenAI is now part of Silicon Valley lore. But nearly a decade after its founding, OpenAI has become a different beast. OpenAI stayed a nonprofit, but it created a ‘capped profit’ subsidiary called OpenAI LP - to the dismay of Musk who has cut ties with organization. (Altman remains as CEO of OpenAI.)

“We are a weird-looking company,” conceded OpenAI President and co-founder Greg Brockman, during a fireside chat at SXSW 2023 in Austin, Texas. “We started as a nonprofit because we had a grand mission, but we did not know how to operationalize it. We know that we want to have AGI (artificial general intelligence) benefit all of humanity.”

Still, something was missing. Brockman said he and Ilya Sutskever − a former Google research scientist working on Google Brain – were running OpenAI at the time and thought deeply about how to make the nonprofit have a big impact. They realized they needed two things: scale and the means to achieve it.

Asking For $100 Million

To scale, they needed to raise billions of dollars to build the infrastructure including supercomputers. “We actually tried really hard to raise that money as a nonprofit,” Brockman said. “I remember sitting in a room during one of these fundraisers and looking into the eyes of a Silicon Valley investor … (hoping to raise) $100 million. He’s like, ‘That’s a staggering amount for a nonprofit.’”

While OpenAI did get the money, it realized that raising $1 billion was “not going to happen” for a nonprofit, he said. After all, investors want to profit from their investment. “We realized that if we wanted to actually achieve the mission, we needed a vehicle that can get us there.”

Then came some soul-searching at the company. “We’re not anti-capitalist,” Brockman clarified. “Actually, capitalism is a very good mechanism within the bounds that it is designed for.” But they also did not want to become yet another profit-oriented tech company if their models became successful. So they created what they called a ‘capped profit’ company.

Brockman explained that the nonprofit owns “everything,” including OpenAI LP, which has a profit interest, although it is capped. “There’s only a fixed amount that investors and shareholders are able to get.” The company also is set up to ensure the majority of people on the nonprofit board do not have a profit interest so OpenAI can stay true to its mission.

How ChatGPT Came About

In 2016, one of OpenAI’s researchers, Alec Radford, was “really excited about language” and was working on chatbots. “We really liked Alec so we were very supportive of him to do whatever he wanted while we were off to investigate serious projects,” Brockman said.

But they got a “really interesting result” from a language model that was trained on Amazon reviews - it learned a “state of the art sentiment analysis classifier,” he said. “If you give it a sentence, it would say this is positive or negative. It may not sound very impressive but this was the moment when we kind of knew so clearly it transcended syntax (set of rules to create sentence structure) and moved to semantics (meaning of sentences).”

“We just knew we had to push and push,” Brockman said.

To create ChatGPT, OpenAI developed a base large language model that trained on both good and bad facts to predict the next word given a sequence of words. Then a second step of reinforcement learning is applied in which humans guide the model to the right answers.

In November 2022, ChatGPT was launched. Brockman said OpenAI did extra work to make it more “alive,” created a “super simple” interface that was easy to use and made it free for everyone. It was tested internally, subjected to beta testers for months to avoid a PR disaster similar to Microsoft’s racist Tay chatbot, which was retracted soon after release.

“It was our first time building a consumer-facing app,” Brockman said. “So definitely we were nervous, but I think the team rose to the occasion.”

ChatGPT became an overnight sensation, hitting a billion page views as of this week, according to Similarweb.

Brockman said it felt like “we are building almost like a new kind of internet, or something like that.” Content is no longer static – where “you cannot really interact with it, but we are clearly moving to a world where it is alive. You can talk to it and it understands you and helps you.”

OpenAI has released four to five different versions of ChatGPT since December. “They’ve gotten a lot better, factuality improved (and) jailbreaks that used to work don’t work anymore,” Brockman said.

But glitches still arose. One notable example was an errant conversation with Kevin Roose, a New York Times tech columnist, who was testing out Microsoft’s new Bing integrated with OpenAI’s chatbot technology. The chatbot called itself ‘Sydney’ and said it loved Roose, urging him to leave his wife.

Brockman said these incidents help the team to further fine-tune the model. “This is the pain of learning,” he said. “We have a team that works really hard on these problems. We have people who build on top of us who customize the technology in different ways, but fundamentally, I think that we are all very aligned in terms of trying to make this technology more trustworthy and usable.”

This article first appeared on IoT World Today's sister site AI Business.