top of page
Search
  • Writer's pictureEric Nentrup
    • 6 days ago
    • 3 min read

The Week in Review: Responsible Skepticism

Updated: 6 days ago

The big story this week is OpenAI's release of GPT-4, the successor of GPT-3, the backend LLM for ChatGPT. Early reactions in the media indicate that it is both exciting and unnverving. If we think that seems too quick to be possible, normal, or otherwise, we best get used to such announcements—the changes are happening quickly and aren't done accelerating. But how is it actually different? OpenAI claims:

We spent 6 months making GPT-4 safer and more aligned. GPT-4 is 82% less likely to respond to requests for disallowed content and 40% more likely to produce factual responses than GPT-3.5 on our internal evaluations.

Such claims sound compelling until you pause for a moment to be skeptical. Consider some of the other advances in GPT-4 over its predecessor:

In adopting a skeptical disposition for the claims we're hearing, it's more useful to think in concrete terms, though admittedly over simplistic (like calorie counting or batting averages or standardized test scores). Yet up until recently, a common measure of an LLM's power was comparing the scale of one model's parameter set to another, like we see here in this diagram.


Image courtesy of Simon Høiberg © 2023


That's GPT-3's 175B unique parameters that trained and are contained with the algorithm, compared to GPT-4's 100T. We're into billions and trillions from this point forward. What will we see in GPT-5 and beyond, let alone from the other LLMs in the field?


The same day as OpenAI's announcement, Sal Khan posted his explanation of how Khan Academy had been working to implement GPT-4 in their products. The video is compelling and positive, but also telling on a couple fronts:

  1. Firstly, have you ever a public announcement from the CEO of a massive edtech company prioritize announcing a product improvement that was exclusively about the inclusion of one specific piece of technology that they did not develop internally within their flagship product's roadmap? Steve Jobs 2010 open letter to Adobe—a nail in the coffin for Flash—comes to mind, but nothing in edtech.

  2. Secondly, it appears from the video the Khan team has been working with OpenAI for some time to develop their chatbot assistant, "Khanmigo", a result of their "Khan Labs". Both of these details shouldn't be lost and definitely support the necessity our work at EdSAFE regarding transparency and explainability to keep the field safe when employing emerging technology in teaching and learning while encouraging innovation with an appropriate sense of urgency.

To be clear, the Khan Academy in its essential mission is an equity win for all learners that benefit from it. The always-on lessons have empowered learning in ways we couldn't have imagined. Khan has done this by leveraging the technology and interconnectedness of the present, relative to slow-to-evolve education systems. Yet this announcement deserves a moment of pause to consider the implications when other big brands in edtech release their updated products.


Therefore, we shall stay vigilant but not policing, skeptical but not cynical. Outside of OpenAI's disruptions, the other big tech companies have been vying similarly for mainstream attention and we may take a closer look at that in near future stories.


In the meantime, here are some of the other relevant stories from the week.


SAFETY


ACCOUNTABILITY


FAIRNESS


EFFICACY



22 views0 comments
bottom of page