ChatGPT Deep Research needs more scrutiny

ChatGPT’s Deep Research just rolled out to us $20 ChatGPT Plus users. It was formerly locked behind the $200 Pro tier. I threw it a quick question “Why is the birth rate so low in Singapore” and it took 7 minutes to research the question. My quick take on the result:

  1. The depth and breadth of analysis seemingly surpasses the average person’s ability. ChatGPT looked at 20 different sources and due to its inherent writing ability, made the research article easy to read and sound very insightful. I’ve posted the link in the comments – at first glance, it looks like an A+ essay at the tertiary level.
  2. However, upon close scrutiny, the sources cited are too random or irrelevant. I don’t consider the Sammyboy site to be an authoritative voice, given that its reputation for smut and anti-establishment views. A lot of of the info is extracted from a HK Legislative Council study on Singapore. The latest birth rate data stops at 2023, when we already have 2024 statistics. I was not able to activate both the “Search” and “Deep Research” button.
  3. For relevant sites like IMF and CNA, it tends to look at aggregated studies or commentaries about the topic, rather than fresh news from the government or the media.

In other words, if a student were to use ChatGPT Deep Research and not check EVERY SINGLE fact, the essay would not survive a knowledgable teacher. On the other hand, the excellent structure of the generated piece means that the student’s main job is to go fill in the blanks and join the dots. It takes a lot of the legwork out of traditional info gathering.

Anyway, to fellow teachers, I always encourage you to try a month of $20 ChatGPT. Deep Research is powerful when wielded in the right way, and educators all need to redesign our assignments to deal with Skynet-level machine intelligence.