ChatGPT on Advent

I have a paper that emphasizes ChatGPT errors. It is important to recognize that LLMs can make mistakes. However, someone could look at our data and emphasize the opposite potential interpretation. On many points, and even when coming up with citations, the LLM generated correct sentences. More than half of the content was good.

You can read ChatGPT’s take on a wide variety of topics within economics, in the appendix of our paper. The journal hosts it at https://journals.sagepub.com/doi/suppl/10.1177/05694345231218454/suppl_file/sj-pdf-1-aex-10.1177_05694345231218454.pdf If that link does not work then the appendix has been up on SSRN since June in the form of the old version of the paper.

Apparently, LLMs just solved an unsolvable math problem. Is there anything they can’t do? Considering how much of human expression and culture revolves around religion, we can expect AI’s to get involved in that aspect of life.

Alex thinks it will be a short hop from Personal Jesus Chatbot to a whole new AI religion. We’ll see. People have had “LLMs” in the form of human pastors, shaman, or rabbis for a long time, and yet sticking to one sacred text for reference has been stable. I think people might feel the same way in the AI era – stick to the canon for a common point of reference. Text written before the AI era will be considered special for a long time, I predict. Even AI’s ought to be suspicious of AI-generated content, just in the way that humans are now (or are they?).

Many religious traditions have lots of training literature. (In our ChatGPT errors paper, we expect LLMs to produce reliable content on topics for which there is plentiful training literature.)

I gave ChatGPT this prompt:

Can you write a Bible study? I’d like this to be appropriate for the season of Advent, but I’d like most of the Bible readings to be from the book of Job. I’d like to consider what Job was going through, because he was trying to understand the human condition and our relationship to God before the idea of Jesus. Job had a conception of the goodness of God, but he didn’t have the hope of the Gospel. Can you work with that?

This is the first part of ChatGPT’s response:

Certainly! Creating a Bible study that combines the Advent season and the book of Job can provide a rich and reflective experience for your small group. Here’s a structure you can use:

Title: “Hope in the Midst of Suffering: Lessons from Job in the Advent Season”

Objective: To explore the themes of hope, suffering, and the human condition through the lens of Job, considering how our understanding of God’s goodness is deepened by the Advent of Jesus.

Session 1: Job’s Understanding of God

Opening Prayer: Begin with a prayer for wisdom and understanding.

Scripture Reading: Job 1:1-22

Discussion Questions:

How does the opening passage of Job set the stage for the challenges he will face?

In what ways does Job express his understanding of God’s character in these verses?

How might Job’s perspective on God be different from our own, considering he did not have the knowledge of Jesus and the Gospel?

Reflection and Application: Encourage group members to reflect on their own conceptions of God and how the knowledge of Jesus and the Gospel shapes their understanding.

Session 2: Wrestling with God

Opening Prayer: Pray for openness and receptivity to God’s Word.

Scripture Reading: Job 9:1-35

Discussion Questions:

What is Job expressing in his response to Bildad about his desire for a mediator?

How does Job grapple with the tension between God’s justice and his suffering?

How might Job’s perspective on a mediator or advocate be different from ours, knowing Jesus as our mediator?

A human expert could improve this response, but it’s pretty serviceable on the first try. AI can give you religious content if you ask for it. A non-expert will not be able to tell any difference between what a human expert would write in response to the same prompt.

You may have heard that ChatGPT is “woke.” If you play around with questions about morality, even if you ask it to reference the Bible explicitly, then you will notice a leftward slant in the answers. Don’t confuse knowing the Bible with being right-wing, in the case of ChatGPT.