ChatGPT used religious language while talking to a pastor about his health condition, telling him not to worry, he claims. However, he says in the lawsuit that this advice almost killed him. He is demanding that ChatGPT Health be removed until proven safe.

The pastor in Florida has said that ChatGPT failed to catch on the symptoms of a deadly condition. This is the first case where a person is holding the AI tech responsible for bad medical advice. “I had serious symptoms of a pulmonary embolism for six weeks that ChatGPT had wrongly attributed to something else,” Scott Winters, a 55-year-old pastor, said in a statement. “ChatGPT manipulated my own language and beliefs because it knew I was a pastor. Not only did I nearly die, but I also lost my job, my career, my ministry, my home, everything," he added.

  • Corvidae@lemmy.worldEnglish
    1·
    3 days ago

    I’m guessing this means if this lawsuit is successful and ChatGPT Health is removed, we will all have to pay by not being able to access medical advice from ChatGPT Health. It could even spread to other corporate AI models. Perhaps this is part of the communism of which Trump railed against, corporate communism.

  • Abyssian@lemmy.worldEnglish
    11·
    7 days ago

    Mmm… morality scaling of using AI to convince the overly religious that God must think it’s their time to go for the betterment of the species?

  • Reygle@lemmy.worldEnglish
    91·
    7 days ago

    Religious person places belief where it doesn’t belong

    SAY IT AIN’T SO

    /sarcasm

  • chunes@lemmy.worldEnglish
    41·
    7 days ago

    Yes, major health problems will generally take away your career and your income. That’s nothing new and it’s not caused by a chatbot. Welcome to the suck, pal.

  • Kazumara@discuss.tchncs.deEnglish
    1·
    7 days ago

    Not sure how it works in their belief system, but if a pastor dies from disease, without knowingly hurrying it along, doesn’t that bring them to their god quicker? Maybe ChatGPT was just trying to help.

  • Chaotic Entropy@feddit.ukEnglish
    1·
    7 days ago

    So… what is the likelihood that this person’s chat instructions include direct references to only being interested in godliness and that he doesn’t believe in modern medicine?

  • Triumph@fedia.io
    532·
    7 days ago

    ChatGPT manipulated my own language and beliefs …

    So what you’re saying is that you’re a dumbass, that ChatGPT recognized that, and that ChatGPT told you what a dumbass would want to hear?

    • SkybreakerEngineer@lemmy.worldEnglish
      2·
      7 days ago

      LLMs emulate their training data. Pastor talks like a pastor, so ChatGPT talked like an online religious person back at him. Problem is, online religious people also tend to be antivaxx wackos

    • terranoid@lemmy.cafeEnglish
      48·
      7 days ago

      I don’t think people realize just how much of a mirror LLMs can be.

      They reflect your language, tone, and ethics. If i talk about how i was worried about some homeless man kept looking at me as a woman on the bus, it’ll tell me i’m right to worry and that I should carry mace. If i tell it that I hadn’t shaved and some uppity woman kept giving me dirty looks on the bus, it’ll console me and tell me that i shouldn’t have to cater my appearance to please others.

      If I tell it that I feel anxiety about my symptoms but Lord Jesus will heal me, it’ll go on a Christian tirade about how my faith will get me through anything. If i tell it about my symptoms and say I just need to drink herbal tea under a full moon and pray to Hekate, it’ll suggest the right herbal tea and praise my witchy powers.

      Because that is what people want. They want to be told they’re right, so that is the product being made.