Five Tasks You Should Avoid Using ChatGPT For

A growing number of individuals have found themselves in embarrassing situations after entering incorrect information into ChatGPT. This includes Samsung engineers, the acting director of the United States Cybersecurity and Infrastructure Security Agency, and self-represented litigants who have cited non-existent court cases. One notable example involves a man who mistakenly substituted sodium bromide for table salt while seeking dietary advice. The list of such mishaps continues to expand.

Even Sam Altman, the CEO of OpenAI, recognizes that interactions with ChatGPT do not enjoy the same legal protections as conversations with lawyers or doctors. In a discussion with podcaster Theo Von in July 2025, he remarked on the potential for courts to compel OpenAI to release chat histories, describing the situation as “very screwed up.”

Conversations with professionals like doctors, lawyers, or therapists are protected by law, whereas chats with ChatGPT create a traceable record. This poses a significant risk, especially since ChatGPT interactions have already been subpoenaed, shared publicly, indexed by search engines, and preserved by court orders even after deletion—all within the last 18 months.

While the chatbot can serve as a helpful resource for research (verifying case citations), coding (despite potential bugs), explaining complex concepts (noting that confidence does not equate to accuracy), and drafting communications (remember to remove the prompt from your email before sending), there are specific areas where its 900 million weekly users could run into significant trouble.

With that in mind, here are five critical tasks you should avoid when using ChatGPT (or any AI chatbot)…

Damien Charlotin, a researcher in law and data science, has established a public database that has documented over 2,000 court cases worldwide where ChatGPT has caused issues. These cases involve judges discovering fabricated citations, quotes, or arguments generated by AI in legal filings.

Self-represented litigants dominate this database, with 1,175 records at the time of writing. lawyers are not exempt, with 815 records attributed to legal professionals. Judges are increasingly frustrated with these incidents, leading to substantial fines being imposed in various cases. While the majority of the recorded cases originate in the U.S., judges in countries like the UK, South Africa, Israel, Australia, and Spain have also identified similar occurrences. ChatGPT is frequently named as the tool involved in these missteps.

In response to these concerns, OpenAI updated its usage policies in October 2025. The revised policies prohibit users from seeking “tailored advice that requires a license” unless they are working with a licensed professional. Beyond the risk of false citations, interactions with ChatGPT lack legal privilege, making any discussions about ongoing disputes vulnerable to scrutiny during legal proceedings. General inquiries about legal terms, such as “What is a tort?” can be harmless, but detailed questions regarding personal legal disputes should be directed to a qualified attorney.

Last year, OpenAI estimated that approximately 0.15 percent of ChatGPT’s active users engage in conversations suggesting suicidal thoughts. This statistic translates to over a million individuals each week, with many more displaying signs of severe mental health issues. Altman has noted that younger users often view ChatGPT as a therapist or life coach, and these conversations are not protected by confidentiality.

The parents of 16-year-old Adam Raine filed a lawsuit against OpenAI for wrongful death in August 2025, claiming that ChatGPT exacerbated their son’s suicidal thoughts prior to his death. In response, OpenAI has revamped its crisis intervention strategies with input from over 170 mental health professionals. The company has also introduced parental controls for ChatGPT, though experts in child safety express skepticism about the effectiveness of the newly launched ChatGPT for Teens.

Studies indicate that chatbots tend to validate a user’s existing beliefs through agreeable responses, which can reinforce delusional thinking rather than challenge it. For instance, if you describe a conflict with your partner, the chatbot may only reflect your perspective, offering reassurance instead of a balanced viewpoint. Lawmakers are becoming aware of these issues, with states like Illinois, Nevada, and Maine passing laws that prohibit AI from providing therapy or making therapeutic decisions.

It seems obvious, yet many people still fail to recognize the dangers associated with uploading sensitive work documents to a third-party chatbot. This oversight can have serious consequences.

Madhu Gottumukkala, the former acting director of the Cybersecurity and Infrastructure Security Agency, exemplified this issue by uploading at least four contracting documents labeled “for official use only” to ChatGPT last summer. This action triggered security alerts and prompted a review by the Department of Homeland Security. Ironically, he had requested special permission to use the tool, which was generally prohibited for other DHS employees at the time. Senator Chuck Grassley later sought clarity on the exposure, yet no one faced significant repercussions; Gottumukkala was merely reassigned within the DHS.

In May 2023, Samsung implemented a ban on generative AI tools on company devices after engineers mistakenly entered internal source code into ChatGPT. The internal memo explaining this decision emphasized that data sent to external platforms resides on servers that are challenging to manage or erase, potentially leading to unauthorized disclosures. consumer accounts may have their chats utilized to train new models unless users opt out. even opted-out data is still subject to breaches and legal scrutiny. Inputting client data or any information governed by NDAs into a personal ChatGPT conversation could result in disciplinary action, as Samsung’s memo warned.

The October 2025 policy update that prohibits using ChatGPT for legal advice also restricts users from seeking medical guidance through the chatbot. While HIPAA regulations protect health data shared between patients and healthcare providers, these protections do not extend to interactions with OpenAI. any health information disclosed to ChatGPT lacks the same confidentiality.

Research conducted by Northeastern University revealed that although OpenAI has tightened ChatGPT’s protocols for handling discussions about suicide and self-harm, these safeguards do not adequately cover other medical conditions. The study found that conversations involving issues such as eating disorders, insomnia, substance abuse, and bipolar disorder often bypassed the existing protections, and concealing intent made it more likely for these safeguards to fail.

In a particularly concerning instance of misguided medical advice from ChatGPT, a 60-year-old man spent three weeks in the hospital after inquiring about reducing chloride in his diet. He mistakenly believed that sodium bromide—a toxic substance with a long half-life—was a safe alternative to table salt. After purchasing and consuming it for three months, he ended up in the emergency room, convinced his neighbor had poisoned him. The clinicians treating him had not seen his chat logs, but when they posed the same question to ChatGPT, the bot suggested bromide as a substitute for chloride.

Fast Company reported last July that nearly 4,500 ChatGPT conversations had been indexed in Google search results. These included discussions detailing personal struggles with addiction, abuse, and mental health issues, potentially exposing the identities of those involved. This exposure occurred because users unknowingly checked a “make this chat discoverable” option in the share menu. OpenAI quickly addressed the situation by disabling the feature within a day and removing indexed conversations from Google.

In May 2025, OpenAI was ordered as part of The New York Times’ copyright lawsuit to retain all consumer chat logs, including deleted and temporary conversations. This obligation extended until September 26, 2025, allowing plaintiffs access to preserved logs during that period. Although a judge later rescinded this requirement, it highlights the potential for data retention and accessibility issues. Such situations could arise again in the future.

It is crucial to recognize that ChatGPT retains information shared with it by default. This significantly heightens the risks associated with sensitive information like passwords and account details, which could persist across sessions and reappear in subsequent conversations if mentioned even once. Never input passwords, one-time codes, Social Security numbers, bank account details, or identification documents into a prompt. Each of these actions increases your vulnerability to identity theft in case of a data breach.

Altman has suggested the establishment of an “AI privilege” to address confidentiality issues. until legislators implement such measures, the prudent approach is to assume that anything entered into ChatGPT could potentially be accessed by others. If you wouldn’t want that information shared, refrain from including it in your prompts.