Artificial Intelligence and tools like ChatGPT are developing technologies. This can impact how it's used and any policies. This guide will continue to be updated as new resources and ideas are published. Students should check with their professors and relevant university departments for guidelines.
Limitations of Generative AI
Limitations of Generative AI (Like ChatGPT)
As any technology evolves, there will be weaknesses that need to be considered and that impact how it can be used. These faults can also be improved as the tools are continuously developed & trained so these limitations may change.
Generative AI and tools such as ChatGPT suffer from these downsides including:
| Limitation | Example |
|---|---|
| Inaccuracies or "hallucinations" |
There are many reports of false information in responses. Tools built around large language models are using words to "predict" accurate information and may make mistakes. EXAMPLE: ChatGPT can produce "fake" or "made up" citations when was asked to provide a list of sources on a topic. |
| Not up to date |
Unless the tool is actively connected to the web, it will not be trained on current information which will impact its responses. EXAMPLE: When asked for current trends on a topic or for who the current President is, certain tools will not be able to provide that information. |
| Bias of the training material |
Since the tools are trained on materials written by biased humans, the response may also be bias in some way. EXAMPLE: If asked to create images of CEO's or prisoners, the people in the images will reflect stereotypical images like those we may see in our modern media, which continues harmful biases. |
| Transparency of information/Source Evaluation |
We do not know exactly what information is used in training data. The tools are also not "searching" the training data like a search engine or database. The content is completely stripped of context and authority. EXAMPLE: When you search on Google for current trends in legal scholarship, you are able to evaluate if the person writing is a legal scholar, a law student, or someone completely removed from the field. |
| Information behind paywalls | Generative AI tools do not have access to information behind paywalls, which is frequently more quality information than what is freely accessible to tools that access the web to provide responses. |
| Limits on conversations | Due to its capabilities to be used for nefarious purposes, many generative AI tools have "guardrails" which prevent it from answering certain types of questions, including those related to politics, "nonsense", and other sensitive topics. |
- Can AI Do That? The Challenges, Limitations, and Opportunities of Generative AIAn article that outlines (with references) specific examples of limitations of generative AI tools.
- Can ChatGPT fact-check? We tested.While sometimes reaching accurate conclusions, ChatGPT struggled to give consistent answers, and sometimes was just plain wrong.
- USC Libraries: A Word About Generative AI Large Language ModelsThe USC Libraries outlines some of the limitations and ethical concerns around generative AI and large language models.
Considerations Before Using AI
- It is not a database of factual or reliable information.
- You may get different responses each time you prompt it for information. This also makes it difficult to evaluate it as a "source".
- It is not "thinking" or using logic, but "predicting" the next word in a sequence. So it is not very creative and doesn't critically think.
- Any responses you receive should be evaluated for accuracy and logical issues. It has not been proven capable of creating entirely new information but ideas based on its training data.
- They're not always accurate! So it can give you extra work to evaluate the content.
- Since AI tools can make things up (called hallucinating) or be inaccurate, you may have to do extra work to verify the information. By going directly to sources that you can verify and synthesizing the information yourself, you are eliminating the "fact checking" needed when using AI.
- They're hard to cite since they are not original work & could lead to unethical uses.
- As we learned in the "What is Generative AI?" tab, AI is not generating new ideas but predicting the most relevant next words and ideas based on its training. There is contention about how this should actually be cited as "work". See our tab on the left on Writing and Citing for more information.