AI Paper 2: Some Weaknesses of Large Language Models
Paper
For this class, we’re focusing most on classic algorithmic AI, probabilistic AI, and some machine learning approaches. However, this isn’t what non-computer scientists typically think of when they’re considering AI. Right now, large language models (LLMs) (like ChatGPT, Github Copilot, or Facebook’s Llama) have become the popular face of AI and in many ways represent the potential of AI. These models can produce novel content that outstrips what we’ve previously seen from AI or machine learning and have collected a dedicated community of enthusiasts and professionals who seek to profit from it or explore it.
In this context, it’s important to consider the weaknesses of large language models and their output. This assignment is to consider LLMs, along with how they’re used, and evaluate their effectiveness and potential weaknesses. You can evaluate a specific LLM or LLMs more generally, and you can evaluate their performance on a particular task (ex., coding, paper writing, or producing art) or their general performance. Your topic could relate to their ability to produce human-like output or to produce output that successfully accomplishes a certain task. As before, this is a very broad topic area and so there are many potential topics and approaches that could result in a good grade.
Note: While there are many issues with LLMs, like environmental issues, authorship/attribution, and dishonesty/cheating, the focus of this paper is specifically on LLM performance and weaknesses. Your paper should focus not on how the output is used per se but more on how effective the output is. If you have any questions about this or are interested in a topic that’s in a bit of a gray area, please feel free to ask me about it!
Paper and Grades
For this paper, I am most interested in your ideas about AI. I want to hear your voice and what you think is important in this subject area. If you have a personal experience relating to AI, feel free to share that in your paper. If you’re unfamiliar with AI use, I recommend researching the above topic or even reading a book related to LLM use. A good paper will incorporate research from several different sources and viewpoints.
I expect this paper to be your own work; do not use ChatGPT (or any other internet/personal source) to write your paper, with one exception*. If you do this, the penalties (outlined in the syllabus) will be harsh. Use of outside sources is strongly encouraged and is important for a good paper! When you use an outside source, cite the source in your works cited and reference. Direct quotes should always be attributed to the source. Finally, I encourage you to consider what sources you are using; some sources are much less reliable than others and may have hidden motivations. (As an example, imagine an article written by Facebook discussing the benefits of social network use.)
*In this paper, I encourage you to spend some time using a LLM. You may discuss examples and personal observations from your use of the LLM and use those to support your arguments about the effectiveness and potential shortcomings of LLMs. Do not use an LLM to do anything illegal or dishonest in your research.
I encourage you to first research and develop your ideas, write an outline, and then start writing a draft. An excellent resource at any stage in this process is the writing center; they can help you solidify your ideas, design your paper’s structure, and correct any errors or issues you might miss in writing the paper.
Topics
Some uses of AI (and thus potential topics) include:
- recommendations,
- natural language processing,
- customer service virtual assistants,
- translation
- medical imaging analysis,
- fraud detection,
- weather analysis,
- surveillance and image analysis,
- loan decisions,
- statistical analysis,
- predictive applications,
- targetted marketing,
- spam filtering,
- pattern recognition,
- game playing,
- autonomous vehicle control,
- and many, many more.
Submission
When submitting your paper on Canvas, make sure that:
- Your name is on the paper, along with the Rhodes honor pledge “I have neither given nor received unauthorized aid on this program.”
- Your paper should be long enough to fully explore your topic. I am not creating a formal length requirement but remember that your work should fully explore the topic.
- You have a works cited page with the relevant sources you used. Include in-text citations when you reference your sources.