AI Convergence - A Core Limitation of LLMs
As someone who tries to read at least 25 books a year, I have always gone way out of my way to find really good books which I would define as a book that provides me with new insights, new perspectives, new information and new frameworks throughout its pages. The idea is that I want to get better as a result of reading a book and improve in some dimension of my life whether it's business, fitness, family or just learning about something unique and esoteric - how did the Church plan the Crusades for example (How to Plan a Crusade)! I have found that books with very few reviews (less than 200 on Amazon) and purchases tend to be much better books than those watered down for mass-market appeal.
Now what does this have to do with AI?
Large language models (LLMs) are trained on all of the information on the internet and an inherent challenge with this type of "training" is the information that these models are trained on. Most of the data, writing, books, websites and training material on the internet tends to converge and be quite average and boring much like how most of the mass market books are watered down. What this means is that LLMs have an extremely strong tendency to give you normally distributed and average responses. This is fine if you want to be normal and average but if you are looking for divergent thinking, contrarian viewpoints or long-tail strategies, LLMs are quite poor with this unless you get very clever with your prompting and even then they are quite limited. These models tend to have several types of convergent behavior:
Expert Bias: Imagine all of the experts and professional organizations in a field such as physical fitness tell us that kids cannot work out before fourteen because it limits bone growth and that hydration during physical activity creates muscle cramping. These experts then generate hundreds of thousands of papers, publications and pieces of material supporting these claims. Now imagine that a single researcher publishes a comprehensive study showing that all of these experts are wrong. Today, LLMS would provide incorrect answers to these questions on physical fitness based upon the authority of the experts instead of the 'correct' answer. This expert bias is a big fundamental problem with LLMs in that history has shown us that frequently the beliefs of an entire field are overturned by a single researcher or paper.
Another great example of this is in the educational space where educational experts decided to do away with phonics despite phonics being the underpinning for language development for thousands of years. We later found out that all of these experts were 100% wrong (who would have thought!) and phonics has fortunately reemerged despite negatively affecting millions of kids. This is why it is so important to remember that LLMs are just abstractions of the data they are fed and are not actually arbiters of truth or knowledge.
Data Weight Bias: Much like the expert bias authority problem with LLMs, they also suffer from a data weight problem where LLMs do not inherently know how to weigh conflicting pieces of information and especially so if they have differing volumes of data supporting them. For example, ask an LLM why the Hepatitis B vaccine was previously administered to all babies in the US (even those with Hep B negative mothers) and not in the EU and it will do mental gymnastics to try and explain the inconsistency to you. Suggesting EU health officials are making a better decision than US health officials is inconceivable in the LLM framework. Part of the reason for this is that truth can be subjective and is based on morals, norms, culture and values (i.e., is it better to mildly harm several kids to prevent one from getting Hepatitis B?). One of the scariest things about LLM's to me is that the cultural underpinning of the model is not always well understood and might not be at all aligned with the interests of the people using the LLM.
Mean Bias: One of my biggest hang-ups with LLMs is their mean bias which simply means that they tend to provide 'average' responses to prompts and that they are heavily weighted to normalcy. The reason is that the information on the internet reflects the collective conscious and knowledge of individuals and it tends to be average. For example, imagine how differently an LLM would work that was only trained on data from millionaires or billionaires vs one trained on all of the information on the internet.
Now how does this convergence and bias effect you? It surely affects you anytime you interact with an LLM and is starting to have real negative effects such as in the job search process. When a company now posts a role, they receive hundreds of convergent resumes and cover letters which are all exactly the same and it becomes difficult to determine which candidates are ideal and which are not. This lack of differentiation leads companies to implement less meritocratic processes for selecting candidates such as ranking by college or leaning heavily on recommendations and networks. This is great if you have these connections, but it makes it much more challenge for those who don’t to rise up.
My suggestion for individuals in leveraging LLMs that want to avoid convergence is to do the following:
Prompt for divergence: e.g., how will every other resume for this job look and how can I stand out and get the job?
Find divergent data sources: Read books, papers and materials from beautifully strange and different people. I believe the human mind and our ability to think divergently will never be replicated and thus you have to seek out the insights from truly special humans.
Ask for the data supporting an LLM generated viewpoint: We often take an LLM at its response and don’t ask where the feedback actually came from or what it was based on. For example, just because every medical association suggests a specific medical pathway for your pregnancy does not mean that it is the right pathway for YOUR pregnancy. What if that guidance is based on broad circumstances that do not pertain to you or even worse, what if it is based on bad data and has yet to be confirmed like the phonics example above.
Stay human and divergent!
