April 18, 2024: Sundar Pichai is CEO of Alphabet Inc. and its subsidiary Google. New York. Shutterstock
On May 14, Google announced the rollout of a new AI-powered search tool, AI Overviews, a Gemini model customized for Google Search. This new search experience promises to generate dynamic content tailored to individual queries and provide more diverse and inclusive results. Google says it will be rolled out to more than 1 billion people by the end of the year. As an early experimental user of the technology and an information professional with a background in information literacy, I have been eagerly awaiting the announcement and watching the discussion unfold. Many conversations around AI Overviews have explored its potential to have a significant impact on online commerce by influencing web traffic flows. The economic impact may indeed be significant. But what most of the discussion so far has ignored is the much more significant change this will create in the information ecosystem.
Every information discovery tool shapes how users interact with information, but the way AI Overviews enables users to ask repetitive questions, generates answers on the fly, and displays information and source material differently will redefine how users access and understand information. AI Overviews appear at the top of the search results page above the traditional Page Rank list of source hyperlinks and previews, and are a customized response to each user’s search. While the technology boasts improved user experience, it also raises concerns about whether it will enable users to intuitively understand how information is created, distributed, and accessed, a core competency of information literacy.
Given the growing research and policy interest in how AI will change the information environment and how it will impact information intake and global democracy, careful monitoring of how users use such search tools in different contexts will be paramount in monitoring the health of the online information ecosystem. This is not only because of the growing research and policy interest, but also because these and related tools will completely change how information is accessed and used. There may be unintended and unexpected consequences that we are not prepared to monitor, let alone mitigate.
Fascinating Synthesis
AI summaries are appealing because they summarize and consolidate search results in seconds. Their convenience often hides the startling reality that we are witnessing the birth of entirely new information objects: a kind of concierge Wikipedia-like tertiary sources that aim to answer increasingly complex user queries with advanced reasoning capabilities. Unlike traditional forms of search, these tertiary responses are generated with less emphasis on the primary and secondary sources from which the generated answer originates.
Synthesis is a higher-order cognitive skill on a par with analysis. AI overviews reduce the effort of synthesis, allowing us to spend the saved time and energy on other, more meaningful tasks. But we don’t yet know what those tasks are. Over-reliance on such systems risks “fading,” compromising our ability to synthesize and analyze on our own. These are cognitive skills, but moral decision-making processes and other ethical skills may also be affected.
Ultimately, evaluating information will be a difficult task of navigating and evaluating uncertainty and complexity. As massive skills are lost in this area, as information consumers increasingly cede value-based decisions to automated systems, and as critical thinking skills are essentially outsourced to AI, this will undoubtedly impact the public’s ability to form a deep and nuanced understanding of the world around them.
AI Overview acts as a semi-transparent veil between the traditional user and information source relationship. Of course, Google Search also previously offered a highlighted preview within the search results for a question. While Google Search algorithms have improved to predict results based on user behavior and other user data points (including ads), AI Overview goes further to guide users in making sense of the collected information. It acts as an intermediary that shapes users’ interaction with search content.
They also have the power to subtly influence how users think about that information, by helping them understand it. The power of these summaries is enormous and needs to be understood within various communities. This includes understanding the impact of users’ increased reliance on information summaries that may be useful but superficial, the potential for summaries to affect broader understanding of sensitive topics, and the potential for them to further polarize online communities already driven by distrust of mainstream media.
Information taken out of context
AI Overview values generated answers over individual sources, drawing and integrating those sources from their original context. Sources are embedded through an expandable carrot or horizontal carousel scrolling feature. Because sources are embedded in the answer, the Overview has a sense of completeness that subtly discourages deeper exploration actions, such as comparing and contrasting sources or personally corroborating information with primary source material. In this sense, AI Overview ultimately prioritizes summarization at the expense of exploration and browsing.
Information professionals have long tried, and failed, to teach users that the first search result is not necessarily the best. AI summaries are both the first search result and represent a machine-determined consensus of many search results. But in reality, AI summaries can never represent everything. While AI summaries appear comprehensive, there are well-documented biases in LLM systems, and the information environment on the web is already unevenly represented, so they risk overlooking diverse perspectives, especially from historically marginalized communities.
Whether it is because the generated summary lacks diverse information or because users must take the extra effort to look at multiple sources and understand the context rather than consulting an AI-generated result, it also undermines the expression of user engagement with not only the answer but also the diverse sources.Further questions regarding source attribution arise when users rely solely on and cite the generated answer and not the source material from which the answer was drawn.
Artificial objectivity and reliability
Google explains that in the experimental version of AI Overviews, they “fine-tuned the model to provide objective, neutral answers that are backed up by web results.” This is to ensure that the model does not take on or reflect a persona that may affect users on an emotional level. However, a disinterested, neutral tone can also appeal emotionally and logically to the reader, masking inherent biases within the system. An unbiased presentation can be very persuasive, especially when the answers are useful and generally trustworthy. However, all LLMs still experience hallucinations of confidently generating false or meaningless information, further highlighting the need for critical evaluation of the output.
While possible hallucinations and incomplete/biased training data make neutrality impossible for a human-created system, a neutral tone of performance makes it seem quite the opposite. Source material, even data-driven content, can be misleading depending on its packaging and presentation. Users often want a quick marker of reliability, and while the objective presentation of an AI overview provides a semblance of reliability, they ultimately prioritize convenience and quick judgment over in-depth analysis. Ultimately, the AI overview is designed to encourage a quick superficial review over in-depth investigation, and a neutral tone ensures that users do not dwell too long anywhere or think too deeply about anything in order to feel that the results are “true.”
Recommendations
Because of the potential for significant impact on user behavior and, as a result, the information ecosystem, I recommend the following:
Having access to robust user data for researchers is essential to understand how the information ecosystem evolves and the transformation of information consumption patterns that impact everything from the economy to democracy. Making such data available for broad academic research is paramount to fostering ethical search interfaces, prioritizing transparency, and encouraging tools for users to access and critically evaluate original source materials. Relatedly, schools should develop robust curricula that include teaching students how to search critically using generative AI-powered information discovery tools. Information literacy has historically been taught unevenly, so developing curricula quickly and swiftly is crucial to empowering users to use this technology responsibly. Google should constantly monitor and evolve its interface design to help users understand the limitations of its AI overview. Google should also analyze user data with the goal of helping users not only get instant answers but also fostering thoughtful online search behavior.
This overview of AI is just scratching the surface of how generative AI will impact the online information ecosystem. Because this is just the beginning, it’s essential to have meaningful conversations early on about how users are adapting their behavior and how that will impact the online information environment. Doing so will help us continually understand how the system works and what the impacts will be for the people who use it.
