If life feels too hard, talk to Claude.
From soysuwan123
When I think about the challenges of understanding complex systems, I often recall something that happened during my time at TripAdvisor. I was helping the machine learning team with analytics to understand what customer behaviors predict high LTV for the growth marketing team. We worked with a talented PhD data scientist who trained a logistic regression model and outputted the coefficients as a first pass.
When we looked at the results of the analysis together with the Growth team, they were perplexed: logistic regression coefficients are hard to interpret because they’re not linear in scale, and the features that proved to be the most predictive were not ones the Growth team could easily influence. We all sighed for a moment and then opened a ticket for a follow-up analysis, but as is often the case, both teams quickly moved on to the next bright idea. The data scientists had some high-priority work to do on our search ranking algorithms, so the Growth team essentially threw the results of the analysis in the trash.
I still think about that exercise: Did we give up too soon? What if the feedback loop had been tighter? What would have happened if we had both kept digging? What would have become clear on a second or third pass?
The above anecdote describes an exploratory analysis gone wrong. Exploratory analysis differs from descriptive analysis, which only aims to describe what is going on. Exploratory analysis aims to gain a deeper understanding of the system rather than a clearly defined question. Consider the following types of questions you might encounter in a business context:
Notice that exploratory questions are open-ended and aimed at improving understanding of a complex problem domain. Exploratory analysis often requires more cycles and a closer partnership between the “domain expert” and the person actually performing the analysis, who is rarely the same person. In the anecdote above, the partnership wasn’t close enough, the feedback loop wasn’t short enough, and not enough cycles were allowed.
Because of these challenges, many experts recommend a “paired analysis” approach to data exploration. Similar to pair programming, in paired analysis, analysts and decision makers collaborate to conduct exploration in real time. Unfortunately, resource and time constraints rarely allow for such close collaboration between analysts and decision makers.
Now think about the organization where you work. What if every decision maker was paired with an experienced analyst, who could give them their full attention and ask follow-up questions at will? What if that analyst could easily switch contexts, free-associating ideas and hypotheses and following his or her partner’s stream of consciousness?
This is the opportunity that the LLM offers to the field of analytics, the promise that anyone can perform exploratory analysis with the help of a technical analyst.
Let’s see how this manifests in practice. The following case study and demo shows how decision makers with domain expertise can effectively collaborate with AI analysts who can query and visualize data. We compare the data exploration experience with ChatGPT’s 4o model to manual analysis with Tableau, which also acts as an error check against potential hallucinations.
Data Privacy Note: The video demo linked in the next section uses a fully synthetic data set that aims to mimic realistic business patterns. For general privacy and security notes for AI analysts, see Data Privacy.
Imagine this: you’re a busy executive at an e-commerce apparel site. You have an Executive Summary dashboard of predefined high-level KPIs, but one morning you look at it and see something that concerns you: your marketing revenue is down 45% month-over-month, and it’s not immediately clear why.
Your mind is racing in several directions at once. What is causing the decline in revenue? Is it just a problem with a particular channel? Is the issue isolated to a particular message type?
But beyond that, what can we do? What’s been working well lately? What’s not working? What seasonal trends are you seeing around this time of year? How can you take advantage of them?
Answering such open-ended questions requires conducting moderately complex multivariate analysis, exactly the type of task that an AI analyst can help with.
First, let’s take a closer look at the alarming decline in month-over-month revenue.
In this example, we see a significant decrease in overall revenue that can be attributed to marketing efforts. As an analyst, there are two parallel schools of thought to begin diagnosing the root cause:
Split the overall revenue into multiple input metrics.
Total messages sent: Were you sending fewer messages? Open rate: Were people opening these messages? In other words, was there something wrong with the subject line of your message? Click rate: Was your recipient less likely to click on your message? In other words, was there something wrong with the content of your message? Conversion rate: Was your recipient less likely to buy after clicking? In other words, was there something wrong with the landing experience?
Separating these trends across different categorical dimensions
Channel: Was this issue observed on all channels or only a subset?Message Type: Was this issue observed on all message types?
In this case, with just a few prompts, the LLM can identify that there was a big difference in the types of messages sent during these two periods, i.e., the 50% sale ran in July but not in August.
Now, price drops are OK, but you can’t run 50% off sales every month. What else can you do to get the most out of your marketing touch points? Look at your highest-performing campaigns and see if anything other than sales drives makes it into the top 10.
Data visualization tools support point-and-click interfaces to build data visualizations, and now tools like ChatGPT and Julius AI can faithfully replicate the iterative data visualization workflow.
These tools leverage Python libraries to create and render both static data visualizations and interactive graphs directly within the chat UI. The ability to tweak and iterate on these visualizations using natural language is incredibly smooth. The introduction of code modules, image rendering, and interactive graph elements brings the chat interface closer to the familiar “notebook” format popularized by Jupyter notebooks.
In many cases, with just a few prompts, you can set up a data visualization as quickly as a power user of a data visualization tool like Tableau — and in this case, you didn’t even need to refer to the help documentation to learn how Tableau’s dual axis charts work.
Here we can see that “new” messages generate higher revenue per recipient, even at higher sending volumes.
“New Arrivals” seem to be popular, but what types of new arrivals should we be sure to introduce next month? We’re approaching September and want to understand how customer buying patterns change during this time of year. Which product categories are expected to increase? Which are expected to decrease?
Once again, a few prompts gave me a clear and accurate data visualization, and I didn’t even have to understand how to use Tableau’s complex quick table calculation functionality.
Now that you know which product categories are likely to increase next month, you may need to adjust your cross-sell recommendations. So if men’s athletic outerwear is going to see the biggest increase, how do you know what other categories are most commonly purchased alongside those items?
This is commonly referred to as a “market basket analysis,” and the data transformations required to perform it are somewhat complex — in fact, it’s virtually impossible to perform a market basket analysis in Excel without using cumbersome add-ons — but in the LLM, all it takes is a moment of pause to clarify your question.
“GPT, for orders that include men’s athletic outerwear items, what product types do the same customers purchase most frequently in the same cart?”
The demo above provides an example of how LLM can support data-driven decision making at scale. Leading enterprises are recognizing this opportunity, and the ecosystem is rapidly evolving to incorporate LLM into their analytics workflows. Consider the following:
When OpenAI released its “Code Interpreter” beta last year, they quickly renamed the feature to “Advanced Data Analysis” to align with how early adopters were using the feature. With GPT4o, OpenAI now supports rendering interactive charts. This includes the ability to change color coding, render tooltips on hover, sort/filter charts, select chart columns and apply calculations, and more. Tools like Julius.ai have emerged to specialize in key analytics use cases and can access multiple models as needed. Julius has access to both OpenAI and Anthropic models. Providers are expanding from static file uploads to Google Sheets connectors and more advanced API options, making data sharing increasingly easier. Tools like Voiceflow have emerged to support AI app development focused on Search Augmented Generative (RAG) use cases (e.g. data analytics). This makes it increasingly easy for third-party developers to connect custom data sets to various LLMs across providers.
With this in mind, let’s imagine how BI analytics will evolve over the next 12 to 24 months. Here are some predictions: