Synthetic Insights With GPT-4
How will AI change the way marketers find insights?
Spoiler alert: this post comes with a GPT-4-powered “synthetic insights generator” that you can try out here.
I've long been fascinated by the role insights play within marketing and advertising. Most marketers will tell you that insights are a quintessential ingredient in successful campaigns. Yet, if you ask a dozen of industry pros what makes for a good insight, you'll likely get a different answer from each one. It’s not surprising that brands will spend countless hours and millions of dollars to unearth these elusive, difficult-to-define drivers of marketing success.
How might finding insights change with the advent of mega language models like GPT-4? There must be a more effective way to discover insights than the months-long process that takes place within brands and agencies today.
At Addition, we've been exploring the future of insights. In this post, I'll demonstrate a data-driven approach to insights discovery and share how this approach can scale to help teams discover high-quality insights faster.
Before I do, I want to clarify that this is not a post about how to coax ChatGPT into generating insights. There are plenty of resources on ChatGPT prompt-hacks, including half of my Linkedin feed. While these approaches have their merits, they require a great deal of patience, lack interpretability, and can't be automated. So while some of what we share here can be reverse-engineered by the ChatGPT enthusiast, we will focus on a solution that can be automated and scaled via an AI system while still affording humans the ability to steer the AI strategically and creatively.
Synthetic Insights
In a nutshell here’s our approach to generating synthetic insights with GPT-4:
Ground the AI in “source of truth” data. Controlling the AI’s generative powers with high-quality data sources enables you to get to insight that are backed up with verifiable information versus speculation from a black box. High-quality data can also lead to higher quality insights. Data sources can include anything form journalism, to social listening, to PDFs, power points, 1st party research, or all of the above.
Give the AI examples of the insights you want. If humans can’t agree on a good insight AI certainly will miss the mark when you ask it for one. We’ve found that the most effective and versatile way to accomplish this is to show GPT-4 a handful of high quality examples to teach it what you’re looking for along with a stylistic description.
Scale the process of going from data to insights. There are different ways of doing this, from human assisted (as you’ll see in the demo), to more autonomous methods. For an AI-driven approach to bear fruit, you’ll want to generate lots of synthetic insights and then hone in on the juicy nuggets that will work best for you.
How It Works
To demonstrate this AI-powered approach to insights we’ve created a “synthetic insight generator” that autonomously conducts desk research and uses the content it finds to generate insights. This system is trained on examples of marketing and advertising insights we curated. Each example is a pithy, 1-2 sentence revelation about a particular subject that help you see it from a different perspective. By constraining all of our training examples to a similar format, the the AI learns the output format we’re after.
To demonstrate how it works, let's have it come up with an insight about the rapidly evolving world of Generative AI.
Me: How will generative AI will impact the future of brands
The AI generates search queries and begins reading news articles to collect information related to the subject. Once it feels like it's conducted enough research, it uses the most relevant information to synthesize an insight.
AI: While brands have long relied on human creativity to differentiate themselves, generative AI offers the potential to elevate brand identities by harmoniously blending human and machine-generated content, ultimately unlocking a new realm of brand expression that has never been seen before.
The AI then goes on to cite the sources it used to generate the insight.
Verbatims:
1. "Generative AI has multiple use cases across different industry verticals and, in marketing especially, can enable brands to accelerate the delivery of personalized marketing content while adhering to the brand's writing style and tone." - Forbes, "Generative AI In Marketing: 5 Use Cases" (https://www.forbes.com/sites/forbescommunicationscouncil/2023/04/03/generative-ai-in-marketing-5-use-cases/)
2. "Automation is ideally about unlocking human potential to do tasks differently and do different, higher-value tasks. In our book, Human + Machine: Reimaging Work in the Age of AI, we detailed how leading companies were using artificial intelligence to augment human capabilities, not replace them. The same dynamic will drive the use of generative AI as companies find even more creative ways to tap the power of human-machine collaboration." - Harvard Business Review, "Generative AI Will Enhance — Not Erase — Customer Service Jobs" (https://hbr.org/2023/03/generative-ai-will-enhance-not-erase-customer-service-jobs)
Here is a demo that you can play with at home.
Just like humans, this AI won't come up with earth-shattering insights all of the time. But if you're patient and willing to iterate with your prompting, it's likely to surprise you with what it can do.
Scaling It Up
To reap the benefits of AI-powered insights, you'll want to scale things up. This means giving the AI more data, generating more insights, and getting the model to identify the best ones. Doing this requires more time and effort, but it's the way to go for high-stakes projects.
Here's how to approach generating synthetic insights at scale:
More Data: First, you'll want to curate an extensive, strategic data set that aligns with your objective. For example, if you're looking for cultural insights, you may lean into news and social listening, whereas for consumer insights, you may want to use 1st party survey data or IDIs. Using search embeddings for information retrieval, you can scale your data sources well beyond the token limit of the GPT-4.
More Insights: To achieve scale, you'll want to get the AI to generate lots and lots of insights on its own. Automated Prompt Engineering (APE) can enable the AI system to generate prompts based on your brief and some examples. From there, we can instruct the system to run until it's generated a desired number of potential insights.
Identifying the Best Ones: Instead of reviewing a hundred insights, you'll want the AI to curate the best outputs for human review. By instructing the AI system to rank the outputs using the original examples as a benchmark, GPT-4 can do this work for you. Additional considerations can also be incorporated here, such as rating the quality of the source materials used.
Up Next: Qualitative Insights at Scale
Conventional industry wisdom suggests that marketers must accept trade-offs between quantitative and qualitative methods. Quantitative methods come with some degree of rigor and verifiability. But they lack nuance and creative juice. Qualitative methods are juicy, but verifying them at scale is prohibitive. Generative models like GPT-4 can defy this, enabling marketers to find nuanced creative insights backed by data at scale.
In the coming weeks, we'll share more examples and demos about how we're working with brands to scale these approaches up.
In the meantime, don't forget to take our synthetic insight generator for a spin and reply to this email with any questions or comments.
Want to learn more about Addition?
Read about us in the Wall Street Journal
Follow us on Twitter
Reply to this email to speak with a human