Upload
colin-rogers
View
305
Download
0
Embed Size (px)
Citation preview
NLP + Brandwatch AnalyticsDeriving insights from social conversations using Natural Language Processing and the Brandwatch Analyt ics API
What we will cover today
What do we want to answer? (and why)
Our approach to social data
Leveraging the Brandwatch API to extract data
Deriving insight from personas
Identifying key topics of conversation
Segmenting on those topics to develop personas
Replicating back into Brandwatch
What we’re working on next
3 |
§ Provides an in-‐situ portrait based on exhibited behavior not on elicited feedback
§ Highly relevant as it can be updated in near-‐real time
§ Enables research budget to be focused on insights rather than data collection
Social intelligence enables new ways of answering traditional business questions and driving data driven actions
What are the sort of questions we want to answer?
How can a financial services company reach out to cyclists?
How can we get small business owners to engage with their cell phone
provider online?
What is the customer journey for a motorcycle enthusiast?
4 |
There are seven stages to the analytical process of developing utilizing personas with social data
Our approach to working with social data
ExtractDevelop the dataset
Linguistic model Segment Analyze TrackQuery data
Prepare
§ Need to truly understand your data before any analysis
§ Iterative query/dataset development through virtual ethnography
§ Use the Brandwatch API to extract the full text mentions
Model
§ Employ Natural Language processing to model how people talk
§ Use either qualitative methods or clustering algorithms to segment
Understand
§ Through visualization and analysis we can understand thoughts, feelings and preferences
§ Replicate back into Brandwatchas sub-‐categories to monitor on an ongoing basis
5 |
• Provide the basis for a ‘corpus’ in NLP jargon from which to model
• We have built a library of functions using python to retrieve and format the data
• The output format of the API is in JSON so there is some work to turn it into a table we can read and use
Extracted BW data has many use cases, today we will be primarily focused on full text mentions
Leveraging the API to extract the dataset
Example API function:def get_mentions_query_URL( startdate,enddate,project_id, query_id,access_token,fullText):
query_def = "data/mentions” end_date = "endDate=" + end_date + "T00:00:00.000Z”start_date = "startDate=" + start_date + "T00:00:00.000Z"request_URL="https://newapi.brandwatch.com/projects/" +str(project_id) + "/" +query_defif fullText == True:
request_URL = request_URL + "/fulltext"
request_URL = request_URL + "?" + "queryId=" + str(query_id) + "&" + start_date + "&" + end_date + "&pageSize=5000" + "&access_token=" + access_token
return request_URL
Read more: blog.tahzoo.com/tech-‐thursday-‐brandy-‐py-‐a-‐python-‐library-‐for-‐brandwatch/
Github: https://github.com/BillmanH/brandy.py/
6 |
Linguistic model -‐ Identifying the topics in a conversation
pumpkinsugar HEALTHY LIVING
PUMPKIN SPICECONVERSATIONS
TEXT ANALYSIS
TOPIC MODEL
1 Break down each conversation into the words and sentences to probabilistically assess each word’s relationship with each other word
2 Analyze to uncover the most common “topics” of conversation
3 Run clustering analysis to segment on topics4 Iterate on topics until we develop a solid segmentation
Four steps to targeting personas
7 |
("pumpkin spice latte") NOT("vue pack" OR "value pack" OR "how to make" OR "win free" OR "latte cake" OR "black friday" OR "pack of" OR "My TL right now iOS7 Hump Day iOS7" ORsite:(twitter.comOR kdvr.com OR fox59.com OR news.google.com))
An example: who discusses Pumpkin Spice Lattes?
Our query…Excluded because of irrelevant recipes
Purposefully broad query to capture full
range of conversations
Exclude Twitter as it would overwhelm the results and we couldn’t export full text mentions
8 |
Do it Yourself Starbucks Nutrition Healthy
living Style Urban living restaurants PS recipes Amazing treat
Pumpkin Spice
ingredientsPS Flavor Coffee at
home
people pumpkin grams squash fall city binary milk love food pumpkin home
make spice fat healthy wear place victoire pumpkin time hari spice inch
things latte calories recipe fashion food restaurants coffee day babe pie coffee
life starbucks sugar food boots park options sugar good ingredients latte green
time fall registers recipes style street time recipe back sugar flavor set
thing psl data copycat color local pst spice week science flavored keurig
feel drink saturated favorite wearing free visit cup great cancer seasonal count
find coffee carbs soup dress art trading make home organic year price
years today sodium paleo black event restaurant cream made found taste mountain
world lattes pos version top restaurant september syrup work chemical food make
The topics
9 |
DIY Example:“I get annoyed when a recipe calls for pumpkin pie spice. It's not that people use it that annoys me, it's the mere existence of it as a single spice. … I guess I'm just a purist at heart. Since I haven't seen pumpkin pie spice here in France I now need to make our own pumpkin pie spice mixture, and then figure out the right proportions for my dreamboat pumpkin spice latte. Nothing that a Google search won't solve, but annoying nonetheless. And don't worry, when I do I'll be sure to share it with you. Maybe you'll even get some rainbows. Fingers crossed.
An example of how this analysis works
Treat/Reward Routine Example:“I thought splurging on a venti pumpkin spice latte would make me feel better this morning, (or maybe even the three cups of green tea with lotsa honey in it!) ...but as my ears pop, my nose runs, and my throat feels like somebody took sandpaper to it last nite, I guess it's time to finally suck it up & take some meds 󾌮ó¾�‚ I blame you! Rodney Deal!! Haha kidding kidding ; )
Below are two pieces of verbatim content that we used in our model. The first post is connected with the DIY (62% relevant) topic and the second with Treat/Reward (73% relevant)
62% 73%
DIY TREAT / REWARD PS FLAVOR FALL (SEASON) PSL RECIPES HEALTHY
LIVINGFILLER/
INFREQUENT WORDS
TOPICS:
10 |
• K-‐means highlights clusters of conversations based on the topics they discuss
• This creates a segmentation that reflects how people discuss a subject
• Keys in on the pattern of topics in a conversation
We use the k means clustering algorithm to segment the conversations based on the topics in order to create the personas
Segmenting on the topics
11 |
UrbanLiving
Fall (season) Dessert
Starbucks drinks
Pumpkin flavor
Treat / reward
Pumpkin (recipes)
DIY
Fall (season)
Treat / reward
Pumpkin(flavor)
Dessert
StyleDesserts
Treat / Reward
Being Healthy
Urban Living
Fall (Season)
Starbucks Drinks
Fall (season)
DIY Desserts
Pumpkinspice recipes
Treat/ Reward
LESS IMPORTANT
MORE IMPORTANT
Grouping the topics that are core to each segment we can see where differences break down
Mapping topics to personas
12 |
Plotting continuums to understand the personas
Why they like it
What it stands for
NOVELTYNOSTALGIA
GUILTY PLEASURE
DAILY RITUAL
OPPORTUNISTICTRADITIONAL
PERENNIALSEASONAL
OFTEN
OCCASIONAL
EXPECTED
EARNED
13 |
What we found
22%
PSL PAMPERER A pumpkin spice latte is a treat to be savored
after it’s earned or after a tough a Monday morning,
“What a weekend. Hello, slow Monday. Oh what's that? I should get a pumpkin spice latte? Well, if
you insist...”
34%
LATTECHEMIST
They make their own lattes in the comfort of their own home or tinker with the official version
“Here is an awesome home version of Starbucks Pumpkin Spice Latte. Very simple to make and alotcheaper… personally I like it better because you control the amounts of ingredients you put in it
according to your taste.”
38%
FALLFANATIC Pumpkin spice is part of what makes fall special
for them, a pumpkin spice latte is one part of their fall tradition
“Pumpkin Spice Latte at Panera. Oh yeah, I need one of those! Bring on fall! Looking forward to bonfires in my fire pit and my newly refinished
fireplace.”
6%
PUMPKINTRADITIONALIST Loves everything pumpkin from pumpkin pie to
lattes, fall is just an excuse to get their fix of pumpkin
“Are you ready for a Pumpkin Spice Latte!?!?! Or how about a Pumpkin Bar???? Well tomorrow they
both will be available!!!!”
14 |
Replicating back into Brandwatch
PSL PAMPERER
“morning treat” OR “Savedmy morning” OR ((rough OR bad OR terrible* OR awful OR stressful)) NEAR/4 (morning ORday OR week))
LATTE CHEMIST
((myOR I OR Mine OR “made a”) NEAR/2f (organic OR make OR recipe OR mixture)) OR homemade OR “the perfect” OR ((coffee) NEAR/3 (dessert OR “sweet tooth”)
FALL FANATIC
(I OR MY) NEAR/3 (“love fall” OR “finally here” OR “the season” OR autumn) OR ((making OR made OR bake OR baked) NEAR/4f (cake OR pie OR pastry))
PUMPKIN TRADITIONALIST
((pumpkin) AND (candle OR products OR cake OR pie)) OR“pumpkin flavor” OR ((“I need a” OR “must have” OR “must get”) NEAR/3f (latte))
We conduct a careful qualitative analysis of persona mentions to translate the topic model into Brandwatch rules
• Allows us to visualize and track in Brandwatch• Create each persona as a sub-‐category• Creating the persona rules are iteratively written
Hypo
thetical ru
les
15 |
Custom geo-‐mapping for DMA’s
Persona use cases
Typing tools
Scoring conversation relevance
IDENTIFYING TARGET SEGMENTS
Commentator
DIY PS Flavor Fall (Season)
Treat / Reward
Focused on others
Traditional
16
Next level – what we’re working on now
§ Ability to use the model to tag incoming mentions in Brandwatch
§ Determining demographic characteristics from language
§ Utilizing topics to predict outcomes
Thank you
Bil l Harding – Data Scientistbil [email protected]
Colin Rogers – Direction of Content [email protected]