ROI for Social Media Marketing: It's Complicated

IStock_000006788336XSmallAt a recent Social Media Breakfast in Boston, Harvard Business School associate professor Andy McAfee had this to say about the age-old Return On Investment (ROI) question:

“There is not enough ROI for figuring out ROI.  It is an intellectually bankrupt exercise.”

Got that?  McAfee is not saying that measurement is not important, he suggests instead that it is simply not worth the trouble!

To be clear, when McAfee talks about the challenges in determining ROI, he is referring to Info Tech spending, as laid out in two posts published back in 2006.  According to McAfee, the sum of his argument was succinctly laid out in his Harvard colleague Bob Kaplan’s seminal “Strategy Maps” tome.  I am going to excerpt and bastardize Kaplan’s text, for my fellow marketers:

“No marketing program has value that can be measured separately or independently.  The value of marketing derives from its ability to help the organization implement its strategy … Marketing programs seldom have a direct impact on financial outcomes such as increased revenues, lowered costs, and higher profits.  (Rather,) Marketing affects financial outcomes through chains of cause-and-effect relationships.”

This is even more true in the Social Media era.  Jason Falls wrote a rock-solid – and highly popular – post about Social Media ROI last month, containing this gem:

“The problem with trying to determine ROI for social media is you are trying to put numeric quantities around human interactions and conversations, which are not quantifiable.”

Or as David Meerman Scott is fond of saying, “What is the ROI of putting on your pants?”

 

At this point you’re thinking that I am anti-measurement.  I’m not.  I believe in measurement-by-objective: once you know which needle you want to move, decide how to make it move and how to keep track of progress.

The brilliant Peter Kim, a former Forrester analyst, outlined a framework for measuring Social Media in a September post:

  1. Attention.  The amount of traffic to your content for a given period of time.  Similar to the standard web metrics of site visits and page/video views.
  2. Participation.  The extent to which users engage with your content in a channel.  Think blog comments, Facebook wall posts, YouTube ratings, or widget interactions.
  3. Authority.  A la Technorati, the inbound links to your content – like trackbacks and inbound links to a blog post or sites linking to a YouTube video.
  4. Influence.  The size of the user base subscribed to your content.  For blogs, feed or email subscribers; followers on Twitter or Friendfeed; or fans of your Facebook page.

2909071349_8c1b3a6f76_oThere’s an “x-factor” that comes into play well:  Sentiment.  The spirit driving user participation matters.  The net result of these adds up to a score for social media engagement.

So what’s the monetary value of a visit, comment, link, or friend?  Well, the only honest answer is “it depends.”  Only you know how much these interactions matter to your brand, regardless of industry, channel, or competitive results.

The just-as-brilliant current Forrester analyst, Jeremiah Owyang, also offers a whole series of posts related to the measurement issue – but my favorite of the group notes the fundamental difficulty:

462007028_b840d0be18_m(Until) we can measure the impact of a conversation between an employee and a prospect at a coffee shop, it (will be) difficult to measure social media…

What are you trying to accomplish? … (The) trick is to figure out what your goal is first – is it to spread a message among a community? Is it to reduce support costs? Is it to learn from your community? In each of these cases you’ll have to then assign the right attributes to measure against.

So, yea, basically it’s tough to measure ROI for Social Media. This doesn’t mean it is not worth doing!  It just means that the justifications that professional marketers will make to qualify for budget are as likely to be anecdotal as analytical/quanitifiable.

IStock_000003845983SmallWhat is the dollar value of responding humanely to an angry customer blogger in their public forum, or, say, on Amazon.com?  Have you made that customer happier?  Maybe.  Have you made other visitors feel good about your publicly-displayed good intentions and responsiveness?  No doubt. 

But did the effort motivate more sales?  How could you tell?  There is no “I am buying this product because ____” form in the Amazon Checkout Cart. 

How much money do you save by having the PR agency monitor Amazon.com, or the blogosphere, so that your customer support reps can instead be on-the-phone with live end-users?  There could be a formula for that, but I’m no math whiz. 

IStock_000003845983SmallHow much value can you place on the fact that the professional communicators at a PR firm trained the Customer Support reps on tone, responsiveness, etc., before letting them loose to represent the brand in the blogosphere?  You can’t measure the value of something that didn’t happen (i.e., a blow-up in the blogs based on a poorly-trained public spokesperson). 

Doth I protest too much?

Ironically, for all the hemming and hawing, it is sometimes easier to show “needle movement” via traditional PR.  For one start-up client, “new users” was the metric we were asked to improve.  They came to us with a base of 5,000 alpha users, and our outreach to traditional and social media outlets led to an additional 200,000 users in just two weeks!  How’s that for ROI?

Lessons?

Know your objectives in advance.  Start small, when possible: think “proof of concept.”  Track the metrics obsessively.  Make sure that your interactive marketing efforts are tied to the sales funnel (e.g., your vp of sales ought to be able to determine where most leads are coming from online).  Report frequently. 

Lather, rinse, repeat.

 



Posted on: November 17, 2008 at 12:47 pm By Todd Defren
31 Responses to “ROI for Social Media Marketing: It's Complicated”

 

Comments
  • “measuring ROI is a fool’s errand.” I disagree completely. Measuring ROI is the backbone of a well executed marketing plan!

  • Excellent article, thanks for the insightful information. It certianly will take hard data to convince many that social networking will increase the bottom line, but even now some of the nay-sayers are coming around.

  • Excellent article! I especially like the quote from Jeremiah Owyang. Trying to measure ROI in Social media is just like trying to put a dollar value on the family photo album, it all depends on who is looking at it.

  • Tom Zoethout says:

    It is still an actual problem. It would be great if you were able to give an update on this, a year after…
    But great article.

    Tom

  • Great insights, and it sounds like we’re all getting these questions from current and potential clients. I think we definitely need to strike a balance between defining what metrics are meaningful and going after those, and being open and honest that there are some social interactions that are difficult to put a dollar amount on.

  • No, no, no, no and no. With all due respect to Dr. McAfee, he doesn’t know what he is talking about. That is the problem with many academics: Too much time in the classroom or the lab, and not enough time actually doing the work in the trenches.

    1. ROI is media-agnostic. The method for measuring ROI is the same regardless of the media… or the activity or exercise. With ROI, you measure the financial-impact of an activity or series of activities. It’s purely a P&L exercise in which ROI is either a cost reduction or an increase in revenue. Period. When a change in the existing strategy or tactical execution of a program also creates a change in either revenue or cost control, you’re at the start of an ROI conversation.

    2. R.O.I. is a financial calculation. Things like attention, authority, participation and influence are “impacts”, not R.O.I. Do they play into the connect-the-dots process of action–>reaction–>impact–>ROI? Yes. So measuring the right things matters, but they do not in and of themselves represent ROI. Not in this or any universe.

    3. The scientific method applied to business is the bulk of the process that leads to the determination of the ROI of an action (or series of actions). All things remaining the same (set up a control group to test your theory if you must), a change in the system resulting in a change in behavior resulting in a change in revenue is pretty easy to spot. Whether correlation or cause-and-effect, connecting the dots between action, reaction, impact and financial deltas isn’t hard.

    4. The actual ROI calculation (money in, money out) is the last step in the connecting-the-dots process and is VERY simple. If you know how much a program has cost and you can calculate the delta in your P&L’s outcome, you’re 80% of the way there. Any first year MBA student should be able to do this.

    The fact that so many “experts” don’t understand how to calculate ROI does not mean that the ROI of Social Media cannot be calculated. It can, and it is. You just can’t rely on “experts” whose background has been in questionable measurement to begin with (media and/or marketing measurement) to suddenly figure out that what they have been measuring all these years was either irrelevant, secondary to real business measurement or in some cases completely bogus. (Impressions? Really?) Statements like Dr. McAfee’s betray a lack of understanding of the process rather than validation of his uninformed opinion. Don’t fall into that trap, Todd. These people are dragging you down the wrong path with this BS. The Earth isn’t flat, I swear. ;D

  • Yeph – it WAS complicated

    @KDpain – were were not sure either. But there are good reasons to do so.

    The Social Media Academy worked on a research to identify a method to calculate ROI that is mathematical correct (currency on both sides of the equation) and is relevant to a CFO.
    CM / IC = SM-ROI
    You find the details here: http://xeeurl.com/A0567
    And a discussion on Social Media Today here: http://xeeurl.com/A0568

  • Pt says:

    I totally disagree with the statement that “Marketing programs seldom have a direct impact on financial outcomes such as increased revenues, lowered costs, and higher profits.”

    I think people are tending to put marketing into a bucket of white paper creators and creative staff. Marketing IS the thing that generates revenue, etc.

    I also think that things like white papers, etc. CAN be measured to impact revenue – the problem is that marketing is so poorly organized and operationally immature, often subordinate to sales who is caught up in their own oedipus complex with marketing, that they give away the farm without focusing on results, let alone measuring them.

    Just my 2 cents.

  • PR Squared says:

    PR Helps Raise Venture Capital

    Wanna talk (yet again) about the ROI of Social Media & PR? Let’s ask the startup companies who rely on the VC community’s support.  Did PR help them net funding? A survey sponsored by BIGfrontier Communications Group asked 300 U.S….

  • excellent video…ROI is still the largest challenge for most social networks.

    research, plan, execute and measure….

    sandy
    megastarmedia.com

  • Coach Adam says:

    I also agree with Mike and would take it one step further. The ideas mentioned above from Jeremiah are spot on “is it to spread a message among a community? Is it to reduce support costs? Is it to learn from your community?” and each of those are tied to value drivers and KPI’s of the business. Then, if we focus on the business processes, the various tactics we employ (social media, traditional marketing, tho9ught leadership, or others) can be more readily and effectively measured against impact and results.

  • KDPaine says:

    Great discussion going on here. When I first read your post, I almost nominated Andy McAffee for Menace of the Month — because if measuring ROI of social media isn’t worth it, then what am I doing in business? But in reality, I tend to agree with him.

    There are some things that can’t and shouldn’t be measured— like your pants — and Shel Israel said that long before David Meerman Scott, by the way. But I’m sure when pants first came on the scene, there probably WAS a discussion about whether they were a good or a bad idea.

    The telephone sparked endless debates about ROI, as well.

    Yes, at some point we can stop worrying about measuring these things because it will simply be an accepted fact that yes, you need to listen to your customers on Twitter, or wherever. But right now, when organizations are trying to figure out what IS working and is NOT working, and what old fashioned tactic can be dropped in favor of a new approach, measurement is critical.

  • Andy Angelos says:

    Agree with Mike – measuring social media campaigns or any other advertising, outreach platform is imperative for success. A more critical aspect, however, is altering the conceptions of success for a clients. As the quote from Jeremiah mentions, “(The) trick is to figure out what your goal is first – is it to spread a message among a community? Is it to reduce support costs? Is it to learn from your community?”

    If you are wanting pageviews to your website, you shouldn’t be surprised when that facebook group fails deliver. There is still a lot of education necessary before social media metrics can reach the status of nielsen, zagat, or other entrenched rating systems.

    Andy

  • Dave Peck says:

    Great Post. I do think though there are ways to measure Social Media results once you figure out what your goals are. Though the items you list are very very important.

  • Frank says:

    I live in a world of metrics and performanced based advertising and media models but I always keep in mind the words of Tim Ambler, Senior Fellow, at the London Business School

    “Marketing Metrics create the illusion of certainty and control.  

    Cannabis has much the same effect.” 

  • Todd, it’s true, measuring ROI can be tricky. And there is a myriad of other metrics that companies should be looking at as well.

    As David Meerman Scott mentions in his video, companies should be measuring how many people are exposed to their ideas, how many people are talking about their products/services or employees and so on.

    But there are also tools out there that allow measuring ROI for certain campaigns. And it’s pretty easy. I’m not associated with any of the companies – just a happy customer, but Salesforce, HubSpot and Google Analytics work very good, especially when they are combined.

  • Mike Spataro says:

    Todd,

    Go ahead and call me jaded because I work for a company that measures social media for brands, but companies are getting smarter in this space everyday and anyone in marketing who thinks that their social media programs can’t and shouldn’t be measured is just setting themselves up for problems down the road.

  • Jed says:

    Excellent post. We just delivered a Facebook outreach strategy to a client. We recommended using a Facebook Page, but pointed out that a lot of the metrics we can look at on Facebook don’t necessarily point directly to business objectives.

    We don’t think we’ll have much trouble helping them get fans of their page, but we wanted to make sure that they look beyond purely social numbers to measure success. As you say, it’s complicated. We’re telling them to think qualitatively, examining the nature of conversations in addition to the number of conversations.

    As Michael Daehn says, this is how you build goodwill. The value of these exercises may be intangible, but that doesn’t mean they’re insignificant. I’d trade consistent feedback and steady growth of goodwill over X huge amount of fans or followers any day.

  • Awesome Todd. I just love McAfee’s comment.

  • Matt says:

    marc’s comment brings up an interesting point, and a bit of a catch-22 in my eyes. selling social media remains difficult, even though we continue to learn more and more about working in the space.

    in my experience, the “sale” is based on demonstrating value. you can do that in various way, but it always comes down to the question, “why is this my best choice?”

    demonstrating the value of social media can be difficult as we don’t have a silver bullet approach to measuring social media ROI. Thus, the ROI conversation becomes moot because we can’t even get in the door.

    here at my agency, i’m trying to get us to move towards more social media engagement for our clients. touch to do for out heavily b-to-b client list though.

    get at me on twitter, @mdd044, if you have any thoughts to add.

  • Adrian Chan says:

    Todd,

    Good post! I like how you’re covering the topic. I have a couple thoughts I want to share on this.

    I believe that there are major, if not fundamental challenges to measuring ROI in social media marketing (PR and advertising too).

    To Andy McAfee’s point, measuring ROI is a fool’s errand. For the simple reason that the greatest gains (as noted by Jeremiah) are often internal to the organization (learning), between organizations and audiences (relationships, trust, respect, credibility, etc), and in the age-old intangibles of marketing (impressions, awareness, brand image), etc.

    What social media does by its very nature escapes easy observation — the net effects of a conversational campaign are unknown until something makes itself obvious. At which point the success story becomes a better reference point and proof of success for a company than than any numbers put behind it.

    I think the issue here is “Who’s asking for ROI?” People who count things. Which means analysts such as ourselves (we get paid for results and use tools by which to provide some measure of them) and campaign managers (used to measuring ad and web traffic, hits, etc).

    So I think the question of ROI is a question asked by those in the process of reaching a comfort level with social media marketing. It’s about creating a community of practice within and among organizations that accepts the “lost” or unobserved data but which can see value without having to see numbers.

    At the point at which we understand the value of social media campaigns, we’ll be more confident in how to tell stories, create participatory branding campaigns, and deliver value without having to measure more than baseline traffic and reach. Nobody questions the value of a web site, or of search marketing and SEO — and tracking is used to supplement but not make the case for a web presence.

    There are things we can’t measure. We’ll stop talking about ROI when we understand that we don’t need to.

    cheers,
    adrian

  • Paul Dunay says:

    Great post Todd

    I really like the line — “No marketing program has value that can be measured separately or independently” – which is really true – I think it is arrogant of a marketer to think one white paper or webinar means that anyone wants to buy anything from them

    BUT

    I think the opportunity to take and measure all marketing activities that drive to increased sales via lead nurturing is there – I have been doing that and it works

    I spoke about this on my blog here – http://buzzmarketingfortech.blogspot.com/2008/07/marketing-metrics-rethinking-them-again.html

    also whats the ROI of putting on my pants >??
    well if I am going to work – the ROI is – it allows me to keep my job!

    p

  • In the old days we called this “Good Will”.

  • Marc Meyer says:

    Todd, over the weekend I pitched a client who has a very analytical, accounting background. The pitch was for a complete rebranding a la web 2.0, with the core being a better social media presence starting with his blog. At the end of the 2 hours, I felt that I had failed in convincing him that there was roi in what we were going to do. return on engagemnet yes-but his thinking was i spend a dollar i want to make a dollar or save a dollar. Lets not forget how hard the sale is before you even get a chance to measure.

  • Rick Burnes says:

    Nice job spelling out the nuances of this discussion, Todd. Great post.

    If you’re looking for the full video of the breakfast, I posted it here: http://blog.hubspot.com/blog/tabid/6307/bid/4405/How-Do-You-Measure-the-ROI-of-Social-Media-You-Don-t.aspx

  • Chris Kovac says:

    Another great post on social media. Thanks so much!



Leave a Reply




Show some social media love would ya?





RSS logo Subscribe by Email

RSS logo RSS Feed

logo




PostRank Topblogs 2009 - #3 in PR















View Todd 

Defren's profile on LinkedIn


Brink