How good are Social media monitoring vendors at searching… well, Social sites?

By on March 3, 2011 in Tools & Technologies

I will generalise and not refer to any particular vendor I’ve tried, it’s also a very non-scientific post, however my conclusions are not without reason. If you scan the market for Social media monitoring tools you soon realized the plethora of providers. These range from free to use twitter dashboards to state of the art monitoring tools. For the ones claiming that they are social media monitoring tools, we assume this means that you get a good overview of tweets, public facebook posts from profiles, youtube movies and comments etc, how good are the tools really?

Let’s start of with a list of things we want to see in-order to accept them as monitoring social sites, and not just traditional webpages and rss-feeds. As you would monitor a set of keywords we want to see data from these keywords being fetched from:

  • Facebook posts, comment and updates from publicly available content. Meaning all content from users not having their privacy settings set to showing content for “Everyone”
  • Tweets
  • Images and comments on such from picture sites
  • Videos and comments on such from video sites
  • Public posts, comments and updates on other  social networks such as LinkedIn, Hyves, Mixi etc, all depending on where in the world you operate
  • Blog posts and comments
  • Forums

In other words, you want to be able to fetch as much keywords as possible from the public data in the Conversation prism, but let’s start of slowly and look at Facebook, twitter and Youtube i.e the big players.

My general conclusion around social media monitoring on Facebook:

I should start with saying that I’m not a programmer nor an analyst, although I know my way around the Internet. So I want to make it clear that these are my assumptions based on what I’ve seen.

The first thing we need to understand about Facebook is that far from all content is possible to index due to people privacy settings. I saw on a blog post one comment saying that in Australia only around 3% of the Facebook accounts are set to “Everyone”. Ryan Strynatka from radian6 elaborates further on the topic and what he claims Radian6 to cover.

The way social media monitoring vendors seems to get the data is via Facebook’s Social Graph API. This allows anyone to pull data from Facebook, again, only data that is public. An easy way of making a keyword search on Facebook is by typing in https://graph.facebook.com/search?q=watermelon&type=post in your browser where “watermelon” is your keword. This will return search results from peoples posts. So far so good, right?

Here are my problems:

  • Some of the most well known social media monitoring vendors don’t even return this search data but less. A very random sample that I cannot figure out.
  • The ones returning the same data coming directly from the Facebook API seems to completely have missed that Facebook delivers different search results depending on your language settings for the profile your are logged into. If I want to see the search results I need to log in to Facebook or else my returned search data is less, however I’m still only looking at public data not my friends or connections. How does the major social media monitoring vendors solve this? It seems like they only return results for one language setting which may very well explain why the results are so limited.  Any one from the big vendors reading this, please elaborate and explain!
  • The data that is returned sometimes lack influence data (apologizes for using the term influence incorrectly) which is highly interesting in order to value the search.

Social media monitoring on Twitter seems to be the easy part:

Everyone seems to handle twitter fairly good. They pull the data straight from the Twitter Search API, the “fire-hose”. You would generally get some insights such as number of followers and/or klout-score from the twitter user who mentioned you. You’d also most of the time have the possibility to reply straight from the tool, this is if they provide you with an engagement console.

Youtube and other video sites:

Now it starts to be difficult again. Well, we’re a traditional web monitoring company that just put “social” in our tag-line the last year so wee need to provide some search results from Youtube… hm. Search video header and description text seems to be easy for everyone, but when it comes to delivering results of keywords in comments, channel profiles etc, well then you’re disappointed. Again, getting the video metrics data such as number of views and ratings is highly interesting.

Other social networks:

LinkedIn, Hyves? Nope, generally forget it.

Blogs and forums:

Blogs is what social media monitoring vendors do good, or actually really good. Forums as well if they have RSS-feeds. generally they seem happy to look at trying to add more sites by request.

So, adding up my absolutely non-scientific post about social media monitoring I draw the very simply conclusion that the tools are still in their very early stages of integrating “social” but fast to claim that they cover it.

My tip for you; do your homework carefully and have a clear idea of your monitoring needs and have a look at what Forrester Research are saying about Social media monitoring.

, ,

  • http://www.peppervirtualassistant.com/ Sarah De Leon

    Yes. It seems that the social media monitoring is still vague on the data they present. I’d say it’s because of the privacy concerns of many users. But, for FB, they made use of all information so that advertisers can use the demographics to target the market that they want. Maybe in the future these social media accounts will use these private information at their own business advantage.

    • http://joakimnilsson.com Joakim Nilsson

      I think it’s a great thing that the monitoring companies seem to show respect for each networks privacy settings. However, which Facebook in particular, most vendors I’ve tried don’t even return the full amount of data that’s available from the Social Graph API.

  • Samuel

    Very good points, Joakim.

    Covering the complete web (all social media websites) is non-trivial. Even Google misses out on blog comments and long-tail of less well-known websites.

    The idea is to cover enough to spot any threats, opportunities before its too late.

    Best,
    Samuel

  • http://www.dayngrzone.com/ Dayngr

    Joakim, as you mentioned coverage of Facebook is built on Facebook’s Graph API and that means results from Facebook only represent a snapshot of the potential total brand mentions. If a mention is posted in a closed or private area of Facebook, we won’t be able to include it in your results. That will absolutely impact the way you can measure and account for Facebook results in your monitoring and measurement activities.

    As for languages, what data each user pulls in depends on how they build their “topic profile” and they have the option to bring in results from at least 15 different languages.

    I hope I understood correctly and that my humble reply helps a bit more. If not, please feel free to reach out and I’ll do my best to help.

    Trish (@Dayngr)
    Community Manager | Radian6

    • http://joakimnilsson.com Joakim Nilsson

      Thanks for stepping in Trish (I love your Facebook song btw) – I think this comment in Jeremiah Owyang’s Google group “Social CRM pioneers” sums it up very well: https://groups.google.com/d/msg/social-crm-pioneers/Xx5i4Yd1DO8/NDdM6UmK_pUJ

      I had a demo with Radian6 recently but I cannot say anything about the coverage as I have not yet played around with it. What is still evident though, as confirmed by many other users, is that actual coverage from what we as users consider as social content, is fairly poor for some vendors.

      I’d love to see a search data comparison chart from Radian6 where you test yourself against the competition and prove in a channel table that you’re superior.

      • http://www.dayngrzone.com/ Dayngr

        Thanks Joakim, I thought people would get a kick out of that song.

        With regard to data comparisons, Radian6 pull in data from over 150 million sources daily (see: http://www.radian6.com/get-started/faq/#coverage) but, it’s important to note that our software only pulls in social media-specific sites using our own special mixture of RSS feeds, proprietary crawlers, and access to APIs like the Twitter firehose (see: http://www.radian6.com/platform-blog/2011/02/dear-radian6-howd-you-get-that-data/). Additionally, we respect the privacy, TOS and rights of other sites – meaning that if they’ve regulated crawler access, we don’t pull in that information – which may be why you’re seeing a difference in the data being returned.

        I would like to add that if you have a specific area of coverage that you’re looking for, let us know. We’re open to feedback and have a dedicated product manager for data coverage.

        I do hope this provides a bit more insight!

        All the best,

        Trish (@Dayngr)
        Community Manager | Radian6

        • http://joakimnilsson.com Joakim Nilsson

          Thanks Trish – I’ve been in touch with your people by e-mail over the last couple of days regarding your actual coverage of Facebook. Your colleague Ryan states in his blog post that “public wall posts or status updates” are covered. That is true for the US English region settings in Facebook, but from what I’ve seen your do not pull in data from the rest of the +60 regions in Facebook. This means that Facebook monitoring in Radian6 is more or less insignificant for business outside Europe.

          Now I’d be very interested if this is due to what you refer to as privacy, TOS and rights of other sites or just a technical difficulties of crawling the data?

          Please correct me if any of my conclusions are incorrect, again I’m only seeking to clarify this ones and for all.

          • http://www.dayngrzone.com/ Dayngr

            Joakim, thanks for your patience! Ok, I did a little research and wanted to follow up with you. We do get public content from Facebook from all countries for which we have language coverage – at the moment that’s 15 languages. However, since Facebook’s API doesn’t provide location information and since it is a .com site, we use the United States as it’s region (even though we’re pulling information in for all the countries we have language coverage).

            Wishing you the best,

            Trish (@Dayngr)
            Community Manager | Radian6

          • http://joakimnilsson.com Joakim Nilsson

            I know, and it means that you only cover Facebook content on public wall posts of users having US English as region/language setting. For any non US brand, that means close to zero coverage. It may be due to a problem with the data available in the API but I’d just like you to clarify on the facts. Correct or not?

  • http://twitter.com/senya03 Peter Senyi

    Great post. I have similar experiences with sm monitoring tools. They still deliver rather inaccurate results in some areas. It is important to keep this in mind when creating reports or making comparisons on the basis of these data. Here is a useful free tool that also makes use of the FB API but the format of the results is a bit better as with the method you wrote about: http://youropenbook.org/

    • http://joakimnilsson.com Joakim Nilsson

      Thanks for you suggestion Peter. I had a look at Youopenbook.org and it delivers the same results as if you use the search box and select “posts by everyone”.

      As you will notice, you search results will depend on if your’re logged-in or out and which language setting you have.

      Try this search using different options and you’ll see: http://youropenbook.org/?q=bwin&gender=any

      Wonder why no monitoring vendor allow you to connect your Facebook profile and login in.order to see more results. I’m sure Facebook are considering selling this data to whomever are ready to pay for it, just as Twitter are selling the firehose data.

      • http://twitter.com/senya03 Peter Senyi

        Yes you are right; it depends on the language settings and if you are logged in. I think we do not have to wait too long for tools to also include Facebook search results. I just received an update this week from the the monitoring company we partner with, that they will improve this feature in the near future. It will still be only a snapshot of the total mentions but it is better than nothing.

        • http://joakimnilsson.com Joakim Nilsson

          Interesting, you mind disclosing the vendor?

  • Anonymous

    Good post, Joakim. I would also recommend taking a look at the Forrester reports on social media monitoring, because they are investigating each one (I work with Synthesio, a global web monitoring company that monitors in 30 languages, and we have been talking with them regularly).

    Synthesio has its proprietary technology and database for monitoring social media, but I’ve warned people (including my parents, most recently) about social media companies and their promises.

    Until you know the technology that is behind what you’re buying, it’s like buying a car without knowing what’s under the hood..

    Glad to be in touch, in any case :)

    Michelle @Synthesio

    • http://joakimnilsson.com Joakim Nilsson

      Read them yes. Econsultancy has a good one from end of last year as well. Thx for stopping by Michelle!

    • http://joakimnilsson.com Joakim Nilsson

      Thanks Michelle. I’ve been through the Forrester report as well as the latest one from Econsultancy. The problem I have is that actual coverage is being grouped as “Social networks, microblogs, news, blogs etc” – I’d like to see Forrester extending the details with how the data is pulled from the various sources. We can leave blogs and forums out as that’s pretty straight forward. But for Facebook there’s a huge difference in the data depending on how you gather it. With Twitter, having the firehose access or not make a difference and so on.

      Simply saying that “we monitor Facebook and twitter” is providing little or no value for a buyer.

      I think there are huge opportunities for the vendors ready to really innovate and deliver a remarkable product.

  • Brooke Hennon

    Hi Joakim- Interesting blog post! Social Media monitoring tools do often have some sources that they track a more in-depth data set than others. The most important thing about social media is that it’s a great sampling of data, which most of the time can be interpreted as candid pieces of info. As time evolves, the picture will only become more accurate at 360 degree. Having said that, Facebook can be a little bit more limiting that other resources. @Sysomos Marketwire we pull from various different sources blogs, forums, twitter, Facebook & traditional media. I’m happy to explain how we do it & introduce you to our tool – should you be interested!
    @bmhennon bhennon@marketwire.com

    • http://joakimnilsson.com Joakim Nilsson

      Thanks for reaching out Brooke. I’ve been testing Sysomos a couple of times.

      I finally concluded that doing a general benchmarking between monitoring vendors is close to impossible. Although if you are a buyer and have a specific need in mind it’s easier, needless to say maybe.

      I think you have a great product with a nice interface. But as a general comment to all vendors, improving monitoring of social networks is desired. An honest elaboration on the difficulties you as a vendor are facing would be far more interesting to read, rather that these outreaches with generic sales pitches (just look at who commented in this blog).

      What are the challenges with getting facebook data, how are you tackling that? Are you solely in the hands of what the APIs provide or do you negotiate deals with the networks to get premium data access (twitter firehose)? I understand must of such would be considered confidential, but you could certainly guide the people who are interested in cashing up and investing in your service.

      Any vendor wishing to cover this in their respective company blog? Brandwatch, Radian6, Sysomos, Alterian SM2, Silverbakk, Lissly, Trackur, Meltwater Buzz, Attentio, Synthesio anyone?

      • brooke

        Hi Joakim -
        First & foremost, apologies for the delay. It varies depending upon the service.  To be honest, some of it would be limited with the API structure.  As far as Facebook data, most of us are in the same boat.  We’re tracking public information – these are conversations that we’d likely have off-line & I can get you in touch with our technical representative should you be interested in learning more. Feel free to email me @ Bhennon@marketwire.com if interested.  Thanks for your thoughtful responses.  Interesting to see the thread of conversation below that transpired.

  • Pingback: Let’s call it social and it sells | Joakim Nilsson's blog - Social Media and Online marketing for business

  • Pingback: 4 Easy Steps to Super Charge Your Contact Center using Social Media « Joakim Nilsson's blog – Social Media and Online marketing for business