Can Quora “cross the chasm”?

Last month, Tawanda Sibanda and I decided to analyze community engagement on Quora, as part of research with Misiek Piskorski. We all love Quora, and were trying to understand whether it could become a mainstream resource.

Bottom-line View
Quora’s product design makes it hard for the site to grow beyond tech users. Consider 3 elements that perpetuate Quora’s appeal to its original techie audience (thereby alienating non-techies):

  1. Newsfeed: Unlike Facebook, the Quora feed is not threaded, i.e., every subsequent answer to a popular question gets a separate feed entry. As a result, the entire feed can be dominated by discussion around a single, trending topic (iOS vs. AndroidiPad 2).
  2. Votes: our data suggests that more views = more votes (not surprising). Therefore, given the newsfeed design, the only way to get votes / community love is to write about trending topics. For this community, that means writing about major issues in tech/entrepreneurship.
  3. Tech celebrities: users like Reed Hastings and Dustin Moskovitzeasily get more than 50 votes per answer and have massive follower counts. A response by them can dominate a newsfeed.

If you add them up, those 3 elements drive a self-reinforcing focus on tech. If you are non-techie visiting Quora for the first time, you might not find much value. Let’s go into the data….

What we did

Over a sample set of 30 answers, we tracked a variety of metrics to see what would predictive of high community engagement (Votes,Thanks on Quora, etc.). We know this is a small sample. We were just testing the waters!

Sample Input Metrics (data that should be correlated with whether answers would be read) included prior number of answers / votes (across all answers) to the question, prior number views and followers and whether or not the question fell within our areas of expertise

Sample Output Metrics (how the community responded to our answers) included, number of votes / comments, number of views / followers and how our follower count grew after answering.

What we found

Part 1: Choosing the question is half the battle

Using a 2-by-2 matrix below, we found that if we answered popular questions, we were much more likely to receive votes and comments from the community. On average, relative to questions rated Low-Low, answers to questions rated High-High received 3X more votes (2.6 vs 0.9), 4X more comments (0.4 vs 0.1) and 13X more subsequent views. In short, writers are rewarded for talking about things the community cares about, which means tech.

Community Engagement, Mapped to Matrix of Question Popularity

Axes Definitions
Read: Number of views for a question prior to our answer. High defined as greater than 130 views (the median) and Low less than 130.
Write: Number of answers for a question prior to our answer. High defined as greater than 3 answers (the median) and Low less than 3.

To further explore this “head” bias in Quora responses, we plotted the chart below of the cumulative percent of views and votes driven off our 30 answers. In short, 20% of questions drive 87% of total views and 57% of votes.

Cumulative Percent of Votes / Views vs. % of Questions

Part 2: Quality matters, but only if you pick a “sexy” topic
We computed the correlation coefficient between the number of votes an answer received and various metrics from our sample set. Our data says that only two factors are statistically significant in getting votes: thelength of the answer and the change in the number of views.

The importance of length is not surprising.  Anecdotally, we found that writing long, well-thought out answers increases the chance of receiving votes. The statistical significance of the change in views supports the idea that hot topics receive more votes. Being able to predict and answer questions that will have a dramatic rise in Quora viewership is a tactic to securing up-votes.


Part 3: Reputation / Identity matters
Despite our best efforts to write detailed and analytical responses, the highest number of votes we achieved was 11 on a response related to venture capital (my job). On average we received 1.7 votes per answer. To put this in perspective, on average, Reed Hastings writes 19 word answers and receives 109.8 votes (versus our 220 word average).

The chart below provides some examples of Quora “celebrities”. Almost everything they write generates significant activity and further reinforces the site’s bias towards “head” technology or entrepreneurship content.

DISCLAIMER: Tim, Dustin Drew and Reed are rockstars. They obviously deserve the love they get on Quora. We’re just trying to make a point around how that level of attention narrows the focus of the site.

What all of this means

While our sample set of 30 writes is not a very large one, we do think it raises some interesting questions for Quora’s future growth. In its current avatar, it may remain a niche resource serving only the tech community because it has slightly circular logic:

  • The existing user base primarily consists of technophiles, which follows topics related to tech trends / entrepreneurship
  • As a result, the newsfeed algorithmically prioritizes those topics
  • Consequently, newer users also click through to the same content, amplifying votes, answers, and other community engagement within those topics
  • Writers on those topics receive community validation, encouraging similar posts – good responses in other content areas and written by non-tech-celebrities are not rewarded
  • And so the cycle continues


Some food for thought:

  • Could Quora offer more community visibility / status incentives for pioneering new topics?
  • Could we encourage users to vote on answers that are truly helpful, e.g., anonymize a celebrity author until a user votes?


Thoughts welcome – looking forward to the debate.

Advertisements

About Indy Guha

Indy is a partner at Bain Capital Ventures and co-founded their Bay Area office. He invests in early and growth stage Software-as-a-Service companies, with a focus on sales and marketing technologies. Current investments include 6Sense, Wrike, BloomReach, Optimizely, etc. He also leads BCV’s Pulse.Commerce program, a custom demo day for Fortune 5000 CMOs. Indy works alongside entrepreneurs on everything from product strategy to acquisitions to hiring, including periodically taking operational roles within our companies. He was an early Product & Engagement Manager at BloomReach, working on key go-to-market issues such as performance attribution and pricing. He was also the interim Head of Marketing at Captora, helping launch the company out of stealth and managing 2 product releases.

Posted on June 28, 2011, in Community Management, Quora, Social Media and tagged , , , , , , , . Bookmark the permalink. Leave a comment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: