Last month, Tawanda Sibanda and I decided to analyze community engagement on Quora, as part of research with Misiek Piskorski. We all love Quora, and were trying to understand whether it could become a mainstream resource.

Bottom-line View
Quora’s product design makes it hard for the site to grow beyond tech users. Consider 3 elements that perpetuate Quora’s appeal to its original techie audience (thereby alienating non-techies):

1. Newsfeed: Unlike Facebook, the Quora feed is not threaded, i.e., every subsequent answer to a popular question gets a separate feed entry. As a result, the entire feed can be dominated by discussion around a single, trending topic (iOS vs. Android, iPad 2).
2. Votes: our data suggests that more views = more votes (not surprising). Therefore, given the newsfeed design, the only way to get votes / community love is to write about trending topics. For this community, that means writing about major issues in tech/entrepreneurship.
3. Tech celebrities: users like Reed Hastings and Dustin Moskovitzeasily get more than 50 votes per answer and have massive follower counts. A response by them can dominate a newsfeed.

If you add them up, those 3 elements drive a self-reinforcing focus on tech. If you are non-techie visiting Quora for the first time, you might not find much value. Let’s go into the data….

What we did

Over a sample set of 30 answers, we tracked a variety of metrics to see what would predictive of high community engagement (Votes,Thanks on Quora, etc.). We know this is a small sample. We were just testing the waters!

Sample Input Metrics (data that should be correlated with whether answers would be read) included prior number of answers / votes (across all answers) to the question, prior number views and followers and whether or not the question fell within our areas of expertise

Sample Output Metrics (how the community responded to our answers) included, number of votes / comments, number of views / followers and how our follower count grew after answering.

What we found

Part 1: Choosing the question is half the battle

Using a 2-by-2 matrix below, we found that if we answered popular questions, we were much more likely to receive votes and comments from the community. On average, relative to questions rated Low-Low, answers to questions rated High-High received 3X more votes (2.6 vs 0.9), 4X more comments (0.4 vs 0.1) and 13X more subsequent views. In short, writers are rewarded for talking about things the community cares about, which means tech.

Community Engagement, Mapped to Matrix of Question Popularity

Axes Definitions
Read: Number of views for a question prior to our answer. High defined as greater than 130 views (the median) and Low less than 130.
Write: Number of answers for a question prior to our answer. High defined as greater than 3 answers (the median) and Low less than 3.

To further explore this “head” bias in Quora responses, we plotted the chart below of the cumulative percent of views and votes driven off our 30 answers. In short, 20% of questions drive 87% of total views and 57% of votes.

Cumulative Percent of Votes / Views vs. % of Questions

Part 2: Quality matters, but only if you pick a “sexy” topic
We computed the correlation coefficient between the number of votes an answer received and various metrics from our sample set. Our data says that only two factors are statistically significant in getting votes: thelength of the answer and the change in the number of views.

The importance of length is not surprising.  Anecdotally, we found that writing long, well-thought out answers increases the chance of receiving votes. The statistical significance of the change in views supports the idea that hot topics receive more votes. Being able to predict and answer questions that will have a dramatic rise in Quora viewership is a tactic to securing up-votes.

Part 3: Reputation / Identity matters
Despite our best efforts to write detailed and analytical responses, the highest number of votes we achieved was 11 on a response related to venture capital (my job). On average we received 1.7 votes per answer. To put this in perspective, on average, Reed Hastings writes 19 word answers and receives 109.8 votes (versus our 220 word average).

The chart below provides some examples of Quora “celebrities”. Almost everything they write generates significant activity and further reinforces the site’s bias towards “head” technology or entrepreneurship content.

DISCLAIMER: Tim, Dustin Drew and Reed are rockstars. They obviously deserve the love they get on Quora. We’re just trying to make a point around how that level of attention narrows the focus of the site.

What all of this means

While our sample set of 30 writes is not a very large one, we do think it raises some interesting questions for Quora’s future growth. In its current avatar, it may remain a niche resource serving only the tech community because it has slightly circular logic:

• The existing user base primarily consists of technophiles, which follows topics related to tech trends / entrepreneurship
• As a result, the newsfeed algorithmically prioritizes those topics
• Consequently, newer users also click through to the same content, amplifying votes, answers, and other community engagement within those topics
• Writers on those topics receive community validation, encouraging similar posts – good responses in other content areas and written by non-tech-celebrities are not rewarded
• And so the cycle continues

Some food for thought:

• Could Quora offer more community visibility / status incentives for pioneering new topics?
• Could we encourage users to vote on answers that are truly helpful, e.g., anonymize a celebrity author until a user votes?

Thoughts welcome – looking forward to the debate.