Stack Overflow Mentorship Research Project

I’m Kristina, the first (and so far, only) User Experience Researcher here at Stack Overflow. As you probably know, on SO we have a bit of a problem with our new user experience. People new to SO (or maybe even new to programming!) come to us to find an answer to a programming issue, and that means that sometimes they ask a question themselves. When they ask a question, we show them a long page of rules for how to ask a question, and then throw them into it. (In case you forget what that looks like, here it is below.)

screenshot of ask a question rules

From here, one of several things happens:

  1. They are too intimidated by the wall of text (or get distracted or any number of things) and they just don’t ask it
  2. They ask a question, but it’s badly formed and it gets downvoted or
    deleted or snarked at
  3. They ask a question, but it’s a duplicate and gets closed right away
    (which is good for SO overall, but probably doesn’t feel great
    in-the-moment)
  4. They ask an appropriately-formed question and get a useful response

What we’re working on is reducing the instances of scenarios #1-3. We’re working with Denae Ford, a CS PhD student at NCSU (you might recognize her from her wonderful research work on gender-based barriers to Stack Overflow usage. Our first attempt to resolve this problem and make Stack Overflow friendlier to new folks (and hopefully also write a paper about it!), is to test a lightweight mentorship program.

Here’s how we envision this test working (about a month long, starting hopefully within the next month):

  1. When we see a first-time asker in the Ask a Question page, we show
    them a link to a chatroom in the sidebar
  2. We have a specially-created, moderated chatroom for people from the
    Ask a Question page and for people who volunteer to mentor
  3. The mentors help with question formation, strategies for searching
    for existing questions, or with tagging. Mentors don’t actually
    answer programming questions in chat.

We hope that this or something like it will give new users a better first-time experience, thus increasing their likelihood of participating in the future, and generally make them better-equipped members of the Stack Overflow community. We also hope that the ratio of useful questions on the site will increase.

So, here’s what I’m asking from y’all:

  • What do you think about this idea? We’re just starting to put this
    test together and would love your feedback and thoughts – we want to
    give it as much likelihood to succeed as we can.
  • What concerns do you have with how we’re deciding to implement this
    initially?
    We’re trying to go as light on the technical requirements
    as possible, but we want to make sure we haven’t missed anything
    glaring.
  • And most importantly: are you interested in volunteering to be a
    mentor?
    A project like this would never work without knowledgeable
    and experienced Stack users to actually do some mentoring. As this is
    a test, we’d probably also want to talk to you about your experience
    afterwards.

If you want to be a mentor, sign up here.