At Udacity, we’re constantly exploring ways to help our students succeed, and recently we’ve been focused on a couple key questions:
What would happen if we opened up new ways for our students to help each other?
Could students help each other achieve their long-term learning goals?
Every month, Udacity welcomes a new cohort of learners, each learner with a career goal in mind and a desire to master vital skills. They’re signing up for Nanodegree programs: 6–12 month hands-on learning experiences during which each student builds a portfolio of career-relevant projects. Upon finishing, a student emerges with a Nanodegree certificate, a credential that will open up doors to achieve their next career goal in tech.
Here’s the thing: Online learning is both incredibly rewarding and has moments where it can feel incredibly difficult. Most of our students are juggling one or more jobs and have a full plate of life commitments. We’ve heard from students that online learning can present similar dynamics to dieting or exercise —they know that their daily actions will get them to a long term goal, but it can be difficult to dedicate time to learning regularly while balancing other priorities. Success requires tenacity, energy, and patience.
Here’s the other thing: Most of our students who set a career goal do it on their own. They may share the goal with their friends or a significant other, but ultimately the day to day accountability can feel solitary. We know that it’s all too easy to tell yourself that today just isn’t the day or there’s a temptation to put off working on your Nanodegree project in favor of another commitment.
A product development principle that we believe in is to follow the grain of users’ daily routines, but find ways to nudge users toward certain behavior patterns we want them to adopt, within those routines. Given the competing commitments we know our students juggle, we wanted to figure out a way to help them consistently dedicate time to learning.
Our hypothesis: Real connections with a small group of other students would help students get the support they need, create a sense of accountability, and ultimately help them master career-relevant skills.
We knew that the students who graduate from Nanodegree programs and ultimately reach their career goals consistently submit their first projects by their designated deadlines. We decided to use this metric as the proof point for our hypothesis–if we could significantly improve the rate at which our general population of students hit their first project deadline, we would have a clear path forward to supporting students’ ultimate success.
To test our hypothesis, we built a low fidelity prototype of a chat client on our site and used it to answer some basic questions about how students would connect with each other.
Would students connect with each other via chat? Yes. We were inspired daily by instances of complete strangers stepping up to help each other out when they got stuck while working on projects.
Would students value having a small group? Yes. When offered the opportunity, we saw a strong signal that students would opt into a smaller learning community.
Would students stay active over time on their own? Generally, no. When we put students into small groups and let them connect over chat and email, only the groups where an individual stepped up into an organizer role persisted — and that happened for only a small segment of groups.
The Concept: Teams
Based on our early experimentation with chat and small groups, we had enough conviction to fuel deeper investment. We built out a new onboarding experience that would match students into teams of 10–12 upon signing up for a Nanodegree program and gave each team a chat room, powered by Layer. We incorporated a few key attributes that added up to a compelling and cohesive experience.
Guides — Each team is matched with a student who is farther along in, or graduated from, the Nanodegree program. This student, or Guide as we call them, serves as facilitator, mentor, and role model. Guides facilitate weekly study sessions where students can work through issues they run into during the week and discuss what they’re learning.
Visibility — A friendly bot announces learning milestones that team members achieve, to both celebrate achievements and create a sense of accountability to stay on track.
Access anytime, anywhere — Mobile apps allow students to get notifications whenever there’s activity with their team. It’s a powerful experience to receive helpful responses from your team members within minutes of getting stuck on your project at 10:30 pm.
Multiple modes of collaboration — Shortly after launch, we got feedback from students that they wanted more ways to communicate than basic chat. In response, we gave students an easy way to connect via video chat and the ability to share code snippets. These modes allow teams to establish the norms and workflows that work best for them.
The Results (so far)
Retention and engagement metrics for online learning are tough to budge, as we’ve seen from the various tacks we and other online education providers have taken.
We’ve been thrilled to see that Nanodegree program students who learn with teams display markedly improved engagement on all fronts.
These students stick with the program over time at higher rates. They spend more time learning on a weekly basis. And ultimately, we saw a 25% increase in the rate students submitted projects by their deadlines, the proof point we defined to signal whether teams were having the desired impact. For such a historically stubborn behavior pattern, 25% was huge.
The signals we continue to see in both quantitative engagement data and observational data— the way students are stepping up to help and encourage each other — suggest that social learning is a powerful motivator for our students.
The Secret Sauce
People, not widgets: One of the biggest lessons we learned through teams is the power of interpersonal connections outstrips that of less dynamic features. The impact of these tiny ecosystems of support and presence for each student has far outperformed other approaches we’ve tried to change student behavior by presenting them with personalized goals and motivational data points.
Test it early: When we came up with the idea for teams, we knew that it very well might not pan out. We were almost guaranteed to make incorrect assumptions about what students would want or need from each other. By simulating the dynamics of teams early with a low fidelity prototype and a limited population of students, we gained insights into how students would interact and potential points of failure before we either exposed a broader set of students to it or invested too much engineering effort. Early testing allowed us to hone in on the right set of initial features and execute those well.
It should just work: Given the frequency with which we wanted students to interact with their Teams, we knew that ease of use would be critical. People have to be able to access it with just a few taps or clicks and it should be easy for them to integrate it into their daily routines. Pairing a web experience with mobile apps was key here, which our integration with Layer (a robust messaging platform), made infinitely faster. We also had to be vigilant about keeping our scope as small as possible in order to execute well in a reasonable amount of time.
This is just the beginning of our exploration of how students can help each other stay on track to achieve their learning and career goals. We’ve been incredibly inspired by how students have stepped up to support each other in teams and are looking forward to continuing to invest in making students’ experiences with their teams even richer.
Oh, and we’re hiring. Join us!