Monday, June 24, 2013

Sebastian Thrun: Thoughts and Financial Transparency on our Masters in Computer Science with Georgia Tech

It has now been six weeks since we announced our collaboration with Georgia Tech and AT&T on a new online Masters program, and we are well underway creating the content and technology necessary to bring an affordable online computer science education to the public.

In December 2012, I participated in the development of a Student Bill of Rights. In it, a group of 12 scholars defined what we believed to be inalienable rights for students in the digital age. We postulated financial transparency for the students as a basic right.

Even though we at Udacity knew we might be subjected to intense criticism and debate, we opened up access to the budget of our new degree, and we are committed to publishing actual financials as this degree is being rolled out. The OMS CS is truly a moon shot, and much of what we project are speculations at this point. But if this model stands, it could serve as a blueprint for making higher education more accessible in the 21st century.

Up front, I believe that online education will not replace face to face education, and neither is it supposed to. Just as film never replaced theater plays and many of us prefer to watch sports live in big stadiums, online will not abolish face to face interaction. It is a different modality with the potential to fundamentally change access and affordability. Online education is also nothing new. A significant fraction of US higher education degrees is already conferred through online programs.

I want to clarify another common confusion about MOOCs. Some proclaim that MOOCs can be offered at "effectively zero dollars marginal cost per additional student." While this might be true for MOOCs, I don't believe this is true for meaningful and high quality education. In fact, our data suggests the opposite. In a few weeks, we will be ready to release data from our SJSU pilot program. In this program, we combined the very best of MOOC education with the very best of traditional online best practices. We offered students online mentoring and access to help and guidance when they needed it. The National Science Foundation and the Gates Foundation are funding a study on the effect of these hybrid online classes, and we are eager to share the data when the study is complete. From the very preliminary data I have seen, I believe that human contact and mentoring make a substantial difference in learning outcomes. The belief that education can be replaced by a computer program is a myth, driven in part by media eager to play upon people's fears in an otherwise important, constructive debate.

One recent article in particular caught my eye: an essay by Dr. Christopher Newfield from UC Santa Barbara, published in Inside Higher Education.

Dr. Newfield suggests that "[t]he two entities together will spend about $3.1 million running a program for an estimated 200 students in the first semester. This comes to around $15,700 per year per enrolled student. The figure is close to what the University of California Office of the President says it spends educating all UC students averaged together."

This analysis is misleading. The bulk of the initial costs arise from course digitization costs, and they are amortized over many years and a much larger number of students. The limited enrollment of 200 in Year 1 is a precautionary step so we can debug the program and improve it before opening it up more broadly. Even with just 2,000 students taking these classes, the amortized content costs will run a factor of ten lower than projected in the budget.

The budget's up-front costs cover content digitization, not platform development, as the contract clearly states. In reference to the digitization costs, Dr. Newfield writes, "[b]ut Udacity is supposed to have already solved higher education’s 'cost disease' with its technology. We’ll note that Year 1 is not plug-and-play."

Think about what his statement entails. Should a top-notch Masters degree really be the result of throwing together and repackaging existing course content? Georgia Tech, Udacity, and AT&T are committed to the highest quality of education. Digitization of content incurs costs. This is not just about translating and putting an offline course online, but transforming existing course content such that is developed specifically for the medium and for the online learning experience.

Dr. Newfield also analyzes in detail our enrollment projections. He points out assumptions on student enrollment when he writes,"$19 million of Year 3 revenue requires 8,700 paying student FTE. This figure is larger than the total number of computer science masters degrees granted in 2009-10 in the United States." 

It is speculative for us to predict enrollment numbers. Fortunately for us, about three quarters of the costs of the program in steady state are variable costs. Variable costs scale with student enrollment. If we have fewer students, we would have less revenue but also incur lower costs.

But more importantly, we project the majority of tuition revenue to come from non-degree seeking students. Dr. Newfield may have overlooked this aspect of our plan, as I concur with him that the present market for degree-seeking students is limited. But our project is about expanding access and increasing education opportunities beyond a degree.

In the United States alone, we have 2 million open jobs in high tech, and over 11 million unemployed people. Tech companies across the nation have been urging Congress to open the flood gates for foreign workers to enter the high tech industry -- much of which falls squarely into the domain of this program. This new degree is fundamentally about access. Our MOOC and SJSU data shows that the vast majority of students come from outside the conventional academic system, enjoying the new access paths that our programs provide. If we only educate 1% of underemployed Americans to become proficient in computer science through Georgia Tech's new program, we will vastly exceed our enrollment projections. And we are doing something that is good for the nation!

Dr. Newfield moves on to proclaim, "How would the OMS CS get to this cost? The simple answer is that it wouldn’t."

Here is how we plan to do it. As the document clearly states, and Dr. Newfield notes, we are planning for 3 hours of personalized face time per student per course. This time does not count the time it takes to make the digital content and to develop the fast computerized feedback (a.k.a. gamification) for which Udacity MOOCs are known. It does not even count the time for exam proctoring. It is personal time given to individual students in form of mentoring, tutoring, discussion, and grading. I spent the past year going from institution to institution in higher education asking professors and TAs, "How much time do you give to each individual student?" The answers vary widely, but 3 hours per course is on the higher end. This is how we arrived at this number. As I said, much of the program is speculative, and we might have to revise this number as we learn more. We are committed to putting quality and student outcomes ahead of everything else. Even if we had to double this number to 6 hours per student per
course, the cost per credit would only go up by less than $50. Hardly a threat to the basic economic model.

There is one thing Dr. Newfield gets right. He says, "[t]his is an important admission that a MOOC is 'as good or better' than hands-on instruction only through much hands-on instruction." Dr. Newfield is very observant when he notices that we integrated hands-on mentoring and student help into the program. In that sense, our program is not a MOOC. It is based on MOOC technology and pedagogy, and leverages the best of many different worlds into online education. Yet throughout the article, Dr. Newfield expresses doubts that our new program is changing the cost structure of higher education. He says, "there is no sign in the Udacity spreadsheets of massive online cost-cutting services." He even entitles his essay, "Where Are the Savings?" and states that our program "disappoint[s]."

If Dr. Newfield is right and our program is not changing the basic cost structure of higher education, then it must be the case that other graduate programs operate at similar costs. Yet tuition fees at comparable institutions are several times higher. This raises the obvious question: where does all the tuition money and the state support go? Can others, such as perhaps Dr. Newfield's own institution, follow Georgia Tech's lead and make high quality Masters degrees more affordable?

I believe these are great times for higher education. US higher education is rightfully admired by many around the world. To maintain our position, we should strive to innovate. Innovation means that we might fail along the way, and that we are willing learn from our failures and to forge on. I am excited that through our collaboration with Georgia Tech and AT&T, we are redefining access and cost of an entire computer science degree. We will measure it by the students we reach who would otherwise not have had this opportunity, and their successes in advancing their education and their careers.