Follow-up: Unanswered Questions From Our App Engine | Google Developer’s Live Appearance

I enjoyed the opportunity to participate in a live Google Developer Live hangout on the topic of how Udacity uses App Engine.  Unfortunately there was not enough time to respond to all the great questions on air.  What follows is my best attempt to respond to the remaining questions.

krmboya: “In your opinion, how low-level should the average modern programmer go? Is it still worth learning assembly language and various pointer-manipulation tricks?

I recommend you should always learn as much about computing as you can.  You can be a productive software engineer without knowing Assembly or C.  However, I do think it is necessary to eventually become familiar with Assembly or C if you want to be a really great software engineer.

Kenny: “What are the disadvantages of using Google App Engine and how does it compare to a service like Amazon’s AWS?

Shahriar: “What makes Google app engine a better choice over using the AWS stack?

Comparing Google App Engine and AWS is like comparing a box of cake mix to a counter full of flour, sugar, water and eggs.  App Engine represents the final product of a complete platform whereas AWS gives you the ingredients that still need to be combined into a complete platform.  App Engine’s advantage can also be its disadvantage:  On the one hand you don’t have to do any “sys-admining” but on the other hand you can’t do any sys-admining when something is going wrong.

ppmt: “I have seen the cost of using GAE but as a mere mortal it is difficult to imagine how it translated in real life. Would you be able to say how much it cost Udacity to be hosted on GAE?

They vary greatly depending on what is trending with our course traffic as well as any ad-hoc analysis and data-dumping we have going on at the moment.  I can say, though, that even before our funding when our budget was tighter, the costs were reasonable enough that price was never a factor when determining where to implement new features.

Travis: “What is the biggest hurdle to developing an application on the cloud?

I think that developing a solid product vision for the application is harder than building it, especially if you build it in App Engine.  My recommendation is to spend one day building a simple app in App Engine and then the next day building the same app in AWS with Elastic Beanstalk against DynamoDB + ElastiCache + CloudFront.  One is going to feel better to you than the other depending on your comfort level with the operations side of building web applications.  My advice is to just go with the one that feels better and don’t look back — if and when it comes time to move to something else, it will be obvious what type of platform you need.  In the meantime, don’t let yourself get stuck trying to future-proof every decision.  It’s better to have a successful app on the wrong platform than no app at all!

As for the biggest hurdle developing an application specifically on App Engine:  I would say the first hurdle is setting up your first application.  Once you have that done, you get to cruise for an surprisingly long while you build out features.  The next hurdle is re-arranging things to perform better once your app takes off.  Udacity is working through the second hurdle now and it took us about a year to get from the first to second.  The best part is that you have plenty of time to develop the level of engineering sophistication needed to get past the second hurdle — you shouldn’t expect the same kind of beating Steve describes in his course when learning how to scale Reddit.

Eric: “What types of projects would you use GAE for a real application?

I would use App Engine for any website or mobile backend that doesn’t require specialized libraries for processing.  For example, I wouldn’t build Voxer on App Engine because they require things like audio stream processing, which, are going to eventually rely on low-level unix libraries written in C.  Clash of Clans on the other hand probably has a relatively straight-forward backend for saving state and managing scheduled tasks — all things very native to App Engine with additional benefit of horizontal scale-out when the app becomes ultra popular.

Anthony: “I have been wondering what the best way to implement a chat server on the GAE would be?

Interesting idea.  Have you looked at the XMPP Service and the Channel API?  I would expect an implementation to use one or both of them.

Мурат: “What is the copyright status of the CS253 course? Does it go under Creative Commons license? Can we translate the course into another language and share it in our websites?

All our current courses are offered under the Creative Commons license and I am not aware of any desire to change that.  New translations are very much appreciated!  You can find existing translations and instructions for submitting new ones at our Universal Subtitles team page.

Bobi: “I really want to know if you’re going to have more technology related programming classes as well. Like: – Low level programming for microcontrollers – RTOS – Embedded Design basics Thanks and keep being awesome!

We are currently producing a course that is somewhat in this direction called Functional Hardware Verification (CS348).  Much of Udacity’s team has a robotics background and we love things like hardware and programming micro controllers.  I think it is safe to expect more courses is this area, especially since it is a very employable field.  The challenge, obviously, is figuring out how to organize project-based course materials.

Teaching Thousands of Students to Program on Udacity with App Engine, Part Two

Cross-posted with the Google App Engine blog:

This post is part two of a two-part series on how Udacity is using App Engine to teach thousands of students how to program online. Today’s guest blogger is Chris Chew, senior software engineer at Udacity, which offers free online courses in programming and other subjects.  Chris shares how Udacity itself is built using App Engine.

Steve Huffman blogged yesterday about how App Engine enables the project-based learning that makes his web development course so powerful.  People are often surprised to learn that Udacity itself is built on App Engine.

The choice to use App Engine originally came from Mike Sokolsky, our CTO and cofounder, after his experience keeping the original version of our extremely popular AI course running on a series of virtual machines.  Mike found App Engine’s operational simplicity extremely compelling after weeks of endlessly spinning up additional servers and administering MySQL replication in order to meet the crazy scale patterns we experience.

Close to a year later, with ten months of live traffic on App Engine, we continue to be satisfied customers.  While there are a few things we do outside App Engine, our choice to continue using App Engine for our core application is clear:  We prefer to spend our time figuring out how to scale personalized education, not memcached.  App Engine’s infrastructure is better than what we could build ourselves, and it frees us to focus on behavior rather than operations.

How Udacity Uses App Engine

The App Engine features we use most include a pretty broad swath of the platform:

A high-level representation of our “stack” looks something like this:


Trails and Trove are two libraries developed in-house mainly by Piotr Kaminski.  Trails supplies very clean semantics for creating families of RESTful endpoints on top of a webapp2.RequestHandler with automagic marshalling.  Trove is a wrapper around NDB that adds common property types (e.g. efficient dereferencing of key properties), yet another layer of caching for entities with relations (both in-process and memcache), and an event “watcher” framework for reliably triggering out-of-band processing when data changes.

Something notable that is not represented in the drawing above is a specific set of monkey patches from Trove we apply to NDB to create better hooks similar to the existing pre/post-put/delete hooks.  These custom hooks power a “watcher” abstraction that provides targeted pieces of code the opportunity to react to changes in the data layer.  Execution of each watcher is deferred and runs outside the scope of the request so as to not increase response times.


During our first year of scaling on App Engine we learned its performance is a complex thing to understand.  Response time is a function of several factors both inside and outside our control.  App Engine’s ability to “scale-out” is undeniable, but we have observed high variance in response times for a given request, even during periods with low load on the system.  As a consequence we have learned to do a number of things to minimize the impact of latency variance:

  • Converting usage of the old datastore API to the new NDB API
  • Using NDB.tasklet coroutines as much as possible to enable parallelism during blocking RPC operations
  • Not indexing fields by default and adding an index only when we need it for a query
  • Carefully avoiding index hotspots by indexing fields with predictable values only when necessary (i.e. auto-now DateTime and enumerated “choices” String properties).
  • Materializing data views very aggressively so we can limit each request to the fewest datastore queries possible

This last point is obvious in the sense that naturally you get faster responses when you do less work.  But we have taken pre-materializing views to an extreme level by denormalizing several aspects of our domain into read-optimized records.  For example, the read-optimized version of a user’s profile record might contain standard profile information, plus privacy configuration, course enrollment information, course progress, and permissions — all things a data modeler would normally want to store separately.  We pile it together into the equivalent of a materialized view so we can fetch it all in one query.


App Engine is an amazingly complete and reliable platform that works astonishingly well for a huge number of use cases.  It is very apparent the services and APIs have been designed by people who know how to scale web applications, and we feel lucky to have the opportunity to ride on the shoulders of such giants.  It is trivial to whip together a proof-of-concept for almost any idea, and the subsequent work to scale your app is significantly less than if you had rolled your own infrastructure.

As with any platform, there are tradeoffs.  The tradeoff with App Engine is that you get an amazing suite of scale-ready services at the cost of relentlessly optimizing to minimize latency spikes.  This is an easy tradeoff for us because App Engine has served us well through several exciting usage spikes and there is no question the progress we have already made towards our mission is significantly more than if we were also building our own infrastructure.  Like most choices in life, this choice can be boiled down to a bumper sticker:


Editor’s note: Chris Chew and Steve Huffman will be participating in a Google Developers Live Hangout on Thursday, November 1st. 

Here is the full Hangout conversation on Youtube:

Udacity Showcase: Share your Udacious projects with the world

Today, Udacity is proud to launch Udacity Showcase, a platform where Udacity students can put their original projects on display. This project was conceived and built by one of Udacity’s own Web Development students, Gian Carlo Martinelli from Brazil. In collaboration with Udacity’s all-start intern Anthony Teate, Martinelli has been helping to redefine and refine his project. Martinelli states, “I originally built this to scratch my own itch, since I was curious about projects that other Udacians were working on.”

In addition to Web Development, Martinelli has also completed Intro to Computer Science. Before taking Udacity classes he studied Business Administration and has been studying computer science consistently for the past year. He states, “Udacity gave me the confidence that I could build something — that I could start with nothing, take the problem apart and solve it step-by-step.”

Udacity Showcase is a website that gives students the opportunity to show off projects they’ve created from what they have learned in their Udacity courses. The platform makes it easy for students to post their projects and share them with the world. Just create an account, submit a title, screenshot, description, and link for your project — then you’re done!

Each project page allows for students to comment on projects, to provide suggestions, improvements, note any bugs or errors, and any other constructive criticism. It also allows students to submit a link to either view or download a project, so they are free to link to a Github, Bitbucket, or any other open-source project hosting website. This way students can view, clone, and fix/improve the original project.

Udacity is so excited to provide this platform for our students. Teate notes that, “I’m amazed at what Udacity students are capable of, and I’ve seen their projects on the forums. But the forums aren’t necessarily designed to showcase work, especially if all of the projects are scattered throughout one thread. With this platform, Gian and I wanted to give students a place to post their projects, know that they would be seen, and make it easy for them to gather feedback on their work — in short, make it easy for them to show off!”

Note: Currently, anyone can sign up on Udacity Showcase to submit a project. User accounts on Udacity Showcase are not tied in with accounts on Udacity, so you have to create another account. (We hope to compensate for this in the future with something like OpenID).

Who are Udacity’s Top Ten Meetup Communities?

Saturday marks Udacity’s first ever Global Meetup Day! Over the course of just a few weeks, Udacity students have done a tremendous job self-organizing for this event in cities across the globe.

We are excited to have Steve Huffman joining us and hosting our San Francisco Meetup at Hipmunk, and Michael Littman as a special guest at the New York City Meetup! In Palo Alto, Sebastian Thrun and the Udacity team will be joined by Peter Norvig and Steve Blank. And in Germany, Joern Loviscach will be joining the Berlin Meetup and potentially bringing along some internet friends.

We are also really happy to announce the top ten communities that have grown the largest and who will be receiving a video phone call, via Skype, from the Udacity team during their event!

  1. Koyampattur
  2. Bangalore
  3. Accra
  4. London
  5. Delhi
  6. Tanjah
  7. Pune
  8. Barcelona
  9. Beijing
  10. Bombay

Meetup organizers in these cities should look out for an email from Udacity. We look forward to meeting you this Saturday!

If you still want to attend a Meetup in your area and have not yet RSVPed, do it today on Udacity’s Meetup Everywhere page!

Also, don’t forget, you can add photos from your event to the Udacity Meetup Everywhere Flickr group!

Sebastian Thrun on Ed Tech Panel at DisruptSF with Sal Khan and Joel Klein

Sebastian Thrun, CEO of Udacity was a guest speaker at this year’s TechCrunch Disrupt San Francisco, a four day event that brings together the world’s leaders in innovation. TechCrunch’s Disrupt events are hosted in both San Francisco and New York City — they are the most anticipated technology conferences of the year. As part of this year’s lectures, Thrun spoke today about the future of education, alongside Joel Klein, CEO of the Education Division for News Corporation, and Sal Khan, Founder and Executive Director of Khan Academy.

The panel focused on engaging students and empowering teachers. Sebastian spoke about careers being the ultimate outcome in higher ed, citing our Career Team’s work, as well as our new courses — particularly How to Build a Startup: The Lean LaunchPad taught by Steve Blank, starting this Friday — as means of addressing this outcome.

Check out some photos from the event on our Flickr Set.

What do you want to know about Udacity? #MeetUdacity

With less than a week until Udacity’s first Global Meetup event, we want to know: what do you want to know about Udacity?

As part of our live broadcast through USTREAM, Sebastian Thrun, Udacity CEO and founder, will be answering YOUR questions. Whether you will be tuning into the live broadcast or re-viewing it on our USTREAM channel, we want to hear your questions.

To submit a question, please tweet it and include #MeetUdacity. We will be collecting all of the question submissions and picking a handful to answer during the broadcast.

In addition to answering your questions, there is a bunch of Udacity team members and instructors who will be joining in to say “hello.”

From our Palo Alto location, you can expect to hear from: Sebastian Thrun, Udacity CEO and instructor for Intro to Statistics, Peter Norvig, instructor for Design of Computer Programs, Steve Blank, instructor for How to Build a Startup, Dave Evans, instructor for Intro to Computer Science, and Andy Brown, instructor for Intro to Physics.

From our San Francisco location, Steve Huffman, co-founder of Hipmunk and instructor for Web Development; from New York City, Michael Littman, instructor for Algorithms, from New York City, and from Berlin, Joern Loviscach, instructor for Making Math Matter!

Thank you to everyone! Your participation and enthusiasm surrounding this event has been truly inspiring to the Udacity team. We can’t wait to meet all of you on September 15!

Secondary School Challenge winners announced; More than 14,000 units completed

Over the summer months, students around the world have been taking Udacity classes for a chance to win an all expenses-paid trip to Silicon Valley, sponsored by Battelle. On their trip, Sebastian Thrun, Udacity CEO, will take the winners on a tour through the Udacity offices and the Stanford Intelligence Labs.

Since June, 408 teams competed with over 2,591 students participating in the challenge, and seven team have been named the winners:

The Top Teams are:

  • Team Renaissance from Bergen Country Academies, NJ
  • Team NYC from The Dalton School, NY
  • Team Freehold from Freehold High School, NJ
  • Gerard’s Team from San Francisco de Asis, Dominican Republic
  • Team Daphne and Aaron from Needham High School, MA
  • Team FlyingNinjas from Texas Academy of Math and Sciences, TX
  • Team DEE35 from Calday Grammar School, UK

The rules were simple: two team captains, who must be attending secondary school, can form a team. From there, captains recruit teammates, who do not have to be attending secondary school, and accumulate as many completed course units as they can for points. On August 24, Udacity tallied up the numbers of completed units to determine the winners. The results were impressive — across all of the teams, 14,862 units were completed!

Throughout the summer, Udacity saw secondary school students cultivating their own learning communities through the Udacity forums and through social outlets. Some teams set up Facebook groups where they could help each other with problem sets and programming. One team even turned the challenge into a fundraiser for The School Fund by soliciting sponsors for each unit completed.

Feedback from students has been tremendous. As one student stated (summarizing the many emails and posts we have received), “Udacity classes are a way to learn things I wouldn’t be learning in school.” In a survey, 99% of students polled said that they would like to continue taking Udacity courses, even during future vacations!

High schools have taken note as well. At Battelle STEMx network high school in Ohio — eSTEM Academy of Reynoldsburg — has announced that they enrolled 41 students in Udacity’s Introduction to Statistics and 49 students in Introduction to Physics this fall semester for credit. Their goal is to help drive high school students to excel in college-level courses.

Both challenge sponsor, Battelle, and challenge partner, the Conrad Foundation, helped promote the competition and are now exploring how to further promote online learning opportunities for high school students. Ultimately, the motivation and achievement show by high school students taking and completing college-level courses in STEM has been inspiring.