Aral Balkan

This is a quick screencast to show you the data export solution I've created for Google App Engine that lets you backup your application's datastore and restore it either locally on your development machine or on the same Google App Engine application on the deployment environment or on a different Google App Engine application (which you can use as a staging environment).

That's right, this solves one of the big criticisms I've seen levelled at Google App Engine, which is that there is no way to export your data from your applications.

With this solution, not only can you export your data but you can also easily restore it.

I am going to be sharing the solution with you in the coming days as a separate open source project in the form of a Django application that you can plug into your own applications as part of the Singularity Web Conference Open Initiatives program (stay tuned for updates on that front).

In the meanwhile, watch the screencast to see how the backup and restore solution works.

View the full-resolution screencast on screencast.com

Comments

• Very nice Aral. I'll want to use that. I have a question about building Singularity on GAE: are you at all concerned you might hit your account limits? Considering this is a commercial application, and HOPEFULLY you'll get tons of traffic and stuff going on for Singularity, it would be quite bad if you hit those limits and users couldn't participate in the conference or see the information. How do you keep within your limits? The reason I ask is because I'd like to host a commercial app on GAE, but ensure I stay under the limits or be able to buy more resources by the time I exceed the free limits. Thanks for any feedback!

  by Jacob Wright on 2008-08-12 18:41:11

• Aral, Any idea when this will be ready for others? I could *really* use it to do some testing in my local environment using live data! Thanks for the time + effort!

  by Chris on 2008-08-27 22:51:15

• That's very cool Aral. Congrats. Did you take try approaches other than serializing into a .py file. Perhaps into pickles that are zipped and sliced. You dump them into memcache, and later retrieve from memcache, unzip, unpickle and store into local.... I'm not sure if that's gonna work, just throwing ideas :)

  by Ozgur Cem S. on 2008-08-12 23:57:27

• Is this a dead project? I noticed the last update time is August this year ...

  by Guest on 2008-10-18 05:14:48

• I had read your post on the Google group, and I was pretty skeptical, but this looks *really* good! Nice work--I look forward to playing around with this.

  by Mike on 2008-08-15 20:28:53

• Nice screencast Aral.

  by Ben the Indefatigable on 2008-08-14 13:27:31

• [...] it's time for me to go back to my Google App Engine backup solution, which broke after Google introduced artificial limits on query offsets, and update it to use [...]

  by Aral Balkan - Google App Engine SDK 1.1.6 released, brings sortable datastore keys on 2008-11-25 13:39:51

• Nice screencast

  by Oyunlar on 2010-02-20 15:22:02