Bypassing local server restrictions on Google App Engine

The local server that comes with the Google App Engine SDK tries to emulate the deployment environment as much as possible. This is a very important goal and I hope to see this improve in the future. Currently, the modules that are not supported on the deployment server are deleted from the various models. Those modules that are semi-supported at monkeypatched.

The problem is that the unsupported modules are completely deleted. Instead, they should really be renamed and kept on so that they can be re-enabled temporarily if and when you need that functionality locally.

I rant into a real-world use case for this today when building the feature to download a remote datastore backup to the local environment. I need to create folders and os.mkdir is one of the disabled modules.

I tried to find a way of doing this without having to modify the local server but couldn't find one. Instead I modified it as follows: 

Line 1119:
        for symbol in set(module.__dict__) - set(allowed_symbols):
          if not (symbol.startswith('__') and symbol.endswith('__')):
+           module.__dict__['old_'+symbol] = module.__dict__[symbol]
            del module.__dict__[symbol]

Update: In AppEngine 1.1.7, apply the patch to Line 1148 of dev_appserver.py. (You can find dev_appserver.py in /Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/ on a default install.)

Linux and Windows users: You may have to set the files in google_appengine to writable (chmod +w -R google_appengine) before you can edit it.

Windows users: You may have to set your security settings so that you can actually edit the dev_appserver.py file (Properties → Security → Users and set Full Control on).

Then, in my download handler (which only runs on the local environment), I monkeypatch mkdir back temporarily. I also add back the write mode to file: 

from google.appengine.tools import dev_appserver

# Add 'w' to the allowed modes
OLD_ALLOWED_MODES = dev_appserver.FakeFile.ALLOWED_MODES
dev_appserver.FakeFile.ALLOWED_MODES = frozenset(['r', 'rb', 'U', 'rU', 'w'])

# Add mkdir back to os
os.mkdir = os.old_mkdir

Finally, at the end of the method, when the download is complete, I return things to the way they were: 

# Remove 'w' from the allowed modes for file.
dev_appserver.FakeFile.ALLOWED_MODES = OLD_ALLOWED_MODES

# Remove mkdir from os
del os.mkdir

I'd love it if dev_appserver.py worked this way without the patch so I've opened Issue 616. I know this sounds like a very edge case but it's quite a central part of the backup/restore process I'm creating.

It's not a big deal if it's not patched -- it will simply mean that you'll have to apply a patch before you can use the backup/restore solution -- but it would be nice to have.

(And, in general, I feel it's a good practice to keep a copy of the old function when you monkeypatch to provide flexibility for cases just like this.)

Comments