_memoize_version generates invalid keys

[tlong-dev]

The function _memoize_version which is used within the @cache.memoize() decorator can generate keys that are invalid for memcached.

I am using pylibmc==1.5.0 and Flask-Cache==0.13.1

I instantiate the pylibmc client with the behavior of verify_keys.

cache = Cache(app, config={
    'CACHE_TYPE': 'memcached',
    # The reason we pass in a client directly is because the keys that flask-cache uses somehow break
    # in Python 3. I don't know why, but I managed to trace the code far enough to know that if we were
    # to send a client directly in then the Werkzeug cache backend would use it directly. This, as far
    # as I know is the only way to get Pylibmc options into the client init.
    'CACHE_MEMCACHED_SERVERS': pylibmc.Client(['127.0.0.1:11211'], behaviors={'verify_keys': True}),
})

The reason I do this is because when I switched to Python 3 the cache keys were breaking and verify keys seemed to fix it. As I look back on it now, maybe instead of fixing it it simply caused the errors to be hidden.

I see two potential issues:

1. You do not look at the response from the set command to see if the set was successful, thus hiding potential issues when setting a key does not succeed.
2. Some functions use raw un-hashed keys which are incompatible with some cache backends.

Switching my client from pylibmc to python-memcached it properly throws an exception for bad key errors coming from _memoize_version.

[tlong-dev]

PR #132

[tlong-dev]

Switching pylibmc to binary=True also seems to avoid this issue. I assume since the key content no longer matters.

[thadeusb]

Can you include the exception that you are receiving, as well as a failing unit test?

The pull request proposed causes some of the existing unit tests to fail.

[tlong-dev]

The exception is:

_pylibmc.SocketCreateError
_pylibmc.SocketCreateError: error 11 from memcached_set_multi: SUCCESS

If you switch to python-memcached you get a more reasonable error. Something along the lines of "Cannot use space/control characters in key".

As far as the PR, I sort of expected it to break stuff. It was mostly to verify that part of the code was the thing generating the fault. I can research adding a failing test.

[thadeusb]

Yeah, we need to figure out "why" you are getting invalid keys.

The _memoize_make_cache_key function should be creating an MD5 for you already

https://github.com/thadeusb/flask-cache/blob/master/flask_cache/__init__.py#L402

[tlong-dev]

That one is easy.

_memoize_make_cache_key calls _memoize_version and that function interacts with the cache directly.

[thadeusb]

What is the name of the function you are memoizing?

[thadeusb]

I see now it is using the function namespace as-is without any translations

[thadeusb]

If you could add a print statement or debug breakpoint to get the value of fetch_keys in the _memoize_version function would be super helpful.

[tlong-dev]

I've got a unit test for you... Just working on PR.

It will only work if you use memcached as the CACHE_TYPE

[tlong-dev]

The issue is that some objects have a __repr__ that have spaces for debug/readability and also to make sure that memoize() is unique enough. These spaces cause the cache to fail in the two ways listed above.

[tlong-dev]

I will retract my earlier comment. Setting the pylibmc binary=True does not fix this issue. The only work around currently is to alter modules to produce spaceless cache keys.