OdooGap Blog / Making Odoo Webshop and Website faster

Making Odoo Webshop and Website faster


Caching Odoo with Memcached

Using cache to speed up your website

I once read somewhere:

"If you are asking me about chache then probably your websites are painfully slow."

And I totally agree, independently of the technology you use, caching systems will always be the base of an efficient website. We need to think about the webserver as an html producer and then consider having a cache distribution system.

With Odoo website and webshop it's not different than any other, so how do we cache Odoo?

What to cache and what not to cache

Back in early 2000 I used a language called Coldfusion and used something like:

<cfcache timeout="360">
    <div id="some-id">Hello World!</div>
</cfcache>

Doing this allowed me to specify parts of the code that I wouldn't expect to be so important to have real time on the webpage. There was also a similar mechanism on the database queries, where I specified a timeout diferent than zero and the query would be also cached.

Now, I thought about altering the xml RNG validation to add another attribute to t-call, that would be interesting but the problem is that you can call the same template passing diferent values. Since I had a project where I urgently needed results, I decided to create a caching key using the following form:

cache_key = "%(lang)s%(url)s" % {'url': request.httprequest.url, 'template_id': template_key, 'lang': lang}

Then decided that I would only cache some templates and leave out others that could be a problem. For the situation I had it solves 80% of the speed problem.

The decision for the cache system

I ended up finding myself with the decision between using Redis or Memcached. Ended up deciding for Memcached because most of the Benchmarks and opinions I found supported that Memcached performs better for caching HTML.

https://www.memcached.org/

Memcached is an in-memory key-value store for small chunks of arbitrary data (strings, objects) from results of database calls, API calls, or page rendering.

Since i'm using cache on the same server as Odoo we can opt for Unix Sockets connection. Using memcached it's as simple as:

from pymemcache.client.base import Client


client = Client('/tmp/memcached.sock')

client.set('some_key', 'some_value')
result = client.get('some_key')

For deciding what to cache and for how long I just added a dictionary where I stored the expiry of the keys:

# What to cache and respective expiry in seconds
CACHE_TEMPLATES = {
    'website.submenu': {'expiry': 20},
    'web.assets_common': {'expiry': 20},
    'web.assets_frontend': {'expiry': 20}
}

Then, I just overrided the render method for the ir.qweb and decided to cache depending on template

class QWeb(models.AbstractModel):
    _inherit = 'ir.qweb'

    def render(self, template, values=None, **options):
        # caching code
        return super(QWeb, self).render(template, values, **options)

You can find the full code here but be sure that you know what templates to cache and for how long. In doubt if a template is used in more than one place, also use the url to restrict caching.

The results

If we consider that not all templates are being cached, just a few the result it's really great. Average response for these pages goes from 234 ms to 78 ms that's 3x faster.

Locust Results with Memcached

Locust Results with Memcached

Locust Results without Memcached

Locust Results without Memcached

The testing was done on a 8 core 16GB RAM laptop using a Odoo 12.0 hited by Locust with 400 users hatch rate 15. I stoped around 5000 requests for both cases. My laptop could do more but it's hosting Postgresql, Locust, Memcached and Odoo so I decided to just spin Odoo with 8 workers and all other default Odoo settings.

The full code

I did this for a 12.0 CE but 13.0 it's not so different.

from odoo import models, api
from pymemcache.client.base import Client
from pymemcache.exceptions import MemcacheUnknownError
import logging

_logger = logging.getLogger(__name__)


# What to cache and respective expiry in seconds
CACHE_TEMPLATES = {
    'website.submenu': {'expiry': 20},
    'web.assets_common': {'expiry': 20},
    'web.assets_frontend': {'expiry': 20},
    'base.contact': {'expiry': 20},
    'website_blog.blog_post_short': {'expiry': 20},
    'website.contactus': {'expiry': 20},
    # This was a template that I tried and ended up creating problems
    # But try to uncomment and check what happens to product image
    # 'website_sale.shop_product_carousel': {'expiry': 1200},
    'website_sale.product': {'expiry': 20},
    'website_sale.products': {'expiry': 20},
    'website.500': {'expiry': 20},
}

# MEMCACHED_SETTINGS = False
# MEMCACHED_SETTINGS = Client(('localhost', 11211))
MEMCACHED_SETTINGS = Client('/tmp/memcached.sock')


class QWeb(models.AbstractModel):
    _inherit = 'ir.qweb'

    def render(self, template, values=None, **options):
        """
        to fine tune use --log-handler "odoo.addons.website_cache:DEBUG"

        :param template:
        :param values:
        :param options:
        :return:
        """
        if not MEMCACHED_SETTINGS:
            return super(QWeb, self).render(template, values, **options)
        if isinstance(template, int):
            template_key = self.env['ir.ui.view'].browse(template).key
        else:
            template_key = template
        settings = CACHE_TEMPLATES.get(template_key, False)
        user_id = values.get('user_id', False)
        if settings and user_id:
            request = values.get('request', False)
            env = values.get('env', False)
            lang = values.get('lang', False)
            if user_id.id == env.ref('base.public_partner').id:
                cache_result = False
                cache_key = "%(lang)s%(url)s" % {'url': request.httprequest.url, 'template_id': template_key, 'lang': lang}
                for origin, target in [('http:', ''), ('https:', ''), ('/', '_')]:
                    cache_key = cache_key.replace(origin, target)
                client = MEMCACHED_SETTINGS
                try:
                    cache_result = client.get(cache_key)
                except MemcacheUnknownError as e:
                    _logger.error("WEBSITE_CACHE: Failed getting key from cache %s" % cache_key)

                if cache_result:
                    _logger.debug("WEBSITE_CACHE: Getting from cache %s" % cache_key)
                    return cache_result
                else:
                    content = super(QWeb, self).render(template, values, **options)
                    expiry = int(settings.get('expiry', 0))
                    client.set(cache_key, content, expiry)
                    _logger.debug("WEBSITE_CACHE: Adding to cache %s" % cache_key)
                    return content
            else:
                _logger.debug("WEBSITE_CACHE: No user, no caching at all - template: %s" % template_key)
                return super(QWeb, self).render(template, values, **options)
        else:
            _logger.debug("WEBSITE_CACHE: Not caching at all - template: %s" % template_key)
            return super(QWeb, self).render(template, values, **options)

Hope you liked it! We're hopping that Odoo 14.0 has caching embebed on the QWeb templating and maybe also on the controllers for caching database calls. If that doesn't happen then maybe it's worth to structure the website url's in a way that we can use NGINX cache.