Skip to content

Index.rst

Latest commit

 

History

History
281 lines (224 loc) · 11.3 KB

Index.rst

File metadata and controls

281 lines (224 loc) · 11.3 KB

Varnish integration for Neos

This package provides a out-of-the-box seamless integration between Varnish and Neos. It basically makes Neos send Cache-Control headers and BAN requests to Varnish for all document nodes.

When installed, Neos send headers for cache lifetime and cache invalidation requests.

Configuration

There are several configuration options can/needs to be set:

  • The URL for the Varnish proxy/proxies (By default the Varnish cache is expected to run on http://127.0.0.1)
    Flowpack.Varnish.varnishUrl allows string or array with URLs (skip trailing slash)
  • Cache header for default shared maximum age (smax-age) - default cache and maximum cache TTL, e.g. 86400 for 24 h
    Flowpack.Varnish.cacheHeaders.defaultSharedMaximumAge accepts integers (seconds) - (defaults to NULL) If not set, the Varnish configuration needs to cache by default since no Cache-Control header will be sent
  • Disable sending of cache headers - can be used to disable Varnish on staging environment e.g.
    Flowpack.Varnish.cacheHeaders.disabled accepts boolean value (defaults to FALSE)
  • Since 4.1.0: The generated Cache Tags can be shortened using this option
    Flowpack.Varnish.cacheHeaders.shortenCacheTags accepts boolean value (defaults to FALSE)
  • Since 4.1.0: If shortenCacheTags is enabled, this option controls the length of the used md5 for tags
    Flowpack.Varnish.cacheHeaders.cacheTagLength accepts integer value (defaults to 8)
  • Reverse lookup port can be set to allow debugging in the backend module if the web server port is not 80
    Flowpack.Varnish.reverseLookupPort accepts integer (defaults to NULL)
  • Ignored cache tags can be used to ignore certain cache tags from being cleared at all (useful for optimizing)
    Flowpack.Varnish.ignoredCacheTags accepts array of strings (defaults to NULL) E.g. 'Neos.Neos:Document' which is used in 'Neos.Neos:Menu' elements
  • To disable the Varnish caching for deployment reasons you can set this option to FALSE
    Flowpack.Varnish.enabled accepts boolean value (defaults to TRUE)

How it works

Varnish works by caching HTTP requests based on their Cache-Control headers and every url is a unique cache entry.

This package works by integrating with the TypoScript Content Cache in Neos for sending proper Cache-Control headers in the HTTP response for document nodes. Connected by a signal it checks if the controller is the NodeController and the node is in the live workspace. Additionally checks for node being caching being disabled for the node using the disableVarnishCache (automatically added to node properties for document nodes types) property and sending cache headers haven't been disabled. A Time-to-live property is also available, making it possible to set a custom TTL for a specific document node.

*Note* The Varnish cache is only enabled for the live workspace.

If the document node is can be cached (no uncached segments) it will add a Cache-Control header as well as two custom headers X-Site and X-Cache-Tags. The X-Site header is auto-generated and used to separate installations and the X-Cache-Tags contains all tags used on the page, needed for cache clearing. The tags are collected by replacing the TypoScript Content Cache frontend with a custom one, that allows for retrieving the meta data (tags) for existing entries. This is taken being done in the CacheControlService. If a uncached segment needs to be ignored for determining if a page can be cached, the flowpackVarnishIgnoreUncached variable can be used in the TypoScript cache configuration. This can be useful in case of a uncached segment that depends on GET parameters, which means the URL is different thus a separate Varnish cache entry, but the page is mostly identical except for the segment. Alternatively this can be solved by adding the GET parameter to the cache entryIdentifier.

Example:

@cache {
      mode = 'uncached'
      context {
              1 = 'documentNode'
      }
      flowpackVarnishIgnoreUncached = true
}

*Note* For a page to be cached, it must not contains any uncached parts (e.g. plugins which are uncachable by default).

When a node is published to the live workspace a ban request is send to the Varnish proxy with the node's cache tags.

For custom flushing of the cache, e.g. on node import, either use flushForNode. The ContentCacheFlusherService generates cache tags for all published nodes and will send one ban request, using the VarnishBanService, containing all the tags to be cleared. The VarnishBanService has two methods banAll accepting domain & contentType (MIME type) and banByTags accepting tags and domain.

The package supports content dimensions as long as they are part of the URL.

The package uses the FriendsOfSymfony package FOSHttpCache for sending requests to the Varnish proxy/proxies and lends inspiration for how to interact with Varnish. More information can be found in it's documentation.

Debugging

For Varnish to cache pages correctly, the additional headers need to be available when requesting the page directly from the server. These include X-Site & X-Cache-Tags. If those headers aren't present the page won't be cached. This is likely due to the page not being cacheable or headers being disabled.

Enable debug logging by changing the configuration setting Flowpack.Varnish.log.backendOptions.severityThreshold to '%LOG_DEBUG%'

Also make sure the setting Flowpack.Varnish.cacheHeaders.disabled is not enabled.

Command

A command for clearing the Varnish cache is available with ./flow varnish:clear, which accepts two optional parameters domain and contentType (MIME type).

Backend module

To make controller and debugging the Varnish cache proxy easier, a Neos backend module is available. It allows for clearing cache all cache for the site with an optional content-type filter. Additionally allows for clearing cache by certain tags. Lastly it allows for searching for individual document nodes for clearing cache and fetching cache information for each of the found nodes. The module is accessible to

Additionally the configuration options are visible.

Screenshot of cache clearing in Neos Backend Module
Screenshot of node search in Neos Backend Module

Shared Varnish support

A unique token for every Flow installation is generated if one doesn't already exist. This is used to separate cache entries in Varnish for every installation to only clear for the correct one. This token is located in Data/Persistent/FlowpackVarnishSiteToken/VarnishSiteToken and can be copied to keep across installations.

Multi-site support

When having multiple sites the cache entries in Varnish are separated by only clearing for the first active domain for a site. This prevents clearing cache for all sites in a installation.

Surf deployment

When using Surf for deploying, it's recommended to clear the Varnish cache after a deployment. That can be done in your Surf deployment script:

$workflow->defineTask('x:varnishban', 'typo3.surf:typo3:flow:runcommand', array('command' => 'varnish:clear'));
$workflow->afterStage('switch', array('x:varnishban'), $application);

Required Varnish VCL

The package expects Varnish to handle BAN requests with the HTTP-Headers X-Host, X-Content-Type and X-Cache-Tags. This can be done by using the following example vcl:

Varnish 4:

vcl 4.0;
backend default {
        .host = "127.0.0.1";
        .port = "8080";
}

acl invalidators {
        "127.0.0.1";
}

sub vcl_recv {
        if (req.method == "BAN") {
                if (!client.ip ~ invalidators) {
                        return (synth(405, "Not allowed"));
                }

                if (req.http.X-Cache-Tags) {
                        ban("obj.http.X-Host ~ " + req.http.X-Host
                                + " && obj.http.X-Url ~ " + req.http.X-Url
                                + " && obj.http.content-type ~ " + req.http.X-Content-Type
                                + " && obj.http.X-Cache-Tags ~ " + req.http.X-Cache-Tags
                                + " && obj.http.X-Site ~ " + req.http.X-Site
                        );
                } else {
                        ban("obj.http.X-Host ~ " + req.http.X-Host
                                + " && obj.http.X-Url ~ " + req.http.X-Url
                                + " && obj.http.content-type ~ " + req.http.X-Content-Type
                                + " && obj.http.X-Site ~ " + req.http.X-Site
                        );
                }

                return (synth(200, "Banned"));
        }
}

sub vcl_backend_response {
        # Set ban-lurker friendly custom headers
        set beresp.http.X-Url = bereq.url;
        set beresp.http.X-Host = bereq.http.host;
        set beresp.http.X-Cache-TTL = beresp.ttl;
}

sub vcl_deliver {
        # Send debug headers if a X-Cache-Debug header is present from the client or the backend
        if (req.http.X-Cache-Debug || resp.http.X-Cache-Debug) {
                if (resp.http.X-Varnish ~ " ") {
                        set resp.http.X-Cache = "HIT";
                } else {
                        set resp.http.X-Cache = "MISS";
                }
        } else {
                # Remove ban-lurker friendly custom headers when delivering to client
                unset resp.http.X-Url;
                unset resp.http.X-Host;
                unset resp.http.X-Cache-Tags;
                unset resp.http.X-Site;
                unset resp.http.X-Cache-TTL;
        }
}

Varnish 3:

backend default {
        .host = "127.0.0.1";
        .port = "8080";
}

acl invalidators {
        "127.0.0.1";
}

sub vcl_recv {
        if (req.request == "BAN") {
                if (!client.ip ~ invalidators) {
                        error 405 "Not allowed.";
                }

                if (req.http.X-Cache-Tags) {
                        ban("obj.http.X-Host ~ " + req.http.X-Host
                                + " && obj.http.X-Url ~ " + req.http.X-Url
                                + " && obj.http.content-type ~ " + req.http.X-Content-Type
                                + " && obj.http.X-Cache-Tags ~ " + req.http.X-Cache-Tags
                                + " && obj.http.X-Site ~ " + req.http.X-Site
                        );
                } else {
                        ban("obj.http.X-Host ~ " + req.http.X-Host
                                + " && obj.http.X-Url ~ " + req.http.X-Url
                                + " && obj.http.content-type ~ " + req.http.X-Content-Type
                                + " && obj.http.X-Site ~ " + req.http.X-Site
                        );
                }

                error 200 "Banned";
        }
}

sub vcl_fetch {
        # Set ban-lurker friendly custom headers
        set beresp.http.X-Url = req.url;
        set beresp.http.X-Host = req.http.host;
        set beresp.http.X-Cache-TTL = beresp.ttl;
}

sub vcl_deliver {
        # Send debug headers if a X-Cache-Debug header is present from the client or the backend
        if (req.http.X-Cache-Debug || resp.http.X-Cache-Debug) {
                if (obj.hits > 0) {
                        set resp.http.X-Cache = "HIT";
                } else {
                        set resp.http.X-Cache = "MISS";
                }
        } else {
                # Remove ban-lurker friendly custom headers when delivering to client
                unset resp.http.X-Url;
                unset resp.http.X-Host;
                unset resp.http.X-Cache-Tags;
                unset resp.http.X-Site;
                unset resp.http.X-Cache-TTL;
        }
}

*Note* Example of full VCL configuration file (Varnish 3) – Use with care!