How can I view the objects that are removed from the varnish cache in real-time? I know that I can monitor for PURGE events using varnishlog, and I know that I can monitor varnishstat to see the counts of objects evicted, banned, and lru nuked. But how can I actually view *what* is removed from varnish's cache live (as it happens)?

I have Two Server One which have install magento 2 (ip - 129.89.188.244 port 80) and Varnish on other server (ip - 129.89.188.245 port 80) My Vranish Configuration : File /etc/default/varnish:- DAEMON_OPTS="-a :80 \ -T 127.0.0.1:6082 \ -b 129.89.188.244:80 \ -f /etc/varnish/default.vcl \ -S /etc/varnish/secret \ -s malloc,256m" netstat -tulpn :- tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 1288/sshd tcp 0 0 127.0.0.1:6082 0.0.0.0:* LISTEN 11115/varnishd tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 11115/varnishd tcp6 0 0 :::22 :::* LISTEN 1288/sshd tcp6 0 0 :::80 :::* LISTEN 11115/varnishd /etc/varnish/default.vcl : - # VCL version 5.0 is not supported so it should be 4.0 even though actually used Varnish version is 5 vcl 4.0; import std; # The minimal Varnish version is 5.0 # For SSL offloading, pass the following header in your proxy server or load balancer: 'X-Forwarded-Proto: https' backend default { .host = "129.89.188.244"; .port = "80"; .first_byte_timeout = 600s; .probe = { .url = "/pub/health_check.php"; .timeout = 2s; .interval = 5s; .window = 10; .threshold = 5; } } acl purge { "129.89.188.245"; "127.0.0.1"; "localhost"; } sub vcl_recv { if (req.method == "PURGE") { if (client.ip !~ purge) { return (synth(405, "Method not allowed")); } # To use the X-Pool header for purging varnish during automated deployments, make sure the X-Pool header # has been added to the response in your backend server config. This is used, for example, by the # capistrano-magento2 gem for purging old content from varnish during it's deploy routine. if (!req.http.X-Magento-Tags-Pattern && !req.http.X-Pool) { return (synth(400, "X-Magento-Tags-Pattern or X-Pool header required")); } if (req.http.X-Magento-Tags-Pattern) { ban("obj.http.X-Magento-Tags ~ " + req.http.X-Magento-Tags-Pattern); } if (req.http.X-Pool) { ban("obj.http.X-Pool ~ " + req.http.X-Pool); } return (synth(200, "Purged")); } if (req.method != "GET" && req.method != "HEAD" && req.method != "PUT" && req.method != "POST" && req.method != "TRACE" && req.method != "OPTIONS" && req.method != "DELETE") { /* Non-RFC2616 or CONNECT which is weird. */ return (pipe); } # We only deal with GET and HEAD by default if (req.method != "GET" && req.method != "HEAD") { return (pass); } # Bypass shopping cart, checkout and search requests if (req.url ~ "/checkout" || req.url ~ "/catalogsearch") { return (pass); } # Bypass health check requests if (req.url ~ "/pub/health_check.php") { return (pass); } # Set initial grace period usage status set req.http.grace = "none"; # normalize url in case of leading HTTP scheme and domain set req.url = regsub(req.url, "^http[s]?://", ""); # collect all cookies std.collect(req.http.Cookie); # Compression filter. See https://www.varnish-cache.org/trac/wiki/FAQ/Compression if (req.http.Accept-Encoding) { if (req.url ~ "\.(jpg|jpeg|png|gif|gz|tgz|bz2|tbz|mp3|ogg|swf|flv)$") { # No point in compressing these unset req.http.Accept-Encoding; } elsif (req.http.Accept-Encoding ~ "gzip") { set req.http.Accept-Encoding = "gzip"; } elsif (req.http.Accept-Encoding ~ "deflate" && req.http.user-agent !~ "MSIE") { set req.http.Accept-Encoding = "deflate"; } else { # unkown algorithm unset req.http.Accept-Encoding; } } # Remove Google gclid parameters to minimize the cache objects set req.url = regsuball(req.url,"\?gclid=[^&]+$",""); # strips when QS = "?gclid=AAA" set req.url = regsuball(req.url,"\?gclid=[^&]+&","?"); # strips when QS = "?gclid=AAA&foo=bar" set req.url = regsuball(req.url,"&gclid=[^&]+",""); # strips when QS = "?foo=bar&gclid=AAA" or QS = "?foo=bar&gclid=AAA&bar=baz" # Static files caching if (req.url ~ "^/(pub/)?(media|static)/") { # Static files should not be cached by default return (pass); # But if you use a few locales and don't use CDN you can enable caching static files by commenting previous line (#return (pass);) and uncommenting next 3 lines #unset req.http.Https; #unset req.http.X-Forwarded-Proto; #unset req.http.Cookie; } return (hash); } sub vcl_hash { if (req.http.cookie ~ "X-Magento-Vary=") { hash_data(regsub(req.http.cookie, "^.*?X-Magento-Vary=([^;]+);*.*$", "\1")); } # For multi site configurations to not cache each other's content if (req.http.host) { hash_data(req.http.host); } else { hash_data(server.ip); } # To make sure http users don't see ssl warning if (req.http.X-Forwarded-Proto) { hash_data(req.http.X-Forwarded-Proto); } } sub vcl_backend_response { set beresp.grace = 3d; if (beresp.http.content-type ~ "text") { set beresp.do_esi = true; } if (bereq.url ~ "\.js$" || beresp.http.content-type ~ "text") { set beresp.do_gzip = true; } if (beresp.http.X-Magento-Debug) { set beresp.http.X-Magento-Cache-Control = beresp.http.Cache-Control; } # cache only successfully responses and 404s if (beresp.status != 200 && beresp.status != 404) { set beresp.ttl = 0s; set beresp.uncacheable = true; return (deliver); } elsif (beresp.http.Cache-Control ~ "private") { set beresp.uncacheable = true; set beresp.ttl = 86400s; return (deliver); } # validate if we need to cache it and prevent from setting cookie # images, css and js are cacheable by default so we have to remove cookie also if (beresp.ttl > 0s && (bereq.method == "GET" || bereq.method == "HEAD")) { unset beresp.http.set-cookie; } # If page is not cacheable then bypass varnish for 2 minutes as Hit-For-Pass if (beresp.ttl = 0s) { # Hit within TTL period return (deliver); } if (std.healthy(req.backend_hint)) { if (obj.ttl + 300s > 0s) { # Hit after TTL expiration, but within grace period set req.http.grace = "normal (healthy server)"; return (deliver); } else { # Hit after TTL and grace expiration return (miss); } } else { # server is not healthy, retrieve from cache set req.http.grace = "unlimited (unhealthy server)"; return (deliver); } } Now issue is when i open url :- 129.89.188.244 magento open but it's not getting from varnish but when i call varnish url :- 129.89.188.245 it will redirect to my magento url (129.89.188.244). in my varnish log page already cache but magento not using that page from varnish. If any help it's really appreciate.

RAM load rises to 85% from 32G I have varnish and nginx installed on my server. Recently I noticed that the load on RAM is very high. I am afraid that it will continue to grow and the site will stop working. My varnish has duplicated 200 treads in htop. it does not seem that if you reduce them, then the load will be less. I changed this in the daemon's natsryok but the threads are still 200 after rebooting the server In htop, I have from large loads Redis. mysqld and 200 varnish threads

SMA.s0.g_bytes  44.33M  0.00 . 44.31M 44.31M 44.31M
SMA.s0.g_space 6.81G 0.00 . 6.81G 6.81G 6.81G
SMA.Transient.g_bytes 602.84K 0.00 . 601.85K 601.85K 601.85K

MAIN.cache_hit 24576 0.00 12.21 0.39 0.43 0.43
MAIN.cache_miss 3730 0.00 1.85 1.92 2.07 2.07
MAIN.threads 200 0.00 . 200.00 200.00 200.00

How do I reconfigure the system to work correctly? and did not load my RAM activity on the site is now small. Server characteristics: CPU(s): 64 x Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz (2 Sockets) RAM: 32G 1TB NVME SSD Varnish configs: /lib/systemd/system/varnish.service

[Service]
Type=simple
LimitNOFILE=131072
LimitMEMLOCK=82000
ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,7018m
ExecReload=/usr/share/varnish/varnishreload
ProtectSystem=full
ProtectHome=true
PrivateTmp=true
PrivateDevices=true

[Install]
WantedBy=multi-user.target

/etc/default/varnish

DAEMON_OPTS="-a :6081 \
#             -T localhost:6082 \
#             -f /etc/varnish/default.vcl \
#             -S /etc/varnish/secret \
             -p http_resp_hdr_len=65536 \
             -p http_resp_size=98304 \
	     -p vcc_allow_inline_c=on \
             -p thread_pool_add_delay=2 \
             -p thread_pools=2 \
             -p thread_pool_min=25 \
             -p thread_pool_max=70 \
             -p timeout_linger=50 \
             -p first_byte_timeout=300 \
             -p pipe_timeout=300 \
             -p cli_buffer=65536 \
             -p syslog_cli_traffic=off \
             -p workspace_backend=64k \
             -p feature=+esi_disable_xml_check,+esi_ignore_other_elements,+esi_ignore_https \
             -T 127.0.0.1:6082 \
             -u varnish -g varnish \
             -f /etc/varnish/default.vcl \
             -S /etc/varnish/secret \
	     -t 172800 \
             -s malloc,7018m"

I am using nginx + varnish. I noticed that I have ~ 22GB / 32GB in use. I looked in htop and noticed that there are about 170 duplicate varnish processes. Tell me what could be the problem? Where to start looking?

I did install varnish with this guide . Then I want to start it, but FAILD and I cannot start varnish. I ran it in debug mode and I got this error: No VCL Available ! What is the problem here? Also enabled varnishlog, but no error log there.

need some help here. I have HAProxy that's load balancing 2 servers. HAProxy - 192.168.1.20:80 Server 1 - 192.168.1.18:80 Server 2 - 192.168.1.19:80 Now I would like to configure Varnish Cache to work with HAProxy. In the default.vcl file of Varnish I setted up the IP and port of the HAProxy, but when I when I type -curl -I 192.168.1.20 (IP of the HAProxy) I don't get a message from it. Any idea where I go wrong?

How can I get the installed Varnish cache version string from the command line?

Randomly our production site encounters a 503 error; each time after a varnish service restart, the requested site comes up normally. I looked at the varnish log and all I see is "Service Unavailable" but nothing else. I'm posting a portion of the varnishlog, the default.vcl and the varnish.params > * > 857875 - Begin bereq 857874 fetch - Timestamp Start: 1553896092.649343 0.000000 0.000000 - BereqMethod GET - BereqURL /huf_en_us/search/ajax/suggest/?_=1553896063503&q=pot+leaf+sock - BereqProtocol HTTP/1.0 - BereqHeader X-Real-IP: 172.16.204.112 - BereqHeader X-Forwarded-Proto: https - BereqHeader SSL-OFFLOADED: https - BereqHeader X-Forwarded-Port: 443 - BereqHeader Host: www.hufworldwide.com - BereqHeader CF-IPCountry: US - BereqHeader CF-RAY: 4bf50bf2cb163852-LAX - BereqHeader CF-Visitor: {"scheme":"https"} - BereqHeader accept: application/json, text/javascript, */*; q=0.01 - BereqHeader x-requested-with: XMLHttpRequest - BereqHeader accept-language: en-us - BereqHeader listrak-listening: 1 - BereqHeader referer: https://www.hufworldwide.com/huf_en_us/triple-triangle-pullover-hoodie-pf00100?color=tropical-green - BereqHeader dnt: 1 - BereqHeader user-agent: Mozilla/5.0 (iPhone; CPU iPhone OS 12_1_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/12.0 Mobile/15E148 Safari/604.1 - BereqHeader x-newrelic-id: VQcDV1JVABAEUVJVDgkOXw== - BereqHeader cookie: store=huf_en_us; _vuid=5759a66b-8175-43cc-a614-51a32c0d7a08; PHPSESSID=los3vfdipo3ako096qoo830m05; form_key=QY53qr4aLFMypLqi; private_content_version=927393d221b9f95d0d23c147b7a38d6b; section_data_ids=%7B%22directory-data%22%3A1553894394%2C%22cus - BereqHeader CF-Connecting-IP: 2606:a000:1216:8689:a5a4:dde1:69bd:10ac - BereqHeader CF-Pseudo-IPv4: 252.241.164.167 - BereqHeader True-Client-IP: 2606:a000:1216:8689:a5a4:dde1:69bd:10ac - BereqHeader CDN-Loop: cloudflare - BereqHeader X-Forwarded-For: 172.16.204.112, 127.0.0.1 - BereqHeader Accept-Encoding: gzip - BereqProtocol HTTP/1.1 - BereqHeader X-Varnish: 857875 - VCL_call BACKEND_FETCH - VCL_return fetch - BackendOpen 54 web02(172.16.204.102,,80) 172.16.204.104 60836 - Backend 54 d web02(172.16.204.102,,80) - Timestamp Bereq: 1553896092.649819 0.000476 0.000476 - Timestamp Beresp: 1553896093.563128 0.913785 0.913309 - BerespProtocol HTTP/1.1 - BerespStatus 503 - BerespReason Service Unavailable - BerespHeader Date: Fri, 29 Mar 2019 21:48:12 GMT - BerespHeader Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips mod_fcgid/2.3.9 PHP/7.0.30 - BerespHeader X-Powered-By: PHP/7.0.30 - BerespHeader X-Content-Type-Options: nosniff - BerespHeader X-XSS-Protection: 1; mode=block - BerespHeader X-Frame-Options: SAMEORIGIN - BerespHeader Expires: Thu, 19 Nov 1981 08:52:00 GMT - BerespHeader Cache-Control: no-store, no-cache, must-revalidate - BerespHeader Pragma: no-cache - BerespHeader Set-Cookie: PHPSESSID=los3vfdipo3ako096qoo830m05; expires=Sun, 28-Apr-2019 21:48:13 GMT; Max-Age=2592000; path=/; domain=www.hufworldwide.com; secure; HttpOnly - BerespHeader X-NewRelic-App-Data: PxQGUFJVDAUIR1VXBAYOXl0IFB9AMQYAZBBZDEtZV0ZaClc9HiBQFg1ZWT1JJUpcXhAiDVlFRQkIXVNBPkkuA1cHVhZXZGh0QQRUBEYOQQk4anYRFj9kdUILDxZ0XlkSFl5aXwcUPz55DEwHSltRQkcKS0MdUR1SVAYHUUpTFgoAXVRQGxwGSkYCAwZbUltQB1oMWw0MBQVWRxUHUA1ABzk= - BerespHeader Connection: close - BerespHeader Transfer-Encoding: chunked - BerespHeader Content-Type: text/html; charset=UTF-8 - TTL RFC -1 -1 -1 1553896094 1553896094 1553896092 375007920 0 - VCL_call BACKEND_RESPONSE - TTL VCL 0 10 0 1553896094 - VCL_return deliver - BerespHeader Content-Encoding: gzip - Storage malloc Transient - ObjProtocol HTTP/1.1 - ObjStatus 503 - ObjReason Service Unavailable - ObjHeader Date: Fri, 29 Mar 2019 21:48:12 GMT - ObjHeader Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips mod_fcgid/2.3.9 PHP/7.0.30 - ObjHeader X-Powered-By: PHP/7.0.30 - ObjHeader X-Content-Type-Options: nosniff - ObjHeader X-XSS-Protection: 1; mode=block - ObjHeader X-Frame-Options: SAMEORIGIN - ObjHeader Expires: Thu, 19 Nov 1981 08:52:00 GMT - ObjHeader Cache-Control: no-store, no-cache, must-revalidate - ObjHeader Pragma: no-cache - ObjHeader Set-Cookie: PHPSESSID=los3vfdipo3ako096qoo830m05; expires=Sun, 28-Apr-2019 21:48:13 GMT; Max-Age=2592000; path=/; domain=www.hufworldwide.com; secure; HttpOnly - ObjHeader X-NewRelic-App-Data: PxQGUFJVDAUIR1VXBAYOXl0IFB9AMQYAZBBZDEtZV0ZaClc9HiBQFg1ZWT1JJUpcXhAiDVlFRQkIXVNBPkkuA1cHVhZXZGh0QQRUBEYOQQk4anYRFj9kdUILDxZ0XlkSFl5aXwcUPz55DEwHSltRQkcKS0MdUR1SVAYHUUpTFgoAXVRQGxwGSkYCAwZbUltQB1oMWw0MBQVWRxUHUA1ABzk= - ObjHeader Content-Type: text/html; charset=UTF-8 - ObjHeader Content-Encoding: gzip - Fetch_Body 2 chunked - - Gzip G F E 16777 6581 80 52568 52578 - Timestamp BerespBody: 1553896093.572716 0.923373 0.009588 - BackendClose 54 web02(172.16.204.102,,80) - Length 6581 - BereqAcct 3058 0 3058 869 16789 17658 - End default.vcl > vcl 4.0; > > import std; import directors; > > REM The minimal Varnish version is 4.0 For SSL offloading, pass the following header in your proxy server or load balancer: > 'SSL-OFFLOADED: https' > > backend web01 { > .host = "172.16.204.101"; > .port = "80"; > .connect_timeout = 1600s; > .first_byte_timeout = 1600s; > .between_bytes_timeout = 1600s; > .max_connections = 20000; } > > backend web02 { > .host = "172.16.204.102"; > .port = "80"; > .connect_timeout = 1600s; > .first_byte_timeout = 1600s; > .between_bytes_timeout = 1600s; > .max_connections = 20000; } > > backend web03 { > .host = "172.16.204.120"; > .port = "80"; > .connect_timeout = 1600s; > .first_byte_timeout = 1600s; > .between_bytes_timeout = 1600s; > .max_connections = 20000; } > > backend web04 { > .host = "172.16.204.121"; > .port = "80"; > .connect_timeout = 1600s; > .first_byte_timeout = 1600s; > .between_bytes_timeout = 1600s; > .max_connections = 20000; } varnish.params > REM Varnish environment configuration description. This was derived from > REM the old style sysconfig/defaults settings > > REM Set this to 1 to make systemd reload try to switch VCL without restart. RELOAD_VCL=1 > > REM Set WARMUP_TIME to force a delay in reload-vcl between vcl.load and vcl.use > REM This is useful when backend probe definitions need some time before declaring > REM configured backends healthy, to avoid routing traffic to a non-healthy backend. > REM WARMUP_TIME=0 > > REM Main configuration file. You probably want to change it. VARNISH_VCL_CONF=/etc/varnish/default.vcl > > REM Default address and port to bind to. Blank address means all IPv4 > REM and IPv6 interfaces, otherwise specify a host name, an IPv4 dotted > REM quad, or an IPv6 address in brackets. > REM VARNISH_LISTEN_ADDRESS=192.168.1.5 VARNISH_LISTEN_PORT=8080 > > REM Admin interface listen address and port VARNISH_ADMIN_LISTEN_ADDRESS=127.0.0.1 VARNISH_ADMIN_LISTEN_PORT=6082 > > REM Shared secret file for admin interface VARNISH_SECRET_FILE=/etc/varnish/secret > > REM Backend storage specification, see Storage Types in the varnishd(5) > REM man page for details. VARNISH_STORAGE="malloc,24G" > > REM User and group for the varnishd worker processes VARNISH_USER=builder VARNISH_GROUP=builder > > REM Other options, see the man page varnishd(1) DAEMON_OPTS="-p thread_pool_min=200 -p thread_pool_max=2000 -p thread_pool_timeout=600 > -p http_resp_hdr_len=90000 -p http_max_hdr=200"

Is possible with Varnish configure multiple backends for one site? For example I have high avalibility site with multiple backends. For example I have site www.goodies.com which is running on php 5.6. So I have 3 www-nodes as backend www-node5.6a www-node5.6b www-node5.6c I have configured backends in Varnish like this: backend www-node5.6a { .host = "10.0.0.11"; .port = "80"; .connect_timeout = 6000s; .first_byte_timeout = 6000s; .between_bytes_timeout = 6000s; } backend www-node5.6b { .host = "10.0.0.12"; .port = "80"; .connect_timeout = 6000s; .first_byte_timeout = 6000s; .between_bytes_timeout = 6000s; } backend www-node5.6c { .host = "10.0.0.13"; .port = "80"; .connect_timeout = 6000s; .first_byte_timeout = 6000s; .between_bytes_timeout = 6000s; } my question is, how to configure Varnish so user will be randomly use to any of this backends for this specific one site?

In My server, running Linux Redhat, I have NGINX, PHP Installed already. Now I want to install PHP-FPM, VARNISH, APC, FASTCGI as well Memcache to speed up my server for my wordpress website. I read this and so I want to install it in my server. http://www.danielmiessler.com/blog/optimizing-wordpress-with-nginx-varnish-w3-total-cache-amazon-s3-and-memcached . I have excidently done update and after that I get following error yum install varnish Loaded plugins: fastestmirror, priorities, protectbase, replace Loading mirror speeds from cached hostfile * base: centos.mirror.secureax.com * epel: mirror.nus.edu.sg * extras: centos.mirror.secureax.com * mratwork-epel: mirror.nus.edu.sg * remi: mirror.smartmedia.net.id * updates: centos.mirror.secureax.com http://centos.alt.ru/repository/centos/6/x86_64/repodata/repomd.xml : [Errno 14] PYCURL ERROR 22 - "The requested URL returned error: 403 Forbidden" Trying other mirror. Error: Cannot retrieve repository metadata (repomd.xml) for repository: mratwork-centalt. Please verify its path and try again How to install varnish? **Files** **CentOS-Base.repo** # CentOS-Base.repo # # The mirror system uses the connecting IP address of the client and the # update status of each mirror to pick mirrors that are updated to and # geographically close to the client. You should use this for CentOS updates # unless you are manually picking other mirrors. # # If the mirrorlist= does not work for you, as a fall back you can try the # remarked out baseurl= line instead. # # [base] name=CentOS-$releasever - Base mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os #baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 #released updates [updates] name=CentOS-$releasever - Updates mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates #baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 #additional packages that may be useful [extras] name=CentOS-$releasever - Extras mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=extras #baseurl=http://mirror.centos.org/centos/$releasever/extras/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 #additional packages that extend functionality of existing packages [centosplus] name=CentOS-$releasever - Plus mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=centosplus #baseurl=http://mirror.centos.org/centos/$releasever/centosplus/$basearch/ gpgcheck=1 enabled=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 #contrib - packages by Centos Users [contrib] name=CentOS-$releasever - Contrib mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=contrib #baseurl=http://mirror.centos.org/centos/$releasever/contrib/$basearch/ gpgcheck=1 enabled=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 **CentOS-Debuginfo.repo** # CentOS-Debug.repo # # The mirror system uses the connecting IP address of the client and the # update status of each mirror to pick mirrors that are updated to and # geographically close to the client. You should use this for CentOS updates # unless you are manually picking other mirrors. # # All debug packages from all the various CentOS-5 releases # are merged into a single repo, split by BaseArch # # Note: packages in the debuginfo repo are currently not signed # [debug] name=CentOS-6 - Debuginfo baseurl=http://debuginfo.centos.org/6/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-Debug-6 enabled=0 **CentOS-Media.repo** # CentOS-Media.repo # # This repo can be used with mounted DVD media, verify the mount point for # CentOS-6. You can use this repo and yum to install items directly off the # DVD ISO that we release. # # To use this repo, put in your DVD and use it with the other repos too: # yum --enablerepo=c6-media [command] # # or for ONLY the media repo, do this: # # yum --disablerepo=\* --enablerepo=c6-media [command] [c6-media] name=CentOS-$releasever - Media baseurl=file:///media/CentOS/ file:///media/cdrom/ file:///media/cdrecorder/ gpgcheck=1 enabled=0 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 **CentOS-Vault.repo** # CentOS-Vault.repo # # CentOS Vault holds packages from previous releases within the same CentOS Version # these are packages obsoleted by the current release and should usually not # be used in production #----------------- [C6.0-base] name=CentOS-6.0 - Base baseurl=http://vault.centos.org/6.0/os/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.0-updates] name=CentOS-6.0 - Updates baseurl=http://vault.centos.org/6.0/updates/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.0-extras] name=CentOS-6.0 - Extras baseurl=http://vault.centos.org/6.0/extras/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.0-contrib] name=CentOS-6.0 - Contrib baseurl=http://vault.centos.org/6.0/contrib/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.0-centosplus] name=CentOS-6.0 - CentOSPlus baseurl=http://vault.centos.org/6.0/centosplus/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 #----------------- [C6.1-base] name=CentOS-6.1 - Base baseurl=http://vault.centos.org/6.1/os/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.1-updates] name=CentOS-6.1 - Updates baseurl=http://vault.centos.org/6.1/updates/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.1-extras] name=CentOS-6.1 - Extras baseurl=http://vault.centos.org/6.1/extras/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.1-contrib] name=CentOS-6.1 - Contrib baseurl=http://vault.centos.org/6.1/contrib/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.1-centosplus] name=CentOS-6.1 - CentOSPlus baseurl=http://vault.centos.org/6.1/centosplus/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 #----------------- [C6.2-base] name=CentOS-6.2 - Base baseurl=http://vault.centos.org/6.2/os/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.2-updates] name=CentOS-6.2 - Updates baseurl=http://vault.centos.org/6.2/updates/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.2-extras] name=CentOS-6.2 - Extras baseurl=http://vault.centos.org/6.2/extras/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.2-contrib] name=CentOS-6.2 - Contrib baseurl=http://vault.centos.org/6.2/contrib/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.2-centosplus] name=CentOS-6.2 - CentOSPlus baseurl=http://vault.centos.org/6.2/centosplus/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 #----------------- [C6.3-base] name=CentOS-6.3 - Base baseurl=http://vault.centos.org/6.3/os/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.3-updates] name=CentOS-6.3 - Updates baseurl=http://vault.centos.org/6.3/updates/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.3-extras] name=CentOS-6.3 - Extras baseurl=http://vault.centos.org/6.3/extras/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.3-contrib] name=CentOS-6.3 - Contrib baseurl=http://vault.centos.org/6.3/contrib/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.3-centosplus] name=CentOS-6.3 - CentOSPlus baseurl=http://vault.centos.org/6.3/centosplus/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 #----------------- [C6.4-base] name=CentOS-6.4 - Base baseurl=http://vault.centos.org/6.4/os/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.4-updates] name=CentOS-6.4 - Updates baseurl=http://vault.centos.org/6.4/updates/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.4-extras] name=CentOS-6.4 - Extras baseurl=http://vault.centos.org/6.4/extras/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.4-contrib] name=CentOS-6.4 - Contrib baseurl=http://vault.centos.org/6.4/contrib/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0 [C6.4-centosplus] name=CentOS-6.4 - CentOSPlus baseurl=http://vault.centos.org/6.4/centosplus/$basearch/ gpgcheck=1 gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-6 enabled=0

I need to have varnishncsa split logs to separate files based on vhost. I know I can make an override configuration in /etc/systemd/system/varnishncsa.service.d that will filter on a specific vhost. Something like this somedomain.conf: [Service] RuntimeDirectory=varnishncsa Type=forking User=varnishlog Group=varnish ExecStart= ExecStart=/usr/bin/varnishncsa -q "ReqHeader ~ '^Host: somedomain.com'" -D -a -w /var/log/varnish/somedomain.log -P /run/varnishncsa/varnishncsa.pid -F '%%{X-Forwarded-For}i %%l %%u %%t "%%r" %%s %%b "%%{Referer}i" "%%{User-agent}i"' ExecReload=/bin/kill -HUP $MAINPID PrivateDevices=true PrivateNetwork=true PrivateTmp=true ProtectHome=true ProtectSystem=full PIDFile=/run/varnishncsa/varnishncsa.pid But how can i make it generate a separate log file for each of many vhosts? I want to feed these per domain logs into awstats.

Operating system: debian 9, varnish version: 5.0.0 I have turned off IPv6 support on my machine and varnish crashes upon start with the following error: Error: Cannot open socket: :6081: Address family not supported by protocol I know this question has been asked here here , but it does not solve my problem. Also, I can't find any IP-related configuration option in the config files. So, how can I run varnish cache without IPv6 support?

This is my setup : Server 1 = Nginx is receiving the request on port 443 and is used as a reverse proxy to send it to Varnish 5, on the same server on port 80. Varnish is load balancing requests on servers 2 and 3 (which are identical) on port 443. Server 2 & 3 = Apache is receiving the requests on port 443 and access to the app. SSL certificates are installed on all servers. When I try to access the website I have this error 400: > Your browser sent a request that this server could not understand. Reason: You're speaking plain HTTP to an SSL-enabled server port. Instead use the HTTPS scheme to access this URL, please. Here are my configs : Nginx : server { listen 443 ssl; server_name server.mydomain.com; ssl_certificate /etc/letsencrypt/live/server.mydomain.com/fullchain.pem; ssl_certificate_key/etc/letsencrypt/live/server.mydomain.com/privkey.pem; location / { proxy_pass http://127.0.0.1:80 ; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Proto https; proxy_set_header X-Forwarded-Port 443; proxy_set_header X-Secure on; } } Varnish: backend server1 { .host = "xx.xx.xx.xxx"; .port = "443"; } backend server2 { .host = "xx.xx.xx.xxx"; .port = "443"; } sub vcl_recv { if (req.restarts == 0) { if (req.http.x-forwarded-for) { set req.http.X-Forwarded-For = req.http.X-Forwarded-For + ", " + client.ip; } else { set req.http.X-Forwarded-For = client.ip; } } if (req.http.X-Real-IP) { set req.http.X-Forwarded-For = req.http.X-Real-IP; } else { set req.http.X-Forwarded-For = client.ip; } ... } Apache: ServerName server.mydomain.com DocumentRoot /var/www/mydomain/ AllowOverride All Order allow,deny allow from all SSLEngine on SSLCertificateFile /etc/letsencrypt/live/server.mydomain.com/cert.pem SSLCertificateKeyFile /etc/letsencrypt/live/server.mydomain.com/privkey.pem SSLCertificateChainFile /etc/letsencrypt/live/server.mydomain.com/chain.pem SSLProtocol all -SSLv2 -SSLv3 SSLHonorCipherOrder on SSLCompression off SSLOptions +StrictRequire SSLCipherSuite ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-E$ LogLevel warn ErrorLog ${APACHE_LOG_DIR}/server.mydomain.com-error.log CustomLog ${APACHE_LOG_DIR}/server.mydomain.com-access.log combined I understand the problem, but didn't find he solution. Any advice? Regards

I try to read the varnish version from a Linux Command line, but the following does not work: varnishd -V | grep -P '(?<=varnish-)[0-9.]+' -o This always returns this varnishd (varnish-3.0.3 revision 9e6a70f) Copyright (c) 2006 Verdens Gang AS Copyright (c) 2006-2011 Varnish Software AS instead of just this 3.0.3 Any hints on what I do wrong? Thanks

This is my script to install Varnish. I run it each time I raise up a new server environment on a VPS. cd ~ apt-get update apt-get install varnish -y sed -i 's/Listen 80/Listen 8080/g' /etc/apache2/ports.conf sed -i 's/\*\:80/\*\:8080/g' /etc/apache2/sites-available/000-default.conf sed -i 's/\*\:80/\*\:8080/g' /etc/apache2/sites-available/domain1.tld.conf && a2ensite domain1.tld.conf sed -i 's/\*\:80/\*\:8080/g' /etc/apache2/sites-available/domain2.tld.conf && a2ensite domain2.tld.conf sed -i 's/\*\:80/\*\:8080/g' /etc/apache2/sites-available/domain3.tld.conf && a2ensite domain3.tld.conf sed -i 's/\*\:80/\*\:8080/g' /etc/apache2/sites-available/domain4.tld.conf && a2ensite domain4.tld.conf mkdir -p /etc/systemd/system/varnish.service.d # Be aware! You might not need this in the future. cat /etc/systemd/system/varnish.service.d/customexec.conf [Service] ExecStart= ExecStart=/usr/sbin/varnishd -j unix,user=vcache -F -a :80 -T localhost:6082 -f /etc/varnish/default.vcl -S /etc/varnish/secret -s malloc,256m VARNISH systemctl restart apache2.service && systemctl daemon-reload && systemctl restart varnish.service This code segment seems quite "heavy", especially the repetitiveness of the sed operations regarding domain.tld. This get's even "heavier" because I have a code segment which is similar in length that uses to me uninstall varnish and revert all changes just in case of desire. ## My question: What strategy will you take to make the installation script shorter in general (at least less rows, maybe also less commands), and in particular, to lower amount of sed operations. ## Notes: * I would assume that the first thing to do is to somehow unify ports.conf, 000-default.conf, and each .conf file of each site, all into one operation. Maybe via a for loop on /etc/apache2/ports.conf/ && /etc/apache2/sites-available/*/.

I am running varnish-4.0.4-3 on centos7. In the varnish logs, I find: 23 ReqUnset c X-Forwarded-For: 172.16.1.2 23 ReqHeader c X-Forwarded-For: 172.16.1.2, 127.0.0.1 How can I disable X-Forwarded-For modification please?

This seems like it should be simple, but I am struggling to make it work. I am trying to set up a systemd service so output from a logging program (varnishncsa) that outputs log entries from memory can be piped to cronolog. The only way I have been able to make it work is in the foreground. I am trying to make it work as a systemd service. The script that takes over the foreground is: #!/bin/bash /usr/bin/varnishncsa -F '%{X-Real-IP}i %l %u %t "%r" %s %b "%{Referer}i" "%{User-agent}i"' -q "ReqHeader:Host ~ '(^|\.)example\.com$'" -C |/usr/sbin/cronolog "/example/varnish_access_log.%Y-%m-%d" The systemd service setup I am trying is: [Unit] Description=Example website Varnish Cache HTTP accelerator NCSA logging daemon After=varnish.service [Service] RuntimeDirectory=varnishncsa Type=forking PIDFile=/run/varnishncsa/varnishncsa-example.pid User=varnish Group=varnish ExecStart=/usr/local/bin/varnishncsa-example.sh ExecReload=/bin/kill -HUP $MAINPID [Install] WantedBy=multi-user.target Which is based on the provided systemd service for running as a daemon and writing to a file (I can't run it directly as a daemon, using -D and -P, and pipe the output): [Unit] Description=Varnish Cache HTTP accelerator NCSA logging daemon After=varnish.service [Service] RuntimeDirectory=varnishncsa Type=forking PIDFile=/run/varnishncsa/varnishncsa.pid User=varnish Group=varnish ExecStart=/usr/bin/varnishncsa -a -w /var/log/varnish/varnishncsa.log -D -P /run/varnishncsa/varnishncsa.pid ExecReload=/bin/kill -HUP $MAINPID [Install] WantedBy=multi-user.target I have tried many options to make it run in the background, but can't get anything to work, and can't find anything online that deals with this situation specifically. Probably because systemd is still quite new, or others doing this have more idea what they are doing! Any help greatly appreciated.

I have a server running Varnish and nginx. The frontend website reports a 503 error: > Error 503 Service Unavailable > > Service Unavailable > > Guru Meditation:
XID: 317911182 > > Varnish cache server This is the content of /etc/varnish/foo.vcl: backend default { .host = "127.0.0.1"; .port = "8080"; .connect_timeout = 1s; .first_byte_timeout = 90s; .between_bytes_timeout = 90s; } varnishlog apparently shows no issue: 0 CLI - Rd ping 0 CLI - Wr 200 19 PONG 1486463718 1.0 0 CLI - Rd ping 0 CLI - Wr 200 19 PONG 1486463721 1.0 0 CLI - Rd ping 0 CLI - Wr 200 19 PONG 1486463724 1.0 0 CLI - Rd ping 0 CLI - Wr 200 19 PONG 1486463727 1.0 However, there is no service running on port 8080 on localhost: server# netstat -anp | grep 8080 server# telnet localhost 8080 Trying ::1... telnet: connect to address ::1: Connection refused Trying 127.0.0.1... telnet: connect to address 127.0.0.1: Connection refused and in fact the nginx log reports an error while trying to connect to it: 2017/02/07 11:51:11 [error] 2008#0: *188 connect() failed (111: Connection refused) while connecting to upstream, client: 10.2.3.4, server: _, request: "GET /mydir/ HTTP/1.1", upstream: "http://127.0.0.1:8080/mydir/ ", host: "myhost.example.com" What could I check to further troubleshoot the issue?

The following error message is taken from my log message. varnishd server is getting killed for over use of memory. We have done all the possible settings to avoid it, Still the same problem happens with multiple VMs, not just one rare incident. Aug 26 08:26:25 host kernel: [15332346.565597] Out of memory in UB 206: OOM killed process 24966 (varnishd) score 0 vm:520484kB, rss:203660kB, swap:12124kB Aug 26 08:26:25 host varnishd: Child (24966) died signal=9 We are having multiple VMs, most of them are running websites. In some VMs, where the load is high, due to limited RAM, the varnishd is getting killed. Some relavent configuration value from /etc/sysconfig/varnish VARNISH_MAX_THREADS=1000 VARNISH_STORAGE_SIZE=256M Server is having 2GB RAM. What configuration needs to be done for varnishd to avoid this problem?

Do we need to deploy varnish if our Web application is using memcached or do we need memcached if we are using varnish to cache web contents. Can someone recommend some scenarios where should we use one or the other or may be both.