justquick/python-varnish

Simple Python interface for the Varnish management port

Simple Python interface for the Varnish management port

Author

Justin Quick <[email protected]>, Sandy Walsh <[email protected]>

Version

0.2

pip install python-varnish==0.2.1

If you are running a version of varnish older than 3.0 then install python-varnish==0.1.2 instead.

Varnish is a state-of-the-art, high-performance HTTP accelerator. For more information checkout Varnish Site

Varnish provides a simple telnet management interface for doing things like:

• reloading configurations
• purging URLs from cache
• view statistics
• start and stop the server

This Python API takes full advantage of the available commands and can run across multiple Varnish instances. Here are the features of this python module (compared to python-varnishadm)

• Uses telnetlib instead of raw sockets
• Implements threading module
• Can run commands across multiple Varnish instances
• More comprehensive methods, closely matching the management API (purge_*, vcl_*, etc.)
• Unittests

Example:

manager = VarnishManager( ('server1:6082', 'server2:6082') )
manager.run('ping')
manager.run('ban.url', '^/secret/$')
manager.run('ban.list')
manager.run('purge.url', 'http://mydomain.com/articles/.*')
manager.close()

Testing:

python runtests.py

python-varnish's Issues

Socket Error

When I attempt to manager.run('ping') I get a socket error:

>>> import varnish
>>> manager = varnish.VarnishManager( ('varnserver.int:6082'))
>>> manager.run('ping')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/adm/config/python/lib/python2.7/site-packages/varnish.py", line 313, in run
    for server in self.servers]
  File "/usr/local/adm/config/python/lib/python2.7/site-packages/varnish.py", line 289, in run
    handler = VarnishHandler(addr, **kwargs)  
  File "/usr/local/adm/config/python/lib/python2.7/site-packages/varnish.py", line 65, in __init__
    Telnet.__init__(self, *host_port_timeout)
  File "/usr/local/python/lib/python2.7/telnetlib.py", line 209, in __init__
    self.open(host, port, timeout)
  File "/usr/local/python/lib/python2.7/telnetlib.py", line 225, in open
    self.sock = socket.create_connection((host, port), timeout)
  File "/usr/local/python/lib/python2.7/socket.py", line 553, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno -2] Name or service not known

Are there any gotchas I should know about?

if multiple servers, commands are sent multiple times

def run(self, *commands, **kwargs):
    threaded = kwargs.pop('threaded', False)
    for server in self.servers:
        if threaded:
            [ThreadedRunner(server, *commands, **kwargs).start()
                for server in self.servers]
        else:
            return [run(server, *commands, **kwargs)
                        for server in self.servers]

the variable server is ignored and is looped again. A loop of servers per server is done resulting in (number of servers) ^ 2 commands run.

delete the line:

for server in self.servers:

Refactor whole app

I wrote this a long time ago when I worked for a company that used varnish in production and it was very helpful for our automation framework along with justquick/django-varnish. Since then I really havent had any work applications to use varnish so I left this app in disrepair. It's always fun looking back at your old code and wondering wtf I was thinking. here are the goals im aiming for in the next major refactor

1. varnish 4 support
2. python 3 support
3. implement threadpoolexecutor from concurrent.futures
4. better testing esp with auth

Varnish 3.0 compatibility

Justa heads up after coming across this projects in a google search, in >= 3.0 "purge.url' is no longer valid, this uses

ban.url

PyPi is missing a few bugfixes.

Hello,
would it be easy for you to make another PyPi release? Some important bug fixes (e.g. for run()) are missing...

Thanks!

TypeError is raised when timeout is passed in connection string

If we pass connection string to VarnishHandler as below

handler = VarnishHandler('localhost:6082:0.1')

TypeError is raised by socket lib:

  File "/Users/ziollek/python-varnish/varnish.py", line 65, in __init__
    Telnet.__init__(self, *host_port_timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/telnetlib.py", line 211, in __init__
    self.open(host, port, timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/telnetlib.py", line 227, in open
    self.sock = socket.create_connection((host, port), timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 559, in create_connection
    sock.settimeout(timeout)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
TypeError: a float is required

vcl_show fails due to missing formatting string

The following trivial patch resolves it: https://gist.github.com/1034854

No tagged versions

I was looking for version 3.0.3 but found no tagged versions in the repository.
Would be useful to have tags

python logging configuration

in varnish.py:

import logging

logging.basicConfig(
    level = logging.DEBUG,
    format = '%(asctime)s %(levelname)s %(message)s',
)

Why is it reconfiguring the logger? and why isn't it using the standard: logger = logging.getLogger(name) ?

Is this compatible with Varnish 4.0 +?

Hi Justquick,

I'm interested in utilising this module alongside an Odoo (runbot.odoo.com - username: admin password: admin) module to allow purging and other varnish functions from the Odoo interface.

I'd just like to know if it would be compatible with Varnish 4.0 + as this is the version i'll be going with for my module. I would be very interested in any input you might have on this.

varnish close generates error when secret needed

When you run varnish commands you can pass in secret like so:

manager.run('ban.url', '^%s' % url, secret=VARNISH_SECRET)

But if you call close it generates a 107 error. That's because it's calling close without a secret and there is no way to pass it in.

def close(self):
    self.run('close', threaded=True)
    self.servers = ()

Work around is to run

manager.run('close', secret=VARNISH_SECRET)

Socket Error

When I attempt to manager.run('ping') I get a socket error:

>>> import varnish
>>> manager = varnish.VarnishManager( ('example.int:6082'))
>>> manager.run('ping')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/adm/config/python/lib/python2.7/site-packages/varnish.py", line 313, in run
    for server in self.servers]
  File "/usr/local/adm/config/python/lib/python2.7/site-packages/varnish.py", line 289, in run
    handler = VarnishHandler(addr, **kwargs)  
  File "/usr/local/adm/config/python/lib/python2.7/site-packages/varnish.py", line 65, in __init__
    Telnet.__init__(self, *host_port_timeout)
  File "/usr/local/python/lib/python2.7/telnetlib.py", line 209, in __init__
    self.open(host, port, timeout)
  File "/usr/local/python/lib/python2.7/telnetlib.py", line 225, in open
    self.sock = socket.create_connection((host, port), timeout)
  File "/usr/local/python/lib/python2.7/socket.py", line 553, in create_connection
    for res in getaddrinfo(host, port, 0, SOCK_STREAM):
socket.gaierror: [Errno -2] Name or service not known

Are there any gotchas I should know about?

passing secret to varnishadm

I need to pass the secret file as a parameter. it seems it's not possible with python-varnis.

Maybe with another parameter on init?