Welcome to graypy’s documentation!¶
This code is open source, and is available on GitHub.
graypy¶
Description¶
Python logging handlers that send log messages in the Graylog Extended Log Format (GELF).
graypy supports sending GELF logs to both Graylog2 and Graylog3 servers.
Installing¶
Using pip¶
Install the basic graypy python logging handlers:
pip install graypy
Install with requirements for GELFRabbitHandler
:
pip install graypy[amqp]
Using easy_install¶
Install the basic graypy python logging handlers:
easy_install graypy
Install with requirements for GELFRabbitHandler
:
easy_install graypy[amqp]
Usage¶
graypy sends GELF logs to a Graylog server via subclasses of the python logging.Handler class.
Below is the list of ready to run GELF logging handlers defined by graypy:
GELFUDPHandler
- UDP log forwardingGELFTCPHandler
- TCP log forwardingGELFTLSHandler
- TCP log forwarding with TLS supportGELFHTTPHandler
- HTTP log forwardingGELFRabbitHandler
- RabbitMQ log forwarding
UDP Logging¶
UDP Log forwarding to a locally hosted Graylog server can be easily done with
the GELFUDPHandler
:
import logging
import graypy
my_logger = logging.getLogger('test_logger')
my_logger.setLevel(logging.DEBUG)
handler = graypy.GELFUDPHandler('localhost', 12201)
my_logger.addHandler(handler)
my_logger.debug('Hello Graylog.')
RabbitMQ Logging¶
Alternately, use GELFRabbitHandler
to send messages to RabbitMQ and
configure your Graylog server to consume messages via AMQP. This prevents log
messages from being lost due to dropped UDP packets (GELFUDPHandler
sends
messages to Graylog using UDP). You will need to configure RabbitMQ with a
gelf_log
queue and bind it to the logging.gelf
exchange so messages
are properly routed to a queue that can be consumed by Graylog (the queue and
exchange names may be customized to your liking).
import logging
import graypy
my_logger = logging.getLogger('test_logger')
my_logger.setLevel(logging.DEBUG)
handler = graypy.GELFRabbitHandler('amqp://guest:guest@localhost/', exchange='logging.gelf')
my_logger.addHandler(handler)
my_logger.debug('Hello Graylog.')
Django Logging¶
It’s easy to integrate graypy
with Django’s logging settings. Just add a
new handler in your settings.py
:
LOGGING = {
'version': 1,
# other dictConfig keys here...
'handlers': {
'graypy': {
'level': 'WARNING',
'class': 'graypy.GELFUDPHandler',
'host': 'localhost',
'port': 12201,
},
},
'loggers': {
'django.request': {
'handlers': ['graypy'],
'level': 'ERROR',
'propagate': True,
},
},
}
Traceback Logging¶
By default log captured exception tracebacks are added to the GELF log as
full_message
fields:
import logging
import graypy
my_logger = logging.getLogger('test_logger')
my_logger.setLevel(logging.DEBUG)
handler = graypy.GELFUDPHandler('localhost', 12201)
my_logger.addHandler(handler)
try:
puff_the_magic_dragon()
except NameError:
my_logger.debug('No dragons here.', exc_info=1)
Default Logging Fields¶
By default a number of debugging logging fields are automatically added to the GELF log if available:
function
pid
process_name
thread_name
You can disable automatically adding these debugging logging fields by
specifying debugging_fields=False
in the handler’s constructor:
handler = graypy.GELFUDPHandler('localhost', 12201, debugging_fields=False)
Adding Custom Logging Fields¶
graypy also supports including custom fields in the GELF logs sent to Graylog. This can be done by using Python’s LoggerAdapter and Filter classes.
Using LoggerAdapter¶
LoggerAdapter makes it easy to add static information to your GELF log messages:
import logging
import graypy
my_logger = logging.getLogger('test_logger')
my_logger.setLevel(logging.DEBUG)
handler = graypy.GELFUDPHandler('localhost', 12201)
my_logger.addHandler(handler)
my_adapter = logging.LoggerAdapter(logging.getLogger('test_logger'),
{'username': 'John'})
my_adapter.debug('Hello Graylog from John.')
Using Filter¶
Filter gives more flexibility and allows for dynamic information to be added to your GELF logs:
import logging
import graypy
class UsernameFilter(logging.Filter):
def __init__(self):
# In an actual use case would dynamically get this
# (e.g. from memcache)
self.username = 'John'
def filter(self, record):
record.username = self.username
return True
my_logger = logging.getLogger('test_logger')
my_logger.setLevel(logging.DEBUG)
handler = graypy.GELFUDPHandler('localhost', 12201)
my_logger.addHandler(handler)
my_logger.addFilter(UsernameFilter())
my_logger.debug('Hello Graylog from John.')
Contributors¶
Sever Banesiu
Daniel Miller
Tushar Makkar
Nathan Klapstein
graypy.handler module¶
Logging Handlers that send messages in Graylog Extended Log Format (GELF)
-
class
graypy.handler.
BaseGELFHandler
(chunk_size=1420, debugging_fields=True, extra_fields=True, fqdn=False, localname=None, facility=None, level_names=False, compress=True)[source]¶ Bases:
logging.Handler
,abc.ABC
Abstract class defining the basic functionality of converting a
logging.LogRecord
into a GELF log. Provides the boilerplate for all GELF handlers defined within graypy.-
__init__
(chunk_size=1420, debugging_fields=True, extra_fields=True, fqdn=False, localname=None, facility=None, level_names=False, compress=True)[source]¶ Initialize the BaseGELFHandler.
- Parameters
chunk_size (int) – Message chunk size. Messages larger than this size will be sent to Graylog in multiple chunks.
debugging_fields (bool) – If
True
add debug fields from the log record into the GELF logs to be sent to Graylog.extra_fields (bool) – If
True
add extra fields from the log record into the GELF logs to be sent to Graylog.fqdn (bool) – If
True
use the fully qualified domain name of localhost to populate thehost
GELF field.localname (str or None) – If specified and
fqdn
isFalse
, use the specified hostname to populate thehost
GELF field.facility (str) – If specified, replace the
facility
GELF field with the specified value. Also add a additional_logger
GELF field containing theLogRecord.name
.level_names (bool) – If
True
use python logging error level name strings instead of syslog numerical values.compress (bool) – If
True
compress the GELF message before sending it to the Graylog server.
-
makePickle
(record)[source]¶ Convert a
logging.LogRecord
into bytes representing a GELF log- Parameters
record (logging.LogRecord) –
logging.LogRecord
to convert into a GELF log.- Returns
bytes representing a GELF log.
- Return type
-
-
class
graypy.handler.
ChunkedGELF
(message, size)[source]¶ Bases:
object
Class that chunks a message into a GELF compatible chunks
-
class
graypy.handler.
GELFHTTPHandler
(host, port=12203, compress=True, path='/gelf', timeout=5, **kwargs)[source]¶ Bases:
graypy.handler.BaseGELFHandler
GELF HTTP handler
-
__init__
(host, port=12203, compress=True, path='/gelf', timeout=5, **kwargs)[source]¶ Initialize the GELFHTTPHandler
- Parameters
host (str) – GELF HTTP input host.
port (int) – GELF HTTP input port.
compress (bool) – If
True
compress the GELF message before sending it to the Graylog server.path (str) – Path of the HTTP input. (see http://docs.graylog.org/en/latest/pages/sending_data.html#gelf-via-http)
timeout (int) – Number of seconds the HTTP client should wait before it discards the request if the Graylog server doesn’t respond.
-
emit
(record)[source]¶ Convert a
logging.LogRecord
to GELF and emit it to Graylog via a HTTP POST request- Parameters
record (logging.LogRecord) –
logging.LogRecord
to convert into a GELF log and emit to Graylog via a HTTP POST request.
-
-
class
graypy.handler.
GELFTCPHandler
(host, port=12201, **kwargs)[source]¶ Bases:
graypy.handler.BaseGELFHandler
,logging.handlers.SocketHandler
GELF TCP handler
-
__init__
(host, port=12201, **kwargs)[source]¶ Initialize the GELFTCPHandler
Attention
GELF TCP does not support compression due to the use of the null byte (
\0
) as frame delimiter.Thus,
handler.GELFTCPHandler
does not support settingcompress
toTrue
and is locked toFalse
.
-
makePickle
(record)[source]¶ Add a null terminator to generated pickles as TCP frame objects need to be null terminated
- Parameters
record (logging.LogRecord) –
logging.LogRecord
to create a null terminated GELF log.- Returns
Null terminated bytes representing a GELF log.
- Return type
-
-
class
graypy.handler.
GELFTLSHandler
(host, port=12204, validate=False, ca_certs=None, certfile=None, keyfile=None, **kwargs)[source]¶ Bases:
graypy.handler.GELFTCPHandler
GELF TCP handler with TLS support
-
__init__
(host, port=12204, validate=False, ca_certs=None, certfile=None, keyfile=None, **kwargs)[source]¶ Initialize the GELFTLSHandler
- Parameters
host (str) – GELF TLS input host.
port (int) – GELF TLS input port.
validate (bool) – If
True
, validate the Graylog server’s certificate. In this case specifyingca_certs
is also required.ca_certs (str) – Path to CA bundle file.
certfile (str) – Path to the client certificate file.
keyfile (str) – Path to the client private key. If the private key is stored with the certificate, this parameter can be ignored.
-
-
class
graypy.handler.
GELFUDPHandler
(host, port=12202, **kwargs)[source]¶ Bases:
graypy.handler.BaseGELFHandler
,logging.handlers.DatagramHandler
GELF UDP handler
graypy.rabbitmq module¶
Logging Handler integrating RabbitMQ and Graylog Extended Log Format (GELF)
-
class
graypy.rabbitmq.
ExcludeFilter
(name)[source]¶ Bases:
logging.Filter
A subclass of
logging.Filter
which should be instantiated with the name of the logger which, together with its children, will have its events excluded (filtered out)-
__init__
(name)[source]¶ Initialize the ExcludeFilter
- Parameters
name (str) – Name to match for within a
logging.LogRecord
’sname
field for filtering.
-
-
class
graypy.rabbitmq.
GELFRabbitHandler
(url, exchange='logging.gelf', exchange_type='fanout', virtual_host='/', routing_key='', **kwargs)[source]¶ Bases:
graypy.handler.BaseGELFHandler
,logging.handlers.SocketHandler
RabbitMQ / GELF handler
Note
This handler ignores all messages logged by amqplib.
-
__init__
(url, exchange='logging.gelf', exchange_type='fanout', virtual_host='/', routing_key='', **kwargs)[source]¶ Initialize the GELFRabbitHandler
-
makePickle
(record)[source]¶ Convert a
logging.LogRecord
into bytes representing a GELF log- Parameters
record (logging.LogRecord) –
logging.LogRecord
to convert into a GELF log.- Returns
bytes representing a GELF log.
- Return type
-