Skip to content

Commit ba6936b

Browse files
grahamalamagrahamalama
authored
Add MozlogHandler that integrates renamed formatter (#112)
* Add MozlogHandler that integrates renamed formatter * Remove unused fixture from test * Slightly refactor logging test setup - Pass handler and formatter as fixtures to tests - Reset logging after each test * Add tests to assert how `logger_name` is attached to records * Ruff fixes * Use formatter in `assert_records` assertions * Set caplog to INFO in fastapi rid test * Make assertions about LogRecord and formatted output
1 parent 391f9ba commit ba6936b

10 files changed

Lines changed: 146 additions & 109 deletions

File tree

‎docs/django.rst‎

Lines changed: 6 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -61,8 +61,8 @@ To install ``python-dockerflow``'s Django support please follow these steps:
6161
]
6262

6363
#. :ref:`Configure logging <django-logging>` to use the
64-
:class:`~dockerflow.logging.JsonLogFormatter`
65-
logging formatter for the ``request.summary`` logger (you may have to
64+
:class:`~dockerflow.logging.MozlogHandler`
65+
logging handler for the ``request.summary`` logger (you may have to
6666
extend your existing logging configuration!).
6767

6868
.. _`Kubernetes liveness checks`: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/
@@ -405,20 +405,15 @@ spec:
405405
Logging
406406
-------
407407

408-
Dockerflow provides a :class:`~dockerflow.logging.JsonLogFormatter` Python
409-
logging formatter class.
408+
Dockerflow provides a :class:`~dockerflow.logging.MozlogHandler` Python
409+
logging handler class. This handler formats logs according to the Mozlog schema
410+
and emits them to stdout.
410411

411412
To use it, put something like this in your Django ``settings`` file and
412413
configure **at least** the ``request.summary`` logger that way::
413414

414415
LOGGING = {
415416
'version': 1,
416-
'formatters': {
417-
'json': {
418-
'()': 'dockerflow.logging.JsonLogFormatter',
419-
'logger_name': 'myproject'
420-
}
421-
},
422417
'filters': {
423418
'request_id': {
424419
'()': 'dockerflow.logging.RequestIdLogFilter',
@@ -427,8 +422,7 @@ configure **at least** the ``request.summary`` logger that way::
427422
'handlers': {
428423
'console': {
429424
'level': 'DEBUG',
430-
'class': 'logging.StreamHandler',
431-
'formatter': 'json',
425+
'class': 'dockerflow.logging.MozlogHandler',
432426
'filters': ['request_id']
433427
},
434428
},

‎docs/fastapi.rst‎

Lines changed: 4 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,7 @@ To install ``python-dockerflow``'s FastAPI support please follow these steps:
5454

5555
.. seealso:: :ref:`fastapi-versions` for more information
5656

57-
#. Configure logging to use the ``JsonLogFormatter`` logging formatter for the
57+
#. Configure logging to use the ``MozlogHandler`` logging handler for the
5858
``request.summary`` logger (you may have to extend your existing logging
5959
configuration), see :ref:`fastapi-logging` for more information.
6060

@@ -280,8 +280,8 @@ spec:
280280
Logging
281281
-------
282282

283-
Dockerflow provides a :class:`~dockerflow.logging.JsonLogFormatter` Python
284-
logging formatter class.
283+
Dockerflow provides a :class:`~dockerflow.logging.Mozlog` Python
284+
logging handler class.
285285

286286
To use it, put something like this **BEFORE** your FastAPI app is initialized
287287
for at least the ``request.summary`` logger:
@@ -292,12 +292,6 @@ for at least the ``request.summary`` logger:
292292
293293
dictConfig({
294294
'version': 1,
295-
'formatters': {
296-
'json': {
297-
'()': 'dockerflow.logging.JsonLogFormatter',
298-
'logger_name': 'myproject'
299-
}
300-
},
301295
'filters': {
302296
'request_id': {
303297
'()': 'dockerflow.logging.RequestIdLogFilter',
@@ -306,9 +300,8 @@ for at least the ``request.summary`` logger:
306300
'handlers': {
307301
'console': {
308302
'level': 'DEBUG',
309-
'class': 'logging.StreamHandler',
303+
'class': 'dockerflow.logging.MozlogHandler',
310304
'filters': ['request_id'],
311-
'formatter': 'json'
312305
},
313306
},
314307
'loggers': {

‎docs/flask.rst‎

Lines changed: 4 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ To install ``python-dockerflow``'s Flask support please follow these steps:
5656

5757
.. seealso:: :ref:`flask-versions` for more information
5858

59-
#. Configure logging to use the ``JsonLogFormatter`` logging formatter for the
59+
#. Configure logging to use the ``MozlogHandler`` logging handler for the
6060
``request.summary`` logger (you may have to extend your existing logging
6161
configuration), see :ref:`flask-logging` for more information.
6262

@@ -425,8 +425,8 @@ spec:
425425
Logging
426426
-------
427427

428-
Dockerflow provides a :class:`~dockerflow.logging.JsonLogFormatter` Python
429-
logging formatter class.
428+
Dockerflow provides a :class:`~dockerflow.logging.MozlogHandler` Python
429+
logging handler class.
430430

431431
To use it, put something like this **BEFORE** your Flask app is initialized
432432
for at least the ``request.summary`` logger::
@@ -435,12 +435,6 @@ for at least the ``request.summary`` logger::
435435

436436
dictConfig({
437437
'version': 1,
438-
'formatters': {
439-
'json': {
440-
'()': 'dockerflow.logging.JsonLogFormatter',
441-
'logger_name': 'myproject'
442-
}
443-
},
444438
'filters': {
445439
'request_id': {
446440
'()': 'dockerflow.logging.RequestIdLogFilter',
@@ -449,8 +443,7 @@ for at least the ``request.summary`` logger::
449443
'handlers': {
450444
'console': {
451445
'level': 'DEBUG',
452-
'class': 'logging.StreamHandler',
453-
'formatter': 'json',
446+
'class': 'dockerflow.logging.MozlogHandler',
454447
'filters': ['request_id']
455448
},
456449
},

‎docs/logging.rst‎

Lines changed: 4 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -3,8 +3,8 @@
33
Logging
44
=======
55

6-
python-dockerflow provides a :class:`~dockerflow.logging.JsonLogFormatter`
7-
Python logging formatter that produces messages following the JSON schema
6+
python-dockerflow provides a :class:`~dockerflow.logging.MozlogHandler`
7+
Python logging handler that produces messages following the JSON schema
88
for a common application logging format defined by the illustrious
99
Mozilla Cloud Services group.
1010

@@ -33,12 +33,6 @@ this::
3333

3434
cfg = {
3535
'version': 1,
36-
'formatters': {
37-
'json': {
38-
'()': 'dockerflow.logging.JsonLogFormatter',
39-
'logger_name': 'myproject'
40-
}
41-
},
4236
'filters': {
4337
'request_id': {
4438
'()': 'dockerflow.logging.RequestIdLogFilter',
@@ -47,8 +41,7 @@ this::
4741
'handlers': {
4842
'console': {
4943
'level': 'DEBUG',
50-
'class': 'logging.StreamHandler',
51-
'formatter': 'json',
44+
'class': 'dockerflow.logging.MozlogHandler',
5245
'filters': ['request_id']
5346
},
5447
},
@@ -109,7 +102,7 @@ thing as the dictionary based configuratio above:
109102
filters = request_id
110103
111104
[formatter_json]
112-
class = dockerflow.logging.JsonLogFormatter
105+
class = dockerflow.logging.MozlogFormatter
113106
114107
Then load the ini file using the :mod:`logging` module function
115108
:func:`logging.config.fileConfig`:

‎docs/sanic.rst‎

Lines changed: 5 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,7 @@ To install ``python-dockerflow``'s Sanic support please follow these steps:
5353

5454
.. seealso:: :ref:`sanic-versions` for more information
5555

56-
#. Configure logging to use the ``JsonLogFormatter`` logging formatter for the
56+
#. Configure logging to use the ``MozlogHandler`` logging handler for the
5757
``request.summary`` logger (you may have to extend your existing logging
5858
configuration), see :ref:`sanic-logging` for more information.
5959

@@ -405,8 +405,8 @@ spec:
405405
Logging
406406
-------
407407

408-
Dockerflow provides a :class:`~dockerflow.logging.JsonLogFormatter` Python
409-
logging formatter class.
408+
Dockerflow provides a :class:`~dockerflow.logging.MozlogFormatter` Python
409+
logging handler class.
410410

411411
To use it, pass something like this to your Sanic app when it is initialized
412412
for at least the ``request.summary`` logger::
@@ -415,12 +415,6 @@ for at least the ``request.summary`` logger::
415415

416416
log_config = {
417417
'version': 1,
418-
'formatters': {
419-
'json': {
420-
'()': 'dockerflow.logging.JsonLogFormatter',
421-
'logger_name': 'myproject'
422-
}
423-
},
424418
'filters': {
425419
'request_id': {
426420
'()': 'dockerflow.logging.RequestIdLogFilter',
@@ -429,8 +423,7 @@ for at least the ``request.summary`` logger::
429423
'handlers': {
430424
'console': {
431425
'level': 'DEBUG',
432-
'class': 'logging.StreamHandler',
433-
'formatter': 'json',
426+
'class': 'dockerflow.logging.MozlogHandler',
434427
'filters': ['request_id']
435428
},
436429
},
@@ -440,7 +433,7 @@ for at least the ``request.summary`` logger::
440433
'level': 'DEBUG',
441434
},
442435
}
443-
})
436+
}
444437

445438
sanic = Sanic(__name__, log_config=log)
446439

‎src/dockerflow/fastapi/middleware.py‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,6 @@
11
from __future__ import annotations
22

33
import logging
4-
import sys
54
import time
65
import urllib
76
from typing import Any, Dict
@@ -14,7 +13,7 @@
1413
HTTPScope,
1514
)
1615

17-
from ..logging import JsonLogFormatter, get_or_generate_request_id, request_id_context
16+
from ..logging import MozlogHandler, get_or_generate_request_id, request_id_context
1817

1918

2019
class RequestIdMiddleware:
@@ -57,9 +56,8 @@ def __init__(
5756
if logger is None:
5857
logger = logging.getLogger("request.summary")
5958
logger.setLevel(logging.INFO)
60-
handler = logging.StreamHandler(sys.stdout)
59+
handler = MozlogHandler()
6160
handler.setLevel(logging.INFO)
62-
handler.setFormatter(JsonLogFormatter())
6361
logger.addHandler(handler)
6462
self.logger = logger
6563

‎src/dockerflow/logging.py‎

Lines changed: 29 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -9,17 +9,31 @@
99
import sys
1010
import traceback
1111
import uuid
12+
import warnings
1213
from contextvars import ContextVar
1314
from typing import ClassVar, Optional
1415

1516

17+
class MozlogHandler(logging.StreamHandler):
18+
def __init__(self, stream=None, name="Dockerflow"):
19+
if stream is None:
20+
stream = sys.stdout
21+
super().__init__(stream=stream)
22+
self.logger_name = name
23+
self.setFormatter(MozlogFormatter())
24+
25+
def emit(self, record):
26+
record.logger_name = self.logger_name
27+
super().emit(record)
28+
29+
1630
class SafeJSONEncoder(json.JSONEncoder):
1731
def default(self, o):
1832
return repr(o)
1933

2034

21-
class JsonLogFormatter(logging.Formatter):
22-
"""Log formatter that outputs machine-readable json.
35+
class MozlogFormatter(logging.Formatter):
36+
"""Log formatter that outputs json structured according to the Mozlog schema.
2337
2438
This log formatter outputs JSON format messages that are compatible with
2539
Mozilla's standard heka-based log aggregation infrastructure.
@@ -58,9 +72,10 @@ class JsonLogFormatter(logging.Formatter):
5872
"levelname",
5973
"levelno",
6074
"lineno",
75+
"logger_name",
76+
"message",
6177
"module",
6278
"msecs",
63-
"message",
6479
"msg",
6580
"name",
6681
"pathname",
@@ -75,15 +90,7 @@ class JsonLogFormatter(logging.Formatter):
7590
)
7691

7792
def __init__(self, fmt=None, datefmt=None, style="%", logger_name="Dockerflow"):
78-
parent_init = logging.Formatter.__init__
79-
# The style argument was added in Python 3.1 and since
80-
# the logging configuration via config (ini) files uses
81-
# positional arguments we have to do a version check here
82-
# to decide whether to pass the style argument or not.
83-
if sys.version_info[:2] < (3, 1):
84-
parent_init(self, fmt, datefmt)
85-
else:
86-
parent_init(self, fmt=fmt, datefmt=datefmt, style=style)
93+
super().__init__(fmt=fmt, datefmt=datefmt, style=style)
8794
self.logger_name = logger_name
8895
self.hostname = socket.gethostname()
8996

@@ -104,7 +111,7 @@ def convert_record(self, record):
104111
out = {
105112
"Timestamp": int(record.created * 1e9),
106113
"Type": record.name,
107-
"Logger": self.logger_name,
114+
"Logger": getattr(record, "logger_name", self.logger_name),
108115
"Hostname": self.hostname,
109116
"EnvVersion": self.LOGGING_FORMAT_VERSION,
110117
"Severity": self.SYSLOG_LEVEL_MAP.get(
@@ -143,6 +150,15 @@ def format(self, record):
143150
return json.dumps(out, cls=SafeJSONEncoder)
144151

145152

153+
class JsonLogFormatter(MozlogFormatter):
154+
def __init__(self, *args, **kwargs):
155+
warnings.warn(
156+
"JsonLogFormatter has been deprecated. Use MozlogFormatter instead",
157+
DeprecationWarning,
158+
)
159+
super().__init__(*args, **kwargs)
160+
161+
146162
def safer_format_traceback(exc_typ, exc_val, exc_tb):
147163
"""Format an exception traceback into safer string.
148164
We don't want to let users write arbitrary data into our logfiles,

0 commit comments

Comments
 (0)