logging - Context-based logging¶

Changed in version 2.0.0.

Overview¶

This module provides context-based logging system built on top of standard logging module. It also solves the common logging management problem when various modules use hard-coded separate loggers, and provides several types of message wrappers that allow lazy message interpolation using f-string, brace (str.format) or dollar (string.Template) formats.

The context-based logging:

Adds context information into LogRecord, that could be used in logging entry formats.
Allows assignment of loggers to specific contexts.

Basics¶

Normally, when you want to use a logger, you call the logging.getLogger function and pass it a logger name (or not to get the root logger). The common (and recommended) practice is to use getLogger(__name__) which often leads to logging configuration problems as complex applications with many modules (including used libraries) may create complex logger hierarchy.

Our logging module solves the problem by replacing the logger name with agent identification. The agent is typically an unit of code that works in specific execution contexts. For example a code that process client request in web application (where request is the context), or executes SQL command (the context could be a database connection). In most cases, the agent is an instance of some class.

So, from user’s perspective, the context logging is used similarly to normal Python logging - but you pass the agent identification instead logger name to get_logger function. If agent identification is a string, it’s used as is. If it’s an object, it uses value of its _agent_name_ attribute if defined, otherwise it uses name in “MODULE_NAME.CLASS_QUALNAME” format. If _agent_name_ value is not a string, it’s converted to string.

The typical usage pattern inside a class is therefore:

logger = get_logger(self)

or for direct use:

get_logger(self).debug("message")

The get_logger also has an optional topic string parameter that could be used to differentiate between various logging “streams”. For example the trace module uses context logging with topic “trace”, so it’s possible to configure the logging system to handle “trace” output in specific way.

The underlying machinery behind get_logger function maps the agent and its context to particular Logger, and returns a ContextLoggerAdapter that you can use as normal logger. This adapter is responsible to add context information into LogRecord.

Context information¶

The conext information added by ContextLoggerAdapter into LogRecord consists from next items:

agent:: String representation of the agent identification described above.
context:: Agent context that could be defined via log_context attribute on agent instance, or by assigning its value directly to extra['context'] on adapter returned by get_logger() function.
domain:: A name assigned to a group of agents (more about that later).
topic:: Name of a logging stream.

They could be used in Formatter templates. If you want to use logging that combines normal and context logging, it’s necessary to assign ContextFilter to your logging.Handler to add (empty) context information into LogRecords that are produced by normal loggers.

LoggingManager¶

The firebird.base.logging module defines a global LoggingManager instance logging_manager that manages several mappings and implements the get_logger method.

Some methods are also provided as global functions for conveniense: get_logger, set_agent_mapping, set_domain_mapping and get_agent_name.

Mappings and Logger names¶

The Logger wrapped by the get_logger function is determined by applying the values of several parameters to the logger name format. These parameters are:

Domain: String used to group output from agents.
Topic: String identification of particular logging stream.

The logger name format is a list that can contain any number of string values and at most one occurrence of DOMAIN or TOPIC enum values. Empty strings are removed.

The final Logger name is constructed by joining elements of this list with dots, and with sentinels replaced with domain and topic names.

For example, if values are defined as:

logger_fmt = ['app', DOMAIN, TOPIC]
domain = 'database'
topic = 'trace'

the Logger name will be: “app.database.trace”

The logger name format is defined in LoggingManager.logger_fmt property, and it’s an empty list by default, which means that get_logger function always maps to root logger.

The domain is determined from agent passed to get_logger. You can use set_domain_mapping to assign agent identifications to particular domain. The agents that are not assigned to domain belong to default domain specifid in LoggingManager.default_domain, which is None by default.

It’s also possible to change agent identification used for logger name mapping porposes to different value with set_agent_mapping function.

Lazy Formatting Messages¶

This module also provides message wrapper classes (FStrMessage, BraceMessage, DollarMessage) that defer string formatting until the log record is actually processed by a handler. This avoids the performance cost of formatting messages that might be filtered out due to log levels.

Basic Setup Example¶

import logging
from firebird.base.logging import (
    get_logger, LogLevel, ContextFilter, logging_manager,
    DOMAIN, TOPIC # For logger_fmt
)

# 1. Configure standard logging (handlers, formatters)
log_format = "[%(levelname)-8s] %(asctime)s %(name)s (%(agent)s) - %(message)s"
formatter = logging.Formatter(log_format)
handler = logging.StreamHandler()
handler.setFormatter(formatter)

# 2. Add ContextFilter to handler(s) to ensure context fields exist
handler.addFilter(ContextFilter())

# 3. Get the root logger or specific standard loggers and add the handler
root_logger = logging.getLogger()
root_logger.addHandler(handler)
root_logger.setLevel(LogLevel.DEBUG) # Use LogLevel enum or logging constants

# 4. (Optional) Configure logging_manager mappings
logging_manager.logger_fmt = ['app', DOMAIN, TOPIC] # Logger name format
logging_manager.default_domain = 'web'            # Default domain if not mapped
logging_manager.set_domain_mapping('db', ['myapp.database.Connection']) # Map agent to domain

# 5. Use in your code
class RequestHandler:
    _agent_name_ = 'myapp.web.RequestHandler' # Optional explicit agent name
    log_context = None # Can be set per request, e.g., client IP

    def handle(self, request_id):
        self.log_context = f"ReqID:{request_id}"
        logger = get_logger(self, topic='requests') # Get context logger
        logger.info("Handling request...")
        # ... processing ...
        logger.debug("Request handled successfully.")

class DbConnection:
    _agent_name_ = 'myapp.database.Connection'
    log_context = None # e.g., DB user

    def query(self, sql):
        self.log_context = "user:admin"
        logger = get_logger(self) # Use default topic (None)
        logger.debug("Executing query: %s", sql) # Standard formatting works too
        # ... execute ...

# --- Execution ---
handler_instance = RequestHandler()
db_conn = DbConnection()

handler_instance.handle("12345")
db_conn.query("SELECT * FROM T")

# --- Example Output ---
# [INFO    ] 2023-10-27... app.web.requests (myapp.web.RequestHandler) - Handling request...
# [DEBUG   ] 2023-10-27... app.web.requests (myapp.web.RequestHandler) - Request handled successfully.
# [DEBUG   ] 2023-10-27... app.db (myapp.database.Connection) - Executing query: SELECT * FROM T

Enums & Flags¶

class firebird.base.logging.FormatElement(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]¶

Bases: Enum

Sentinels used within LoggingManager.logger_fmt list.

DOMAIN = 1¶

TOPIC = 2¶

class firebird.base.logging.LogLevel(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]¶

Bases: IntEnum

Mirrors standard logging levels for convenience and type hinting.

Provides symbolic names (e.g., LogLevel.DEBUG) corresponding to the integer values used by the standard logging module (logging.DEBUG).

CRITICAL = 50¶

DEBUG = 10¶

ERROR = 40¶

FATAL = 50¶

INFO = 20¶

NOTSET = 0¶

WARN = 30¶

WARNING = 30¶

Constants¶

firebird.base.logging.DOMAIN: FormatElement = FormatElement.DOMAIN¶: Sentinel representing the domain element in LoggingManager.logger_fmt.

firebird.base.logging.TOPIC: FormatElement = FormatElement.TOPIC¶: Sentinel representing the topic element in LoggingManager.logger_fmt.

Functions¶

firebird.base.logging.get_logger(agent: Any, topic: str | None = None) → ContextLoggerAdapter¶

Shortcut to global LoggingManager.get_logger function.

Parameters:

agent (Any) –
topic (str | None) –

Return type:

ContextLoggerAdapter

firebird.base.logging.set_domain_mapping(domain: str, agents: Iterable[str] | str | None, *, replace: bool = False) → None¶

Shortcut to global LoggingManager.set_domain_mapping function.

Parameters:

domain (str) –
agents (Iterable[str] | str | None) –
replace (bool) –

Return type:

None

firebird.base.logging.set_agent_mapping(agent: str, new_agent: str | None) → None¶

Shortcut to global LoggingManager.set_agent_mapping function.

Parameters:

agent (str) –
new_agent (str | None) –

Return type:

None

firebird.base.logging.get_agent_name(agent: Any) → str¶

Shortcut to global LoggingManager.get_agent_name function.

Parameters:: agent (Any) –
Return type:: str

Adapters and Filters¶

class firebird.base.logging.ContextLoggerAdapter(logger, domain: str | None, topic: str | None, agent: Any, agent_name: str)[source]¶

Bases: LoggerAdapter

Logger adapter injecting context (domain, topic, agent, context) info.

Wraps a standard logging.Logger. When a logging method (e.g., info, debug) is called, it adds the context information into the extra dictionary, making it available as attributes on the resulting logging.LogRecord.

Parameters:

logger – The standard logging.Logger instance to wrap.
domain (str | None) – Context Domain name (or None).
topic (str | None) – Context Topic name (or None).
agent (Any) – The original agent object or string passed to get_logger.
agent_name (str) – The resolved string name for the agent.

process(msg: Any, kwargs: dict[str, Any]) → tuple[Any, dict[str, Any]][source]¶

Process the logging message and keyword arguments passed in to a logging call to insert contextual information.

Ensures self.extra contains domain, topic, and agent (from init).
Adds context to self.extra, taking it from self.agent.log_context if available, otherwise None.
Merges the adapter’s extra dictionary with any extra dictionary passed in kwargs, giving precedence to keys in kwargs['extra'].
Stores the final merged extra dictionary into kwargs['extra'].

Returns:

The possibly modified msg and kwargs.

Parameters:

msg (Any) –
kwargs (dict[str, Any]) –

Return type:

tuple[Any, dict[str, Any]]

class firebird.base.logging.ContextFilter(name='')[source]¶

Bases: Filter

Logging filter ensuring context fields exist on LogRecord instances.

Checks for domain, topic, agent, and context attributes on each log record. If any are missing (e.g., for records from standard loggers not using ContextLoggerAdapter), it adds them with a value of None.

Usage:: Attach an instance of this filter to logging.Handler objects to ensure formatters expecting these fields do not raise AttributeError.

Example:

handler = logging.StreamHandler()
handler.addFilter(ContextFilter())
# ... add handler to logger ...

filter(record) → bool[source]¶

Determine if the specified record is to be logged.

Returns True if the record should be logged, or False otherwise. If deemed appropriate, the record may be modified in-place.

Return type:: bool

Logging manager¶

class firebird.base.logging.LoggingManager[source]¶

Bases: object

Logging manager.

_get_logger_name(domain: str | None, topic: str | None) → str[source]¶

Returns logging.Logger name.

Parameters:

domain (str | None) –
topic (str | None) –

Return type:

str

get_agent_domain(agent: str) → str | None[source]¶

Returns domain name assigned to agent.

Parameters:: agent (str) – Agent name.
Returns:: Domain assigned to agent or None.
Return type:: str | None

get_agent_mapping(agent: str) → str | None[source]¶

Returns current name mapping for agent.

Parameters:: agent (str) – Agent name.
Returns:: Reassigned agent name or None.
Return type:: str | None

get_agent_name(agent: Any) → str[source]¶

Determine the canonical string name for a given agent identifier.

Parameters:: agent (Any) – Agent identifier (string, or object).
Returns:: The resolved agent name (string).
Return type:: str

Logic:

If agent is a string, it’s used directly.
If agent is an object: - Uses agent._agent_name_ if defined (converting to string if needed). - Otherwise, constructs name as module.ClassQualname.
Applies any agent name mapping defined via set_agent_mapping to the name determined in steps 1 or 2.
Ensures the final result is a string.

Example:

> from firebird.base.logging import logging_manager
> logging_manager.get_agent_name(logging_manager)
'firebird.base.logging.LoggingManager'

get_domain_mapping(domain: str) → set[str] | None[source]¶

Returns current agent mapping for domain.

Parameters:: domain (str) – Domain name.
Returns:: Set of agent names assigned to domain or None.
Return type:: set[str] | None

get_logger(agent: Any, topic: str | None = None) → ContextLoggerAdapter[source]¶

Get a ContextLoggerAdapter configured for the specified agent and topic.

This is the primary function for obtaining a logger in the context logging system. It determines the appropriate underlying logging.Logger based on the agent’s domain and the topic, then wraps it in a ContextLoggerAdapter to inject context information.

Parameters:

agent (Any) – The agent identifier (object or string). Used to determine the agent_name and domain.
topic (str | None) – Optional topic string for the logging stream (e.g., ‘network’, ‘db’).

Returns:

A ContextLoggerAdapter instance ready for logging.

Return type:

ContextLoggerAdapter

Process Flow:

Determine agent_name using get_agent_name(agent).
Determine domain by looking up agent_name in the domain mapping, falling back to self.default_domain.
Apply topic mapping to the input topic (if any).
Construct the final underlying logging.Logger name using self.logger_fmt, substituting the determined domain and mapped topic.
Get/create the logging.Logger instance using self._logger_factory with the constructed name.
Create and return a ContextLoggerAdapter wrapping the logger and carrying the domain, mapped topic, original agent, and agent_name.

get_logger_factory() → Callable[source]¶

Return a callable which is used to create a Logger.

Return type:: Callable

get_topic_mapping(topic: str) → str | None[source]¶

Returns current name mapping for topic.

Parameters:: topic (str) – Topic name.
Returns:: Reassigned topic name or None.
Return type:: str | None

reset() → None[source]¶

Resets manager to “factory defaults”: no mappings, no logger_fmt and undefined default_domain.

Return type:: None

set_agent_mapping(agent: str, new_agent: str | None) → None[source]¶

Sets or removes the mapping of an agent name to another name.

Parameters:

agent (str) – Agent name.
new_agent (str | None) – New agent name or None to remove the mapping. Empty string is like None.

Return type:

None

Important

Does not validate the new_agent value type, instead it’s converted to string.

set_domain_mapping(domain: str, agents: Iterable[str] | str | None, *, replace: bool = False) → None[source]¶

Sets, updates, or removes agent name mappings to a domain.

Parameters:

domain (str) – Domain name.
agents (Iterable[str] | str | None) – Iterable with agent names, single agent name, or None.
replace (bool) – When True, the new mapping replaces the current one, otherwise the mapping is updated.

Return type:

None

Important

Passing None to agents removes all agent mappings for specified domain, regardless of replace value.

set_logger_factory(factory) → None[source]¶

Set a callable which is used to create a Logger.

Parameters:: factory – The factory callable to be used to instantiate a logger.
Return type:: None

The factory has the following signature: factory(name, *args, **kwargs)

set_topic_mapping(topic: str, new_topic: str | None) → None[source]¶

Sets or removes the mapping of an topic name to another name.

Parameters:

topic (str) – Topic name.
new_topic (str | None) – Either None or new topic name.

Return type:

None

When new_topic is a string, it maps topic to new_topic. Empty string is like None.
When new_topic is None, it removes any mapping.

Important

Does not validate the new_topic value type, instead it’s converted to string.

property default_domain: str | None¶: Default domain. Could be either a string or None.

Important

When assigned, it does not validate the value type, but converts it to string.

property logger_fmt: list[str | FormatElement]¶

Logger format.

The list can contain any number of string values and at most one occurrence of DOMAIN or TOPIC enum values. Empty strings are removed.

The final logging.Logger name is constructed by joining elements of this list with dots, and with sentinels replaced with domain and topic names.

Example:

logger_fmt = ['app', DOMAIN, TOPIC]
domain = 'database'
topic = 'trace'

Logger name will be: "app.database.trace"

Messages¶

class firebird.base.logging.FStrMessage(fmt: str, /, *args, **kwargs)[source]¶

Bases: object

Lazy logging message wrapper using f-string semantics via eval.

Defers the evaluation of the f-string until the message is actually formatted by a handler, improving performance if the message might be filtered out by log level settings.

Note

Uses eval() internally. Ensure the format string and arguments do not contain untrusted user input.

Example:

logger.debug(FStrMessage("Processing item {item_id} for user {user!r}",
                        item_id=123, user="Alice"))
# Formatting only happens if DEBUG level is enabled for the logger/handler.

Parameters:: fmt (str) –

class firebird.base.logging.BraceMessage(fmt: str, /, *args, **kwargs)[source]¶

Bases: object

Lazy logging message wrapper using brace (str.format) style formatting.

Defers the call to str.format() until the message is actually formatted by a handler, improving performance for potentially filtered messages.

Example:

logger.warning(BraceMessage("Connection failed: host={0}, port={1}",
                            'server.com', 8080))
logger.warning(BraceMessage(("Message with coordinates: ({point.x:.2f}, {point.y:.2f})",
                             point=point))

Parameters:: fmt (str) –

class firebird.base.logging.DollarMessage(fmt: str, /, **kwargs)[source]¶

Bases: object

Lazy logging message wrapper using dollar (string.Template) style formatting.

Defers the substitution using string.Template until the message is actually formatted by a handler, improving performance for potentially filtered messages.

Example:

from string import Template # Not strictly needed for caller
logger.info(DollarMessage("Task $name completed with status $status",
                          name='Cleanup', status='Success'))

Parameters:: fmt (str) –

Globals¶

firebird.base.logging.logging_manager: LoggingManager¶: Context logging manager.