Python Logging, Configuration, and Environment Variables

Why Logging Beats print()

I get it. print() is fast and easy. But when your code runs on a server in production, you won't see stdout. You need logs that you can:

• Save to files with timestamps and severity levels
• Filter by urgency (DEBUG, INFO, WARNING, ERROR, CRITICAL)
• Route to different outputs (file, console, external logging service)
• Format consistently across your entire application

The logging module is Python's standard solution, and it's surprisingly elegant once you understand the layers.

Logging Architecture

Before you write a single line of logging code, it pays to understand what you're actually working with. Python's logging system is not a single function, it's a pipeline of components that each play a distinct role. Misunderstanding the architecture is the root cause of most logging bugs: logs disappearing unexpectedly, duplicate messages appearing, or configuration changes having no effect.

Python's logging system has four key components that work together:

Loggers are the entry point, you call them from your code with logger.info("message"). Think of them as labeled channels. When you call logging.getLogger(__name__), you get a logger named after your module. Loggers form a hierarchy: myapp.db is a child of myapp, which is a child of the root logger. Any message not handled by a child propagates up to the parent.

Handlers determine where logs go (console, file, network). A single logger can have multiple handlers, you can simultaneously write to a file for persistence and to the console for immediate visibility. Handlers have their own log level filter, independent of the logger's level, which gives you fine-grained control over what appears where.

Formatters control how the log message looks, what information gets included and in what order. The timestamp, the logger name, the severity level, the file and line number, all of these are optional fields you compose using format strings. A formatter attached to a handler shapes every message that handler emits.

Filters optionally restrict which records are processed, allowing you to drop messages based on any attribute of the log record, not just level. This is an advanced feature you reach for when the level hierarchy alone is not granular enough.

Here's how they connect: Logger receives the call → checks its level → passes to Handlers → each Handler checks its own level → passes to Formatter → writes to Output. Understanding this chain means you can diagnose any logging problem by walking the pipeline step by step.

Getting Started: basicConfig

The quickest way to set up logging is basicConfig(), which configures the root logger in a single call. This is the right choice for scripts and small applications where you just need logging to work without thinking about it much. The call is intentionally simple, give it a level, a format string, and you're ready.

import logging
 
logging.basicConfig(
    level=logging.DEBUG,
    format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
    datefmt="%Y-%m-%d %H:%M:%S"
)
 
logger = logging.getLogger(__name__)
logger.info("Application started")
logger.warning("This is a warning")
logger.error("Something went wrong!")

Run this and you'll see:

2026-02-25 14:32:15 - __main__ - INFO - Application started
2026-02-25 14:32:15 - __main__ - WARNING - This is a warning
2026-02-25 14:32:15 - __main__ - ERROR - Something went wrong!

Notice %(name)s, that's the logger name. By using logging.getLogger(__name__), each module gets its own logger named after the module path. This matters when debugging; you can see exactly where a log came from. When you're looking at 10,000 log lines trying to reproduce a bug, knowing that the message came from myapp.db.connection rather than just __main__ saves you significant time.

The level=logging.DEBUG means "show me everything from DEBUG level and above." The hierarchy is: DEBUG < INFO < WARNING < ERROR < CRITICAL. Set your root logger to INFO for production (less noise) and DEBUG for development (more detail). You will find that most third-party libraries log at DEBUG level by default, so without level filtering, your logs will be overwhelmed with framework noise.

Gotcha: basicConfig() only works if no handlers are configured yet. If you've already added handlers, calling it again does nothing. Set it up early, before importing modules that set up their own loggers.

Why basicConfig Isn't Enough

basicConfig() is fine for scripts, but real applications need flexibility. You might want:

• Different log levels for different modules
• Logs to a file AND console, with different formats
• A rotating file handler that creates new files when the current one gets too big

That's where dictConfig comes in.

Production-Grade Setup: dictConfig

Instead of imperative code, you define logging as a dictionary. This is cleaner, more testable, and easier to load from external configuration. The dictionary approach also makes logging configuration something you can diff in version control, review in code review, and reason about without running the application. When a colleague asks "how does logging work in this app," you hand them one dictionary and they have the complete picture.

import logging.config
 
LOGGING_CONFIG = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "simple": {
            "format": "%(levelname)s - %(name)s - %(message)s"
        },
        "detailed": {
            "format": "%(asctime)s - %(name)s - %(levelname)s - %(filename)s:%(lineno)d - %(message)s",
            "datefmt": "%Y-%m-%d %H:%M:%S"
        }
    },
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "level": "INFO",
            "formatter": "simple",
            "stream": "ext://sys.stdout"
        },
        "file": {
            "class": "logging.handlers.RotatingFileHandler",
            "level": "DEBUG",
            "formatter": "detailed",
            "filename": "logs/app.log",
            "maxBytes": 10485760,  # 10 MB
            "backupCount": 5
        }
    },
    "loggers": {
        "myapp": {
            "level": "DEBUG",
            "handlers": ["console", "file"]
        },
        "third_party_lib": {
            "level": "WARNING"  # Only show warnings from external libs
        }
    },
    "root": {
        "level": "INFO",
        "handlers": ["console"]
    }
}
 
logging.config.dictConfig(LOGGING_CONFIG)
logger = logging.getLogger("myapp")

Let's unpack this:

• formatters: Define reusable format patterns. detailed includes the file and line number; simple is lean.
• handlers: Map each output destination. The file handler uses RotatingFileHandler, which automatically creates new files when reaching 10 MB (keeping 5 backups).
• loggers: Configure individual loggers. myapp gets DEBUG level and both handlers; third_party_lib only shows WARNINGs to reduce noise.
• root: Catches everything else at INFO level.

Now different parts of your app can log with different verbosity. You control it all in one place, no code changes needed. As your application grows to dozens of modules, this centralized control becomes invaluable, you can dial up verbosity for a specific subsystem while keeping everything else quiet, purely through configuration.

Structured Logging for Production

In production, you often send logs to a centralized logging service (ELK, Datadog, Splunk). These services parse JSON better than raw text. Structured logging means outputting JSON instead of human-readable strings. The payoff is enormous: instead of writing regex parsers to extract data from log strings, your logging service can immediately filter, aggregate, and alert on specific fields. Find all errors from a specific user ID? One query. Measure the distribution of response times from a specific endpoint? Trivially easy with structured logs.

import logging
import logging.config
import json
 
class JSONFormatter(logging.Formatter):
    def format(self, record):
        log_obj = {
            "timestamp": self.formatTime(record, self.datefmt),
            "level": record.levelname,
            "logger": record.name,
            "message": record.getMessage(),
            "module": record.module,
            "function": record.funcName,
            "line": record.lineno
        }
        return json.dumps(log_obj)
 
LOGGING_CONFIG = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "json": {
            "()": JSONFormatter
        }
    },
    "handlers": {
        "file": {
            "class": "logging.handlers.RotatingFileHandler",
            "formatter": "json",
            "filename": "logs/app.json",
            "maxBytes": 10485760,
            "backupCount": 5
        }
    },
    "root": {
        "level": "INFO",
        "handlers": ["file"]
    }
}
 
logging.config.dictConfig(LOGGING_CONFIG)
logger = logging.getLogger(__name__)
logger.info("User login", extra={"user_id": 123, "ip": "192.168.1.1"})

Now your logs look like:

{
  "timestamp": "2026-02-25 14:35:22",
  "level": "INFO",
  "logger": "__main__",
  "message": "User login",
  "module": "__main__",
  "function": "<module>",
  "line": 47
}

Logging services can easily parse and search this. You can also pass extra dict to add custom fields. Notice how the extra fields like user_id and ip can be added to the JSONFormatter.format() method to include them in the output, this is where structured logging truly shines, letting you add business-context fields to any log entry without changing the format string.

Environment Variables: The Basics

Your application needs different settings for different environments. Development might use a local database, while production uses a cloud instance. Staging might use a different API key than production. These differences should never require a code change, they should be driven entirely by configuration that lives outside the codebase. That's what environment variables are for, configuration that changes without code changes.

The simplest approach is os.environ:

import os
 
DEBUG = os.environ.get("DEBUG", "False").lower() == "true"
DATABASE_URL = os.environ["DATABASE_URL"]  # Raises KeyError if missing
SECRET_KEY = os.environ.get("SECRET_KEY", "dev-key-change-in-prod")
 
print(f"Debug mode: {DEBUG}")
print(f"Database: {DATABASE_URL}")

When you run this:

$ DEBUG=True DATABASE_URL="postgres://localhost/mydb" python app.py
Debug mode: True
Database: postgres://localhost/mydb

Key point: os.environ.get() with a default is safer than os.environ[] because it won't crash if the variable is missing. But for truly required variables (like DATABASE_URL), use [] to fail loudly during startup, not later when you try to connect. The principle is: fail early and fail clearly. A crash on startup with a missing variable message is far better than a cryptic connection error that surfaces ten seconds into a request when your application is already serving traffic.

Environment Variable Patterns

Working with environment variables at scale means more than just reading strings. There are a handful of patterns worth internalizing because they come up in nearly every production application.

Boolean coercion is the most common pitfall. Environment variables are always strings. os.environ["DEBUG"] == "True" is True, but os.environ["DEBUG"] == "true" is False. Normalize to lowercase before comparing: .lower() == "true". Pydantic handles this automatically, which is one of the many reasons to use it.

Optional vs. required is a design decision you make at the variable level. Required variables have no default and should raise immediately if missing. Optional variables have a sensible default for development but you must document the expected production value in your .env.example. Do not blur this line, every variable is one or the other.

Namespace your variables for complex applications. Instead of DATABASE_URL, use MYAPP_DATABASE_URL. This prevents collisions when your application runs alongside other software that sets its own environment variables. Pydantic settings supports a env_prefix configuration option that applies this namespace automatically.

Avoid dynamic construction of variable names. Code like os.environ[f"{env}_DATABASE_URL"] is hard to grep, hard to document, and hard to audit for security. Prefer explicit variable names even if it means more lines of code.

The .env File Pattern

Typing environment variables on every run is tedious. During development, you want a .env file that captures all the variables your application needs in a single place you can check once and forget about until the variables change.

DEBUG=True
DATABASE_URL=postgres://localhost/mydb
SECRET_KEY=this-is-a-dev-key
API_TIMEOUT=30

Then use python-dotenv to load it:

pip install python-dotenv

from dotenv import load_dotenv
import os
 
load_dotenv()  # Loads .env into os.environ
 
DEBUG = os.environ.get("DEBUG", "False").lower() == "true"
DATABASE_URL = os.environ["DATABASE_URL"]
SECRET_KEY = os.environ.get("SECRET_KEY")

Call load_dotenv() at the very start of your app, before importing modules that might read environment variables. The function is idempotent (safe to call multiple times) and will not override variables already set in the actual environment, which means your CI/CD system's real environment variables always win over the .env file.

Critical rule: Never commit .env to version control. Add it to .gitignore:

.env
.env.local

Instead, commit .env.example with placeholder values:

DEBUG=False
DATABASE_URL=postgres://user:password@localhost/dbname
SECRET_KEY=change-me-in-production
API_TIMEOUT=30

This documents what variables your app needs without exposing real credentials. Think of .env.example as the API contract between your codebase and whoever deploys it. New team members clone the repo, copy .env.example to .env, fill in real values, and they're running. No undocumented setup steps, no hunting through Slack history for forgotten config.

The Modern Way: pydantic-settings

Raw os.environ works, but it lacks type validation and documentation. If someone sets API_TIMEOUT=hello, your code won't know until it tries to use that value as an int. Now imagine a production incident where logs just say "invalid literal for int()" and you're debugging blindly at 2 AM. The entire stack trace leads you to a config value that was wrong from the moment the application started.

Enter pydantic-settings, it treats environment variables like a Pydantic model:

pip install pydantic-settings

from pydantic_settings import BaseSettings
 
class Settings(BaseSettings):
    debug: bool = False
    database_url: str  # Required, no default
    secret_key: str = "dev-key"
    api_timeout: int = 30
    log_level: str = "INFO"
 
    class Config:
        env_file = ".env"
        case_sensitive = False
 
# Load once, reuse everywhere
settings = Settings()
 
print(f"Debug: {settings.debug} (type: {type(settings.debug)})")
print(f"Timeout: {settings.api_timeout} (type: {type(settings.api_timeout)})")

Create a .env:

DEBUG=true
DATABASE_URL=postgres://localhost/mydb
SECRET_KEY=super-secret
API_TIMEOUT=45
LOG_LEVEL=debug

Run it:

$ python app.py
Debug: True (type: <class 'bool'>)
Timeout: 45 (type: <class 'int'>)

Notice that DEBUG is a proper boolean and API_TIMEOUT is an integer, not strings. Pydantic validated and converted them. If you set API_TIMEOUT=not-a-number, Pydantic raises a ValidationError immediately on startup, before your code breaks. You also get free documentation: the Settings class itself serves as the canonical list of every configuration variable your application accepts, with its type, default, and the fact that it's required (no default) or optional (has default), all visible at a glance.

Combining Logging and Settings

Here's where the magic happens, use your settings to configure logging. This closes the loop between the two systems: your settings control your log level, your log file path, and any other logging-related configuration. Change LOG_LEVEL=DEBUG in your environment and you get verbose logs. Deploy to production with LOG_LEVEL=WARNING and your logs are quiet. No code changes, no redeployment of new code, just configuration.

from pydantic_settings import BaseSettings
import logging.config
import os
 
class Settings(BaseSettings):
    debug: bool = False
    database_url: str
    log_level: str = "INFO"
    log_file: str = "logs/app.log"
 
    class Config:
        env_file = ".env"
 
settings = Settings()
 
LOGGING_CONFIG = {
    "version": 1,
    "disable_existing_loggers": False,
    "formatters": {
        "simple": {
            "format": "%(levelname)s - %(name)s - %(message)s"
        }
    },
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "formatter": "simple",
            "level": settings.log_level
        },
        "file": {
            "class": "logging.handlers.RotatingFileHandler",
            "filename": settings.log_file,
            "formatter": "simple",
            "level": settings.log_level,
            "maxBytes": 10485760,
            "backupCount": 5
        }
    },
    "root": {
        "level": settings.log_level,
        "handlers": ["console", "file"]
    }
}
 
logging.config.dictConfig(LOGGING_CONFIG)
logger = logging.getLogger(__name__)

Now your log level is environment-driven. Set LOG_LEVEL=DEBUG in development for detailed logs; set LOG_LEVEL=WARNING in production to reduce noise. The pattern extends naturally: add LOG_FORMAT=json to your settings, then conditionally choose between your JSON formatter and your human-readable formatter based on that value. Development gets readable logs; production gets structured JSON for log aggregation services.

Twelve-Factor App Principles

The "Twelve-Factor App" methodology is a design pattern for modern web applications. Two principles directly apply here:

III. Store config in the environment: Configuration that varies between deploys should live in environment variables, not hardcoded. Your .env file is for development; production uses actual environment variables set by your deployment platform (Docker, Kubernetes, Heroku, etc.).

VI. Strictly separate logs from config: Logs are output streams (stdout, files). Config is input data (environment variables, config files). Keep them separate.

This means:

• Write logs to stdout/files, never try to read from them
• Read configuration from environment variables, never log raw secrets
• Use .env only in development; production gets variables from the platform

# GOOD: Log structured data, read env vars
logger.info("Database connected", extra={"host": settings.database_url.split("@")[1]})
 
# BAD: Logging secrets
logger.info(f"Connecting to {settings.database_url}")  # Exposes password!
 
# BAD: Hardcoded config
DATABASE_URL = "postgres://admin:password@prod.example.com/db"

Common Logging Mistakes

Even experienced Python developers make predictable logging mistakes. Knowing them in advance lets you avoid them entirely rather than discovering them through painful debugging sessions.

Calling basicConfig too late is perhaps the most common. If any module you import has already called logging.getLogger() or set up a handler, your basicConfig() call will silently do nothing. The fix: call basicConfig() or dictConfig() at the very top of your entry point, before any application imports. If you're still seeing double-logged messages or missing output, this is almost always the cause.

Propagation surprises bite developers who set up handlers on both a child logger and the root logger. When a child logger emits a message, it propagates up to root by default, causing the message to appear twice, once from the child's handler and once from root's. Either set propagate: False on your child logger in dictConfig, or only attach handlers to root and control verbosity through levels.

String formatting overhead is a subtle performance problem. When you write logger.debug("Processing %d items: %s", count, expensive_repr()), Python evaluates both count and expensive_repr() before passing them to the logger, even if DEBUG is disabled and the message will never be emitted. For cheap values this is fine. For expensive computations, guard with if logger.isEnabledFor(logging.DEBUG) or use lazy evaluation patterns.

Swallowing exceptions in logging calls is a real gotcha. If your log formatter raises an exception (say, your JSON formatter can't serialize an object you passed as extra data), the default behavior is to call logging.lastResort, which prints to stderr. You may not notice this in development but it can mask serious bugs in production. Test your formatters with the full range of data you intend to pass.

Misconfigured log file paths cause entire logging configurations to fail silently at startup. The RotatingFileHandler will raise a FileNotFoundError if the logs/ directory does not exist. Create the directory in your startup code before calling dictConfig, or use os.makedirs("logs", exist_ok=True).

Secrets Management: The Checklist

You'll handle secrets (database passwords, API keys, encryption keys). Here's the non-negotiable list:

1. Never hardcode secrets in source code. Period.
2. Never commit .env files. Commit .env.example instead.
3. Use strong defaults for non-sensitive config; require real values for secrets in production.
4. Rotate secrets periodically, don't use the same API key for years.
5. Log without leaking, if you log a database URL, mask the password.
6. Use a secrets manager in production (AWS Secrets Manager, HashiCorp Vault, etc.).

Here's a safe pattern:

from pydantic_settings import BaseSettings
from pydantic import Field
 
class Settings(BaseSettings):
    # Secrets (required in production, optional in dev)
    database_password: str = Field(default="", alias="DB_PASSWORD")
    api_key: str = Field(default="dev-key", alias="API_KEY")
 
    # Non-secrets (safe to have defaults)
    debug: bool = False
    database_host: str = "localhost"
    log_level: str = "INFO"
 
    class Config:
        env_file = ".env"
        case_sensitive = False
 
settings = Settings()
 
# In production, if api_key is still "dev-key", something's wrong
if not settings.debug and settings.api_key == "dev-key":
    raise ValueError("API_KEY not set in production!")

The .env.example:

DB_PASSWORD=change-me
API_KEY=change-me
DEBUG=False

Your deployment platform (Docker, Kubernetes, CI/CD) sets real values. Your code refuses to run with defaults in production. This startup validation is a hard wall between development conveniences and production requirements, exactly where that wall should be.

Real-World Example: Putting It Together

Let's build a small app that ties everything together. This pattern, a separate config.py, a separate logging_config.py, and a thin main.py, scales from small projects to large ones. Each file has a single responsibility: defining what configuration exists, setting up how logging works, and running the application. Keeping them separate means you can test each in isolation.

# config.py
from pydantic_settings import BaseSettings
from pydantic import Field
 
class Settings(BaseSettings):
    app_name: str = "MyApp"
    debug: bool = False
    database_url: str
    api_key: str = Field(default="", alias="API_KEY")
    log_level: str = "INFO"
    log_file: str = "logs/app.log"
 
    class Config:
        env_file = ".env"
        case_sensitive = False
 
settings = Settings()

# logging_config.py
import logging.config
 
def setup_logging(settings):
    """Configure logging based on settings."""
    LOGGING_CONFIG = {
        "version": 1,
        "disable_existing_loggers": False,
        "formatters": {
            "standard": {
                "format": "%(asctime)s [%(levelname)s] %(name)s: %(message)s",
                "datefmt": "%Y-%m-%d %H:%M:%S"
            }
        },
        "handlers": {
            "console": {
                "class": "logging.StreamHandler",
                "level": settings.log_level,
                "formatter": "standard",
                "stream": "ext://sys.stdout"
            },
            "file": {
                "class": "logging.handlers.RotatingFileHandler",
                "level": settings.log_level,
                "formatter": "standard",
                "filename": settings.log_file,
                "maxBytes": 10485760,
                "backupCount": 5
            }
        },
        "root": {
            "level": settings.log_level,
            "handlers": ["console", "file"]
        }
    }
    logging.config.dictConfig(LOGGING_CONFIG)

# main.py
import logging
from config import settings
from logging_config import setup_logging
 
setup_logging(settings)
logger = logging.getLogger(__name__)
 
def main():
    logger.info(f"Starting {settings.app_name}")
    logger.debug(f"Debug mode: {settings.debug}")
 
    if not settings.debug and settings.api_key == "":
        logger.error("API_KEY is not set in production")
        raise ValueError("API_KEY required in production")
 
    logger.info("Application running successfully")
 
if __name__ == "__main__":
    main()

.env:

APP_NAME=MyApp
DEBUG=True
DATABASE_URL=postgres://localhost/mydb
API_KEY=dev-key-for-testing
LOG_LEVEL=DEBUG
LOG_FILE=logs/app.log

Run it:

$ python main.py
2026-02-25 14:45:12 [DEBUG] __main__: Debug mode: True
2026-02-25 14:45:12 [INFO] __main__: Application running successfully

Change .env to DEBUG=False and API_KEY= (empty), then run again, it fails with a clear error message. The failure is immediate, specific, and actionable. Perfect.

Common Pitfalls and How to Avoid Them

Not setting log level before other imports: If your modules configure loggers in their imports, the root level might already be set. Call setup_logging() or basicConfig() at the very top of your main module, before other imports if possible.

Logging secrets: Review your log statements. Never log raw environment variables or full URLs with passwords. Use string formatting carefully:

# BAD
logger.info(f"Connecting: {DATABASE_URL}")
 
# GOOD
logger.info("Database connection established", extra={"host": get_safe_hostname()})

Too many loggers: You don't need a logger per function. Use loggers per module (using __name__). That gives you the right granularity.

Forgetting to load .env: A common mistake is assuming environment variables are loaded when they're not. Always call load_dotenv() explicitly at startup, before you read any env vars.

Mixing configuration approaches: Don't use both raw os.environ and Pydantic settings in the same codebase. Pick one and stick with it. Pydantic is better for new projects.

Testing Configuration and Logging

You've set up logging and configuration, but how do you test code that depends on environment variables? You don't want tests to use your production database or real API keys. Even worse, tests might pollute your real logs or trigger production alerts.

Isolation is the key principle: each test should have its own clean environment. Use monkeypatch in pytest to override environment variables during tests:

# test_config.py
import pytest
from config import Settings
 
def test_settings_from_env(monkeypatch):
    """Test that settings correctly reads environment variables."""
    monkeypatch.setenv("DEBUG", "true")
    monkeypatch.setenv("DATABASE_URL", "postgres://test/db")
    monkeypatch.setenv("API_KEY", "test-key")
 
    settings = Settings()
    assert settings.debug is True
    assert "test/db" in settings.database_url
    assert settings.api_key == "test-key"
 
def test_settings_validation_error(monkeypatch):
    """Test that missing required fields raise ValidationError."""
    monkeypatch.delenv("DATABASE_URL", raising=False)
 
    with pytest.raises(ValueError):  # Pydantic raises this for missing required fields
        Settings()
 
def test_logging_configuration(caplog):
    """Test that logging outputs correctly."""
    import logging
    logger = logging.getLogger("test")
 
    with caplog.at_level(logging.INFO):
        logger.info("Test message")
 
    assert "Test message" in caplog.text

This approach keeps tests isolated. Each test gets its own clean environment. Production config never bleeds into test runs. The monkeypatch fixture automatically restores the original environment after each test, so there are no side effects between tests. You can test the validation behavior, the type coercion, and the startup checks all without touching a real environment.

Advanced Logging: Filters and Custom Handlers

For more complex scenarios, you might want to filter logs by criteria or send them to custom destinations. When your application integrates with 10 different libraries, each with its own logger, your logs become noise unless you're selective.

Here's a filter that only logs messages from your app, hiding noise from third-party libraries:

import logging
 
class AppOnlyFilter(logging.Filter):
    def filter(self, record):
        # Only allow logs from your app, not from libraries
        return record.name.startswith("myapp")
 
# Add to your logging config
LOGGING_CONFIG = {
    "version": 1,
    "filters": {
        "app_only": {
            "()": AppOnlyFilter
        }
    },
    "handlers": {
        "console": {
            "class": "logging.StreamHandler",
            "filters": ["app_only"]
        }
    },
    "root": {
        "handlers": ["console"]
    }
}

For a custom handler that sends logs to an external service:

import logging
import requests
 
class HTTPHandler(logging.Handler):
    def __init__(self, url):
        super().__init__()
        self.url = url
 
    def emit(self, record):
        try:
            log_entry = self.format(record)
            requests.post(self.url, json={"message": log_entry})
        except Exception:
            self.handleError(record)
 
# Use it
handler = HTTPHandler("https://logs.example.com/ingest")
logger.addHandler(handler)

This is useful if you're shipping logs to a centralized logging service. In production, you might use the pythonjsonlogger library to do this more robustly:

pip install python-json-logger

from pythonjsonlogger import jsonlogger
 
handler = logging.StreamHandler()
formatter = jsonlogger.JsonFormatter()
handler.setFormatter(formatter)
logger.addHandler(handler)

This automatically outputs valid JSON without writing custom formatters.

Environment Variables Across Different Platforms

Your application might run on different platforms, local laptop, Docker container, Kubernetes cluster, serverless function. Each has a different way to set environment variables, but the beautiful part is your code doesn't care. The abstraction holds everywhere:

Local development: Use .env file and python-dotenv.

# Manual
export DEBUG=True
python app.py
 
# Or via .env
python app.py  # Loads .env automatically

Docker: Set in Dockerfile or docker-compose:

FROM python:3.11
ENV DEBUG=False
ENV DATABASE_URL=postgres://db:5432/mydb

Or in docker-compose.yml:

services:
  app:
    build: .
    environment:
      DEBUG: "False"
      DATABASE_URL: postgres://db:5432/mydb

Kubernetes: Use ConfigMaps for non-sensitive config, Secrets for sensitive data:

apiVersion: v1
kind: ConfigMap
metadata:
  name: app-config
data:
  DEBUG: "false"
  LOG_LEVEL: "INFO"
 
---
apiVersion: v1
kind: Secret
metadata:
  name: app-secrets
type: Opaque
stringData:
  DATABASE_URL: postgres://user:pass@db:5432/mydb
  API_KEY: secret-key-123
 
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  template:
    spec:
      containers:
        - name: app
          image: myapp:latest
          envFrom:
            - configMapRef:
                name: app-config
            - secretRef:
                name: app-secrets

The point: your Python code doesn't change. It just reads from os.environ. The platform determines how those variables get there.

Performance Considerations

Logging has performance implications, especially in high-traffic applications. Keep these in mind:

Use appropriate log levels: DEBUG is verbose; INFO is normal; WARNING and ERROR are sparse. In production, set to INFO or WARNING to reduce I/O.

Asynchronous handlers: Heavy logging can block your application. Use QueueHandler to log asynchronously:

import logging
import logging.handlers
from logging.handlers import QueueHandler, QueueListener
from queue import Queue
 
# Create a queue
log_queue = Queue()
queue_handler = QueueHandler(log_queue)
 
# Create actual handlers that write to the queue
file_handler = logging.handlers.RotatingFileHandler("app.log")
file_handler.setFormatter(logging.Formatter("%(levelname)s - %(message)s"))
 
# Start listener in background
listener = QueueListener(log_queue, file_handler, respect_handler_level=True)
listener.start()
 
# Use queue handler in your logger
logger = logging.getLogger(__name__)
logger.addHandler(queue_handler)
logger.setLevel(logging.INFO)
 
# Later, before exiting:
listener.stop()

Now logging is non-blocking, your application doesn't wait for disk I/O.

Conditional expensive computations: If a log message would require expensive computation, check the level first:

# BAD: Expensive work happens even if not logged
logger.debug(f"User data: {expensive_function()}")
 
# GOOD: Only compute if debug is enabled
if logger.isEnabledFor(logging.DEBUG):
    logger.debug(f"User data: {expensive_function()}")

Real Pitfalls: Case Studies

Let me share some real problems I've seen in production to make this concrete.

Pitfall 1: Commit .env by accident

A developer committed .env with a real database password. It stayed in git history for months before someone noticed. Lesson: Add .env to .gitignore before creating it. Use a pre-commit hook (you'll learn about this in article 48) to reject commits with .env files.

# .gitignore
.env
.env.local
.env.*.local

Pitfall 2: Silent configuration failures

A new deployment server didn't have the DATABASE_URL environment variable set. The code defaulted to localhost, so it tried to connect to the deployment server's own machine. It took hours to find because the error message was "connection refused" without context.

Better approach:

class Settings(BaseSettings):
    database_url: str  # Required, no default
 
    def __init__(self, **data):
        super().__init__(**data)
        if not self.database_url:
            raise ValueError(
                "DATABASE_URL environment variable is not set. "
                "Set it before starting the application."
            )

Fail loudly at startup if required variables are missing. Don't hide behind defaults.

Pitfall 3: Log format inconsistency

Different parts of the code were logging with different formats. Some had timestamps, some didn't. Some included module names, some didn't. Parsing logs in a centralized logging system became a nightmare.

This is why centralized configuration via dictConfig matters. One format everywhere.

Pitfall 4: Accidental secret leaks in logs

A developer tried to debug a database connection issue by logging the entire connection string:

logger.error(f"Connection failed: {database_url}")

The logs went to an external service. Three months later, someone reviewing logs found the password. This is a compliance violation in regulated industries.

Always mask sensitive data:

# Parse URL to hide password
from urllib.parse import urlparse
parsed = urlparse(database_url)
safe_url = f"{parsed.scheme}://***@{parsed.netloc}/{parsed.path}"
logger.error(f"Connection failed: {safe_url}")

Best Practices Summary

Here's a checklist for professional logging and configuration:

1. Use logging module, never raw print() for anything important.
2. Configure centrally via dictConfig in one place.
3. Different levels by module, verbose for your code, quiet for libraries.
4. Use Pydantic settings for type-safe environment variable handling.
5. Load .env at startup, before importing application modules.
6. Never commit .env or secrets; commit .env.example instead.
7. Validate secrets in production, fail fast if required values are missing.
8. Use structured logging (JSON) in production for easier parsing.
9. Log without leaking secrets, mask passwords and API keys.
10. Test configuration with pytest's monkeypatch.
11. Consider async handlers if logging volume is high.
12. Follow Twelve-Factor principles, config in environment, logs to stdout.

Connecting to the Broader Picture

You now have three critical pieces in place:

• Article 46 (pyproject.toml): Your code is packaged, dependencies are managed.
• Article 47 (this one): Your code logs properly and reads configuration safely.
• Article 48 (coming next): Your code passes quality checks (linting, formatting).

These three layers form the foundation of production code. Without any one of them, your application is half-baked. With all three, you're ready to deploy with confidence.

The configuration patterns you've learned scale to complex applications. A microservices architecture with 50 services? Each reads its config from the environment. A monolith with hundreds of settings? Still just environment variables, validated by Pydantic. The pattern holds.

Wrapping Up

The patterns in this article are not optional extras, they are the difference between code that works on your machine and code that operates reliably in the world. Logging gives you visibility into a running system without touching it. Environment-based configuration gives your application the flexibility to run anywhere without modification. Together they define the operational maturity of your codebase.

You now have a complete toolkit: a logging system tuned for both development readability and production observability, type-validated configuration that fails loudly on bad input, a secret management discipline that keeps credentials out of your codebase, and testing strategies that keep all of it honest. These are not advanced topics you revisit later, they are habits to build now, from the start of every project.

Take these patterns into your packaged project from article 46. Wire up the settings class, configure dictConfig, and feel how clean it is when you can change log verbosity or swap database URLs through environment variables alone. You will notice the difference immediately, and so will everyone who operates your software after you. That is what professional Python development feels like.

In your next article, you'll add code quality tooling (linters, formatters, pre-commit hooks) to catch bugs and enforce consistency before they reach production. The three-part foundation, packaging, observability, and quality gates, is almost complete.

Welcome to the professional side of Python development. You're building systems that others can run, debug, and maintain. That's the real skill.