Building Generic HTTP Responses with Pydantic

When building APIs or applications that return structured HTTP responses, it’s common to encounter scenarios where the response data can vary widely in type and structure.

Pydantic, a powerful data validation library in Python, provides tools to handle such cases elegantly. In this article, we’ll explore how to create a generic HTTP response model using Pydantic, addressing common challenges and providing a reusable solution.

The Problem

In many applications, HTTP responses need to include both data and metadata. For example, you might want to return a list of items along with pagination details or status information.

However, the data portion of the response can vary significantly—it could be a list of objects, a single object, or even a complex nested structure. Additionally, some data types are not natively supported by Pydantic, making it difficult to include them in responses without customization.

Key Challenges

1. Handling Arbitrary Data Types: Pydantic typically requires data to conform to its supported types. When working with external libraries or custom types, this can be restrictive.

2. Reusability: Creating a separate response model for each type of data is inefficient and leads to code duplication.

3. Metadata Inclusion: Responses often need to include metadata (e.g., pagination details, counts), which should be standardized across all responses.

The Solution: A Generic Response Model

Pydantic’s support for generics and configuration options like arbitrary_types_allowed provides a clean solution to these challenges. By creating a generic response model, we can handle arbitrary data types while maintaining a consistent structure for metadata.

The Code

Here’s the implementation of a generic HTTP response model using Pydantic:

"""The base classes and models to represent all HTTP responses."""

from typing import Generic, TypeVar

from pydantic import BaseModel, Field


class ResponseMetadata(BaseModel):
    """
    The meta information of the results of the response.

    Attributes
    ----------
    page : int
        The current page, and -1 if not paginated.
    page_size : int
        The current total pages, and -1 if not paginated.
    count : int | None
        The total count of items.
    """

    page: int = Field(
        default=-1,
        description="The current page, and -1 if not paginated.",
    )
    page_size: int = Field(
        default=-1,
        description="The current total pages, and -1 if not paginated.",
    )
    count: int | None = Field(default=-1, description="The total count of items.")


T = TypeVar("T")


class BaseHttpResponse(BaseModel, Generic[T]):
    """
    The base class for all HTTP responses.

    Attributes
    ----------
    metadata : ResponseMetadata
        The meta information of the results.
    data : T
        The data result(s) of the response.
    """

    class Config:
        """
        The config for the Pydantic model.

        Attributes
        ----------
        arbitrary_types_allowed : bool
            Allow arbitrary types.
        """

        arbitrary_types_allowed: bool = True

    metadata: ResponseMetadata = Field(
        description="The meta information of the results.",
        default=ResponseMetadata(count=-1),
    )
    data: T = Field(..., description="The data result(s) of the response.")

How It Works

1. Generic Type Support: The BaseHttpResponse class uses Generic[T] to allow the data field to accept any type. This makes the model reusable for different response types.

2. Arbitrary Types Allowed: By setting arbitrary_types_allowed=True in the Config class, Pydantic allows the data field to include types that are not natively supported.

3. Standardized Metadata: The metadata field provides a consistent structure for including pagination details, counts, or other metadata.

Example Usage

Here’s how you can use the BaseHttpResponse model in an application:

from dataclasses import dataclass

@dataclass()
class Example:
    name: str

# Example data
pod_statuses = [Example(...), Example(...)]

# Create a response
response = BaseHttpResponse(
    metadata=ResponseMetadata(count=len(pod_statuses)),
    data=pod_statuses,
)

print(response)

metadata=ResponseMetadata(page=-1, page_size=-1, count=2) data=[Example(name=Ellipsis), Example(name=Ellipsis)]

Benefits

1. Reusability: The generic model can be used for any type of response, reducing
   code duplication.

2. Flexibility: The arbitrary_types_allowed configuration allows integration with external libraries and custom types.

3. Consistency: Standardized metadata ensures that all responses follow the same structure, improving maintainability.

4. Type Safety: Pydantic’s type hints and validation ensure that the response data is correctly structured.

Conclusion

By leveraging Pydantic’s generic support and configuration options, we can create a flexible and reusable HTTP response model that handles arbitrary data types while maintaining a consistent structure.

This approach simplifies API development, improves code quality, and ensures that responses are both standardized and adaptable.