etlplus.api

etlplus.api package.

High-level helpers for building REST API clients with pagination, retry, rate limiting, and transport configuration.

Summary

Use etlplus.api.EndpointClient to register relative endpoint paths under a base URL and paginate responses. The client can apply rate limits between requests and perform exponential-backoff retries with full jitter.

Examples

Page-based pagination

>>> from etlplus.api import EndpointClient
>>> client = EndpointClient(
...     base_url="https://api.example.com/v1",
...     endpoints={"list_users": "/users"},
... )
>>> page_cfg = {
...     "type": "page",               # or "offset"
...     "records_path": "data.items", # dotted path into payload
...     "page_param": "page",
...     "size_param": "per_page",
...     "start_page": 1,
...     "page_size": 100,
... }
>>> rows = client.paginate(
...     "list_users",
...     query_parameters={"active": "true"},
...     pagination=page_cfg,
... )

Retries and network errors

>>> client = EndpointClient(
...     base_url="https://api.example.com/v1",
...     endpoints={"list": "/items"},
...     retry={"max_attempts": 5, "backoff": 0.5, "retry_on": [429, 503]},
...     retry_network_errors=True,
... )
>>> items = client.paginate(
...     "list", pagination={"type": "page", "page_size": 50}
... )

Absolute URLs

Use EndpointClient.paginate_url() for an already composed absolute URL. It accepts the same pagination config and returns either the raw JSON object (no pagination) or a list of record dicts aggregated across pages.

Notes

• EndpointClient.endpoints is read-only at runtime.

• Pagination defaults are centralized on the client (page, per_page,

  cursor, limit; start page 1; page size 100).

• Retries are opt-in via the retry parameter; backoff uses jitter.

• Use retry_network_errors=True to also retry timeouts/connection errors.

• Prefer JSONRecords (list of JSONDict) for paginated

  responses; scalar/record aliases are exported for convenience.

• The underlying Paginator is exported for advanced scenarios that

  need to stream pages manually.

See also

-

mod:etlplus.api.pagination for pagination helpers and config shapes

-

mod:etlplus.api.rate_limiting for rate-limit helpers and config shapes

-

mod:etlplus.api._errors for API error exceptions

-

mod:etlplus.api._retry_manager for retry policies

-

mod:etlplus.api._transport for HTTPAdapter helpers

-, error, and

Functions

build_http_adapter(cfg)	Build a requests HTTPAdapter from a configuration mapping.
build_session_with_adapters(adapters_cfg)	Mount adapters described by adapters_cfg onto a new session.
compose_api_request_env(cfg, source_obj, ex_opts)	Compose the API request environment.
compose_api_target_env(cfg, target_obj, ...)	Compose the API target environment.
paginate_with_client(client, endpoint_key, ...)	Paginate using the given client.
resolve_request(method, *[, session, timeout])	Resolve a request callable and effective timeout for an HTTP method.

Classes

EndpointClient(base_url, endpoints[, ...])	Immutable registry of endpoint path templates rooted at a base URL.
EndpointCredentialsBearer(*, token_url, ...)	Bearer token authentication via the OAuth2 Client Credentials flow.
RetryManager(*, policy[, ...])	Centralized retry logic for HTTP requests.
ApiConfig(*, base_url[, headers, endpoints, ...])	Configuration for a REST API service.
ApiProfileConfig(*, base_url[, headers, ...])	Profile configuration for a REST API service.
EndpointConfig(*, path[, method, ...])	Configuration for a single API endpoint.
RequestOptions(*[, params, headers, timeout])	Immutable snapshot of per-request options.
RetryStrategy(max_attempts, backoff, ...)	Normalized retry settings derived from a RetryPolicyDict.
HttpMethod(*values)	Supported HTTP verbs with a helper for request-body allowance.
ApiConfigDict	Top-level API config shape for etlplus.api.ApiConfig.
ApiProfileConfigDict	Shape accepted for one API profile entry.
ApiProfileDefaultsDict	Defaults block available under a profile (all keys optional).
EndpointConfigDict	Shape accepted by etlplus.api.EndpointConfig.from_obj().
HTTPAdapterMountConfigDict	Configuration mapping for mounting an HTTPAdapter on a Session.
HTTPAdapterRetryConfigDict	Retry configuration for urllib3 Retry.
RetryPolicyDict	Optional retry policy for HTTP requests.

Exceptions

ApiAuthError(*, url[, status, attempts, ...])	Authentication/authorization failure (e.g., 401/403).
ApiRequestError(*, url[, status, attempts, ...])	Base error for API request failures with rich context.
PaginationError(*, url[, status, attempts, ...])	Error raised during pagination with page context.