The http.client module implements the client side of HTTP and HTTPS at a lower level than urllib.request. It exposes connections, requests, and responses directly, giving precise control over paths, headers, bodies, streaming, connection reuse, proxy tunneling, and TLS contexts.
This control is useful for learning HTTP, testing servers, building specialized clients, and diagnosing integrations. Higher-level libraries are usually more productive for normal APIs. This guide covers the correct HTTPSConnection lifecycle, bounded reads, JSON and file uploads, persistent connections, errors, proxies, and security.
When to use http.client
Use it when you need explicit HTTP/1.1 behavior, custom protocol handling, or no external dependency. For complete URLs, redirects, cookies, and authentication, the urllib.request guide describes a higher standard-library layer. Large applications often benefit from a maintained client with pooling and richer timeout controls.
A first HTTPS connection
import http.client
connection = http.client.HTTPSConnection(
"www.python.org",
timeout=10,
)
try:
connection.request(
"GET",
"/",
headers={
"Host": "www.python.org",
"Accept": "text/html",
"User-Agent": "MyTool/1.0",
},
)
response = connection.getresponse()
print(response.status, response.reason)
body = response.read(200_000)
finally:
connection.close()
The constructor receives a host and optional port, not a complete URL. The request target is normally an absolute path such as /docs?page=1. Always set a timeout.
HTTPS and SSLContext
HTTPSConnection verifies certificates and hostnames by default. Load a private CA with a secure context when necessary.
import ssl
context = ssl.create_default_context(cafile="company-ca.pem")
connection = http.client.HTTPSConnection(
"internal-api.example",
timeout=10,
context=context,
)
Do not pass an unverified context to work around failures. Repair the trust chain. The Python ssl guide explains secure TLS settings.
The request and getresponse lifecycle
Send a request, obtain the response, and fully read or close that response before sending another request on the same connection.
connection.request("GET", "/first")
first = connection.getresponse()
first_data = first.read()
connection.request("GET", "/second")
second = connection.getresponse()
second_data = second.read()
Unread bytes prevent the next response from being framed correctly. Large bodies should be consumed in chunks until EOF.
Bounded response reads
MAX_BYTES = 5 * 1024 * 1024
def read_limited(response, maximum=MAX_BYTES) -> bytes:
chunks = []
total = 0
while chunk := response.read(64 * 1024):
total += len(chunk)
if total > maximum:
response.close()
raise ValueError("Response exceeded the limit")
chunks.append(chunk)
return b"".join(chunks)
Content-Length is useful but not sufficient. Count the bytes actually received. Stream downloads to a temporary file rather than retaining every chunk.
Reading headers
response = connection.getresponse()
content_type = response.getheader("Content-Type")
content_length = response.getheader("Content-Length")
all_headers = response.getheaders()
getheader() joins repeated values with commas. That is not correct for every field, especially Set-Cookie. Use getheaders() when duplicate fields must remain separate.
Sending JSON
import json
payload = json.dumps({"name": "Example"}).encode("utf-8")
headers = {
"Content-Type": "application/json",
"Accept": "application/json",
"Content-Length": str(len(payload)),
}
connection.request("POST", "/items", body=payload, headers=headers)
response = connection.getresponse()
raw = read_limited(response, 500_000)
For a bytes body, the library can calculate Content-Length automatically. An explicit value must match the exact number of bytes.
Uploading a file or iterable
When a body is a file or general iterable and no length is provided, the library normally uses chunked transfer encoding.
with open("file.bin", "rb") as file:
connection.request(
"PUT",
"/upload",
body=file,
headers={"Content-Type": "application/octet-stream"},
)
response = connection.getresponse()
response.read()
Some legacy servers do not accept chunked request bodies. Provide a correct length when required. A file body may not be safely replayable after authentication or network failure unless repositioned.
Streaming a download
from pathlib import Path
connection.request("GET", "/download")
response = connection.getresponse()
if response.status != 200:
response.read(20_000)
raise RuntimeError(f"HTTP {response.status}")
with Path("download.tmp").open("wb") as output:
total = 0
while chunk := response.read(64 * 1024):
total += len(chunk)
if total > 100 * 1024 * 1024:
response.close()
raise ValueError("File is too large")
output.write(chunk)
Verify a digest before moving the file into place. The Python hashlib guide covers SHA-256 verification.
Handling HTTP status
http.client does not raise an exception for 404 or 500. Inspect response.status yourself.
if 200 <= response.status < 300:
data = read_limited(response)
elif response.status == 404:
response.read(20_000)
raise LookupError("Resource not found")
else:
response.read(20_000)
raise RuntimeError(f"Server returned {response.status}")
Redirects, 429, Retry-After, and authentication challenges are also application responsibilities.
Persistent connections
HTTP/1.1 allows reuse, reducing TCP and TLS handshakes. Reuse only after consuming each response. Do not use the same connection concurrently from multiple threads without strict coordination.
If RemoteDisconnected or another connection failure occurs before an idempotent operation, close and reconnect. Do not blindly replay POST because the server may have processed it.
HEAD requests
connection.request("HEAD", "/file.zip")
response = connection.getresponse()
print(response.status, response.getheader("Content-Length"))
response.read()
HEAD has no response body, but complete the response lifecycle. Reported metadata does not replace actual limits during a later GET.
CONNECT proxy tunnels
proxy = http.client.HTTPSConnection("proxy.example", 8443, timeout=10)
proxy.set_tunnel(
"www.python.org",
443,
headers={"Host": "www.python.org:443"},
)
proxy.request("GET", "/")
response = proxy.getresponse()
Protect proxy credentials and validate the destination certificate. Since Python 3.12, CONNECT uses HTTP/1.1 and get_proxy_response_headers() exposes the proxy response headers.
Step-by-step request APIs
putrequest(), putheader(), endheaders(), and send() expose lower protocol stages. Use them only when request() is insufficient; manual header and chunk framing mistakes are easy to introduce.
Exceptions and uncertain state
try:
connection.request("GET", "/")
response = connection.getresponse()
data = read_limited(response)
except (TimeoutError, OSError, http.client.HTTPException) as error:
connection.close()
raise RuntimeError("HTTP failure") from error
Important exceptions include IncompleteRead, BadStatusLine, LineTooLong, ResponseNotReady, and RemoteDisconnected. Close the connection whenever framing or state is uncertain.
Debug output
set_debuglevel(1) prints protocol details to stdout. Use it only in controlled development because authorization and other sensitive headers may be displayed.
SSRF protection
If host and port come from an external user, validate DNS and resolved IPs before connecting. Block loopback, private, link-local, and cloud metadata ranges. Direct host connections remain vulnerable to SSRF and DNS rebinding.
Choosing a higher-level client
Use urllib.request or a third-party HTTP client when redirects, cookies, URL parsing, authentication, and connection pools are needed. http.client is best for precise control, education, and narrowly scoped infrastructure.
The REST API guide covers a complete integration workflow.
Common mistakes
Common errors are missing timeouts, passing a full URL as a normal path, failing to consume a response, sharing one connection unsafely, unlimited body reads, disabling TLS validation, replaying POST after failure, and enabling debug output around credentials.
Best practices
Use HTTPS with a validated context, explicit timeout, bounded bodies, streaming, and guaranteed cleanup. Consume or close each response. Reuse only healthy connections, distinguish idempotent methods, and restrict external hosts when possible.
Conclusion
http.client provides direct access to Python’s HTTP/1.1 client. It offers control over connections, requests, responses, streaming, and proxies, while leaving redirects, status policy, retries, and limits to your code. Use it when that control justifies the added responsibility.
Read the official http.client documentation and RFC 9112 for HTTP/1.1.
