The selectors module provides a portable layer for monitoring many I/O objects and discovering which ones are ready for reading or writing. It selects an efficient implementation available on the operating system, such as epoll, kqueue, poll, or select. It is useful for servers, proxies, concurrent clients, consoles, and protocols with many nonblocking connections.
A selector does not perform I/O for you. It only reports readiness. The program must still accept connections, handle partial reads and writes, maintain buffers, detect EOF, and enforce timeouts. For larger systems, asyncio may provide a richer abstraction, but selectors exposes the essential event-loop layer.
Create the default selector
DefaultSelector chooses the recommended implementation for the platform.
import selectors
selector = selectors.DefaultSelector()
Close it during shutdown to release internal descriptors.
Register a listening socket
The socket must be nonblocking.
import socket
server = socket.socket()
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
server.bind(("127.0.0.1", 9000))
server.listen()
server.setblocking(False)
selector.register(server, selectors.EVENT_READ, data=None)
Read readiness on a listening socket means at least one connection can probably be accepted.
Call select
selector.select(timeout) returns pairs containing a SelectorKey and an event mask.
events = selector.select(timeout=1.0)
for key, mask in events:
if key.data is None:
accept(key.fileobj)
else:
service(key, mask)
An empty result after the timeout is normal and gives the loop a chance to perform maintenance.
SelectorKey
The key contains the registered object, descriptor, requested events, and associated data.
Use data for connection state such as the remote address, input buffer, output buffer, and protocol phase.
Accept every pending connection
In nonblocking mode, accept until BlockingIOError.
def accept(server):
while True:
try:
connection, address = server.accept()
except BlockingIOError:
break
connection.setblocking(False)
state = Connection(address=address, incoming=bytearray(), outgoing=bytearray())
selector.register(connection, selectors.EVENT_READ, data=state)
Accepting only once may leave other ready clients waiting for another event-loop cycle.
Read without blocking
A read event means the operation is likely to make progress.
try:
block = sock.recv(4096)
except BlockingIOError:
return
if block:
state.incoming.extend(block)
else:
close_connection(sock)
An empty byte string indicates an orderly peer shutdown.
TCP is a stream
One read does not equal one application message. Data may arrive split or combined.
Implement framing with a delimiter, length prefix, or documented format. See Python socket.
Input buffers
Accumulate bytes until a complete frame is available.
state.incoming.extend(block)
while message_is_complete(state.incoming):
message = extract_message(state.incoming)
process(message, state)
Set a maximum buffer size so a client cannot send endless data without completing a frame.
Partial writes
send() may write only part of the output buffer.
if state.outgoing:
try:
sent = sock.send(state.outgoing)
except BlockingIOError:
sent = 0
del state.outgoing[:sent]
Keep remaining bytes for the next write event.
Register EVENT_WRITE only when needed
Sockets are often reported writable most of the time. Keeping write interest enabled permanently can create a busy loop.
Add it when the output buffer becomes nonempty and remove it after the buffer drains.
modify
modify() changes the interest mask and associated data.
events = selectors.EVENT_READ
if state.outgoing:
events |= selectors.EVENT_WRITE
selector.modify(sock, events, data=state)
Centralize mask calculation so state remains consistent.
unregister and close
Remove an object from the selector before closing it.
def close_connection(sock):
try:
selector.unregister(sock)
except Exception:
pass
sock.close()
Production code should catch expected errors specifically and log inconsistent state.
Connection errors
Reset, broken pipe, and network failures can occur during read or write.
Close the affected connection and keep the event loop alive unless the failure affects the listening socket or selector itself.
Per-connection deadlines
The timeout passed to select() is not automatically a protocol timeout.
Track last activity in each connection state and close idle clients after a deadline.
now = time.monotonic()
for key in list(selector.get_map().values()):
state = key.data
if state and now - state.last_activity > 30:
close_connection(key.fileobj)
Use a monotonic clock
Deadlines and durations should use time.monotonic().
Wall-clock adjustments must not extend or shorten network timeouts.
Backpressure
If a peer reads slowly, the output buffer can grow.
Set a limit, pause new request processing, reject work, or close the connection. Never allow unbounded growth.
Fairness
One high-volume client should not monopolize the loop.
Limit bytes, messages, or processing time per connection during each cycle, then return to the selector.
Blocking work
Slow database calls, filesystem operations, and CPU-heavy functions block every connection when run inside the event loop.
Submit them to a bounded pool or use an asynchronous architecture. Wake the loop safely when results become available.
Wake up the selector
An external thread may need to notify the loop. A portable pattern uses socketpair() where available, or a suitable pipe.
Register the read end and write one byte from another thread to interrupt the wait.
Threads
Prefer to perform registration, modification, and closure in the event-loop thread.
Other threads can submit commands through a queue and trigger the wakeup mechanism, reducing races.
Integration with queue
A queue.Queue can transfer commands from workers to the event loop.
The next guide covers synchronized queues, shutdown, and backpressure.
TLS
Nonblocking TLS sockets may alternate between wanting reads and writes during handshakes and normal operations.
Handle SSLWantReadError and SSLWantWriteError by adjusting the interest mask. A high-level framework reduces this complexity for production services.
Windows limitations
Selectability of non-socket objects differs on Windows. Pipes and ordinary files may not behave as they do on Unix.
Test the target platform and use Windows-specific APIs when required.
Regular files
Ordinary files are generally reported ready and gain little from readiness multiplexing.
Use buffered chunk reading or worker threads for file I/O.
Signals and interruptions
Waits may be interrupted by signals depending on platform and version.
Keep signal handlers minimal and use a flag or wakeup mechanism for shutdown.
Graceful shutdown
Stop accepting new connections, allow existing clients to drain output buffers, apply a deadline, and close remaining connections.
Close registrations, the listening socket, wakeup objects, and the selector.
Observability
Record active connections, byte counts, messages, buffer sizes, duration, timeouts, and error classes.
Output-buffer and loop-latency metrics reveal backpressure and blocking callbacks.
Security
Limit connections per source, frame size, buffer size, idle time, and message rate.
Validate protocol fields before allocating memory and do not use IP address alone as identity.
Testing
Test partial reads, partial writes, several frames in one read, EOF, reset, slow clients, full buffers, timeout, wakeup, shutdown, IPv6, and Windows.
Use local sockets and generous deadlines instead of exact sleeps.
Common mistakes
Common failures include keeping EVENT_WRITE enabled permanently, assuming complete messages, ignoring partial sends, running blocking work in the loop, forgetting to unregister, allowing unlimited buffers, and modifying the selector concurrently without a protocol.
Conclusion
selectors provides portable multiplexing for many nonblocking sockets. Register only required interests, maintain per-connection buffers, enforce deadlines and backpressure, and clean up explicitly.
Consult the official selectors documentation, Python socket, and Python select.
