Curl Object

class pycurl.Curl → New Curl object

Creates a new Curl Object which corresponds to a CURL handle in libcurl. Curl objects automatically set CURLOPT_VERBOSE to 0, CURLOPT_NOPROGRESS to 1, provide a default CURLOPT_USERAGENT and setup CURLOPT_ERRORBUFFER to point to a private error buffer.

Implicitly calls pycurl.global_init() if the latter has not yet been called.

The Curl object can be used as a context manager. Exiting the context calls close().

Example:

with pycurl.Curl() as c:
    # perform operations

Curl objects have the following methods:

close() → None

Close handle and end curl session.

Corresponds to curl_easy_cleanup in libcurl. This method is automatically called by pycurl when a Curl object no longer has any references to it, but can also be called explicitly.

setopt(option, value) → None

Set curl session option. Corresponds to curl_easy_setopt in libcurl.

option specifies which option to set. PycURL defines constants corresponding to CURLOPT_* constants in libcurl, except that the CURLOPT_ prefix is removed. For example, CURLOPT_URL is exposed in PycURL as pycurl.URL, with some exceptions as detailed below. For convenience, CURLOPT_* constants are also exposed on the Curl objects themselves:

import pycurl
c = pycurl.Curl()
c.setopt(pycurl.URL, "http://www.python.org/")
# Same as:
c.setopt(c.URL, "http://www.python.org/")

The following are exceptions to option constant naming convention:

  • CURLOPT_FILETIME is mapped as pycurl.OPT_FILETIME

  • CURLOPT_CERTINFO is mapped as pycurl.OPT_CERTINFO

  • CURLOPT_COOKIELIST is mapped as pycurl.COOKIELIST and, as of PycURL 7.43.0.2, also as pycurl.OPT_COOKIELIST

  • CURLOPT_RTSP_CLIENT_CSEQ is mapped as pycurl.OPT_RTSP_CLIENT_CSEQ

  • CURLOPT_RTSP_REQUEST is mapped as pycurl.OPT_RTSP_REQUEST

  • CURLOPT_RTSP_SERVER_CSEQ is mapped as pycurl.OPT_RTSP_SERVER_CSEQ

  • CURLOPT_RTSP_SESSION_ID is mapped as pycurl.OPT_RTSP_SESSION_ID

  • CURLOPT_RTSP_STREAM_URI is mapped as pycurl.OPT_RTSP_STREAM_URI

  • CURLOPT_RTSP_TRANSPORT is mapped as pycurl.OPT_RTSP_TRANSPORT

value specifies the value to set the option to. Different options accept values of different types:

  • Options specified by curl_easy_setopt as accepting 1 or an integer value accept Python integers and booleans:

    c.setopt(pycurl.FOLLOWLOCATION, True)
c.setopt(pycurl.FOLLOWLOCATION, 1)

  • Options specified as accepting strings by curl_easy_setopt accept bytes and str with ASCII code points only. For more information, please refer to String And Unicode Handling. Example:

    c.setopt(pycurl.URL, "http://www.python.org/")
c.setopt(pycurl.URL, b"http://www.python.org/")

  • HTTP200ALIASES, HTTPHEADER, POSTQUOTE, PREQUOTE, PROXYHEADER and QUOTE accept a list or tuple of strings. The same rules apply to these strings as do to string option values. Example:

    c.setopt(pycurl.HTTPHEADER, ["Accept:"])
c.setopt(pycurl.HTTPHEADER, ("Accept:",))

  • READDATA accepts a file object or any Python object which has a read method. READDATA is emulated in PycURL via READFUNCTION. The file should generally be opened in binary mode. Example:

    f = open('file.txt', 'rb')
c.setopt(c.READDATA, f)

  • WRITEDATA and WRITEHEADER accept a file object or any Python object which has a write method. WRITEDATA is emulated in PycURL via WRITEFUNCTION. The file should generally be opened in binary mode. Example:

    f = open('/dev/null', 'wb')
c.setopt(c.WRITEDATA, f)

  • *FUNCTION options accept a function. Supported callbacks are documented in Callbacks. Example:

    import io
b = io.BytesIO()
c.setopt(pycurl.WRITEFUNCTION, b.write)

  • SHARE option accepts a CurlShare Object.

  • STDERR option is not currently supported.

It is possible to set integer options - and only them - that PycURL does not know about by using the numeric value of the option constant directly. For example, pycurl.VERBOSE has the value 42, and may be set as follows:

c.setopt(42, 1)

setopt can reset some options to their default value, performing the job of pycurl.Curl.unsetopt(), if None is passed for the option value. The following two calls are equivalent:

c.setopt(c.URL, None)
c.unsetopt(c.URL)

Raises TypeError when the option value is not of a type accepted by the respective option, and pycurl.error exception when libcurl rejects the option or its value.

perform() → None

Perform a file transfer.

Corresponds to curl_easy_perform in libcurl.

Raises pycurl.error exception upon failure.

perform_rb() → response_body

Perform a file transfer and return response body as a byte string.

This method arranges for response body to be saved in a BytesIO instance, then invokes perform to perform the file transfer, then returns the value of the BytesIO instance which is a bytes instance. Errors during transfer raise pycurl.error exceptions just like in perform.

Use perform_rs to retrieve response body as a str.

Raises pycurl.error exception upon failure.

Added in version 7.43.0.2.

perform_rs() → response_body

Perform a file transfer and return response body as a string.

This method arranges for response body to be saved in a BytesIO instance, then invokes perform to perform the file transfer, then decodes the response body in Python’s default encoding and returns the decoded body as a Unicode string (str instance). Note: decoding happens after the transfer finishes, thus an encoding error implies the transfer/network operation succeeded.

Any transfer errors raise pycurl.error exception, just like in perform.

Use perform_rb to retrieve response body as a byte string (bytes instance) without attempting to decode it.

Raises pycurl.error exception upon failure.

Added in version 7.43.0.2.

getinfo(option) → Result

Extract and return information from a curl session, decoding string data in Python’s default encoding at the time of the call. Corresponds to curl_easy_getinfo in libcurl. The getinfo method should not be called unless perform has been called and finished.

option is a constant corresponding to one of the CURLINFO_* constants in libcurl. Most option constant names match the respective CURLINFO_* constant names with the CURLINFO_ prefix removed, for example CURLINFO_CONTENT_TYPE is accessible as pycurl.CONTENT_TYPE. Exceptions to this rule are as follows:

  • CURLINFO_FILETIME is mapped as pycurl.INFO_FILETIME

  • CURLINFO_COOKIELIST is mapped as pycurl.INFO_COOKIELIST

  • CURLINFO_CERTINFO is mapped as pycurl.INFO_CERTINFO

  • CURLINFO_RTSP_CLIENT_CSEQ is mapped as pycurl.INFO_RTSP_CLIENT_CSEQ

  • CURLINFO_RTSP_CSEQ_RECV is mapped as pycurl.INFO_RTSP_CSEQ_RECV

  • CURLINFO_RTSP_SERVER_CSEQ is mapped as pycurl.INFO_RTSP_SERVER_CSEQ

  • CURLINFO_RTSP_SESSION_ID is mapped as pycurl.INFO_RTSP_SESSION_ID

The type of return value depends on the option, as follows:

  • Options documented by libcurl to return an integer value return a Python int.

  • Options documented by libcurl to return a floating point value return a Python float.

  • Options documented by libcurl to return a string value return a Python str. The data returned by libcurl is decoded using the default string encoding at the time of the call. If the data cannot be decoded using the default encoding, UnicodeDecodeError is raised. Use getinfo_raw to retrieve the data as bytes in these cases.

  • SSL_ENGINES and INFO_COOKIELIST return a list of strings. The same encoding caveats apply; use getinfo_raw to retrieve the data as a list of byte strings.

  • INFO_CERTINFO returns a list with one element per certificate in the chain, starting with the leaf; each element is a sequence of (key, value) tuples where both key and value are strings. String encoding caveats apply; use getinfo_raw to retrieve certificate data as byte strings.

  • For libcurl versions >= 7.45.0, CURLINFO_LASTSOCKET is aliased to CURLINFO_ACTIVESOCKET to avoid unreliable results on some platforms.

Example usage:

import pycurl
c = pycurl.Curl()
c.setopt(pycurl.OPT_CERTINFO, 1)
c.setopt(pycurl.URL, "https://python.org")
c.setopt(pycurl.FOLLOWLOCATION, 1)
c.perform()
print(c.getinfo(pycurl.HTTP_CODE))
# --> 200
print(c.getinfo(pycurl.EFFECTIVE_URL))
# --> "https://www.python.org/"
certinfo = c.getinfo(pycurl.INFO_CERTINFO)
print(certinfo)
# --> [(('Subject', 'C = AU, ST = Some-State, O = PycURL test suite,
         CN = localhost'), ('Issuer', 'C = AU, ST = Some-State,
         O = PycURL test suite, OU = localhost, CN = localhost'),
        ('Version', '0'), ...)]

Raises pycurl.error exception upon failure.

getinfo_raw(option) → Result

Extract and return information from a curl session, returning string data as byte strings. Corresponds to curl_easy_getinfo in libcurl. The getinfo_raw method should not be called unless perform has been called and finished.

option is a constant corresponding to one of the CURLINFO_* constants in libcurl. Most option constant names match the respective CURLINFO_* constant names with the CURLINFO_ prefix removed, for example CURLINFO_CONTENT_TYPE is accessible as pycurl.CONTENT_TYPE. Exceptions to this rule are as follows:

  • CURLINFO_FILETIME is mapped as pycurl.INFO_FILETIME

  • CURLINFO_COOKIELIST is mapped as pycurl.INFO_COOKIELIST

  • CURLINFO_CERTINFO is mapped as pycurl.INFO_CERTINFO

  • CURLINFO_RTSP_CLIENT_CSEQ is mapped as pycurl.INFO_RTSP_CLIENT_CSEQ

  • CURLINFO_RTSP_CSEQ_RECV is mapped as pycurl.INFO_RTSP_CSEQ_RECV

  • CURLINFO_RTSP_SERVER_CSEQ is mapped as pycurl.INFO_RTSP_SERVER_CSEQ

  • CURLINFO_RTSP_SESSION_ID is mapped as pycurl.INFO_RTSP_SESSION_ID

The type of return value depends on the option, as follows:

  • Options documented by libcurl to return an integer value return a Python int.

  • Options documented by libcurl to return a floating point value return a Python float.

  • Options documented by libcurl to return a string value return a Python bytes instance. The string contains whatever data libcurl returned. Use getinfo to retrieve this data as a Unicode string.

  • SSL_ENGINES and INFO_COOKIELIST return a list of byte strings. The same encoding caveats apply; use getinfo to retrieve the data as a list of potentially Unicode strings.

  • INFO_CERTINFO returns a list with one element per certificate in the chain, starting with the leaf; each element is a sequence of (key, value) tuples where both key and value are byte strings. String encoding caveats apply; use getinfo to retrieve certificate data as potentially Unicode strings.

Example usage:

import pycurl
c = pycurl.Curl()
c.setopt(pycurl.OPT_CERTINFO, 1)
c.setopt(pycurl.URL, "https://python.org")
c.setopt(pycurl.FOLLOWLOCATION, 1)
c.perform()
print(c.getinfo_raw(pycurl.HTTP_CODE))
# --> 200
print(c.getinfo_raw(pycurl.EFFECTIVE_URL))
# --> b"https://www.python.org/"
certinfo = c.getinfo_raw(pycurl.INFO_CERTINFO)
print(certinfo)
# --> [((b'Subject', b'C = AU, ST = Some-State, O = PycURL test suite,
         CN = localhost'), (b'Issuer', b'C = AU, ST = Some-State,
         O = PycURL test suite, OU = localhost, CN = localhost'),
        (b'Version', b'0'), ...)]

Raises pycurl.error exception upon failure.

Added in version 7.43.0.2.

reset() → None

Reset all options set on curl handle to default values, but preserves live connections, session ID cache, DNS cache, cookies, and shares.

Corresponds to curl_easy_reset in libcurl.

unsetopt(option) → None

Reset curl session option to its default value.

Only some curl options may be reset via this method.

libcurl does not provide a general way to reset a single option to its default value; pycurl.Curl.reset() resets all options to their default values, otherwise pycurl.Curl.setopt() must be called with whatever value is the default. For convenience, PycURL provides this unsetopt method to reset some of the options to their default values.

Raises pycurl.error exception on failure.

c.unsetopt(option) is equivalent to c.setopt(option, None).

pause(bitmask=PAUSE_ALL) → None

Pause or unpause a curl handle. bitmask defaults to PAUSE_ALL. Pass a value such as PAUSE_RECV, PAUSE_SEND, or PAUSE_CONT to override.

Corresponds to curl_easy_pause in libcurl. The argument should be derived from the PAUSE_RECV, PAUSE_SEND, PAUSE_ALL and PAUSE_CONT constants.

Raises pycurl.error exception upon failure.

recv(buffersize) → data

Receive data from a connection established with CONNECT_ONLY.

Receive up to buffersize bytes and return them as a bytes object. A returned empty bytes object indicates that the peer has closed the connection.

Raises ValueError if buffersize is negative.

Corresponds to curl_easy_recv in libcurl.

Because the underlying socket is used in non-blocking mode internally, this method raises BlockingIOError with errno set to EAGAIN when libcurl returns CURLE_AGAIN.

Raises pycurl.error exception upon failures other than CURLE_AGAIN.

recv_into(buffer[, nbytes]) → nbytes

Receive data from a connection established with CONNECT_ONLY into buffer.

buffer must be a writable bytes-like object.

If nbytes is 0 (the default), receive up to len(buffer) bytes. Otherwise, receive up to nbytes bytes. Returns the number of bytes received.

Raises ValueError if nbytes is negative or larger than len(buffer).

Corresponds to curl_easy_recv in libcurl.

Because the underlying socket is used in non-blocking mode internally, this method raises BlockingIOError with errno set to EAGAIN when libcurl returns CURLE_AGAIN.

Raises pycurl.error exception upon failures other than CURLE_AGAIN.

send(bytes) → count

Send data over a connection established with CONNECT_ONLY.

data may be any bytes-like object.

Returns the number of bytes sent. If fewer than len(data) bytes are sent, the remaining data should be sent in a subsequent call.

Corresponds to curl_easy_send in libcurl.

Because the underlying socket is used in non-blocking mode internally, this method raises BlockingIOError with errno set to EAGAIN when libcurl returns CURLE_AGAIN.

Raises pycurl.error exception upon failures other than CURLE_AGAIN.

errstr() → string

Return the internal libcurl error buffer of this handle as a string.

Return value is a str instance. Error buffer data is decoded using Python’s default encoding at the time of the call. If this decoding fails, UnicodeDecodeError is raised. Use errstr_raw to retrieve the error buffer as a byte string in this case.

errstr_raw() → byte string

Return the internal libcurl error buffer of this handle as a byte string.

Return value is a bytes instance. Unlike errstr, errstr_raw allows reading libcurl error buffer when its contents is not valid in Python’s default encoding.

Added in version 7.43.0.2.

setopt_string(option, value) → None

Set curl session option to a string value.

This method allows setting string options that are not officially supported by PycURL, for example because they did not exist when the version of PycURL being used was released. pycurl.Curl.setopt() should be used for setting options that PycURL knows about.

Warning: No checking is performed that option does, in fact, expect a string value. Using this method incorrectly can crash the program and may lead to a security vulnerability. Furthermore, it is on the application to ensure that the value object does not get garbage collected while libcurl is using it. libcurl copies most string options but not all; one option whose value is not copied by libcurl is CURLOPT_POSTFIELDS.

option would generally need to be given as an integer literal rather than a symbolic constant.

value can be a binary string or a Unicode string using ASCII code points, same as with string options given to PycURL elsewhere.

Example setting URL via setopt_string:

import pycurl
c = pycurl.Curl()
c.setopt_string(10002, "http://www.python.org/")

WebSocket methods (libcurl 7.86.0 or later):

ws_send(data, flags=None, fragsize=0, encoding='utf-8') → count

Send a WebSocket frame. In detached mode this requires CONNECT_ONLY=2; inside an active WRITEFUNCTION callback it may also be used to send a blocking reply.

data may be a str or any bytes-like object. str is encoded with encoding (UTF-8 by default); characters that are not representable in encoding raise UnicodeEncodeError. Passing None raises TypeError — use b"" for an empty payload.

flags is a bitmask built from the frame-type constants WS_TEXT, WS_BINARY, WS_CONT, WS_CLOSE, WS_PING, WS_PONG. When flags is omitted (None), the frame type is inferred: str -> WS_TEXT, bytes-like -> WS_BINARY. Explicit flags win. str + WS_BINARY and str + WS_CLOSE raise TypeError (use ws_close() for close frames, or pass bytes-like data).

fragsize maps to curl_ws_send’s fragsize parameter; 0 means “whole message”. WS_OFFSET is the companion flag for multi-call fragmented sends; see the libcurl docs for the rules.

Returns the number of bytes accepted by libcurl.

Raises BlockingIOError (errno=EAGAIN) in detached mode when libcurl returns CURLE_AGAIN. Inside a WRITEFUNCTION callback libcurl treats the call as blocking and returns only once the frame has been fully sent; BlockingIOError does not apply. Calls from other threads while perform() is running are rejected.

Corresponds to curl_ws_send in libcurl. Requires libcurl 7.86.0 or later. Raises pycurl.error for libcurl failures other than CURLE_AGAIN.

ws_recv(buffersize)

Receive a WebSocket frame chunk on a connection established with CONNECT_ONLY=2.

Receive up to buffersize bytes. Returns a 2-tuple (data, meta) where data is a bytes object containing the received payload chunk and meta is a WsFrame namedtuple carrying the per-frame metadata returned by libcurl for this call (age, flags, offset, bytesleft, len).

A single call may return only part of a frame’s payload: check meta.bytesleft to decide whether to loop. Reassembly of fragmented messages is the caller’s responsibility.

A buffersize of 0 performs a zero-length curl_ws_recv call. This returns (b"", meta) so callers can inspect frame metadata without consuming payload bytes. Frames with empty payload are consumed by this action.

Raises ValueError if buffersize is negative.

Corresponds to curl_ws_recv in libcurl. Requires libcurl 7.86.0 or later.

Because the underlying socket is used in non-blocking mode internally, this method raises BlockingIOError with errno set to EAGAIN when libcurl returns CURLE_AGAIN.

Raises pycurl.error exception upon failures other than CURLE_AGAIN.

ws_recv_into(buffer[, nbytes]) -> (nbytes, meta)

Receive a WebSocket frame chunk on a connection established with CONNECT_ONLY=2 into a caller-owned writable buffer.

buffer must be a writable bytes-like object (e.g. bytearray, memoryview, array.array).

If nbytes is 0 (the default), receive up to len(buffer) bytes. Otherwise, receive up to nbytes bytes.

Returns a 2-tuple (nbytes, meta) where nbytes is the number of bytes written into buffer and meta is a WsFrame namedtuple with the per-frame metadata returned by libcurl for this call.

Raises ValueError if nbytes is negative or larger than len(buffer).

If buffer has length 0, this performs a zero-length curl_ws_recv call and returns (0, meta) so callers can inspect frame metadata without consuming payload bytes. Frames with empty payload are consumed by this action.

Corresponds to curl_ws_recv in libcurl. Requires libcurl 7.86.0 or later.

Because the underlying socket is used in non-blocking mode internally, this method raises BlockingIOError with errno set to EAGAIN when libcurl returns CURLE_AGAIN.

Raises pycurl.error exception upon failures other than CURLE_AGAIN.

ws_meta() → WsFrame or None

Return a snapshot of the current WebSocket frame’s metadata.

This is a callback-context helper. It is intended to be called from inside an active WRITEFUNCTION callback on a WebSocket transfer, where it returns a WsFrame namedtuple with the metadata of the chunk currently being delivered.

Outside that context — including when used in detached mode (CONNECT_ONLY=2), after perform() has returned, or on a non-WebSocket transfer — libcurl’s curl_ws_meta() returns NULL and PycURL maps that NULL to Python None. No exception is raised; callers can use if c.ws_meta() is None to probe context validity.

In detached mode, prefer the metadata returned directly by ws_recv() / ws_recv_into() rather than a separate ws_meta() call.

Corresponds to curl_ws_meta in libcurl. Requires libcurl 7.86.0 or later.

ws_close(code=None, reason=None, encoding='utf-8') → count

Send a WebSocket close frame. In detached mode this requires CONNECT_ONLY=2; inside an active WRITEFUNCTION callback it may also be used to send a blocking reply.

Builds an RFC 6455 §5.5.1 close payload — an optional 2-byte big-endian status code followed by an optional UTF-8 reason — and sends it as a WS_CLOSE control frame. Prefer this over ws_send(bytes, WS_CLOSE): the payload format is non-obvious.

code is the WebSocket close status code. Omitted (None) sends an empty close payload. When specified, must be a valid wire code per RFC 6455 §7.4.1: 1000 (normal), 1001 (going away), 1002, 1003, 1007-1014, or a private-use value in 3000..4999. Codes 1004, 1005, 1006, 1015 are RFC-forbidden to send and rejected.

reason may be a str or any bytes-like object. str is encoded with encoding (UTF-8 by default). The resulting bytes must be valid UTF-8 on the wire; invalid UTF-8 raises UnicodeDecodeError, non-encodable input raises UnicodeEncodeError. reason without code raises ValueError. The combined payload (2-byte code + reason) must not exceed 125 bytes (RFC 6455 §5.5).

Returns the number of bytes accepted by libcurl.

Same blocking / non-blocking semantics as ws_send(). Calls from other threads while perform() is running are rejected.

Corresponds to curl_ws_send with CURLWS_CLOSE. Requires libcurl 7.86.0 or later. Raises pycurl.error for libcurl failures other than CURLE_AGAIN.

PycURL supports libcurl’s two documented WebSocket usage models:

  • Detached mode. Set CONNECT_ONLY to 2, call perform() to drive the handshake, and then drive the connection yourself with ws_send() and ws_recv() (or ws_recv_into()). In this mode, frame metadata is returned as the second element of each receive call’s result.

  • Callback-receive mode. Set a WRITEFUNCTION callback, leave CONNECT_ONLY unset (or 0), and call perform(). libcurl drives the transfer and delivers each received frame chunk to the write callback. Call ws_meta() from inside the callback to retrieve the current chunk’s WsFrame metadata; ws_meta() returns None outside that context.

PycURL does not reassemble fragmented messages or manage the WebSocket close handshake. libcurl automatically replies to ping frames unless WS_NOAUTOPONG or WS_RAW_MODE is enabled.

Frame metadata is exposed through the module-level WsFrame namedtuple (age, flags, offset, bytesleft, len). The flags field is a bitmask of the WS_* constants.

Caveats and sharp edges:

  • Do not combine ``CONNECT_ONLY=2`` with ``FORBID_REUSE`` — libcurl tears down the connection after the handshake perform() returns and the first ws_send() fails with CURLE_UNSUPPORTED_PROTOCOL.

  • A WebSocket handle is not thread-safe — see Thread Safety.

  • ``WS_RAW_MODE`` (via ``CURLOPT_WS_OPTIONS``) changes framing semantics. libcurl ignores ws_send’s flags argument and writes bytes verbatim. Raw-mode callers should pass bytes-like data; Python-side str/bytes inference still runs but the flag bits never reach the wire.

  • Replying inside a ``WRITEFUNCTION`` is allowed — see Callbacks. ws_send/ws_close behave as blocking sends in that context; ws_recv / ws_recv_into remain detached-only.

  • Runtime probe: 'ws' in pycurl.version_info()[8]. Compile-time: hasattr(pycurl, 'WS_TEXT').