Compatibility with urllib3-future: `ProgrammingError: expected bytes-like object, not int`

[amotl]

Problem

from crate.client import connect

if __name__ == "__main__":
    conn = connect("http://localhost:4200", username="crate", password="crate")
    cursor = conn.cursor()
    cursor.execute("SELECT 42")
    cursor.close()
    conn.close()

Traceback (most recent call last):
  File "/path/to/site-packages/crate/client/http.py", line 587, in _request
    response = self.server_pool[next_server].request(
        method,
    ...<5 lines>...
        **kwargs,
    )
  File "/path/to/site-packages/crate/client/http.py", line 196, in request
    return self.pool.urlopen(
           ~~~~~~~~~~~~~~~~~^
        method,
        ^^^^^^^
    ...<4 lines>...
        **kwargs,
        ^^^^^^^^^
    )
    ^
  File "/path/to/site-packages/urllib3/connectionpool.py", line 1740, in urlopen
    response = self._make_request(  # type: ignore[call-overload,misc]
        conn,
    ...<15 lines>...
        extension=extension,
    )
  File "/path/to/site-packages/urllib3/connectionpool.py", line 1308, in _make_request
    rp = conn.request(
        method,
    ...<7 lines>...
        on_upload_body=on_upload_body,
    )
  File "/path/to/site-packages/urllib3/connection.py", line 506, in request
    rp = self.endheaders(expect_body_afterward=chunks is not None)
  File "/path/to/site-packages/urllib3/backend/hface.py", line 1323, in endheaders
    self._protocol.submit_headers(
    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^
        self._stream_id,
        ^^^^^^^^^^^^^^^^
        self.__headers,
        ^^^^^^^^^^^^^^^
        end_stream=should_end_stream,
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/path/to/site-packages/urllib3/contrib/hface/protocols/http1/_h11.py", line 184, in submit_headers
    self._h11_submit(headers_to_request(headers))
                     ~~~~~~~~~~~~~~~~~~^^^^^^^^^
  File "/path/to/site-packages/urllib3/contrib/hface/protocols/http1/_h11.py", line 91, in headers_to_request
    return h11.Request(
           ~~~~~~~~~~~^
        method=method,  # type: ignore[arg-type]
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        headers=regular_headers,
        ^^^^^^^^^^^^^^^^^^^^^^^^
        target=target,
        ^^^^^^^^^^^^^^
    )
    ^
  File "/path/to/site-packages/h11/_events.py", line 96, in __init__
    self, "headers", normalize_and_validate(headers, _parsed=_parsed)
                     ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/site-packages/h11/_headers.py", line 164, in normalize_and_validate
    value = bytesify(value)
  File "/path/to/site-packages/h11/_util.py", line 134, in bytesify
    raise TypeError("expected bytes-like object, not int")
TypeError: expected bytes-like object, not int

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "example.py", line 10, in <module>
    cursor.execute("SELECT 42")
    ~~~~~~~~~~~~~~^^^^^^^^^^^^^
  File "/path/to/site-packages/crate/client/cursor.py", line 57, in execute
    self._result = self.connection.client.sql(
                   ~~~~~~~~~~~~~~~~~~~~~~~~~~^
        sql, parameters, bulk_parameters
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/path/to/site-packages/crate/client/http.py", line 508, in sql
    content = self._json_request("POST", self.path, data=data)
  File "/path/to/site-packages/crate/client/http.py", line 639, in _json_request
    response = self._request(method, path, data=data)
  File "/path/to/site-packages/crate/client/http.py", line 632, in _request
    raise ProgrammingError(_ex_to_message(e)) from e
crate.client.exceptions.ProgrammingError: expected bytes-like object, not int

[amotl]

The program works when downgrading to urllib3 1.x.

uv pip install --upgrade "crate" "urllib3<2"

[amotl]

I can only reproduce it in a single repository/folder, but nowhere else. Closing.

[amotl]

This happens to happen when the urllib3-future package is installed. It gets pulled in by the niquests package. Because of this, we can't use the CrateDB Client together with the Grafana Client.

/cc @Ousret

[amotl]

We submitted a report to the authors of urllib3-future with a request to increase compatibility. In the meanwhile, we contributed a patch to adjust crate-python to make it compatible, which will not hurt too much.

• expected bytes-like object, not int jawah/urllib3.future#318
• Fix compatibility with urllib3-future #784