Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions CONTRIBUTING.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,19 @@ When modifying driver files, rebuilding Cython modules is often necessary.
Without caching, each such rebuild may take over a minute. Caching usually brings it
down to about 2-3 seconds.

**Important:** After modifying any ``.py`` file under ``cassandra/`` that is
Cython-compiled (such as ``query.py``, ``protocol.py``, ``cluster.py``, etc.),
extensions must be rebuilt before running tests. If you always use ``uv run``
(e.g. ``uv run pytest``), this is handled automatically via the ``cache-keys``
configuration in ``pyproject.toml``. If you invoke ``pytest`` directly, you can
rebuild with::

uv sync --reinstall-package scylla-driver

Without rebuilding, Python will load the stale compiled extension (``.so`` / ``.pyd``)
instead of your modified ``.py`` source, and your changes will not actually be tested.
The test suite will emit a warning if it detects this situation.

Comment on lines +43 to +55
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No no no, calling setup.py directly is deprecated for many years now and shoud never be done.
The tool to work with the driver is uv, and it is the one that should be used in such guides.

How did you even encounter this problem? In pyproject.toml section we have this section:

[tool.uv]
cache-keys = [
    { file = "pyproject.toml" },
    { file = "setup.py" },
    # Cythonized modules
    { file = "cassandra/io/libevwrapper.c" },
    { file = "cassandra/cmurmur3.c" },
    { file = "cassandra/cluster.py" },
    { file = "cassandra/concurrent.py" },
    { file = "cassandra/connection.py" },
    { file = "cassandra/cqltypes.py" },
    { file = "cassandra/metadata.py" },
    { file = "cassandra/pool.py" },
    { file = "cassandra/protocol.py" },
    { file = "cassandra/query.py" },
    { file = "cassandra/util.py" },
    { file = "cassandra/shard_info.py" },
    { file = "cassandra/*.pyx" },
    { file = "cassandra/*.pxd" },

    # Env variables used in setup.py
    { env = "CASS_DRIVER_LIBEV_INCLUDES" },
    { env = "CASS_DRIVER_LIBEV_LIBS" },
    { env = "CASS_DRIVER_NO_EXTENSIONS" },
    { env = "CASS_DRIVER_NO_LIBEV" },
    { env = "CASS_DRIVER_NO_CYTHON" },
    { env = "CASS_DRIVER_BUILD_CONCURRENCY" },
    { env = "CASS_DRIVER_BUILD_EXTENSIONS_ARE_MUST" },

    # used by setuptools_scm
    { git = { commit = true, tags = true } },
]

It already achieves what this PR is trying to achieve, assuming you use uv (which you should).

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should remove those, not add new ones.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Docs fixed in 2 additional commits. I hope it's OK.

Building the Docs
=================

Expand Down
33 changes: 12 additions & 21 deletions docs/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,6 @@ threads used to build the driver and any C extensions:

.. code-block:: bash

$ # installing from source
$ CASS_DRIVER_BUILD_CONCURRENCY=8 python setup.py install
$ # installing from pip
$ CASS_DRIVER_BUILD_CONCURRENCY=8 pip install scylla-driver

Note that by default (when CASS_DRIVER_BUILD_CONCURRENCY is not specified), concurrency will be equal to the number of
Expand Down Expand Up @@ -108,7 +105,7 @@ installed. You can find the list of dependencies in

Once the dependencies are installed, simply run::

python setup.py install
pip install .


(*Optional*) Non-python Dependencies
Expand All @@ -122,9 +119,9 @@ for token-aware routing with the ``Murmur3Partitioner``,
`libev <http://software.schmorp.de/pkg/libev.html>`_ event loop integration,
and Cython optimized extensions.

When installing manually through setup.py, you can disable both with
the ``--no-extensions`` option, or selectively disable them with
with ``--no-murmur3``, ``--no-libev``, or ``--no-cython``.
Extensions can be selectively disabled using environment variables:
``CASS_DRIVER_NO_EXTENSIONS=1`` (disable all), ``CASS_DRIVER_NO_CYTHON=1``,
or ``CASS_DRIVER_NO_LIBEV=1``.

To compile the extensions, ensure that GCC and the Python headers are available.

Expand All @@ -149,31 +146,25 @@ This is not a hard requirement, but is engaged by default to build extensions of
pure Python implementation.

This is a costly build phase, especially in clean environments where the Cython compiler must be built
This build phase can be avoided using the build switch, or an environment variable::
This build phase can be avoided using an environment variable::

python setup.py install --no-cython
CASS_DRIVER_NO_CYTHON=1 pip install scylla-driver

Alternatively, an environment variable can be used to switch this option regardless of
Alternatively, the environment variable can be used to switch this option regardless of
context::

CASS_DRIVER_NO_CYTHON=1 <your script here>
- or, to disable all extensions:
CASS_DRIVER_NO_EXTENSIONS=1 <your script here>

This method is required when using pip, which provides no other way of injecting user options in a single command::

CASS_DRIVER_NO_CYTHON=1 pip install scylla-driver
CASS_DRIVER_NO_CYTHON=1 sudo -E pip install ~/python-driver

The environment variable is the preferred option because it spans all invocations of setup.py, and will
These environment variables are the preferred option, and will
prevent Cython from being materialized as a setup requirement.

If your sudo configuration does not allow SETENV, you must push the option flag down via pip. However, pip
applies these options to all dependencies (which break on the custom flag). Therefore, you must first install
dependencies, then use install-option::
If your sudo configuration does not allow SETENV, you must first install
dependencies, then install the driver::

sudo pip install futures
sudo pip install --install-option="--no-cython"
sudo CASS_DRIVER_NO_CYTHON=1 pip install scylla-driver


Supported Event Loops
Expand Down Expand Up @@ -205,7 +196,7 @@ install libev using any Windows package manager. For example, to install using
$ vcpkg install libev

If successful, you should be able to build and install the extension
(just using ``setup.py build`` or ``setup.py install``) and then use
(just using ``pip install .``) and then use
the libev event loop by doing the following:

.. code-block:: python
Expand Down
67 changes: 67 additions & 0 deletions tests/conftest.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
# Copyright ScyllaDB, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

import importlib.machinery
import os
import warnings

# Directory containing the Cython-compiled driver modules.
_CASSANDRA_DIR = os.path.join(os.path.dirname(__file__), os.pardir, "cassandra")


def pytest_configure(config):
"""Warn when a compiled Cython extension is older than its .py source.

Python's import system prefers compiled extensions (.so / .pyd) over pure
Python (.py) files. If a developer edits a .py file without rebuilding
the Cython extensions, the tests
will silently run the *old* compiled code, masking any regressions in the
Python source.

This hook detects such staleness at test-session startup so the developer
is alerted immediately.
"""
stale = []
# Iterate over .py sources and, for each module, look for the first
# existing compiled extension in EXTENSION_SUFFIXES order. This mirrors
# how Python's import machinery selects an extension module, and avoids
# globbing patterns like "*{suffix}" that can pick up ABI-tagged
# extensions built for other Python versions.
if os.path.isdir(_CASSANDRA_DIR):
for entry in os.listdir(_CASSANDRA_DIR):
if not entry.endswith(".py"):
continue
module_name, _ = os.path.splitext(entry)
py_path = os.path.join(_CASSANDRA_DIR, entry)
# For this module, find the first extension file Python would load.
for suffix in importlib.machinery.EXTENSION_SUFFIXES:
ext_path = os.path.join(_CASSANDRA_DIR, module_name + suffix)
if not os.path.exists(ext_path):
continue
if os.path.getmtime(py_path) > os.path.getmtime(ext_path):
stale.append((module_name, ext_path, py_path))
# Only consider the first matching suffix; this is the one
# the import system would actually use.
break

if stale:
names = ", ".join(m for m, _, _ in stale)
warnings.warn(
f"Stale Cython extension(s) detected: {names}. "
f"The .py source is newer than the compiled extension — tests "
f"will run the OLD compiled code, not your latest changes. "
f"Rebuild with: uv sync --reinstall-package scylla-driver\n"
f"Or use 'uv run pytest' which handles rebuilds automatically.",
stacklevel=1,
)
Loading