Skip to content

Latest commit

 

History

History
1276 lines (1053 loc) · 64.7 KB

changelog.rst

File metadata and controls

1276 lines (1053 loc) · 64.7 KB
.. cpp:namespace:: nanobind

Changelog

nanobind uses a semantic versioning policy for its API. It also has a separate ABI version that is not subject to semantic versioning.

The ABI version is relevant whenever a type binding from one extension module should be visible in another nanobind-based extension module. In this case, both modules must use the same nanobind ABI version, or they will be isolated from each other. Releases that don't explicitly mention an ABI version below inherit that of the preceding release.

Version 2.2.0 (TBA)

  • nanobind can now target free-threaded Python, which replaces the Global Interpreter Lock (GIL) with a fine-grained locking scheme (see PEP 703) to better leverage multi-core parallelism. A separate documation page explains this in detail.

  • nanobind has always used PEP 590 vector calls to efficiently dispatch calls to function and method bindings, but it lacked the ability to do so for constructors (e.g., MyType(arg1, arg2, ...)).

    Version 2.2.0 adds this missing part, which accelerates object construction by up to a factor of 2×. The difference is especially pronounced when passing keyword arguments to constructors. Note that this improvement only applies to Python version 3.9 and newer (commits #24907b0, #e24d7f3, #0acecb4).

  • The NVIDIA CUDA compiler (nvcc) is now explicitly supported and included in nanobind's CI test suite.

Version 2.1.0 (Aug 11, 2024)

Version 2.0.0 (May 23, 2024)

The 2.0.0 release of nanobind is entirely dedicated to types [1]! The project has always advertised seamless Python ↔ C++ interoperability, and this release tries to bring a similar level of interoperability to static type checkers like MyPy, PyRight, PyType, and editors with interactive autocompletion like Visual Studio Code, PyCharm, and many other LSP-compatible IDEs.

This required work on three fronts:

  1. Stub generation: the above tools all analyze Python code statically without running it. Because the import mechanism of compiled extensions depends the Python interpreter, these tools weren't able to inspect the contents of nanobind-based extensions.

    The usual solution involves writing stubs that expose the module contents to static analysis tools. However, writing stubs by hand is tedious and error-prone.

    This release adds tooling to automatically extract stubs from existing extensions. The process is fully integrated into the CMake-based build system and explained in a :ref:`new documentation section <stubs>`.

  2. Better default annotations: once stubs were available, this revealed the next problem: the default nanobind-provided function and class signatures were too rudimentary, and this led to a user poor experience.

    The release therefore improves many builtin type caster so that they produce more accurate type signatures. For example, the STL std::vector<T> caster now renders as collections.abc.Sequence[T] in stubs when it is used as an input, and list[T] when it is used as part of a return value. The :cpp:func:`nb::make_*_iterator() <make_iterator>` family of functions return typed iterators, etc.

  3. Advanced customization: a subset of the type signatures in larger binding projects will generally require further customization. The features listed below aim to enable precisely this:

    • In Python, many built-in types are generic and can be parameterized (e.g., list[int]). The :cpp:class:`nb::typed\<T, Ts...\> <typed>` wrapper enables such parameterization within C++ (for example, the int-specialized list would be written as nb::typed<nb::list, int>). :ref:`Read more <typing_generics_parameterizing>`.

    • The opposite is also possible: passing :cpp:class:`nb::is_generic() <is_generic>` to the class binding constructor

      nb::class_<MyType>(m, "MyType", nb::is_generic())

      produces a generic type that can be parameterized in Python (e.g. MyType[int]). :ref:`Read more <typing_generics_creating>`.

    • The :cpp:class:`nb::sig <sig>` annotation overrides the signature of a function or method, e.g.:

      m.def("f", &f, nb::sig("def f(x: Foo = Foo(0)) -> None"), "docstring");

      Each binding of an overloaded function can be customized separately. This feature can be used to add decorators or control how default arguments are rendered. :ref:`Read more <typing_signature_functions>`.

    • The :cpp:class:`nb::sig <sig>` annotation can also override class signatures in generated stubs. Stubs often take certain liberties in deviating somewhat from the precise type signature of the underlying implementation. For example, the following annotation adds an abstract base class advertising that the class implements a typed iterator.

      using IntVec = std::vector<int>;

nb::class_<IntVec>(m, "IntVec",
                   nb::sig("class IntVec(collections.abc.Iterable[int])"));

      Nanobind can't subclass Python types, hence this declaration is technically untrue. On the flipside, such a declaration can assist static checkers and improve auto-completion in visual IDEs. This is fine since these tools only perform a static analysis and never import the actual extension. :ref:`Read more <typing_signature_classes>`.

    • The :cpp:struct:`nb::for_setter <for_setter>` and :cpp:struct:`nb::for_getter <for_getter>` annotations enable passing function binding annotations (e.g., signature overrides) specifically to the setter or the getter part of a property.

    • The :cpp:class:`nb::arg("name") <arg>` argument annotation (and "name"_a shorthand) now have a :cpp:func:`.sig("signature") <arg::sig>` member to control how a default value is rendered in the stubs and docstrings. This provides more targeted control compared to overriding the entire function signature.

    • Finally, nanobind's stub generator supports :ref:`pattern files <pattern_files>` containing custom stub replacement rules. This catch-all solution addresses the needs of advanced binding projects, for which the above list of features may still not be sufficient.

Most importantly, it was possible to support these improvements with minimal changes to the core parts of nanobind.

These release breaks API and ABI compatibility, requiring a new major version according to SemVer. The following changes are noteworthy:

  • The :cpp:class:`nb::enum_\<T\>() <enum_>` binding declaration is now a wrapper that creates either a :py:class:`enum.Enum` or :py:class:`enum.IntEnum`-derived type. Previously, nanobind relied on a custom enumeration base class that was a frequent source of friction for users.

    This change may break code that casts entries to integers, which now only works for arithmetic (:py:class:`enum.IntEnum`-derived) enumerations. Replace int(my_enum_entry) with my_enum_entry.value to work around the issue.

  • The :cpp:func:`nb::bind_vector\<T\>() <bind_vector>` and :cpp:func:`nb::bind_map\<T\>() <bind_map>` interfaces were found to be severely flawed since element access (__getitem__) created views into the internal state of the STL type that were not stable across subsequent modifications.

    This could lead to unexpected changes to array elements and undefined behavior when the underlying storage was reallocated (i.e., use-after-free).

    nanobind 2.0.0 improves these types so that they are safe to use, but this means that element access must now copy by default, potentially making them less convenient. The documentation of :cpp:func:`nb::bind_vector\<T\>() <bind_vector>` discusses the issue at length and presents alternative solutions.

  • The functions :cpp:func:`nb::make_iterator() <make_iterator>`, :cpp:func:`nb::make_value_iterator() <make_value_iterator>` and :cpp:func:`nb::make_key_iterator() <make_key_iterator>` suffer from the same issue as :cpp:func:`nb::bind_vector() <bind_vector>` explained above.

    nanobind 2.0.0 improves these operations so that they are safe to use, but this means that iterator access must now copy by default, potentially making them less convenient. The documentation of :cpp:func:`nb::make_iterator() <make_iterator>` discusses the issue and presents alternative solutions.

  • The nb::raw_doc annotation was found to be too inflexible and was removed in this version.

  • The nb::typed wrapper listed above actually already existed in previous nanobind versions but was awkward to use, as it required the user to provide a custom type formatter. This release makes the interface more convenient.

  • The nb::any placeholder to specify an unconstrained :cpp:class:`nb::ndarray <ndarray>` axis was removed. This name was given to a new wrapper type :cpp:class:`nb::any` indicating typing.Any-typed values.

    All use of nb::any in existing code must be replaced with -1 (for example, nb::shape<3, nb::any, 4> → nb::shape<3, -1, 4>).

  • :ref:`Keyword-only arguments <kw_only>` are now supported, and can be indicated using the new :cpp:struct:`nb::kw_only() <kw_only>` function annotation. (PR #448).

  • nanobind classes now permit overriding __new__, in order to support C++ singletons, caches, and other types that expose factory functions rather than ordinary constructors. Read the section on :ref:`customizing Python object creation <custom_new>` for more details. (PR #473).

  • When binding methods on a class T, nanobind will now produce a Python function that expects a self argument of type T. Previously, it would use the type of the member pointer to determine the Python function signature, which could be a base of T, which would create problems if nanobind did not know about that base. (PR #471).

  • nanobind can now handle keyword arguments that are not interned, which avoids spurious TypeError exceptions in constructs like fn(**pickle.loads(...)). The speed of normal function calls (which generally do have interned keyword arguments) should be unaffected. (PR #469).

  • The owner=nb::handle() default value of the :cpp:class:`nb::ndarray <ndarray>` constructor was removed since it was bug-prone. You now have to specify the owner explicitly. The previous default (nb::handle()) continues to be a valid argument.

  • There have been some changes to the API for type casters in order to avoid undefined behavior in certain cases. (PR #549).

    • Type casters that implement custom cast operators must now define a member function template can_cast<T>(), which returns false if operator cast_t<T>() would raise an exception and true otherwise. can_cast<T>() will be called only after a successful call to from_python(), and might not be called at all if the caller of operator cast_t<T>() can cope with a raised exception. (Users of the NB_TYPE_CASTER() convenience macro need not worry about this; it produces cast operators that never raise exceptions, and therefore provides a can_cast<T>() that always returns true.)
    • Many type casters for container types (std::vector<T>, std::optional<T>, etc) implement their from_python() methods by delegating to another, "inner" type caster (T in these examples) that is allocated on the stack inside from_python(). Container casters implemented in this way should make two changes in order to take advantage of the new safety features:
      • Wrap your flags (received as an argument of the outer caster's from_python method) in flags_for_local_caster<T>() before passing them to inner_caster.from_python(). This allows nanobind to prevent some casts that would produce dangling pointers or references.
      • If inner_caster.from_python() succeeds, then also verify inner_caster.template can_cast<T>() before you execute inner_caster.operator cast_t<T>(). A failure of can_cast() should be treated the same as a failure of from_python(). This avoids the possibility of an exception being raised through the noexcept load_python() method, which would crash the interpreter.

    The previous cast_flags::none_disallowed flag has been removed; it existed to avoid one particular source of exceptions from a cast operator, but can_cast<T>() now handles that problem more generally.

  • ABI version 14.

Footnote

[1]The author of this library had somewhat of a revelation after switching to a new editor and experiencing the benefits of interactive Python code completion and type checking for the first time. This experience also showed how nanobind-based extension were previously a second-class citizen in this typed world, prompting the changes in this release.

Version 1.9.2 (Feb 23, 2024)

Version 1.9.0-1.9.1 (Feb 18, 2024)

Releases withdrawn because of a regression. The associated changes are listed above in the 1.9.2 release notes.

Version 1.8.0 (Nov 2, 2023)

  • nanobind now considers two C++ std::type_info instances to be equal when their mangled names match. The previously used pointer comparison was fast but fragile and often caused multi-part extensions to not recognize each other's types. This version introduces a two-level caching scheme (search by pointer, then by name) to fix such problems once and for all, while avoiding the cost of constantly comparing very long mangled names. (commit b515b1).
  • Fixed casting of complex-valued constant :cpp:class:`nb::ndarray\<T\> <ndarray>` instances. (PR #338, commit ba8c7f).
  • Added a type caster for std::nullopt_t (PR #350).
  • Added the missing C++ → Python portion of the type caster for Eigen::Ref<..> (PR #334).
  • Minor fixes and improvements.
  • ABI version 12.

Version 1.7.0 (Oct 19, 2023)

New features

Bugfixes

  • The behavior of the :cpp:class:`nb::keep_alive\<Nurse, Patient\> <keep_alive>` function binding annotation was changed as follows: when the function call requires the implicit conversion of an argument, the lifetime constraint now applies to the newly produced argument instead of the original object. The change was rolled into a minor release since the former behavior is arguably undesirable and dangerous. (commit 9d4b2e).
  • STL type casters previously raised an exception when casting a Python container containing a None element into a C++ container that was not able to represent nullptr (e.g., std::vector<T> instead of std::vector<T*>). However, this exception was raised in a context where exceptions were not allowed, causing the process to be abort()-ed, which is very bad. This issue is now fixed, and such conversions are refused. (PR #318, commits d1ad3b and 5f25ae).
  • The STL sequence casters (std::vector<T>, etc.) now refuse to unpack str and bytes objects analogous to pybind11. (commit 7e4a88).

Version 1.6.2 (Oct 3, 2023)

  • Added a missing include file used by the new intrusive reference counting sample implementation from v1.6.0. (commit 31d115).

Version 1.6.1 (Oct 2, 2023)

  • Added missing namespace declaration to the :cpp:class:`ref` intrusive reference counting RAII helper class added in version 1.6.0. (commit 3ba352).

Version 1.6.0 (Oct 2, 2023)

New features

Bugfixes

  • Fixed a serious issue involving combinations of bound types (e.g., T) and type casters (e.g., std::vector<T>), where nanobind was too aggressive in its use of move semantics. Calling a bound function from Python taking such a list (e.g., f([t1, t2, ..])) would destruct t1, t2, .. if the type T exposed a move constructor, which is highly non-intuitive and no longer happens as of this fix.

    Further investigation also revealed inefficiencies in the previous implementation where moves were actually possible but not done (e.g., for functions taking an STL vector by value). Some binding projects may see speedups as a consequence of this change. (issue #307, commit 122015).

Version 1.5.2 (Aug 24, 2023)

Version 1.5.1 (Aug 23, 2023)

  • Fixed serious reference counting issue introduced in nanobind version 1.5.0, which affected the functions :cpp:func:`python_error::traceback()` and :cpp:func:`python_error::what()`, causing undefined behavior via use-after-free. Also addressed an unrelated minor UB sanitizer warning. (issue #277, commits 30d30c and c48b18).
  • Extended the internal data structure tag so that it isolates different MSVC versions from each other (they are often not ABI compatible, see pybind11 issue #4779). This means that nanobind 1.5.1 effectively bumps the ABI version to "10.5" when compiling for MSVC, and the internals will be isolated from extensions built with nanobind v1.5.0 or older. (commit c7f3cd).
  • Incorporated fixes so that nanobind works with PyPy 3.10. (commits fb5508 and 2ed10a).
  • Fixed type caster for std::vector<bool>. (PR #256).
  • Fixed compilation in debug mode on MSVC. (PR #253).

Version 1.5.0 (Aug 7, 2023)

Version 1.4.0 (June 8, 2023)

  • Improved the efficiency of the function dispatch loop. (PR #227).
  • Significant improvements to the Eigen type casters (generalized stride handling to avoid unnecessary copies, support for conversion via nb::cast(), many refinements to the Eigen::Ref<T> interface). (PR #215).
  • Added a NB_DOMAIN parameter to :cmake:command:`nanobind_add_module` which can isolate extensions from each other to avoid binding clashes. See the associated :ref:`FAQ entry <type-visibility>` for details. (commit 977119).
  • Reduced the severity of nanobind encountering a duplicate type binding (commits f3b0e6, and 2c9124).
  • Support for pickling/unpickling nanobind objects. (commit 59843e).
  • ABI version 9.

Version 1.3.2 (June 2, 2023)

  • Fixed compilation on 32 bit processors (only i686 tested so far). (PR #224).
  • Fixed compilation on PyPy 3.8. (commit cd8135).
  • Reduced binary bloat of musllinux wheels. (commit f52513).

Version 1.3.1 (May 31, 2023)

  • CMake build system improvements for stable ABI wheel generation. (PR #222).

Version 1.3.0 (May 31, 2023)

This is a big release. The sections below cover added features, efficiency improvements, and miscellaneous fixes and improvements.

New features

Efficiency improvements:

  • Reduced the per-instance overhead of nanobind by 1 pointer and simplified the internal hash table types to crunch libnanobind. (commit de018d).
  • Supplemental type data specified via :cpp:class:`nb::supplement\<T\>() <supplement>` is now stored directly within the type object instead of being referenced through an indirection. (commit d82ca9).
  • Reduced the number of exception-related exports to further crunch libnanobind. (commit 763962).
  • Reduced the size of nanobind type objects by 5 pointers. (PR #194, #195, and commit d82ca9).
  • Internal nanobind types (nb_type, nb_static_property, nb_ndarray) are now constructed on demand. This reduces the size of the libnanobind component in static (NB_STATIC) builds when those features are not used. (commits 95e45a, 375083, and e033c8).
  • Added a small function cache to improve code generation in limited API builds. (commit f0f4aa).
  • Refined compiler and linker flags across platforms to ensure compact binaries especially in NB_STATIC builds. (commit 5ead9f)
  • nanobind enums now take advantage of :ref:`supplemental data <supplement>` to improve the speed of object and name lookups. Note that this prevents use of nb::supplement<T>() with enums for other purposes. (PR #195).

Miscellaneous fixes and improvements

  • Use the new PEP-697 interface to access data in type objects when compiling stable ABI3 wheels. This improves forward compatibility (the Python team may at some point significantly refactor the layout and internals of type objects). (PR #211):
  • Added introspection attributes __self__ and __func__ to nanobind bound methods, to make them more like regular Python bound methods. Fixed a bug where some_obj.method.__call__() would behave differently than some_obj.method(). (PR #216).
  • Updated the implementation of :cpp:class:`nb::enum_ <enum_>` so it does not take advantage of any private nanobind type details. As a side effect, the construct nb::class_<T>(..., nb::is_enum(...)) is no longer permitted; use nb::enum_<T>(...) instead. (PR #195).
  • Added the :cpp:class:`nb::type_slots_callback` class binding annotation, similar to :cpp:class:`nb::type_slots` but allowing more dynamic choices. (PR #195).
  • nanobind type objects now treat attributes specially whose names begin with @. These attributes can be set once, but not rebound or deleted. This safeguard allows a borrowed reference to the attribute value to be safely stashed in the type supplement, allowing arbitrary Python data associated with the type to be accessed without a dictionary lookup while keeping this data visible to the garbage collector. (PR #195).
  • Fixed surprising behavior in enumeration comparisons and arithmetic (PR #207):
    • Enum equality comparisons (== and !=) now can only be true if both operands have the same enum type, or if one is an enum and the other is an int. This resolves some confusing results and ensures that enumerators of different types have a distinct identity, which is important if they're being put into the same set or used as keys in the same dictionary. All of the following were previously true but will now evaluate as false:
      • FooEnum(1) == BarEnum(1)
      • FooEnum(1) == 1.2
      • FooEnum(1) == "1"
    • Enum ordering comparisons (<, <=, >=, >) and arithmetic operations (when using the :cpp:struct:`is_arithmetic` annotation) now require that any non-enum operand be a Python number (an object that defines __int__, __float__, and/or __index__) and will avoid truncating non-integer operands to integers. Note that unlike with equality comparisons, ordering and arithmetic operations do still permit two operands that are enums of different types. Some examples of changed behavior:
      • FooEnum(1) < 1.2 is now true (used to be false)
      • FooEnum(2) * 1.5 is now 3.0 (used to be 2)
      • FooEnum(3) - "2" now raises an exception (used to be 1)
    • Enum comparisons and arithmetic operations with unsupported types now return NotImplemented rather than raising an exception. This means equality comparisons such as some_enum == None will return unequal rather than failing; order comparisons such as some_enum < None will still fail, but now with a more informative error.
  • ABI version 8.

Version 1.2.0 (April 24, 2023)

  • Improvements to the internal C++ → Python instance map data structure to improve performance and address type confusion when returning previously registered instances. (commit 716354, discussion 189).
  • Added up-to-date nanobind benchmarks on Linux including comparisons to Cython. (commit 834cf3 and 39e163).
  • Removed the superfluous nb_enum metaclass. (commit 9c1985).
  • Fixed a corner case that prevented nb::cast<char> from working. (commit 9ae320).

Version 1.1.1 (April 6, 2023)

Version 1.1.0 (April 5, 2023)

Version 1.0.0 (March 28, 2023)

  • Nanobind now has a logo. (commit b65d31).
  • Fixed a subtle issue involving function/method properties and the IPython command line interface. (PR #151).
  • Added a boolean type to the :cpp:class:`nb::ndarray\<..\> <ndarray>` interface. (PR #150).
  • Minor fixes and improvements.

Version 0.3.1 (March 8, 2023)

  • Added a type caster for std::filesystem::path. (PR #138 and commit 0b05cd).
  • Fixed technical issues involving implicit conversions (commits 022935 and 5aefe3) and construction of type hierarchies with custom garbage collection hooks (commit 022935).
  • Re-enabled the 'chained fixups' linker optimization for recent macOS deployment targets. (commit 2f29ec).

Version 0.3.0 (March 8, 2023)

  • Botched release, replaced by 0.3.1 on the same day.

Version 0.2.0 (March 3, 2023)

Version 0.1.0 (January 3, 2023)

  • Allow nanobind methods on non-nanobind) classes. (PR #104).
  • Fix dangling tp_members pointer in type initialization. (PR #99).
  • Added a runtime setting to suppress leak warnings. (PR #109).
  • Added the ability to hash nb::enum_<..> instances (PR #106).
  • Fixed the signature of nb::enum_<..>::export_values(). (commit 714d17).
  • Double-check GIL status when performing reference counting operations in debug mode. (commit a1b245).
  • Fixed a reference leak that occurred when module initialization fails. (commit adfa9e).
  • Improved robustness of nb::tensor<..> caster. (commit 633672).
  • Upgraded the internally used tsl::robin_map<> hash table to address a rare overflow issue discovered in this codebase. (commit 3b81b1).
  • Various minor fixes and improvements.
  • ABI version 6.

Version 0.0.9 (Nov 23, 2022)

  • PyPy 7.3.10 or newer is now supported subject to certain limitations. (commits f935f93 and b343bbd).
  • Three changes that reduce the binary size and improve runtime performance of binding libraries. (commits 07b4e1fc, 9a803796, and cba4d285).
  • Fixed a reference leak in python_error::what() (commit 61393ad).
  • Adopted a new policy for function type annotations. (commit c855c90).
  • Improved the effectiveness of link-time-optimization when building extension modules with the NB_STATIC flag. This leads to smaller binaries. (commit f64d2b9).
  • Nanobind now relies on standard mechanisms to inherit the tp_traverse and tp_clear type slots instead of trying to reimplement the underlying CPython logic (commit efa09a6b).
  • Moved nanobind internal data structures from builtins to Python interpreter state dictionary. (issue #96, commit ca23da7).
  • Various minor fixes and improvements.

Version 0.0.8 (Oct 27, 2022)

  • Caster for std::array<..>. (commit be34b16).
  • Caster for std::set<..> and std::unordered_set (PR #87).
  • Ported nb::make[_key_,_value]_iterator() from pybind11. (commit 34d0be1).
  • Caster for untyped void * pointers. (commit 6455fff).
  • Exploit move constructors in nb::class_<T>::def_readwrite() and nb::class_<T>::def_readwrite_static() (PR #94).
  • Redesign of the std::function<> caster to enable cyclic garbage collector traversal through inter-language callbacks (PR #95).
  • New interface for specifying custom type slots during Python type construction. (commit 38ba18a).
  • Fixed potential undefined behavior related to nb_func garbage collection by Python's cyclic garbage collector. (commit 662e1b9).
  • Added a workaround for spurious reference leak warnings caused by other extension modules in conjunction with typing.py (commit 5e11e80).
  • Various minor fixes and improvements.
  • ABI version 5.

Version 0.0.7 (Oct 14, 2022)

  • Fixed a regression involving function docstrings in pydoc. (commit 384f4a).

Version 0.0.6 (Oct 14, 2022)

  • Fixed undefined behavior that could lead to crashes when nanobind types were freed. (commit 39266e).
  • Refactored nanobind so that it works with Py_LIMITED_API (PR #37).
  • Dynamic instance attributes (PR #38).
  • Intrusive pointer support (PR #43).
  • Byte string support (PR #62).
  • Casters for std::variant<..> and std::optional<..> (PR #67).
  • Casters for std::map<..> and std::unordered_map<..> (PR #73).
  • Caster for std::string_view<..> (PR #68).
  • Custom exception support (commit 41b7da).
  • Register nanobind functions with Python's cyclic garbage collector (PR #86).
  • Various minor fixes and improvements.
  • ABI version 3.

Version 0.0.5 (May 13, 2022)

  • Enumeration export.
  • Implicit number conversion for numpy scalars.
  • Various minor fixes and improvements.

Version 0.0.4 (May 13, 2022)

  • Botched release, replaced by 0.0.5 on the same day.

Version 0.0.3 (Apr 14, 2022)

  • DLPack support.
  • Iterators for various Python type wrappers.
  • Low-level interface to instance creation.
  • Docstring generation improvements.
  • Various minor fixes and improvements.

Version 0.0.2 (Mar 10, 2022)

  • Initial release of the nanobind codebase.
  • ABI version 1.

Version 0.0.1 (Feb 21, 2022)

  • Placeholder package on PyPI.