Source ownership¶
treams-rs has three layers. The Rust crate treams-core computes every value
and every derivative. The bindings crate treams-py builds the extension
module treams_rs._native: it converts NumPy arrays and holds the data that a
gradient needs. The Python package treams_rs gives the results their physical
meaning: bases, media, polarization conventions and named results. The
design pages explain why.
Ownership table¶
Each row names the files that change together. Paths are relative to
crates/treams-core/src/, crates/treams-py/src/, python/treams_rs/ and the
repository root. Run the Python checks as uv run --no-sync pytest ... after
just build-ext.
Rust module (treams-core) |
Binding file (treams-py) |
Python module | Tests | Focused check |
|---|---|---|---|---|
fpenv, numerics |
lib.rs, context.rs, convert.rs, broadcast.rs, args.rs, ufunc/ |
_native.pyi |
tests/bindings/ |
pytest tests/bindings; cargo test -p treams-core -- fpenv numerics |
linalg |
linalg.rs |
diff.py (solve, eig, svdvals) |
tests/linalg/, properties/linalg.rs |
pytest tests/linalg; cargo test -p treams-core -- linalg |
special |
special.rs, integrals.rs, coordinates.rs, ufunc/registry.rs |
special.py |
tests/special/, properties/special.rs |
pytest tests/special; cargo test -p treams-core -- special |
lattice |
lattice.rs |
lattice.py, _lattice.py, _periodic.py, misc.py |
tests/lattice/, properties/lattice/ |
pytest tests/lattice; cargo test -p treams-core -- lattice |
sw, cw, basis, rotation, vectorwaves, fields |
translation.rs, expansion.rs, rotation.rs, vectorwaves.rs, fields.rs |
sw.py, cw.py, _bases.py, _modes.py, _waves.py, _fields.py |
tests/waves/, properties/waves.rs |
pytest tests/waves; cargo test -p treams-core -- sw:: cw:: vectorwaves fields properties::waves |
pw, channels |
plane.rs, channels.rs |
pw.py, _bases.py (PlaneWaveBasis, PlaneWavePorts), _waves.py (plane waves) |
tests/plane/, properties/plane.rs |
pytest tests/plane; cargo test -p treams-core -- channels properties::plane |
coeffs, tmatrix, ebcm |
coeffs.rs, tmatrix.rs, ebcm.rs |
coeffs.py, ebcm.py, _material.py, _tmatrix.py |
tests/tmatrix/, properties/tmatrix.rs |
pytest tests/tmatrix; cargo test -p treams-core -- coeffs ebcm properties::tmatrix |
cluster |
cluster.rs, iterative.rs |
_cluster.py, iterative.py |
tests/tmatrix/, properties/tmatrix.rs |
pytest tests/tmatrix/test_illumination.py tests/tmatrix/test_iterative.py tests/tmatrix/test_particle_cluster.py; cargo test -p treams-core -- cluster:: properties::tmatrix |
smatrix |
smatrix.rs |
_smatrix.py, coeffs.py (fresnel) |
tests/smatrix/, properties/smatrix.rs |
pytest tests/smatrix; cargo test -p treams-core -- smatrix |
| - | - | _array.py, _operators.py, _operator_objects.py, operators.py, _polarization.py, _results.py, _validation.py |
tests/api/, tests/waves/test_polarization.py |
pytest tests/api/test_operators.py tests/api/test_matrix_methods.py tests/api/test_physics_api.py |
| - | every record and context | diff.py, _records.py, testing.py |
tests/autodiff/test_testing.py, tests/autodiff/test_adjoint_identities.py, the gradients tests of every domain |
pytest -m gradients |
| - | - | _dispatch.py, _promotion.py, _autodiff_functions.py, advect.py, jax.py, torch.py, autograd.py, _framework*.py |
tests/autodiff/ |
pytest tests/autodiff |
| - | - | __init__.py, __main__.py, _catalog.py, _upstream.py |
tests/api/test_support_catalog.py, tests/api/test_namespaces.py, tests/api/test_docs.py |
pytest tests/api/test_support_catalog.py tests/api/test_namespaces.py; just docs-check |
| - | - | io.py |
tests/api/test_io.py |
pytest tests/api/test_io.py |
test_support |
testing.rs (*_jet hooks) |
- | tests/_support.py, tests/_scripts.py, tests/test_suite_rules.py |
pytest tests/test_suite_rules.py |
The crate docs
and the design module table pair each Rust module
with its treams_rs and treams namespaces. Testing describes the
test directories.
After a change to Rust or to the bindings, run just build-ext, then the
module's Rust tests and its Python test directory. The Python tests check what
the Rust tests cannot: broadcasting, memory layouts, argument checks and
complete workflows through the public API.
Rules¶
Use these rules for new and changed code. They describe the intended contracts; the checks and open gaps below distinguish what is enforced today. The core's conventions define names, numerical conventions and derivative state in more detail.
- Keep formulas in the core. Rust owns numerical values, analytic
pushforwards and pullbacks. Bindings and frameworks adapt ownership, arrays
and execution. Finite differences belong only in tests and
treams_rs.testing. - Follow the dependency layers. Production code uses its own or lower layers in the core's module table; tests may cross layers. Keep one numerical core crate until measured build or distribution needs justify a split.
- Establish invariants at boundaries. Validate public inputs and restored state before indexing, allocating or computing. Prefer constructors and private fields that preserve validated domains over repeated checks or combinations of policy booleans. Bindings still validate their array shapes and Python still provides messages in treams terms; those are distinct duties.
- Own only useful derivative state. A residual keeps what later derivative
calls need, with private invariant-bearing fields. Both
pushforward(&self, …)andpullback(&self, …)borrow it for repeated calls at fixed primal inputs. Returned gradient records may be plain data. Consume an owned argument when its caller no longer needs it, aslinalg::solve_owneddoes, and reuse its storage where possible; do not clone merely to fit an interface. - Keep saved state coherent. Shape-derived sizes, writing and restoration
must agree. Restore validates lengths, tags, domains and allocation sizes;
round trips preserve derivatives and deferred errors, including zero
directions. These bytes are private invocation state, not a persistent file
format. A mechanical refactor preserves the layout; changing a payload or
padding requires an explicit compatibility decision and matching
state_spec. - Share concrete work. Use the smallest helper or closure that removes actual duplicated assembly or numerical work. Avoid a generic residual hierarchy, speculative extension points or a second framework implementation of a core derivative.
- Branch on error identity. Use a typed reason when code branches on an error or encodes it; display wording must not decide behavior. Numerical fallback catches only the failures it can repair and propagates resource and programming errors. Document which failures leave partial output.
- Specify parallel arithmetic. Route parallel work through the owned
threadspool. Define reduction chunks and accumulation order independently of the worker count. Wrap Python entry-point bodies infpenv::ieee, and preserve the caller's floating-point mode, including failure and single-thread paths. Benchmark before changing thresholds or regrouping sums. - Make unsafe obligations local. Keep unsafe code in the existing allowed
modules, with minimal blocks and
SAFETYcomments. Express operand lifetimes, read/write ownership, aliasing and dtype layout in types where possible. Contain panics before they can cross a C ABI boundary; do not disguise them as numerical failures. - State numerical and resource limits. Reserve advertised large outputs
and workspaces fallibly with the existing
numericshelpers. Define finite output and accuracy contracts, check both value and derivative paths, and return an appropriate error when a result cannot meet them. This is not a promise that every internal allocation is fallible. - Require evidence. Start a numerical repair with a reproducible failing case and an independent reference or physical identity. Check adjoint pairings separately from finite differences, cover both sides of thresholds, and record measured reasons for tuning constants. Never loosen a tolerance to hide a failure. Measure proposed caches, allocation tradeoffs and numerical regrouping before adopting them; preserve archived measurements under the evidence policy.
One source and its checks¶
Each formula, layout, capability and shared limit has one owner. Generate other representations from it, or check their agreement when generation would obscure the code. Do not add a second hand-maintained registry to make a check pass.
| Contract | Existing example or check | Open work |
|---|---|---|
| Core layers and saved state | SavedState, linalg codecs, test_native_contexts.py |
#30: codec composition, fallible writer and Rust-only layer/codec checks; #27: deferred metric errors |
| Validated domains and shared label bounds | BlochLattice validates construction |
#32: basis/residual invariants and Python/Rust bound agreement |
| Error identity, finite outputs and allocation | Error, numerics::memory |
#33: typed reasons and deterministic errors |
| Owned pool, IEEE mode and reduction order | test_thread_pool.py, test_float_environment.py, parallel |
#11, #35: one-thread, failure, foreign-pool and extraction paths; #34: ufunc ownership and reduction aliasing |
| Native names, contexts and ufunc signatures | _native.pyi checked against the module by test_native_contexts.py and test_ufunc_contract.py |
Extend these checks with each binding |
| Public API and page inventory | support_catalog() reads modules/docstrings; mkdocs.yml owns page order. just docs generates reference pages, tables and llms.txt; just docs-check checks them |
Never edit generated output by hand |
| Shared metadata and reference evidence | generate_references.py, test_support |
#40: coverage, determinism/default metadata agreement and fixture provenance enforcement |
The ordinary Python API selects an optional adapter in _dispatch.py before
array coercion. _promotion.py converts constant physics objects,
_autodiff_functions.py preserves NumPy callable contracts, and _framework*.py
objects compose records; adapters supply the framework bridge. Keep this map
current as part of #9, which owns
package restructuring and import checks.
Every public rename gets a bullet under "Unreleased" in CHANGELOG.md. A name
that differs from treams also gets an entry in _upstream.py, which owns the
name map and the treams-name AttributeError.
Adding a binding¶
The crate docs of treams-py, in
crates/treams-py/src/lib.rs, hold the
checklist under "Adding a binding". Start with the core function and its tests.
Add the binding, context, pushforward and pullback to the file for that core module.
Then add the export, stub, Python caller and a CASES entry in
tests/bindings/test_native_contexts.py (or a REGISTRY_ROWS row for a ufunc).
Finish with just docs.
Adding a physics feature¶
For a new function X that computes a value:
- Rust. Write
Xin the relevanttreams-coremodule. If it supports gradients, it returns(value, XResidual), where the residual holds what both derivative directions need. When the pullback reads the value, it returns onlyXResidualwith avalue()accessor, ascoeffs::miedoes (see 'Names' in the crate docs). Its pullback,XResidual::pullback(&self, cotangent), turns the cotangent (the gradient of a real loss with respect to the value) into the input gradientsXGradient(glossary). Add the pushforward with the matching input tangents as well. Test the implementation in the module's inlinetestsmodule and the physics inproperties/<domain>.rs. - Binding. Follow the checklist above.
- Python. Add the record, a function that returns the value and a context
for its gradients, to
diff.py:diff.x(...) -> (value, XContext). Then add the upstream-style function of its namespace or the method of the physics object. Check label bounds and argument types in Python, with messages in treams terms. - Framework adapters. If a physics object exposes the feature, route it
through
_framework_*.py, so that Advect, JAX, PyTorch and HIPS Autograd differentiate it. - Tests. Add tests to
tests/<domain>/: a comparison with the pinned treams version or another reference, a physical identity, andcheck_pushforward/check_pullbackfor the derivatives (adding a test). - Docs. Write the docstring with units, shapes and conventions, add a
runnable example to the matching guide page, update
capabilities, run
just docsand add aCHANGELOG.mdbullet.
Naming rules¶
The glossary gives one name per concept across Rust,
the bindings, Python and treams. The crate docs state the rules for each
layer: "Names" in the
treams-core docs
and "Naming rules" in
crates/treams-py/src/lib.rs. The rules
that cross layers:
- Residuals and contexts. The Rust function
XreturnsXResidual, and its pullback returnsXGradient. The native context of the recorddiff.xis<X>Context, the diff name in CamelCase:diff.smatrix_trreturns anSMatrixTrContext. - Value-only functions. A function that returns the value of a record
without a context ends in
_value:smatrix::tr_valuein Rust,smatrix_tr_valuein the bindings. - treams counterparts. The doc of a Rust item that mirrors treams ends with
Upstream: `treams.sw.translate`., followed byDifferences: ...where it deviates. A Python docstring saysMirrors ``treams.sw.translate``.and names its differences. A module without a counterpart says "treams-rs extension". - Labels. The degree is
land the order ism; radial kinds are "regular" and "singular"; two-basis functions takedestinationandsource; expansion centres arepositions.