Skip to content

Overhaul of vector#184

Merged
arthurmccray merged 12 commits intoelectronmicroscopy:devfrom
cophus:vector-update
Mar 6, 2026
Merged

Overhaul of vector#184
arthurmccray merged 12 commits intoelectronmicroscopy:devfrom
cophus:vector-update

Conversation

@cophus
Copy link
Contributor

@cophus cophus commented Mar 3, 2026

What this PR does

This PR completely rewrites the Vector class.

The new implementation is substantially simpler. Public proxy types such as FieldView and CellView are removed, and all indexing/selection operations now return Vector. The explicit NumPy escape hatches are:

  • .flatten() for concatenated rowwise data
  • .array for selections containing exactly one cell

The main syntax change is field selection. Instead of:
v["kx"]
use:
v.select_fields("kx")

This makes fixed-grid indexing with [] distinct from field selection.

select_fields(...) returns a write-through Vector view, so field-selected updates still modify the parent Vector.

This PR also adds:

  • rowwise update helpers such as set_flattened(...)
  • append_rows(...) and compact()
  • expanded arithmetic support, including **, %, //, unary -, abs(...)
  • elementwise NumPy ufunc support such as np.sin(vector) and np.maximum(vector, 0)
  • Minor fix for the 2D histogram calculation function.

File I/O is also much cleaner. The internal storage no longer serializes nested per-cell containers, which previously created large Zarr metadata overhead. The new layout stores one numeric row buffer plus per-cell offsets/lengths. In practice this makes Vector saves substantially smaller and reduces read/write overhead.

What PR reviewer(s) should do

  • Run the Vector test suite and confirm downstream tests still behave as expected.
  • Run the attached notebook examples.
  • Call out any missing functionality or awkward syntax before this API is finalized.

Notebook demo for syntax:

vector_new_01.ipynb

@arthurmccray
Copy link
Collaborator

arthurmccray commented Mar 6, 2026

Okay this looks good to me. I reckon there will be some changes that need to be made as this gets used in features, but it does everything I want for now at least.

It's not the easiest to understand all of the code, but unless we start subclassing it, then I don't think that should ever be a problem. The tests we have are fairly robust I think.

I also created a tutorial at electronmicroscopy/quantem-tutorials#5

@arthurmccray arthurmccray merged commit 787585e into electronmicroscopy:dev Mar 6, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants