Repository navigation
SparseVector keeps duplicate coordinates from a SciPy sparse array #162
Description
After fd74f78, a SparseVector from a coo_array disagrees with itself. to_coo() returns the SciPy answer. to_list() and to_numpy() return a different vector, from the same object. _from_sparse copies duplicate coordinates through one for one.
$ PYTHONPATH=/var/tmp/p11L1/pgv python pgv_sparse_probe.py # master fd74f78, scipy 1.18.1, numpy 2.5.3
# input: coo_array(([1.0, 2.0], ([0, 0], [1, 1])), shape=(1, 4))
sv.to_coo().toarray() : [[0.0, 3.0, 0.0, 0.0]]
sv.to_list() : [0.0, 2.0, 0.0, 0.0]
sv.to_numpy() : [0.0, 2.0, 0.0, 0.0]
sv.to_text() : {2:1.0,2:2.0}/4
SparseVector(sv.to_coo()) == sv : True
# input: coo_array(([1.0, 0.0], ([0, 0], [0, 2])), shape=(1, 4))
to_binary nnz, _from_sparse : 2
to_binary nnz, _from_dense : 1
to_binary nnz, _from_dict : 1
Postgres refuses both inputs. CheckIndex (src/sparsevec.c:129) rejects the duplicate on the text path and the binary path alike. sparsevec_in drops the explicit zero (:322) and sparsevec_recv refuses it (:549). pgvector/asyncpg/register.py registers sparsevec with format='binary' and no text codec.
# docker.io/pgvector/pgvector:pg17, pgvector 0.8.7 on PostgreSQL 17, table t (v sparsevec(4))
postgres=# SELECT '{2:1.0,2:2.0}/4'::sparsevec;
ERROR: sparsevec indices must not contain duplicates
postgres=# SELECT '{1:1.0,3:0.0}/4'::sparsevec;
{1:1}/4
# dup.bin and zero.bin wrap each to_binary() payload in a one-row PGCOPY frame
$ psql -c 'COPY t FROM STDIN WITH (FORMAT binary)' < dup.bin
ERROR: sparsevec indices must not contain duplicates
CONTEXT: COPY t, line 1, column v
$ psql -c 'COPY t FROM STDIN WITH (FORMAT binary)' < zero.bin
ERROR: binary representation of sparsevec cannot contain zero values
CONTEXT: COPY t, line 1, column v
README.md:777 builds the vector from a coo_array triplet. The SciPy docstring for that constructor says it "permits duplicate entries". The same triplet through csr_array arrives at nnz 1, so coo is the one input _from_sparse ever sees non-canonical.
#160 took the ordering and left this half open, with no reproduction to judge it on. tocoo(copy=False) returns the array the caller passed, so an in-place sum_duplicates() would rewrite it. @ankane, should _from_sparse fold duplicates and drop zeros while it builds elements, or raise instead?
Activity
ankane commentedon Oct 4, 2026
Hi @chrikrah, thanks for reporting. Fixed in cd18c3f and the commit above.