feat: add options support to Store for format-specific write options#58
Conversation
There was a problem hiding this comment.
Clean addition — the options field is well-designed with defensive copying, empty-dict normalization, and backward-compatible wire format. PEP 249 compliance is unaffected: store was already a vendor extension kwarg on execute(), and this PR only adds a field to the vendor-specific Store dataclass. No cursor, connection, or driver signatures change.
Two suggestions worth considering as the Store surface area grows: extracting serialization into Store.to_dict() (eliminates duplication in both connection.py and tests), and eventually locking down mutability.
Pre-existing note (not caused by this PR): cursor.fetchone() / fetchall() will TypeError when store is configured, because __get_results() returns None and the callers index into it unconditionally. Worth a follow-up ticket.
Address review feedback: - Add to_dict() method on Store for WebSocket request serialization - Use store.to_dict() in Connection.__execute_sql instead of inline dict - Replace duplicated _serialize_store() helper in tests with to_dict() - Remove test_options_dict_is_mutable to avoid enshrining mutability
Summary
options: dict[str, str] | Nonefield to theStoredataclass, allowing users to pass format-specific SparkDataFrameWriteroptions (e.g.header,delimiter,ignoreNullFields) when storing query results to cloud storage.None, and the field is omitted from the WebSocket request when not set (backward compatible).Store.for_download()to accept an optionaloptionsparameter.PEP 249 (DB-API 2.0) compliance
This change does not alter the PEP 249 interface. The
storekeyword parameter oncursor.execute()was introduced upstream in #55/#56 as a vendor extension, which PEP 249 explicitly permits:The standard
execute(operation [, parameters])signature is preserved —storeis keyword-only and optional, so callers using the PEP 249 API are unaffected. This PR only adds an attribute (options) to the vendor-specificStoredataclass; no cursor, connection, or driver signatures change.Usage
Related
optionsfield on Store (merged/ready)