Skip to content

feat: support partial writes#213

Open
alespour wants to merge 26 commits into
mainfrom
feat/partial-writes
Open

feat: support partial writes#213
alespour wants to merge 26 commits into
mainfrom
feat/partial-writes

Conversation

@alespour
Copy link
Copy Markdown
Contributor

@alespour alespour commented May 18, 2026

Proposed Changes

This PR adds partial-write support in the Python client and finalizes write API selection behavior for product compatibility.

Feature highlights:

  • Adds structured partial-write errors via InfluxDBPartialWriteError with per-line details:
    • line_number
    • error_message
    • original_line
  • accept_partial defaults to true (server default behavior)
  • accept_partial=false is sent only when partial writes are explicitly disabled
  • use_v2_api routes writes between:
    • V2 API endpoint: /api/v2/write
    • V3 API endpoint: /api/v3/write_lp

Default behavior and option semantics:

  • Default writes use the V2 API endpoint (use_v2_api=true).
  • no_sync requires use_v2_api=false and is rejected otherwise.
  • accept_partial applies only to V3 endpoint writes and is not used on V2 endpoint writes.

Configuration surfaces:

  • Write options: accept_partial, use_v2_api, no_sync
  • Constructor kwargs: write_accept_partial, write_use_v2_api, write_no_sync
  • Environment variables: INFLUX_WRITE_ACCEPT_PARTIAL, INFLUX_WRITE_USE_V2_API, INFLUX_WRITE_NO_SYNC

See Partial writes for reference.

Example (partial-write handling):

try:
    client.write(lp)
except InfluxDBPartialWriteError as e:
    for line_err in e.line_errors:
        print(f"line {line_err.line_number} failed: {line_err.error_message} ({line_err.original_line})")
except InfluxDBError as e:
    print(e.message)

Example (enable V3 write options):

client = InfluxDBClient3(
    host=host,
    token=token,
    database=database,
    write_use_v2_api=False,
    write_no_sync=True,
    write_accept_partial=False,
)

client.write(lp)

Checklist

  • CHANGELOG.md updated
  • Rebased/mergeable
  • A test has been added if appropriate
  • Tests pass
  • Commit messages are conventional

@codecov
Copy link
Copy Markdown

codecov Bot commented May 18, 2026

Codecov Report

❌ Patch coverage is 99.04306% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 77.45%. Comparing base (f8a0805) to head (c3f5407).

Files with missing lines Patch % Lines
influxdb_client_3/__init__.py 95.83% 1 Missing ⚠️
influxdb_client_3/exceptions/exceptions.py 99.09% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #213      +/-   ##
==========================================
+ Coverage   76.05%   77.45%   +1.40%     
==========================================
  Files          37       37              
  Lines        2543     2710     +167     
==========================================
+ Hits         1934     2099     +165     
- Misses        609      611       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds partial-write support to the InfluxDB 3 Python client by introducing a structured partial-write exception and by finalizing write endpoint selection logic to support both the V2 compatibility endpoint (/api/v2/write) and the V3 LP endpoint (/api/v3/write_lp). It also exposes new write configuration surfaces (options, constructor kwargs, and env vars) and updates tests and documentation accordingly.

Changes:

  • Introduces InfluxDBPartialWriteError / InfluxDBPartialWriteLineError and structured parsing/formatting of partial-write responses.
  • Adds write routing and option semantics via use_v2_api, accept_partial, and no_sync (including validation and env/kwargs wiring).
  • Updates unit/integration tests and README/CHANGELOG to cover and document the new behaviors.

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
influxdb_client_3/write_client/client/write_api.py Adds default write options (accept_partial, use_v2_api), per-request option resolution, and validation before dispatch.
influxdb_client_3/write_client/service/write_service.py Routes requests to /api/v2/write vs /api/v3/write_lp, adds partial-write error raising, and improves 405 compatibility errors.
influxdb_client_3/exceptions/exceptions.py Adds partial-write detection/parsing helpers and the new structured partial-write exception types.
influxdb_client_3/exceptions/__init__.py Exports the new partial-write exception types.
influxdb_client_3/__init__.py Adds env vars + constructor kwargs for new write booleans and updates env parsing helper.
README.md Documents partial-write handling and endpoint compatibility configuration.
CHANGELOG.md Records the new feature and option semantics.
tests/test_api_client.py Expands error parsing tests to assert structured partial-write errors and fallback behavior.
tests/test_influxdb_client_3.py Updates default option assertions, env/kwargs boolean parsing tests, and adds async multi-precision list behavior test.
tests/test_influxdb_client_3_integration.py Updates integration coverage to expect InfluxDBPartialWriteError on partial write failures.
tests/test_write_local_server.py Adds/updates local-server routing tests for use_v2_api, accept_partial, and no_sync validation.
tests/test_polars.py Updates mock expectations to include new write parameters.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread influxdb_client_3/write_client/client/write_api.py
Comment thread influxdb_client_3/write_client/service/write_service.py Outdated
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Comment thread influxdb_client_3/write_client/service/write_service.py Outdated
Comment thread README.md Outdated
Comment thread tests/test_influxdb_client_3_integration.py
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 3 comments.

Comment thread influxdb_client_3/write_client/service/write_service.py Outdated
Comment thread tests/test_influxdb_client_3_integration.py
Comment thread tests/test_influxdb_client_3_integration.py
@alespour alespour marked this pull request as ready for review May 18, 2026 13:57
@alespour alespour requested a review from Copilot May 18, 2026 14:08
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Comment on lines 144 to +175
@@ -157,13 +159,40 @@ def post_write_with_http_info(self, org, bucket, body, **kwargs): # noqa: E501,
_request_timeout=local_var_params.get('_request_timeout'),
collection_formats={},
urlopen_kw=kwargs.get('urlopen_kw', None))
if local_var_params.get('async_req'):
original_get = result.get

def translated_get(timeout=None):
try:
return original_get(timeout=timeout)
except ApiException as e:
raise self._translate_write_exception(e, use_v2_api)

result.get = translated_get
return result
except ApiException as e:
no_sync = 'no_sync' in local_var_params and local_var_params['no_sync']
if no_sync and e.status == HTTPStatus.METHOD_NOT_ALLOWED:
message = "Server doesn't support write with no_sync=true " \
"(supported by InfluxDB 3 Core/Enterprise servers only)."
raise ApiException(status=0, reason=message)
raise e
raise self._translate_write_exception(e, use_v2_api)

@dburton-influxdata
Copy link
Copy Markdown

Is this Partial Write logic being considered or has it been implemented in Telegraf?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants