-
Notifications
You must be signed in to change notification settings - Fork 228
job_runs: add lifecycle.triggers.on_file_change #6309
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
60 commits
Select commit
Hold shift + click to select a range
3099afb
job_runs: add lifecycle.triggers.on_file_change
radakam a61e662
job_runs: fix exhaustruct and regenerate refschema for on_file_change
radakam 568540f
job_runs: tighten on_file_change path and directory checks
radakam fa6de6f
job_runs: store on_file_change as path-to-hash only
radakam ce24a87
job_runs: use maps.Copy in on_file_change hash merge
radakam eb34c41
job_runs: drop filepath.Glob jargon from on_file_change schema docs
radakam 3595e3a
acc: refresh on_file_change goldens for resource-key progress lines
radakam 58392c9
acc: cover on_file_change globs, errors, and appear/disappear
radakam ca1f5d4
acc: expand on_file_change glob coverage and skip ignored files
radakam 426a9f3
acc: force-add gitignored on_file_change fixture
radakam 1b3f797
acc: drop READPLAN from the on_file_change glob test
radakam 13bf6f6
acc: restore READPLAN on the on_file_change glob test
radakam 41fc58b
job_runs: address on_file_change review comments
radakam 756b446
job_runs: list every trigger field to satisfy exhaustruct
radakam c2d364d
job_runs: keep empty nested lifecycle on state and remote
radakam c2e04c9
job_runs: keep the lifecycle fingerprint out of the remote type
radakam 575f266
job_runs: store on_file_change hashes on the trigger itself
radakam 2892947
acc: refresh job_runs goldens for the omitted empty trigger
radakam 19b4448
job_runs: error when on_file_change hashes nothing
radakam ae51339
acc: fix on_file_change_errors diagnostic line numbers
radakam ad85edf
acc: removing on_file_change is unchanged, not an update
radakam 7b715fc
job_runs: resolve on_file_change relative to the defining YAML file
radakam 8e61fa1
job_runs: reject absolute on_file_change patterns
radakam 0934d61
job_runs: warn instead of erroring when on_file_change hashes nothing
radakam 8670de6
acc: stop Git Bash rewriting the on_file_change absolute pattern
radakam 94fb2e5
acc: read the absolute on_file_change pattern from a committed file
radakam f24efe8
job_runs: match on_file_change patterns against synced files and drop…
radakam 380cda9
job_runs: persist trigger fingerprints as lifecycle.triggers_state
radakam aa26223
job_runs: store triggers_state as a pointer on state
radakam 427b320
acc: keep LF when rewriting on_file_change include fixtures
radakam 3dcb9d4
job_runs: key on_file_change fingerprints by pattern
radakam 123ffa4
job_runs: aggregate file trigger fingerprints by pattern
radakam abe08c1
job_runs: fingerprint on_file_change after build and persist cleared …
radakam 85f00b7
job_runs: drop redundant on_file_change validation and unit tests
radakam d097bf9
nextchanges: conform on_file_change fragment to the bullet+PR-link fo…
denik 318fb12
acc: record the inherited DMS matrix in the new on_file_change goldens
denik 5fda712
job_runs: fingerprint on_file_change once, after the predeploy script
denik a66e358
job_runs: drop the duplicate on_file_change pre-scan
denik 2eb2468
job_runs: drop removed-trigger changes instead of persisting cleared …
denik 4f39ed8
job_runs: revert unrelated churn in type_test.go and expand the overr…
denik 4d2480b
update test
denik 0dbf58c
acc: inline plan_no_changes in the on_file_change script
denik 04e72c8
Revert "acc: inline plan_no_changes in the on_file_change script"
denik 1fdcae8
acc: commit the on_file_change create-plan golden
denik df0746e
job_runs: inline fileTriggerDiag
denik dcd6024
job_runs: reject Windows-rooted on_file_change patterns on every host
denik 6961c6a
job_runs: give JobRunTriggersState an IsEmpty method
denik 73d4865
job_runs: note why on_file_change hashes the candidate sync set
denik ea1c789
job_runs: match OverrideChangeDesc paths by string like other resources
denik 6c87810
job_runs: move JobRunLifecycle types to job_run.go
denik f894190
job_runs: regenerate pydabs and refschema for on_file_change
denik 59f5817
acc: mask plan_version in the on_file_change create-plan golden
denik b39cfed
shorten desc
denik c27a0b7
job_runs: regenerate for main rebase (DMS version_id, refschema)
denik a55df32
job_runs: drop stale AI-snapshot mention from the include comment
denik 964ccdd
job_runs: drop the prevent_destroy trigger validation, rely on the ge…
denik d2c544e
job_runs: drop stale generated-overlays mention from the fingerprint …
denik a32d777
job_runs: drop vfs.Overlay from the file-trigger hash test
denik 6391342
job_runs: regenerate pydabs docstrings after dropping the prevent_des…
denik f964f6f
job_runs: reject only genuinely absolute on_file_change patterns
denik File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| * direct: resources.job\_runs: new lifecycle.triggers.on\_file\_change setting to restart the run when monitored files change. Can be set to a series of paths or globs. ([#6309](https://github.com/databricks/cli/pull/6309)) | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
18 changes: 18 additions & 0 deletions
18
acceptance/bundle/resources/job_runs/on_file_change/databricks.yml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| bundle: | ||
| name: job-runs-on-file-change | ||
|
|
||
| resources: | ||
| jobs: | ||
| my_job: | ||
| name: my-job | ||
| tasks: | ||
| - task_key: main | ||
| notebook_task: | ||
| notebook_path: /Workspace/test | ||
|
|
||
| job_runs: | ||
| my_run: | ||
| job_id: ${resources.jobs.my_job.id} | ||
| lifecycle: | ||
| triggers: | ||
| - on_file_change: seed.txt | ||
|
radakam marked this conversation as resolved.
|
||
57 changes: 57 additions & 0 deletions
57
acceptance/bundle/resources/job_runs/on_file_change/out.plan.create.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,57 @@ | ||
| { | ||
| "plan_version": [PLAN_VERSION], | ||
| "cli_version": "[CLI_VERSION]", | ||
| "plan": { | ||
| "resources.job_runs.my_run": { | ||
| "depends_on": [ | ||
| { | ||
| "node": "resources.jobs.my_job", | ||
| "label": "${resources.jobs.my_job.id}" | ||
| } | ||
| ], | ||
| "action": "create", | ||
| "new_state": { | ||
| "value": { | ||
| "job_id": 0, | ||
| "lifecycle": { | ||
| "triggers_state": { | ||
| "on_file_change": { | ||
| "seed.txt": "[FILE_HASH]" | ||
| } | ||
| } | ||
| }, | ||
| "result_state": "SUCCESS" | ||
| }, | ||
| "vars": { | ||
| "job_id": "${resources.jobs.my_job.id}" | ||
| } | ||
| } | ||
| }, | ||
| "resources.jobs.my_job": { | ||
| "action": "create", | ||
| "new_state": { | ||
| "value": { | ||
| "deployment": { | ||
| "kind": "BUNDLE", | ||
| "metadata_file_path": "/Workspace/Users/[USERNAME]/.bundle/job-runs-on-file-change/default/state/metadata.json" | ||
| }, | ||
| "edit_mode": "UI_LOCKED", | ||
| "format": "MULTI_TASK", | ||
| "max_concurrent_runs": 1, | ||
| "name": "my-job", | ||
| "queue": { | ||
| "enabled": true | ||
| }, | ||
| "tasks": [ | ||
| { | ||
| "notebook_task": { | ||
| "notebook_path": "/Workspace/test" | ||
| }, | ||
| "task_key": "main" | ||
| } | ||
| ] | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } |
4 changes: 4 additions & 0 deletions
4
acceptance/bundle/resources/job_runs/on_file_change/out.test.toml
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.