Skip to content

acc: remove engine-specific logic from the acceptance harness - #6938

Merged
denik merged 3 commits into
mainfrom
denik/tf-cleanup-acc-harness
Oct 6, 2026
Merged

denik merged 3 commits into
mainfrom
denik/tf-cleanup-acc-harness

Conversation

@denik

@denik denik commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Changes

No test varies DATABRICKS_BUNDLE_ENGINE anymore, so remove what the harness and tests still do for the terraform engine:

  • Subset selection hashes over all variants (no engine grouping / 10x direct weighting); drop the empty-engine-matrix validation, CHECKPOINT_DISABLE, and the DBR runner's engine ENVFILTER.
  • read_state.py/print_state.py: drop the terraform branch; print_state.py decides from the state file, not the engine env var.
  • Make engine-conditional assertions unconditional, drop the direct-vs-terraform permissions comparison and direct_drymigrate telemetry filtering, and the terraform-only exclusions in empty_string_dropped (fixture regenerated).
  • Update testing rules describing the two-engine matrix.

This pull request and its description were written by Isaac.

@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: aa27dad

Run: 37468237849

Env 🔄​flaky ✅​pass 🙈​skip Time
✅​ aws linux 284 27 11:40
🔄​ aws windows 3 283 25 14:24
🔄​ azure linux 3 280 27 13:38
✅​ azure windows 285 25 6:49
✅​ gcp linux 284 27 11:23
🔄​ gcp windows 3 283 25 9:32
6 interesting tests: 6 flaky
Test Name aws windows azure linux gcp windows
🔄​ TestAccept 🔄​f 🔄​f 🔄​f
🔄​ TestAccept/bundle/resources/dashboards/change-serialized-dashboard 🔄​f ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/dashboards/change-serialized-dashboard/DMS=/READPLAN= ✅​p ✅​p 🔄​f
🔄​ TestAccept/bundle/resources/dashboards/change-serialized-dashboard/DMS=true/READPLAN=1 🔄​f ✅​p ✅​p
🔄​ TestAccept/bundle/resources/volumes/uppercase-name ✅​p 🔄​f ✅​p
🔄​ TestAccept/bundle/resources/volumes/uppercase-name/DMS= ✅​p 🔄​f ✅​p
Top 16 slowest tests (at least 2 minutes):
duration env testname
3:48 aws linux TestAccept
3:47 gcp linux TestAccept
3:40 aws windows TestAccept/bundle/resources/dashboards/change-serialized-dashboard/DMS=/READPLAN=
3:16 azure windows TestAccept
3:15 gcp windows TestAccept/bundle/resources/dashboards/change-serialized-dashboard/DMS=true/READPLAN=1
2:45 aws linux TestAccept/bundle/resources/dashboards/change-serialized-dashboard/DMS=/READPLAN=
2:37 azure windows TestAccept/bundle/resources/dashboards/change-serialized-dashboard/DMS=/READPLAN=1
2:19 azure linux TestFilerWorkspaceFilesExtensionsReadDir
2:14 azure linux TestAccept/bundle/resources/dashboards/change-serialized-dashboard/DMS=true/READPLAN=
2:12 azure windows TestFilerWorkspaceFilesExtensionsReadDir
2:11 gcp windows TestAccept/bundle/resources/dashboards/change-serialized-dashboard/DMS=true/READPLAN=
2:08 gcp windows TestAccept/bundle/resources/dashboards/change-serialized-dashboard/DMS=/READPLAN=
2:07 gcp windows TestAccept/bundle/resources/volumes/uppercase-name/DMS=true
2:07 azure windows TestAccept/bundle/resources/dashboards/change-serialized-dashboard/DMS=true/READPLAN=1
2:07 aws windows TestAccept/bundle/resources/dashboards/change-serialized-dashboard/DMS=true/READPLAN=
2:05 aws linux TestFilerWorkspaceFilesExtensionsReadDir

@denik
denik force-pushed the denik/tf-cleanup-acc-harness branch from 6b9349f to b69c827 Compare October 6, 2026 10:15
@denik
denik force-pushed the denik/tf-cleanup-tf-only-refs branch from 4e9d99e to 8ab9370 Compare October 6, 2026 10:15
@denik
denik changed the base branch from denik/tf-cleanup-tf-only-refs to main October 6, 2026 10:15
denik and others added 3 commits October 6, 2026 15:01
The terraform engine is gone, so no test varies DATABRICKS_BUNDLE_ENGINE anymore.

- Drop the engine-aware subset grouping and 10x direct weighting; subset selection
  now hashes over all variants.
- Drop the EnvMatrix.DATABRICKS_BUNDLE_ENGINE = [] validation, the terraform
  CHECKPOINT_DISABLE env, and the DBR runner's engine ENVFILTER.
- Gate eventual-consistency injection on INJECT_STALE_ON_DIRECT alone. This makes
  postgres_synced_tables/recreate_capped exercise the capped wait it was written
  for (its golden previously showed a successful recreate).
- read_state.py/print_state.py: drop the terraform-engine branch; print_state.py
  decides on DMS recording from the state file, not the engine env var.
- Make engine-conditional test assertions unconditional, drop the dead
  permissions direct-vs-terraform comparison, and the terraform-only exclusions in
  empty_string_dropped/gen_empty_config.py (fixture and goldens regenerated).
- Update testing rules that described the two-engine matrix.

Co-authored-by: Isaac <no-reply@databricks.com>
DryRunMigrationTelemetry is gone, so these keys are never emitted.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@denik
denik marked this pull request as ready for review October 6, 2026 13:02
@denik
denik requested review from a team as code owners October 6, 2026 13:02
@denik
denik enabled auto-merge October 6, 2026 13:02
@denik
denik force-pushed the denik/tf-cleanup-acc-harness branch from b69c827 to aa27dad Compare October 6, 2026 13:04
@github-actions github-actions Bot added DABs DABs related issues PyDABs labels Oct 6, 2026
@denik
denik added this pull request to the merge queue Oct 6, 2026
Merged via the queue into main with commit 4184b09 Oct 6, 2026
28 checks passed
@denik
denik deleted the denik/tf-cleanup-acc-harness branch October 6, 2026 14:31
andreamatt pushed a commit to andreamatt/databricks-cli that referenced this pull request Oct 9, 2026
…atabricks#6941) (databricks#6969)

Re-applies databricks#6941. databricks#6938 put the `DATABRICKS_BUNDLE_ENGINE=direct` check
back into `staleOnceEnabled`: its "leave the stale-read injection fix to
databricks#6941" commit was rebased after databricks#6941 merged and reverted it. Nothing
sets that variable any more, so the testserver's stale-read injection
was off and `postgres_synced_tables/recreate_capped` again recorded a
successful recreate instead of the capped-wait failure.

This pull request and its description were written by Isaac.

Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

DABs DABs related issues PyDABs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants