Merge branch 'main' into feature/DRM/simplify-java-installation-in-devcontainer

Author: davem-bis

.github/workflows/pr.yaml

@@ -457,7 +457,7 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        dbt-version: ['1.3', '1.4', '1.5', '1.6', '1.7', '1.8', '1.9', '1.10', '1.11']
+        dbt-version: ['1.3', '1.4', '1.5', '1.6', '1.7', '1.8', '1.9', '1.10', '1.11', '1.12']
     steps:
       - uses: actions/checkout@v7
       - name: Set up Python

Makefile

@@ -33,7 +33,7 @@ install-dev-dbt-%:
 	echo "Installing dbt version: $$version"; \
 	cp pyproject.toml pyproject.toml.backup; \
 	$(SED_INPLACE) 's/"pydantic>=2.0.0"/"pydantic"/g' pyproject.toml; \
-	if [ "$$version" = "1.10.0" ] || [ "$$version" = "1.11.0" ]; then \
+	if [ "$$version" = "1.10.0" ] || [ "$$version" = "1.11.0" ] || [ "$$version" = "1.12.0" ]; then \
 		echo "Applying special handling for dbt $$version"; \
 		$(SED_INPLACE) -E 's/"(dbt-core)[^"]*"/"\1~='"$$version"'"/g' pyproject.toml; \
 		$(SED_INPLACE) -E 's/"(dbt-(bigquery|duckdb|snowflake|athena-community|clickhouse|redshift|trino))[^"]*"/"\1"/g' pyproject.toml; \

docs/concepts/overview.md

@@ -52,7 +52,9 @@ Bad data is worse than no data. The best way to keep bad data out of your system
 ### [Tests](./tests.md)
 SQLMesh "tests" are similar to unit tests in software development, where the unit is a single model. SQLMesh tests validate model *code* — you specify the input data and expected output, then SQLMesh runs the test and compares the expected and actual output.
 
-SQLMesh automatically runs tests when you apply a `plan`, or you can run them on demand with the [`test` command](../reference/cli.md#test).
+SQLMesh automatically runs all unit tests when a `plan` is created. Use `--test-changed-only` to run tests only for models included in the plan (added, modified, or restated), `--skip-tests` to skip, run tests for specific models with [`sqlmesh test --select-model`](../reference/cli.md#test), or run the full suite on demand with the [`test` command](../reference/cli.md#test).
+
+Learn more in the [testing guide](../guides/testing.md).
 
 ### [Audits](./audits.md)
 In contrast to tests, SQLMesh "audits" validate the results of model code applied to your actual data.

docs/concepts/tests.md

@@ -1,6 +1,8 @@
 # Testing
 
-Testing allows you to protect your project from regression by continuously verifying that the output of each model matches your expectations. Unlike [audits](audits.md), tests are executed either on demand (for example, as part of a CI/CD job) or every time a new [plan](plans.md) is created.
+Testing allows you to protect your project from regression by continuously verifying that the output of each model matches your expectations. Unlike [audits](audits.md), tests are executed either on demand (for example, as part of a CI/CD job or via [`sqlmesh test`](../reference/cli.md#test)) or when a new [plan](plans.md) is created.
+
+By default, `sqlmesh plan` runs all unit tests. Use `--test-changed-only` to run tests only for models included in the plan (added, modified, or restated), or `--skip-tests` to run none. With both `--select-model` and `--test-changed-only`, tests run only for selected models that changed.
 
 Similar to unit testing in software development, SQLMesh evaluates the model's logic against predefined inputs and then compares the output to expected outcomes provided as part of each test.
 

docs/faq/faq.md

@@ -102,7 +102,7 @@
     SQLMesh's default behavior is appropriate for most deployments, but you can override where SQLMesh creates physical tables and views with [schema configuration options](../guides/configuration.md#environment-schemas).
 
 ??? question "What's the difference between a `test` and an `audit`?"
-    A SQLMesh [`test`](../concepts/tests.md) is analogous to a "unit test" in software engineering. It tests *code* based on known inputs and outputs. In SQLMesh, the inputs and outputs are specified in a YAML file, and SQLMesh automatically runs them when `sqlmesh plan` is executed.
+    A SQLMesh [`test`](../concepts/tests.md) is analogous to a "unit test" in software engineering. It tests *code* based on known inputs and outputs. In SQLMesh, the inputs and outputs are specified in a YAML file, and SQLMesh runs all unit tests when `sqlmesh plan` is executed (use `--test-changed-only` to run only tests for models in the plan).
 
     Writing YAML is annoying and error-prone, so SQLMesh's [`create_test` command](../concepts/tests.md#automatic-test-generation) allows you to automatically generate YAML test files based on queries of existing data tables.
 

docs/guides/models.md

@@ -161,7 +161,7 @@ Reverting to a previous model version is a quick operation since no additional w
 
 SQLMesh automatically validates your models in order to ensure the quality and accuracy of your data. This is done via the following:
 
-* Running unit tests by default when you execute the `plan` command. This ensures all changes to applied to any environment are logically validated. Refer to [testing](../concepts/tests.md) for more information.
+* Running all unit tests when you execute the `plan` command (use `--test-changed-only` to run only tests for models in the plan). This ensures changes applied to any environment are logically validated. Refer to [testing](../concepts/tests.md) for more information.
 * Running audits whenever data is loaded to a table (either for backfill or loading on a cadence). This way you know all data present in any table has passed all defined audits. Refer to [auditing](../concepts/audits.md) for more information.
 
 SQLMesh also provides automatic validation via CI/CD by automatically creating a preview environment.

docs/guides/testing.md

@@ -12,11 +12,19 @@ OK
 ```
 As the unit tests run, SQLMesh will identify any that fail.
 
+By default, `sqlmesh plan` runs all unit tests. Use `--test-changed-only` to run tests only for models included in the plan (added, modified, or restated), or `--skip-tests` to run none. With both `--select-model` and `--test-changed-only`, tests run only for selected models that changed.
+
 For more information about tests, refer to [testing](../concepts/tests.md).
 
 ### Test changes to a specific model
 
-To run a specific model test, pass in the suite file name followed by `::` and the name of the test; for example: `sqlmesh test tests/test_suite.yaml::test_example_full_model`.
+To run unit tests for a specific model, use `--select-model`:
+
+```bash
+$ sqlmesh test --select-model sqlmesh_example.full_model
+```
+
+Alternatively, pass in the suite file name followed by `::` and the name of the test; for example: `sqlmesh test tests/test_suite.yaml::test_example_full_model`.
 
 ### Run a subset of tests
 

docs/integrations/dbt.md

@@ -287,9 +287,12 @@ If set to `False`, then the snapshot will be skipped and a warning will be logge
 Support for this will be added soon.
 
 ## Tests
-SQLMesh uses dbt tests to perform SQLMesh [audits](../concepts/audits.md) (coming soon).
 
-Add SQLMesh [unit tests](../concepts/tests.md) to a dbt project by placing them in the "tests" directory.
+dbt and SQLMesh use the word "test" differently. See [Workflow differences](#workflow-differences-between-sqlmesh-and-dbt) above for an overview.
+
+SQLMesh converts dbt tests (singular and generic `.sql` files in the `tests/` directory) into SQLMesh [audits](../concepts/audits.md). These run against data that already exists in your tables, just like dbt tests.
+
+Add SQLMesh [unit tests](../concepts/tests.md) to a dbt project by placing YAML files named `test*.yaml` or `test*.yml` in the `tests/` directory. Unit tests validate model query logic with predefined inputs and expected outputs; they are separate from dbt's SQL test files.
 
 ## Seed column types
 

docs/reference/cli.md

@@ -357,6 +357,8 @@ Options:
                                   Default: prod.
   --skip-tests                    Skip tests prior to generating the plan if
                                   they are defined.
+  --test-changed-only             Run unit tests only for models included in
+                                  the plan instead of all tests.
   --skip-linter                   Skip linting prior to generating the plan if
                                   the linter is enabled.
   -r, --restate-model TEXT        Restate data for specified models and models
@@ -626,6 +628,8 @@ Options:
   -v, --verbose        Verbose output.
   --preserve-fixtures  Preserve the fixture tables in the testing database,
                        useful for debugging.
+  --select-model TEXT  Select specific models to run unit tests for. Can be
+                       specified multiple times.
   --help               Show this message and exit.
 ```
 

docs/reference/notebook.md

@@ -96,7 +96,7 @@ options:
 #### plan
 ```
 %plan [--start START] [--end END] [--execution-time EXECUTION_TIME]
-            [--create-from CREATE_FROM] [--skip-tests]
+            [--create-from CREATE_FROM] [--skip-tests] [--test-changed-only]
             [--restate-model [RESTATE_MODEL ...]] [--no-gaps]
             [--skip-backfill, --dry-run] [--forward-only]
             [--effective-from EFFECTIVE_FROM] [--no-prompts] [--auto-apply]
@@ -120,6 +120,8 @@ options:
                         The environment to create the target environment from
                         if it doesn't exist. Default: prod.
   --skip-tests, -t      Skip the unit tests defined for the model.
+  --test-changed-only   Run unit tests only for models included in the plan
+                        instead of all tests.
   --restate-model <[RESTATE_MODEL ...]>, -r <[RESTATE_MODEL ...]>
                         Restate data for specified models (and models
                         downstream from the one specified). For production
@@ -131,7 +133,8 @@ options:
                         comparing to existing snapshots for matching models in
                         the target environment.
   --skip-backfill, --dry-run
-                        Skip the backfill step and only create a virtual update for the plan.
+                        Skip the backfill step and only create a virtual
+                        update for the plan.
   --forward-only        Create a plan for forward-only changes.
   --effective-from EFFECTIVE_FROM
                         The effective date from which to apply forward-only
@@ -430,7 +433,8 @@ options:
 
 #### run_test
 ```
-%run_test [--pattern [PATTERN ...]] [--verbose] [--preserve-fixtures] [tests ...]
+%run_test [--pattern [PATTERN ...]] [--verbose] [--preserve-fixtures]
+          [--select-model [SELECT_MODEL ...]] [tests ...]
 
 Run unit test(s).
 
@@ -443,6 +447,8 @@ options:
   --verbose, -v         Verbose output.
   --preserve-fixtures   Preserve the fixture tables in the testing database,
                         useful for debugging.
+  --select-model <[SELECT_MODEL ...]>
+                        Select specific models to run unit tests for.
 ```
 
 #### audit