tests: update testing docs

Author: espy

.github/workflows/ci.yml

@@ -277,7 +277,7 @@ jobs:
       matrix:
         node: [20, 22]
         cmd:
-          - COUCH_HOST=http://admin:password@127.0.0.1:5984 CLIENT=firefox npm run test-webpack
+          - CLIENT=firefox npm run test-webpack
           - AUTO_COMPACTION=true npm test
           - npm run test-unit
           - npm run test-component

TESTING.md

@@ -1,12 +1,35 @@
-# Running PouchDB Tests
+# Running PouchDB Tests <!-- omit from toc -->
 
 This document covers all the types of tests the PouchDB project has and how to
 run them. PouchDB has been primarily developed on Linux and macOS, if you are
 using Windows then these instructions will have problems, we would love your
 help fixing them though.
 
-
-## Installation
+> [!WARNING] VERY IMPORTANT
+> **Always set the `COUCH_HOST` env var to a proper CouchDB for the integration tests!**
+> 
+> If you don’t, the integration tests currently fall back to `pouchdb-server`, which is 
+> no longer reliable and causes random test failures. (20000ms timeouts).
+>
+> Until this is resolved, please set the `COUCH_HOST` env var:
+> `$ CLIENT=firefox COUCH_HOST=http://admin:admin@127.0.0.1:5984 npm test`
+>
+> This is necessary for both `npm test` and `npm run dev`.
+
+## Table of Contents <!-- omit from toc -->
+
+- [Running the integration tests](#running-the-integration-tests)
+  - [Test options](#test-options)
+- [Other sets of tests](#other-sets-of-tests)
+  - [`find` and `mapreduce`](#find-and-mapreduce)
+  - ["Fuzzy" tests](#fuzzy-tests)
+  - [Performance tests](#performance-tests)
+  - [Running tests in the browser](#running-tests-in-the-browser)
+- [Other test tasks](#other-test-tasks)
+- [Troubleshooting and useful info when working on tests](#troubleshooting-and-useful-info-when-working-on-tests)
+  - [Where do I get a CouchDB from?](#where-do-i-get-a-couchdb-from)
+  - [Utils won’t update](#utils-wont-update)
+  - [Flaky tests and random timeouts](#flaky-tests-and-random-timeouts)
 
 The PouchDB test suite expects an instance of CouchDB (version 1.6.1 and above)
 running in [Admin Party](http://guide.couchdb.org/draft/security.html#party) on
@@ -38,6 +61,9 @@ The main test suite can be run using the following command:
 
     $ npm test
 
+> [!NOTE]
+> If the tests don‘t pass on `main`, try building once before running the tests: `$npm run build`.
+
 PouchDB runs in the browser and on Node.js, and has multiple different storage
 backends known as _adapters_. In the browser these are `idb`, `indexeddb` and
 `memory` and on Node.js they're `leveldb` and `memory`.
@@ -59,7 +85,7 @@ replication to a remote server, and by default we start an instance of
 The integration tests support the following options, configured via environment
 variables.
 
-#### `ADAPTERS` (default: depends on `CLIENT`)
+#### `ADAPTERS` (default: depends on `CLIENT`) <!-- omit from toc -->
 
 Comma-separated list of preferred adapter backends that PouchDB will use for local
 databases. These are selected automatically based on the execution environment,
@@ -74,21 +100,21 @@ a remote server, identified by `COUCH_HOST`. This is not necessary for
 integration tests since they use a mixture of local and remote databases to
 check compatibility, but it's useful for the `find` and `mapreduce` suites.
 
-#### `AUTO_COMPACTION` (default: `0`)
+#### `AUTO_COMPACTION` (default: `0`) <!-- omit from toc -->
 
 Set this to `1` to enable automatic compaction of PouchDB databases by default.
 
-#### `BAIL` (default: `1`)
+#### `BAIL` (default: `1`) <!-- omit from toc -->
 
 Normally the test runner will halt as soon as it discovers a failing test. Set
 this to `0` to prevent this behaviour.
 
-#### `CLIENT` (default: `node`)
+#### `CLIENT` (default: `node`) <!-- omit from toc -->
 
 Sets the target platform the tests will execute on. Set this to
 `firefox`, `chromium` or `webkit` to execute the tests in the browser.
 
-#### `COUCH_HOST`
+#### `COUCH_HOST` <!-- omit from toc -->
 
 Some tests perform replication between local and remote databases. When we
 create a remote database, we get the URL of the remote server from `COUCH_HOST`.
@@ -98,45 +124,45 @@ CORS enabled.
 If not set explicitly, this variable is set automatically based on the other
 configuration values.
 
-#### `FETCH` (default: `0`)
+#### `FETCH` (default: `0`) <!-- omit from toc -->
 
 Set this to `1` to stop PouchDB falling back to `XMLHttpRequest` if `fetch()` is
 not available.
 
-#### `GREP`
+#### `GREP` <!-- omit from toc -->
 
 Use this to request that a specific test is run; if you set `GREP='name of
 test'` then only those tests whose names include the string `name of test` will
 run.  Regular expressions are also supported.
 
-#### `PLUGINS` (default: empty)
+#### `PLUGINS` (default: empty) <!-- omit from toc -->
 
 Comma-separated list of additional plugins that should be loaded into the test
 environment. For example:
 
     $ PLUGINS=pouchdb-find npm test
 
-#### `POUCHDB_SRC`
+#### `POUCHDB_SRC` <!-- omit from toc -->
 
 This overrides the path used to load PouchDB in the browser. We use this in CI
 to select different builds of the PouchDB library, for example to test the
 Webpack version, etc.
 
 This is an alternative to `SRC_ROOT` and `USE_MINIFIED`.
 
-#### `SRC_ROOT`
+#### `SRC_ROOT` <!-- omit from toc -->
 
 This overrides the path used to load all PouchDB files in the browser. We use
 this in performance tests to allow easily comparing two different versions of
 PouchDB, including plugin and adapter implementations.
 
-#### `USE_MINIFIED`
+#### `USE_MINIFIED` <!-- omit from toc -->
 
 This changes the file extension used for loading PouchDB files in the browser.
 This can be used in CI and performance testing to select the minified version of
 PouchDB and its adapters, plugins, etc.
 
-#### `SERVER` (default: `pouchdb-express-router`)
+#### `SERVER` (default: `pouchdb-express-router`) <!-- omit from toc -->
 
 To support remote replication tests, we start a server in the background that
 speaks the CouchDB replication protocol. This variable controls how that is
@@ -151,11 +177,11 @@ following values:
 - `couchdb-master`: use this value if you already have CouchDB running; it
   causes `COUCH_HOST` to be set to the correct value.
 
-#### `SKIP_MIGRATION` (default: `0`)
+#### `SKIP_MIGRATION` (default: `0`) <!-- omit from toc -->
 
 Set this to `1` to skip the migration tests.
 
-#### `VIEW_ADAPTERS` (default: `memory`)
+#### `VIEW_ADAPTERS` (default: `memory`) <!-- omit from toc -->
 
 Comma-separated list of preferred view adapter backends that PouchDB will use. 
 This variable overrides the default choice and causes additional adapters to
@@ -226,7 +252,7 @@ This supports most of the same options as the integration suite, particularly
 the `CLIENT`, `ADAPTERS` and `GREP` options. It has some additional options of
 its own:
 
-#### `ITERATIONS`
+#### `ITERATIONS` <!-- omit from toc -->
 
 Sets the number of iterations each test uses by default.
 
@@ -252,15 +278,20 @@ server and opening a browser window yourself. To run the server:
 
     $ npm run dev
 
+You will almost always want to include the `find` plugin though, without it, all the `find` tests will fail:
+
+    $ PLUGINS=pouchdb-find npm run dev
+
 Then you can open the page for any of the test suites via the following URLs:
 
 - `http://127.0.0.1:8000/tests/integration/`
 - `http://127.0.0.1:8000/tests/find/`
 - `http://127.0.0.1:8000/tests/mapreduce/`
 - `http://127.0.0.1:8000/tests/performance/`
 
-The test options are controlled by editing the query string; some of the common
-command-line options and their query string equivalents are:
+You can re-run tests by reloading, and only run specific suites by clicking on the suite names, this sets the `grep` query string mentioned below. This works well in conjunction with setting `.only` on individual tests in that suite.
+
+The test options are controlled by editing the query string; some of the common command-line options and their query string equivalents are:
 
 | Environment variable | Query-string param |
 | -------------------- | ------------------ |
@@ -281,29 +312,61 @@ command-line options and their query string equivalents are:
 There are a few other tasks we run during CI and which you will find useful to
 run during development.
 
-### `npm run eslint`
+### `npm run eslint` <!-- omit from toc --> 
 
 Checks that all code in the project follows our formatting and style guide. This
 runs before any other tasks are run during our CI build.
 
-### `npm run test-unit`
+### `npm run test-unit` <!-- omit from toc --> 
 
 Runs the unit tests; running these can give more precise feedback about key
 building blocks that are not working.
 
-### `npm run test-component`
+### `npm run test-component` <!-- omit from toc --> 
 
 Tests some additional components besides the core database functionality, for
 example authentication and read-only replication.
 
-### `npm run test-coverage`
+### `npm run test-coverage` <!-- omit from toc --> 
 
 Runs the test suite with coverage analysis turned on.
 
-### `npm run test-webpack`
+### `npm run test-webpack` <!-- omit from toc --> 
 
 Checks that the Webpack build of PouchDB works correctly.
 
-### `npm run verify-build`
+### `npm run verify-build` <!-- omit from toc --> 
 
 Checks that the build is correct.
+
+## Troubleshooting and useful info when working on tests
+
+### Where do I get a CouchDB from?
+
+See the [official CouchDB documentation](https://docs.couchdb.org/en/stable/install/index.html) for a guide on how to install CouchDB.
+
+Your CouchDB will most likely then run at `http://127.0.0.1:5984`. All you need to do now is  enable CORS so CouchDB will accept requests from PouchDB in the tests. Add `http://127.0.0.1:8000`, the test server’s domain. There are several ways to do this, easiest first:
+
+1. In the [CouchDB admin UI](http://127.0.0.1:5984/_utils), click on `Configuration` -> `CORS`. 
+2. Via cURL (`_local` is the node name in single-node databases, which your local dev CouchDB probably is):
+   ```sh
+   curl 'http://admin:password@127.0.0.1:5984/_node/_local/_config/cors/origins' -X PUT -d '"http://127.0.0.1:8000"'
+   ```
+3. There’s also an older [npm package](https://github.com/pouchdb/add-cors-to-couchdb) to help you do this porgrammatically.
+
+### Utils won’t update
+
+If you’re modifying the test utils (`/tests/integration/utils.js` etc.) or adding logs to them, you need to rebuild these before running your tests again, eg.:
+
+```sh
+$ npm run build && CLIENT=firefox npm test
+```
+
+### Flaky tests and random timeouts
+
+If you regularly run into random timeouts, please re-read the top of this file and set the `COUCH_HOST` env var when running tests or the dev server, eg.:
+
+```sh
+$ CLIENT=firefox COUCH_HOST=http://admin:admin@127.0.0.1:5984 npm test
+```
+