# Automated Testing

Varbase 11.0.x includes a comprehensive automated functional acceptance testing suite built with [**Playwright**](https://playwright.dev/), [**Cucumber-JS**](https://github.com/cucumber/cucumber-js), and [**Webship-js**](https://github.com/webship/webship-js).

## Overview

The testing suite provides end-to-end testing of Varbase features using Behaviour-Driven Development (BDD). Tests are written in plain language (Gherkin syntax) making them readable by all team members, and are executed against real browser instances.

### Technology Stack

| Tool                                                       | Role                                                                                                         |
| ---------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
| [**Playwright**](https://playwright.dev/)                  | Browser automation and end-to-end testing framework. Supports Chromium, Firefox, and WebKit.                 |
| [**Cucumber-JS**](https://github.com/cucumber/cucumber-js) | BDD test runner that executes tests written in Gherkin (Given/When/Then) plain language syntax.              |
| [**Webship-js**](https://github.com/webship/webship-js)    | Automated functional acceptance testing tool with pre-built step definitions for Drupal Core and Drupal CMS. |

## Prerequisites

* [DDEV](https://ddev.readthedocs.io/) local development environment
* Node.js >= 20
* Yarn 4 (enabled via corepack)

## Quick Start

### Fresh Build (New Site)

```bash
# 1. Start a fresh DDEV environment
ddev start

# 2. Install Varbase and initialize testing
ddev init-full-automated-testing

# 3. Install Playwright browsers
ddev yarn install
ddev npx playwright install-deps
ddev npx playwright install

# 4. Run tests
ddev yarn test:chromium
```

### Already Installed Site

```bash
# 1. Initialize testing on an existing site (adds test users, prepares settings)
ddev init-minimal-automated-testing

# 2. Install Playwright browsers
ddev yarn install
ddev npx playwright install-deps
ddev npx playwright install

# 3. Run tests
ddev yarn test:chromium
```

## DDEV Commands for Testing Varbase

### `ddev install-varbase`

Installs Varbase from scratch using `drush site:install varbase` with the Varbase profile and Drupal recipes.

```bash
# Minimal install: core Varbase only (no extra recipes)
ddev install-varbase minimal

# Full install: core + dev, i18n, api, auth recipes + social auth modules
ddev install-varbase full
```

### `ddev init-full-automated-testing`

Full initialization for automated testing. Handles everything from a fresh `ddev start`:

1. Installs Varbase if the database is empty
2. Applies optional Varbase recipes (`varbase_dev_base`, `varbase_i18n_base`, `varbase_api_base`, `varbase_auth_base`, `varbase_ai_base`)
3. Enables social auth modules (`social_auth_facebook`, `social_auth_linkedin`)
4. Adds testing users for each Varbase role
5. Disables the antibot module (required for automated browser testing)
6. Disables CSS/JS aggregation
7. Sets verbose error logging

### `ddev init-minimal-automated-testing`

Minimal initialization for automated testing on an already installed site. Does not install Varbase or apply recipes:

1. Verifies Drupal is installed (exits with error if not)
2. Adds testing users for each Varbase role
3. Disables the antibot module
4. Disables CSS/JS aggregation
5. Sets verbose error logging

### `ddev add-testing-users` / `ddev delete-testing-users`

Manage testing user accounts individually.

## Running Tests

```bash
# Run all tests with Chromium
ddev yarn test:chromium

# Run all tests with Firefox
ddev yarn test:firefox

# Run all tests with WebKit
ddev yarn test:webkit
```

### Running Specific Tests

```bash
# Run scenarios matching a name (regex filter)
LAUNCH_URL=https://VARBASE_PROJECT.ddev.site BROWSER=chromium \
  node ./node_modules/@cucumber/cucumber/bin/cucumber.js \
  --config cucumber.js --name "Canvas editor"

# Run tests by tag
LAUNCH_URL=https://VARBASE_PROJECT.ddev.site BROWSER=chromium \
  node ./node_modules/@cucumber/cucumber/bin/cucumber.js \
  --config cucumber.js --tags "@check"
```

## Test Structure

```
tests/
  features/
    01-website-base-requirements/    # Registration, roles, input formats, languages, accessibility
    02-user-management/              # Login, passwords, role assignment, login redirect
    03-admin-management/             # Admin pages, masquerade, media, JSON:API
    04-content-structure/            # Content types, Canvas pages, blog, homepage, contact, breadcrumbs
    05-content-management/           # Entityqueues, media library, workflows, scheduling, cloning, trash
  step-definitions/
    varbase-step-definitions.js      # Varbase-specific step definitions
  assets/
    flag-earth.jpg                   # Test asset for media upload scenarios
```

Step definitions from [Webship-js](https://github.com/webship/webship-js) are loaded automatically from `node_modules/webship-js/tests/step-definitions/`.

## Test Suites

### 01 - Website Base Requirements

| Feature               | Description                                                                                   |
| --------------------- | --------------------------------------------------------------------------------------------- |
| Welcome Tour          | Admin dashboard displays after login                                                          |
| User Registration     | Only admins can register new users                                                            |
| User Roles            | Default roles (Content editor, Content Admin, SEO Admin, Site Admin, Super Admin) are present |
| Input Formats         | Rich text editor formats are available to authorized users                                    |
| Website Languages     | Internationalization configuration is accessible                                              |
| Accessibility Checker | Editoria11y accessibility checker permissions                                                 |

### 02 - User Management

| Feature                       | Description                                                       |
| ----------------------------- | ----------------------------------------------------------------- |
| Request New Password          | Password reset functionality                                      |
| Create Users and Assign Roles | Site admins can create accounts and assign roles                  |
| User Login                    | Login functionality for existing accounts                         |
| Persistent Login              | Session persistence during browser sessions                       |
| User Protect                  | User account protection for admin accounts                        |
| Role Assign                   | Role assignment permissions and delegation                        |
| Login Redirect                | Admin roles redirect to dashboard, authenticated users to profile |

### 03 - Admin Management

| Feature             | Description                                                   |
| ------------------- | ------------------------------------------------------------- |
| Admin Pages         | Pre-check of important admin and development pages            |
| Masquerade          | Site admins can switch users to view the site as another user |
| Keyboard Navigation | Navigate admin with Coffee module keyboard shortcut           |
| Disable Users       | Admins can block user accounts                                |
| Media Usage         | Image media and usage list page for admins                    |
| JSON:API            | JSON:API admin interface and services checks                  |
| Audit Trail         | Admin audit trail access permissions                          |
| Media Bulk Upload   | Media bulk upload access permissions                          |

### 04 - Content Structure

| Feature          | Description                                  |
| ---------------- | -------------------------------------------- |
| Page Permissions | Page content type permissions by role        |
| Canvas Pages     | Canvas page builder permissions              |
| Breadcrumbs      | Standard breadcrumb navigation               |
| Blog Permissions | Blog post content type permissions by role   |
| Blog Page        | Blog listing page renders correctly          |
| Contact Us       | Contact form page is accessible              |
| Homepage         | Homepage loads and displays expected content |
| Canvas Editor    | Drupal Canvas visual page editor access      |

### 05 - Content Management

| Feature            | Description                                 |
| ------------------ | ------------------------------------------- |
| Entityqueues       | Entityqueue management permissions          |
| Cloning            | Content and entity cloning via Entity Clone |
| Media Library      | Media library access permissions            |
| Linkit             | Internal content linking configuration      |
| Content Workflows  | Content moderation workflow configuration   |
| Content Scheduling | Scheduled publishing and unpublishing       |
| Trash Management   | Soft-deleted content management             |

## Testing Users

The following test users are created during testing initialization:

| Username       | Email                            | Role            |
| -------------- | -------------------------------- | --------------- |
| webmaster      | <webmaster@vardot.com>           | administrator   |
| Normal user    | <test.authenticated@vardot.com>  | authenticated   |
| Content editor | <test.content_editor@vardot.com> | content\_editor |
| Content admin  | <test.content_admin@vardot.com>  | content\_admin  |
| SEO admin      | <test.seo_admin@vardot.com>      | seo\_admin      |
| Site admin     | <test.site_admin@vardot.com>     | site\_admin     |
| Super admin    | <test.super_admin@vardot.com>    | administrator   |

All test user passwords: `dD.123123ddd`

## Configuration Files

### [**cucumber.js**](https://git.drupalcode.org/project/varbase_project/-/blob/11.0.x/cucumber.js)

Cucumber-JS configuration file defining:

* Test timeout (40 seconds)
* Step definition paths (Webship-js + Varbase custom)
* Feature file paths
* World parameters (launch URL, wait times, test user credentials)

The `launchUrl` defaults to the `LAUNCH_URL` environment variable, then `DDEV_PRIMARY_URL`, then `https://localhost`.

### [**playwright.config.ts**](https://git.drupalcode.org/project/varbase_project/-/blob/11.0.x/playwright.config.ts)

Playwright browser configuration:

* Headless mode enabled by default
* SlowMo set to 300ms for stability
* HTTPS certificate errors ignored (for DDEV self-signed certs)
* Chromium launched with sandbox disabled for CI compatibility
* Browser selected via the `BROWSER` environment variable (defaults to `chromium`)

## Example Local Varbase Automated Testing with DDEV

```bash
mkdir my-varbase-site ;
cd my-varbase-site ;
ddev config --project-type=drupal11 --docroot=web --php-version=8.4 ;
ddev start ;
ddev composer create-project "drupal/varbase_project:11.0.x-dev" --no-interaction ;
ddev init-full-automated-testing ;
ddev yarn install ;
ddev npx playwright install-deps chromium ;
ddev npx playwright install chromium ;
ddev yarn test:chromium ;
```

### To See the Test in Your local Browser

1- Change `headless: true,` to `headless: false,` in the `playwright.config.js` file.

2- Change to `launchUrl: 'https://my-varbase-site.ddev.site',` in the `cucumber.js` file.

3- Run in your local development, not with DDEV

<pre class="language-bash"><code class="lang-bash"><strong>yarn install ;
</strong>npx playwright install-deps chromium ;
npx playwright install chromium ;
yarn test:chromium ;
</code></pre>

## Custom Step Definitions

Varbase provides custom step definitions in `tests/step-definitions/varbase-step-definitions.js` that extend the Webship-js base:

| Step                            | Example                                                                        |
| ------------------------------- | ------------------------------------------------------------------------------ |
| Login as a configured user      | `Given I am a logged in user with the "Content admin" user`                    |
| Browse as anonymous             | `Given I am an anonymous user`                                                 |
| Check a checkbox by label       | `When I check the box "Remember me"`                                           |
| Assert checkbox checked         | `Then I should see the "Remember me" checkbox checked`                         |
| Assert checkbox unchecked       | `Then I should see the "Remember me" checkbox unchecked`                       |
| Wait for page load              | `And wait`                                                                     |
| Click tour next button          | `When I click next button in tour`                                             |
| Assert element in container     | `Then I should see the "#edit-title" element in the ".form-wrapper" field`     |
| Assert element not in container | `Then I should not see the "#edit-title" element in the ".form-wrapper" field` |

Additional step definitions are loaded from [Webship-js](https://webship.co/docs/webship-js/2.0.x/step-definitions) which provides a comprehensive library of Drupal-aware steps for navigation, forms, content, media, and more.

## Writing New Tests

Feature files use Gherkin syntax. Each scenario describes a user story with Given/When/Then steps:

```gherkin
Feature: Content Structure - Blog post permissions
      As a site admin user
      I want to control who can create and manage Blog posts
      So that only authorized users can publish blog content.

  @javascript @check @local @development @staging @production
  Scenario: Check that the webmaster can access the blog listing
    Given I am a logged in user with the "webmaster" user
     When I go to "/admin/content"
      And wait
     Then I should see "Content"
```

### Tags

Tests use tags to control execution scope:

| Tag            | Purpose                                    |
| -------------- | ------------------------------------------ |
| `@javascript`  | Requires browser execution                 |
| `@check`       | Quick verification tests                   |
| `@local`       | Suitable for local development             |
| `@development` | Suitable for development environments      |
| `@staging`     | Suitable for staging environments          |
| `@production`  | Suitable for production (read-only checks) |

Place new feature files in the appropriate numbered directory under `tests/features/` following the existing naming convention.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.varbase.vardot.com/developers/automated-testing.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.