Automate Wazuh Indexer Backups to Google Cloud Storage (Part 10)

How I backed up Wazuh index data to Google Cloud Storage, kept snapshots for 9 days, and trimmed live index data to 7 days with three small moves.

Published Jun 24, 2026 · Republished on Medium

WazuhAnsibleDevOpsGoogle Cloud PlatformSecurity

Picture by Photoholgic on Unsplash

In this lab, the Wazuh all-in-one node runs on Google Compute Engine using an ARM instance, with daily operations handled from Google Cloud Console.

Because the deployment already sits on GCE, Google Cloud Storage becomes the natural long-term backup target for the indexer layer. It keeps retained snapshot data outside the live Wazuh disk while still being easy to inspect and manage operationally.

The model in this repo is intentionally small because this lab only uses 100 GB of storage:

Keep live Wazuh data queryable for 7 days
Keep GCS snapshots for 9 days
Manage the repository, snapshot policy, and retention policy from one playbook

This gives a 2-day recovery buffer outside the live cluster without turning the indexer into long-term storage or overloading the small lab disk.

The current implementation uses GCS, but the repository role is not cloud-locked. The same pattern can be switched to S3 later if the environment changes or a secondary object-storage target is needed.

Files in This Post

bash

playbooks/
└── wazuh-indexer-backup.yml

roles/
├── wazuh_snapshot_repository/
├── wazuh_snapshot_policy/
└── wazuh_index_retention/

inventories/lab/files/wazuh/config/indexer/
├── snapshot_repository.yml
├── snapshot_policy.yml
└── index_retention.yml

external secret manager or encrypted Ansible Vault
└── GCS repository credential material

Depends on: Part 1. The indexer must already exist.

One Playbook, Three Steps

The deployment flow is intentionally linear:

yaml

# playbooks/wazuh-indexer-backup.yml
- name: Configure Wazuh indexer backup and retention
  hosts: aio:wazuh_indexer
  become: true
  gather_facts: true
  roles:
    - role: wazuh_snapshot_repository
    - role: wazuh_snapshot_policy
    - role: wazuh_index_retention

That sequence matters:

Install plugin and register snapshot repositories
Create daily snapshot schedules
Enforce 7-day live index retention

Part 1: Register the GCS Repository

The repository role reads snapshot_repository.yml.

Illustrative config (the secret reference must resolve outside plaintext source control):

yaml

repo_on: true
repo_plugin: repository-gcs
repo_keys: {}
repo_files:
  gcs.client.default.credentials_file: "{{ secured_runtime_dir }}/creds-backup.json"
repos:
  - name: wazuh-snapshots
    type: gcs
    settings:
      bucket: gcs-backup-wazuh-labs
      base_path: wazuh
      compress: true
  - name: wazuh-core-snapshots
    type: gcs
    settings:
      bucket: gcs-backup-wazuh-labs
      base_path: core
      compress: true

This splits backups into two paths in the same bucket:

wazuh/ for wazuh-* data
core/ for OpenSearch and dashboard internals

For this environment, that is the right tradeoff:

live query data stays on the Wazuh AIO node
durable retained snapshot data moves to GCS
operational management stays inside the Google Cloud environment already in use

The role does four things:

verifies every indexer node runs the expected OpenSearch/Wazuh version
installs the matching repository-gcs plugin on every node, using a rolling maintenance procedure when restart is required
loads short-lived or secret-managed credentials into the OpenSearch keystore without logging them
registers and verifies each repository through the authenticated indexer API

Even though this chapter uses repository-gcs, the role itself was written to stay portable. If object storage needs to move to Amazon S3 later, the same role can be reused by swapping the plugin, secure settings, secure files, and repository definition.

The credential is staged with restrictive permissions only long enough to inject it into the keystore, then removed. Prefer workload identity or the narrowest supported non-exportable identity mechanism; otherwise grant a dedicated service account access only to the required bucket prefix. Do not grant project-wide Owner, Editor, or storage administration.

That is the important hardening detail in this chapter.

Part 2: Schedule Daily Snapshots

Snapshot policy config lives in snapshot_policy.yml.

Current config:

yaml

snap_on: true
snap_items:
  - name: wazuh-daily-snapshots
    start: true
    definition:
      description: Daily Wazuh data snapshots to GCS with 9-day retention.
      creation:
        schedule:
          cron:
            expression: "30 1 * * *"
            timezone: Asia/Jakarta
      deletion:
        schedule:
          cron:
            expression: "0 3 * * *"
            timezone: Asia/Jakarta
        condition:
          max_age: 9d
          min_count: 1
      snapshot_config:
        repository: wazuh-snapshots
        indices: "wazuh-*"

  - name: wazuh-core-daily-snapshots
    start: true
    definition:
      description: Daily core OpenSearch and Dashboards snapshots to GCS.
      creation:
        schedule:
          cron:
            expression: "45 1 * * *"
            timezone: Asia/Jakarta
      deletion:
        schedule:
          cron:
            expression: "15 3 * * *"
            timezone: Asia/Jakarta
        condition:
          max_age: 9d
          min_count: 1
      snapshot_config:
        repository: wazuh-core-snapshots
        indices: ".opensearch_dashboards*,.kibana*,.plugins*,.opendistro*,.tasks,.security*"

The local rule here is:

Snapshot creation runs daily
Snapshot cleanup also runs daily
GCS snapshot retention is 9 days

Why 9 days?

7 days live in Wazuh
2 extra days in backup

That is enough buffer for restore mistakes or delayed investigation without keeping too much stale data in the bucket.

Snapshot creation is asynchronous unless the request waits for completion. A scheduler accepting the request does not prove a usable backup exists: poll snapshot status until SUCCESS, treat PARTIAL, FAILED, INCOMPATIBLE, and overdue IN_PROGRESS states as failures, and alert when the last successful recovery point exceeds the RPO.

Part 3: Keep Live Data for 7 Days

Live retention config lives in index_retention.yml.

Current config:

yaml

retention_on: true
retention_name: wazuh-retention-7d
retention_patterns:
  - wazuh-alerts-*
  - wazuh-archives-*
  - wazuh-monitoring-*
  - wazuh-statistics-*
  - wazuh-states-*
retention_body:
  policy:
    description: Retain Wazuh indexes for 7 days and then delete them.
    default_state: hot
    schema_version: 1
    ism_template:
      - index_patterns:
          - wazuh-alerts-*
          - wazuh-archives-*
          - wazuh-monitoring-*
          - wazuh-statistics-*
          - wazuh-states-*
        priority: 100
    states:
      - name: hot
        actions: []
        transitions:
          - state_name: delete
            conditions:
              min_index_age: 7d
      - name: delete
        actions:
          - delete: {}

This policy does not back anything up. It only governs live index deletion inside OpenSearch.

That distinction is easy to miss:

Snapshot policy controls backup creation and backup deletion
ISM retention controls live index lifecycle

They are separate systems and both are needed.

The 7-day ISM deletion must not race ahead of a failed snapshot. Before enabling destructive retention, prove successful snapshots cover the intended indices and dates. Object-storage lifecycle rules must also be longer than or coordinated with OpenSearch Snapshot Management; deleting repository objects directly can corrupt repository metadata and make otherwise listed snapshots unrestorable.

Effective Retention Model

With the current repo state:

Live Wazuh indexes: 7 days
GCS snapshots: 9 days

This is not “16 days total of live data”.

It means:

The cluster itself keeps 7 days
Older recovery points remain in GCS for up to 9 days from snapshot time

So the operational safety margin is in the backup layer, not in the live query layer.

Verification

Run the playbook:

bash

ansible-playbook -i inventories/lab/hosts.ini playbooks/wazuh-indexer-backup.yml

Useful dashboard checks:

Snapshot Management shows both policies enabled