use rsync to push restore archive to remote, avoiding OOM on large backups

README.md

@@ -1,415 +1,29 @@
-# Docker Compose App Role
+# patrickj.docker_compose_app
 
-A flexible Ansible role for deploying and managing Docker Compose applications with features including backup/restore and healthchecks.
+Ansible collection providing a single reusable role for deploying and managing Docker Compose applications.
 
-## Overview
+## Contents
 
-This role provides a standardized way to deploy containerized applications using Docker Compose. It handles directory creation, template rendering, and container lifecycle management.
+- `patrickj.docker_compose_app.app` — the base role. See [`roles/app/README.md`](roles/app/README.md) for usage, variables, and examples.
 
-## Features
+## Installation
-
-- **Template-based deployment** with Jinja2 templating
-- **Dual backup system** (controller + remote host)
-- **Health checks** and deployment verification
-- **Multiple deployment modes** (template, file, or inline content)
-- **Flexible directory management**
-- **Configurable backup retention policies**
-
-## Quick Start
-
-### 1. Create a New Role
-
-Use the provided script to create a new application skeleton role:
 
 ```bash
-./create_app_role.sh my-app
+ansible-galaxy collection install git+https://git.jaroszew.ski/ansible/ansible-collection-docker-compose-app.git
 ```
 
-This creates:
+Or as a dependency in another collection's `galaxy.yml`:
-```
-roles/my-app/
-├── defaults/main.yml     # Default variables
-├── meta/main.yml         # Role metadata and dependencies
-└── templates/
-    └── compose.yml.j2    # Docker Compose template
-```
-
-### 2. Configure Your Application
-
-Edit `roles/my-app/defaults/main.yml`:
 
 ```yaml
----
-app_role_name: my-app
-
-my_app_container_name: "{{ app_name | default('my-app') }}"
-my_app_container_version: latest
-my_app_restart_policy: "{{ app_restart_policy }}"
-my_app_http_port: 8080
-my_app_data_path: "{{ app_dir }}/data"
-my_app_config_path: "{{ app_dir }}/config"
-
-# Optional: directories to create
-app_subdirectories:
-  - "{{ my_app_data_path }}"
-  - "{{ my_app_config_path }}"
-
-# Optional: backup configuration
-app_backup_subdirectories:
-  - "{{ my_app_data_path }}"
-  - "{{ my_app_config_path }}"
-```
-
-### 3. Create Docker Compose Template
-
-Edit `roles/my-app/templates/compose.yml.j2`:
-
-```yaml
----
-services:
-  my-app:
-    image: "my-app:{{ my_app_container_version }}"
-    container_name: "{{ my_app_container_name }}"
-    restart: "{{ my_app_restart_policy }}"
-    ports:
-      - "{{ my_app_http_port }}:8080"
-    volumes:
-      - "{{ my_app_data_path }}:/data"
-      - "{{ my_app_config_path }}:/config"
-    environment:
-      - TZ={{ app_timezone | default('UTC') }}
-```
-
-### 4. Deploy Your Application
-
-Add to your playbook:
-
-```yaml
----
-- name: Deploy my application
-  hosts: localhost
-  roles:
-    - role: my-app
-```
-
-## Advanced Usage
-
-### Backup and Restore
-
-#### Create a New Backup
-
-Create an on-demand backup using tags:
-
-```bash
-# Backup a specific application
-ansible-playbook --tags backup playbook.yml
-```
-
-Configure backup retention and options in your role's meta vars:
-
-```yaml
-# In roles/my-app/meta/main.yml
 dependencies:
-  - role: patrickj.docker_apps.docker_compose_app
+  "patrickj.docker_compose_app": ">=1.0.0"
-    vars:
-      app_backup_subdirectories:
-        - "{{ my_app_data_path }}"
-        - "{{ my_app_config_path }}"
-      app_backup_retention_days_controller: 90  # Keep backups for 90 days
-      app_backup_retention_days_remote: 7       # Keep remote backups for 7 days  
-      app_backup_stop_services: true            # Stop containers during backup
 ```
 
-#### Automated Backups
+## Scaffolding a new consuming role
 
-For scheduled backups:
+`create_app_role.sh` at the repo root scaffolds a new "shim" role that depends on `patrickj.docker_compose_app.app`. Run it from the `roles/` directory of your consuming collection or playbook:
 
 ```bash
-# Run daily backup at 2 AM - only runs backup tasks
+cd path/to/your/collection/roles
-0 2 * * * ansible-playbook -i inventory --tags backup my-playbook.yml
+/path/to/this/repo/create_app_role.sh my-app
 ```
-
-### Tag-Based Execution
-
-The role supports selective execution using tags:
-
-```bash
-# Setup only - doesn't start the containers
-ansible-playbook --tags setup playbook.yml
-
-# Full deployment - setup + start containers
-ansible-playbook --tags deploy playbook.yml
-
-# Restore and deploy
-ansible-playbook --tags restore,deploy playbook.yml
-
-# Backup only
-ansible-playbook --tags backup playbook.yml
-
-# Update only
-ansible-playbook --tags update playbook.yml
-
-# Backup then update
-ansible-playbook --tags backup,update playbook.yml
-```
-
-#### Restore from Backup
-
-Restore from the latest backup:
-
-```bash
-# Setup directories, restore from backup, then deploy
-ansible-playbook --tags setup,restore,deploy playbook.yml
-```
-
-Configure restore options in your role's meta vars:
-
-```yaml
-# In roles/my-app/meta/main.yml
-dependencies:
-  - role: patrickj.docker_apps.docker_compose_app
-    vars:
-      app_restore_source: controller  # or 'remote'
-      app_restore_max_age_days: 30    # Only restore backups newer than 30 days
-```
-
-Restore from a specific archive by setting the variable at runtime:
-
-```bash
-ansible-playbook --tags restore playbook.yml \
-  -e app_restore_archive=/path/to/specific/backup-20240315-120000.tar.gz
-```
-
-### Health Checks
-
-Configure health monitoring:
-
-```yaml
-- role: my-app
-  vars:
-    app_name: my-app
-    app_health_check: true
-    app_health_check_method: http
-    app_health_check_url: "http://localhost:8080/health"
-    app_health_check_retries: 30
-    app_health_check_delay: 10
-```
-
-### Updates
-
-Update application containers using tags:
-
-```bash
-# Update a specific application
-ansible-playbook --tags update playbook.yml
-
-# Update all applications in a playbook
-ansible-playbook --tags update site.yml
-```
-
-**Update process:**
-1. Stops the application
-2. Pulls latest images for all services
-3. Recreates containers with new images
-4. Runs health checks (if enabled)
-
-
-### Rollbacks
-
-Rollback using the restore functionality:
-
-```bash
-# First, create a backup then update
-ansible-playbook --tags backup,update playbook.yml
-
-# If update fails, rollback from backup
-ansible-playbook --tags restore playbook.yml
-```
-
-**Important Notes:**
-- Updates do NOT create automatic backups - create backups manually beforehand
-- Rollbacks must be triggered manually using the restore functionality
-- Using `latest` tags prevents effective rollbacks since the old image is overwritten
-- For production, use specific version tags instead of `latest`
-
-
-### Multiple Instances
-
-Deploy multiple instances of the same application:
-
-```yaml
-- role: jellyfin
-  vars:
-    app_name: jellyfin-movies
-    jellyfin_http_port: 8096
-
-- role: jellyfin  
-  vars:
-    app_name: jellyfin-tv
-    jellyfin_http_port: 8097
-```
-
-
-### Custom Templates and Files
-
-#### Override template location:
-
-```yaml
-- role: my-app
-  vars:
-    app_compose_template: /path/to/custom/compose.yml.j2
-```
-
-#### Use a static compose file:
-
-If you have an existing `docker-compose.yml` file that doesn't need templating:
-
-```yaml
-- role: my-app
-  vars:
-    app_name: my-app
-    app_compose_file: /path/to/existing/docker-compose.yml
-```
-
-The file will be copied as-is to the application directory without Jinja2 processing.
-
-#### Use inline content:
-
-```yaml
-- role: patrickj.docker_apps.docker_compose_app
-  vars:
-    app_name: simple-app
-    app_role_name: simple-app
-    app_compose_content: |
-      services:
-        app:
-          image: nginx:alpine
-          ports:
-            - "80:80"
-```
-
-## Configuration Reference
-
-### Core Variables
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `app_name` | Required | Unique application instance name |
-| `role_name` | Required | Role name for template paths |
-| `app_dir` | `{{ host_root_path }}/{{ app_name }}` | Application directory |
-| `app_uid` | `{{ ansible_facts.user_uid }}` | File ownership UID |
-| `app_gid` | `{{ ansible_facts.user_gid }}` | File ownership GID |
-| `app_permission_mode` | `"0640"` | File permission mode |
-
-### Deployment Options
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `app_compose_template` | `{{ app_templates_path }}/compose.yml.j2` | Template file path |
-| `app_compose_file` | - | Static compose file path |
-| `app_compose_content` | - | Inline compose content |
-| `app_compose_validate` | `true` | Validate compose syntax |
-| `app_compose_pull` | `policy` | Image pull strategy |
-| `app_compose_recreate` | `auto` | Container recreate strategy |
-
-### Backup Configuration
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `app_backup_subdirectories` | `[]` | Specific directories to backup |
-| `app_backup_stop_services` | `true` | Stop containers during backup |
-| `app_backup_retention_days_controller` | `90` | Controller backup retention |
-| `app_backup_retention_days_remote` | `7` | Remote backup retention |
-
-### Restore Configuration
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `app_restore_source` | `controller` | Restore source (`controller`/`remote`) |
-| `app_restore_archive` | - | Specific archive file to restore (overrides latest) |
-| `app_restore_max_age_days` | `30` | Maximum backup age for restore (when finding latest) |
-
-### Health Check Configuration
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `app_health_check` | `true` | Enable health checks |
-| `app_health_check_method` | `docker` | Check method (`docker`/`http`) |
-| `app_health_check_url` | - | HTTP endpoint for health check |
-| `app_health_check_retries` | `30` | Number of check retries |
-| `app_health_check_delay` | `10` | Delay between checks (seconds) |
-| `app_health_check_status_codes` | `[200, 201, 202]` | Valid HTTP status codes |
-
-### Directory Management
-
-| Variable | Default | Description |
-|----------|---------|-------------|
-| `app_subdirectories` | `[]` | Directories to create in app_dir |
-| `app_extra_templates` | `[]` | Additional templates to render |
-
-Example `app_extra_templates`:
-```yaml
-app_extra_templates:
-  - src: config.json.j2
-    dest: "{{ app_dir }}/config/app.json"
-  - src: env.j2  
-    dest: "{{ app_dir }}/.env"
-```
-
-
-## Best Practices
-
-1. **Use semantic versioning** for container tags instead of `latest`
-2. **Test restore procedures** regularly to ensure backups are valid
-3. **Use specific directories** for backups instead of entire app directory
-4. **Monitor disk usage** on backup storage locations
-
-## Architecture
-
-The role is organized into logical task files for selective execution:
-
-```
-├── setup.yaml             # Validation, directories, and templates
-├── restore.yaml           # Backup restoration logic
-├── deploy.yaml            # Docker Compose deployment and startup
-├── health_check.yaml      # Health monitoring and verification  
-├── backup.yaml            # Backup creation and management
-├── update.yaml            # Container update and image pulling
-└── manage_compose.yaml    # Docker Compose lifecycle management
-```
-
-### Task Execution Flow
-
-1. **Setup** (`setup.yaml`) - [setup, deploy]
-   - Docker/Compose validation
-   - Variable validation and security checks
-   - Directory creation
-   - Additional template deployment
-
-2. **Restore** (`restore.yaml`) - [restore, never]
-   - Backup file selection and validation
-   - Service stopping
-   - Archive extraction
-   - Service restart
-
-3. **Deploy** (`deploy.yaml`) - [deploy]
-   - Docker Compose file creation
-   - Container startup and management
-
-4. **Health Check** (`health_check.yaml`) - [deploy, healthcheck]
-   - Service health verification
-   - Endpoint monitoring
-
-5. **Backup** (`backup.yaml`) - [backup, never]
-   - Service stopping (optional)
-   - Archive creation
-   - Backup copying and cleanup
-   - Service restart
-
-6. **Update** (`update.yaml`) - [update, never]
-   - Service stopping
-   - Image pulling
-   - Container recreation
-   - Health verification
-
-Each task file is designed to be idempotent and can be run multiple times safely.

create_app_role.sh

@@ -0,0 +1,106 @@
+roles_path=../roles
+
+# Convert string to valid Ansible variable name
+convert_to_ansible_var() {
+    local input="$1"
+    
+    # Replace hyphens, and spaces with underscores
+    local result=$(echo "$input" | tr '-' '_' | tr ' ' '_')
+    
+    # Remove any characters that aren't alphanumeric or underscore
+    result=$(echo "$result" | tr -cd '[:alnum:]_')
+    
+    # Ensure it doesn't start with a number (prepend underscore if it does)
+    if [[ "$result" =~ ^[0-9] ]]; then
+        result="_${result}"
+    fi
+    
+    # Convert to lowercase (Ansible convention)
+    result=$(echo "$result" | tr '[:upper:]' '[:lower:]')
+    
+    echo "$result"
+}
+
+# Check if any arguments were provided
+if [ $# -eq 0 ]; then
+    echo "No arguments provided."
+    echo "Usage: $0 role1 role2 ..."
+    exit 1
+fi
+
+pushd $roles_path
+
+for arg in "$@"; do
+    echo "Creating directory $arg"
+    mkdir $arg
+    cd $arg
+    echo "Creating subdirectories"
+    mkdir {defaults,templates,meta}
+    echo "Creating main.yml files"
+    touch {defaults,meta}/main.yml
+    touch templates/compose.yml.j2
+    app_name=$(convert_to_ansible_var "$arg")
+
+    # create files
+    cat > defaults/main.yml << EOF
+---
+app_name: $arg
+
+# Container configuration
+${app_name}_container_name: "{{ app_name | default('$arg') }}"
+${app_name}_container_version: latest
+${app_name}_restart_policy: "{{ app_restart_policy }}"
+
+# Network configuration
+# ${app_name}_http_port: 8080
+
+# Volume paths
+# ${app_name}_config_path: "{{ app_dir }}/config"
+# ${app_name}_data_path: "{{ app_dir }}/data"
+
+# App-specific configuration
+# Add custom variables here
+
+# Directory structure
+# app_subdirectories:
+#   - config
+#   - data
+
+# Backup configuration
+# app_backup_subdirectories:
+#   - config
+EOF
+    
+
+    cat > meta/main.yml << EOF
+---
+galaxy_info:
+  author: <your_name>
+  description: Deploy $arg with Docker Compose
+  license: MIT
+
+dependencies:
+  - role: patrickj.docker_compose_app.app
+    vars:
+      app_role_name: $arg
+
+EOF
+
+    cat > templates/compose.yml.j2 << EOF
+---
+services:
+  ${arg}:
+    image: "your-image:{{ ${app_name}_container_version }}"
+    container_name: "{{ ${app_name}_container_name }}"
+    restart: "{{ ${app_name}_restart_policy }}"
+    # ports:
+    #   - "{{ ${app_name}_http_port }}:8080"
+    # volumes:
+    #   - "{{ ${app_name}_config_path }}:/config"
+    #   - "{{ ${app_name}_data_path }}:/data"
+EOF
+
+    cd ..
+
+done
+popd

galaxy.yml

@@ -0,0 +1,19 @@
+---
+namespace: patrickj
+name: docker_compose_app
+version: 1.0.0
+readme: README.md
+authors:
+  - Patrick Jaroszewski
+description: Base Ansible role for deploying Docker Compose applications
+license:
+  - MIT
+tags:
+  - docker
+  - compose
+  - containers
+dependencies:
+  "community.docker": ">=1.10.0"
+  "ansible.posix": ">=1.4.0"
+repository: https://git.jaroszew.ski/ansible/ansible-collection-docker-compose-app
+issues: https://git.jaroszew.ski/ansible/ansible-collection-docker-compose-app/issues

roles/app/README.md

@@ -0,0 +1,435 @@
+# Docker Compose App Role
+
+A flexible Ansible role for deploying and managing Docker Compose applications with features including backup/restore and healthchecks.
+
+## Overview
+
+This role provides a standardized way to deploy containerized applications using Docker Compose. It handles directory creation, template rendering, and container lifecycle management.
+
+## Requirements
+
+- Collections: `community.docker` (>=1.10.0) and `ansible.posix` (>=1.4.0). Both are declared as collection dependencies and installed automatically with the collection.
+- The backup workflow copies archives to the controller with `ansible.posix.synchronize`, so **`rsync` must be installed on both the controller and the managed host**.
+
+## Features
+
+- **Template-based deployment** with Jinja2 templating
+- **Dual backup system** (controller + remote host)
+- **Health checks** and deployment verification
+- **Multiple deployment modes** (template, file, or inline content)
+- **Flexible directory management**
+- **Configurable backup retention policies**
+
+## Quick Start
+
+### 1. Create a New Role
+
+Use the provided script to create a new application skeleton role:
+
+```bash
+./create_app_role.sh my-app
+```
+
+This creates:
+```
+roles/my-app/
+├── defaults/main.yml     # Default variables
+├── meta/main.yml         # Role metadata and dependencies
+└── templates/
+    └── compose.yml.j2    # Docker Compose template
+```
+
+### 2. Configure Your Application
+
+Edit `roles/my-app/defaults/main.yml`:
+
+```yaml
+---
+app_role_name: my-app
+
+my_app_container_name: "{{ app_name | default('my-app') }}"
+my_app_container_version: latest
+my_app_restart_policy: "{{ app_restart_policy }}"
+my_app_http_port: 8080
+my_app_data_path: "{{ app_dir }}/data"
+my_app_config_path: "{{ app_dir }}/config"
+
+# Optional: directories to create
+app_subdirectories:
+  - "{{ my_app_data_path }}"
+  - "{{ my_app_config_path }}"
+
+# Optional: backup configuration
+app_backup_subdirectories:
+  - "{{ my_app_data_path }}"
+  - "{{ my_app_config_path }}"
+```
+
+### 3. Create Docker Compose Template
+
+Edit `roles/my-app/templates/compose.yml.j2`:
+
+```yaml
+---
+services:
+  my-app:
+    image: "my-app:{{ my_app_container_version }}"
+    container_name: "{{ my_app_container_name }}"
+    restart: "{{ my_app_restart_policy }}"
+    ports:
+      - "{{ my_app_http_port }}:8080"
+    volumes:
+      - "{{ my_app_data_path }}:/data"
+      - "{{ my_app_config_path }}:/config"
+    environment:
+      - TZ={{ app_timezone | default('UTC') }}
+```
+
+### 4. Deploy Your Application
+
+Add to your playbook:
+
+```yaml
+---
+- name: Deploy my application
+  hosts: localhost
+  roles:
+    - role: my-app
+```
+
+## Advanced Usage
+
+### Backup and Restore
+
+#### Create a New Backup
+
+Create an on-demand backup using tags:
+
+```bash
+# Backup a specific application
+ansible-playbook --tags backup playbook.yml
+```
+
+Configure backup retention and options in your role's meta vars:
+
+```yaml
+# In roles/my-app/meta/main.yml
+dependencies:
+  - role: patrickj.docker_apps.docker_compose_app
+    vars:
+      app_backup_subdirectories:
+        - "{{ my_app_data_path }}"
+        - "{{ my_app_config_path }}"
+      app_backup_retention_days_controller: 90  # Keep backups for 90 days
+      app_backup_retention_days_remote: 7       # Keep remote backups for 7 days  
+      app_backup_stop_services: true            # Stop containers during backup
+```
+
+When `app_backup_stop_services` is enabled the containers are stopped only while the tarball is created on the managed host, then restarted immediately — the transfer of the archive back to the controller (via rsync) happens after the application is running again, so downtime is limited to the snapshot itself.
+
+#### Automated Backups
+
+For scheduled backups:
+
+```bash
+# Run daily backup at 2 AM - only runs backup tasks
+0 2 * * * ansible-playbook -i inventory --tags backup my-playbook.yml
+```
+
+### Tag-Based Execution
+
+The role supports selective execution using tags:
+
+```bash
+# Setup only - creates directories and renders the compose file, but doesn't start the containers
+ansible-playbook --tags setup playbook.yml
+
+# Full deployment - setup + start containers
+ansible-playbook --tags deploy playbook.yml
+
+# Restore and deploy
+ansible-playbook --tags restore,deploy playbook.yml
+
+# Backup only
+ansible-playbook --tags backup playbook.yml
+
+# Update only
+ansible-playbook --tags update playbook.yml
+
+# Backup then update
+ansible-playbook --tags backup,update playbook.yml
+```
+
+#### Restore from Backup
+
+Restore from the latest backup:
+
+```bash
+# Setup directories, restore from backup, then deploy
+ansible-playbook --tags setup,restore,deploy playbook.yml
+```
+
+Configure restore options in your role's meta vars:
+
+```yaml
+# In roles/my-app/meta/main.yml
+dependencies:
+  - role: patrickj.docker_apps.docker_compose_app
+    vars:
+      app_restore_source: controller  # or 'remote'
+      app_restore_max_age_days: 30    # Only restore backups newer than 30 days
+```
+
+Restore from a specific archive by setting the variable at runtime:
+
+```bash
+ansible-playbook --tags restore playbook.yml \
+  -e app_restore_archive=/path/to/specific/backup-20240315-120000.tar.gz
+```
+
+### Health Checks
+
+Configure health monitoring:
+
+```yaml
+- role: my-app
+  vars:
+    app_name: my-app
+    app_health_check: true
+    app_health_check_method: http
+    app_health_check_url: "http://localhost:8080/health"
+    app_health_check_retries: 30
+    app_health_check_delay: 10
+```
+
+### Updates
+
+Update application containers using tags:
+
+```bash
+# Update a specific application
+ansible-playbook --tags update playbook.yml
+
+# Update all applications in a playbook
+ansible-playbook --tags update site.yml
+```
+
+**Update process:**
+1. Stops the application
+2. Pulls images according to `app_compose_pull` (defaults to `policy`; set to `always` to force fresh pulls)
+3. Recreates containers with new images
+4. Runs health checks (if enabled)
+
+
+### Rollbacks
+
+Rollback using the restore functionality:
+
+```bash
+# First, create a backup then update
+ansible-playbook --tags backup,update playbook.yml
+
+# If update fails, rollback from backup
+ansible-playbook --tags restore playbook.yml
+```
+
+**Important Notes:**
+- Updates do NOT create automatic backups - create backups manually beforehand
+- Rollbacks must be triggered manually using the restore functionality
+- Using `latest` tags prevents effective rollbacks since the old image is overwritten
+- For production, use specific version tags instead of `latest`
+
+
+### Multiple Instances
+
+Deploy multiple instances of the same application:
+
+```yaml
+- role: jellyfin
+  vars:
+    app_name: jellyfin-movies
+    jellyfin_http_port: 8096
+
+- role: jellyfin  
+  vars:
+    app_name: jellyfin-tv
+    jellyfin_http_port: 8097
+```
+
+
+### Custom Templates and Files
+
+#### Override template location:
+
+```yaml
+- role: my-app
+  vars:
+    app_compose_template: /path/to/custom/compose.yml.j2
+```
+
+#### Use a static compose file:
+
+If you have an existing `docker-compose.yml` file that doesn't need templating:
+
+```yaml
+- role: my-app
+  vars:
+    app_name: my-app
+    app_compose_file: /path/to/existing/docker-compose.yml
+```
+
+The file will be copied as-is to the application directory without Jinja2 processing.
+
+#### Use inline content:
+
+```yaml
+- role: patrickj.docker_apps.docker_compose_app
+  vars:
+    app_name: simple-app
+    app_role_name: simple-app
+    app_compose_content: |
+      services:
+        app:
+          image: nginx:alpine
+          ports:
+            - "80:80"
+```
+
+## Configuration Reference
+
+### Required Externals
+
+These variables have no defaults and must be set by the calling playbook (typically as group_vars or host_vars). They are consumed by the defaults shown further down; if you override every derived variable that uses them, you can leave the corresponding external unset.
+
+| Variable | Description | Consumed by |
+|----------|-------------|-------------|
+| `host_root_path` | Base directory on the **target host** under which each app's directory is created (e.g. `/srv/apps`, `/opt/docker`) | `app_dir` |
+| `backup_path` | Base directory on the **controller** where fetched backup archives are stored (e.g. `~/backups`) | `app_backup_path` |
+
+The `templates/` directory of the calling shim role is discovered automatically via `ansible_parent_role_paths` — no path variable to set.
+
+### Core Variables
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `app_name` | Required | Unique application instance name |
+| `app_dir` | `{{ host_root_path }}/{{ app_name }}` | Application directory on the target host |
+| `app_uid` | `{{ ansible_facts.user_uid }}` | File ownership UID |
+| `app_gid` | `{{ ansible_facts.user_gid }}` | File ownership GID |
+| `app_permission_mode` | `"0640"` | File permission mode |
+
+### Deployment Options
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `app_compose_template` | `compose.yml.j2` | Template file path — relative names resolve via Ansible's role templates search path (finds the calling shim role's `templates/`); absolute paths also accepted |
+| `app_compose_file` | - | Static compose file path |
+| `app_compose_content` | - | Inline compose content |
+| `app_compose_validate` | `true` | Validate compose syntax |
+| `app_compose_pull` | `policy` | Image pull strategy (also used by `update` — set to `always` to force pulls on update) |
+| `app_compose_recreate` | `auto` | Container recreate strategy |
+| `app_compose_project_name` | `{{ app_dir \| basename }}` | Compose project name; override when your compose file sets a non-default project name |
+
+### Backup Configuration
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `app_backup` | `true` | Whether backups run when invoked with `--tags backup`; set `false` to opt a host out |
+| `app_backup_subdirectories` | `[]` | Specific directories to backup |
+| `app_backup_stop_services` | `true` | Stop containers during backup |
+| `app_backup_retention_days_controller` | `90` | Controller backup retention |
+| `app_backup_retention_days_remote` | `7` | Remote backup retention |
+
+### Restore Configuration
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `app_restore_source` | `controller` | Restore source (`controller`/`remote`) |
+| `app_restore_archive` | - | Specific archive file to restore (overrides latest) |
+| `app_restore_max_age_days` | `30` | Maximum backup age for restore (when finding latest) |
+
+### Health Check Configuration
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `app_health_check` | `true` | Enable health checks |
+| `app_health_check_method` | `docker` | Check method (`docker`/`http`) |
+| `app_container_name` | `{{ app_name }}` | Container name inspected by the `docker` health check; override when your compose service uses a different `container_name` |
+| `app_health_check_url` | - | HTTP endpoint for health check |
+| `app_health_check_retries` | `30` | Number of check retries |
+| `app_health_check_delay` | `10` | Delay between checks (seconds) |
+| `app_health_check_status_codes` | `[200, 201, 202]` | Valid HTTP status codes |
+
+### Directory Management
+
+| Variable | Default | Description |
+|----------|---------|-------------|
+| `app_subdirectories` | `[]` | Directories to create in app_dir |
+| `app_extra_templates` | `[]` | Additional templates to render |
+
+Example `app_extra_templates`:
+```yaml
+app_extra_templates:
+  - src: config.json.j2
+    dest: "{{ app_dir }}/config/app.json"
+  - src: env.j2  
+    dest: "{{ app_dir }}/.env"
+```
+
+
+## Best Practices
+
+1. **Use semantic versioning** for container tags instead of `latest`
+2. **Test restore procedures** regularly to ensure backups are valid
+3. **Use specific directories** for backups instead of entire app directory
+4. **Monitor disk usage** on backup storage locations
+
+## Architecture
+
+The role is organized into logical task files for selective execution:
+
+```
+├── setup.yml              # Validation, directories, and templates
+├── restore.yml            # Backup restoration logic
+├── deploy.yml             # Docker Compose deployment and startup
+├── health_check.yml       # Health monitoring and verification
+├── backup.yml             # Backup creation and management
+├── update.yml             # Container update and image pulling
+└── manage_compose.yml     # Docker Compose lifecycle management
+```
+
+### Task Execution Flow
+
+1. **Setup** (`setup.yml`) - [setup, deploy]
+   - Docker/Compose validation
+   - Variable validation and security checks
+   - Directory creation
+   - Additional template deployment
+
+2. **Restore** (`restore.yml`) - [restore, never]
+   - Backup file selection and validation
+   - Service stopping
+   - Archive extraction
+   - Service restart
+
+3. **Deploy** (`deploy.yml`) - [deploy]
+   - Docker Compose file creation
+   - Container startup and management
+
+4. **Health Check** (`health_check.yml`) - [deploy, healthcheck]
+   - Service health verification
+   - Endpoint monitoring
+
+5. **Backup** (`backup.yml`) - [backup, never]
+   - Service stopping (optional)
+   - Archive creation
+   - Backup copying and cleanup
+   - Service restart
+
+6. **Update** (`update.yml`) - [update, never]
+   - Service stopping
+   - Image pulling
+   - Container recreation
+   - Health verification
+
+Each task file is designed to be idempotent and can be run multiple times safely.

roles/app/defaults/main.yml

@@ -2,12 +2,13 @@
 app_dir: "{{ host_root_path }}/{{ app_name }}"
 app_uid: "{{ ansible_facts.user_uid }}"
 app_gid: "{{ ansible_facts.user_gid }}"
-app_permission_mode: "0640"
+app_permission_mode: "0640"    # files (compose.yml, rendered templates)
+app_dir_mode: "0755"           # directories (app_dir and app_subdirectories)
 app_restart_policy: unless-stopped
 
 app_backup_path: "{{ backup_path }}/{{ app_name }}"
 app_backup_path_remote: "{{ app_dir }}/backups"
-app_backup: false
+app_backup: true
 app_backup_subdirectories: []
 app_backup_stop_services: true
 app_backup_retention_days_controller: 90
@@ -21,15 +22,19 @@ app_compose_validate: true
 app_compose_pull: policy
 app_compose_recreate: auto
 app_compose_start: true
+app_compose_project_name: "{{ app_dir | basename }}"
 
+app_container_name: "{{ app_name }}"
 app_health_check: true
 app_health_check_method: docker
 app_health_check_retries: 30
 app_health_check_delay: 10
 app_health_check_status_codes: [200, 201, 202]
 
-app_templates_path: "{{ app_roles_path + '/' + app_role_name }}/templates"
+# Relative src — Ansible's template resolver searches loaded roles' templates/
-app_compose_template: "{{ app_templates_path }}/compose.yml.j2"
+# directories, so this finds the calling (shim) role's compose.yml.j2 without
+# any absolute-path plumbing.
+app_compose_template: compose.yml.j2
 
 app_subdirectories: []
 # - config

roles/app/meta/main.yml

@@ -7,3 +7,4 @@ galaxy_info:
 
 collections:
   - community.docker
+  - ansible.posix

roles/app/tasks/backup.yml

@@ -26,6 +26,7 @@
         state: directory
         mode: "0750"
       delegate_to: localhost
+      become: false
 
     - name: Create remote backup directory on host
       ansible.builtin.file:
@@ -51,12 +52,19 @@
         mode: "{{ app_permission_mode | default('0644') }}"
       register: remote_backup
 
+    - name: Restart application after snapshot
+      community.docker.docker_compose_v2:
+        project_src: "{{ app_dir }}"
+        state: present
+      when: app_backup_stop_services | default(true)
+
     - name: Copy backup to controller
-      ansible.builtin.fetch:
+      ansible.posix.synchronize:
         src: "{{ remote_backup.dest }}"
         dest: "{{ app_backup_path }}/{{ app_name }}-{{ ansible_date_time.iso8601_basic_short }}.tar.gz"
-        flat: true
+        mode: pull
       when: remote_backup is succeeded
+      become: false
 
     - name: Clean old backups on controller
       ansible.builtin.find:
@@ -65,6 +73,7 @@
         age: "{{ app_backup_retention_days_controller | default(90) }}d"
       register: old_backups_controller
       delegate_to: localhost
+      become: false
 
     - name: Remove old backups from controller
       ansible.builtin.file:
@@ -72,6 +81,7 @@
         state: absent
       loop: "{{ old_backups_controller.files | default([]) }}"
       delegate_to: localhost
+      become: false
 
     - name: Clean old backups on remote host
       ansible.builtin.find:
@@ -87,7 +97,7 @@
       loop: "{{ old_backups_remote.files | default([]) }}"
 
   always:
-    - name: Restart application after backup
+    - name: Ensure application is running (safety net)
       community.docker.docker_compose_v2:
         project_src: "{{ app_dir }}"
         state: present

roles/app/tasks/deploy.yml

@@ -0,0 +1,6 @@
+---
+- name: "{{ app_name }} - Deploy Docker Compose configuration"
+  block:
+    - ansible.builtin.include_tasks: manage_compose.yml
+      when: app_compose_start | default(true)
+  tags: [deploy]

roles/app/tasks/health_check.yml

@@ -0,0 +1,27 @@
+---
+- name: Verify application HTTP endpoint
+  ansible.builtin.uri:
+    url: "{{ app_health_check_url }}"
+    method: GET
+    status_code: "{{ app_health_check_status_codes | default([200, 201, 202]) }}"
+  register: http_health_check
+  until: http_health_check.status in (app_health_check_status_codes | default([200, 201, 202]))
+  retries: "{{ app_health_check_retries | default(30) }}"
+  delay: "{{ app_health_check_delay | default(10) }}"
+  when: app_health_check | default(true) and app_health_check_method | default('docker') == 'http' and app_health_check_url is defined
+
+- name: Wait for compose services to be running (and healthy if applicable)
+  ansible.builtin.command:
+    cmd: docker compose ps --all --format json
+    chdir: "{{ app_dir }}"
+  register: _compose_ps
+  changed_when: false
+  retries: "{{ app_health_check_retries | default(30) }}"
+  delay: "{{ app_health_check_delay | default(10) }}"
+  vars:
+    _services: "{{ _compose_ps.stdout_lines | map('from_json') | list }}"
+  until:
+    - _services | length > 0
+    - _services | rejectattr('State', 'equalto', 'running') | list | length == 0
+    - _services | selectattr('Health', 'defined') | selectattr('Health', 'in', ['unhealthy', 'starting']) | list | length == 0
+  when: app_health_check | default(true) and app_health_check_method | default('docker') == 'docker'

roles/app/tasks/main.yml

@@ -0,0 +1,14 @@
+---
+
+- ansible.builtin.include_tasks: setup.yml
+  tags: [setup, deploy]
+- ansible.builtin.include_tasks: restore.yml
+  tags: [restore, never]
+- ansible.builtin.include_tasks: deploy.yml
+  tags: [deploy]
+- ansible.builtin.include_tasks: health_check.yml
+  tags: [deploy, healthcheck]
+- ansible.builtin.include_tasks: backup.yml
+  tags: [backup, never]
+- ansible.builtin.include_tasks: update.yml
+  tags: [update, never]

roles/app/tasks/restore.yml

@@ -52,18 +52,16 @@
       when: app_restore_source | default('controller') == 'remote' and remote_backups.files | length > 0 and app_restore_archive is not defined
 
     - name: Copy backup from controller to remote if needed
-      ansible.builtin.copy:
+      ansible.posix.synchronize:
         src: "{{ _restore_backup }}"
         dest: "{{ app_dir }}/restore_backup.tar.gz"
-        owner: "{{ app_uid | default(omit) }}"
+        mode: push
-        group: "{{ app_gid | default(omit) }}"
-        mode: "{{ app_permission_mode | default('0644') }}"
       when: app_restore_source | default('controller') == 'controller' and _restore_backup is defined
       register: copied_backup
 
     - name: Set restore path for remote backup
       ansible.builtin.set_fact:
-        _restore_path: "{{ _restore_backup if app_restore_source | default('controller') == 'remote' else copied_backup.dest }}"
+        _restore_path: "{{ _restore_backup if app_restore_source | default('controller') == 'remote' else app_dir + '/restore_backup.tar.gz' }}"
       when: _restore_backup is defined
 
     - name: Stop application for restore
@@ -72,21 +70,6 @@
         state: stopped
       when: _restore_path is defined
 
-    - name: Debug directory permissions before deletion
-      ansible.builtin.shell: |
-        ls -la "{{ item }}"
-        ls -la "{{ item | dirname }}"
-        whoami
-        id
-      loop: "{{ app_backup_subdirectories | default([]) }}"
-      when: _restore_path is defined
-      become: true
-      register: debug_perms
-
-    - name: Show debug output
-      ansible.builtin.debug:
-        var: debug_perms
-
     - name: Remove existing backup subdirectories for clean restore
       ansible.builtin.file:
         path: "{{ item }}"

roles/app/tasks/setup.yml

@@ -18,11 +18,12 @@
         that:
           - app_name is defined
           - app_name | length > 0
-          - app_name is match('^[a-zA-Z0-9_-]+$')
+          - app_name is match('^[a-zA-Z0-9_.-]+$')
           - host_root_path is defined
           - host_root_path | length > 0
-          - host_root_path is match('^/[a-zA-Z0-9/_-]*$')
+          - host_root_path is match('^/[a-zA-Z0-9./_-]*$')
-        fail_msg: "Validation failed for {{ app_name | default('undefined') }}: Required variables missing or contain invalid characters. app_name and host_root_path must be defined and contain only alphanumeric, underscore, hyphen, and slash characters."
+          - not (host_root_path is match('.*\.\..*'))
+        fail_msg: "Validation failed for {{ app_name | default('undefined') }}: Required variables missing or contain invalid characters. app_name and host_root_path must be defined, contain only alphanumeric, dot, underscore, hyphen, and slash characters, and must not contain path traversal sequences."
 
     - name: Set compose source type
       ansible.builtin.set_fact:
@@ -43,25 +44,10 @@
     - name: Validate directory paths
       ansible.builtin.assert:
         that:
-          - app_dir is match('^/[a-zA-Z0-9/_-]*$')
+          - app_dir is match('^/[a-zA-Z0-9./_-]*$')
           - not (app_dir is match('.*\.\..*'))
         fail_msg: "Security validation failed for {{ app_name }}: app_dir '{{ app_dir }}' contains invalid characters or path traversal sequences"
 
-    - name: Check if compose template exists
-      ansible.builtin.stat:
-        path: "{{ app_compose_template }}"
-      register: _compose_template_stat
-      delegate_to: localhost
-      become: false
-      when: app_compose_template is defined and app_compose_template | length > 0
-
-    - name: Assert compose template exists
-      ansible.builtin.assert:
-        that:
-          - _compose_template_stat.stat.exists
-        fail_msg: "Template file not found for {{ app_name }}: '{{ app_compose_template }}' does not exist"
-      when: app_compose_template is defined and app_compose_template | length > 0
-
     - name: Check if compose file exists
       ansible.builtin.stat:
         path: "{{ app_compose_file }}"
@@ -86,16 +72,58 @@
         state: directory
         owner: "{{ app_uid }}"
         group: "{{ app_gid }}"
-        mode: "{{ app_permission_mode }}"
+        mode: "{{ app_dir_mode }}"
 
+    # Items may be plain path strings, or dicts with per-item overrides:
+    #   - /some/path                                  # uses app_uid/app_gid/app_dir_mode
+    #   - path: /some/path                            # same
+    #     owner: 999                                  # per-item overrides for containers
+    #     group: 999                                  # that expect a specific uid/gid
+    #     mode: "0700"
     - name: Create application subdirectories
       ansible.builtin.file:
-        path: "{{ item }}"
+        path: "{{ item.path | default(item) }}"
         state: directory
+        owner: "{{ item.owner | default(app_uid) }}"
+        group: "{{ item.group | default(app_gid) }}"
+        mode: "{{ item.mode | default(app_dir_mode) }}"
+      loop: "{{ app_subdirectories }}"
+  tags: [setup, deploy]
+
+- name: "{{ app_name }} - Render Docker Compose configuration"
+  block:
+    - name: Copy Docker Compose template
+      ansible.builtin.template:
+        src: "{{ app_compose_template }}"
+        dest: "{{ app_compose_dest }}"
+        mode: "{{ app_permission_mode }}"
         owner: "{{ app_uid }}"
         group: "{{ app_gid }}"
+        validate: "{{ 'docker compose -f %s config -q' if app_compose_validate else omit }}"
+      when: _compose_type == 'template'
+      register: compose_file
+
+    - name: Copy Docker Compose file
+      ansible.builtin.copy:
+        src: "{{ app_compose_file }}"
+        dest: "{{ app_compose_dest }}"
         mode: "{{ app_permission_mode }}"
-      loop: "{{ app_subdirectories }}"
+        owner: "{{ app_uid }}"
+        group: "{{ app_gid }}"
+        validate: "{{ 'docker compose -f %s config -q' if app_compose_validate else omit }}"
+      when: _compose_type == 'file'
+      register: compose_file
+
+    - name: Copy Docker Compose content from YAML string
+      ansible.builtin.copy:
+        content: "{{ app_compose_content }}"
+        dest: "{{ app_compose_dest }}"
+        mode: "{{ app_permission_mode }}"
+        owner: "{{ app_uid }}"
+        group: "{{ app_gid }}"
+        validate: "{{ 'docker compose -f %s config -q' if app_compose_validate else omit }}"
+      when: _compose_type == 'content'
+      register: compose_file
   tags: [setup, deploy]
 
 - name: "{{ app_name }} - Deploy additional templates"

roles/app/tasks/update.yml

@@ -9,7 +9,7 @@
     - name: Pull latest container images
       community.docker.docker_compose_v2:
         project_src: "{{ app_dir }}"
-        pull: always
+        pull: "{{ app_compose_pull | default('always') }}"
 
     - name: Start application with new images
       community.docker.docker_compose_v2:
@@ -19,7 +19,7 @@
       register: update_restart
 
     - name: Verify application health after update
-      ansible.builtin.include_tasks: health_check.yaml
+      ansible.builtin.include_tasks: health_check.yml
       when: app_health_check | default(true)
 
   rescue:

tasks/deploy.yaml

@@ -1,39 +0,0 @@
----
-- name: "{{ app_name }} - Deploy Docker Compose configuration"
-  block:
-    - name: Copy Docker Compose template
-      ansible.builtin.template:
-        src: "{{ app_compose_template }}"
-        dest: "{{ app_compose_dest }}"
-        mode: "{{ app_permission_mode }}"
-        owner: "{{ app_uid }}"
-        group: "{{ app_gid }}"
-        validate: "{{ 'docker compose -f %s config -q' if app_compose_validate else omit }}"
-      when: _compose_type == 'template'
-      register: compose_file
-
-    - name: Copy Docker Compose file
-      ansible.builtin.copy:
-        src: "{{ app_compose_file }}"
-        dest: "{{ app_compose_dest }}"
-        mode: "{{ app_permission_mode }}"
-        owner: "{{ app_uid }}"
-        group: "{{ app_gid }}"
-        validate: "{{ 'docker compose -f %s config -q' if app_compose_validate else omit }}"
-      when: _compose_type == 'file'
-      register: compose_file
-
-    - name: Copy Docker Compose content from YAML string
-      ansible.builtin.copy:
-        content: "{{ app_compose_content }}"
-        dest: "{{ app_compose_dest }}"
-        mode: "{{ app_permission_mode }}"
-        owner: "{{ app_uid }}"
-        group: "{{ app_gid }}"
-        validate: "{{ 'docker compose -f %s config -q' if app_compose_validate else omit }}"
-      when: _compose_type == 'content'
-      register: compose_file
-
-    - ansible.builtin.include_tasks: manage_compose.yaml
-      when: app_compose_start | default(true)
-  tags: [deploy]

tasks/health_check.yaml

@@ -1,49 +0,0 @@
----
-- name: Wait for containers to be healthy
-  community.docker.docker_container_info:
-    name: "{{ app_name }}"
-  register: container_info
-  until: container_info.container.State.Health.Status | default('healthy') == 'healthy'
-  retries: "{{ app_health_check_retries | default(30) }}"
-  delay: "{{ app_health_check_delay | default(10) }}"
-  when: app_health_check | default(true) and app_health_check_method | default('docker') == 'docker'
-  ignore_errors: true
-
-- name: Verify application HTTP endpoint
-  ansible.builtin.uri:
-    url: "{{ app_health_check_url }}"
-    method: GET
-    status_code: "{{ app_health_check_status_codes | default([200, 201, 202]) }}"
-  register: http_health_check
-  until: http_health_check.status in (app_health_check_status_codes | default([200, 201, 202]))
-  retries: "{{ app_health_check_retries | default(30) }}"
-  delay: "{{ app_health_check_delay | default(10) }}"
-  when: app_health_check | default(true) and app_health_check_method | default('docker') == 'http' and app_health_check_url is defined
-  ignore_errors: true
-
-- name: Check if all containers are running
-  ansible.builtin.command:
-    cmd: docker compose ps --services --filter status=running
-    chdir: "{{ app_dir }}"
-  register: running_services
-  when: app_health_check | default(true)
-
-- name: Check total services
-  ansible.builtin.command:
-    cmd: docker compose ps --services
-    chdir: "{{ app_dir }}"
-  register: total_services
-  when: app_health_check | default(true)
-
-- name: Verify deployment success
-  ansible.builtin.assert:
-    that:
-      - running_services.stdout_lines | length == total_services.stdout_lines | length
-    fail_msg: "Not all containers are running. Running: {{ running_services.stdout_lines | length }}, Total: {{ total_services.stdout_lines | length }}"
-    success_msg: "All containers are running successfully."
-  when: app_health_check | default(true) and total_services.stdout_lines is defined
-
-- name: Log deployment status
-  ansible.builtin.debug:
-    msg: "Deployment {{ 'successful' if (running_services.stdout_lines | length == total_services.stdout_lines | length) else 'failed' }}"
-  when: app_health_check | default(true) and total_services.stdout_lines is defined

tasks/main.yaml

@@ -1,14 +0,0 @@
----
-
-- ansible.builtin.include_tasks: setup.yaml
-  tags: [setup, deploy]
-- ansible.builtin.include_tasks: restore.yaml
-  tags: [restore, never]
-- ansible.builtin.include_tasks: deploy.yaml
-  tags: [deploy]
-- ansible.builtin.include_tasks: health_check.yaml
-  tags: [deploy, healthcheck]
-- ansible.builtin.include_tasks: backup.yaml
-  tags: [backup, never]
-- ansible.builtin.include_tasks: update.yaml
-  tags: [update, never]