# TaskEngine Worker Pool Configuration

This directory contains launchd plist templates for running TaskEngine workers with different queue configurations.

## Priority Queue Strategy

The TaskEngine supports three priority queues:
- **high**: Critical, time-sensitive jobs (push notifications, SMS, real-time sync)
- **default**: Normal priority jobs (scheduled syncs, reports, metrics)
- **low**: Background jobs (cleanup, archival, batch imports)

### The Problem

If all workers process all queues, a long-running low-priority job can tie up a worker, preventing it from processing high-priority jobs. For example:
- Worker A picks up a 10-minute data import job (low priority)
- While Worker A is busy, a push notification job (high priority) arrives
- If all other workers are also busy with low-priority work, the push notification waits

### The Solution: Dedicated Priority Workers

By dedicating some workers to only process specific queues, you ensure high-priority jobs always have available workers:

| Worker Type | Queues | Purpose |
|-------------|--------|---------|
| `worker-high` | `high` only | Always available for critical jobs |
| `worker-default` | `high,default` | General processing, no low priority |
| `worker-all` | `high,default,low` | Catch-all including background jobs |

## Recommended Configuration

For a production system:

| Worker Type | Count | Purpose |
|-------------|-------|---------|
| High Priority | 2 | Dedicated to critical jobs |
| Default Priority | 4 | General job processing |
| All Queues | 2 | Handles low priority/background |

**Total: 8 workers**

## Installation (macOS)

1. **Copy templates to LaunchAgents:**
   ```bash
   cp scripts/launchd/com.buyerkiosk.taskengine-worker-*.plist ~/Library/LaunchAgents/
   ```

2. **Edit paths in plist files:**
   - Update `ProgramArguments` with correct PHP path
   - Update `WorkingDirectory` with project path
   - Update log paths as needed

3. **Create multiple workers by copying with unique labels:**
   ```bash
   # Create 2 high-priority workers
   for i in 1 2; do
       sed "s/worker-high/worker-high-$i/g" \
           scripts/launchd/com.buyerkiosk.taskengine-worker-high.plist \
           > ~/Library/LaunchAgents/com.buyerkiosk.taskengine-worker-high-$i.plist
   done

   # Create 4 default-priority workers
   for i in 1 2 3 4; do
       sed "s/worker-default/worker-default-$i/g" \
           scripts/launchd/com.buyerkiosk.taskengine-worker-default.plist \
           > ~/Library/LaunchAgents/com.buyerkiosk.taskengine-worker-default-$i.plist
   done

   # Create 2 all-queues workers
   for i in 1 2; do
       sed "s/worker-all/worker-all-$i/g" \
           scripts/launchd/com.buyerkiosk.taskengine-worker-all.plist \
           > ~/Library/LaunchAgents/com.buyerkiosk.taskengine-worker-all-$i.plist
   done
   ```

4. **Load the workers:**
   ```bash
   launchctl load ~/Library/LaunchAgents/com.buyerkiosk.taskengine-worker-*.plist
   ```

5. **Verify workers are running:**
   ```bash
   php userfrosting/bin/task worker:manager --status
   ```

## Management Commands

```bash
# Check worker status (shows queue assignments)
php userfrosting/bin/task worker:manager --status

# View queue depths
php userfrosting/bin/task queue:status --detailed

# Start all workers
launchctl load ~/Library/LaunchAgents/com.buyerkiosk.taskengine-worker-*.plist

# Stop all workers
launchctl unload ~/Library/LaunchAgents/com.buyerkiosk.taskengine-worker-*.plist

# Restart a specific worker
launchctl unload ~/Library/LaunchAgents/com.buyerkiosk.taskengine-worker-high-1.plist
launchctl load ~/Library/LaunchAgents/com.buyerkiosk.taskengine-worker-high-1.plist

# View worker logs
tail -f logs/task-worker-high.log
tail -f logs/task-worker-default.log
tail -f logs/task-worker-all.log
```

## Job Priority Guidelines

When defining jobs, assign queues based on urgency:

| Queue | Use For | Timeout |
|-------|---------|---------|
| `high` | Push notifications, SMS, real-time sync, alerts | < 60s |
| `default` | Scheduled syncs, reports, metrics aggregation | < 5min |
| `low` | Data cleanup, archival, batch imports, analytics | < 30min |

## Heartbeat Behavior

Workers now send heartbeats during long-running job execution:
- Default heartbeat throttle: Every 10 seconds
- Jobs can call `$this->checkpoint()` to heartbeat + check for abort
- This prevents workers from appearing stale during long jobs

## Monitoring

The worker status now shows which queues each worker monitors:

```
php userfrosting/bin/task worker:manager --status

Active Workers:
ID                                      Hostname         PID     Queues              Status      Last Heartbeat
---------------------------------------------------------------------------------------------------------------
a1b2c3d4-e5f6-7890-abcd-ef1234567890    MacBook-Pro      12345   high                idle        2024-01-15 10:30:45
b2c3d4e5-f6g7-8901-bcde-fg2345678901    MacBook-Pro      12346   high,default        busy        2024-01-15 10:30:42
c3d4e5f6-g7h8-9012-cdef-gh3456789012    MacBook-Pro      12347   high,default,low    idle        2024-01-15 10:30:44
```