# Server Hardening Backlog — buyerkiosk.com (50.28.85.71)
**Created**: 2026-04-03 (post-outage audit)
**Server**: AlmaLinux 9.7 | 64 cores (2x Xeon Gold 6130) | 251 GB RAM | 3.5 TB disk
---
## Completed (2026-04-03)
- [x] **P0**: Cleaned up Apache MPM config conflicts — removed duplicate directives from `pre_main_global.conf`, consolidated to `worker_performance_override.conf` as single source of truth
- [x] **P0**: Fixed httpd LimitNOFILESoft (1,024 → 524,288) — created `/etc/systemd/system/httpd.service.d/limits.conf`
- [x] **P1**: Reduced MySQL wait_timeout (60 → 30) — persisted in `/etc/my.cnf`
- [x] **P1**: Expanded ephemeral port range (28K → 64K ports) — persisted in `/etc/sysctl.d/99-buyerkiosk-tcp.conf`
- [x] **P1**: Reduced tcp_fin_timeout (60 → 30), tcp_keepalive_time (7200 → 600)
- [x] **P1**: Reduced vm.swappiness (60 → 10)
- [x] **P1**: Capped PHP-FPM pm.max_children (2,500 → 1,000) — prevents OOM if all workers spawn (2026-04-09)
---
## Remaining Items
### P2 — Should Address Soon
#### 1. OPcache Undersized for Codebase
**Risk**: PHP recompiling scripts on every evicted file — wasted CPU
**Current**:
```ini
opcache.max_accelerated_files = 4000 # too low for 239+ classes + deps
opcache.interned_strings_buffer = 8 # MB, default
opcache.memory_consumption = 128 # MB
```
**Target**:
```ini
opcache.max_accelerated_files = 20000
opcache.interned_strings_buffer = 32
opcache.memory_consumption = 256
```
**File**: `/opt/cpanel/ea-php84/root/etc/php.ini` (or via WHM MultiPHP INI Editor)
**Restart**: `systemctl restart ea-php84-php-fpm`
---
#### 2. MySQL table_definition_cache Too Low for Multi-DB
**Risk**: Cache thrashing with many store databases (kiosk_ou00, kiosk_pc80xxx, etc.)
**Current**:
```
table_definition_cache = 400
table_open_cache = 2000
```
**Target**:
```
table_definition_cache = 4000
table_open_cache = 4000
```
**File**: `/etc/my.cnf` under `[mysqld]`
**Apply runtime**: `SET GLOBAL table_definition_cache = 4000; SET GLOBAL table_open_cache = 4000;`
---
#### 3. Security: Root Filesystem Exposed via Apache
**Risk**: `00-buyerkiosk-urgent-fix.conf` grants Apache access to entire filesystem
**Current** (`/etc/apache2/conf.d/00-buyerkiosk-urgent-fix.conf`):
```apache
Require all granted
AllowOverride All
Options All # enables directory listing, CGI, SSI, etc.
```
**Target**: Remove the `` block entirely. The blocks for `/home/bkweb` and `/home/bkweb/public_html` in the same file are sufficient. Replace `Options All` with `Options FollowSymLinks` on the remaining blocks.
**Test**: `apachectl configtest` then `apachectl graceful`, verify site loads
---
#### 4. mod_evasive Duplicate Config Blocks
**Risk**: Two `` blocks with conflicting thresholds — unpredictable behavior
**File**: `/etc/apache2/conf.d/mod_evasive_custom.conf`
**Current**: First block has DOSPageCount 4, second has DOSPageCount 500
**Target**: Remove the first (restrictive cPanel default) block, keep only the tuned lenient config
---
#### 5. InnoDB I/O Capacity (if SSD)
**Risk**: Dirty page flushing throttled at spinning-disk speeds on SSD hardware
**Current**:
```
innodb_io_capacity = 200
innodb_io_capacity_max = 2000
```
**Target** (verify SSD first with `lsblk -d -o name,rota` — rota=0 means SSD):
```
innodb_io_capacity = 2000
innodb_io_capacity_max = 10000
```
**File**: `/etc/my.cnf` under `[mysqld]`
**Apply runtime**: `SET GLOBAL innodb_io_capacity = 2000; SET GLOBAL innodb_io_capacity_max = 10000;`
---
### P3 — Address When Convenient
#### 6. Redis maxmemory Only 1GB
**Risk**: Future eviction ceiling if caching/session usage grows
**Current**: `maxmemory 1073741824` (1GB), using 5.26MB
**Target**: `maxmemory 4294967296` (4GB)
**File**: `/etc/redis/redis.conf`
**Apply runtime**: `redis-cli CONFIG SET maxmemory 4294967296`
**Restart**: `systemctl restart redis`
---
#### 7. Log Rotation Policy
**Risk**: Logs consuming disk/IO and slowing backups
**Current sizes**:
```
5.0 GB /home/bkweb/logs/stores
2.8 GB buyerkiosk.com-ssl_log-Mar-2026.gz (compressed!)
1.3 GB dev_log.txt.1.gz
```
**Target**: Create `/etc/logrotate.d/buyerkiosk` with:
- Access logs: rotate daily, keep 7 days, compress
- Error logs: rotate weekly, keep 30 days, compress
- App logs (stores/): rotate daily, keep 14 days, compress
---
#### 8. max_input_vars PHP Limit
**Risk**: Forms with 1000+ fields silently truncate data (bulk edit pages)
**Current**: `max_input_vars = 1000`
**Target**: `max_input_vars = 5000`
**File**: php.ini or WHM MultiPHP INI Editor
---
#### 9. Network Buffer Sizes
**Risk**: Suboptimal throughput under high load
**Current**:
```
net.core.rmem_max = 212992 (208KB)
net.core.wmem_max = 212992 (208KB)
```
**Target**:
```
net.core.rmem_max = 16777216 (16MB)
net.core.wmem_max = 16777216 (16MB)
```
**File**: Append to `/etc/sysctl.d/99-buyerkiosk-tcp.conf`
---
#### 10. Redis Version End-of-Life
**Current**: Redis 6.2.20 (EOL)
**Target**: Redis 7.x (significant performance improvements)
**Notes**: Plan upgrade during maintenance window, test with staging first
---
## Server Quick Reference
| Service | Config File(s) | Restart Command |
|---------|---------------|-----------------|
| Apache MPM | `/etc/apache2/conf.d/worker_performance_override.conf` | `apachectl graceful` |
| Apache systemd | `/etc/systemd/system/httpd.service.d/limits.conf` | `systemctl daemon-reload && systemctl restart httpd` |
| PHP-FPM pool | `/opt/cpanel/ea-php84/root/etc/php-fpm.d/buyerkiosk.com.conf` | `systemctl restart ea-php84-php-fpm` |
| MySQL/MariaDB | `/etc/my.cnf` | `systemctl restart mariadb` |
| Redis | `/etc/redis/redis.conf` | `systemctl restart redis` |
| Sysctl tuning | `/etc/sysctl.d/99-buyerkiosk-tcp.conf` | `sysctl --system` |
| Kernel limits | `/etc/security/limits.conf` | Relogin/reboot |
## SSH Access
```bash
ssh root@50.28.85.71 # uses ~/.ssh/id_rsa
```