# Lobster Workflows Documentation

## Overview

This repository contains personal automation workflows powered by the Lobster workflow engine. All workflows are scheduled via systemd user timers and run autonomously.

## Repository Structure

```
lobster-workflows/
├── workflows/          # .lobster workflow definitions (31 total)
├── scripts/            # Helper scripts called by workflows
└── docs/              # Documentation
```

## Active Workflows (17 with timers)

### Daily Health & Wellness
| Workflow | Timer | Schedule | Description |
|----------|-------|----------|-------------|
| `morning-note-prompt.lobster` | lobster-morning-note | 8:20am | Morning check-in with Oura data |
| `daily-oura-hybrid.lobster` | lobster-daily-oura-hybrid | 8:00am | Oura hybrid daily report |
| `daily-oura-note.lobster` | lobster-daily-oura | 8:15am | Daily Oura note creation |
| `health-report.lobster` | lobster-health | 8:15am | Consolidated health briefing |
| `fitbit-report.lobster` | lobster-fitbit | 8:00am | Daily Fitbit metrics |

### Work & Productivity
| Workflow | Timer | Schedule | Description |
|----------|-------|----------|-------------|
| `daily-work-standup.lobster` | lobster-standup | Mon-Fri 8:30am | Daily work standup |
| `eod-task-review.lobster` | lobster-eod | Mon-Fri 6:00pm | End of day task review |
| `evening-reflection.lobster` | lobster-evening-reflection | 8:00pm | Evening reflection prompt |

### Financial & Markets
| Workflow | Timer | Schedule | Description |
|----------|-------|----------|-------------|
| `morning-market.lobster` | lobster-market-morning | Mon-Fri 6:30am | German morning market briefing |
| `evening-market.lobster` | lobster-market-evening | Mon-Fri 1:00pm | German evening market briefing |
| `stocks-dashboard.lobster` | lobster-stocks | Mon-Fri 2:00pm | Update stocks dashboard |
| `earnings-calendar.lobster` | lobster-earnings | Sun 7:00am | Weekly earnings calendar |

### Weekly Tasks
| Workflow | Timer | Schedule | Description |
|----------|-------|----------|-------------|
| `weekly-sleep.lobster` | lobster-weekly-sleep | Sun 8:00am | Weekly sleep report |
| `weekly-areas-index.lobster` | lobster-weekly-areas | Sun 10:00am | Update Obsidian areas index |
| `weekly-work-review.lobster` | lobster-weekly | Mon 9:00am | Weekly work review |

### Maintenance
| Workflow | Timer | Schedule | Description |
|----------|-------|----------|-------------|
| `research-sync.lobster` | lobster-research | Sun 11:00am | Research data sync |
| `qmd-update` | lobster-qmd-update | 3:00am daily | QMD index update |

## Manual Workflows (14 without timers)

These workflows are run on-demand, not scheduled:

### Obsidian Management
- `areas-index.lobster` - Update areas index
- `reference-index.lobster` - Update reference index
- `monthly-archive-audit.lobster` - Monthly archive audit
- `personal-standup.lobster` - Personal task standup

### Development & Tools
- `pr-monitor.lobster` - GitHub PR monitoring
- `qmd-search.lobster` - QMD knowledge search
- `qmd-search-telegram.lobster` - QMD search (Telegram format)
- `test-pipeline.lobster` - Test/demo workflow

### Work & Business
- `daily-standup.lobster` - Old standup version
- `daily-standup-cron.lobster` - Current standup workflow
- `shapescale-deal-summary.lobster` - ShapeScale deal summaries
- `equity-research.lobster` - Stock equity research

### Special Projects
- `niemand-boerse-news-alerts.lobster` - Finance news alerts workflow

## Scripts

### JavaScript Helpers
- `format-pr-summary.js` - Format GitHub PR summaries
- `gh-pr-view.js` - Fetch PR data via gh CLI
- `market-briefing.js` - Format market briefing output

### Python Scripts
- `daily-note.py` - Create daily Obsidian note with health data
- `daily-oura-report-hybrid.py` - Oura hybrid report generator

### Shell Scripts
- `daily-fitbit-report.sh` - Fitbit daily metrics
- `daily-oura-report-hybrid.sh` - Oura hybrid report wrapper
- `oura-weekly-sleep-alert.sh` - Weekly sleep report

## Creating New Workflows

### 1. Create Workflow File

```bash
cd ~/projects/lobster-workflows/workflows
vim my-workflow.lobster
```

Example workflow structure:
```yaml
name: my-workflow
description: Brief description of what this does
steps:
  - id: step1
    command: echo "Hello world"
  - id: step2
    command: python3 /home/art/niemand/scripts/my-script.py
    stdin: $step1.stdout
```

### 2. Add Supporting Scripts (if needed)

```bash
cd ~/projects/lobster-workflows/scripts
vim my-script.py
chmod +x my-script.py
```

### 3. Test Workflow

```bash
cd ~/projects/lobster
node ./bin/lobster.js run --file ~/projects/lobster-workflows/workflows/my-workflow.lobster
```

### 4. Create Systemd Timer (if scheduling)

Create service file:
```bash
vim ~/.config/systemd/user/lobster-my-workflow.service
```

```ini
[Unit]
Description=My Workflow Description

[Service]
Type=oneshot
WorkingDirectory=/home/art/projects/lobster
ExecStart=/usr/bin/env node ./bin/lobster.js run --file /home/art/projects/lobster-workflows/workflows/my-workflow.lobster
StandardOutput=append:/home/art/niemand/logs/lobster-my-workflow.log
StandardError=append:/home/art/niemand/logs/lobster-my-workflow.log

[Install]
WantedBy=multi-user.target
```

Create timer file:
```bash
vim ~/.config/systemd/user/lobster-my-workflow.timer
```

```ini
[Unit]
Description=My Workflow Timer

[Timer]
OnCalendar=*-*-* 09:00:00
Persistent=true

[Install]
WantedBy=timers.target
```

Enable and start:
```bash
systemctl --user daemon-reload
systemctl --user enable --now lobster-my-workflow.timer
systemctl --user list-timers | grep my-workflow
```

### 5. Commit to Repo

```bash
cd ~/projects/lobster-workflows
git add workflows/my-workflow.lobster scripts/my-script.py
git commit -m "Add my-workflow for doing X"
git push
```

## Best Practices

### Security
- **Never hardcode secrets** in workflow files
- Use environment variables from `~/.config/systemd/user/secrets.conf`
- Reference secrets in workflows: `${ENV_VAR_NAME}`

### Script Paths
- Always use **absolute paths** in workflows
- Lobster scripts: `/home/art/niemand/scripts/`
- Skill scripts: `/home/art/niemand/skills/<skill-name>/scripts/`
- CLI tools: `/home/art/.local/bin/<tool>`

### Logging
- Systemd services automatically log to specified files
- Use `journalctl --user -u lobster-workflow-name -f` to tail logs
- Log files location: `/home/art/niemand/logs/lobster-*.log`

### Testing
- Always test workflows manually before scheduling
- Use `--dry-run` if the workflow supports it
- Check workflow output before committing

## Troubleshooting

### Workflow not running
```bash
# Check timer status
systemctl --user list-timers | grep lobster

# Check service logs
journalctl --user -u lobster-workflow-name -n 50

# Manually run workflow
cd ~/projects/lobster
node ./bin/lobster.js run --file ~/projects/lobster-workflows/workflows/workflow-name.lobster
```

### Script not found
- Verify absolute path in workflow file
- Check script exists: `ls -la /home/art/niemand/scripts/`
- Verify script is executable: `chmod +x script.sh`

### Permission denied
- Check systemd service has correct working directory
- Verify script permissions
- Check gateway writable paths if using OpenClaw

## Migration Notes

This repo was migrated from `~/.config/lobster/workflows/` on 2026-01-30. Old workflows are backed up at:
- `~/.config/lobster/workflows.backup-20260130`

Scripts were previously scattered across:
- `/home/art/projects/openbb/scripts/` (old, non-existent)
- `/home/art/niemand/scripts/`
- `/home/art/.config/lobster/scripts/`

All Lobster-specific scripts are now centralized in this repo.

## Related Resources

- **Lobster Engine**: https://github.com/moltbot/lobster
- **Systemd User Timers**: `~/.config/systemd/user/lobster-*.timer`
- **Log Files**: `/home/art/niemand/logs/`
- **Secrets Config**: `~/.config/systemd/user/secrets.conf`
