/home/techb158/cosmic-risk.abdallabala.com/docs
Edit: /home/techb158/cosmic-risk.abdallabala.com/docs/integration-pull-push-plan.md (7384B)
# COSMIC Pull-Push Integration Plan
## Objective
Build bidirectional integration between COSMIC AI-Risk Dashboard and project management tools (Trello, Jira, Asana, Microsoft Planner).
- **Pull**: Import boards/projects/tasks from PM tools → create Risks in COSMIC
- **Push**: Export COSMIC Risks → create/update items in PM tools (existing)
- **Bidirectional**: Full sync with conflict resolution based on timestamps
## Repository & Deployment
| Item | Value |
|---|---|
| **GitHub repo** | `github.com/Bala-Group/cosmic-pull-push-integrations` |
| **Base code** | Fork of `saas-app/` directory |
| **Subdomain** | `cosmic-risk.abdallabala.com` |
| **Deployment** | Docker + Cloudflare Tunnel on VPS |
| **App port** | `8094` (host) → `8090` (container) |
## Architecture
The implementation stays within the existing `saas-app` layered framework:
```
docs/
integration-pull-push-plan.md ← this plan
src/
domain/
integrations.js ← add externalStatusToLocalStatus()
integrations/
pm-clients/
trello-client.js ← add fetchWorkItems()
jira-client.js ← add fetchWorkItems()
asana-client.js ← add fetchWorkItems()
planner-client.js ← add fetchWorkItems()
services/
import-service.js ← NEW: pull/import orchestration
integration-service.js ← extended: bidirectionalSync()
app/api/integrations/[id]/
pull/route.js ← NEW: POST /pull
pull-preview/route.js ← NEW: GET /pull-preview
bidirectional/route.js ← NEW: POST /bidirectional
app/integrations/[id]/
page.jsx ← extended: Import UI panel
tests/
live-pm-clients.test.js ← extended: pull tests
integration-service.test.js ← extended: import service tests
```
## Implementation Phases
### Phase 0 — Backup & Repo Setup ✓
1. ✅ Create zip backup of `saas-app/`
2. ✅ Create private GitHub repo `cosmic-pull-push-integrations`
3. ✅ Push saas-app code as initial commit
4. ⬜ Deploy to `cosmic-risk.abdallabala.com` (Docker + Cloudflare Tunnel)
### Phase 1 — Pull (PM → COSMIC)
#### 1a. Add `fetchWorkItems()` to each PM client
| Client | API Call | Returns |
|---|---|---|
| `trello-client.js` | `GET /1/lists/{id}/cards` | cards with id, name, desc, labels, due, members |
| `jira-client.js` | `GET /rest/api/3/search?jql=project={key}` | issues with id, key, summary, description, status |
| `asana-client.js` | `GET /api/1.0/projects/{gid}/tasks` | tasks with gid, name, notes, assignee, due_on |
| `planner-client.js` | `GET /v1.0/planner/plans/{id}/tasks` | tasks with id, title, dueDateTime, percentComplete |
#### 1b. Add `externalStatusToLocalStatus()` in domain
Reverse mapping function per provider:
| Provider | External Status | COSMIC Status |
|---|---|---|
| Trello | "Approved for deployment" | CLOSED |
| Trello | "In mitigation" | IN_MITIGATION |
| Trello | "Blocked by risk" / other | OPEN |
| Jira | "Done" | CLOSED |
| Jira | "In Progress" | IN_MITIGATION |
| Jira | "To Do" / other | OPEN |
| Asana | "Complete" | CLOSED |
| Asana | "In progress" | IN_MITIGATION |
| Asana | "Open" / other | OPEN |
| Planner | "Completed" | CLOSED |
| Planner | "In progress" | IN_MITIGATION |
| Planner | "Not started" / "Blocked bucket" | OPEN |
#### 1c. Create `src/services/import-service.js`
Functions:
- **`getImportPreview(integrationId)`** — dry-run: fetch external items, compare against existing mappings, return list of new items that would be imported
- **`pullFromProvider(integrationId, actor, request)`** — actual import:
1. Load integration + credentials
2. Create PM client
3. Call `fetchWorkItems()`
4. For each item, check if mapping exists (dedup by external ID)
5. Create Risk record with mapped fields
6. Create ExternalWorkItemMapping (bidirectional lookup)
7. Write audit event
8. Return summary (created count, skipped count, failed count)
#### 1d. API Routes
| Method | Route | Handler |
|---|---|---|
| `GET` | `/api/integrations/{id}/pull-preview` | `getImportPreview()` |
| `POST` | `/api/integrations/{id}/pull` | `pullFromProvider()` |
#### 1e. UI — Import Panel
Add to `src/app/integrations/[integrationId]/page.jsx`:
- "Import from [Provider]" section
- "Preview Import" button → shows table of items to be imported
- "Import" button → triggers pull, shows toast with summary
### Phase 2 — Bidirectional Sync
#### 2a. Add `bidirectionalSync()` to integration-service
Logic:
1. **Push**: Call existing `liveSync()` (COSMIC → PM)
2. **Pull**: Call new `pullFromProvider()` (PM → COSMIC)
3. **Conflict resolution**: Compare `updatedAt` on COSMIC Risk vs `lastSyncedAt` on mapping. If COSMIC is newer → push wins. If external is newer → pull wins. If same → skip.
4. Create single IntegrationSyncRun recording both directions
#### 2b. API Route
| Method | Route | Handler |
|---|---|---|
| `POST` | `/api/integrations/{id}/bidirectional` | `bidirectionalSync()` |
### Phase 3 — Tests & Deployment
#### 3a. Tests
Update `tests/live-pm-clients.test.js`:
- Mock fetch for each client's `fetchWorkItems()`
- Verify returned items have expected shape
Update `tests/integration-service.test.js`:
- Test `externalStatusToLocalStatus()` mapping
- Test import preview (dry-run, no DB writes)
#### 3b. Cloudflare Tunnel
```
cloudflared tunnel create cosmic-risk
cloudflared tunnel route dns cosmic-risk cosmic-risk.abdallabala.com
```
config.yml:
```yaml
tunnel:
credentials-file: /root/.cloudflared/.json
ingress:
- hostname: cosmic-risk.abdallabala.com
service: http://localhost:8094
- service: http_status:404
```
#### 3c. Docker Compose
Update `docker-compose.yml` port mapping:
```yaml
ports:
- "8094:8090"
```
## Field Mapping
Pull (PM → COSMIC):
| COSMIC Field | Trello | Jira | Asana | Planner |
|---|---|---|---|---|
| title | name | summary | name | title |
| description | desc | description | notes | description (from details) |
| dimension | From label (parsed) | From label/component | From tag/custom field | From category |
| status | From list name | From status name | From section/status | From bucket/progress |
| dueDate | due | duedate | due_on | dueDateTime |
| owner | member (fullName) | assignee (displayName) | assignee (name) | assigneeId (resolved) |
| probability | Default 3 | Default 3 | Default 3 | Default 3 |
| impact | Default 3 | Default 3 | Default 3 | Default 3 |
| externalId | id | id | gid | id |
| externalUrl | shortUrl | browse URL | permalink_url | tasks.office.com URL |
## Files Modified/Created
### Modified files
- `src/integrations/pm-clients/trello-client.js`
- `src/integrations/pm-clients/jira-client.js`
- `src/integrations/pm-clients/asana-client.js`
- `src/integrations/pm-clients/planner-client.js`
- `src/domain/integrations.js`
- `src/services/integration-service.js`
- `src/app/integrations/[integrationId]/page.jsx`
- `tests/live-pm-clients.test.js`
- `tests/integration-service.test.js`
### New files
- `src/services/import-service.js`
- `src/app/api/integrations/[integrationId]/pull/route.js`
- `src/app/api/integrations/[integrationId]/pull-preview/route.js`
- `src/app/api/integrations/[integrationId]/bidirectional/route.js`
- `docs/integration-pull-push-plan.md`
