ca-grow-ops-manager/specs/paperless-integration.md

Paperless-ngx Integration Specification

Overview

Integrate CA Grow Ops Manager with a Paperless-ngx instance for long-term document storage, archival, and retrieval. Paperless-ngx provides OCR, full-text search, tagging, and archival capabilities that complement our application's document management needs.

Architecture

┌─────────────────────────────────────────────────────┐
│                  CA Grow Ops Manager                │
│  ┌───────────────┐    ┌─────────────────────────┐  │
│  │  Documents    │    │  Paperless Service      │  │
│  │  (metadata)   │◄──►│  (API wrapper)          │  │
│  └───────────────┘    └───────────┬─────────────┘  │
└───────────────────────────────────┼─────────────────┘
                                    │ REST API
                                    ▼
                          ┌─────────────────────┐
                          │   Paperless-ngx     │
                          │   (Document Store)  │
                          │   - OCR             │
                          │   - Full-text       │
                          │   - Tagging         │
                          │   - Archival        │
                          └─────────────────────┘

Use Cases

1. SOPs and Policies

• Upload SOPs from CA Grow Ops to Paperless for long-term archival
• Retrieve and display archived versions
• Full-text search across all SOPs

2. Compliance Documents

• Store state inspection reports
• Archive METRC reports and manifests
• Maintain license documentation

3. Batch Documentation

• Archive completed batch records
• Store harvest manifests
• Archive lab test results (COAs)

4. Visitor Logs

• Archive signed NDAs
• Store visitor badges and access logs

5. Photos (Optional)

• Archive walkthrough photos
• Store plant progress photos with metadata

Paperless-ngx API Integration

Authentication

// Environment variables
PAPERLESS_URL=https://paperless.runfoo.run
PAPERLESS_TOKEN=your_api_token

API Endpoints Used

Endpoint	Method	Purpose
/api/documents/	GET	List/search documents
/api/documents/	POST	Upload new document
/api/documents/{id}/	GET	Get document metadata
/api/documents/{id}/download/	GET	Download original file
/api/documents/{id}/preview/	GET	Get thumbnail/preview
/api/tags/	GET/POST	Manage tags
/api/correspondents/	GET/POST	Manage correspondents (document owners)
/api/document_types/	GET/POST	Manage document types

Implementation

Backend Service (paperless.service.ts)

interface PaperlessDocument {
    id: number;
    title: string;
    content: string;
    tags: number[];
    document_type: number | null;
    correspondent: number | null;
    created: string;
    added: string;
    archive_serial_number: string | null;
    original_file_name: string;
}

interface UploadDocumentOptions {
    title: string;
    file: Buffer | ReadStream;
    filename: string;
    documentType?: string;
    tags?: string[];
    correspondent?: string;
    archiveSerialNumber?: string;
}

class PaperlessService {
    async uploadDocument(options: UploadDocumentOptions): Promise<PaperlessDocument>;
    async getDocument(id: number): Promise<PaperlessDocument>;
    async searchDocuments(query: string, tags?: string[]): Promise<PaperlessDocument[]>;
    async downloadDocument(id: number): Promise<Buffer>;
    async deleteDocument(id: number): Promise<void>;
    
    // Tag management
    async createTag(name: string, color?: string): Promise<Tag>;
    async getOrCreateTag(name: string): Promise<Tag>;
    
    // Document type management
    async getOrCreateDocumentType(name: string): Promise<DocumentType>;
}

API Routes (paperless.routes.ts)

// Upload document to Paperless
POST /api/paperless/documents
  Body: multipart/form-data
  - file: File
  - title: string
  - documentType: string (e.g., "SOP", "COMPLIANCE", "BATCH_RECORD")
  - tags: string[] (e.g., ["gorilla-glue", "batch-001"])

// Search Paperless documents
GET /api/paperless/search?q=keyword&tags=tag1,tag2

// Get document preview
GET /api/paperless/documents/:id/preview

// Download original document
GET /api/paperless/documents/:id/download

// Sync local document to Paperless
POST /api/paperless/sync/:localDocumentId

// Get sync status for local document
GET /api/paperless/sync-status/:localDocumentId

Database Schema Addition

model Document {
    // ... existing fields ...
    
    // Paperless sync fields
    paperlessId       Int?      @unique
    paperlessSyncedAt DateTime?
    paperlessUrl      String?
}

Tagging Convention

Consistent tagging for easy retrieval:

Tag Pattern	Example	Purpose
app:grow-ops	app:grow-ops	Source application
type:{category}	type:sop, type:compliance	Document category
batch:{name}	batch:gg4-b001	Associated batch
room:{name}	room:flower-a	Associated room
year:{year}	year:2025	Year for archival
user:{email}	user:mike	Document owner

Document Types in Paperless

Pre-create these document types:

1. SOP - Standard Operating Procedures
2. Policy - Company policies
3. Compliance - State/regulatory documents
4. Batch Record - Completed batch documentation
5. Lab Result - COA and test results
6. Visitor Log - NDA, access records
7. Photos - Facility/plant photos
8. Invoice - Purchase records

Sync Workflow

Manual Sync

1. User clicks "Archive to Paperless" on a document
2. Backend uploads file to Paperless API
3. Backend stores paperlessId and paperlessUrl in local DB
4. UI shows sync status indicator

Automatic Sync (Optional)

1. Cron job runs nightly
2. Finds documents meeting criteria (e.g., approved SOPs, completed batches)
3. Uploads to Paperless if not already synced
4. Updates local records with Paperless IDs

Retrieval

1. User searches in CA Grow Ops
2. Search includes both local and Paperless documents
3. For Paperless-only documents, proxy the download through our API

UI Components

Document Card (Extended)

<DocumentCard>
    <DocumentTitle>{doc.title}</DocumentTitle>
    <DocumentMeta>
        <Badge>{doc.type}</Badge>
        {doc.paperlessId && (
            <Badge variant="success">
                <CloudArchive /> Archived
            </Badge>
        )}
    </DocumentMeta>
    <Actions>
        <Button onClick={viewDocument}>View</Button>
        {!doc.paperlessId && (
            <Button onClick={archiveToPaperless}>
                Archive to Paperless
            </Button>
        )}
    </Actions>
</DocumentCard>

Archive Modal

<ArchiveModal>
    <Title>Archive to Paperless</Title>
    <TagSelector 
        selected={selectedTags}
        onChange={setSelectedTags}
        suggestions={["batch:current", "type:sop"]}
    />
    <DocumentTypeSelector 
        value={documentType}
        onChange={setDocumentType}
    />
    <Button onClick={handleArchive}>Archive</Button>
</ArchiveModal>

Configuration

Environment Variables

# Paperless Connection
PAPERLESS_ENABLED=true
PAPERLESS_BASE_URL=https://paperless.runfoo.run
PAPERLESS_API_TOKEN=pk_xxxxxxxxxxxxx

# Sync Settings
PAPERLESS_AUTO_SYNC=false
PAPERLESS_SYNC_COMPLETED_BATCHES=true
PAPERLESS_SYNC_APPROVED_SOPS=true

# Defaults
PAPERLESS_DEFAULT_CORRESPONDENT=777wolfpack

Settings Page Addition

Add Paperless configuration section to Settings:

• Connection status indicator
• Test connection button
• Enable/disable sync
• Configure auto-sync rules

Error Handling

Error	Handling
Paperless unavailable	Queue for retry, show warning
Upload failed	Store locally, mark for retry
Authentication error	Alert admin, disable sync
Document not found	Remove paperlessId from local record

Security Considerations

1. API Token Security: Store token in encrypted environment variable
2. Access Control: Only users with document permissions can trigger archival
3. Audit Logging: Log all Paperless operations to audit log
4. Network Security: Require HTTPS for Paperless connection

Implementation Phases

Phase 1: Basic Integration

• Paperless service with upload/download
• Manual sync button on documents
• Sync status indicators

Phase 2: Search Integration

• Combined search across local + Paperless
• Tag management UI
• Preview/thumbnail support

Phase 3: Automatic Sync

• Cron job for batch archival
• Auto-archive completed batches
• Auto-archive approved SOPs

Phase 4: Advanced Features

• Bulk archive UI
• Archive reports/analytics
• Retention policies

Testing

Unit Tests

• Mock Paperless API responses
• Test upload/download flows
• Test error handling

Integration Tests

• Set up test Paperless instance
• End-to-end document workflow
• Sync status verification

Dependencies

{
    "form-data": "^4.0.0",
    "node-fetch": "^3.3.0"
}

References

• Paperless-ngx Documentation
• Paperless API Reference
• Paperless Docker Setup