Skip to content

METADATA_GUIDE.md

Latest commit

 

History

History
328 lines (262 loc) · 10.6 KB

METADATA_GUIDE.md

File metadata and controls

328 lines (262 loc) · 10.6 KB

Validation Rule Metadata: Current Implementation & Future Vision

📋 Overview

This document explains how metadata is integrated with validation rules in the CouchDB Rules Engine, covering both the current manual implementation and the future automated web interface approach.

🔧 Current Implementation (Manual Process)

How Metadata Works Today

Currently, metadata is manually defined and integrated into validation rules through a structured process:

1. Metadata Definition (utils/rule-metadata.js)

Pre-configured metadata is defined in the VALIDATOR_METADATA object:

const VALIDATOR_METADATA = {
    householdIncome: {
        name: 'Household Income Validator',
        description: 'Validates that household income does not exceed $25,000 threshold for program eligibility',
        version: '1.0.0',
        author: 'CouchDB Rules Engine',
        tags: ['income', 'eligibility', 'financial', 'threshold'],
        status: 'active',
        changeNotes: 'Initial implementation with $25,000 income threshold'
    }
    // ... other validators
};

2. Validator Enhancement (validators/*.js)

Each validator exports both the validation function and metadata:

const { createRuleMetadata, VALIDATOR_METADATA } = require('../utils/rule-metadata');

// Validation rule for household income
exports.householdIncome = function (doc) {
    if (doc.income > 25000) {
        throw ({
            forbidden: 'Income must be lower than $25,000'
        });
    }
    return true;
};

// Export metadata for this validator
exports.metadata = createRuleMetadata(VALIDATOR_METADATA.householdIncome);

3. CouchDB Integration (couchLoader.js)

The loader automatically includes metadata when creating design documents:

for (let validator in validators) {
    let doc = {};
    doc._id = `_design/${validator}`;
    doc.validate_doc_update = `${validators[validator][validator]}`;
    
    // Include metadata if available
    if (validators[validator].metadata) {
        doc.rule_metadata = validators[validator].metadata;
        console.log(`Adding metadata for ${validator} validator`);
    }
    
    // POST to CouchDB...
}

4. Resulting CouchDB Structure

This creates design documents with embedded metadata:

{
  "_id": "_design/householdIncome",
  "_rev": "1-abc123...",
  "validate_doc_update": "function(doc) { if(doc.income > 25000) throw({forbidden: 'Income must be lower than $25,000'}); return true; }",
  "rule_metadata": {
    "name": "Household Income Validator",
    "description": "Validates that household income does not exceed $25,000 threshold for program eligibility",
    "version": "1.0.0",
    "author": "CouchDB Rules Engine",
    "tags": ["income", "eligibility", "financial", "threshold"],
    "status": "active",
    "created_date": "2025-09-11T13:53:33.930Z",
    "modified_date": "2025-09-11T13:53:33.930Z",
    "change_notes": "Initial implementation with $25,000 income threshold"
  }
}

🎯 Adding New Validation Rules (Current Manual Process)

Step 1: Define Metadata

Add metadata definition to utils/rule-metadata.js:

const VALIDATOR_METADATA = {
    // ... existing validators
    
    newValidatorName: {
        name: 'My New Validation Rule',
        description: 'Detailed description of what this rule validates',
        version: '1.0.0',
        author: 'Rule Author Name',
        tags: ['category', 'type', 'eligibility'],
        status: 'active',
        changeNotes: 'Initial implementation with specific business requirements'
    }
};

Step 2: Create Validator Module

Create validators/newValidatorName.js:

const { createRuleMetadata, VALIDATOR_METADATA } = require('../utils/rule-metadata');

// Validation logic
exports.newValidatorName = function (doc) {
    if (/* validation condition */) {
        throw ({
            forbidden: 'Validation error message'
        });
    }
    return true;
};

// Export metadata
exports.metadata = createRuleMetadata(VALIDATOR_METADATA.newValidatorName);

Step 3: Register in Main Index

Add to index.js:

const newValidatorName = require('./validators/newValidatorName');

module.exports = {
    // ... existing validators
    newValidatorName: newValidatorName
};

Step 4: Create Tests

Follow the TESTING_GUIDE.md to create comprehensive tests.

Step 5: Deploy to CouchDB

node couchLoader.js your_database_name

🚀 Future Vision: Web Interface Automation (Phase 1.3+)

How This Will Change

The upcoming Basic Web Interface (CRUD Operations) will automate and streamline this entire process:

1. Rule Creation Form

<!-- Future web interface mockup -->
<form id="new-rule-form">
    <h2>Create New Validation Rule</h2>
    
    <!-- Metadata Fields -->
    <fieldset>
        <legend>Rule Information</legend>
        <input type="text" name="name" placeholder="Rule Name" required>
        <textarea name="description" placeholder="Detailed description" required></textarea>
        <input type="text" name="version" value="1.0.0" pattern="\\d+\\.\\d+\\.\\d+">
        <input type="text" name="author" placeholder="Author Name">
        <input type="text" name="tags" placeholder="tag1, tag2, tag3">
        <select name="status">
            <option value="draft">Draft</option>
            <option value="active">Active</option>
            <option value="inactive">Inactive</option>
        </select>
    </fieldset>
    
    <!-- Validation Logic -->
    <fieldset>
        <legend>Validation Function</legend>
        <textarea name="validation_code" placeholder="function(doc) { /* validation logic */ }"></textarea>
    </fieldset>
    
    <button type="submit">Create Rule</button>
</form>

2. Automated File Generation

The web interface will automatically:

// Future implementation preview
async function createNewRule(formData) {
    // 1. Generate metadata object
    const metadata = createRuleMetadata({
        name: formData.name,
        description: formData.description,
        version: formData.version,
        author: formData.author,
        tags: formData.tags.split(',').map(t => t.trim()),
        status: formData.status,
        changeNotes: `Created via web interface on ${new Date().toISOString()}`
    });
    
    // 2. Create design document
    const designDoc = {
        _id: `_design/${formData.ruleName}`,
        rule_metadata: metadata,
        validate_doc_update: formData.validation_code
    };
    
    // 3. Save directly to CouchDB
    const result = await fetch(`${couchDB_url}`, {
        method: 'POST',
        headers: { 'Content-Type': 'application/json' },
        body: JSON.stringify(designDoc)
    });
    
    // 4. Auto-generate test scaffolding
    generateTestFile(formData.ruleName, metadata);
    
    // 5. Update rule registry
    updateRuleIndex(formData.ruleName);
}

3. Rule Management Dashboard

// Future rule listing interface
function displayRulesList(rules) {
    return rules.map(rule => `
        <div class="rule-card">
            <h3>${rule.rule_metadata.name}</h3>
            <p>${rule.rule_metadata.description}</p>
            <div class="rule-tags">
                ${rule.rule_metadata.tags.map(tag => `<span class="tag">${tag}</span>`).join('')}
            </div>
            <div class="rule-actions">
                <button onclick="editRule('${rule._id}')">Edit</button>
                <button onclick="testRule('${rule._id}')">Test</button>
                <button onclick="viewHistory('${rule._id}')">History</button>
            </div>
        </div>
    `).join('');
}

🔄 Migration Path

Current State → Future State

Aspect Current (Manual) Future (Web Interface)
Rule Creation Manual file editing Web form with validation
Metadata Definition Pre-configured constants Dynamic form inputs
File Management Manual file creation Automated generation
CouchDB Deployment Command line tool One-click deployment
Testing Manual test creation Auto-generated test scaffolding
Version Control Git-based Built-in version management

Backward Compatibility

The web interface will:

  • ✅ Recognize existing rules created manually
  • ✅ Import current metadata from existing design documents
  • ✅ Maintain file structure for rules created via web interface
  • ✅ Support hybrid workflows (manual + web interface)

📚 Best Practices (Current Manual Process)

Until the web interface is available:

1. Consistent Metadata

  • Always use semantic versioning (1.0.0, 1.1.0, 2.0.0)
  • Include relevant business tags for categorization
  • Write clear, business-focused descriptions
  • Document change notes for version updates

2. Validation Function Quality

  • Keep validation logic simple and focused
  • Use descriptive error messages that business users understand
  • Test edge cases and boundary conditions
  • Follow existing patterns from current validators

3. Testing Requirements

  • Create comprehensive unit tests for validation logic
  • Add integration tests for CouchDB behavior
  • Include metadata validation tests
  • Follow the TESTING_GUIDE.md patterns

4. Documentation

  • Update metadata when changing validation logic
  • Increment version numbers for any changes
  • Document business rationale in change notes
  • Keep tags current and relevant

🎯 Benefits of This Approach

Current Benefits

  • Rich Documentation: Every rule is self-documenting
  • Version Tracking: Clear history of rule changes
  • Categorization: Tag-based organization
  • Consistency: Standardized metadata structure

Future Benefits (Web Interface)

  • User-Friendly: Non-technical users can create rules
  • Validation: Form validation prevents metadata errors
  • Automation: Reduces manual file management overhead
  • Integration: Seamless CouchDB deployment
  • Testing: Auto-generated test scaffolding
  • Governance: Built-in approval workflows

🔗 Related Documentation

💡 Key Takeaway: The current manual metadata process provides a solid foundation that will seamlessly transition to automated web-based rule management, maintaining backward compatibility while dramatically improving usability.