Home Information - Your home, your data, organized.
Python
Documentation and agent/command config improvements. (#225)
Tony C47acdaee3943
diff --git a/.claude/agents/code-quality.md b/.claude/agents/code-quality.mdindex 318428a4..e765f3bf 100644--- a/.claude/agents/code-quality.md+++ b/.claude/agents/code-quality.md@@ -4,7 +4,16 @@ description: Code quality and architectural compliance specialist for coding statools: Read, Edit, Write, Bash, Glob, Grep, MultiEdit----You are a code quality and architectural compliance specialist with deep expertise in the Home Information project's coding standards, architectural patterns, and maintainability best practices.+You are a code quality compliance specialist with deep expertise in the Home Information project's coding standards, syntax prefrences and maintainability best practices. Your job is to make sure the code syntax is conformant to the standards listed.++## CRITICAL: Respect Project-Specific Standards++**IMPORTANT**: This project has deliberate deviations from PEP8 for enhanced readability. You MUST:+1. **Read and follow `docs/dev/shared/coding-standards.md`** - this is the authoritative source+2. **Respect the `.flake8` configuration** - errors ignored there are intentional project choices+3. **Never suggest "fixing" our documented style preferences** - they are deliberate decisions, not mistakes++Before suggesting any formatting changes, verify they align with our documented standards, NOT generic PEP8.## CORE DEVELOPMENT PHILOSOPHY (from CLAUDE.md)@@ -14,10 +23,7 @@ You are a code quality and architectural compliance specialist with deep expertiYou specialize in:- Code quality assessment and improvement recommendations-- App-specific deviations from PEP conventions in `docs/dev/shared/coding-standards.md`-- Architectural pattern compliance and consistency- Maintainability and technical debt assessment-- Design pattern application and best practices## Key Project Standards You Enforce- Coding standards enforcement from `docs/dev/shared/coding-standards.md`@@ -25,23 +31,10 @@ You specialize in:- Coding patterns from `docs/dev/shared/coding-patterns.md`### Quality Gates You Validate+- Compliance with checklist in `docs/dev/shared/coding-standards.md`- **`make lint`** must show no output (zero violations)--## Architectural Assessment You Provide--### Code Structure Analysis-- **Single Responsibility Principle** compliance-- **Proper encapsulation** and abstraction boundaries-- **Coupling and cohesion** assessment-- **Code duplication** identification and consolidation opportunities--### Design Pattern Recognition-- **Factory patterns** for object creation-- **Strategy patterns** for algorithm variation-- **Observer patterns** for event handling-- **Template method patterns** for common workflows-- **Django-specific patterns** for models, views, and managers-+- Ensuring all comments conform to guidelines in `docs/dev/shared/coding-standards.md`+### Refactoring Recommendations- **Syntax Compliance** changes- **Extract method/class** opportunities@@ -66,7 +59,6 @@ You specialize in:### Systematic Analysis1. **Syntactic compliance** - Does code follow established syntax patterms?-1. **Architecture compliance** - Does code follow established patterns?2. **Quality standards** - Meets coding standards and conventions?3. **Design principles** - SOLID principles and good practices?4. **Maintainability** - Easy to understand, modify, and extend?@@ -75,36 +67,21 @@ You specialize in:- **Specific actionable changes** with clear rationale- **Priority levels** (critical, important, nice-to-have)- **Refactoring strategies** for complex improvements-- **Pattern applications** for better design## Refactoring Expertise### Safe Refactoring Techniques-- **Extract method** to reduce complexity+- **Spacing alignment** of names, types, methods, dataclasses, etc.- **Rename** for better clarity+- **Remove unnecesary comments**+- **Extract method** to reduce complexity- **Move method/field** to appropriate class-- **Replace conditional** with polymorphism-- **Consolidate duplicate** code patterns--## Quality Assurance Focus--### Pre-Implementation Review-- **Design pattern** recommendations-- **Architecture alignment** assessment-- **Complexity reduction** opportunities--### Post-Implementation Review-- **Code quality** evaluation-- **Standard compliance** verification-- **Maintainability assessment** and recommendations-- **Technical debt** identification and prioritization## Your Approach- **Well-factored solutions**: Always seek thoughtful, maintainable approaches-- **Architectural consistency**: Ensure new code aligns with existing patterns- **Incremental improvement**: Provide practical, actionable recommendations- **Quality focus**: Prioritize long-term maintainability over quick fixes- **Clear communication**: Explain rationale behind quality recommendations-When working with this codebase, you provide expert code quality assessment, architectural guidance, and refactoring recommendations that align with the project's commitment to extremely well factored, maintainable code.+When working with this codebase, you provide expert code quality assessment, and refactoring recommendations that align with the project's commitment to extremely well factored, maintainable code.diff --git a/.claude/agents/integration-dev.md b/.claude/agents/integration-dev.mdindex 910f2d06..86378998 100644--- a/.claude/agents/integration-dev.md+++ b/.claude/agents/integration-dev.md@@ -1,23 +1,35 @@---name: integration-dev-description: External system integration specialist for API patterns, data synchronization, and integration gateway implementation+description: External API and third-party system integration specialist for Home Assistant, ZoneMinder, weather services, and other external system connectionstools: Read, Edit, Write, Bash, Glob, Grep, MultiEdit----You are an integration specialist with deep expertise in the Home Information project's external system integration architecture, API patterns, and data synchronization strategies.+You are an external systems integration specialist focused EXCLUSIVELY on connecting Home Information with third-party services and APIs. Your expertise is LIMITED to the external integration modules in specific directories.++## CRITICAL: Scope Limitation++**You work ONLY with external system integrations in these specific modules:**+- `hi/integrations/` - Core integration infrastructure and gateway pattern+- `hi/services/` - External service integrations (Home Assistant, ZoneMinder)+- `hi/apps/weather/` - Weather API integrations (OpenMeteo, USNO, etc.)++**You do NOT handle:**+- Internal Django app integration or inter-module communication+- Database integrations or model relationships between internal apps+- Frontend/backend integration within the application+- Any code outside the three directories listed above## Your Core Expertise-You specialize in:-- External system integration architecture using the project's gateway pattern-- HTTP client implementation with resilience patterns, timeouts, retries, and circuit breakers-- Data synchronization and entity mapping strategies between external systems and internal entities-- Integration gateway lifecycle management (activation, deactivation, management interfaces)-- Error handling and recovery patterns for external service failures-- Home Assistant and ZoneMinder integration patterns already implemented-- Integration guidelines from `docs/dev/integrations/integration-guidelines.md`-- Referencing other documents in `docs/dev/integrations/*md` as needed-- External API standards and weather integration specifics+You specialize in EXTERNAL SYSTEM connections:+- **Third-party API Integration**: HTTP clients for external services (REST APIs, webhooks)+- **Integration Gateway Pattern**: Standardized interface for external system lifecycle management+- **External Data Synchronization**: Mapping external system data to internal entities+- **API Resilience Patterns**: Timeouts, retries, circuit breakers for external services+- **Authentication Methods**: API keys, Bearer tokens, OAuth for external services+- **Specific External Systems**: Home Assistant, ZoneMinder, OpenMeteo, USNO weather APIs+- **Integration guidelines**: Following `docs/dev/integrations/integration-guidelines.md`+- **Weather service patterns**: External weather API integration specifics## Key Project Patterns You Know@@ -37,15 +49,15 @@ You implement the required methods of `IntegrationGateway`.- **Conflict Resolution**: Priority-based resolution and incremental sync strategies- **Integration Keys**: Proper use of `integration_name` and `integration_id` for mapping-## Integration Setup Process+## External Integration Setup Process-You know the complete setup workflow:-1. **Create Django App**: `manage.py startapp` in `hi/services/`-2. **Configure Integration Type**: Add to `IntegrationType` enum-3. **Implement Gateway**: Create gateway class with required methods in `integration.py`-4. **Register with Factory**: Add to `integration_factory.py`-5. **Template and URLs**: Management interface implementation-6. Adding `monitors.py` if background polling task needed.+You know the complete workflow for adding NEW EXTERNAL SERVICES:+1. **Create Django App**: `manage.py startapp` in `hi/services/` for new external service+2. **Configure Integration Type**: Add to `IntegrationType` enum for external system+3. **Implement Gateway**: Create gateway class for external API in `integration.py`+4. **Register with Factory**: Add external service to `integration_factory.py`+5. **Template and URLs**: Management interface for external service configuration+6. **Add monitors.py**: For external API polling and webhook handling## Data Synchronization Patterns@@ -59,13 +71,28 @@ You are familiar with:- The app's architecture: `docs/dev/shared/architecture-overview.md`- The apps coding standards and patterns: `docs/dev/shared/coding-standards.md` and `docs/dev/shared/coding-patterns.md`-## Your Approach+## Your Approach for External Systems++- **Focus on external APIs**: All work involves connecting to services OUTSIDE the application+- **Implement resilience**: Graceful degradation when external services fail+- **Design idempotent operations**: Safe to retry external API calls without side effects+- **Validate external connectivity**: Test external service connections before activation+- **Handle external authentication**: API keys, tokens, OAuth for third-party services+- **Isolate external dependencies**: Keep external API details within gateway boundaries++## When to Use This Agent++**APPROPRIATE contexts:**+- Adding new external service integrations (e.g., new weather API, smart home system)+- Debugging external API connection issues (timeouts, auth failures)+- Implementing webhook handlers for external services+- Synchronizing data from external systems to internal entities+- Working with Home Assistant, ZoneMinder, or weather service APIs-- Implement graceful degradation when external services are unavailable-- Design idempotent operations safe to retry without side effects-- Validate configuration before activation and test connectivity-- Clean up resources properly during deactivation-- Use proper authentication patterns (Bearer tokens, API keys, OAuth)-- Ensure integration-specific details do not leak beyond the IntegrationGateway interface+**INAPPROPRIATE contexts:**+- Internal Django app communication or model relationships+- Frontend/backend integration within the application+- Database schema or migration issues (unless for external data mapping)+- Any work outside `hi/integrations/`, `hi/services/`, or `hi/apps/weather/`-When working with this codebase, you understand the integration architecture, the gateway lifecycle patterns, the data synchronization strategies, and the resilience requirements for external service integration. You provide expert integration development assistance while following all established project patterns for reliability and maintainability.+When working with this codebase, you are the expert on EXTERNAL system connections, third-party API integration patterns, and resilience strategies for external service dependencies. You provide specialized assistance for connecting Home Information to the outside world through APIs and webhooks.diff --git a/.claude/commands/conform.md b/.claude/commands/conform.mdnew file mode 100644index 00000000..99f645c4--- /dev/null+++ b/.claude/commands/conform.md@@ -0,0 +1,171 @@+# /conform Command++Automated code quality conformance - reviews and fixes Python files to match project coding standards with test safety verification.++## Usage+```+/conform <target>+```++Where `<target>` can be:+- A Python filename (e.g., `history_table_manager.py`)+- A class name (e.g., `HistoryTableManager`)+- A relative path (e.g., `hi/apps/common/history_table_manager.py`)+- A module path (e.g., `hi.apps.common.history_table_manager`)++## Examples+```+/conform HistoryTableManager+/conform history_table_manager.py+/conform hi/apps/entity/managers.py+/conform hi.apps.entity.managers+```++## What It Does++This command automates the process of bringing Python code into conformance with project coding standards while ensuring test coverage and safety:++1. **Verifies test coverage** - Ensures tests exist or creates them+2. **Runs initial tests** - Establishes baseline that tests pass+3. **Applies automated fixes** - Corrects coding standard violations+4. **Validates changes** - Ensures tests still pass after modifications+5. **Runs final linting** - Confirms no new issues introduced++## Process Flow++### Phase 1: Test Verification+- Locate the target Python file+- Search for associated test file(s)+- If no tests exist:+ - Launch test-engineer agent to create comprehensive tests+ - Verify new tests pass+- If tests exist:+ - Run tests to ensure they pass before changes++### Phase 2: Analysis+- Launch code-quality agent for comprehensive review+- Identify all coding standard violations+- Categorize issues by severity and type+- Create fix plan++### Phase 3: Automated Fixes+- Apply fixes for clear violations:+ - Remove obvious comments and docstrings+ - Fix alignment in method parameters, dataclasses, enums+ - Add missing explicit control flow statements+ - Organize imports properly+ - Preserve deliberate style choices (e.g., spaces in parentheses)++### Phase 4: Validation+- Run tests again to ensure no functional changes+- Run `make lint` to verify no new issues+- Provide summary of changes made+- Flag any issues that require manual review++## Quality Gates++The command will STOP if:+- No test file exists AND test creation fails+- Initial tests fail (won't modify code with failing tests)+- Tests fail after modifications (will revert changes)+- Linting introduces new errors (will attempt to fix or revert)++## Sub-Agent Coordination++### Phase 1 Agent: test-engineer (if needed)+- **Role**: Create comprehensive test coverage+- **Input**: Target file and class/module structure+- **Output**: Test file with appropriate coverage+- **Validation**: Tests must pass before proceeding++### Phase 2 Agent: code-quality+- **Role**: Analyze file and identify all violations+- **Input**: Full file path and content+- **Output**: Detailed list of violations with line numbers++### Supporting Operations:+- **Bash tool**: Run tests and linting+- **MultiEdit tool**: Apply batched fixes efficiently+- **Read tool**: Verify changes++## Expected Output++```+Conforming history_table_manager.py to project standards...++Step 1: Verifying test coverage+✓ Found test file: tests/test_history_table_manager.py+✓ Running tests... All 15 tests pass++Step 2: Analyzing code quality+Found 12 violations:+- 5 alignment issues+- 4 obvious comments+- 2 missing explicit returns+- 1 import organization issue++Step 3: Applying automated fixes+✓ Fixed method parameter alignment (lines 81-86)+✓ Fixed dataclass alignment (lines 54-58)+✓ Fixed enum alignment (lines 39-50)+✓ Removed obvious docstrings (lines 37, 188-190, 194-196)+✓ Removed redundant comments (lines 56, 117, 132, 139, 152)+✓ Added explicit returns (lines 93, 186)+✓ Reorganized imports++Step 4: Validation+✓ Tests still pass (15/15)+✓ make lint: No violations++Successfully conformed history_table_manager.py to project standards.+12 issues fixed automatically, all tests still passing.+```++## Error Handling++- **No test file found**:+ - Automatically creates tests via test-engineer agent+ - If test creation fails, aborts the cleanup++- **Tests fail before changes**:+ - Aborts with message to fix tests first+ - Suggests using `/fixtests` command++- **Tests fail after changes**:+ - Automatically reverts all changes+ - Reports which changes caused test failures+ - Suggests manual review++- **Lint failures after fixes**:+ - Attempts secondary fixes+ - If still failing, reverts and reports++## Safety Features++1. **Test-First Approach**: Won't modify code without passing tests+2. **Incremental Changes**: Applies fixes in logical batches+3. **Automatic Rollback**: Reverts if tests fail after changes+4. **Change Validation**: Each change verified against tests+5. **Audit Trail**: Lists every change made with line numbers++## Important Notes++- **Preserves functionality** - Changes only syntax/style, never logic+- **Respects project preferences** - Maintains deliberate PEP8 deviations+- **Comments philosophy** - Removes only obviously redundant comments+- **Variable assignments** - Maintains named variables over inlining+- **Test coverage** - Ensures or creates tests before any modifications++## Limitations++- Works only with Python files in the project+- Cannot fix complex architectural issues+- Requires ability to run tests (proper environment setup)+- May not handle all edge cases in test discovery+- Does not modify third-party or system files++## Related Commands+- `/fixtests` - Fix failing tests before running conform+- `/review` - Code review without automatic fixes+- `/commit` - Commit changes after conformance+- `/test` - Run tests for a specific modulediff --git a/.claude/commands/execute.md b/.claude/commands/execute.mdindex 935510cd..f93cb5f5 100644--- a/.claude/commands/execute.md+++ b/.claude/commands/execute.md@@ -120,13 +120,21 @@ Execute complete strategic-to-delivery workflow with intelligent coordination:Task: code-quality - "Review only coding standards and project convention compliance"```- **MANDATORY CHECKPOINT**: After code review completion, PAUSE execution and present review results to user for examination before proceeding with commit/PR creation.+ **CRITICAL: Address ALL code review feedback before proceeding:**+ - Collect all review agent feedback+ - Implement ALL suggested improvements and fixes+ - Make iterative changes based on review recommendations+ - The code presented to the user must be the FINAL version after addressing all review comments+ - Do NOT just show review results - actually fix the issues first++ **MANDATORY CHECKPOINT**: After implementing all review feedback, present the FINAL improved code to user for examination before proceeding with commit/PR creation.8. **Phase 6: User Review Checkpoint** - Mandatory pause for human review:- - Present all agent review results to user- - Summarize key findings and recommendations+ - Present the FINAL code that already incorporates all review feedback+ - Summarize what improvements were made based on agent reviews+ - Show that all quality gates still pass after improvements- **PAUSE**: Wait for explicit user approval before proceeding- - Address any concerns or make requested changes+ - Address any additional concerns or make requested changes- Only proceed to Phase 7 after user approval9. **Phase 7: PR Creation** - Automated pull request generation (only after user approval):diff --git a/.gitignore b/.gitignoreindex a3ba1a86..f2ac85b6 100644--- a/.gitignore+++ b/.gitignore@@ -29,3 +29,4 @@ __pycache__//docs/**/*.html!.github/+/CLAUDE.mddiff --git a/Makefile b/Makefileindex a2b16e36..15a8c20e 100644--- a/Makefile+++ b/Makefile@@ -7,10 +7,10 @@ SCRIPTS = deploy/env-generate.py deploy/run_container.sh.DEFAULT_GOAL := fix-permissionstest:- cd src && ./manage.py test --keepdb+ cd src && ./manage.py test --keepdb $(filter-out $@,$(MAKECMDGOALS))test-fast:- cd src && ./manage.py test --keepdb --parallel 4+ cd src && ./manage.py test --keepdb --parallel 4 $(filter-out $@,$(MAKECMDGOALS))lint:cd src && flake8 --config=.flake8-ci hi/ 2>/dev/null@@ -54,3 +54,6 @@ fix-permissions:@echo "Setting execute permissions for $(SCRIPTS)"chmod +x $(SCRIPTS)+%:+ @:+diff --git a/docs/CLAUDE.md b/docs/CLAUDE.mdindex acc053d3..e1c7a84b 100644--- a/docs/CLAUDE.md+++ b/docs/CLAUDE.md@@ -2,9 +2,9 @@This file provides AI-specific guidance for Claude Code when working with this repository.-## Development Philosophy - Our Prime Directive+## Key Philosophy - Our Prime Directive-In all code we write, we strive for extremely well factored code. We are thoughtful about responsibility boundaries, encapsulation, readability and maintainability. We do not write the first code we can think of to solve the problem - we find a well factored version that does the job.+In all code we write, we strive for extremely well factored code. We are thoughtful about responsibility boundaries, encapsulation, readability and maintainability. We do not write the first code we can think of to solve the problem - we find a well factored version that does the job. We avoid assumptions and seek clarification and verification before acting.## Custom Command Ecosystem@@ -82,6 +82,13 @@ This project has extensive custom commands for workflow automation. Use them pro- **Follow command guidance** - they reference authoritative documentation- **Trust the process** - commands implement project best practices+## Tone and Behavior+- **Criticism is welcome** Please tell me when I am wrong or mistaken, or even when you think I might be wrong or mistaken.+- **Correct** Please tell me if there is a better approach than the one I am taking.+- **Be concise** I will let you know when I need longer explanations.+- **Do not flatter** Do not give compliments unless I am specifically asking for your judgement.+- **Ask** Feel free to ask many questions. If you are in doubt of my intent, don't guess. Ask.+## Key Project References- **Project Structure** `docs/dev/shared/project-structure.md`diff --git a/docs/dev/shared/coding-standards.md b/docs/dev/shared/coding-standards.mdindex 01177445..9e0e252f 100644--- a/docs/dev/shared/coding-standards.md+++ b/docs/dev/shared/coding-standards.md@@ -61,6 +61,12 @@ Checklists for writing and reviewing code.- [ ] Templates appear in a subdirectory matching their purpose: modals, panes, pages.- [ ] Template tags `load` statments near the top of the file.+**Comments**:+- [ ] No comments that state what is obvious from the naming and typing and context.+- [ ] No method docstrings that simply restate what the method name already says.+- [ ] No inline comments that describe what the code is doing (vs why).+- [ ] No comments that refer to the past, future or current work in progress.+## Code Conventions Details### No "magic" strings@@ -111,6 +117,34 @@ For readability, besides adding type hints to method parameters, we adhere to thlocation : Location, svg_path_str: str ) -> EntityPath:```+### Variable Assignment vs Inlining++We prefer explicit variable assignment over inlining function calls. This is not about minimizing lines of code - it's about readability and debuggability.++**Good** - Named intermediate values+```python+table_name = self.queryset.model._meta.db_table+logger.debug( f"Processing table: {table_name}" )++cutoff_date = datetimeproxy.now() - timedelta( days=30 )+old_records = queryset.filter( created__lt=cutoff_date )+```++**Bad** - Inlined function calls+```python+logger.debug( f"Processing table: {self.queryset.model._meta.db_table}" )++old_records = queryset.filter(+ created__lt=datetimeproxy.now() - timedelta( days=30 )+)+```++Benefits of variable assignment:+- Provides semantic naming that clarifies intent+- Easier to debug (can inspect intermediate values)+- Improves readability by breaking complex expressions+- Allows reuse without recalculation+### Explicit BooleansWe prefer to wrap all expression that evaluate to a boolean in `bool()` to make it explicit what type we are expecting:@@ -122,7 +156,7 @@ We prefer to wrap all expression that evaluate to a boolean in `bool()` to make**Bad***```- my_variable = len(my_list) == 4+ my_variable = len(my_list) == 4```### Complex Boolean Expressions@@ -184,6 +218,16 @@ def process_items(items):- Examples: `x = y + z`, `result += value`, `if count == 0`- Exception: Don't add spaces in function keyword arguments (`func(x=y)`) or type annotations+### Parentheses Spacing (Deliberate PEP8 Deviation)+- **We prefer spaces inside parentheses for enhanced readability**+- This is a deliberate deviation from PEP8 standards (E201, E202)+- Examples:+ - Good: `if ( condition ):`+ - Good: `my_function( param1, param2 )`+ - Good: `result = calculate( x + y )`+- This applies to all parentheses: function calls, conditionals, expressions+- Rationale: Extra spacing improves readability by visually separating content from delimiters+### Boolean ExpressionsWhen assigning or returning boolean values, wrap expressions in `bool()` to make intent explicit:@@ -201,6 +245,10 @@ in_modal_context = request.POST.get('context') == 'modal'The project uses two different flake8 configurations:- Development Configuration (`src/.flake8`) : Our preferred style for daily development work, with specific whitespace deviations from PEP8 for enhanced readability:+ - **E201, E202**: We use spaces inside parentheses for better visual separation+ - **E221**: We align operators and values in multi-line declarations+ - **E251**: We use spaces around keyword parameters for consistency+ - **Note**: These are deliberate choices for improved code readability, not oversights- CI Configuration (`src/.flake8-ci`): GitHub Actions enforces these standards and blocks PR merging if violations exist.## Commenting Guidelines@@ -292,19 +340,28 @@ The project uses two different flake8 configurations:#### BAD Comments - What NOT To Include-1. **Avoid commenting obvious variable purposes**+1. **Method docstrings that restate the obvious**+ - Bad: `"""Get the total number of records in the history table."""` for `_get_record_count()` in a HistoryTableManager class+ - Bad: `"""Delete records with the given IDs."""` for `_delete_records( ids )`+ - Bad: `"""Types of cleanup operation results."""` for an enum called CleanupResultType+ - The method/class name + parameters + return type already convey this information+ - Exception: Public API methods may need docstrings for documentation generation++2. **Avoid commenting obvious variable purposes**- Bad: `# Store original entity_type_str to detect changes`- - The variable name should make this clear+ - Bad: `# Human-readable message for health status` when field is named `reason: str`+ - The variable name and type should make this clear-2. **Remove work-stream artifacts**+3. **Remove work-stream artifacts**- Bad: Comments explaining why tests were removed or referencing specific issues- Comments should be timeless, not tied to particular development contexts-3. **Redundant descriptions of clear code**+4. **Redundant descriptions of clear code**- Bad: `# Track EntityType change and response needed after transaction`- - When variable names already convey this information+ - Bad: `# Check if we're over the total record limit` before `if total_count <= self.max_records_limit:`+ - When variable names and conditionals already convey this information-4. **Cryptic references**+5. **Cryptic references**- Bad: `# Recreate to preserve "max" to show new form`- If unclear, either explain properly or removediff --git a/docs/dev/testing/testing-guidelines.md b/docs/dev/testing/testing-guidelines.mdindex cbf6a9f8..02253edd 100644--- a/docs/dev/testing/testing-guidelines.md+++ b/docs/dev/testing/testing-guidelines.md@@ -12,11 +12,18 @@ Run individual tests using Django with:```bashcd $PROJ_DIR-./src/manage.py test --keepdb <test-module-goes-here>+make test <hi.test.module.goes.here>```++The make targets use the Django testing framework.+## Overall Guidelines-We only want to mock as external system boundaries and avoid mocking anything that is internal to this code base. We need to know how our internal components integrate and the test are the best way to know if something has chnaged in an incompatible way. We rarely want "pure" unit testing since mocking internal app boundaries masks issues.+YOU **MUST NOT** use mocking unless absolutely necessary. You use real system components and create test data for all our tests (using synthetic-data.py pattern). We need to know how our internal components integrate and the test are the best way to know if something has changed in an incompatible way. We rarely want "pure" unit testing since mocking internal app boundaries masks issues.++We only want to mock as external system boundaries. Anything that interacts with systems outside the code base, or that would leave remaining artifacts (e.g., API services, MEDIA_ROOT and other filesystem writes).++YOU **NEVER** write test that depend on log messages, human readable strings or other hard-coded strings. You test outcomes, not implementation details and artifacts.## High-Value vs Low-Value Testing Criteriadiff --git a/src/hi/apps/common/history_table_manager.py b/src/hi/apps/common/history_table_manager.pyindex 5fbf0363..180ccde9 100644--- a/src/hi/apps/common/history_table_manager.py+++ b/src/hi/apps/common/history_table_manager.py@@ -34,7 +34,6 @@ logger = logging.getLogger(__name__)class CleanupResultType(LabeledEnum):- """Types of cleanup operation results."""UNDER_LIMIT = ( 'Under Limit','Table is under the record limit, no cleanup needed' )@@ -53,7 +52,7 @@ class CleanupResultType(LabeledEnum):@dataclassclass CleanupResult:result_type : CleanupResultType- reason : str # Human-readable message for health status+ reason : strdeleted_count : intduration_seconds : float = 0.0@@ -90,7 +89,7 @@ class HistoryTableManager:self.min_days_retention = min_days_retentionself.max_records_limit = max_records_limitself.deletion_batch_size = deletion_batch_size- return+ return Nonedef cleanup_next_batch(self) -> CleanupResult:"""@@ -114,7 +113,6 @@ class HistoryTableManager:logger.debug(f"Starting cleanup batch for table '{table_name}'")try:- # Check if we're over the total record limittotal_count = self._get_record_count()if total_count <= self.max_records_limit:@@ -129,14 +127,12 @@ class HistoryTableManager:duration_seconds=(datetimeproxy.now() - start_time).total_seconds())- # Find records older than retention periodcutoff_date = datetimeproxy.now() - timedelta(days=self.min_days_retention)old_records_qs = self.queryset.filter(**{f"{self.date_field_name}__lt": cutoff_date}).order_by(self.date_field_name)- # Check if there are any old records eligible for deletionif not old_records_qs.exists():logger.debug(f"No records older than {cutoff_date} ({self.min_days_retention} days) "@@ -149,7 +145,6 @@ class HistoryTableManager:duration_seconds=(datetimeproxy.now() - start_time).total_seconds())- # Get IDs of next batch to delete (oldest first)ids_to_delete = list(old_records_qs.values_list('pk', flat=True)[:self.deletion_batch_size])if not ids_to_delete:@@ -161,7 +156,6 @@ class HistoryTableManager:duration_seconds=(datetimeproxy.now() - start_time).total_seconds())- # Delete the batchdeleted_count = self._delete_records(ids_to_delete)duration = (datetimeproxy.now() - start_time).total_seconds()@@ -185,15 +179,9 @@ class HistoryTableManager:raisedef _get_record_count(self) -> int:- """- Get the total number of records in the history table.- """return self.queryset.count()def _delete_records( self, ids: List[int] ) -> int:- """- Delete records with the given IDs in a single transaction.- """if not ids:return 0@@ -209,3 +197,4 @@ class HistoryTableManager:f"with IDs: {ids[:5]}{'...' if len(ids) > 5 else ''}")return deleted_count+ return 0diff --git a/src/hi/apps/user/tests/test_middleware.py b/src/hi/apps/user/tests/test_middleware.pyindex 0ac9ae83..cce5c455 100644--- a/src/hi/apps/user/tests/test_middleware.py+++ b/src/hi/apps/user/tests/test_middleware.py@@ -35,6 +35,7 @@ class TestAuthenticationMiddleware(BaseTestCase):# Verify exempt URL names are definedexpected_exempt_urls = {'admin',+ 'manifest','user_signin','user_signin_magic_code','user_signin_magic_link',