Skip to content

armgit / ARES

Public Imported from GitHub
0

ARES - authorized red-team engagement automation with dashboard, campaign scope, module orchestration, OPSEC controls, encrypted vault, and reporting.

fix(data): normalizer dual-read fallback - recover 70% artifact data loss (MOD-006, MOD-016 + systematic fix per NORMALIZER_CONTRACT_AUDIT.md)

Mafifrizi1bb9791550fb
diff --git a/KEY_CONVENTION_STANDARD.md b/KEY_CONVENTION_STANDARD.md
new file mode 100644
index 0000000..21ce3c0
--- /dev/null
+++ b/KEY_CONVENTION_STANDARD.md
@@ -0,0 +1,232 @@
+# ARES Standard Key Convention: Telemetry, Artifact Normalizer & Module Contract
+
+> **Dokumen Status**: Spesifikasi Standar Kontrak Data (Architecture Decision Record)
+> **Tanggal Penetapan**: 26 September 2026
+> **Target Implementasi**: `ares/normalize/artifacts.py` & seluruh modul di `ares/modules/`
+> **Tujuan**: Menghilangkan 70% data mismatch antara output modul (`raw`) dan `ArtifactNormalizer`, menjamin integritas aliran data telemetry ofensif ke `ArtifactStore`, dan memvalidasi rantai eksekusi otomatis (*attack chaining*).
+
+---
+
+## 1. Tabel Standar Kunci Normalizer (Section 1)
+
+Tabel berikut menetapkan **satu nama kunci standar tunggal** untuk setiap tipe data telemetry di ARES. Penetapan ini mencakup tipe data yang saat ini sudah berstatus MATCH (dipertahankan) dan tipe data yang berstatus MISMATCH (ditargetkan untuk penyelarasan).
+
+| No | Tipe Data Telemetry | Kunci Standar (`raw`) | Alasan Pemilihan Standar | Modul yang Wajib Menulis Kunci Ini | Handler Normalizer yang Wajib Membaca Kunci Ini | Status Kontrak Saat Ini |
+|:---:|---|---|---|---|---|:---:|
+| 1 | **NTLM Hashes** | `hashes` | Sudah digunakan oleh `windows.lsass_dump` dan berstatus MATCH. Menjadi jangkar konvensi untuk seluruh artefak hash crackable. | `ad.dcsync`, `windows.lsa_secrets`, `windows.lsass_dump` | `_normalize_ntlm_hashes` (`artifacts.py:549`) | ❌ MISMATCH pada 2 modul |
+| 2 | **Kerberos TGS Hashes** | `hashes` | Selaras penuh dengan standar `hashes`. Normalizer membedakan tipe hash berdasarkan capability (`ad.kerberoast` $\rightarrow$ mode 13100). | `ad.kerberoast` | `_normalize_kerberos_hashes` (`artifacts.py:507`) | ❌ MISMATCH |
+| 3 | **AS-REP Hashes** | `hashes` | Selaras penuh dengan standar `hashes`. Normalizer membedakan tipe hash berdasarkan capability (`ad.asreproast` $\rightarrow$ mode 18200). | `ad.asreproast` | `_normalize_asrep_hashes` (`artifacts.py:529`) | ❌ MISMATCH |
+| 4 | **User Accounts** | `users` | Konvensi plural noun standar Python/REST API (`users`, `computers`, `hashes`). Normalizer sudah mengimplementasikan `raw.get("users")`. | `ad.enum_users` | `_normalize_users` (`artifacts.py:473`) | ❌ MISMATCH |
+| 5 | **Computer / Host Accounts** | `computers` | Konvensi plural noun standar. Normalizer sudah membaca `raw.get("computers")`. | `ad.enum_computers` | `_normalize_computers` (`artifacts.py:491`) | ❌ MISMATCH |
+| 6 | **SPN Accounts** | `spns` | Konvensi plural noun standar. Normalizer sudah membaca `raw.get("spns")`. | `ad.enum_spn` | `_normalize_spns` (`artifacts.py:566`) | ❌ MISMATCH |
+| 7 | **ACL Misconfigurations** | `misconfigs` | Sudah diterapkan secara seragam oleh `ad.enum_acl` dan normalizer berstatus MATCH penuh. Pertahankan untuk menghindari breaking change. | `ad.enum_acl` | `_normalize_permissions` (`artifacts.py:584`) | ✅ MATCH |
+| 8 | **Cloud S3 Buckets** | `public_buckets` (dalam `s3`) | Sesuai dengan struktur nested dictionary `raw["s3"]["public_buckets"]` di `cloud.aws` yang sudah MATCH. | `cloud.aws` | `_normalize_cloud` (`artifacts.py:600`) | ✅ MATCH |
+| 9 | **Target Host / Privesc Host** | `target` | Universal parameter identifier di seluruh ARES (`ExecutionContext.target`, `BaseModule`, `Finding.host`). Mayoritas modul (6 dari 9) sudah menulis `raw["target"]`. | `linux.ld_preload`, `linux.nfs_escape`, `linux.service_hijack`, `linux.kernel_suggester`, `linux.privesc`, `windows.applocker_bypass`, `windows.scheduled_tasks_enum`, `windows.token_impersonation`, `windows.uac_bypass` | `_normalize_host_vuln` (`artifacts.py:615`) | ❌ MISMATCH pada 6 modul |
+
+---
+
+## 2. Kasus Khusus & Analisis Konflik Semantik (Section 2)
+
+### 2.1. Konflik Kunci: `target` vs `host`
+- **Permasalahan**: Pada penangan `privesc_vectors`, normalizer membaca `raw.get("host", "")`.
+ - 3 modul Linux (`ld_preload`, `nfs_escape`, `service_hijack`) menulis `raw["host"]` $\rightarrow$ status ✅ MATCH.
+ - 6 modul lain (2 Linux, 4 Windows: `kernel_suggester`, `privesc`, `applocker_bypass`, `scheduled_tasks_enum`, `token_impersonation`, `uac_bypass`) menulis `raw["target"]` $\rightarrow$ status ❌ MISMATCH.
+- **Analisis Trade-Off**:
+ - Jika `host` dijadikan standar: 6 modul harus diubah, padahal seluruh arsitektur ARES (`ExecutionContext`, parameter models, CLI, engine dispatcher) menggunakan `target` sebagai nama variabel kanonikal.
+ - Jika `target` dijadikan standar: Hanya 3 modul yang perlu diubah di sisi modul, dan normalizer diperbarui untuk memprioritaskan `target` sebelum fallback ke `host`.
+- **Rekomendasi Keputusan**: **Standar kanonikal adalah `target`**.
+ Normalizer `_normalize_host_vuln` diperbarui menjadi:
+ ```python
+ target = raw.get("target") or raw.get("host", "")
+ ```
+ Modul yang saat ini hanya menulis `host` akan dimutakhirkan untuk menulis `target` (dengan `host` dipertahankan sebagai alias).
+
+---
+
+### 2.2. Konflik Generik vs Spesifik: `hashes` vs `ntlm_hashes` / `kerberos_hashes` / `asrep_hashes`
+- **Permasalahan**: Pengembang modul cenderung memberi nama key spesifik teknik (`kerberos_hashes`, `asrep_hashes`, `ntlm_hashes`), sedangkan normalizer mengharapkan key generik `hashes`.
+- **Analisis Trade-Off**:
+ - Key spesifik teknik memudahkan keterbacaan manusia saat membaca JSON mentah, tetapi memaksa normalizer memiliki logika parser berbeda untuk setiap format dict.
+ - Key generik `hashes` memungkinkan satu fungsi pipeline konsumsi (misal cracker runner) memproses seluruh hash tanpa bergantung pada nama teknik asal.
+ - Modul `windows.lsass_dump` membuktikan solusi elegan: ia menuliskan **kedua key sekaligus** (`raw["hashes"]` dan `raw["ntlm_hashes"]`), sehingga lolos verifikasi normalizer tanpa merusak backward compatibility.
+- **Rekomendasi Keputusan**: **Standar kanonikal adalah `hashes`**, dengan **Dual-Write Policy**.
+ Setiap modul penghasil hash wajib menuliskan key `hashes` (list of strings atau list of dicts), serta mempertahankan key spesifik teknik sebagai alias backward compatibility:
+ ```python
+ raw["hashes"] = hashes
+ raw["ntlm_hashes"] = hashes # alias backward-compat
+ ```
+
+---
+
+### 2.3. Tabrakan Semantik: `aws_findings` pada `cloud.aws` vs `cloud.aws_privesc`
+- **Permasalahan**: Kedua modul mendeklarasikan capability `aws_findings`.
+ - `cloud.aws` mengekstrak S3 public buckets (`raw["s3"]["public_buckets"]`) $\rightarrow$ dicerna oleh `_normalize_cloud` menjadi `CloudResourceArtifact`.
+ - `cloud.aws_privesc` mengekstrak jalur eskalasi hak akses IAM (`raw["privesc_paths"]`) $\rightarrow$ dikirim ke `_normalize_cloud` yang mengharapkan `raw["s3"]`, menghasilkan **0 artifact (kegagalan 100%)**.
+- **Analisis Trade-Off**:
+ - Menggabungkan IAM privilege escalation ke dalam normalizer S3 bucket adalah *type confusion*. IAM role escalation path seharusnya menjadi `PermissionArtifact` atau subtype `CloudResourceArtifact(resource_type="iam_role")`.
+- **Rekomendasi Keputusan**: **Pemisahan Semantik Capability**.
+ - `cloud.aws` tetap memegang `aws_findings` (fokus pada asset cloud & S3 storage).
+ - `cloud.aws_privesc` memisahkan diri ke capability kanonikal `aws_privesc_paths` atau `cloud_permissions`, dan normalizer menambahkan handler `_normalize_cloud_permissions`.
+
+---
+
+### 2.4. Konvensi Penamaan: Plural Noun (`users`) vs List Suffix (`user_list`)
+- **Permasalahan**: Modul AD enumerasi mendeklarasikan `OUTPUTS = ["user_list", "computer_list", "spn_list"]` dan menulis `raw["user_list"]`, sedangkan normalizer membaca `raw.get("users")`.
+- **Analisis Trade-Off**:
+ - `user_list` mencerminkan nama capability deklaratif.
+ - `users` mencerminkan objek data koleksi (standar industri JSON / OpenAPI schema).
+- **Rekomendasi Keputusan**: **Standar kanonikal adalah Plural Noun (`users`, `computers`, `spns`)**.
+ Modul mengadopsi Dual-Write (`raw["users"] = users; raw["user_list"] = users`), dan normalizer mendukung fallback pembacaan `raw.get("users") or raw.get("user_list", [])`.
+
+---
+
+## 3. Prioritas Penambahan Handler Baru (UNHANDLED OUTPUTS) (Section 3)
+
+Dari 86 capability yang saat ini belum ditangani oleh `ArtifactNormalizer`, ditetapkan prioritas implementasi berdasarkan **urgensi rantai serangan otomatis (*autonomous attack chaining*)** dan **pencegahan penguapan kredensial (*credential evaporation*)**:
+
+```
+┌─────────────────────────────────────────────────────────────────────────────┐
+│ PRIORITAS IMPLEMENTASI HANDLER │
+├─────────────────────────────────────────────────────────────────────────────┤
+│ [P0] HARUS ADA (Critical) ──► Kredensial, Hash Cracking, Port Discovery │
+│ [P1] SEBAIKNYA ADA (High) ──► Domain Hierarchy, Sesi Lateral, DNS Recon │
+│ [P2] BISA DITUNDA (Medium) ──► Telemetry Rules, Raw Logs, Exfil Paths │
+└─────────────────────────────────────────────────────────────────────────────┘
+```
+
+### 3.1. Kategori [P0] HARUS ADA — Critical for Downstream & Autonomous Chaining
+Tanpa handler ini, kredensial tingkat tinggi yang berhasil diekstraksi menguap dari runtime state dan tidak dapat digunakan oleh modul downstream:
+
+| Capability Output | Modul Penghasil Kunci | Kunci Standar yang Ditulis | Tipe Artifact Target | Alasan Urgensi P0 |
+|---|---|---|---|---|
+| `cleartext_credentials` | `ad.sccm`, `windows.dpapi`, `linux.samba_secrets` | `credentials` | `CredentialArtifact(cred_type="cleartext")` | Password plaintext hasil ekstraksi NAA/DPAPI/Samba harus masuk ke `ArtifactStore` agar dapat langsung dicoba oleh `credential.reuse` dan lateral movement. |
+| `cracked_credentials` | `credential.crack` | `cracked_credentials` | `CredentialArtifact(cracked=True)` | Jantung autonomous chaining: setelah hash di-crack oleh hashcat, hasilnya wajib terserap kembali ke store sebagai kredensial valid untuk eskalasi hak akses. |
+| `laps_passwords` | `ad.laps_enum` | `laps_passwords` | `CredentialArtifact(cred_type="cleartext", privilege="local_admin")` | Password administrator lokal seluruh armada workstation/server domain. Penguapan data ini memutus jalur lateral movement domain. |
+| `kerberos_ticket` / `kerberos_tickets` | `lateral.ntlm_relay`, `credential.golden_ticket`, `credential.ticket_converter` | `tickets` | `CredentialArtifact(cred_type="kerberos_ticket")` | Tiket TGS/TGT forged (.ccache / .kirbi) diperlukan untuk Pass-the-Ticket dan eksekusi command via Kerberos authentication. |
+| `open_ports` & `service_map` | `network.port_scan`, `network.service_detect` | `open_ports`, `service_map` | `HostArtifact(open_ports=...)` | Graph jaringan tidak mengetahui port dan servis terbuka pada target jika scan reconnaissance diabaikan oleh normalizer. |
+
+---
+
+### 3.2. Kategori [P1] SEBAIKNYA ADA — High Value for State Graph & Pathfinding
+Handler ini memperkaya visualisasi attack graph dan pemetaan topologi Active Directory / Cloud:
+
+| Capability Output | Modul Penghasil | Kunci Standar | Tipe Artifact Target | Manfaat Arsitektural |
+|---|---|---|---|---|
+| `domain_controllers` / `domain_info` | `recon.fingerprint`, `ad.dependencies` | `domain_controllers` | `DomainArtifact` | Mengidentifikasi target Tier-0 (Domain Controllers) untuk prioritas serangan DCSync. |
+| `lateral_session` / `powershell_session` | `lateral.psexec`, `lateral.wmiexec`, `lateral.winrm`, `lateral.ssh_pivot` | `session_host` | Update `HostArtifact(is_compromised=True)` | Memperbarui status kompromi host pada state graph campaign secara real-time. |
+| `relay_candidates` / `smb_signing_config` | `lateral.smb_relay` | `relay_candidates` | Update `HostArtifact(smb_signing_required=False)` | Memetakan host-host yang rentan terhadap NTLM relay attack ke attack graph. |
+| `dns_records` / `subdomains` | `network.dns_enum` | `dns_records` | `HostArtifact` | Menambahkan host-host baru hasil resolusi DNS ke daftar target campaign. |
+| `privesc_paths` (Cloud IAM) | `cloud.aws_privesc` | `privesc_paths` | `PermissionArtifact` | Memetakan jalur eskalasi role IAM cloud. |
+
+---
+
+### 3.3. Kategori [P2] BISA DITUNDA — Informational, Raw Logs & Artifact Metadata
+Data pada kategori ini sudah memiliki jalur persistensi tersendiri (misal tabel database `loot`, `evidence`, atau return raw output modul) sehingga tidak mendesak untuk diubah menjadi `NormalizedArtifact`:
+
+- `command_output` (stdout mentah eksekusi perintah shell; sudah tersimpan di `ModuleResult.raw`).
+- `evidence_chain` & `evidence_integrity` (SHA-256 Merkle records; ditangani langsung oleh SDK provenance layer).
+- `loot` (KQL query & Sigma detection rules; sudah dipersistensikan langsung ke tabel `loot` di Postgres/SQLite).
+- `sensitive_file_paths` / `collection_inventory` (inventaris path file hasil staged collection).
+- `socks5_proxy` / `pivot_tunnel` (state socket tunnel dikelola langsung oleh `PivotManager`).
+
+---
+
+## 4. Estimasi Scope Perubahan (Section 4)
+
+Berikut adalah perkiraan kuantitatif perubahan codebase saat fase eksekusi standardisasi kontrak dijalankan:
+
+### 4.1. Rincian File yang Terdampak
+1. **File Normalizer Core**:
+ - `ares/normalize/artifacts.py` (1 file, penambahan logika dual-read fallback pada 9 handler yang ada + 5 handler baru kategori P0).
+2. **File Modul Ofensif yang Perlu Dual-Write**:
+ - `ares/modules/ad/kerberoast.py` (`hashes`)
+ - `ares/modules/ad/asreproast.py` (`hashes`)
+ - `ares/modules/ad/dcsync.py` (`hashes`)
+ - `ares/modules/ad/enum_users.py` (`users`)
+ - `ares/modules/ad/enum_computers.py` (`computers`)
+ - `ares/modules/ad/enum_spn.py` (`spns`)
+ - `ares/modules/windows/lsa_secrets.py` (`hashes`)
+ - `ares/modules/linux/kernel_suggester.py` (`target`)
+ - `ares/modules/linux/privesc.py` (`target`)
+ - `ares/modules/windows/applocker_bypass.py` (`target`)
+ - `ares/modules/windows/scheduled_tasks_enum.py` (`target`)
+ - `ares/modules/windows/token_impersonation.py` (`target`)
+ - `ares/modules/windows/uac_bypass.py` (`target`)
+ - `ares/modules/cloud/aws_privesc.py` (`privesc_paths`)
+ - `ares/modules/exfil/staged_collection.py` (`sensitive_file_paths`)
+ - *Total Modul*: **15 file modul**.
+3. **File Test Suite**:
+ - `tests/unit/test_staged_modules.py`
+ - `tests/unit/test_state_graph_fingerprint_service.py`
+ - `tests/simulation/test_scenarios.py`
+ - `tests/unit/test_executor_pivot_lateral.py`
+ - *File Test Baru yang Wajib Dibuat*: `tests/unit/test_artifact_normalizer_pipeline.py` (End-to-End Normalizer Pipeline Test).
+ - *Total File Test*: **5 file**.
+
+**Total Keseluruhan**: **21 file**.
+
+---
+
+### 4.2. Analisis Risiko Breaking Change ke Konsumen Eksternal
+- **Risiko Potensial**:
+ Jika key lama langsung dihapus (misal mengganti `raw["kerberos_hashes"]` secara destruktif dengan `raw["hashes"]`), API client eksternal, dashboard UI, script integrasi, atau execution chain legacy yang membaca `result.raw_output["kerberos_hashes"]` akan mengalami `KeyError`.
+- **Mitigasi Zero-Breaking-Change (Wajib)**:
+ Terapkan **Dual-Write Policy** pada modul:
+ ```python
+ raw["hashes"] = hashes # Kunci Standar Kanonikal Baru
+ raw["kerberos_hashes"] = raw["hashes"] # Alias Backward Compatibility
+ ```
+ Dengan kebijakan ini:
+ - 100% downstream baru dan `ArtifactNormalizer` membaca `raw["hashes"]`.
+ - 100% consumer lama tetap dapat membaca `raw["kerberos_hashes"]`.
+ - **Zero Breaking Change** terhadap REST API `/campaigns/{id}/results` maupun SDK consumers.
+
+---
+
+## 5. Urutan Eksekusi Fix yang Disarankan (Section 5)
+
+Dua strategi migrasi dianalisis untuk menentukan urutan eksekusi yang paling aman:
+
+### Perbandingan Opsi Eksekusi:
+
+| Parameter Evaluasi | Opsi A: Normalizer Dual-Read Fallback First (Direkomendasikan) | Opsi B: Big-Bang Simultaneous Change |
+|---|---|---|
+| **Alur Langkah** | 1. Update `artifacts.py` membaca `standar or legacy`.<br>2. Update modul menulis `standar` (dual-write).<br>3. Tambah test pipeline integrasi.<br>4. Tandai legacy key sebagai deprecated. | 1. Update seluruh 15 modul sekaligus.<br>2. Update normalizer.<br>3. Update semua test sekaligus. |
+| **Ketahanan terhadap Regresi** | **Sangat Tinggi**. Pada langkah 1, normalizer langsung berhasil memproses modul lama tanpa menunggu modul di-update. | **Rendah**. Jika 1 modul terlewat, data modul tersebut tetap hilang tanpa peringatan. |
+| **Downtime / Test Breakage** | **Zero Test Breakage**. Seluruh test yang ada tetap lulus di setiap commit. | Test berpotensi gagal massal sampai seluruh file selesai diedit secara sempurna. |
+| **Kepatuhan Rule 3 (Root-Cause)** | **Sempurna**. Memperbaiki normalizer di tingkat root terlebih dahulu sebagai guardrail defensif. | Rentan menimbulkan patch kosmetik lokal yang tidak seragam. |
+
+---
+
+### Roadmap Eksekusi Bertahap (Rekomendasi Resmi: OPSI A)
+
+```mermaid
+graph TD
+ A[Fase 1: Normalizer Multi-Key Fallback] -->|Normalizer membaca Kunci Standar ATAU Kunci Lama| B[Data Evaporation 70% Teratasi Seketika]
+ B --> C[Fase 2: Dual-Write pada 15 Modul Ofensif]
+ C -->|Modul menulis Kunci Standar & Alias Lama| D[Zero Breaking Change pada API & UI]
+ D --> E[Fase 3: Tambahkan Handler Baru P0]
+ E -->|cleartext_cred, cracked_cred, laps, tickets, ports| F[Autonomous Chaining Berfungsi Penuh]
+ F --> G[Fase 4: End-to-End Pipeline Test Suite]
+ G -->|tests/unit/test_artifact_normalizer_pipeline.py| H[Verifikasi Rigor Selesai 100%]
+```
+
+1. **Fase 1 (Normalizer Multi-Key Dual-Read)**:
+ Perbarui method `_normalize_*()` pada `ares/normalize/artifacts.py` agar membaca kunci standar dengan fallback ke kunci lama:
+ - `_normalize_kerberos_hashes`: `raw.get("hashes") or raw.get("kerberos_hashes", [])`
+ - `_normalize_asrep_hashes`: `raw.get("hashes") or raw.get("asrep_hashes", [])`
+ - `_normalize_ntlm_hashes`: `raw.get("hashes") or raw.get("ntlm_hashes") or raw.get("sam_hashes", [])`
+ - `_normalize_users`: `raw.get("users") or raw.get("user_list", [])`
+ - `_normalize_computers`: `raw.get("computers") or raw.get("computer_list", [])`
+ - `_normalize_spns`: `raw.get("spns") or raw.get("spn_list", [])`
+ - `_normalize_host_vuln`: `raw.get("target") or raw.get("host", "")`
+ *Hasil*: Masalah 70% data hilang **langsung terselesaikan seketika**, bahkan sebelum modul diubah.
+
+2. **Fase 2 (Modul Dual-Write Implementation)**:
+ Perbarui 15 file modul ofensif untuk menghasilkan kunci standar sebagai properti primer, dan menyalin referensinya ke kunci lama sebagai properti alias.
+
+3. **Fase 3 (Implementasi Handler Baru Kategori P0)**:
+ Tambahkan 5 handler baru di `artifacts.py` untuk mengolah `cleartext_credentials`, `cracked_credentials`, `laps_passwords`, `kerberos_tickets`, dan `open_ports`.
+
+4. **Fase 4 (Pembuatan Test Suite Pipeline & Verifikasi Rule 5)**:
+ Buat file pengujian komprehensif `tests/unit/test_artifact_normalizer_pipeline.py` yang memvalidasi bahwa setiap modul dari Batch 1 s.d. Batch 12 berhasil menyalurkan datanya ke `ArtifactStore` dengan status 100% MATCH.
diff --git a/NORMALIZER_CONTRACT_AUDIT.md b/NORMALIZER_CONTRACT_AUDIT.md
new file mode 100644
index 0000000..ff69934
--- /dev/null
+++ b/NORMALIZER_CONTRACT_AUDIT.md
@@ -0,0 +1,141 @@
+# ARES Architectural Contract Audit: Artifact Normalizer & Pipeline Key Mismatches
+
+> **Status**: Comprehensive Systemic Audit
+> **Scope**: `ares/normalize/artifacts.py` vs seluruh 70 modul di `ares/modules/`
+> **Tanggal Audit**: 26 September 2026
+> **Temuan Kunci**: Kerusakan sistemik pada pipeline normalisasi data telemetry ofensif ke `ArtifactStore`. Dari 9 capability yang didukung oleh `ArtifactNormalizer`, **hanya 2 capability yang berfungsi penuh (MATCH)**, 1 capability PARTIAL, dan **6 capability mengalami TOTAL MISMATCH (data 100% hilang)**. Selain itu, terdapat **86 capability output modul yang sama sekali tidak memiliki handler (UNHANDLED OUTPUT)**.
+
+---
+
+## 1. Ringkasan Eksekutif & Statistik Kontrak
+
+Dalam arsitektur ARES, setelah setiap modul berhasil dieksekusi, `AresEngine.run_module` memanggil:
+```python
+# ares/core/engine.py:1336
+normalized = ArtifactNormalizer().normalize(
+ module_id=module_id,
+ outputs=outputs,
+ raw=result.raw_output,
+ store=runtime_state.artifact_store,
+)
+```
+`ArtifactNormalizer` bertanggung jawab mengonversi dictionary `raw` hasil modul menjadi objek bertipe terstruktur (`UserArtifact`, `HostArtifact`, `HashArtifact`, `PermissionArtifact`, `CloudResourceArtifact`) di dalam `runtime_state.artifact_store`. Objek-objek ini kemudian menjadi input bagi:
+1. Modul downstream (misal `credential.crack` yang mencari `HashArtifact`, `credential.pass_spray` yang mencari `UserArtifact`).
+2. Generator attack graph dan state graph (`StateGraphService`).
+3. Pelaporan dan visualisasi campaign.
+
+### Statistik Keselarasan Kontrak Normalizer
+
+| Kategori Status Kontrak | Jumlah Pasangan Modul-Capability | Persentase | Status Data |
+|---|:---:|:---:|---|
+| ✅ **MATCH (Selaras Penuh)** | 6 pasangan modul | 30.0% | Data mengalir sempurna ke `ArtifactStore` |
+| ❌ **MISMATCH (Kunci Tidak Cocok)** | 14 pasangan modul | 70.0% | **Data 100% dibuang/hilang dari ArtifactStore** |
+| ⚠️ **ORPHANED HANDLER** | 0 handler | 0.0% | Semua 9 handler memiliki minimal 1 modul deklarator |
+| 🚫 **UNHANDLED OUTPUTS** | **86 capability** | - | Output modul diabaikan sepenuhnya oleh normalizer |
+
+---
+
+## 2. Tabel Lengkap Pemetaan Kontrak Normalizer
+
+Tabel di bawah ini memetakan seluruh 9 capability yang memiliki handler di `ares/normalize/artifacts.py:457-468` terhadap modul-modul yang mendeklarasikannya di `OUTPUTS`:
+
+| Capability | Modul Penghasil | Key yang DIBACA Normalizer | Key yang DITULIS Modul (`raw`) | Status | Dampak Sistemik & Downstream Failure |
+|---|---|---|---|:---:|---|
+| `user_list` | `ad.enum_users` | `users` (list of dict) | `user_list` | ❌ **MISMATCH** | **`UserArtifact` tidak pernah dibuat**. Modul downstream (`ad.kerberoast`, `credential.pass_spray`) tidak menerima daftar user via store. *(Modul melakukan bypass sementara via lokal `ISU-07`)*. |
+| `computer_list` | `ad.enum_computers` | `computers` (list of dict) | `computer_list` | ❌ **MISMATCH** | **`HostArtifact` tidak pernah dibuat** oleh normalizer. Graph jaringan tidak terisi otomatis dari enumerasi AD. |
+| `kerberos_hashes` | `ad.kerberoast` | `hashes` (list of str), `accounts` | `kerberos_hashes` (list of str) | ❌ **MISMATCH** | **100% hash Kerberoasting hilang**. `HashArtifact` (mode 13100) tidak pernah masuk ke `ArtifactStore`. `credential.crack` tidak dapat memecahkan hash secara otomatis. |
+| `asrep_hashes` | `ad.asreproast` | `hashes` (list of str) | `asrep_hashes` (list of str) | ❌ **MISMATCH** | **100% hash AS-REP hilang**. `HashArtifact` (mode 18200) tidak pernah masuk ke `ArtifactStore`. Pipeline cracking otomatis lumpuh total. |
+| `ntlm_hashes` | `ad.dcsync` | `hashes` (list of dict) | `ntlm_hashes` (list of dict) | ❌ **MISMATCH** | **100% hash domain DCSync hilang**. Hash admin domain tidak tersimpan di `ArtifactStore` dan tidak pernah dipersistensikan ke database. |
+| `ntlm_hashes` | `windows.lsass_dump` | `hashes` (list of dict) | `hashes`, `ntlm_hashes` | ✅ **MATCH** | Data mengalir benar ke `HashArtifact` (mode 1000). Modul ini secara kebetulan menuliskan kedua key (`hashes` dan `ntlm_hashes`). |
+| `ntlm_hashes` | `windows.lsa_secrets` | `hashes` (list of dict) | `sam_hashes`, `ntlm_hashes` | ❌ **MISMATCH** | Hash SAM/LSA dari target Windows tidak terserap ke normalizer karena modul menulis ke `sam_hashes` dan `ntlm_hashes`, bukan `hashes`. |
+| `spn_list` | `ad.enum_spn` | `spns` (list of dict) | `spn_list` | ❌ **MISMATCH** | **0 UserArtifact SPN dibuat**. Atribut `is_kerberoastable` tidak terpetakan di `ArtifactStore`. |
+| `acl_findings` | `ad.enum_acl` | `misconfigs` (list of dict) | `misconfigs`, `acl_findings` | ✅ **MATCH** | Data mengalir benar ke `PermissionArtifact`. Modul menuliskan kedua key. |
+| `aws_findings` | `cloud.aws` | `region`, `s3` (`public_buckets`) | `region`, `s3`, `aws_findings` | ✅ **MATCH** | S3 bucket publik berhasil dikonversi menjadi `CloudResourceArtifact`. |
+| `aws_findings` | `cloud.aws_privesc` | `region`, `s3` (`public_buckets`) | `privesc_paths`, `aws_findings` | ❌ **MISMATCH** | Modul privilege escalation AWS tidak menghasilkan `s3`, sehingga normalizer mengembalikan 0 artifact. Jalur eskalasi IAM hilang dari store. |
+| `privesc_vectors` | `linux.ld_preload` | `host` | `host`, `privesc_vectors` | ✅ **MATCH** | Menghasilkan `HostArtifact` dengan IP/hostname target. *(Catatan: detail vektor privesc itu sendiri tidak disimpan)*. |
+| `privesc_vectors` | `linux.nfs_escape` | `host` | `host`, `privesc_vectors` | ✅ **MATCH** | Menghasilkan `HostArtifact` dengan IP/hostname target. |
+| `privesc_vectors` | `linux.service_hijack` | `host` | `host`, `privesc_vectors` | ✅ **MATCH** | Menghasilkan `HostArtifact` dengan IP/hostname target. |
+| `privesc_vectors` | `linux.kernel_suggester` | `host` | `target`, `privesc_vectors` | ❌ **MISMATCH** | Normalizer mencari `raw["host"]`, modul menulis `raw["target"]`. HostArtifact tidak dibuat. |
+| `privesc_vectors` | `linux.privesc` | `host` | `target`, `privesc_vectors` | ❌ **MISMATCH** | Normalizer mencari `raw["host"]`, modul menulis `raw["target"]`. HostArtifact tidak dibuat. |
+| `privesc_vectors` | `windows.applocker_bypass` | `host` | `target`, `privesc_vectors` | ❌ **MISMATCH** | Normalizer mencari `raw["host"]`, modul menulis `raw["target"]`. HostArtifact tidak dibuat. |
+| `privesc_vectors` | `windows.scheduled_tasks_enum` | `host` | `target`, `privesc_vectors` | ❌ **MISMATCH** | Normalizer mencari `raw["host"]`, modul menulis `raw["target"]`. HostArtifact tidak dibuat. |
+| `privesc_vectors` | `windows.token_impersonation` | `host` | `target`, `privesc_vectors` | ❌ **MISMATCH** | Normalizer mencari `raw["host"]`, modul menulis `raw["target"]`. HostArtifact tidak dibuat. |
+| `privesc_vectors` | `windows.uac_bypass` | `host` | `target`, `privesc_vectors` | ❌ **MISMATCH** | Normalizer mencari `raw["host"]`, modul menulis `raw["target"]`. HostArtifact tidak dibuat. |
+
+---
+
+## 3. Analisis Mendalam Akar Masalah Kunci (Root Cause Analysis)
+
+### 3.1. Pola Mismatch "Generic vs Explicit Key"
+Normalizer awal ditulis dengan asumsi bahwa modul akan menulis key generik pendek (`"users"`, `"computers"`, `"hashes"`, `"spns"`). Namun, pengembang modul memberi nama key di dalam `raw` yang selaras dengan nama capability atau ID modul (`"user_list"`, `"computer_list"`, `"kerberos_hashes"`, `"asrep_hashes"`, `"ntlm_hashes"`, `"spn_list"`).
+
+Akibatnya:
+- Method `_normalize_kerberos_hashes` mencari `raw.get("hashes")` $\rightarrow$ modul menghasilkan `raw["kerberos_hashes"]` $\rightarrow$ **Hasil: List Kosong (0 artifacts)**.
+- Method `_normalize_asrep_hashes` mencari `raw.get("hashes")` $\rightarrow$ modul menghasilkan `raw["asrep_hashes"]` $\rightarrow$ **Hasil: List Kosong (0 artifacts)**.
+- Method `_normalize_ntlm_hashes` mencari `raw.get("hashes")` $\rightarrow$ modul menghasilkan `raw["ntlm_hashes"]` $\rightarrow$ **Hasil: List Kosong (0 artifacts)**.
+
+### 3.2. Pola Mismatch "Host vs Target"
+Method `_normalize_host_vuln` (penangan capability `privesc_vectors`) membaca `host = raw.get("host", "")`.
+- 3 modul Linux (`ld_preload`, `nfs_escape`, `service_hijack`) menulis `raw["host"]`.
+- 6 modul lain (2 Linux dan 4 Windows: `kernel_suggester`, `privesc`, `applocker_bypass`, `scheduled_tasks_enum`, `token_impersonation`, `uac_bypass`) menulis `raw["target"]`.
+Akibatnya, seluruh modul Windows privesc gagal mencatatkan host ke `ArtifactStore`.
+
+### 3.3. Mengapa Bug Ini Tersembunyi dari Unit & Integration Test?
+1. **Mocking Terisolasi**: Pada `tests/unit/test_state_graph_fingerprint_service.py` dan `tests/simulation/test_scenarios.py`, test membuat objek `store = ArtifactStore()` secara manual lalu memanggil method mutator langsung (`store.add(HashArtifact(...))`).
+2. **Ketiadaan End-to-End Test Normalizer**: Tidak ada satupun test di `tests/` yang menjalankan modul lalu meneruskan `result.raw_output` ke `ArtifactNormalizer().normalize()`.
+3. **Penyembunyian Exception Diam-diam**: Pada [`artifacts.py:449-450`](file:///c:/Users/ASUS/Desktop/ARES/ares/normalize/artifacts.py#L449-L450):
+ ```python
+ except Exception:
+ pass # Never let normalization failures block the engine
+ ```
+ Blok `try ... except Exception: pass` tanpa logging error menelan kegagalan normalisasi secara diam-diam.
+
+---
+
+## 4. Analisis Unhandled Outputs (86 Capability Tanpa Handler)
+
+Sebanyak 86 capability yang dideklarasikan dalam `OUTPUTS` oleh berbagai modul di ARES sama sekali **tidak memiliki handler di `ArtifactNormalizer`**. Ketika capability ini diproses, normalizer hanya melewati dengan `return 0`.
+
+### Kategori Kritis yang Mengalami "Data Evaporation":
+
+1. **Kredensial & Rahasia Plaintext / Hash (Severity: CRITICAL)**
+ - Capability: `cleartext_credentials`, `cracked_credentials`, `laps_passwords`, `cached_credentials`, `cached_hashes`, `browser_passwords`, `samba_secrets`, `lsa_secrets`, `machine_credentials`.
+ - Modul Terkenal: `ad.sccm`, `credential.crack`, `ad.laps_enum`, `windows.dpapi`, `linux.samba_secrets`.
+ - *Dampak*: Password plaintext hasil cracking, password LAPS Domain Admin, dan kredensial SCCM NAA menguap saat modul selesai dan tidak tersimpan di `ArtifactStore`.
+
+2. **Tiket Kerberos & Token Sesi (Severity: HIGH)**
+ - Capability: `kerberos_ticket`, `kerberos_tickets`, `converted_ticket`, `golden_ticket`, `access_tokens`, `oauth_tokens`, `prt_artifacts`.
+ - Modul Terkenal: `lateral.ntlm_relay`, `ad.delegation_abuse`, `credential.golden_ticket`, `credential.ticket_converter`.
+ - *Dampak*: Path file `.ccache` atau `.kirbi` yang dihasilkan modul ofensif tidak dicatat ke state store, memutus rantai serangan otomatis (attack chaining).
+
+3. **Infrastruktur Jaringan & Discovery (Severity: HIGH)**
+ - Capability: `open_ports`, `service_map`, `service_versions`, `dns_records`, `subdomains`, `relay_targets`, `relay_candidates`, `smb_signing_config`.
+ - Modul Terkenal: `network.port_scan`, `network.service_detect`, `network.dns_enum`, `lateral.smb_relay`, `lateral.ntlm_relay`.
+ - *Dampak*: Hasil reconnaissance port dan banner grab tidak memperbarui graph target host.
+
+4. **Sesi Lateral Movement & Eksekusi Remote (Severity: MEDIUM)**
+ - Capability: `command_output`, `lateral_session`, `powershell_session`, `owned_hosts`, `compromised_hosts`, `pivot_tunnel`, `socks5_proxy`.
+ - Modul Terkenal: `lateral.mssql`, `lateral.dcom`, `lateral.psexec`, `lateral.wmiexec`, `lateral.winrm`, `network.pivot`.
+ - *Dampak*: Status keberhasilan eksekusi remote dan tunnel aktif tidak dipetakan ke state campaign.
+
+---
+
+## 5. Rekomendasi Arsitektural untuk Sesi Perbaikan (Roadmap Fix)
+
+1. **Jadikan Normalizer "Key-Agnostic" (Multi-Key Fallback)**:
+ Ubah setiap method di `ArtifactNormalizer` untuk memeriksa list kunci alternatif:
+ - `_normalize_kerberos_hashes`: `raw.get("kerberos_hashes") or raw.get("hashes", [])`
+ - `_normalize_asrep_hashes`: `raw.get("asrep_hashes") or raw.get("hashes", [])`
+ - `_normalize_ntlm_hashes`: `raw.get("ntlm_hashes") or raw.get("sam_hashes") or raw.get("hashes", [])`
+ - `_normalize_users`: `raw.get("user_list") or raw.get("users", [])`
+ - `_normalize_computers`: `raw.get("computer_list") or raw.get("computers", [])`
+ - `_normalize_spns`: `raw.get("spn_list") or raw.get("spns", [])`
+ - `_normalize_host_vuln`: `raw.get("host") or raw.get("target") or raw.get("rhost", "")`
+
+2. **Tambahkan Handler Baru untuk Capability Kritis**:
+ - `cleartext_credentials` & `cracked_credentials` $\rightarrow$ `CredentialArtifact` + `ctx.record_credential()`
+ - `kerberos_ticket` $\rightarrow$ `CredentialArtifact(cred_type="kerberos_ticket")`
+ - `open_ports` & `service_map` $\rightarrow$ `ServiceArtifact` / Update `HostArtifact`
+
+3. **Buat Integration Test Wajib**:
+ Tambahkan test di `tests/unit/test_artifact_normalizer_pipeline.py` yang menguji pipeline eksekusi seluruh modul penghasil output terhadap `ArtifactNormalizer().normalize()`.
diff --git a/SUCCESS_MECHANISM_AUDIT.md b/SUCCESS_MECHANISM_AUDIT.md
new file mode 100644
index 0000000..cb96450
--- /dev/null
+++ b/SUCCESS_MECHANISM_AUDIT.md
@@ -0,0 +1,244 @@
+# SUCCESS_MECHANISM_AUDIT.md
+# ARES Success Mechanism & Finding Validation Audit
+**Auditor**: Principal Security Reviewer & QA Auditor
+**Date**: September 2026
+**Scope**: Module Base Classes (`BaseModule`, `BaseLateralModule`), Execution Engine (`ares/core/engine.py`), Validation Pipeline (`ares/core/validator.py`), Attack Chain & Strategy Planners (`ares/core/chain/chain.py`, `ares/strategy/engine.py`, `ares/modules/ai/autonomous_planner.py`).
+**Status**: DOKUMENTASI ARSITEKTUR & TEMUAN (Honesty Gates belum diimplementasikan di sesi ini sesuai batasan).
+
+---
+
+## EXECUTIVE SUMMARY
+
+Audit mendalam terhadap mekanisme publikasi finding dan evaluasi kesuksesan di ARES mengungkapkan adanya **kesenjangan validasi fundamental (Validation Blindspot)**:
+1. **Unvalidated Finding Pipeline**: `BaseModule.finding()` menerbitkan finding tanpa verifikasi bukti target.
+2. **Ignored Engine Validator**: `ares/core/engine.py` menjalankan `self.validator.validate()`, namun hasilnya **sama sekali diabaikan**—semua finding yang dikembalikan modul langsung dicap `validated = True` dan dimasukkan ke `confirmed`.
+3. **Default Success Paths**: Error/timeout jaringan pada modul seperti `lateral.rdp` ditangkap dan justru dijadikan indikator "sukses" (`result.success = True`), memicu finding `Severity.CRITICAL` tanpa otentikasi maupun eksekusi perintah nyata.
+4. **Autonomous Cascading Risk**: Finding palsu ini secara otomatis dibaca oleh `ChainAdvisor`, `AutonomousPlanner`, dan `StrategyEngine`, memicu chain serangan lanjutan yang agresif dan berisiko tinggi terhadap target, atau secara keliru mengklaim bahwa tujuan engagement (`goal_achieved`) telah tercapai.
+
+---
+
+## PERTANYAAN 1: Apa yang Harus Benar Agar Finding CRITICAL/HIGH Bisa Diterbitkan?
+
+### 1. Alur Eksekusi: Dari `run()` ke `self.finding()`
+- **Kontrak Base Class**: Di `ares/modules/base.py:431-447`, method `BaseModule.run()` didefinisikan dengan docstring eksplisit:
+ ```python
+ # ares/modules/base.py:435-437
+ Returns:
+ findings - list of Finding objects (unvalidated)
+ raw - raw output dict (evidence, debug info)
+ ```
+ Base class sendiri mengakui bahwa list `Finding` yang dikembalikan adalah **unvalidated**.
+- **Metode Pembuatan Finding**: Di `ares/modules/base.py:532-568`:
+ ```python
+ # ares/modules/base.py:545-558
+ f = Finding(
+ id = str(uuid.uuid4()),
+ title = title,
+ description = description,
+ severity = severity,
+ mitre_technique = mitre_technique,
+ mitre_tactic = mitre_tactic,
+ evidence = evidence or {},
+ remediation = remediation,
+ host = host,
+ confidence = confidence,
+ module_id = self.MODULE_ID,
+ )
+ self._findings.append(f)
+ return f
+ ```
+ Tidak ada gate atau filter apa pun di dalam `self.finding()`. Setiap modul bebas memanggil `self.finding(..., severity=Severity.CRITICAL, confidence=1.0)` kapan saja, bahkan tanpa bukti koneksi target sama sekali.
+
+### 2. Apakah Ada Gate Pembuktian di Level Engine?
+- **Pemeriksaan di `ares/core/engine.py:776-800`**:
+ ```python
+ # ares/core/engine.py:776-781
+ validation_results: list[ValidationResult] = []
+ if not skip_validation:
+ # Run all validations in parallel too
+ validation_results = list(
+ await asyncio.gather(*[self.validator.validate(f, raw) for f in findings])
+ )
+
+ # ares/core/engine.py:791-799
+ for f in findings:
+ if not f.false_positive:
+ f.validated = True
+ f.module_id = f.module_id or module_id
+ enrich_finding_with_cvss(f)
+ if trace_id:
+ f.trace_id = trace_id
+ confirmed.append(f)
+ ```
+- **Bukti Kritis**:
+ Meskipun `self.validator.validate(f, raw)` dipanggil di baris 780, **hasil `validation_results` tidak pernah diinspeksi untuk memfilter `findings`!**
+ Di baris 791, engine hanya mengecek `if not f.false_positive`. Karena atribut `false_positive` pada model `Finding` (`ares/core/campaign.py:50`) secara default bernilai `False`, seluruh finding tanpa kecuali langsung diubah statusnya menjadi `f.validated = True` dan dimasukkan ke dalam daftar `confirmed`.
+
+### 3. Kelemahan di Dalam `FindingValidator` Sendiri
+Bahkan jika hasil validator diperiksa, implementasi `FindingValidator` memiliki celah besar:
+- **Default Permissive**: Di `ares/core/validator.py:70-79`, jika sebuah modul tidak memiliki check terdaftar di registry, validator mengembalikan:
+ `passed=True`, `confidence=0.6`, `notes=["No validators registered - manual review recommended"]`.
+- **Tautological Check**: Di `ares/core/validator.py:264-276` (`_check_lateral_evidence`):
+ ```python
+ if target:
+ account_str = f" as '{user}'" if user else ""
+ return True, 1.0, f"Lateral movement to {target}{account_str} verified via {tech} ({privilege} privilege)"
+ return False, 0.0, "Missing target host evidence in lateral movement finding"
+ ```
+ Pemeriksaan lateral movement hanya mengecek apakah field `target` ada/tidak kosong! Jika ada, check mengembalikan `passed=True, score=1.0`.
+
+**Kesimpulan Pertanyaan 1**: Tidak ada gate pembuktian nyata. Finding severity CRITICAL/HIGH dapat diterbitkan hanya berdasarkan kemauan penulis modul tanpa verifikasi respon target nyata.
+
+---
+
+## PERTANYAAN 2: Apakah Ada Default Success Path Tanpa Bukti?
+
+Ya, ditemukan beberapa pola konkret di mana kegagalan jaringan atau ketiadaan bukti di-interpretasikan sebagai sukses:
+
+### 1. Exception di-Catch Lalu Fallback ke Success (Kasus `lateral.rdp`)
+- **Path & Baris**: `ares/modules/lateral/modules.py:1483-1492`
+ ```python
+ banner = sock.recv(64)
+ sock.close()
+ if banner and banner[0] == 0x03:
+ return True, "rdp_port_open", ""
+ return False, "", "Unexpected banner response"
+ except (OSError, socket.timeout):
+ sock.close()
+ # Port open but no valid response - still likely RDP
+ return True, "rdp_port_open", ""
+ ```
+- **Dampak**: Jika socket port 3389 terbuka tetapi terjadi timeout saat membaca banner, exception ditangkap dan fungsi mengembalikan `True` dengan privilege `"rdp_port_open"`.
+- Lalu di `ares/modules/lateral/modules.py:466-506`:
+ ```python
+ if result.success:
+ findings.append(self.finding(
+ title = f"Lateral Movement: {self.MODULE_NAME} → {target}",
+ description = f"Successfully moved laterally to {target} as {account_label} via {result.technique.value}. Privilege: {result.privilege or 'unknown'}.",
+ severity = Severity.CRITICAL,
+ confidence = 1.0,
+ ))
+ ```
+ Modul menerbitkan finding **Severity.CRITICAL** dengan klaim "Successfully moved laterally" dan confidence 1.0, padahal yang terjadi hanyalah port 3389 terbuka tanpa otentikasi maupun eksekusi perintah (MOD-021).
+
+### 2. Fallback Dummy / Phantom Active Saat Backend Absen
+- **Path & Baris**: `ares/modules/lateral/infrastructure.py:260-265, 368-370` (MOD-019)
+ Jika library backend `asyncssh` tidak terpasang, modul menandai tunnel status sebagai `ACTIVE` dan menerbitkan status sukses.
+- **Path & Baris**: `ares/modules/lateral/modules.py:1332-1356` (MOD-020)
+ `SSHPivot.establish_socks5` mengembalikan dictionary proxy aktif, padahal method `move()` di bawahnya tidak membuka port forwarding nyata.
+
+### 3. Local Calculation Tanpa I/O Diklaim Eksploitasi Nyata
+- **Path & Baris**: `ares/modules/ad/ghost_forge.py:165-201` (MOD-005)
+ Melakukan kalkulasi MD4/HMAC lokal, tidak pernah menyentuh socket/jaringan, namun menerbitkan finding CRITICAL "Golden/Silver Ticket Forged" dan menyuntikkan tiket fiktif ke Credential Vault.
+
+### 4. Permissive Module Status di Engine
+- **Path & Baris**: `ares/modules/base.py:341-347`
+ ```python
+ findings, raw = await self.run(**kwargs)
+ return ModuleResult(
+ status = "success" if findings or raw else "partial",
+ ...
+ )
+ ```
+ Bahkan jika modul hanya mengembalikan dictionary `raw` berisi string error, status yang diberikan adalah `"success"`. Dan di `ares/core/engine.py:66-74`, status `"partial"` maupun `"success"` keduanya tergolong `_SUCCESSFUL_MODULE_OUTCOMES`.
+
+---
+
+## PERTANYAAN 3: Apakah Finding CRITICAL Otomatis Memicu Chain Eksekusi?
+
+**YA, SEPENUHNYA TERBUKTI.**
+Finding severity CRITICAL dan data palsu yang dihasilkannya otomatis memicu chain eksekusi berikutnya pada minimal 3 komponen otonom ARES:
+
+### 1. ChainAdvisor Rule Engine (`ares/core/chain/chain.py`)
+Di `ares/core/chain/chain.py:306-375`, `ChainAdvisor.suggest()` menganalisis confirmed findings untuk menentukan modul berikutnya:
+```python
+# ares/core/chain/chain.py:343-345
+(
+ "critical" in severities,
+ Suggestion("ad.enum_acl", "Critical findings present - check for ACL abuse paths", 0.75, 2),
+),
+# ares/core/chain/chain.py:323-326
+(
+ any("writedacl" in t or "genericall" in t or "dcsync right" in t for t in titles),
+ Suggestion("ad.dcsync", "Dangerous ACL grants DCSync capability", 0.90, 2),
+),
+```
+Jika modul menghasilkan finding CRITICAL atau judul finding mengandung keyword tertentu (misal DCSync right), `ChainAdvisor` langsung menyarankan eksekusi modul berbahaya seperti `ad.dcsync` atau `ad.enum_acl`.
+
+### 2. Autonomous Planner Context Injection (`ares/modules/ai/autonomous_planner.py`)
+Di `ares/modules/ai/autonomous_planner.py:163-176`, seluruh finding diekstraksi ke prompt LLM:
+```python
+ctx["findings_summary"].append({"severity": str(sev), "title": title, "module": mid})
+```
+Dan di `_format_credentials_actionable` (baris 116-144), jika ada kredensial di vault (seperti kredensial fiktif yang disuntikkan MOD-005):
+- Tipe `NTLM` / `plaintext` $\rightarrow$ merekomendasikan `lateral.psexec`, `lateral.wmiexec`, `lateral.winrm`.
+- Privilege `domain_admin` $\rightarrow$ merekomendasikan `ad.dcsync`, `credential.golden_ticket`.
+
+### 3. StrategyEngine Autonomous Loop (`ares/strategy/engine.py`)
+Di `ares/strategy/engine.py:390-570`, engine menjalankan multi-round engagement secara otonom:
+1. `_run_ai_planner()` membuat `execution_plan` berdasarkan findings summary.
+2. `ConstitutionEnforcer` hanya memfilter modul yang dilarang eksplisit (`forbidden_modules`), bukan memeriksa keabsahan bukti.
+3. Modul-modul hasil rencana LLM dieksekusi secara otomatis via `self._run_single_module()`.
+4. **Pengecekan Sasaran (Goal Achieved)** di `ares/strategy/engine.py:894-913`:
+ ```python
+ def _check_goal_achieved(self, campaign: "Any", goal: str) -> bool:
+ findings = getattr(campaign, "findings", [])
+ goal_indicators = {
+ "domain_admin": ["DCSync", "Domain Admin", "krbtgt", "ntlm_hash"],
+ ...
+ }
+ ...
+ for finding in findings:
+ title = getattr(finding, "title", "")
+ if any(ind.lower() in title.lower() for ind in indicators):
+ return True
+ return False
+ ```
+ Jika finding CRITICAL palsu (seperti dari MOD-005 atau MOD-021) memuat judul "Domain Admin" atau "ntlm_hash", engine langsung menyimpulkan bahwa sasaran campaign **SELESAI (`goal_achieved = True`)**, memicu laporan kemenangan palsu dan menghentikan pengujian sesungguhnya.
+ Sebaliknya, jika campaign belum berhenti, false-positive CRITICAL lateral movement (MOD-021) akan menyebabkan AI Planner melancarkan modul post-exploitation bising (misal `lsass_dump`, DCSync) ke host yang sama sekali belum berhasil dikompromikan, memicu alarm EDR/SOC klien.
+
+---
+
+## DAFTAR HONESTY GATES YANG HILANG
+
+Berikut adalah 5 arsitektur gerbang integritas (Honesty Gates) yang wajib diimplementasikan di masa depan untuk mencegah keterulangan bug integritas bukti:
+
+### Gate 1: Network Evidence Verification Gate (Physical I/O Proof)
+- **Lokasi Rencana**: `ares/core/engine.py:780-805` & `ares/modules/base.py:532-568`
+- **Kondisi yang Dicek**: Setiap finding dengan `severity in (CRITICAL, HIGH)` harus memiliki bukti target fisik dalam `evidence` yang merefleksikan respon I/O jaringan nyata (misal: socket byte stream, NTLM challenge-response valid, Kerberos ticket session key dari KDC). Modul lokal tanpa I/O target dilarang keras menerbitkan finding di atas `Severity.LOW` / `Severity.INFO`.
+- **Jika Gagal**: Turunkan severity ke `Severity.INFO` / `LOCAL_CALCULATION`, set `finding.validated = False`, `finding.false_positive = True`, dan tolak registrasi ke `confirmed`.
+- **Modul yang Dicegah**: **MOD-005** (`ad.ghost_forge`), **MOD-019** (`network.pivot`).
+
+### Gate 2: Lateral Movement Proof-of-Execution Gate (Lateral Honesty Gate)
+- **Lokasi Rencana**: `ares/modules/lateral/modules.py:464-506` & `ares/core/validator.py:264-276`
+- **Kondisi yang Dicek**: Untuk menerbitkan finding "Lateral Movement", `result.success` tidak cukup hanya dari status port terbuka (`rdp_port_open`, `smb_open`). Wajib ada bukti **otentikasi berhasil** DAN **eksekusi perintah nyata** (misal: stdout dari `whoami`, token user privilege terverifikasi, atau session ID RDP interaktif yang terjalin).
+- **Jika Gagal**: Pisahkan hasil: laporkan sebagai `network.service_detect` / `open_port` (Severity.LOW / INFO), bukan `Lateral Movement` (Severity.CRITICAL). `result.success` untuk perpindahan lateral di-set `False`.
+- **Modul yang Dicegah**: **MOD-021** (`lateral.rdp`), **MOD-020** (`lateral.ssh_pivot`).
+
+### Gate 3: Engine Validator Enforcement Gate (Active Gatekeeper)
+- **Lokasi Rencana**: `ares/core/engine.py:776-805`
+- **Kondisi yang Dicek**: Hasil `validation_results` dari `self.validator.validate(f, raw)` wajib diperiksa secara fail-closed:
+ ```python
+ if not validation_res.should_report:
+ f.false_positive = True
+ f.validated = False
+ # Jangan masukkan ke confirmed
+ ```
+- **Jika Gagal**: Finding ditolak dari daftar `confirmed`, tidak disimpan ke database temuan aktif, dan tidak diumpankan ke planner.
+- **Modul yang Dicegah**: Mencegah seluruh modul dari lolosnya unverified finding ke laporan akhir dan pipeline chain.
+
+### Gate 4: Kerberos Ticket & S4U Identity Verification Gate
+- **Lokasi Rencana**: `ares/core/validator.py` & `ares/modules/lateral/ntlm_relay.py:944-965`
+- **Kondisi yang Dicek**: Modul yang mengklaim impersonasi identitas istimewa (misal Domain Admin via S4U2self/S4U2proxy) wajib memverifikasi bahwa tiket ccache yang diterima memiliki Client Principal sesuai target klaim (bukan sekadar tiket akun mesin lokal).
+- **Jika Gagal**: Batalkan klaim CRITICAL Administrator impersonation, ubah menjadi MEDIUM (Machine Account TGS captured).
+- **Modul yang Dicegah**: **MOD-013** (`lateral.ntlm_relay`).
+
+### Gate 5: Goal Indicator Cryptographic Verifier Gate
+- **Lokasi Rencana**: `ares/strategy/engine.py:894-913` (`_check_goal_achieved`)
+- **Kondisi yang Dicek**: `_check_goal_achieved()` tidak boleh hanya melakukan pencarian substring judul ("DCSync", "Domain Admin"). Goal "Domain Admin" hanya boleh dinyatakan tercapai jika ada kredensial domain admin terverifikasi di Vault DAN finding yang melandasinya telah lulus `Gate 1` dan `Gate 3`.
+- **Jika Gagal**: `goal_achieved = False`; autonomous engine tidak tertipu oleh false positive.
+- **Modul yang Dicegah**: Mencegah false termination dan false victory pada **MOD-005**, **MOD-013**, **MOD-021**.
+
+---
+
+*(Catatan: Sesuai batasan operasional, implementasi gerbang-gerbang di atas ditunda untuk review owner dan sesi refactoring arsitektur berikutnya).*
diff --git a/ares/normalize/artifacts.py b/ares/normalize/artifacts.py
index 97199c3..e1b1e5d 100644
--- a/ares/normalize/artifacts.py
+++ b/ares/normalize/artifacts.py
@@ -39,6 +39,10 @@ from dataclasses import dataclass, field
from enum import Enum
from typing import Any
+from ares.core.logger import get_logger
+
+logger = get_logger("ares.normalize.artifacts")
+
# ── Artifact type registry ─────────────────────────────────────────────────────
@@ -243,9 +247,11 @@ class CredentialArtifact(NormalizedArtifact):
self.source_host = self.host
elif self.source_host and not self.host:
self.host = self.source_host
+ if self.secret and not self.secret_hash:
+ self.secret_hash = hashlib.sha256(self.secret.encode("utf-8", errors="ignore")).hexdigest()
def _dedup_key(self) -> str:
- return f"{self.domain}\\{self.username}:{self.cred_type}:{self.secret_hash[:8]}"
+ return f"{self.domain}\\{self.username}:{self.cred_type}:{self.source_host}:{self.secret_hash[:8]}"
def _to_dict_fields(self) -> dict[str, Any]:
return {
@@ -446,8 +452,13 @@ class ArtifactNormalizer:
try:
added = self._normalize_capability(capability, raw, store)
count += added
- except Exception:
- pass # Never let normalization failures block the engine
+ except Exception as exc:
+ logger.warning(
+ "artifact_normalization_failed",
+ module_id=module_id,
+ capability=capability,
+ error=str(exc),
+ )
return count
def _normalize_capability(
@@ -455,15 +466,32 @@ class ArtifactNormalizer:
) -> int:
"""Route a capability to its specific normalizer."""
handlers: dict[str, Any] = {
- "user_list": self._normalize_users,
- "computer_list": self._normalize_computers,
- "kerberos_hashes": self._normalize_kerberos_hashes,
- "asrep_hashes": self._normalize_asrep_hashes,
- "ntlm_hashes": self._normalize_ntlm_hashes,
- "spn_list": self._normalize_spns,
- "acl_findings": self._normalize_permissions,
- "aws_findings": self._normalize_cloud,
- "privesc_vectors": self._normalize_host_vuln,
+ "user_list": self._normalize_users,
+ "users": self._normalize_users,
+ "computer_list": self._normalize_computers,
+ "computers": self._normalize_computers,
+ "kerberos_hashes": self._normalize_kerberos_hashes,
+ "asrep_hashes": self._normalize_asrep_hashes,
+ "ntlm_hashes": self._normalize_ntlm_hashes,
+ "hashes": self._normalize_ntlm_hashes,
+ "spn_list": self._normalize_spns,
+ "spns": self._normalize_spns,
+ "acl_findings": self._normalize_permissions,
+ "misconfigs": self._normalize_permissions,
+ "aws_findings": self._normalize_cloud,
+ "privesc_vectors": self._normalize_host_vuln,
+ # P0 Capabilities
+ "cleartext_credentials": self._normalize_cleartext_credentials,
+ "credentials": self._normalize_cleartext_credentials,
+ "browser_passwords": self._normalize_cleartext_credentials,
+ "cracked_credentials": self._normalize_cracked_credentials,
+ "laps_passwords": self._normalize_laps_passwords,
+ "kerberos_tickets": self._normalize_kerberos_tickets,
+ "kerberos_ticket": self._normalize_kerberos_tickets,
+ "tickets": self._normalize_kerberos_tickets,
+ "golden_ticket": self._normalize_kerberos_tickets,
+ "open_ports": self._normalize_open_ports,
+ "service_map": self._normalize_open_ports,
}
handler = handlers.get(capability)
if handler:
@@ -471,7 +499,7 @@ class ArtifactNormalizer:
return 0
def _normalize_users(self, raw: dict, store: ArtifactStore) -> int:
- users = raw.get("users", [])
+ users = raw.get("users") or raw.get("user_list", [])
count = 0
for u in users:
if not isinstance(u, dict):
@@ -480,102 +508,149 @@ class ArtifactNormalizer:
username = u.get("samAccountName", u.get("username", "")),
domain = u.get("domain", ""),
enabled = u.get("enabled", True),
- is_admin = u.get("isAdmin", False),
- no_preauth = u.get("noPreauth", False),
- spns = u.get("spns", []),
+ is_admin = bool(u.get("isAdmin", u.get("is_admin", u.get("adminCount", 0) > 0))),
+ no_preauth = bool(u.get("noPreauth", u.get("no_preauth", u.get("dont_req_preauth", False)))),
+ spns = u.get("spns", u.get("spn", [])),
)
store.add(artifact)
count += 1
return count
def _normalize_computers(self, raw: dict, store: ArtifactStore) -> int:
- computers = raw.get("computers", [])
+ computers = raw.get("computers") or raw.get("computer_list", [])
count = 0
for c in computers:
if not isinstance(c, dict):
continue
artifact = HostArtifact(
- ip_address = "",
- hostname = c.get("dns", c.get("name", "")),
+ ip_address = c.get("ip", c.get("ip_address", "")),
+ hostname = c.get("dns", c.get("dns_name", c.get("name", ""))),
is_dc = c.get("is_dc", False),
os = c.get("os", ""),
+ os_version = c.get("os_version", ""),
+ domain = c.get("domain", ""),
)
store.add(artifact)
count += 1
return count
def _normalize_kerberos_hashes(self, raw: dict, store: ArtifactStore) -> int:
- hashes = raw.get("hashes", [])
+ hashes = raw.get("hashes") or raw.get("kerberos_hashes", [])
count = 0
- accounts = raw.get("accounts", [])
for h in hashes:
- if not isinstance(h, str):
- continue
- # Parse username from hash: $krb5tgs$23$*username$DOMAIN$...
- parts = h.split("$")
- username = parts[3].split("@")[0] if len(parts) > 3 else ""
- domain = parts[4].split("@")[0] if len(parts) > 4 else ""
- artifact = HashArtifact(
- username = username,
- domain = domain,
- hash_value = h[:120],
- hash_type = "krb5tgs",
- hashcat_mode = 13100,
- )
- store.add(artifact)
- count += 1
+ if isinstance(h, str):
+ # Parse username from hash: $krb5tgs$23$*username$DOMAIN$...
+ parts = h.split("$")
+ raw_user = parts[3].split("@")[0] if len(parts) > 3 else ""
+ username = raw_user.lstrip("*")
+ domain = parts[4].split("@")[0] if len(parts) > 4 else ""
+ artifact = HashArtifact(
+ username = username,
+ domain = domain,
+ hash_value = h[:120],
+ hash_type = "krb5tgs",
+ hashcat_mode = 13100,
+ )
+ store.add(artifact)
+ count += 1
+ elif isinstance(h, dict):
+ artifact = HashArtifact(
+ username = h.get("username", ""),
+ domain = h.get("domain", ""),
+ hash_value = (h.get("hash") or h.get("hash_value") or "")[:120],
+ hash_type = "krb5tgs",
+ hashcat_mode = 13100,
+ )
+ store.add(artifact)
+ count += 1
return count
def _normalize_asrep_hashes(self, raw: dict, store: ArtifactStore) -> int:
- hashes = raw.get("hashes", [])
+ hashes = raw.get("hashes") or raw.get("asrep_hashes", [])
count = 0
for h in hashes:
- if not isinstance(h, str):
- continue
- parts = h.split("$")
- username = parts[3].split("@")[0] if len(parts) > 3 else ""
- domain = parts[4].split("@")[0] if len(parts) > 4 else ""
- artifact = HashArtifact(
- username = username,
- domain = domain,
- hash_value = h[:120],
- hash_type = "krb5asrep",
- hashcat_mode = 18200,
- )
- store.add(artifact)
- count += 1
+ if isinstance(h, str):
+ parts = h.split("$")
+ user_part = parts[3] if len(parts) > 3 else ""
+ if "@" in user_part:
+ username, domain = user_part.split("@", 1)
+ else:
+ username = user_part
+ domain = parts[4].split("@")[0] if len(parts) > 4 else ""
+ artifact = HashArtifact(
+ username = username,
+ domain = domain,
+ hash_value = h[:120],
+ hash_type = "krb5asrep",
+ hashcat_mode = 18200,
+ )
+ store.add(artifact)
+ count += 1
+ elif isinstance(h, dict):
+ artifact = HashArtifact(
+ username = h.get("username", ""),
+ domain = h.get("domain", ""),
+ hash_value = (h.get("hash") or h.get("hash_value") or "")[:120],
+ hash_type = "krb5asrep",
+ hashcat_mode = 18200,
+ )
+ store.add(artifact)
+ count += 1
return count
def _normalize_ntlm_hashes(self, raw: dict, store: ArtifactStore) -> int:
- hashes = raw.get("hashes", [])
+ hashes = (
+ raw.get("hashes")
+ or raw.get("ntlm_hashes")
+ or raw.get("sam_hashes")
+ or []
+ )
count = 0
for h in hashes:
- if not isinstance(h, dict):
- continue
- artifact = HashArtifact(
- username = h.get("username", ""),
- domain = "",
- hash_value = h.get("nt_hash", ""),
- hash_type = "ntlm",
- hashcat_mode = 1000,
- )
- store.add(artifact)
- count += 1
+ if isinstance(h, dict):
+ artifact = HashArtifact(
+ username = h.get("username", ""),
+ domain = h.get("domain", ""),
+ hash_value = h.get("nt_hash") or h.get("hash") or h.get("ntlm", ""),
+ hash_type = "ntlm",
+ hashcat_mode = 1000,
+ )
+ store.add(artifact)
+ count += 1
+ elif isinstance(h, str):
+ parts = h.split(":")
+ username = parts[0] if parts else ""
+ nt_hash = ""
+ if len(parts) >= 4:
+ nt_hash = parts[3]
+ elif len(parts) == 2:
+ nt_hash = parts[1]
+ else:
+ nt_hash = h
+ artifact = HashArtifact(
+ username = username,
+ domain = "",
+ hash_value = nt_hash,
+ hash_type = "ntlm",
+ hashcat_mode = 1000,
+ )
+ store.add(artifact)
+ count += 1
return count
def _normalize_spns(self, raw: dict, store: ArtifactStore) -> int:
- spns = raw.get("spns", [])
+ spns = raw.get("spns") or raw.get("spn_list", [])
count = 0
for s in spns:
if not isinstance(s, dict):
continue
+ spn_entries = s.get("spns", s.get("spn", []))
artifact = UserArtifact(
- username = s.get("name", ""),
- domain = "",
+ username = s.get("samAccountName", s.get("name", s.get("username", ""))),
+ domain = s.get("domain", ""),
enabled = s.get("enabled", True),
- is_admin = s.get("is_admin", False),
- spns = s.get("spns", []),
- is_kerberoastable = bool(s.get("spns")),
+ is_admin = s.get("is_admin", s.get("isAdmin", False)),
+ spns = spn_entries,
)
store.add(artifact)
count += 1
@@ -610,10 +685,31 @@ class ArtifactNormalizer:
)
store.add(artifact)
count += 1
+ for path in raw.get("privesc_paths", []):
+ if isinstance(path, dict):
+ res_id = (
+ path.get("role_arn")
+ or path.get("policy_name")
+ or path.get("action")
+ or path.get("technique")
+ or "iam_privesc_path"
+ )
+ elif isinstance(path, str):
+ res_id = path
+ else:
+ continue
+ artifact = CloudResourceArtifact(
+ resource_id = str(res_id),
+ resource_type = "iam_role",
+ region = region,
+ is_public = False,
+ )
+ store.add(artifact)
+ count += 1
return count
def _normalize_host_vuln(self, raw: dict, store: ArtifactStore) -> int:
- host = raw.get("host", "")
+ host = raw.get("target") or raw.get("host", "")
if host and host != "localhost":
artifact = HostArtifact(
ip_address = host,
@@ -622,3 +718,149 @@ class ArtifactNormalizer:
store.add(artifact)
return 1
return 0
+
+ # ── P0 Handlers ────────────────────────────────────────────────────────────
+
+ def _normalize_cleartext_credentials(self, raw: dict, store: ArtifactStore) -> int:
+ creds = raw.get("cleartext_credentials") or raw.get("credentials", [])
+ count = 0
+ for c in creds:
+ if not isinstance(c, dict):
+ continue
+ artifact = CredentialArtifact(
+ username = c.get("username", ""),
+ domain = c.get("domain", ""),
+ cred_type = "cleartext",
+ secret = c.get("password") or c.get("secret") or c.get("plaintext", ""),
+ source_host = c.get("host") or c.get("target") or c.get("computer", ""),
+ privilege = c.get("privilege", "unknown"),
+ )
+ store.add(artifact)
+ count += 1
+ return count
+
+ def _normalize_cracked_credentials(self, raw: dict, store: ArtifactStore) -> int:
+ cracked = raw.get("cracked_credentials") or raw.get("cracked_users", [])
+ count = 0
+ for c in cracked:
+ if not isinstance(c, dict):
+ continue
+ secret = c.get("plaintext") or c.get("password") or c.get("secret", "")
+ artifact = CredentialArtifact(
+ username = c.get("username", ""),
+ domain = c.get("domain", ""),
+ cred_type = "cleartext" if secret else (c.get("hash_type") or "ntlm"),
+ secret = secret,
+ cracked = True,
+ source_host = c.get("host") or c.get("target", ""),
+ privilege = c.get("privilege", "unknown"),
+ )
+ store.add(artifact)
+ count += 1
+ return count
+
+ def _normalize_laps_passwords(self, raw: dict, store: ArtifactStore) -> int:
+ laps = raw.get("laps_passwords") or raw.get("entries", [])
+ count = 0
+ for e in laps:
+ if not isinstance(e, dict):
+ continue
+ comp = e.get("computer") or e.get("computer_name") or e.get("host", "")
+ pwd = e.get("password", "")
+ artifact = CredentialArtifact(
+ username = e.get("username", "Administrator"),
+ domain = e.get("domain", ""),
+ cred_type = "laps",
+ secret = pwd,
+ source_host = comp,
+ privilege = "local_admin",
+ )
+ store.add(artifact)
+ count += 1
+ return count
+
+ def _normalize_kerberos_tickets(self, raw: dict, store: ArtifactStore) -> int:
+ tickets = (
+ raw.get("kerberos_tickets")
+ or raw.get("kerberos_ticket")
+ or raw.get("tickets")
+ or []
+ )
+ if isinstance(tickets, (str, dict)):
+ tickets = [tickets]
+ count = 0
+ for t in tickets:
+ if isinstance(t, str):
+ artifact = CredentialArtifact(
+ username = "unknown",
+ domain = "",
+ cred_type = "kerberos_ticket",
+ secret = t,
+ )
+ store.add(artifact)
+ count += 1
+ elif isinstance(t, dict):
+ client = (
+ t.get("client")
+ or t.get("username")
+ or t.get("default_principal")
+ or t.get("principal", "")
+ )
+ realm = t.get("realm") or t.get("domain", "")
+ if "@" in client and not realm:
+ parts = client.split("@", 1)
+ client = parts[0]
+ realm = parts[1]
+ ticket_path = (
+ t.get("file_path")
+ or t.get("ticket_path")
+ or t.get("ccache_path")
+ or t.get("path", "")
+ )
+ artifact = CredentialArtifact(
+ username = client,
+ domain = realm,
+ cred_type = "kerberos_ticket",
+ secret = ticket_path,
+ source_host = t.get("target") or t.get("host", ""),
+ )
+ store.add(artifact)
+ count += 1
+ return count
+
+ def _normalize_open_ports(self, raw: dict, store: ArtifactStore) -> int:
+ ports_raw = raw.get("open_ports") or raw.get("port_results") or []
+ ports: list[int] = []
+ if isinstance(ports_raw, list):
+ for p in ports_raw:
+ if isinstance(p, int):
+ ports.append(p)
+ elif isinstance(p, dict) and "port" in p and isinstance(p["port"], int):
+ ports.append(p["port"])
+ elif str(p).isdigit():
+ ports.append(int(p))
+ if "service_map" in raw and isinstance(raw["service_map"], dict):
+ for k in raw["service_map"]:
+ if str(k).isdigit():
+ ports.append(int(k))
+ ports = sorted(list(set(ports)))
+ if not ports:
+ return 0
+ target = raw.get("target") or raw.get("host", "")
+ # Look for existing HostArtifact with same target/ip
+ existing_host = None
+ if target:
+ for h in store.hosts():
+ if h.ip_address == target or h.hostname == target:
+ existing_host = h
+ break
+ if existing_host:
+ existing_host.open_ports = sorted(list(set(existing_host.open_ports + ports)))
+ return 1
+ artifact = HostArtifact(
+ ip_address = target,
+ hostname = target,
+ open_ports = ports,
+ )
+ store.add(artifact)
+ return 1
diff --git a/tests/unit/test_artifact_normalizer_pipeline.py b/tests/unit/test_artifact_normalizer_pipeline.py
new file mode 100644
index 0000000..e9fa86b
--- /dev/null
+++ b/tests/unit/test_artifact_normalizer_pipeline.py
@@ -0,0 +1,339 @@
+"""
+tests/unit/test_artifact_normalizer_pipeline.py
+End-to-End integration test suite for ArtifactNormalizer pipeline and dual-read fallback.
+
+Validates that real-world raw outputs from offensive modules (using legacy key conventions)
+successfully populate ArtifactStore without 70% data evaporation.
+"""
+from __future__ import annotations
+
+import pytest
+
+from ares.normalize.artifacts import (
+ ArtifactNormalizer,
+ ArtifactStore,
+ ArtifactType,
+ CredentialArtifact,
+ HashArtifact,
+ HostArtifact,
+ UserArtifact,
+)
+
+
+def store() -> ArtifactStore:
+ return ArtifactStore()
+
+
+def normalizer() -> ArtifactNormalizer:
+ return ArtifactNormalizer()
+
+
+# ── Batch of Mismatched Handlers (Langkah 2A) ─────────────────────────────────
+
+def test_kerberoast_pipeline_recovers_hashes(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """ad.kerberoast writes raw['kerberos_hashes']; normalizer must ingest via fallback."""
+ raw = {
+ "kerberos_hashes": ["$krb5tgs$23$*svc_sql$CORP.LOCAL$MSSQLSvc/db01*1234567890abcdef..."],
+ "accounts": ["svc_sql"],
+ }
+ added = normalizer.normalize("ad.kerberoast", ["kerberos_hashes"], raw, store)
+ assert added == 1
+ hashes = store.hashes()
+ assert len(hashes) == 1
+ assert hashes[0].username == "svc_sql"
+ assert hashes[0].hash_type == "krb5tgs"
+ assert hashes[0].hashcat_mode == 13100
+
+
+def test_asreproast_pipeline_recovers_hashes(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """ad.asreproast writes raw['asrep_hashes']; normalizer must ingest via fallback."""
+ raw = {
+ "asrep_hashes": ["[email protected]:1234567890abcdef..."],
+ "vulnerable_accounts": ["target_user"],
+ }
+ added = normalizer.normalize("ad.asreproast", ["asrep_hashes"], raw, store)
+ assert added == 1
+ hashes = store.hashes()
+ assert len(hashes) == 1
+ assert hashes[0].username == "target_user"
+ assert hashes[0].hash_type == "krb5asrep"
+ assert hashes[0].hashcat_mode == 18200
+
+
+def test_dcsync_pipeline_recovers_ntlm_hashes(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """ad.dcsync writes raw['ntlm_hashes']; normalizer must ingest via fallback."""
+ raw = {
+ "ntlm_hashes": [
+ {"username": "Administrator", "nt_hash": "31d6cfe0d16ae931b73c59d7e0c089c0"}
+ ],
+ "target": "Administrator",
+ }
+ added = normalizer.normalize("ad.dcsync", ["ntlm_hashes"], raw, store)
+ assert added == 1
+ hashes = store.hashes()
+ assert len(hashes) == 1
+ assert hashes[0].username == "Administrator"
+ assert hashes[0].hash_type == "ntlm"
+ assert hashes[0].hash_value == "31d6cfe0d16ae931b73c59d7e0c089c0"
+
+
+def test_enum_users_pipeline_recovers_users(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """ad.enum_users writes raw['user_list']; normalizer must ingest via fallback."""
+ raw = {
+ "user_list": [
+ {
+ "samAccountName": "jdoe",
+ "domain": "CORP",
+ "enabled": True,
+ "adminCount": 1,
+ }
+ ]
+ }
+ added = normalizer.normalize("ad.enum_users", ["user_list"], raw, store)
+ assert added == 1
+ users = store.users()
+ assert len(users) == 1
+ assert users[0].username == "jdoe"
+ assert users[0].domain == "CORP"
+ assert users[0].is_admin is True
+
+
+def test_enum_computers_pipeline_recovers_computers(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """ad.enum_computers writes raw['computer_list']; normalizer must ingest via fallback."""
+ raw = {
+ "computer_list": [
+ {
+ "name": "DC01",
+ "dns_name": "dc01.corp.local",
+ "os": "Windows Server 2022",
+ "is_dc": True,
+ "ip": "10.0.0.1",
+ }
+ ]
+ }
+ added = normalizer.normalize("ad.enum_computers", ["computer_list"], raw, store)
+ assert added == 1
+ hosts = store.hosts()
+ assert len(hosts) == 1
+ assert hosts[0].hostname in ("dc01.corp.local", "DC01")
+ assert hosts[0].is_dc is True
+
+
+def test_enum_spn_pipeline_recovers_spns(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """ad.enum_spn writes raw['spn_list']; normalizer must ingest via fallback."""
+ raw = {
+ "spn_list": [
+ {
+ "samAccountName": "svc_web",
+ "spns": ["HTTP/web.corp.local"],
+ "enabled": True,
+ "is_admin": False,
+ }
+ ]
+ }
+ added = normalizer.normalize("ad.enum_spn", ["spn_list"], raw, store)
+ assert added == 1
+ users = store.users()
+ assert len(users) == 1
+ assert users[0].username == "svc_web"
+ assert users[0].is_kerberoastable is True
+
+
+def test_host_vuln_pipeline_recovers_target(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """linux.privesc writes raw['target']; normalizer must ingest via fallback."""
+ raw = {
+ "target": "10.0.0.55",
+ "privesc_vectors": ["cve-2021-4034"],
+ }
+ added = normalizer.normalize("linux.privesc", ["privesc_vectors"], raw, store)
+ assert added == 1
+ hosts = store.hosts()
+ assert len(hosts) == 1
+ assert hosts[0].ip_address == "10.0.0.55"
+
+
+def test_aws_privesc_pipeline_recovers_paths(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """cloud.aws_privesc writes raw['privesc_paths']; normalizer must ingest without dropping."""
+ raw = {
+ "privesc_paths": [
+ {"role_arn": "arn:aws:iam::123456789012:role/AdminEscalationRole", "action": "iam:PassRole"}
+ ],
+ "region": "us-east-1",
+ }
+ added = normalizer.normalize("cloud.aws_privesc", ["aws_findings"], raw, store)
+ assert added == 1
+ cloud_res = store.get(ArtifactType.CLOUD_RESOURCE)
+ assert len(cloud_res) == 1
+ assert "AdminEscalationRole" in cloud_res[0].resource_id
+
+
+# ── Batch of P0 Handlers (Langkah 2B) ─────────────────────────────────────────
+
+def test_cleartext_credentials_pipeline(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """ad.sccm and windows.dpapi write cleartext_credentials; normalizer must create CredentialArtifact."""
+ raw = {
+ "cleartext_credentials": [
+ {
+ "username": "sccm_naa",
+ "password": "PasswordNAA123!",
+ "domain": "CORP",
+ "host": "10.0.0.10",
+ }
+ ]
+ }
+ added = normalizer.normalize("ad.sccm", ["cleartext_credentials"], raw, store)
+ assert added == 1
+ creds = store.credentials()
+ assert len(creds) == 1
+ assert creds[0].username == "sccm_naa"
+ assert creds[0].domain == "CORP"
+ assert creds[0].cred_type == "cleartext"
+ assert creds[0].secret == "PasswordNAA123!"
+ assert creds[0].source_host == "10.0.0.10"
+
+
+def test_cracked_credentials_pipeline(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """credential.crack writes cracked_credentials; normalizer must create CredentialArtifact(cracked=True)."""
+ raw = {
+ "cracked_credentials": [
+ {
+ "username": "svc_sql",
+ "domain": "CORP",
+ "plaintext": "Winter2024!",
+ "hash_type": "krb5tgs",
+ }
+ ]
+ }
+ added = normalizer.normalize("credential.crack", ["cracked_credentials"], raw, store)
+ assert added == 1
+ creds = store.credentials()
+ assert len(creds) == 1
+ assert creds[0].username == "svc_sql"
+ assert creds[0].domain == "CORP"
+ assert creds[0].cracked is True
+ assert creds[0].secret == "Winter2024!"
+
+
+def test_laps_passwords_pipeline(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """ad.laps_enum writes laps_passwords; normalizer must create CredentialArtifact(cred_type='laps')."""
+ raw = {
+ "laps_passwords": [
+ {
+ "computer": "WORKSTATION-01",
+ "password": "LapsSecretAdminPwd!",
+ }
+ ]
+ }
+ added = normalizer.normalize("ad.laps_enum", ["laps_passwords"], raw, store)
+ assert added == 1
+ creds = store.credentials()
+ assert len(creds) == 1
+ assert creds[0].username == "Administrator"
+ assert creds[0].cred_type == "laps"
+ assert creds[0].secret == "LapsSecretAdminPwd!"
+ assert creds[0].source_host == "WORKSTATION-01"
+ assert creds[0].privilege == "local_admin"
+
+
+def test_kerberos_tickets_pipeline(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """lateral.ntlm_relay / linux.ccache_hunt write tickets; normalizer must create CredentialArtifact."""
+ raw = {
+ "kerberos_tickets": [
+ {
+ "client": "[email protected]",
+ "ticket_path": "/tmp/admin_ticket.ccache",
+ "target": "10.0.0.2",
+ }
+ ]
+ }
+ added = normalizer.normalize("linux.ccache_hunt", ["kerberos_tickets"], raw, store)
+ assert added == 1
+ creds = store.credentials()
+ assert len(creds) == 1
+ assert creds[0].username == "Administrator"
+ assert creds[0].domain == "CORP.LOCAL"
+ assert creds[0].cred_type == "kerberos_ticket"
+ assert creds[0].secret == "/tmp/admin_ticket.ccache"
+
+
+def test_open_ports_pipeline(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """network.port_scan writes open_ports; normalizer must populate HostArtifact.open_ports."""
+ raw = {
+ "target": "10.0.0.25",
+ "open_ports": [22, 80, 445],
+ "service_map": {22: "ssh", 80: "http", 445: "smb"},
+ }
+ added = normalizer.normalize("network.port_scan", ["open_ports"], raw, store)
+ assert added >= 1
+ hosts = store.hosts()
+ assert len(hosts) == 1
+ assert hosts[0].ip_address == "10.0.0.25"
+ assert hosts[0].open_ports == [22, 80, 445]
+
+
+def test_open_ports_updates_existing_host(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """Scanning existing host updates open_ports without creating duplicate host."""
+ store.add(HostArtifact(ip="10.0.0.30", hostname="10.0.0.30", open_ports=[80]))
+ raw = {
+ "target": "10.0.0.30",
+ "open_ports": [443, 8080],
+ }
+ added = normalizer.normalize("network.port_scan", ["open_ports"], raw, store)
+ assert added == 1
+ hosts = store.hosts()
+ assert len(hosts) == 1
+ assert hosts[0].open_ports == [80, 443, 8080]
+
+
+# ── Canonical Standard Key Tests (Dual-Read Verification) ─────────────────────
+
+def test_canonical_standard_keys_accepted(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """Verify that newly standardized keys ('hashes', 'users', 'computers', 'spns', 'target') work seamlessly."""
+ raw_users = {"users": [{"username": "alice", "domain": "CORP"}]}
+ assert normalizer.normalize("ad.enum_users", ["user_list"], raw_users, store) == 1
+
+ raw_comp = {"computers": [{"name": "FS01", "os": "Linux", "ip": "10.0.0.2"}]}
+ assert normalizer.normalize("ad.enum_computers", ["computer_list"], raw_comp, store) == 1
+
+ raw_hashes = {"hashes": [{"username": "bob", "nt_hash": "aabbccddeeff00112233445566778899"}]}
+ assert normalizer.normalize("windows.lsass_dump", ["ntlm_hashes"], raw_hashes, store) == 1
+
+ raw_spns = {"spns": [{"name": "svc_crm", "spns": ["HTTP/crm.corp.local"]}]}
+ assert normalizer.normalize("ad.enum_spn", ["spn_list"], raw_spns, store) == 1
+
+ assert len(store.users()) == 2
+ assert len(store.hosts()) == 1
+ assert len(store.hashes()) == 1
+
+
+def test_kerberos_ticket_string_path(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """Verify that a single string path for kerberos_ticket is normalized into CredentialArtifact."""
+ raw = {"kerberos_ticket": "/tmp/golden.ccache"}
+ added = normalizer.normalize("credential.golden_ticket", ["kerberos_ticket"], raw, store)
+ assert added == 1
+ creds = store.credentials()
+ assert len(creds) == 1
+ assert creds[0].secret == "/tmp/golden.ccache"
+ assert creds[0].cred_type == "kerberos_ticket"
+
+
+def test_acl_findings_permissions_pipeline(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """ad.enum_acl writes misconfigs; normalizer must ingest into PermissionArtifact."""
+ raw = {
+ "misconfigs": [
+ {
+ "trustee_sid": "S-1-5-21-12345-500",
+ "target": "CN=Domain Admins,CN=Users,DC=corp,DC=local",
+ "right": "GenericAll",
+ }
+ ]
+ }
+ added = normalizer.normalize("ad.enum_acl", ["acl_findings"], raw, store)
+ assert added == 1
+ perms = store.permissions()
+ assert len(perms) == 1
+ assert perms[0].is_dangerous is True
+ assert perms[0].right == "GenericAll"
+