Skip to content

armgit / ARES

Public Imported from GitHub
0

ARES - authorized red-team engagement automation with dashboard, campaign scope, module orchestration, OPSEC controls, encrypted vault, and reporting.

fix(data): Fase 1 - complete normalizer pipeline handlers Grup A (MASTER_FIX_PLAN.md)

Mafifrizi9f8b111c84a1
diff --git a/NORMALIZER_CONTRACT_AUDIT.md b/NORMALIZER_CONTRACT_AUDIT.md
index 41c321d..397a119 100644
--- a/NORMALIZER_CONTRACT_AUDIT.md
+++ b/NORMALIZER_CONTRACT_AUDIT.md
@@ -28,17 +28,17 @@ normalized = ArtifactNormalizer().normalize(
| Kategori Status Kontrak | Jumlah Pasangan Modul-Capability | Persentase | Status Data |
|---|:---:|:---:|---|
-| ✅ **MATCH (Selaras Penuh)** | 6 pasangan modul | 30.0% | Data mengalir sempurna ke `ArtifactStore` |
-| 🔧 **FIXED (Dual-Read Fallback Diterapkan)** | 14 pasangan modul | 70.0% | **Data dipulihkan via Dual-Read Fallback** |
+| ✅ **MATCH (Selaras Penuh)** | 6 pasangan modul | 15.0% | Data mengalir sempurna ke `ArtifactStore` |
+| 🔧 **FIXED (Dual-Read Fallback & Handler Lengkap)** | 34 pasangan modul | 85.0% | **Data dipulihkan via Dual-Read Fallback & Handler Lengkap Grup A** |
| ⚠️ **ORPHANED HANDLER** | 0 handler | 0.0% | Semua handler memiliki minimal 1 modul deklarator |
-| 🆕 **HANDLER BARU DITAMBAHKAN** | 15 capability | - | `lsa_secrets`, `cached_credentials`, `valid_credentials`, `cleartext_credentials`, `cracked_credentials`, `laps_passwords`, `kerberos_tickets`, `open_ports`, `converted_ticket`, `samba_secrets`, `keytab_keys`, `iam_privesc`, `dns_records`, `service_versions`, `web_fingerprint` |
-| 🚫 **UNHANDLED OUTPUTS** | **71 capability** | - | Output modul diabaikan sepenuhnya oleh normalizer (berkurang dari 78 pasca 7 Group A handlers) |
+| 🆕 **HANDLER SELESAI (FASE 1)** | 35+ capability | - | Seluruh capability Grup A diimplementasikan dan diuji via `test_artifact_normalizer_pipeline.py` |
+| 🚫 **UNHANDLED OUTPUTS (Grup A)** | **0 capability** | - | Seluruh capability target Grup A telah memiliki handler aktif |
---
## 2. Tabel Lengkap Pemetaan Kontrak Normalizer
-Tabel di bawah ini memetakan seluruh 9 capability yang memiliki handler di `ares/normalize/artifacts.py:457-468` terhadap modul-modul yang mendeklarasikannya di `OUTPUTS`:
+Tabel di bawah ini memetakan seluruh 9 capability awal plus puluhan handler baru di `ares/normalize/artifacts.py` terhadap modul-modul yang mendeklarasikannya di `OUTPUTS`:
| Capability | Modul Penghasil | Key yang DIBACA Normalizer | Key yang DITULIS Modul (`raw`) | Status | Dampak Sistemik & Downstream Failure |
|---|---|---|---|:---:|---|
@@ -52,7 +52,7 @@ Tabel di bawah ini memetakan seluruh 9 capability yang memiliki handler di `ares
| `spn_list` | `ad.enum_spn` | `spns` OR `spn_list` (Dual-Read) | `spn_list` | ✅ **FIXED** (Dual-Read Fallback) | SPN data mengalir ke `UserArtifact` via fallback. |
| `acl_findings` | `ad.enum_acl` | `misconfigs` (list of dict) | `misconfigs`, `acl_findings` | ✅ **MATCH** | Data mengalir benar ke `PermissionArtifact`. Modul menuliskan kedua key. |
| `aws_findings` | `cloud.aws` | `region`, `s3` (`public_buckets`) | `region`, `s3`, `aws_findings` | ✅ **MATCH** | S3 bucket publik berhasil dikonversi menjadi `CloudResourceArtifact`. |
-| `aws_findings` | `cloud.aws_privesc` | `region`, `s3` (`public_buckets`) | `privesc_paths`, `aws_findings` | ❌ **MISMATCH** | Modul privilege escalation AWS tidak menghasilkan `s3`, sehingga normalizer mengembalikan 0 artifact. Jalur eskalasi IAM hilang dari store. |
+| `aws_findings` | `cloud.aws_privesc` | `region`, `s3` (`public_buckets`) | `privesc_paths`, `aws_findings` | ✅ **FIXED** (Fase 1) | Handler `_normalize_aws` dan `_normalize_iam_privesc` memulihkan temuan bucket dan jalur IAM ke `CloudResourceArtifact`. |
| `privesc_vectors` | `linux.ld_preload` | `target` OR `host` (Dual-Read) | `host`, `privesc_vectors` | ✅ **MATCH** | Menghasilkan `HostArtifact` dengan IP/hostname target. |
| `privesc_vectors` | `linux.nfs_escape` | `target` OR `host` (Dual-Read) | `host`, `privesc_vectors` | ✅ **MATCH** | Menghasilkan `HostArtifact` dengan IP/hostname target. |
| `privesc_vectors` | `linux.service_hijack` | `target` OR `host` (Dual-Read) | `host`, `privesc_vectors` | ✅ **MATCH** | Menghasilkan `HostArtifact` dengan IP/hostname target. |
@@ -65,40 +65,40 @@ Tabel di bawah ini memetakan seluruh 9 capability yang memiliki handler di `ares
| **`lsa_secrets`** | **`windows.lsa_secrets`** | **`lsa_secrets`** | **`lsa_secrets`** | ✅ **FIXED** (Handler Baru, MOD-028) | Handler `_normalize_lsa_secrets` menghasilkan `CredentialArtifact(cred_type="lsa_secret")`. Data loss 100% teratasi. |
| **`cached_credentials`** | **`windows.lsa_secrets`** | **`cached_credentials`** | **`cached_credentials`** | ✅ **FIXED** (Handler Baru, MOD-028) | Handler `_normalize_cached_domain_credentials` menghasilkan `CredentialArtifact(cred_type="cached_domain")`. |
| **`valid_credentials`** | **`credential.pass_spray`, `ssh_spray`, `pass_the_hash`, `reuse`** | **`valid_credentials`** | **`valid_credentials`** | ✅ **FIXED** (Handler Baru, MOD-033) | Handler `_normalize_valid_credentials` menghasilkan `CredentialArtifact`. Tipe data distandardisasi ke `list[dict]`. |
-| `converted_ticket` | `credential.ticket_converter` | — | `converted_ticket_b64` | ❌ **MISMATCH** (MOD-036) | Output key mismatch (`converted_ticket` vs `converted_ticket_b64`). Tidak ada handler. Tiket konversi hilang dari pipeline. |
-| `container_escape_vectors` | `linux.container` | — | `container_escape_vectors` | ❌ **MISMATCH** (MOD-040) | Tidak ada handler normalizer (`no handler`). Data vektor container escape hilang dari `ArtifactStore`. |
-| `k8s_rbac_findings` | `linux.container` | — | `k8s_rbac_findings` | ❌ **MISMATCH** (MOD-040) | Tidak ada handler normalizer (`no handler`). Data temuan K8s RBAC hilang dari `ArtifactStore`. |
-| `machine_account_hash` | `linux.samba_secrets` | — | `machine_account_hash` | ❌ **MISMATCH** (MOD-041) | Tidak ada handler normalizer (`no handler`). Machine account NTLM hash tidak terserap ke `ArtifactStore`. |
-| `samba_secrets` | `linux.samba_secrets` | — | `samba_secrets` | ❌ **MISMATCH** (MOD-041) | Tidak ada handler normalizer (`no handler`). Samba secrets tidak terserap ke `ArtifactStore`. |
-| `machine_credentials` | `linux.keytab_abuse` | — | `entries` | ❌ **MISMATCH** (MOD-044) | Output key mismatch (`machine_credentials` vs `entries`). Tidak ada handler normalizer. Data kunci mesin hilang dari `ArtifactStore`. |
-| `kerberos_keys` | `linux.keytab_abuse` | — | `silver_tickets` | ❌ **MISMATCH** (MOD-044) | Output key mismatch (`kerberos_keys` vs `silver_tickets`). Tidak ada handler normalizer. Data kunci Kerberos hilang dari `ArtifactStore`. |
-| `cached_hashes` | `linux.sssd_harvest` | — | `cached_hashes`, `hashes` | ❌ **MISMATCH** (MOD-045) | Tidak ada handler normalizer khusus `cached_hashes` (`no handler`). Telah didual-write ke `hashes` dan menunggu handler normalizer serentak. |
-| `domain_users` | `linux.sssd_harvest` | — | `domain_users`, `users` | ❌ **MISMATCH** (MOD-045) | Tidak ada handler normalizer khusus `domain_users` (`no handler`). Telah didual-write ke `users` dan menunggu handler normalizer serentak. |
-| `federation_trusts` | `cloud.identity_federation_abuse` | — | `federation_trusts` | ❌ **MISMATCH** (MOD-051) | Tidak ada handler normalizer (`no handler`). Data trust federasi hilang dari `ArtifactStore`. |
-| `golden_saml_paths` | `cloud.identity_federation_abuse` | — | `golden_saml_paths` | ❌ **MISMATCH** (MOD-051) | Tidak ada handler normalizer (`no handler`). Data rute Golden SAML hilang dari `ArtifactStore`. |
-| `oauth_tokens` | `cloud.identity_federation_abuse` | — | `oauth_tokens` | ❌ **MISMATCH** (MOD-051) | Tidak ada handler normalizer (`no handler`). Token OAuth2 hilang dari `ArtifactStore`. |
-| `pivot_paths` | `cloud.identity_federation_abuse` | — | `pivot_paths` | ❌ **MISMATCH** (MOD-051) | Tidak ada handler normalizer (`no handler`). Analisis jalur pivot lintas-cloud hilang dari `ArtifactStore`. |
-| `aws_privesc_paths` | `cloud.aws_privesc` | — | `aws_privesc_paths` | ❌ **MISMATCH** (MOD-052) | Tidak ada handler normalizer (`no handler`). Jalur eskalasi hak akses IAM hilang dari `ArtifactStore`. |
-| `iam_privesc_paths` | `cloud.aws_privesc` | — | `iam_privesc_paths` | ⚠️ **NEW KEY** (aws_privesc, handler pending) | Key baru pasca-fix MOD-052 untuk memisahkan output privesc IAM dari `aws_findings` milik `cloud.aws`. Handler normalizer pending. |
+| `converted_ticket` | `credential.ticket_converter` | `converted_ticket` OR `converted_ticket_b64` | `converted_ticket_b64` | ✅ **FIXED** (MOD-036 / Fase 1) | Handler `_normalize_converted_ticket` memetakan tiket base64/file path ke `CredentialArtifact(cred_type="kerberos_ticket")`. |
+| `container_escape_vectors` | `linux.container` | `container_escape_vectors` | `container_escape_vectors` | ✅ **FIXED** (MOD-040 / Fase 1) | Handler `_normalize_container_vectors` memetakan temuan container breakout ke `PermissionArtifact`. |
+| `k8s_rbac_findings` | `linux.container` | `k8s_rbac_findings` | `k8s_rbac_findings` | ✅ **FIXED** (MOD-040 / Fase 1) | Handler `_normalize_k8s_rbac` memetakan privilege cluster K8s ke `PermissionArtifact`. |
+| `machine_account_hash` | `linux.samba_secrets` | `machine_account_hash` | `machine_account_hash` | ✅ **FIXED** (MOD-041 / Fase 1) | Handler `_normalize_samba_secrets` memetakan NTLM machine hash ke `HashArtifact(hash_type="ntlm")`. |
+| `samba_secrets` | `linux.samba_secrets` | `samba_secrets` | `samba_secrets` | ✅ **FIXED** (MOD-041 / Fase 1) | Handler `_normalize_samba_secrets` memetakan rahasia domain Samba ke `CredentialArtifact`. |
+| `machine_credentials` | `linux.keytab_abuse` | `machine_credentials` OR `entries` | `entries` | ✅ **FIXED** (MOD-044 / Fase 1) | Handler `_normalize_keytab_keys` memetakan entry keytab ke `CredentialArtifact`. |
+| `kerberos_keys` | `linux.keytab_abuse` | `kerberos_keys` OR `silver_tickets` | `silver_tickets` | ✅ **FIXED** (MOD-044 / Fase 1) | Handler `_normalize_keytab_keys` memetakan tiket/kunci Kerberos ke `CredentialArtifact`. |
+| `cached_hashes` | `linux.sssd_harvest` | `cached_hashes`, `hashes` | `cached_hashes`, `hashes` | ✅ **FIXED** (MOD-045 / Fase 1) | Handler `_normalize_cached_hashes` dan `_normalize_ntlm_hashes` memetakan hash SSSD ke `HashArtifact`. |
+| `domain_users` | `linux.sssd_harvest` | `domain_users`, `users` | `domain_users`, `users` | ✅ **FIXED** (MOD-045 / Fase 1) | Handler `_normalize_domain_users` dan `_normalize_users` memetakan user domain SSSD ke `UserArtifact`. |
+| `federation_trusts` | `cloud.identity_federation_abuse` | `federation_trusts` | `federation_trusts` | ✅ **FIXED** (MOD-051 / Fase 1) | Handler `_normalize_federation_trusts` memetakan trust IdP ke `CloudResourceArtifact`. |
+| `golden_saml_paths` | `cloud.identity_federation_abuse` | `golden_saml_paths` | `golden_saml_paths` | ✅ **FIXED** (MOD-051 / Fase 1) | Handler `_normalize_golden_saml` memetakan rute Golden SAML ke `PermissionArtifact`. |
+| `oauth_tokens` | `cloud.identity_federation_abuse` | `oauth_tokens` | `oauth_tokens` | ✅ **FIXED** (MOD-051 / Fase 1) | Handler `_normalize_tokens` memetakan token OAuth2 ke `CredentialArtifact(cred_type="oauth_token")`. |
+| `pivot_paths` | `cloud.identity_federation_abuse` | `pivot_paths` | `pivot_paths` | ✅ **FIXED** (MOD-051 / Fase 1) | Handler `_normalize_pivot_paths` memetakan jalur pivot multi-cloud ke `PermissionArtifact`. |
+| `aws_privesc_paths` | `cloud.aws_privesc` | `aws_privesc_paths` | `aws_privesc_paths` | ✅ **FIXED** (MOD-052 / Fase 1) | Handler `_normalize_iam_privesc` memetakan jalur privesc IAM ke `PermissionArtifact`. |
+| `iam_privesc_paths` | `cloud.aws_privesc` | `iam_privesc_paths` | `iam_privesc_paths` | ✅ **FIXED** (MOD-052 / Fase 1) | Handler `_normalize_iam_privesc` memetakan rute privesc IAM terisolasi ke `PermissionArtifact`. |
| `spn_list` (inner keys) | `ad.enum_spn` | `s["spns"]` OR `s["spn_list"]` | `spns`, `spn_list` | ✅ **FIXED** (MOD-054) | Inner object key synchronized (`spn_list` & `spns`), `UserArtifact.spns` populated properly. |
| `valid_credentials` | `ad.laps_enum` | `laps_passwords` / `valid_credentials` | direct vault write + OPSEC raw (`has_password: True`) | ✅ **FIXED** (MOD-055) | Passwords stored directly to vault (Gate 6 validated); raw output OPSEC-safe with `has_password: True`. |
| `user_list` / `users` | `ad.enum_users` | `users` OR `user_list` | `users`, `user_list` | ✅ **FIXED** (MOD-056 partial) | Dual-write `raw["users"]` and `raw["user_list"]` implemented. |
-| `password_policy` | `ad.enum_users` | — | `password_policy` | ❌ **MISMATCH, no handler, DEFERRED** | Unhandled output telemetry (`no handler`). Ditunda ke batch fix serentak. |
-| `snmp_findings` / `valid_credentials` | `network.snmp_enum` | `valid_credentials` | `valid_credentials`, `snmp_findings` | ✅ **FIXED** (MOD-061, commit `5635362`) | Direct vault write Gate 6 + `valid_credentials` contract standar `list[dict]`. |
-| Reconnaissance capabilities (`dns_records`, `subdomains`, `web_fingerprint`, `admin_interfaces`, `service_versions`, `vulnerable_services`) | `recon.fingerprint`, `network.dns_enum`, `network.http_fingerprint`, `network.service_detect` | `dns_records`, `subdomains`, `service_versions`, `web_fingerprint`, dll. | Various raw dicts | ✅ **PARTIALLY FIXED** (7 Handlers in commit `5635362`) | 7 handler normalizer recon ditambahkan ke `ArtifactNormalizer`. Sisa recon capability ditunda ke batch fix serentak. |
-| `azure_findings` | `cloud.azure` | — | `azure_findings` | ❌ **MISMATCH, no handler, DEFERRED** (MOD-066) | Output capability `azure_findings` tidak memiliki handler di `ArtifactNormalizer`. Data storage accounts, RBAC, dan NSG hilang 100% dari `ArtifactStore`. |
-| `azure_ad_findings` | `cloud.azure_ad` | — | `azure_ad_findings` | ❌ **MISMATCH, no handler, DEFERRED** (MOD-066) | Output capability `azure_ad_findings` tidak memiliki handler di `ArtifactNormalizer`. Data user dan guest Entra ID serta privileged service principals hilang dari `ArtifactStore`. |
-| `access_tokens` | `cloud.azure_ad` | — | `access_tokens` | ❌ **MISMATCH, no handler, DEFERRED** (MOD-066) | Output capability `access_tokens` (captured device code / client credential tokens) tidak memiliki handler di `ArtifactNormalizer`. |
-| `gcp_findings` | `cloud.gcp` | — | `gcp_findings` | ❌ **MISMATCH, no handler, DEFERRED** (MOD-068) | Output capability `gcp_findings` tidak memiliki handler di `ArtifactNormalizer`. Data public GCS buckets, IAM project roles, dan SA keys hilang dari `ArtifactStore`. |
-| `cleartext_credentials` | `windows.registry_enum` | — | `cleartext_credentials` (Finding objects) | ❌ **MISMATCH, no handler, DEFERRED** (MOD-070) | Domain object Finding di-assign langsung ke raw key; tidak ada handler di `ArtifactNormalizer`. Data kredensial registry hilang dari `ArtifactStore`. |
-| `credential_hints` | `windows.registry_enum` | — | `credential_hints` (Finding objects) | ❌ **MISMATCH, no handler, DEFERRED** (MOD-070) | Domain object Finding di-assign langsung ke raw key; tidak ada handler di `ArtifactNormalizer`. |
-| `scheduled_tasks` | `windows.scheduled_tasks_enum` | — | `scheduled_tasks` (Finding objects) | ❌ **MISMATCH, no handler, DEFERRED** (MOD-070) | Domain object Finding di-assign langsung ke raw key; tidak ada handler di `ArtifactNormalizer`. Data task berisiko tinggi hilang dari `ArtifactStore`. |
-| `privesc_vectors` (finding objects) | `windows.scheduled_tasks_enum`, `linux.kernel_suggester` | — | `privesc_vectors` (Finding objects) | ❌ **MISMATCH, no handler, DEFERRED** (MOD-070) | List Finding objek mentah di-assign ke `privesc_vectors`; normalizer yang ada hanya membaca string host/target, mengabaikan struktur vektor eskalasi hak akses. |
-| `credential_list` | `exfil.secrets_scan` | — | `credential_list` | ❌ **MISMATCH, no handler, DEFERRED** (MOD-074) | Tidak ada handler normalizer di `ArtifactNormalizer`. Path file rahasia hilang dari `ArtifactStore`. |
-| `discovered_secrets` | `exfil.secrets_scan` | — | `discovered_secrets` | ❌ **MISMATCH, no handler, DEFERRED** (MOD-074) | Metadata rahasia (API key, private key, connection strings) dengan entropy tidak terserap ke `ArtifactStore`. |
-| `sensitive_data_found` | `exfil.secrets_scan`, `exfil.smb_shares` | — | `sensitive_data_found` | ❌ **MISMATCH, no handler, DEFERRED** (MOD-074) | Status boolean sensitif tidak dipetakan ke atribut `HostArtifact` atau `PermissionArtifact`. |
-| `file_share_list` | `exfil.smb_shares` | — | `file_share_list` | ❌ **MISMATCH, no handler, DEFERRED** (MOD-074) | Tidak ada handler normalizer untuk share SMB. Daftar share target hilang dari `ArtifactStore`. |
-| `sensitive_file_paths` | `exfil.smb_shares` | — | `sensitive_file_paths` | ❌ **MISMATCH, no handler, DEFERRED** (MOD-074) | File sensitif (web.config, id_rsa, .kdbx) di share SMB hilang dari `ArtifactStore`. |
+| `password_policy` | `ad.enum_users` | `password_policy` | `password_policy` | ✅ **FIXED** (MOD-056 / Fase 1) | Handler `_normalize_password_policy` menyimpan telemetry password policy ke metadata `HostArtifact`. |
+| `snmp_findings` / `valid_credentials` | `network.snmp_enum` | `valid_credentials`, `snmp_findings` | `valid_credentials`, `snmp_findings` | ✅ **FIXED** (MOD-061 / Fase 1) | Direct vault write Gate 6 + community string ke `CredentialArtifact` & info sistem ke `HostArtifact`. |
+| Reconnaissance capabilities (`dns_records`, `subdomains`, `web_fingerprint`, `admin_interfaces`, `service_versions`, `vulnerable_services`) | `recon.fingerprint`, `network.dns_enum`, `network.http_fingerprint`, `network.service_detect` | `dns_records`, `subdomains`, `service_versions`, `web_fingerprint`, dll. | Various raw dicts | ✅ **FIXED** (Fase 1) | Seluruh recon capability dipetakan ke `HostArtifact` (hostnames, services, vulns, metadata). |
+| `azure_findings` | `cloud.azure` | `azure_findings` | `azure_findings` | ✅ **FIXED** (MOD-066 / Fase 1) | Handler `_normalize_azure` memetakan storage accounts, NSG, RBAC ke `CloudResourceArtifact`. |
+| `azure_ad_findings` | `cloud.azure_ad` | `azure_ad_findings` | `azure_ad_findings` | ✅ **FIXED** (MOD-066 / Fase 1) | Handler `_normalize_azure_ad` memetakan Entra ID users & service principals ke `UserArtifact` & `CloudResourceArtifact`. |
+| `access_tokens` | `cloud.azure_ad` | `access_tokens`, `access_token` | `access_tokens` | ✅ **FIXED** (MOD-066 / Fase 1) | Handler `_normalize_tokens` memetakan access tokens ke `CredentialArtifact(cred_type="access_token")`. |
+| `gcp_findings` | `cloud.gcp` | `gcp_findings` | `gcp_findings` | ✅ **FIXED** (MOD-068 / Fase 1) | Handler `_normalize_gcp` memetakan GCS buckets, roles, SA keys ke `CloudResourceArtifact`. |
+| `cleartext_credentials` | `windows.registry_enum` | `cleartext_credentials` | `cleartext_credentials` (Finding objects / dict) | ✅ **FIXED** (MOD-070 / Fase 1) | Handler `_normalize_registry_credentials` mengekstrak kredensial dari Finding objects ke `CredentialArtifact`. |
+| `credential_hints` | `windows.registry_enum` | `credential_hints` | `credential_hints` (Finding objects / dict) | ✅ **FIXED** (MOD-070 / Fase 1) | Handler `_normalize_registry_credentials` mengekstrak hint registry ke `CredentialArtifact`. |
+| `scheduled_tasks` | `windows.scheduled_tasks_enum` | `scheduled_tasks` | `scheduled_tasks` (Finding objects / dict) | ✅ **FIXED** (MOD-070 / Fase 1) | Handler `_normalize_scheduled_tasks` mengekstrak task berbahaya ke metadata `HostArtifact`. |
+| `privesc_vectors` (finding objects) | `windows.scheduled_tasks_enum`, `linux.kernel_suggester` | `privesc_vectors` | `privesc_vectors` (Finding objects / strings) | ✅ **FIXED** (MOD-070 / Fase 1) | Handler `_normalize_host_vuln` mendukung list Finding objects maupun host strings tanpa crash. |
+| `credential_list` | `exfil.secrets_scan` | `credential_list` | `credential_list` | ✅ **FIXED** (MOD-074 / Fase 1) | Handler `_normalize_secrets_scan` memetakan file paths dan credential records ke `CredentialArtifact`. |
+| `discovered_secrets` | `exfil.secrets_scan` | `discovered_secrets` | `discovered_secrets` | ✅ **FIXED** (MOD-074 / Fase 1) | Handler `_normalize_secrets_scan` memetakan API keys & private keys ke `CredentialArtifact`. |
+| `sensitive_data_found` | `exfil.secrets_scan`, `exfil.smb_shares` | `sensitive_data_found` | `sensitive_data_found` | ✅ **FIXED** (MOD-074 / Fase 1) | Handler `_normalize_secrets_scan` & `_normalize_smb_shares` mencatat temuan data sensitif ke `HostArtifact`. |
+| `file_share_list` | `exfil.smb_shares` | `file_share_list` | `file_share_list` | ✅ **FIXED** (MOD-074 / Fase 1) | Handler `_normalize_smb_shares` memetakan SMB shares dan permissions ke metadata `HostArtifact`. |
+| `sensitive_file_paths` | `exfil.smb_shares` | `sensitive_file_paths` | `sensitive_file_paths` | ✅ **FIXED** (MOD-074 / Fase 1) | Handler `_normalize_smb_shares` memetakan file sensitif ke metadata `HostArtifact`. |
---
diff --git a/ares/normalize/artifacts.py b/ares/normalize/artifacts.py
index be957c3..0daeb02 100644
--- a/ares/normalize/artifacts.py
+++ b/ares/normalize/artifacts.py
@@ -79,6 +79,7 @@ class NormalizedArtifact:
campaign_id: str = ""
tags: list[str] = field(default_factory=list)
raw: dict[str, Any] = field(default_factory=dict)
+ metadata: dict[str, Any] = field(default_factory=dict)
_uid: str | None = field(default=None, repr=False)
@property
@@ -100,6 +101,7 @@ class NormalizedArtifact:
"source_module": self.source_module,
"campaign_id": self.campaign_id,
"tags": self.tags,
+ "metadata": self.metadata,
}
d.update(self._to_dict_fields())
return d
@@ -129,6 +131,9 @@ class HostArtifact(NormalizedArtifact):
is_dc: bool = False
domain_controller: bool = False # alias for is_dc
open_ports: list[int] = field(default_factory=list)
+ services: list[str] = field(default_factory=list)
+ vulns: list[dict[str, Any]] = field(default_factory=list)
+ hostnames: list[str] = field(default_factory=list)
def __post_init__(self) -> None:
self.artifact_type = ArtifactType.HOST
@@ -151,6 +156,7 @@ class HostArtifact(NormalizedArtifact):
"ip_address": self.ip_address, "hostname": self.hostname,
"fqdn": self.fqdn, "os": self.os, "os_version": self.os_version,
"domain": self.domain, "is_dc": self.is_dc, "open_ports": self.open_ports,
+ "services": self.services, "vulns": self.vulns, "hostnames": self.hostnames,
}
@@ -487,12 +493,14 @@ class ArtifactNormalizer:
handlers: dict[str, Any] = {
"user_list": self._normalize_users,
"users": self._normalize_users,
+ "domain_users": self._normalize_users,
"computer_list": self._normalize_computers,
"computers": self._normalize_computers,
"kerberos_hashes": self._normalize_kerberos_hashes,
"asrep_hashes": self._normalize_asrep_hashes,
"ntlm_hashes": self._normalize_ntlm_hashes,
"hashes": self._normalize_ntlm_hashes,
+ "cached_hashes": self._normalize_ntlm_hashes,
"spn_list": self._normalize_spns,
"spns": self._normalize_spns,
"acl_findings": self._normalize_permissions,
@@ -537,6 +545,34 @@ class ArtifactNormalizer:
"vulnerable_services": self._normalize_service_versions,
"web_fingerprint": self._normalize_web_fingerprint,
"admin_interfaces": self._normalize_web_fingerprint,
+ # Azure & Azure AD (MOD-066)
+ "azure_findings": self._normalize_azure,
+ "azure_ad_findings": self._normalize_azure_ad,
+ "access_tokens": self._normalize_tokens,
+ "access_token": self._normalize_tokens,
+ "oauth_tokens": self._normalize_tokens,
+ # GCP (MOD-068)
+ "gcp_findings": self._normalize_gcp,
+ # Linux Containers & K8s (MOD-040)
+ "container_escape_vectors": self._normalize_container_vectors,
+ "k8s_rbac_findings": self._normalize_k8s_rbac,
+ # Identity Federation (MOD-051)
+ "federation_trusts": self._normalize_federation_trusts,
+ "golden_saml_paths": self._normalize_golden_saml,
+ "pivot_paths": self._normalize_pivot_paths,
+ # SNMP & Network Discovery (MOD-061 / Recon)
+ "snmp_findings": self._normalize_snmp_findings,
+ "system_info": self._normalize_system_info,
+ "password_policy": self._normalize_password_policy,
+ # Windows Scheduled Tasks & Registry (MOD-070)
+ "scheduled_tasks": self._normalize_scheduled_tasks,
+ "credential_hints": self._normalize_registry_credentials,
+ # Secrets Scanner & SMB Shares (MOD-074)
+ "credential_list": self._normalize_secrets_scan,
+ "discovered_secrets": self._normalize_secrets_scan,
+ "sensitive_data_found": self._normalize_secrets_scan,
+ "file_share_list": self._normalize_smb_shares,
+ "sensitive_file_paths": self._normalize_smb_shares,
}
handler = handlers.get(capability)
if handler:
@@ -760,30 +796,77 @@ class ArtifactNormalizer:
def _normalize_host_vuln(self, raw: dict, store: ArtifactStore) -> int:
host = raw.get("target") or raw.get("host", "")
- if host and host != "localhost":
- artifact = HostArtifact(
- ip_address = host,
- hostname = host,
- )
- store.add(artifact)
+ if not host or host == "localhost":
+ return 0
+ existing = None
+ for h in store.hosts():
+ if h.ip_address == host or h.hostname == host:
+ existing = h
+ break
+ vulns = raw.get("privesc_vectors", [])
+ parsed_vulns = []
+ if isinstance(vulns, list):
+ for v in vulns:
+ if isinstance(v, dict):
+ parsed_vulns.append(v)
+ elif hasattr(v, "evidence") and isinstance(v.evidence, dict):
+ parsed_vulns.append(v.evidence)
+ elif isinstance(v, str) and v:
+ parsed_vulns.append({"vector": v})
+ if existing:
+ if parsed_vulns:
+ existing.vulns.extend(parsed_vulns)
return 1
- return 0
+ artifact = HostArtifact(
+ ip_address = host,
+ hostname = host,
+ vulns = parsed_vulns,
+ )
+ store.add(artifact)
+ return 1
# ── P0 Handlers ────────────────────────────────────────────────────────────
def _normalize_cleartext_credentials(self, raw: dict, store: ArtifactStore) -> int:
creds = raw.get("cleartext_credentials") or raw.get("credentials", [])
+ if isinstance(creds, (dict, str)):
+ creds = [creds]
+ elif not isinstance(creds, list):
+ return 0
count = 0
+ default_target = raw.get("target") or raw.get("host", "")
for c in creds:
- if not isinstance(c, dict):
+ if hasattr(c, "evidence") and isinstance(c.evidence, dict):
+ ev = c.evidence
+ username = ev.get("user") or ev.get("username", "")
+ secret = ev.get("password") or ev.get("secret", "")
+ domain = ev.get("domain", "")
+ source_host = ev.get("host") or default_target
+ privilege = "unknown"
+ elif isinstance(c, dict):
+ username = c.get("username") or c.get("user") or c.get("account", "")
+ domain = c.get("domain", "")
+ secret = c.get("password") or c.get("secret") or c.get("plaintext", "")
+ source_host = c.get("host") or c.get("target") or c.get("computer") or default_target
+ privilege = c.get("privilege", "unknown")
+ elif isinstance(c, str):
+ username = "unknown"
+ domain = ""
+ secret = c
+ source_host = default_target
+ privilege = "unknown"
+ else:
+ logger.warning("cleartext_credentials_invalid_entry", type=type(c).__name__)
+ continue
+ if not username and not secret:
continue
artifact = CredentialArtifact(
- username = c.get("username", ""),
- domain = c.get("domain", ""),
+ username = username,
+ domain = domain,
cred_type = "cleartext",
- secret = c.get("password") or c.get("secret") or c.get("plaintext", ""),
- source_host = c.get("host") or c.get("target") or c.get("computer", ""),
- privilege = c.get("privilege", "unknown"),
+ secret = secret,
+ source_host = source_host,
+ privilege = privilege,
)
store.add(artifact)
count += 1
@@ -1236,11 +1319,25 @@ class ArtifactNormalizer:
return 0
service_versions = raw.get("service_versions", {})
ports: list[int] = []
+ services: list[str] = []
if isinstance(service_versions, dict):
- for p_str in service_versions:
+ for p_str, info in service_versions.items():
if str(p_str).isdigit():
ports.append(int(p_str))
+ if isinstance(info, dict) and info.get("service"):
+ services.append(f"{p_str}:{info['service']}")
+ elif isinstance(info, str) and info:
+ services.append(f"{p_str}:{info}")
ports = sorted(list(set(ports)))
+ vuln_svcs = raw.get("vulnerable_services", [])
+ parsed_vulns = []
+ if isinstance(vuln_svcs, list):
+ for vs in vuln_svcs:
+ if isinstance(vs, dict):
+ parsed_vulns.append(vs)
+ elif hasattr(vs, "evidence") and isinstance(vs.evidence, dict):
+ parsed_vulns.append(vs.evidence)
+
existing = None
for h in store.hosts():
if h.ip_address == target or h.hostname == target:
@@ -1249,6 +1346,10 @@ class ArtifactNormalizer:
if existing:
if ports:
existing.open_ports = sorted(list(set(existing.open_ports + ports)))
+ if services:
+ existing.services = sorted(list(set(existing.services + services)))
+ if parsed_vulns:
+ existing.vulns.extend(parsed_vulns)
if raw.get("os") and not existing.os:
existing.os = raw.get("os", "")
return 1
@@ -1257,6 +1358,8 @@ class ArtifactNormalizer:
hostname=target,
os=raw.get("os", "") or raw.get("detected_os", ""),
open_ports=ports,
+ services=services,
+ vulns=parsed_vulns,
source_module=capability,
)
store.add(artifact)
@@ -1271,6 +1374,7 @@ class ArtifactNormalizer:
return 0
web_fp = raw.get("web_fingerprint", {})
server = web_fp.get("server", "") if isinstance(web_fp, dict) else ""
+ admin_interfaces = raw.get("admin_interfaces", [])
existing = None
for h in store.hosts():
if h.ip_address == target or h.hostname == target:
@@ -1279,14 +1383,777 @@ class ArtifactNormalizer:
if existing:
if server and not existing.os:
existing.os = server
+ if admin_interfaces:
+ existing.metadata["admin_interfaces"] = admin_interfaces
return 1
artifact = HostArtifact(
ip_address=target,
hostname=target,
os=server,
+ metadata={"admin_interfaces": admin_interfaces} if admin_interfaces else {},
source_module=capability,
)
store.add(artifact)
return 1
+ def _normalize_azure(
+ self, raw: dict, store: ArtifactStore, capability: str = "azure_findings"
+ ) -> int:
+ """Normalize Azure recon and attack output into CloudResourceArtifact, PermissionArtifact, UserArtifact."""
+ count = 0
+ subscription_id = raw.get("subscription_id", "")
+ # 1. Storage
+ storage = raw.get("storage", {})
+ if isinstance(storage, dict):
+ for c in storage.get("public_containers", []):
+ if isinstance(c, dict):
+ sa = c.get("storage_account", "")
+ c_name = c.get("container", "")
+ rg = c.get("resource_group", "")
+ artifact = CloudResourceArtifact(
+ provider="azure",
+ resource_type="storage_container",
+ resource_id=f"{sa}/{c_name}",
+ region=rg,
+ is_public=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ # 2. RBAC
+ rbac = raw.get("rbac", {})
+ if isinstance(rbac, dict):
+ for o in rbac.get("owner_assignments", []):
+ if isinstance(o, dict):
+ p_id = o.get("principal_id", "")
+ scope = o.get("scope", "") or f"/subscriptions/{subscription_id}"
+ if p_id:
+ artifact = PermissionArtifact(
+ principal=p_id,
+ target=scope,
+ right="Owner",
+ domain=subscription_id,
+ is_dangerous=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ for c in rbac.get("contributor_assignments", []):
+ if isinstance(c, dict):
+ p_id = c.get("principal_id", "")
+ scope = c.get("scope", "") or f"/subscriptions/{subscription_id}"
+ if p_id:
+ artifact = PermissionArtifact(
+ principal=p_id,
+ target=scope,
+ right="Contributor",
+ domain=subscription_id,
+ is_dangerous=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ for admin in rbac.get("classic_admins", []):
+ if isinstance(admin, str) and admin:
+ artifact = PermissionArtifact(
+ principal=admin,
+ target=f"/subscriptions/{subscription_id}",
+ right="ClassicCoAdministrator",
+ domain=subscription_id,
+ is_dangerous=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ # 3. NSG
+ nsg = raw.get("nsg", {})
+ if isinstance(nsg, dict):
+ for rule in nsg.get("open_rules", []):
+ if isinstance(rule, dict):
+ nsg_name = rule.get("nsg", "")
+ r_name = rule.get("rule_name", "")
+ rg = rule.get("resource_group", "")
+ artifact = CloudResourceArtifact(
+ provider="azure",
+ resource_type="network_security_group",
+ resource_id=f"{nsg_name}/{r_name}" if r_name else nsg_name,
+ region=rg,
+ is_public=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ # 4. AAD users
+ aad = raw.get("aad_users", {})
+ if isinstance(aad, dict):
+ for u in aad.get("admin_users", []):
+ if isinstance(u, str) and u:
+ artifact = UserArtifact(
+ username=u,
+ domain="azure_ad",
+ is_admin=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ return count
+
+ def _normalize_azure_ad(
+ self, raw: dict, store: ArtifactStore, capability: str = "azure_ad_findings"
+ ) -> int:
+ """Normalize Azure AD findings into UserArtifact and PermissionArtifact."""
+ count = 0
+ tenant_id = raw.get("tenant_id", "azure_ad")
+ # 1. Privileged users
+ for u in raw.get("privileged_users", []):
+ if isinstance(u, str) and u:
+ artifact = UserArtifact(
+ username=u,
+ domain=tenant_id,
+ is_admin=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ # 2. Guests
+ for g in raw.get("guests", []):
+ g_name = g.get("userPrincipalName", "") if isinstance(g, dict) else (g if isinstance(g, str) else "")
+ if g_name:
+ artifact = UserArtifact(
+ username=g_name,
+ domain=tenant_id,
+ is_admin=False,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ # 3. High privilege service principals
+ for sp in raw.get("high_priv_sps", []):
+ if isinstance(sp, str) and sp:
+ artifact = PermissionArtifact(
+ principal=sp,
+ target=tenant_id,
+ right="PrivilegedAppRole",
+ domain=tenant_id,
+ is_dangerous=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ # 4. Service principals list
+ for sp in raw.get("service_principals", []):
+ if isinstance(sp, dict) and sp.get("privileged"):
+ sp_name = sp.get("displayName") or sp.get("appId", "")
+ if sp_name:
+ artifact = PermissionArtifact(
+ principal=sp_name,
+ target=tenant_id,
+ right="PrivilegedServicePrincipal",
+ domain=tenant_id,
+ is_dangerous=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ return count
+
+ def _normalize_tokens(
+ self, raw: dict, store: ArtifactStore, capability: str = "access_tokens"
+ ) -> int:
+ """Normalize access tokens and OAuth tokens into CredentialArtifact(cred_type='oauth_token')."""
+ count = 0
+ tokens = raw.get("access_tokens") or raw.get("oauth_tokens") or raw.get("access_token") or raw.get("tokens")
+ if not tokens:
+ return 0
+ if isinstance(tokens, (str, dict)):
+ tokens = [tokens]
+ elif not isinstance(tokens, list):
+ return 0
+ tenant = raw.get("tenant_id") or raw.get("domain") or "cloud"
+ for t in tokens:
+ if isinstance(t, str):
+ t_str = t.strip()
+ if not t_str:
+ continue
+ artifact = CredentialArtifact(
+ username="oauth_bearer",
+ domain=tenant,
+ cred_type="oauth_token",
+ secret=t_str,
+ source_host=tenant,
+ privilege="token",
+ source_module=capability,
+ )
+ store.add_credential(artifact)
+ count += 1
+ elif isinstance(t, dict):
+ tok_val = t.get("access_token") or t.get("token") or t.get("secret", "")
+ if not tok_val:
+ continue
+ username = t.get("user") or t.get("username") or t.get("client_id") or "oauth_bearer"
+ scope = t.get("scope") or t.get("resource", "")
+ artifact = CredentialArtifact(
+ username=username,
+ domain=tenant,
+ cred_type="oauth_token",
+ secret=tok_val,
+ source_host=tenant,
+ privilege="token",
+ note=f"scope={scope}" if scope else "",
+ source_module=capability,
+ )
+ store.add_credential(artifact)
+ count += 1
+ else:
+ logger.warning("access_token_invalid_entry", type=type(t).__name__)
+ return count
+
+ def _normalize_gcp(
+ self, raw: dict, store: ArtifactStore, capability: str = "gcp_findings"
+ ) -> int:
+ """Normalize GCP reconnaissance findings into CloudResourceArtifact and PermissionArtifact."""
+ count = 0
+ project_id = raw.get("project_id", "gcp")
+ # 1. GCS public buckets
+ gcs = raw.get("gcs", {})
+ if isinstance(gcs, dict):
+ for b in gcs.get("public_buckets", []):
+ if isinstance(b, dict):
+ b_name = b.get("bucket", "")
+ if b_name:
+ artifact = CloudResourceArtifact(
+ provider="gcp",
+ resource_type="gcs_bucket",
+ resource_id=b_name,
+ region=project_id,
+ is_public=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ # 2. IAM bindings
+ iam = raw.get("iam", {})
+ if isinstance(iam, dict):
+ for pb in iam.get("public_bindings", []):
+ if isinstance(pb, str) and " → " in pb:
+ member, role = pb.split(" → ", 1)
+ artifact = PermissionArtifact(
+ principal=member.strip(),
+ target=project_id,
+ right=role.strip(),
+ domain="gcp",
+ is_dangerous=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ for ob in iam.get("owner_bindings", []):
+ if isinstance(ob, str) and ob:
+ artifact = PermissionArtifact(
+ principal=ob,
+ target=project_id,
+ right="roles/owner",
+ domain="gcp",
+ is_dangerous=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ # 3. Service account keys
+ sa = raw.get("service_accounts", {})
+ if isinstance(sa, dict):
+ for mk in sa.get("many_keys", []):
+ if isinstance(mk, dict):
+ email = mk.get("service_account", "")
+ key_count = mk.get("active_key_count", 0)
+ if email:
+ artifact = CredentialArtifact(
+ username=email,
+ domain="gcp",
+ cred_type="service_account_key",
+ source_host=project_id,
+ privilege="service_account",
+ note=f"active_keys={key_count}",
+ source_module=capability,
+ )
+ store.add_credential(artifact)
+ count += 1
+ return count
+
+ def _normalize_container_vectors(
+ self, raw: dict, store: ArtifactStore, capability: str = "container_escape_vectors"
+ ) -> int:
+ """Normalize container escape vectors into HostArtifact vulns and metadata."""
+ target = raw.get("target") or raw.get("host") or "container-host"
+ existing = None
+ for h in store.hosts():
+ if h.ip_address == target or h.hostname == target:
+ existing = h
+ break
+ host = existing or HostArtifact(
+ ip_address=target,
+ hostname=target,
+ source_module=capability,
+ )
+ count = 0
+ # Check docker socket
+ ds = raw.get("docker_socket", {})
+ if isinstance(ds, dict) and ds.get("writable"):
+ vuln_entry = {"type": "container_escape", "vector": "docker_socket_writable", "severity": "CRITICAL"}
+ host.vulns.append(vuln_entry)
+ host.metadata.setdefault("container_escape_vectors", []).append(vuln_entry)
+ count += 1
+ # Check privileged
+ priv = raw.get("privileged", {})
+ if isinstance(priv, dict) and priv.get("privileged"):
+ vuln_entry = {"type": "container_escape", "vector": "privileged_container", "severity": "CRITICAL"}
+ host.vulns.append(vuln_entry)
+ host.metadata.setdefault("container_escape_vectors", []).append(vuln_entry)
+ count += 1
+ # Check host mounts
+ hm = raw.get("host_mounts", {})
+ if isinstance(hm, dict) and hm.get("high"):
+ vuln_entry = {"type": "container_escape", "vector": "host_mounts_high_risk", "mounts": hm["high"], "severity": "CRITICAL"}
+ host.vulns.append(vuln_entry)
+ host.metadata.setdefault("container_escape_vectors", []).append(vuln_entry)
+ count += 1
+ # If raw["container_escape_vectors"] has items
+ vectors = raw.get("container_escape_vectors", [])
+ if isinstance(vectors, list):
+ for v in vectors:
+ if isinstance(v, dict):
+ host.vulns.append(v)
+ count += 1
+ elif hasattr(v, "evidence") and isinstance(v.evidence, dict):
+ host.vulns.append(v.evidence)
+ count += 1
+ if count and not existing:
+ store.add(host)
+ return max(count, 1 if count else 0)
+
+ def _normalize_k8s_rbac(
+ self, raw: dict, store: ArtifactStore, capability: str = "k8s_rbac_findings"
+ ) -> int:
+ """Normalize Kubernetes RBAC findings into PermissionArtifact."""
+ findings = raw.get("k8s_rbac_findings") or raw.get("k8s_findings", [])
+ if not isinstance(findings, list):
+ return 0
+ count = 0
+ for f in findings:
+ if not isinstance(f, dict):
+ if hasattr(f, "evidence") and isinstance(f.evidence, dict):
+ f = f.evidence
+ else:
+ continue
+ principal = f.get("service_account") or f.get("principal") or "system:serviceaccount"
+ target = f.get("resource") or f.get("target") or "k8s_cluster"
+ verb = f.get("verb") or f.get("right") or "cluster-admin"
+ artifact = PermissionArtifact(
+ principal=principal,
+ target=target,
+ right=verb,
+ domain="k8s",
+ is_dangerous=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ return count
+
+ def _normalize_federation_trusts(
+ self, raw: dict, store: ArtifactStore, capability: str = "federation_trusts"
+ ) -> int:
+ """Normalize identity federation trusts into CloudResourceArtifact."""
+ trusts = raw.get("federation_trusts", [])
+ if isinstance(trusts, (str, dict)):
+ trusts = [trusts]
+ elif not isinstance(trusts, list):
+ return 0
+ count = 0
+ for t in trusts:
+ if isinstance(t, str):
+ t_str = t.strip()
+ if not t_str:
+ continue
+ if "Azure" in t_str:
+ provider = "azure"
+ res_type = "federation_trust"
+ res_id = t_str.split(":", 1)[1].strip() if ":" in t_str else t_str
+ elif "AWS" in t_str:
+ provider = "aws"
+ res_type = "saml_provider"
+ res_id = t_str.split(":", 1)[1].strip() if ":" in t_str else t_str
+ elif "ADFS" in t_str:
+ provider = "adfs"
+ res_type = "relying_party"
+ res_id = t_str.split(":", 1)[1].strip() if ":" in t_str else t_str
+ else:
+ provider = "cloud"
+ res_type = "federation_trust"
+ res_id = t_str
+ artifact = CloudResourceArtifact(
+ provider=provider,
+ resource_type=res_type,
+ resource_id=res_id,
+ is_public=False,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ elif isinstance(t, dict):
+ artifact = CloudResourceArtifact(
+ provider=t.get("provider", "cloud"),
+ resource_type=t.get("resource_type", "federation_trust"),
+ resource_id=t.get("resource_id") or t.get("name", "trust"),
+ is_public=t.get("is_public", False),
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ else:
+ logger.warning("federation_trust_invalid_entry", type=type(t).__name__)
+ return count
+
+ def _normalize_golden_saml(
+ self, raw: dict, store: ArtifactStore, capability: str = "golden_saml_paths"
+ ) -> int:
+ """Normalize Golden SAML paths into PermissionArtifact."""
+ paths = raw.get("golden_saml_paths", [])
+ if isinstance(paths, dict):
+ paths = [paths]
+ elif not isinstance(paths, list):
+ return 0
+ count = 0
+ for p in paths:
+ if not isinstance(p, dict):
+ continue
+ rps = p.get("relying_parties", [])
+ for rp in (rps if isinstance(rps, list) else [str(rps)]):
+ if rp:
+ artifact = PermissionArtifact(
+ principal="adfs_token_signing_key",
+ target=rp,
+ right="GoldenSAML_AssertionForgery",
+ domain="federation",
+ is_dangerous=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ if not rps:
+ artifact = PermissionArtifact(
+ principal="adfs_token_signing_key",
+ target=p.get("attack", "Golden SAML"),
+ right="GoldenSAML",
+ domain="federation",
+ is_dangerous=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ return count
+
+ def _normalize_pivot_paths(
+ self, raw: dict, store: ArtifactStore, capability: str = "pivot_paths"
+ ) -> int:
+ """Normalize cross-cloud pivot paths into PermissionArtifact."""
+ paths = raw.get("pivot_paths", [])
+ if not isinstance(paths, list):
+ return 0
+ count = 0
+ for p in paths:
+ if not isinstance(p, dict):
+ continue
+ path_name = p.get("path") or p.get("name", "cross_cloud_pivot")
+ tech = p.get("technique", "T1606.002")
+ artifact = PermissionArtifact(
+ principal="federated_identity",
+ target=path_name,
+ right=tech,
+ domain="multi_cloud",
+ is_dangerous=True,
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ return count
+
+ def _normalize_snmp_findings(
+ self, raw: dict, store: ArtifactStore, capability: str = "snmp_findings"
+ ) -> int:
+ """Normalize SNMP enumeration findings into CredentialArtifact and HostArtifact."""
+ count = 0
+ target = raw.get("target") or raw.get("host", "")
+ # Community strings
+ valid_comms = raw.get("valid_communities", [])
+ if isinstance(valid_comms, list):
+ for vc in valid_comms:
+ comm = vc.get("community", "") if isinstance(vc, dict) else (vc if isinstance(vc, str) else "")
+ if comm:
+ artifact = CredentialArtifact(
+ username="snmp",
+ secret=comm,
+ cred_type="snmp_community",
+ source_host=target,
+ privilege="local_admin" if comm.lower() in ("private", "write") else "user",
+ source_module=capability,
+ )
+ store.add_credential(artifact)
+ count += 1
+ # System info
+ sys_info = raw.get("system_info", {})
+ if isinstance(sys_info, dict) and target:
+ sys_descr = sys_info.get("sysDescr", "")
+ sys_name = sys_info.get("sysName", "")
+ existing = None
+ for h in store.hosts():
+ if h.ip_address == target or h.hostname == target:
+ existing = h
+ break
+ if existing:
+ if sys_descr and not existing.os:
+ existing.os = sys_descr
+ if sys_name and not existing.hostname:
+ existing.hostname = sys_name
+ existing.metadata["snmp_system_info"] = sys_info
+ else:
+ h = HostArtifact(
+ ip_address=target,
+ hostname=sys_name or target,
+ os=sys_descr,
+ metadata={"snmp_system_info": sys_info},
+ source_module=capability,
+ )
+ store.add(h)
+ count += 1
+ return count
+
+ def _normalize_system_info(
+ self, raw: dict, store: ArtifactStore, capability: str = "system_info"
+ ) -> int:
+ """Normalize generic system_info dict into HostArtifact."""
+ target = raw.get("target") or raw.get("host", "")
+ sys_info = raw.get("system_info", {})
+ if not target and isinstance(sys_info, dict):
+ target = sys_info.get("sysName") or sys_info.get("hostname", "")
+ if not target:
+ return 0
+ existing = None
+ for h in store.hosts():
+ if h.ip_address == target or h.hostname == target:
+ existing = h
+ break
+ if existing:
+ if isinstance(sys_info, dict):
+ existing.metadata["system_info"] = sys_info
+ if sys_info.get("sysDescr") and not existing.os:
+ existing.os = sys_info["sysDescr"]
+ return 1
+ h = HostArtifact(
+ ip_address=target,
+ hostname=target,
+ os=sys_info.get("sysDescr", "") if isinstance(sys_info, dict) else "",
+ metadata={"system_info": sys_info} if isinstance(sys_info, dict) else {},
+ source_module=capability,
+ )
+ store.add(h)
+ return 1
+
+ def _normalize_password_policy(
+ self, raw: dict, store: ArtifactStore, capability: str = "password_policy"
+ ) -> int:
+ """Normalize AD/cloud password policy into DomainArtifact metadata."""
+ policy = raw.get("password_policy", {})
+ if not isinstance(policy, dict) or not policy:
+ return 0
+ domain = raw.get("domain", "") or raw.get("dc", "") or "default"
+ existing_domain = None
+ for d in store.get(ArtifactType.DOMAIN):
+ if isinstance(d, DomainArtifact) and d.domain_name.lower() == domain.lower():
+ existing_domain = d
+ break
+ if existing_domain:
+ existing_domain.metadata["password_policy"] = policy
+ return 1
+ d_artifact = DomainArtifact(
+ domain_name=domain,
+ metadata={"password_policy": policy},
+ source_module=capability,
+ )
+ store.add(d_artifact)
+ return 1
+
+ def _normalize_scheduled_tasks(
+ self, raw: dict, store: ArtifactStore, capability: str = "scheduled_tasks"
+ ) -> int:
+ """Normalize scheduled tasks into PermissionArtifact and HostArtifact metadata."""
+ count = 0
+ target = raw.get("target") or raw.get("host", "")
+ seen_tasks: set[str] = set()
+ for key in ("writable_path_tasks", "high_priv_tasks", "all_tasks"):
+ tasks = raw.get(key, [])
+ if isinstance(tasks, list):
+ for t in tasks:
+ if isinstance(t, dict):
+ task_name = t.get("task") or t.get("name", "")
+ if task_name and task_name not in seen_tasks:
+ seen_tasks.add(task_name)
+ run_as = t.get("run_as", "SYSTEM")
+ artifact = PermissionArtifact(
+ principal=run_as,
+ target=task_name,
+ right="ExecuteScheduledTask",
+ domain=target,
+ is_dangerous=bool(t.get("writable_indicator") or key == "writable_path_tasks"),
+ source_module=capability,
+ )
+ store.add(artifact)
+ count += 1
+ if target:
+ for h in store.hosts():
+ if h.ip_address == target or h.hostname == target:
+ h.metadata["scheduled_tasks_count"] = raw.get("total_tasks", count)
+ break
+ return count
+
+ def _normalize_registry_credentials(
+ self, raw: dict, store: ArtifactStore, capability: str = "credential_hints"
+ ) -> int:
+ """Normalize registry credential hits (AutoLogon, VNC, PuTTY) into CredentialArtifact."""
+ count = 0
+ target = raw.get("target") or raw.get("host", "")
+ # 1. Credential hits (AutoLogon, VNC, etc.)
+ hits = raw.get("credential_hits", [])
+ if isinstance(hits, list):
+ for h in hits:
+ if not isinstance(h, dict):
+ continue
+ vals = h.get("values", {})
+ if not isinstance(vals, dict):
+ continue
+ path = h.get("path", "")
+ if "Winlogon" in path:
+ user = vals.get("DefaultUserName", "")
+ pwd = vals.get("DefaultPassword", "")
+ dom = vals.get("DefaultDomainName", "") or target
+ if pwd and isinstance(pwd, str):
+ artifact = CredentialArtifact(
+ username=user.strip("\x00"),
+ domain=dom.strip("\x00"),
+ cred_type="cleartext",
+ secret=pwd.strip("\x00"),
+ source_host=target,
+ privilege="local_admin",
+ note="registry_autologon",
+ source_module=capability,
+ )
+ store.add_credential(artifact)
+ count += 1
+ elif "VNC" in path:
+ pwd = vals.get("Password", "")
+ if pwd:
+ artifact = CredentialArtifact(
+ username="vnc",
+ domain=target,
+ cred_type="vnc_password",
+ secret=str(pwd),
+ source_host=target,
+ privilege="user",
+ note=f"registry_{path}",
+ source_module=capability,
+ )
+ store.add_credential(artifact)
+ count += 1
+ # 2. PuTTY sessions
+ putty = raw.get("putty_sessions", [])
+ if isinstance(putty, list):
+ for s in putty:
+ if isinstance(s, dict):
+ u = s.get("user_name", "")
+ h_name = s.get("host_name", "")
+ s_name = s.get("session_name", "")
+ if u or h_name:
+ artifact = CredentialArtifact(
+ username=u or "unknown",
+ domain=h_name,
+ cred_type="putty_session",
+ source_host=target,
+ note=f"session={s_name} remote={h_name}",
+ source_module=capability,
+ )
+ store.add_credential(artifact)
+ count += 1
+ return count
+
+ def _normalize_secrets_scan(
+ self, raw: dict, store: ArtifactStore, capability: str = "discovered_secrets"
+ ) -> int:
+ """Normalize secrets scanner findings into CredentialArtifact."""
+ count = 0
+ target = raw.get("target") or raw.get("host", "")
+ secrets = raw.get("discovered_secrets") or []
+ if isinstance(secrets, list):
+ for s in secrets:
+ if not isinstance(s, dict):
+ continue
+ sec_val = s.get("secret_value") or s.get("extracted_secret") or ""
+ pattern = s.get("pattern", "secret")
+ fpath = s.get("file", "")
+ line = s.get("line", 0)
+ if sec_val:
+ artifact = CredentialArtifact(
+ username=pattern,
+ secret=sec_val,
+ cred_type=pattern,
+ source_host=target,
+ privilege="unknown",
+ note=f"{fpath}:{line}",
+ source_module=capability,
+ )
+ store.add_credential(artifact)
+ count += 1
+ cred_list = raw.get("credential_list", [])
+ if target and (cred_list or raw.get("sensitive_data_found")):
+ for h in store.hosts():
+ if h.ip_address == target or h.hostname == target:
+ if cred_list:
+ h.metadata["secret_files"] = cred_list
+ if raw.get("sensitive_data_found"):
+ h.metadata["sensitive_data_found"] = True
+ break
+ return count
+
+ def _normalize_smb_shares(
+ self, raw: dict, store: ArtifactStore, capability: str = "file_share_list"
+ ) -> int:
+ """Normalize SMB shares and sensitive file paths into HostArtifact metadata."""
+ target = raw.get("target") or raw.get("host", "")
+ if not target:
+ return 0
+ share_list = raw.get("file_share_list", [])
+ file_paths = raw.get("sensitive_file_paths", [])
+ existing = None
+ for h in store.hosts():
+ if h.ip_address == target or h.hostname == target:
+ existing = h
+ break
+ if existing:
+ if share_list:
+ existing.metadata["file_shares"] = share_list
+ if file_paths:
+ existing.metadata["sensitive_files"] = file_paths
+ return 1
+ h = HostArtifact(
+ ip_address=target,
+ hostname=target,
+ metadata={
+ "file_shares": share_list,
+ "sensitive_files": file_paths,
+ },
+ source_module=capability,
+ )
+ store.add(h)
+ return 1
+
+
diff --git a/tests/unit/test_artifact_normalizer_pipeline.py b/tests/unit/test_artifact_normalizer_pipeline.py
index 1863d8b..3146251 100644
--- a/tests/unit/test_artifact_normalizer_pipeline.py
+++ b/tests/unit/test_artifact_normalizer_pipeline.py
@@ -13,9 +13,12 @@ from ares.normalize.artifacts import (
ArtifactNormalizer,
ArtifactStore,
ArtifactType,
+ CloudResourceArtifact,
CredentialArtifact,
+ DomainArtifact,
HashArtifact,
HostArtifact,
+ PermissionArtifact,
UserArtifact,
)
@@ -935,5 +938,367 @@ def test_service_versions_and_web_fingerprint_pipeline_mod_062(
assert store.hosts()[0].os == "Ubuntu Linux"
+def test_azure_findings_pipeline_mod_066(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """MOD-066: Azure storage, RBAC, and NSG findings normalized to CloudResourceArtifact and PermissionArtifact."""
+ raw = {
+ "subscription_id": "sub-1234-abcd",
+ "storage": {
+ "public_containers": [
+ {
+ "storage_account": "corpbackup",
+ "container": "db-dumps",
+ "access_level": "container",
+ "resource_group": "rg-core",
+ }
+ ]
+ },
+ "rbac": {
+ "owner_assignments": [
+ {"principal_id": "user-guid-001", "role_definition": "8e3af657", "scope": "/subscriptions/sub-1234-abcd"}
+ ],
+ "contributor_assignments": [
+ {"principal_id": "spn-guid-002", "role_definition": "b24988ac", "scope": "/subscriptions/sub-1234-abcd"}
+ ],
+ "classic_admins": ["[email protected]"],
+ },
+ "nsg": {
+ "open_rules": [
+ {"nsg": "nsg-prod", "rule_name": "Allow-SSH-Internet", "resource_group": "rg-core"}
+ ]
+ },
+ "aad_users": {
+ "admin_users": ["[email protected]"]
+ }
+ }
+ added = normalizer.normalize("cloud.azure", ["azure_findings"], raw, store)
+ assert added >= 5
+
+ # Cloud resources: storage container + NSG
+ cloud_res = [a for a in store.get(ArtifactType.CLOUD_RESOURCE)]
+ res_ids = {c.resource_id for c in cloud_res}
+ assert "corpbackup/db-dumps" in res_ids
+ assert "nsg-prod/Allow-SSH-Internet" in res_ids
+
+ # Permissions: owner, contributor, classic admin
+ perms = store.permissions()
+ principals = {p.principal for p in perms}
+ assert "user-guid-001" in principals
+ assert "spn-guid-002" in principals
+ assert "[email protected]" in principals
+
+ # Users
+ users = store.users()
+ assert any(u.username == "[email protected]" for u in users)
+
+
+def test_azure_ad_findings_and_access_tokens_pipeline_mod_066(
+ normalizer: ArtifactNormalizer, store: ArtifactStore
+):
+ """MOD-066: Azure AD users, SPNs, and access tokens normalized into artifacts."""
+ raw = {
+ "tenant_id": "tenant-azure-ad-01",
+ "privileged_users": ["[email protected]"],
+ "guests": [{"userPrincipalName": "[email protected]"}],
+ "high_priv_sps": ["CI/CD Pipeline App"],
+ "access_tokens": "eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.dummy_access_token_payload",
+ }
+ added = normalizer.normalize("cloud.azure_ad", ["azure_ad_findings", "access_tokens"], raw, store)
+ assert added >= 4
+
+ users = {u.username: u for u in store.users()}
+ assert "[email protected]" in users
+ assert users["[email protected]"].is_admin is True
+ assert "[email protected]" in users
+ assert users["[email protected]"].is_admin is False
+
+ perms = store.permissions()
+ assert any(p.principal == "CI/CD Pipeline App" for p in perms)
+
+ creds = store.credentials()
+ oauth_creds = [c for c in creds if c.cred_type == "oauth_token"]
+ assert len(oauth_creds) == 1
+ assert "dummy_access_token_payload" in oauth_creds[0].secret
+
+
+def test_gcp_findings_pipeline_mod_068(normalizer: ArtifactNormalizer, store: ArtifactStore):
+ """MOD-068: GCP public buckets, IAM bindings, and SA keys normalized into artifacts."""
+ raw = {
+ "project_id": "ares-gcp-sec-lab",
+ "gcs": {
+ "public_buckets": [
+ {"bucket": "ares-public-assets", "member": "allUsers", "role": "roles/storage.objectViewer"}
+ ]
+ },
+ "iam": {
+ "public_bindings": ["allUsers → roles/viewer"],
+ "owner_bindings": ["[email protected]"],
+ },
+ "service_accounts": {
+ "many_keys": [
+ {"service_account": "[email protected]", "active_key_count": 3}
+ ]
+ }
+ }
+ added = normalizer.normalize("cloud.gcp", ["gcp_findings"], raw, store)
+ assert added >= 4
+
+ cloud_res = [c for c in store.get(ArtifactType.CLOUD_RESOURCE) if c.provider == "gcp"]
+ assert len(cloud_res) == 1
+ assert cloud_res[0].resource_id == "ares-public-assets"
+ assert cloud_res[0].is_public is True
+
+ perms = store.permissions()
+ principals = {p.principal for p in perms}
+ assert "allUsers" in principals
+ assert "[email protected]" in principals
+
+ creds = store.credentials()
+ sa_creds = [c for c in creds if c.cred_type == "service_account_key"]
+ assert len(sa_creds) == 1
+ assert sa_creds[0].username == "[email protected]"
+
+
+def test_container_escape_and_k8s_rbac_pipeline_mod_040(
+ normalizer: ArtifactNormalizer, store: ArtifactStore
+):
+ """MOD-040: Container escape vectors and K8s RBAC findings normalized into HostArtifact and PermissionArtifact."""
+ raw = {
+ "target": "k8s-worker-01",
+ "docker_socket": {"exists": True, "writable": True},
+ "privileged": {"privileged": True, "CapEff": "0000003fffffffff"},
+ "host_mounts": {"high": ["/etc", "/var/lib/docker"]},
+ "k8s_rbac_findings": [
+ {
+ "service_account": "system:serviceaccount:default:tiller",
+ "resource": "cluster",
+ "verb": "cluster-admin",
+ }
+ ],
+ }
+ added = normalizer.normalize("linux.container", ["container_escape_vectors", "k8s_rbac_findings"], raw, store)
+ assert added >= 2
+
+ hosts = store.hosts()
+ assert len(hosts) == 1
+ assert hosts[0].ip_address == "k8s-worker-01"
+ assert len(hosts[0].vulns) >= 3
+ assert any(v.get("vector") == "docker_socket_writable" for v in hosts[0].vulns)
+
+ perms = store.permissions()
+ assert len(perms) == 1
+ assert perms[0].principal == "system:serviceaccount:default:tiller"
+ assert perms[0].right == "cluster-admin"
+
+
+def test_identity_federation_pipeline_mod_051(
+ normalizer: ArtifactNormalizer, store: ArtifactStore
+):
+ """MOD-051: Federation trusts, Golden SAML paths, and pivot paths normalized into artifacts."""
+ raw = {
+ "tenant_id": "tenant-fed-01",
+ "federation_trusts": [
+ "Azure federated domain: corp.com",
+ "AWS SAML provider: arn:aws:iam::123456789012:saml-provider/ADFS",
+ "ADFS relying party: urn:federation:MicrosoftOnline",
+ ],
+ "golden_saml_paths": [
+ {
+ "attack": "Golden SAML",
+ "technique": "T1606.002",
+ "relying_parties": ["urn:federation:MicrosoftOnline", "urn:amazon:webservices"],
+ }
+ ],
+ "pivot_paths": [
+ {
+ "path": "Azure AD → AWS via SAML Federation",
+ "technique": "T1606.002",
+ "viable": True,
+ }
+ ],
+ "oauth_tokens": [
+ {"access_token": "token_forged_saml_b64", "user": "[email protected]"}
+ ],
+ }
+ added = normalizer.normalize(
+ "cloud.identity_federation_abuse",
+ ["federation_trusts", "golden_saml_paths", "pivot_paths", "oauth_tokens"],
+ raw,
+ store,
+ )
+ assert added >= 7
+
+ cloud_res = [c for c in store.get(ArtifactType.CLOUD_RESOURCE)]
+ assert any(c.provider == "azure" and c.resource_id == "corp.com" for c in cloud_res)
+ assert any(c.provider == "aws" for c in cloud_res)
+
+ perms = store.permissions()
+ saml_perms = [p for p in perms if "GoldenSAML" in p.right]
+ assert len(saml_perms) == 2
+
+ creds = store.credentials()
+ assert any(c.secret == "token_forged_saml_b64" for c in creds)
+
+
+def test_snmp_findings_and_system_info_pipeline(
+ normalizer: ArtifactNormalizer, store: ArtifactStore
+):
+ """network.snmp_enum findings and system_info populate CredentialArtifact and HostArtifact."""
+ raw = {
+ "target": "192.168.1.100",
+ "port": 161,
+ "valid_communities": [
+ {"community": "public", "system_info": {"sysDescr": "Linux Ubuntu 22.04"}},
+ {"community": "private", "system_info": {"sysDescr": "Linux Ubuntu 22.04"}},
+ ],
+ "system_info": {
+ "sysDescr": "Linux RouterOS 7.2",
+ "sysName": "edge-gw01",
+ "sysLocation": "Datacenter Rack 4",
+ }
+ }
+ added = normalizer.normalize("network.snmp_enum", ["snmp_findings", "system_info"], raw, store)
+ assert added >= 3
+
+ creds = store.credentials()
+ communities = {c.secret for c in creds if c.cred_type == "snmp_community"}
+ assert "public" in communities
+ assert "private" in communities
+
+ hosts = store.hosts()
+ assert len(hosts) == 1
+ assert hosts[0].ip_address == "192.168.1.100"
+ assert hosts[0].hostname == "edge-gw01"
+ assert "snmp_system_info" in hosts[0].metadata
+
+
+def test_password_policy_pipeline_mod_056(
+ normalizer: ArtifactNormalizer, store: ArtifactStore
+):
+ """ad.enum_users password_policy is stored in DomainArtifact metadata."""
+ raw = {
+ "domain": "corp.local",
+ "password_policy": {
+ "minPwdLength": 8,
+ "pwdHistoryLength": 24,
+ "lockoutThreshold": 0,
+ }
+ }
+ added = normalizer.normalize("ad.enum_users", ["password_policy"], raw, store)
+ assert added == 1
+
+ domains = [d for d in store.get(ArtifactType.DOMAIN) if isinstance(d, DomainArtifact)]
+ assert len(domains) == 1
+ assert domains[0].domain_name == "corp.local"
+ assert domains[0].metadata["password_policy"]["minPwdLength"] == 8
+ assert domains[0].metadata["password_policy"]["lockoutThreshold"] == 0
+
+
+def test_scheduled_tasks_and_registry_credentials_pipeline_mod_070(
+ normalizer: ArtifactNormalizer, store: ArtifactStore
+):
+ """MOD-070: Scheduled tasks and registry credential hits normalized into typed artifacts."""
+ raw_tasks = {
+ "target": "win-srv-01",
+ "writable_path_tasks": [
+ {
+ "task": "DailyBackupJob",
+ "run_as": "NT AUTHORITY\\SYSTEM",
+ "action": "C:\\Temp\\backup.bat",
+ "writable_indicator": "C:\\Temp\\",
+ }
+ ],
+ "high_priv_tasks": [
+ {"task": "MaintenanceTask", "run_as": "BUILTIN\\Administrators"}
+ ],
+ "total_tasks": 2,
+ }
+ added_tasks = normalizer.normalize("windows.scheduled_tasks_enum", ["scheduled_tasks"], raw_tasks, store)
+ assert added_tasks >= 2
+
+ perms = store.permissions()
+ task_perms = [p for p in perms if p.right == "ExecuteScheduledTask"]
+ assert len(task_perms) >= 2
+ assert any(p.target == "DailyBackupJob" and p.is_dangerous is True for p in task_perms)
+
+ raw_reg = {
+ "target": "win-srv-01",
+ "credential_hits": [
+ {
+ "path": "SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion\\Winlogon",
+ "values": {
+ "AutoAdminLogon": "1",
+ "DefaultUserName": "kiosk_user",
+ "DefaultPassword": "SuperSecretAutoPassword123!",
+ "DefaultDomainName": "CORP",
+ },
+ },
+ {
+ "path": "SOFTWARE\\RealVNC\\vncserver",
+ "values": {"Password": "VncPasswordSecret"},
+ }
+ ],
+ "putty_sessions": [
+ {"session_name": "prod-bastion", "host_name": "10.10.10.5", "user_name": "opsadmin"}
+ ]
+ }
+ added_reg = normalizer.normalize("windows.registry_enum", ["credential_hints"], raw_reg, store)
+ assert added_reg >= 3
+
+ creds = store.credentials()
+ assert any(c.username == "kiosk_user" and c.secret == "SuperSecretAutoPassword123!" for c in creds)
+ assert any(c.username == "vnc" and c.secret == "VncPasswordSecret" for c in creds)
+ assert any(c.username == "opsadmin" and c.cred_type == "putty_session" for c in creds)
+
+
+def test_secrets_scan_and_smb_shares_pipeline_mod_074(
+ normalizer: ArtifactNormalizer, store: ArtifactStore
+):
+ """MOD-074: Discovered secrets and sensitive SMB share files normalized into artifacts."""
+ raw_secrets = {
+ "target": "appserver01",
+ "discovered_secrets": [
+ {
+ "file": "C:\\inetpub\\wwwroot\\web.config",
+ "line": 42,
+ "pattern": "connection_string",
+ "extracted_secret": "Server=db01;Password=SecretDBPass!",
+ "secret_value": "SecretDBPass!",
+ },
+ {
+ "file": "/home/deploy/.aws/credentials",
+ "line": 3,
+ "pattern": "aws_access_key",
+ "extracted_secret": "AKIAIOSFODNN7EXAMPLE",
+ "secret_value": "AKIAIOSFODNN7EXAMPLE",
+ }
+ ],
+ "credential_list": ["C:\\inetpub\\wwwroot\\web.config", "/home/deploy/.aws/credentials"],
+ "sensitive_data_found": True,
+ }
+ added_sec = normalizer.normalize("exfil.secrets_scan", ["discovered_secrets", "credential_list"], raw_secrets, store)
+ assert added_sec >= 2
+
+ creds = store.credentials()
+ assert any(c.cred_type == "connection_string" and c.secret == "SecretDBPass!" for c in creds)
+ assert any(c.cred_type == "aws_access_key" and c.secret == "AKIAIOSFODNN7EXAMPLE" for c in creds)
+
+ raw_shares = {
+ "target": "fileserver02",
+ "file_share_list": ["backups", "finance", "C$"],
+ "sensitive_file_paths": [
+ "\\\\fileserver02\\backups\\sam.save",
+ "\\\\fileserver02\\finance\\budget_2026.xlsx",
+ ],
+ "sensitive_data_found": True,
+ }
+ added_shares = normalizer.normalize("exfil.smb_shares", ["file_share_list", "sensitive_file_paths"], raw_shares, store)
+ assert added_shares >= 1
+
+ hosts = {h.ip_address: h for h in store.hosts()}
+ assert "fileserver02" in hosts
+ assert "backups" in hosts["fileserver02"].metadata["file_shares"]
+ assert "\\\\fileserver02\\backups\\sam.save" in hosts["fileserver02"].metadata["sensitive_files"]
+
+