恶意代码相似度与家族聚类分析

---
name: code-similarity-analysis
description: >
  Compare malware samples to identify code reuse, family relationships, and
  shared authorship. Uses multiple similarity metrics including fuzzy hashing
  (ssdeep, TLSH), import table comparison, string overlap, and function-level
  matching. Supports clustering samples into families and generating similarity
  matrices for large sample sets. Use when triaging new samples against known
  families or investigating campaign connections.
---

# Code Similarity Analysis

Compare malware samples using multiple similarity metrics to identify
family relationships, code reuse, and shared tooling across campaigns.

## Prerequisites

- **Python 3.8+**: `hashlib`, `json`, `os`, `math` (standard library)
- **Python packages (optional)**: `ssdeep`, `tlsh`, `pefile`, `numpy`, `scipy`
- **Tools (optional)**: `ssdeep` CLI, `tlsh` CLI, `radare2`, `Ghidra`
- **Input**: Two or more malware samples (PE, ELF, or raw binaries)

## Step-by-Step Instructions

### Step 1: Compute Fuzzy Hashes

Calculate ssdeep and TLSH fuzzy hashes for each sample. These capture structural
similarity even when bytes change between variants.

**Using the similarity analyzer:**
```bash
python3 scripts/similarity_analyzer.py \
  --samples sample1.exe sample2.exe sample3.exe \
  --metrics ssdeep tlsh \
  --output similarity_report.json
```

**Using ssdeep CLI directly:**
```bash
ssdeep -b sample1.exe sample2.exe sample3.exe
ssdeep -b -d sample1.exe sample2.exe  # Compare two files
```

**Using TLSH:**
```bash
python3 -c "
import tlsh
h1 = tlsh.hash(open('sample1.exe', 'rb').read())
h2 = tlsh.hash(open('sample2.exe', 'rb').read())
print(f'Distance: {tlsh.diff(h1, h2)}')  # Lower = more similar
"
```

**Interpreting results:**
- ssdeep: Score 0-100, higher means more similar. >30 is noteworthy, >70 is strong match
- TLSH: Distance score, lower means more similar. <100 is noteworthy, <30 is strong match

### Step 2: Compare Import Tables

Import table overlap is a strong indicator of shared functionality and tooling.

**Using the similarity analyzer:**
```bash
python3 scripts/similarity_analyzer.py \
  --samples sample1.exe sample2.exe \
  --metrics imports \
  --output import_comparison.json
```

**Manual comparison with pefile:**
```bash
python3 -c "
import pefile
pe1 = pefile.PE('sample1.exe')
pe2 = pefile.PE('sample2.exe')

imports1 = set()
imports2 = set()
for entry in pe1.DIRECTORY_ENTRY_IMPORT:
    for imp in entry.imports:
        if imp.name:
            imports1.add(f'{entry.dll.decode()}.{imp.name.decode()}')

for entry in pe2.DIRECTORY_ENTRY_IMPORT:
    for imp in entry.imports:
        if imp.name:
            imports2.add(f'{entry.dll.decode()}.{imp.name.decode()}')

intersection = imports1 & imports2
union = imports1 | imports2
jaccard = len(intersection) / len(union) if union else 0
print(f'Jaccard similarity: {jaccard:.3f}')
print(f'Shared imports: {len(intersection)}/{len(union)}')
"
```

**Interpretation:**
- Jaccard > 0.8: Very similar import profiles, likely same family or tooling
- Jaccard 0.5-0.8: Significant overlap, possible shared components
- Jaccard 0.3-0.5: Moderate overlap, may share some functionality
- Jaccard < 0.3: Low overlap, likely different codebases

### Step 3: Compare String Artifacts

Extract and compare strings to identify shared configurations, C2 infrastructure,
or development artifacts.

**Using the similarity analyzer:**
```bash
python3 scripts/similarity_analyzer.py \
  --samples sample1.exe sample2.exe \
  --metrics strings \
  --output string_comparison.json
```

**Manual string comparison:**
```bash
strings -n 8 sample1.exe | sort -u > strings1.txt
strings -n 8 sample2.exe | sort -u > strings2.txt
comm -12 strings1.txt strings2.txt  # Shared strings
```

**Key strings to compare:**
- PDB paths (reveal build environment)
- Mutex names (execution markers)
- Registry key paths
- C2 URLs and domains
- Error messages and debug strings
- Encryption keys and configuration markers

### Step 4: Build Similarity Matrix

For multiple samples, compute pairwise similarity across all metrics.

**Generate full similarity matrix:**
```bash
python3 scripts/similarity_analyzer.py \
  --samples-dir ./samples/ \
  --metrics all \
  --output similarity_matrix.json \
  --format matrix
```

**Output as CSV for visualization:**
```bash
python3 scripts/similarity_analyzer.py \
  --samples-dir ./samples/ \
  --metrics all \
  --output matrix.csv \
  --format csv
```

### Step 5: Cluster Samples into Families

Use similarity scores to automatically group samples into families.

**Run clustering:**
```bash
python3 scripts/cluster_samples.py \
  --similarity-matrix similarity_matrix.json \
  --threshold 0.6 \
  --output clusters.json
```

**Specify clustering method:**
```bash
python3 scripts/cluster_samples.py \
  --samples-dir ./samples/ \
  --method hierarchical \
  --linkage average \
  --threshold 0.6 \
  --output clusters.json
```

### Step 6: Investigate Shared Code Blocks

For deeper analysis, compare function-level code patterns.

**Using radare2 for function comparison:**
```bash
r2 -qc "aaa; aflj" sample1.exe > funcs1.json
r2 -qc "aaa; aflj" sample2.exe > funcs2.json
```

**Check for shared unique strings in functions:**
```bash
python3 scripts/similarity_analyzer.py \
  --samples sample1.exe sample2.exe \
  --metrics strings imports ssdeep \
  --verbose \
  --output detailed_comparison.json
```

### Step 7: Document Family Classification

Based on similarity analysis, classify samples and document relationships.

**Classification criteria:**
| Similarity Level | Relationship | Action |
|-----------------|--------------|--------|
| >90% (all metrics) | Same sample or recompilation | Merge into single analysis |
| 70-90% | Same family variant | Group as variant, note differences |
| 50-70% | Related family or shared builder | Investigate shared components |
| 30-50% | Possible connection | Note for further investigation |
| <30% | Likely unrelated | Separate analysis tracks |

## Output Format

```json
{
  "comparison_timestamp": "2025-01-15T10:00:00Z",
  "samples": ["sample1.exe", "sample2.exe"],
  "metrics": {
    "ssdeep": {"score": 85, "hash1": "...", "hash2": "..."},
    "tlsh": {"distance": 42, "hash1": "...", "hash2": "..."},
    "imports": {"jaccard": 0.78, "shared": 45, "total_union": 58},
    "strings": {"jaccard": 0.62, "shared": 128, "total_union": 207}
  },
  "overall_similarity": 0.75,
  "classification": "Same family - high confidence",
  "shared_artifacts": {
    "imports": [],
    "strings": [],
    "mutexes": [],
    "pdb_paths": []
  }
}
```

## Tips

- Use multiple metrics together; no single metric is sufficient alone
- ssdeep requires minimum file size (~4KB) to produce useful hashes
- TLSH requires minimum 50 bytes of data and sufficient complexity
- Import comparison only works for PE files with intact import tables
- Packed samples should be unpacked before comparison for meaningful results
- Consider section-level hashing for more granular comparison
- PDB paths are high-value indicators of shared build environments
- Maintain a reference database of known family hashes for rapid classification