HOST

Slow executor host

One executor host runs its tasks far slower than the rest of the cluster.

Outcome: Speed
Scope: Judged per stage
Usual fix: A Spark configuration change
CI gate: No budget flag: gate it with --fail-on-introduced against a baseline

An example finding

Where you see it

Dashboard

The HOST tag marks it in the action region of the dashboard, among the widgets that rank a fix. Every affected stage is flagged, not only the worst.How to read it in the findings guide.

Agent (MCP)

An assistant can read this entry with the list_detectors tool and its background with get_finding_documentation. The call and the response below come from a real run of the MCP server.

Call

{
  "tool": "list_detectors",
  "arguments": {
    "type": "slowHost"
  }
}

Response

{
  "count": 1,
  "detectors": [
    {
      "type": "slowHost",
      "name": "slow executor host",
      "tag": "HOST",
      "tagSharedWith": [],
      "family": "speed",
      "pitch": "One executor host runs its tasks far slower than the rest of the cluster.",
      "region": "action",
      "scope": "stage",
      "fixEffort": "config",
      "thresholdSummary": "a host 2× slower than its peers by mean task time (per-executor figures from 1.33×, and only when the stage's own task-time spread cannot explain the gap)",
      "detectors": [
        {
          "detector": "slowHost",
          "version": 2,
          "thresholds": {
            "minHosts": 3,
            "minTasks": 15,
            "ratioWarn": 2,
            "minShare": 0.2,
            "shareWarn": 0.75,
            "taskShareWarn": 0.5,
            "ratioTiers": [
              1.33,
              1.78,
              3.16,
              10
            ],
            "floorMs": 1000,
            "floorBytes": 67108864,
            "stageFloorPct": 0.005,
            "outlierAlpha": 0.005
          }
        }
      ],
      "pages": {
        "catalog": "docs/detectors/slowHost.html",
        "guide": "docs/user-guide/understanding-findings.html#host"
      },
      "tuning": {
        "kind": "page",
        "anchor": "#bottleneck-slow-host",
        "path": "docs/tuning-reference/bottleneck-slow-host.html#bottleneck-slow-host"
      },
      "gate": null
    }
  ]
}

get_finding_documentation call

{
  "tool": "get_finding_documentation",
  "arguments": {
    "type": "slowHost"
  }
}

Its response, with each Markdown body replaced by its length

{
  "type": "slowHost",
  "name": "Slow Executor Host",
  "detectionDoc": {
    "tag": "HOST",
    "title": "Slow host",
    "content": "<963 characters of Markdown>"
  },
  "tuningDoc": {
    "anchor": "#bottleneck-slow-host",
    "title": "Slow Host",
    "content": "<6885 characters of Markdown>"
  }
}

CI (CLI)

List this entry from the command line. The output below is a real run of the command. Budgets and exit codes are in the CI guide.

sparkforensics-analyze detectors --type slowHost --format md

# Detectors (1)

## Speed

| Tag | Type | Scope | Fix | Gate | What it finds |
| --- | --- | --- | --- | --- | --- |
| HOST | `slowHost` | stage | config |  | One executor host runs its tasks far slower than the rest of the cluster. |

Report only this finding type for a run with --type:

sparkforensics-analyze <event-log> --type slowHost --impact critical,warning,info

It has no budget flag. Fail a build on a new one by comparing against a baseline with --fail-on-introduced.

Airflow

The operator has no budget parameter for it. Use fail_on_introduced with a baseline.

check_spark_job = SparkForensicsOperator(
    task_id="check_spark_job",
    log_source=...,
    backend=SubprocessAnalyzeHook(),
    baseline_log_source=...,
    fail_on_introduced="critical",
)

The operator works per run, and its summary counts findings per impact band. See the operator for its parameters.

How it is measured

Thresholds

A host 2× slower than its peers by mean task time (per-executor figures from 1.33×, and only when the stage's own task-time spread cannot explain the gap).

slowHost version 2

Threshold	Default
`minHosts`	3
`minTasks`	15
`ratioWarn`	2
`minShare`	0.2
`shareWarn`	0.75
`taskShareWarn`	0.5
`ratioTiers`	[1.33,1.78,3.16,10]
`floorMs`	1000
`floorBytes`	67108864
`stageFloorPct`	0.005
`outlierAlpha`	0.005

Each rule's impact bands and fixed fallbacks are in the threshold tables.

Override a default per run with --thresholds; see Tuning detector thresholds.

Docs

This finding has its own tuning page: Spark tuning reference.
Findings guide: HOST in Understanding findings.