Recommendation Confidence
What This Chart Shows
This chart places each marketing lever on a grid to help determine the performance and model confidence. It plots each series as a bubble, where the horizontal position shows performance (ROI or Lift), the vertical position shows confidence in the attribution, and bubble size indicates the magnitude of impact. Series in the upper-right are the "sweet spot"—strong results with reliable attribution. This helps you focus optimization and scaling decisions on channels where you can trust the numbers, rather than chasing metrics that may be noisy or unreliable.
Key Questions This Chart Helps Answer
- Which channels have both strong ROI and high confidence in the attribution?
- Where should I scale spend with the most certainty?
- Which high-performing channels have low confidence and may need more investigation?
Axes, Metrics, and Units
Element | Description |
|---|---|
| X-axis | Attributed ROI (or Lift in no-costs mode): Performance metric for each series. |
| Y-axis | Confidence Score: Ranges from 0 to 1. Higher = more confidence in the attribution. |
| Bubble size | Proportional to absolute lift (larger bubbles = larger impact). |
| Bubble color | Governance state: Green = PASS (validated) Yellow = CLAMPED (adjusted) Red = KILLED (suppressed). |
| Shaded zone | "High Confidence Zone": Series in this zone have strong model confidence. |
| Hover | Shows series name, ROI/Lift, Confidence, Lift value, and Governance state. |
Confidence Score Calculation:
- Confidence = (1 - AI Reliance %) × Governance Factor
- AI Reliance % = How much of the series' attribution comes from the AI model residual vs the physics model. Lower AI reliance = higher confidence.
- Governance Factor = 1 if PASS or CLAMPED, 0 if KILLED. KILLED series always have 0 confidence.
All performance values are modeled outputs.
Control Options Reference
Control | Meaning |
|---|---|
| KPI classification / hierarchy / drilldown | Which outcome (KPI / target) to attribute to. |
| Date range | Time window for attribution. |
How to Interpret the Results
- Upper-right quadrant: High ROI (or Lift) AND high confidence. These are your best candidates for scaling—strong performance with reliable attribution.
- Upper-left quadrant: Low ROI but high confidence. The attribution is reliable, but the channel isn't performing well. Consider optimization or reduced investment.
- Lower-right quadrant: High ROI but low confidence. The performance looks good, but the model is less certain. Investigate further before scaling.
- Lower-left quadrant: Low ROI and low confidence. Deprioritize or investigate data quality.
- Bubble color: Green (PASS) = validated attribution. Yellow (CLAMPED) = adjusted by governance. Red (KILLED) = suppressed—treat with caution.
- High Confidence Zone: Series above the 0.7 line have strong physics-driven attribution with minimal AI residual reliance.
Practical Applications for Marketers
Application | How to use this chart |
|---|---|
| Scaling decisions | Prioritize scaling for series in the upper-right (high ROI + high confidence). |
| Risk assessment | Be cautious about scaling series with high ROI but low confidence—the numbers may be unreliable. |
| Optimization targeting | Focus optimization efforts on upper-left series (high confidence but low ROI)—the attribution is reliable, so improvements will be measurable. |
| Data quality review | Investigate series in the lower half—low confidence may indicate data sparsity or unusual patterns. |
| Governance audit | Use bubble colors to identify CLAMPED or KILLED series that warrant investigation. |
Common Mistakes & Misinterpretations
Mistake | Why it is a problem | How to avoid |
|---|---|---|
| Ignoring confidence when making decisions | High ROI with low confidence may be noise or model artifact. Acting on it could lead to poor outcomes. | Always consider both axes—performance AND confidence—when prioritizing. |
| Treating all CLAMPED/KILLED series as bad | Governance flags indicate modeling uncertainty, not necessarily that the channel is ineffective. They may still be valuable with better data. | Investigate flagged series before cutting; consider improving data quality. |
| Assuming confidence = certainty | Confidence is a relative measure of model trust, not a guarantee. Even high-confidence series can have errors. | Use confidence as a prioritization tool, not as proof of accuracy. |
| Comparing bubble sizes across different time periods | Bubble sizes reflect absolute lift, which can change with time windows. Different periods may not be directly comparable. | Keep time periods consistent when comparing; use hover for exact values. |
| Over-indexing on the high confidence zone threshold | The 0.7 threshold is a guideline, not a hard rule. Series at 0.68 are not dramatically less trustworthy than those at 0.72. | Treat the zone as directional guidance, not a binary cutoff. |
Caveats & Considerations
- No-costs mode: When cost data is unavailable, the X-axis shows Lift instead of ROI, and the title reflects this.
- AI residual allocation: Confidence is reduced when a series relies more on the AI model residual (less interpretable) vs the physics model. This is allocated proportionally by lift share.
- Governance impact: KILLED series automatically have 0 confidence (they are suppressed by governance). CLAMPED series retain their calculated confidence but are flagged by color.
- Display limit: Only the top 500 series by absolute lift are shown to prevent browser slowdown. Smaller series may not appear.
- Bubble overlap: In dense charts, bubbles may overlap. Use hover to see exact values; zoom to inspect specific regions.