Custom scoring model built on closed-won data: how it works and how to validate it

Most scoring models fail because they're built on someone else's data. A custom scoring model built from your closed-won data analyzes the accounts and contacts you've actually closed, identifies the patterns that predict success in your specific business, and weights those signals accordingly.

Why generic scoring fails (and what custom scoring fixes)

Generic scoring models use industry-standard weights: company size gets 20 points, tech stack match gets 15, intent signal gets 10. The problem is those weights came from aggregated data across thousands of companies in dozens of industries. They reflect what works on average, which means they don't reflect what works for you.

Custom scoring fixes this by analyzing your closed-won deals and reverse-engineering the signals that actually predicted those wins. UserGems builds this custom model automatically by ingesting your CRM data, analyzing your closed-won patterns, and creating a scoring algorithm trained specifically on what converts in your business.

How to build a scoring model from closed-won deals

Step 1: pull your closed-won deal history

Start with your CRM. Export every closed-won opportunity from the last 12-24 months. You need account firmographics, contact data, deal characteristics, and engagement history. UserGems automates this step by connecting directly to your CRM.

Step 2: layer in signal data

Closed-won data tells you who converted. Signal data tells you what was happening in their world when they converted. Add job changes, tech stack shifts, intent activity, company events, and first-party engagement. Map these signals back to your closed-won deals.

Step 3: weight signals based on conversion correlation

For each signal, calculate how often it appeared in closed-won deals versus closed-lost or no-decision deals. Signals that appear significantly more often in wins get higher weights.

• 85% of closed-won deals had a job change signal - weight: high
• 60% of closed-won deals were in the 500-2,000 employee range - weight: medium
• 30% of closed-won deals showed intent activity - weight: low

UserGems builds this correlation model automatically using AI trained on your closed-won data.

Step 4: make the model transparent and editable

Most AI scoring models are black boxes. Transparent scoring means you can see exactly which signals contributed to a score and how much weight each one carried. Editable scoring means you can override the model when you know something it doesn't.

What data you need in Salesforce or HubSpot to train scoring

Required fields:

• Opportunity stage history (when deals moved from stage to stage)
• Close date and close reason (won, lost, no decision)
• Deal size and product/service purchased
• Account firmographics (industry, employee count, revenue, location)
• Contact roles (who was involved, what titles, who was the champion)

Recommended fields:

• Lead source and campaign attribution
• Engagement history (emails opened, meetings attended, content consumed)
• Competitor mentioned
• Sales cycle length

If your CRM data is messy or incomplete, start by cleaning up the last 12 months of closed-won deals. That's enough to train a baseline model.

How to know if your scoring model is working

Metric 1: conversion rate by score tier

Segment your prospects into score tiers (90-100, 80-89, 70-79, etc.) and track conversion rate for each tier. If your model is working, higher-scored prospects should convert at a meaningfully higher rate than lower-scored ones.

Metric 2: speed-to-lead and speed-to-opportunity

High-scoring prospects should move through your funnel faster than low-scoring ones. Track days from first touch to meeting booked, days from meeting to opportunity created, and days from opportunity to close.

Metric 3: meeting show rate and reply rate

Scoring should predict engagement, not just conversion. Track reply rate by score tier, meeting show rate by score tier, and sequence completion rate.

Metric 4: win rate by score tier

The ultimate test: do high-scoring prospects close at a higher rate than low-scoring ones? Track closed-won rate by score tier over a 90-day window.

How often should scoring weights change?

• Monthly reviews: check conversion rate by score tier, speed-to-lead, and reply rate.
• Quarterly updates: analyze your last 90 days of closed-won deals and compare signal patterns to the previous quarter.
• Event-driven changes: if your product changes, your ICP shifts, or you enter a new market, revisit your model immediately.

UserGems handles this automatically. The model continuously ingests new closed-won data and adjusts weights based on recent conversion patterns.

How UserGems builds and maintains your custom scoring model

UserGems automates the entire process: data ingestion, pattern analysis, scoring and prioritization, continuous improvement, and transparent reporting. This is what separates UserGems from generic intent platforms. Intent tools give you account-level signals and leave you to figure out what to do with them. UserGems builds a custom scoring model from your data, prioritizes the right contacts, and drafts personalized outreach automatically.

‍