A company signs up for your SaaS product, and all you have is its website. To qualify the account or personalize onboarding, you also need its industry, employee range, headquarters, and contact information. Some of those details are on the company's website. Others aren't.
You can collect the missing information from other sources, but there's a second problem: how do you know the records belong to the same company, and which value should you use when they disagree?
In this guide, we'll build a company enrichment pipeline with Spidra that collects data from company websites and structured company profiles, checks identities before combining records, and tracks where each value came from. The result is a profile your application can inspect instead of a collection of fields it has to trust blindly.
If you're starting from a company name rather than a domain, our earlier company enrichment API tutorial covers website discovery and name matching. Here, we'll start with known domains and focus on combining sources safely.
What a single website can tell you
Consider five developer-focused B2B software companies: PostHog, Cal.com, Resend, Attio, and Plausible. For each one, the November 11, 2026 research collected nine fields: name, description, industry, products or services, target customers, employee range, headquarters, founding year, and public email addresses.
The homepage extraction filled an average of six fields per company. Adding a website contact pass and a structured company-profile source increased the average to 8.4.
Company | Homepage only | Combined profile |
|---|---|---|
PostHog | 4/9 | 7/9 |
Cal.com | 7/9 | 9/9 |
Resend | 6/9 | 9/9 |
Attio | 5/9 | 8/9 |
Plausible | 8/9 | 9/9 |
These results describe this small group, not a general accuracy or coverage rate for company enrichment. A filled field isn't necessarily a correct one, either. The rest of the pipeline is about making that distinction.
Decide which fields you need first
Start with a fixed profile rather than collecting every detail a website might contain. For a signup enrichment or account research workflow, the nine fields above are enough to illustrate the process.
They don't all have the same best source. A company's own website is useful for its description, product, and audience. Its contact pages are better places to look for published email addresses. A structured company profile may provide employee ranges, founding dates, and headquarters that aren't mentioned on a homepage.
Keep social links outside the core completeness score. In this workflow, their main job is to help establish identity. GitHub and LinkedIn links can help tie an extracted website record to an external company profile, but neither is proof by itself.
For each field, the final record should store its value, its source, and whether another source disagreed. Missing data should remain missing.
Extract company details from the homepages
We'll use Python and the Spidra SDK. Install the package and set your API key in the SPIDRA_API_KEY environment variable.
pip install spidraStart with the five company websites and a client:
# enrich_companies.py
import os
from spidra import Spidra
client = Spidra(api_key=os.environ["SPIDRA_API_KEY"])
domains = ["posthog.com", "cal.com", "resend.com", "attio.com", "plausible.io"]
urls = [f"https://{domain}" for domain in domains]Define the fields before scraping. Nullable values help avoid forcing a guessed answer into a field that isn't on the page.
text_fields = ("name", "description", "industry", "targetCustomers", "headquarters", "employeeRange")
properties = {key: {"type": ["string", "null"]} for key in text_fields}
properties["productsOrServices"] = {"type": "array", "items": {"type": "string"}}
properties["foundedYear"] = {"type": ["integer", "null"]}
properties["emails"] = {"type": "array", "items": {"type": "string"}}
properties["socialLinks"] = {"type": "object", "properties": {
"linkedin": {"type": ["string", "null"]},
"github": {"type": ["string", "null"]},
}}
schema = {"type": "object", "properties": properties, "required": list(properties)}Batch scrape applies the same extraction instructions to each URL and returns a separate result for each page.
batch = client.batch_scrape(
urls,
prompt=("Extract only company details explicitly stated on this page. "
"Return null or an empty list when a field is missing. Do not infer facts."),
schema=schema,
)
website_rows = {
item.url: item.result for item in batch.items
if item.status == "completed" and isinstance(item.result, dict)
}Don't assume that an empty founding year means the request failed. The page may have loaded correctly without publishing that information. Separately inspect failed batch items and retry them when appropriate. That distinction matters when you calculate profile completeness.
In the five-company set, PostHog's homepage filled only four of the nine fields. Plausible's filled eight. A homepage's purpose is often to explain the product, not provide every detail needed by a CRM.
Look for contact details on other pages
A homepage isn't always the best place to find a company's email address or social links. Rather than guessing paths such as /contact and /about, Spidra has a Contacts from company websites template that checks a supplied homepage and its linked contact and about pages when available.
contacts_run = client.run_template(
"company-contacts-from-websites",
{"urls": urls},
)
contact_rows = contacts_run.rowsThe template returns information published on the checked pages, including emails, phone numbers, addresses, and social links. It doesn't verify whether an address is deliverable or belongs to a sales contact. See the Templates documentation for how runs and their results work.
That difference showed up clearly with Resend. The contact extraction found ten email strings, but eight were examples or demo addresses appearing in documentation, not useful company contacts. A technically successful extraction could have sent those strings into a CRM.
Treat discovered emails as candidates. Remove obvious documentation and placeholder patterns, retain the page where an address appeared, and don't use an address for outreach until it has passed any required checks. A syntactically valid address isn't necessarily a genuine business contact.
There was another useful limit. PostHog and Attio did not publish an email on the pages this template checked. None of the five companies returned a public phone number from the checked pages. Those are gaps in the available source material, not reasons to invent a contact.
Match an external company profile to the right business
Company-profile sources can fill fields that are missing from a website. But matching by company name or a guessed URL slug is risky. Two unrelated companies can have similar names, and a short slug can belong to someone else.
The safer starting point is the LinkedIn company link published on the company's own website. You can extract the company slug from that link without guessing it.
from urllib.parse import urlparse
def linkedin_slug(url):
parsed = urlparse(url)
if parsed.hostname not in {"linkedin.com", "www.linkedin.com"}:
return None
parts = [part for part in parsed.path.split("/") if part]
return parts[1] if len(parts) >= 2 and parts[0] == "company" else NoneSpidra's LinkedIn company profiles template accepts company slugs or profile links. For sources you're authorized to use, the Python call follows this pattern:
profiles_run = client.run_template(
"linkedin-company-profiles",
{"companies": ["posthog", "cal-com", "resend", "attio", "plausible-analytics"]},
)
profile_rows = profiles_run.rowsThese slugs were obtained from the companies' published links in the supplied research. In your own pipeline, build the input list from the verified links in contact_rows; don't hardcode guessed names. Check the terms and permitted uses of any third-party company-profile source before collecting or reusing its data.
Now verify the returned row before merging it. Use at least two checks, such as the company's published profile link and a matching website domain, or a published link and a consistent company name. If the signals disagree, quarantine the row and return an identity error rather than borrowing another company's employee count or headquarters.
For this five-company set, each company's own site supplied a LinkedIn link and the returned company profiles passed the matching checks in the research. An earlier run against a different set had exposed the danger: the slug linear pointed to an unrelated engineering company. A merge based on the slug alone would have quietly attached the wrong details.
One more distinction: the number of people listing a company on LinkedIn is not the same as that company's employee-size band. In the research, PostHog's size band was 51–200 while its employee-profile count was 234. Store those as different signals, or leave the profile count out if it isn't needed.
Merge fields without hiding conflicts
Once each source has been matched to the correct company, decide which value takes priority for each field.
Use the company's website for its own product description and target customers. Prefer a structured company profile for employee range, founding year, and headquarters when those values are available and have passed identity checks. Use the contact results for published email addresses. Industry can fall back to the structured profile if the homepage doesn't state it.
The merge should also retain disagreements. Here's a small function for selecting a preferred value while keeping alternatives visible:
def choose(candidates, preference):
available = [(source, candidates[source]) for source in preference
if candidates.get(source) not in (None, "", [])]
if not available:
return {"value": None, "source": None, "conflicts": []}
source, value = available[0]
conflicts = [{"source": other, "value": other_value}
for other, other_value in available[1:] if other_value != value]
return {"value": value, "source": source, "conflicts": conflicts}Here's why this matters. The homepage extraction returned 2026 as Cal.com's founding year, while the matched company-profile source reported 2021. The website value appears to have been mistaken for a founding date. With a source preference for this field, the merge can select 2021 and still report the disagreement:
founded = choose({"website": 2026, "company_profile": 2021},
["company_profile", "website"])The resulting value is 2021, sourced from company_profile, with 2026 retained under conflicts. The rule avoids silently overwriting evidence, but it doesn't independently prove that the preferred source is correct. A high-stakes use case may still require another source or manual review.
A different error happened with PostHog. Its homepage didn't supply an industry, but the matched external row did. The initial merge implementation copied headquarters, size, and founding year but forgot to fall back to the external industry field. The final profile therefore had a gap even though the data had been collected. Test your merge policy for missing-field fallbacks as well as conflicts.
Check the finished profile on a new company
It's easy to build rules that happen to work for the companies used during development. A useful next check is a domain that hasn't influenced those rules.
The supplied research used mintlify.com for that purpose. Its resulting record included these fields:
{
"domain": "mintlify.com",
"name": "Mintlify",
"employeeRange": "51-200 employees",
"headquarters": "San Francisco, California",
"foundedYear": 2022,
"completeness": 0.9
}This is an excerpt from the resulting profile, not the entire response. Mintlify had eight of the nine core fields. The missing field was a public email address, which wasn't published on the pages checked. The research also reported no identity errors or recorded conflicts for this run.
A completeness score measures how much of your chosen schema was filled. It does not measure accuracy. Keep that score separate from identity verification and from a field's source information.
Run enrichment as a background job
The next decision is when to collect and refresh the data.
Enriching every company from scratch during an HTTP request makes your application's response depend on several external pages. Some pages will load slowly or fail, even if the rest succeed. A more reliable design is to queue enrichment when a new domain appears and serve an existing profile while a refresh is running.
For example, an application could accept POST /enrich with a domain and return a job ID and 202 Accepted. A worker would run the homepage, contact, and authorized company-profile passes, verify the matches, and merge the results. GET /enrich/{domain} could then return the latest stored profile with its last-updated time and source details.
The API routes above describe an application design, not built-in Spidra endpoints. Your own service still needs authentication, request limits, a job queue, storage, retries, and error handling.
Refresh frequency should follow how the data is used. A company description may change after a repositioning, while employee ranges and contact details can become stale for different reasons. Cache by a normalized domain, record when each source was checked, and refresh fields that matter to the workflow rather than scraping every page on every request.
If you're enriching domains supplied by users, validate them before making outbound requests. A hostname syntax check isn't sufficient protection against malicious DNS, redirects, or internal-network access.
Start with the records your product needs
A useful enrichment pipeline doesn't need the largest possible company database. It needs a defined set of fields, a way to identify the right business, and a record of which source supplied each value.
Start with a few domains already in your product. Use Spidra to collect the public website fields, add contacts and authorized company-profile data where they help, and check how the merged profiles compare with the homepage alone. The Spidra data enrichment workflow is a starting point for bringing those records into your application.