Collecting and verifying a company database for B2B sales

Sales did not need "a list from a directory" but a callable database: no duplicates, current websites, and a clear flag for whether the organisation has a working site at all — which is precisely what separates a warm contact from a cold one.

A naive export by category produces something unusable: the same business appears under several categories, branches duplicate the head record, and a share of the websites stopped responding long ago.

We built a three-stage pipeline. The first stage collects by category and district, respecting the source's rate limits and surviving interruptions: progress is written line by line, so a rerun does not start over.

The second normalises — addresses and phone numbers to a single format, deduplication on a stable key rather than the name, and tagging of branches and multi-category records.

The third checks the websites: parallel requests with timeouts, response classification (working, redirect, unreachable, domain does not resolve), and export to CSV plus an HTML report a manager opens without installing anything.