The result: 2,046 queries across 27 countries turned into one clean, queryable dataset. Weeks of manual research were replaced by a pipeline that can be run again whenever the data needs refreshing.
The problem
A market-intelligence client needed structured data on infrastructure projects across 27 countries. Gathering it meant searching country by country, reading results by hand and copying what mattered into spreadsheets. It was slow, inconsistent, and out of date by the time it was finished.
What we built
- A large-scale collection pipeline on SerpApi covering Google Search and AI Overview results.
- Automated CAPTCHA handling so long runs finish without anyone watching them.
- Structured parsing of every result into consistent fields.
- OpenAI batch processing to classify and extract the details that matter, at a fraction of real-time cost.
- Storage in a queryable SQLite database the team can filter and export.
Stack
SerpApi · OpenAI · Python · SQLite
What this means for your business
If your team researches the same kinds of sources over and over (competitors, tenders, listings, regulations), the same pattern applies: define the sources, define what counts as a match, and let a pipeline collect, clean and structure it.
Related: Data pipeline services · AI for accounting firms · All case studies
Client names are kept private. Kortex Labs is founder-led by Saif Ur Rehman, who built this project.