Scrape LinkedIn CEOs By Sector
Automatically scrapes LinkedIn CEO profiles daily by sector and country using Google Custom Search, then logs results to a Google Sheet for easy tracking.
Daily LinkedIn CEO Scraper — By Sector
This workflow automatically scrapes LinkedIn profiles of CEOs within a specified sector and country, using the Google Custom Search API. It runs daily and records key details like CEO name, job title, company, LinkedIn URL, and more into a Google Sheet for easy tracking.
What It Does
- Starts on a daily schedule (7:00 AM IST) to trigger the scraping process.
- Configures search parameters — sector, country, and maximum pages to scrape.
- Loops through multiple pages of Google Custom Search results querying LinkedIn profiles matching the criteria.
- Parses and enriches the search results, extracting CEO details and profile images.
- Appends each CEO's information as a new row in a pre-configured Google Sheet.
What You Need
- Google Cloud Project with:
- An API Key for accessing the Google Custom Search API.
- The Custom Search API enabled.
- Google Custom Search Engine (CSE) configured to search the entire web.
- Google Sheet prepared to receive output data.
The Google Sheet should have these column headers in Row 1:
| Column | Header |
|---|---|
| A | CEO Name |
| B | Job Title |
| C | Company |
| D | Sector |
| E | Country |
| F | LinkedIn URL |
| G | Snippet |
| H | Profile Image |
| I | Scraped Date |
You also need the following variables configured in the workflow:
| Variable | Description |
|---|---|
GOOGLE_API_KEY | Your Google Cloud API key |
GOOGLE_CX | Custom Search Engine ID |
SPREADSHEET_ID | The target Google Sheet ID |
How the Flow Works
| Node | Description |
|---|---|
| Scheduled Trigger | Runs the workflow every day at 7:00 AM IST to start scraping. |
| Scraper Configuration (Code Node) | Sets the sector (e.g., "AI Automation"), country (e.g., "United States"), and pagination limits. |
| Loop Control (Loop Node) | Controls pagination to loop over multiple pages of search results until the max page count is reached or no more results are found. |
| Google Custom Search Request | Sends HTTP GET requests to the Google Custom Search API using the sector, country, and pagination parameters to find LinkedIn CEO profiles. |
| Parse and Enrich Results (Code Node) | Extracts CEO name, job title, company, LinkedIn URL, snippet, profile image, and scraped date from the API response. Handles errors gracefully. |
| Append Row to Google Sheet | Adds each extracted CEO profile as a new row in the Google Sheet. |
| Loop State Updater (Code Node) | Updates loop control variables based on whether more results exist to continue scraping. |
Output
At the end of each run, you get a Google Sheet populated with up to 100 LinkedIn CEO profiles (based on the pagination limit), including:
- CEO name
- Job title
- Company name
- Industry sector
- Country
- LinkedIn profile URL
- A short snippet from the LinkedIn profile or search result
- Profile thumbnail image URL
- Date the data was scraped
This makes it easy to track CEOs by sector and country daily, helping with lead generation or market research.
One-Time Setup
- Create a Google API Key — Go to Google Cloud Console → APIs & Services → Credentials → Create API Key. Add it as the
GOOGLE_API_KEYvariable in the workflow. - Enable Custom Search API — Visit the Custom Search API page and enable it for your project.
- Create a Custom Search Engine — Go to Programmable Search Engine, create a new engine set to search the whole web, copy the CX ID, and set it as the
GOOGLE_CXvariable. - Prepare your Google Sheet — Create a sheet with the column headers listed above. Copy the Sheet ID from the URL and set it as the
SPREADSHEET_IDvariable. - Set Sector & Country — Edit the Scraper Configuration code node to change the
SECTORandCOUNTRYconstants to target your desired market.
Notes
- The Google Custom Search free tier allows about 100 queries per day. Adjust the
MAX_PAGESvalue accordingly to stay within quota. - The scraper relies on Google's indexing and LinkedIn's public profile data — some results may be incomplete or missing images.
- Customize the
SECTORandCOUNTRYconstants in the configuration node to target different markets or industries. - If the workflow encounters API errors, it logs them and stops further scraping to avoid redundant requests.
- Keep your API keys secure and monitor usage to avoid unexpected costs.
Want to showcase your own workflows?
Become a Needle workflow partner and turn your expertise into recurring revenue.
