To convert the "Ask HN: Who is hiring?" thread on Hacker News into a Google Sheets document, a systematic technical process was implemented leveraging specific tools and APIs.

1. Identifying Thread and Extracting Data:

   • The process begins by accessing the (November) thread URL, such as: https://news.ycombinator.com/item?id=38099086.
   • Extract the unique ID from the webpage and utilize it in the Hacker News Algolia Search API: https://hn.algolia.com/api/v1/search_by_date?tags=comment,story_38099086&hitsPerPage=10.

2. Retrieving Data with API:

   • A GET request is used to retrieve data. For instance, using Axios in JavaScript:

   const response = await axios.get(url, { params: { hitsPerPage: limit, page } });
   

   • The response contains an array of JSON objects within the hits field, each representing a post from the thread. The most significant field is comment_text, containing the actual content of each comment.

3. Processing Data with ChatGPT:

   • For each comment text, a ChatGPT “template” is used to extract specific details.

   // For simplicity, I use a pseudo YAML format, as it is less verbose than the actual JSON
   
   functions:
   	check_skip: Call if you need to keep or skip this item
   		parameters:
   			skip: boolean
   	pull_data: Pull items from Algolia url
   		parameters:
   			url: string
   	append: Append a row to Google sheets document
   		parameters:
   			values: array
   			url: string
   messages:
   	function: 
   		name: pull_data
   		content: ${comment_text}
   	user: Only keep valid job offer
   	assistant:
   		function_call:
   			check_skip: ...
   	user: From this job offer, give me following key points (use NA if needed):
   		- Company
   		- Goal
   		- Position
   		- Location
   		- Job Type: Full time or part time
   		- Remote (Full, Partial, from specific timezone)
   		- Compensation
   		- Experience
   		- Website URL
   		- job offer URL
   	assistant: ...
   	user: Push these informations (keeping order) to <https://docs.google.com/spreadsheets>...
   	assistant:
   		function_call:
   			append: ...
   

4. Iteration and Data Normalization:

   • The process iterates through each post in the thread and executes the ChatGPT template.
   • As a result, a Google Sheets document is populated with all valid job offers, structured according to the requested key points.

5. Result

   The resulting Google Sheets document can be found here: https://docs.google.com/spreadsheets/d/1oOIZmTrppLY3QrDu3l6tosDJPNotio4dlytb0MP2dHc/edit?usp=sharing

By combining the capabilities of Hacker News Algolia API, ChatGPT for data processing, and Google Sheets for data storage, the technical process efficiently converts the thread content into a structured and organized spreadsheet of job offerings.

Arnaud Norman (@myaictf) / X