<?xml version="1.0" encoding="UTF-8"?><rss xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:content="http://purl.org/rss/1.0/modules/content/" xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title><![CDATA[Clean Feeds]]></title><description><![CDATA[Clean Feeds]]></description><link>https://cleanfeeds.hashnode.dev</link><generator>RSS for Node</generator><lastBuildDate>Fri, 23 Oct 2026 06:40:58 GMT</lastBuildDate><atom:link href="https://cleanfeeds.hashnode.dev/rss.xml" rel="self" type="application/rss+xml"/><language><![CDATA[en]]></language><ttl>60</ttl><item><title><![CDATA[Get every open job from any company's career page (no HTML scraping)]]></title><description><![CDATA[Greenhouse, Lever, Ashby and Workday serve job listings as public JSON. Read any company's open jobs with a few lines of Python.]]></description><link>https://cleanfeeds.hashnode.dev/get-every-open-job-from-any-company-s-career-page-no-html-scraping</link><guid isPermaLink="true">https://cleanfeeds.hashnode.dev/get-every-open-job-from-any-company-s-career-page-no-html-scraping</guid><category><![CDATA[Python]]></category><category><![CDATA[web scraping]]></category><category><![CDATA[api]]></category><category><![CDATA[Tutorial]]></category><dc:creator><![CDATA[clean_feeds]]></dc:creator><pubDate>Wed, 21 Oct 2026 08:59:34 GMT</pubDate><content:encoded><![CDATA[<p>If you've ever tried to scrape a company's careers page, you've probably fought with JavaScript rendering, pagination buttons and selectors that break every month.</p>
<p>You usually don't have to. Almost every careers page is hosted on an <strong>applicant tracking system (ATS)</strong>: Greenhouse, Lever, Ashby, Workday and a handful of others. Nearly all of them expose the job list as <strong>public JSON</strong>, either an official API meant for embedding job boards, or the same endpoint the career site itself calls. No login, no API key, no CAPTCHAs.</p>
<p>This post shows how to read four of them with plain <code>requests</code>, plus the gotchas I hit while mapping a dozen of them.</p>
<h2>Step 1: figure out which ATS a company uses</h2>
<p>Open the careers page and look at where the "Apply" links go:</p>
<table>
<thead>
<tr>
<th>Link looks like</th>
<th>ATS</th>
<th>Slug</th>
</tr>
</thead>
<tbody><tr>
<td><code>boards.greenhouse.io/airbnb</code></td>
<td>Greenhouse</td>
<td><code>airbnb</code></td>
</tr>
<tr>
<td><code>jobs.lever.co/spotify</code></td>
<td>Lever</td>
<td><code>spotify</code></td>
</tr>
<tr>
<td><code>jobs.ashbyhq.com/ramp</code></td>
<td>Ashby</td>
<td><code>ramp</code></td>
</tr>
<tr>
<td><code>nvidia.wd5.myworkdayjobs.com/NVIDIAExternalCareerSite</code></td>
<td>Workday</td>
<td>host + tenant + site</td>
</tr>
</tbody></table>
<p>If the careers page is on the company's own domain, view the page source and search for <code>greenhouse</code>, <code>lever.co</code>, <code>ashbyhq</code> or <code>myworkdayjobs</code>. The board URL is almost always in there.</p>
<h2>Step 2: Greenhouse</h2>
<p>Greenhouse has an official, documented Job Board API:</p>
<pre><code class="language-python">import requests

def greenhouse(slug):
    url = f"https://boards-api.greenhouse.io/v1/boards/{slug}/jobs"
    jobs = requests.get(url, params={"content": "true"}, timeout=30).json()["jobs"]
    return [{"title": j["title"], "location": j["location"]["name"], "url": j["absolute_url"]}
            for j in jobs]

print(len(greenhouse("airbnb")))  # 186 open jobs when I ran it
</code></pre>
<p><strong>Gotcha:</strong> with <code>content=true</code> you get the full description, but it's HTML-<em>escaped</em> (<code>&amp;lt;p&amp;gt;</code>), so run it through <code>html.unescape()</code> before parsing. Remote/hybrid isn't a standard field either. It only exists if the company added a custom metadata field.</p>
<h2>Step 3: Lever</h2>
<pre><code class="language-python">def lever(slug):
    for host in ("api.lever.co", "api.eu.lever.co"):  # EU companies live on a separate host
        r = requests.get(f"https://{host}/v0/postings/{slug}", params={"mode": "json"}, timeout=30)
        if r.ok and r.json():
            return [{"title": j["text"], "location": j["categories"].get("location"),
                     "url": j["hostedUrl"]} for j in r.json()]
    return []

print(len(lever("spotify")))  # 82
</code></pre>
<p><strong>Gotchas:</strong> EU-hosted companies only answer on <code>api.eu.lever.co</code>, so try both. Dates (<code>createdAt</code>) are epoch <strong>milliseconds</strong>.</p>
<h2>Step 4: Ashby (best salary data)</h2>
<pre><code class="language-python">def ashby(slug):
    url = f"https://api.ashbyhq.com/posting-api/job-board/{slug}"
    jobs = requests.get(url, params={"includeCompensation": "true"}, timeout=30).json()["jobs"]
    return [{"title": j["title"], "location": j["location"], "remote": j.get("isRemote"),
             "salary": (j.get("compensation") or {}).get("compensationTierSummary"),
             "url": j["jobUrl"]} for j in jobs]

print(ashby("ramp")[0])
# {'title': 'Security Engineer, Cloud', 'location': 'New York, NY (HQ)', 'remote': True,
#  'salary': '$211.4K – $290.6K • Offers Equity', ...}
</code></pre>
<p>Ashby has the most structured compensation data of any ATS I've looked at: there are also numeric min/max/currency fields under <code>compensation</code>.</p>
<h2>Step 5: Workday (where it gets interesting)</h2>
<p>Big companies like NVIDIA, Salesforce and many banks use Workday. There's no official public API, but the career site loads its jobs from a JSON endpoint you can call the same way:</p>
<pre><code class="language-python">def workday(host, tenant, site, search=""):
    url = f"https://{host}/wday/cxs/{tenant}/{site}/jobs"
    body = {"appliedFacets": {}, "limit": 20, "offset": 0, "searchText": search}
    first = requests.post(url, json=body, timeout=30).json()
    return first["total"], first["jobPostings"]

total, page = workday("nvidia.wd5.myworkdayjobs.com", "nvidia", "NVIDIAExternalCareerSite")
print(total)  # 2000
</code></pre>
<p>That <code>2000</code> is a lie: NVIDIA had <strong>2,747</strong> open jobs when I ran this. Workday's gotchas:</p>
<ul>
<li><strong>Max 20 results per page.</strong></li>
<li><strong>Every search stops at 2,000 results.</strong> Past offset 2,000 it silently wraps back to page 1, so a naive pager loops forever or returns duplicates.</li>
<li>The workaround is to <strong>split the search by a facet</strong>: the response includes <code>facets</code> (job category, location…), and if you pick one whose biggest bucket is under 2,000 and page through each value separately, you get everything.</li>
<li><code>total</code> is only reliable on the first page.</li>
<li>It's behind Cloudflare and throttles per IP, so keep concurrency around 2 and back off on HTTP 429.</li>
</ul>
<h2>The other ATSs, briefly</h2>
<table>
<thead>
<tr>
<th>ATS</th>
<th>Endpoint</th>
<th>Watch out for</th>
</tr>
</thead>
<tbody><tr>
<td>SmartRecruiters</td>
<td><code>api.smartrecruiters.com/v1/companies/{id}/postings</code></td>
<td>Unknown companies return <strong>200 with an empty list</strong>, not 404</td>
</tr>
<tr>
<td>Workable</td>
<td><code>apply.workable.com/api/v1/widget/accounts/{slug}?details=true</code></td>
<td>Rate-limits shared IPs (429)</td>
</tr>
<tr>
<td>Recruitee</td>
<td><code>{slug}.recruitee.com/api/offers/</code></td>
<td>Easy, nothing special</td>
</tr>
<tr>
<td>Personio</td>
<td><code>{slug}.jobs.personio.de/xml</code></td>
<td>XML; drops descriptions not written in the requested language</td>
</tr>
<tr>
<td>Teamtailor</td>
<td><code>{slug}.teamtailor.com/jobs.rss</code></td>
<td>Also works on custom domains (<code>careers.oatly.com/jobs.rss</code>)</td>
</tr>
<tr>
<td>Rippling</td>
<td><code>ats.rippling.com/api/v2/board/{slug}/jobs</code></td>
<td>One row <strong>per location</strong>, so merge by job id</td>
</tr>
<tr>
<td>Breezy HR</td>
<td><code>{slug}.breezy.hr/json?verbose=true</code></td>
<td>No descriptions without <code>verbose=true</code>; salary is free text</td>
</tr>
<tr>
<td>Oracle Cloud HCM</td>
<td><code>{host}/hcmRestApi/resources/latest/recruitingCEJobRequisitions?onlyData=true&amp;finder=findReqs;siteNumber=CX_1001,limit=200</code></td>
<td>The <code>finder</code> must <strong>not</strong> be URL-encoded, or you get zero results</td>
</tr>
</tbody></table>
<p>Be a good citizen: these feeds are public because companies <em>want</em> their jobs distributed, but cache results, don't hammer them, and respect 429s.</p>
<h2>The hard part is normalizing</h2>
<p>Each of these returns a different shape: locations as strings vs objects, dates as ISO strings vs epoch ms, salaries structured vs free text, remote flags in five different places. Writing the fetchers takes an afternoon. Keeping a dozen of them normalized and working as APIs change is the real work.</p>
<h2>If you'd rather not maintain it</h2>
<p>Disclosure: I packaged all twelve of these into an <a href="https://apify.com/clean_feeds/ats-jobs-scraper">Apify actor</a> that I built and maintain. You give it company names ("Ramp"), domains or careers URLs, and it detects the ATS and returns every job in one schema: title, location, remote/hybrid, salary min/max/currency, dates and full description. It also handles the Workday 2,000-result cap, and has a "new jobs only" mode for daily monitoring. It's pay-per-result ($2 per 1,000 jobs).</p>
<p>But everything above works fine on its own. Happy to answer questions about any of these endpoints in the comments.</p>
]]></content:encoded></item></channel></rss>