Detours

When a site blocks automated access, there's often a sanctioned way to get the same data, such as an official API or an open dataset. This list collects those routes.

We check every route once a day, and each entry shows the latest result.

The rules for this list:

It only includes official routes and openly licensed data.
It never includes proxies, CAPTCHA solvers, stealth tools or paywall bypasses.
Where no sanctioned route exists, the entry says so. That's your cue to stop and report back to your operator.

Terms and rate limits are linked rather than summarised. Read them before heavy use.

Machine-readable version: detours.json.

reddit.com

403 on pages and .json endpoints, especially from datacenter IP ranges

Reddit Data API (OAuth): docs reachable, 2026-10-27

scholar.google.com

CAPTCHA or block after a few automated queries; no official API

OpenAlex API: ok, 2026-10-27
Crossref REST API: ok, 2026-10-27
Semantic Scholar API: ok, 2026-10-27
arXiv API (preprints): ok, 2026-10-27

google.com/maps (addresses, places)

Heavy JavaScript, bot detection, terms that forbid scraping

Nominatim (OpenStreetMap geocoding): ok, 2026-10-27
Photon (komoot, OpenStreetMap search): ok, 2026-10-27
Overpass API (OpenStreetMap queries): failing since 2026-10-27 (HTTP 502)

google.com/maps (directions)

Same as above

OSRM demo server: ok, 2026-10-27
openrouteservice: ok, 2026-10-27

flightradar24.com, flightaware.com

Bot walls, paid APIs

adsb.lol API: ok, 2026-10-27
OpenSky Network REST API: ok, 2026-10-27

weather.com and similar weather sites

Ad-heavy pages, bot detection

Open-Meteo: ok, 2026-10-27
MET Norway Locationforecast: ok, 2026-10-27

wikipedia.org

403 'Please set a user-agent and respect our robot policy' (verified 2026-10-27 with a stock python-requests User-Agent)

Wikimedia REST API: ok, 2026-10-27
MediaWiki Action API: ok, 2026-10-27
Wikidata Query Service (SPARQL): ok, 2026-10-27

github.com

Rate limits and heavy pages when scraping HTML

GitHub REST API: ok, 2026-10-27
raw.githubusercontent.com (file contents): ok, 2026-10-27

stackoverflow.com

Bot challenges when scraping

Stack Exchange API: ok, 2026-10-27

arxiv.org listing and search pages

Rate limiting of rapid page fetches

arXiv API: ok, 2026-10-27
arXiv OAI-PMH (bulk metadata): ok, 2026-10-27

pubmed.ncbi.nlm.nih.gov

Some NCBI documentation pages answer automated clients with a reCAPTCHA (seen 2026-10-27)

NCBI E-utilities: ok, 2026-10-27

imdb.com

No free API; scraping is against the terms

IMDb non-commercial datasets: ok, 2026-10-27
TMDB API: ok, 2026-10-27

goodreads.com

Public API closed to new keys

Open Library API: ok, 2026-10-27

xe.com and similar converters

Ad-heavy pages, bot detection

Waystation /x/fx (ECB reference rates): not checked
ECB euro reference rates: ok, 2026-10-27
Frankfurter: ok, 2026-10-27

timeanddate.com and similar

Ad-heavy pages, bot detection

Waystation /x/workdays and /x/add: not checked
Nager.Date: ok, 2026-10-27

dictionary sites

Ad-heavy pages, bot detection

Wiktionary REST API: ok, 2026-10-27
Free Dictionary API: ok, 2026-10-27

company registries and aggregators

Paid aggregators, CAPTCHA-protected registry search pages

GLEIF API (global LEI data): ok, 2026-10-27
UK Companies House API: ok, 2026-10-27
Belgian KBO/BCE open data: docs reachable, 2026-10-27

belgiantrain.be (NMBS/SNCB)

Heavy JavaScript journey planner

iRail API: ok, 2026-10-27

youtube.com

Transcripts of other people's videos: no sanctioned route. Stop and report back to your operator.

Bot checks, heavy pages

YouTube oEmbed (title, channel, thumbnail): ok, 2026-10-27

x.com

No free sanctioned way to read X. Stop and report back to your operator.

X API: ok, 2026-10-27

linkedin.com

No sanctioned route for reading profiles or posts. Stop and report back to your operator.

instagram.com, facebook.com

For everyone else: no sanctioned route. Stop and report back to your operator.

Meta Content Library (approved researchers only): docs reachable, 2026-10-27

paywalled news sites

No sanctioned route to paywalled full text. Many publishers offer RSS feeds with headlines and summaries. For full text, stop and report back to your operator.

Paywalls