05-02-2023, 01:50 PM
This is causing me to rethink the idea of making the content on my site accessible by URLs like a normal website.
As it stands, they'd have to write a special crawler to execute the JavaScript functions that fetch and render content. I think they're more interested in hoovering up low-hanging fruit wholesale from the public Web than devoting resources to crawling a backwater site that will probably only ever see five active posters.
As it stands, they'd have to write a special crawler to execute the JavaScript functions that fetch and render content. I think they're more interested in hoovering up low-hanging fruit wholesale from the public Web than devoting resources to crawling a backwater site that will probably only ever see five active posters.
