Rendering the websites in its Chrome headless browser, and returning them as static pages, so that I can then grab the page info, and render OpenGraph data or the fallback data like <title> and <image>.
That's why you can see rich previews of all websites, even if they aren't SSR. Also means you can see rich previews (including content) of websites that would normally block you, for having Javascript turned off.
Login to reply
Replies (2)
I.e. the preview you see isn't directly from the website. I build the website and then scrape off the rich preview tags.
I do a lot of scraping and parsing of hyperlinks. This is not data the website has sent me, for instance, but I think this is more interesting than the hyperlink alone, and then I can actually skip rendering the website. ๐ค
https://wikistr.imwald.eu/trusted-assertions*460c25e682fda7832b52d1f22d3d22b3176d972f60dcdc3212ed8c92ef85065c
https://decentnewsroom.com/mag/newsroom-magazine-43cf24/cat/lifestyle-d22cbb/d/How-Colorful-Are-You-ans9bk
https://coracle.social/notes/naddr1qvzqqqr4gupzplaukurf77496md3984nne7c49ufgekj2knr0wkpa0uxz7c9wszyqq54gmmhv9exguedvykksatdv9hz6envdamj66tw946x2cmgdehkcmm80ykh5mtwdpurvww7weg

ZEUS
ZEUS v0.12.0 is now available. - feat: Nostr Wallet Connect service - feat: Circular Channel Rebalance tool - feat: Battery saver ...
Oh yea I've always wanted to play around with that for scraping tools!