Scraping Hidden Website Data
hewjet last edited by
This could be general to any python API but I want to scrape data (numbers) from a betting site. Requesting an URL displays those numbers in Safari but when I inspect the HTML the data packages and their headers are not always there.
Some websites display the top item with its header and data. Then it seems the following items follow suit but without their header or data as though there is a general element within the string that infills what is wanted. I don't actually see the data.
Other websites hide all headers and data at all times
I see price comparison websites that seem to get around this but how is it done?
JonB last edited by
I find it is helpful to use developer tools on your desktop (F12 on chrome or IE/edge, I suspect safari has something similar), then select the element, and check the XPath.