Get HTML code from URL simply

pllbleu

Hi,
Is there a simple way to get the html code of a web page with a given URL in Pythonista? I tried urllib, using a code from stack overflow, however it didn’t work. Is there another way of doing it?
Thanks for your answer

cvp

@pllbleu if I correctly understand

import requests
url = 'https://www.google.com'
print(requests.get(url).text)

pllbleu

Oh yeah, that works thanks! Also is there a way to sign in to the webpage before getting the html (the content is only given if you’re signed in)?

cvp

@pllbleu if you want that the script does the login programmatically, not manually, you need to know the name of the button in the web page, see here for examples

pllbleu

Okay so I'm currently using this code:

import requests
url='http://www.kwyk.fr/accounts/login/'
login_data=dict(login="username", password="password")
session = requests.session()
session.post(url, data=login_data)
r = session.get("https://www.kwyk.fr/devoirs/419323/?id=23924446")
print(r2.text)

I checked the html code of the web page and I know that they're called login and password but I don't know what the name of the button is. How would I find it and what would I change in my code?
Thanks for your help!

cvp

@pllbleu if you only want to display the page, manually login and then get the html, you can use this and press the button " get html" after your login

import ui
w = ui.WebView()
w.frame = (0,0,400,400)
b = ui.ButtonItem()
b.title = 'get html'
def get_html(sender):
	html = w.eval_js('document.documentElement.outerHTML')
	print(html)
b.action = get_html
w.right_button_items = (b,)
url = 'https://www.google.com'
w.load_url(url)
w.present('sheet')

cvp

@pllbleu said

r = session.get("https://www.kwyk.fr/devoirs/419323/?id=23924446")
print(r2.text)

r <---> r2 ?

pllbleu

The idea in the first place was to send the url of a web page through shortcuts, and have a python program which sent the html code back through the clipboard. However that seems complicated (I'm pretty sure what you're proposing is longer than doing the math exercise on my calculator). I think that I'll juts try to learn some basic js to do it directly on shortcuts.
Thanks anyway

cvp

@pllbleu For info, on your site, the blue button "identifiez-vous" has no name (but a class name) and you can force the submit (=press button) by

	w.eval_js("document.getElementsByClassName('pull-right btn btn-lg btn-primary')[0].submit();")