urlopen connection to you test HTTPS URL

[realpython]

urlopen connection to you test HTTPS URL

[ScriptAutomate]

You are right, that the default code in the Scrape and Parse Text From Websites doesn't work without some modifications to it. The website is likely blocking straight urllib calls without using a common user agent.

I get the following error when using the default code from RealPython, as you have above:

HTTPError: HTTP Error 403: Forbidden

If anyone else if having problems here, this should work (if wanting to continue to use urllib as part of the exercise):

from urllib.request import Request, urlopen

mozilla_request = Request('https://realpython.com/practice/aphrodite.html',
                          headers={'User-Agent': 'Mozilla/5.0'})
html_page = urlopen(mozilla_request)
html_text = html_page.read().decode('utf-8')

print(html_text)

NOTE: I definitely need to credit this StackOverflow for my answer method above

Using the requests package works fine, too. It isn't a standard library, so a pip install requests or pipenv install requests would need to be run beforehand:

import requests

r = requests.get("https://realpython.com/practice/aphrodite.html")
print(r.text)