If you refer to this page, you can set the user agent in the header.
In addition, some services may block per ip if very frequently accessed by scraping.
At that time, it is possible that 403 will be returned, so you should try to access it for at least one second.
If this happens, it will be impossible to access from the same ip, so you should be careful.
The worst case is the police.
For at least UserAgent, see
You can specify the part that opens in such a format by giving the request.Request() object instead of the URL string.
NameError: name 'html' is not defined
def make_network(root_url, urls):
It is probably due toBecause html is in the scope inside the try block, it is different from and undefined in the first argument of
Why don't you try it?
Maybe there is a
placeholder hyperlink where
placeholder hyperlink is a
a element with no
If the href attribute is not specified, the element presents a placeholder hyperlink.
placeholder hyperlink exists, the
urls variable (list) will contain
None in the following parts of the
article.get("href") has a return value of
placeholder hyperlink), change it not to be included in
Try it now.
© 2022 OneMinuteCode. All rights reserved.