Python Web Crawler Tutorial – 4 – Speeding Up the Crawler

[ad_1]
Facebook –
GitHub –
Google+ –
LinkedIn –
reddit –
Support –
thenewboston –
Twitter –


Posted

in

by

Tags:

Comments

16 responses to “Python Web Crawler Tutorial – 4 – Speeding Up the Crawler”

  1. Vaishnav M Avatar

    your website is not a working site
    it shows file not found

  2. Aayushi Johri Avatar

    i m getting this error while creating the file

    /root/PycharmProjects/crawler/venv/bin/python /root/PycharmProjects/crawler/main.py
    Traceback (most recent call last):
    File "/root/PycharmProjects/crawler/main.py", line 27, in <module>
    create_files("crawler", "https://thenewboston.com")
    File "/root/PycharmProjects/crawler/main.py", line 15, in create_files
    write_file(queue, base_url)
    File "/root/PycharmProjects/crawler/main.py", line 22, in write_file
    f = open(path, 'w')
    IOError: [Errno 2] No such file or directory: 'crawler /root/PycharmProjects/crawler/queue.txt '

    Process finished with exit code 1

  3. Hafiz M Ahmad Naeem Avatar

    please reply as soon as possible
    thenewboston!!!!
    what are the pre-required skill for this course

  4. Fullstack Devs Avatar

    man i love you and i hate you at the same time… i love you cz u know everything and you are everything i wanna be. i hate you because you know everything and you are everything i wanna be lol

  5. Sam Gram Avatar

    "Alrighty mofos" haha, such a good intro! XD

  6. Lennart Bank Avatar

    "Allrighty mofos" cracked me up

  7. Johnathan Chivington Avatar

    Anyone know why I can't import HTMLParser?

  8. S1rWakka Avatar

    First video in the series with dislikes so far. Wonder why.

  9. Ribeirao Avatar

    really simple stuff

  10. Justin Fattz Avatar

    Why not put these functions in a class?

  11. Zsombor Udvardy Avatar

    Why aren't you using strip instead of replace at 6:45?

  12. Akash Popat Avatar

    Instead of using append_to_file wouldnt it just be better to open the file in set_to_file once and just keep writing in it ??

  13. Severen Redwood Avatar

    1 million subs only 211 views

  14. tehlolzfactor Avatar

    Thanks for the tut Bucky. I've been waiting forever for a good python web crawler tutorial.

  15. aladeenzweipunktnull Avatar

    its 1:42 in the morning, i cant get ice cream atm.

Leave a Reply

Your email address will not be published. Required fields are marked *