Web Scraping -101

Web scraping

Now, we know that we need web scraping, let us start with web scraping. Today we will scrape very simple data using a very beginner friendly python library.
Today, we will be needing two python libraries at first let us install them using pip and then import them.

Libraries

Scraping

Here I am installing them together, you can install them one by one
pip install requests
pip install bs4

For me it is already installed so it is showing ‘already satisfied’. Your might take a few seconds.

Scraping

And then we save the source code in a variable and then use the bs4 library to parse and extract the required data.

We used BeautifulSoup, which is a class in the bs4 library and it takes the source_code which we got as a response from the website.

Now we will use selector to find out the data we seek.

Let us say we will build a program which will extract the number of Covid-19 case from the worldometers.info website. for that we need to get the url to the page where we will get the data we want from and then use the requests library to send a HTTP request and the website will give us the source code.

Scraping

Parsing the data

Scraping

we can get it easily from the title and then split it to get the number.

Scraping
recovered

Now let us say we are interested in the number of recovered cases which we can not scrape from title. So let us look for the data in the HTML source code or we can inspect the element from the browser and get the tag or class for the element.

We can see the element has a ‘span’ tag and a ‘maincounter-number’ class. which we can use to extract the data.

This is bad, it has spited the number of total cases. Because from the source code we can see those both elements the same class and tag name. So the program has just gave us the first value it encountered.

So, Now you are familiar with scraping, it is your job to find out how you can scrape the number of the recovered cases. And that way you can scrape most of the websites. Thank you. Happy scraping!

106 thoughts on “Web Scraping -101”

  1. I truly love your site.. Great colors & theme. Did you develop this site yourself?
    Please reply back as I’m trying to create my own site and would like to know where
    you got this from or just what the theme is named. Thank you!

  2. Every weekend i used to visit this website, because i want enjoyment, as this this web page conations truly pleasant funny data too. Nerita Clint Panter Dynah Robinson Lattie

  3. I’m really inspired along with your writing
    skills as well as with the structure in your blog. Is that this a paid
    topic or did you customize it yourself? Anyway stay up the excellent quality writing, it is uncommon to see a great
    weblog like this one nowadays..

  4. I want to express my love for your kind-heartedness in support of men who should have assistance with your theme. Your special commitment to getting the solution up and down ended up being particularly advantageous and has specifically allowed guys and women like me to get to their desired goals. Your personal interesting guidelines can mean this much to me and even further to my colleagues. Thanks a lot; from all of us. Candie Morley Rora

  5. I’m more than happy to discover this website.
    I wanted to thank you for your time for this particularly wonderful read!!
    I definitely loved every little bit of it and i
    also have you book marked to see new things in your site.

  6. Hello there. I found your web site by means of Google while searching for a related topic, your website got here up. It appears to be good. I have bookmarked it in my google bookmarks to come back later. Simona Ricoriki Concordia

  7. Do you mind if I quote a few of your posts as long as I provide credit and sources back to your website? My blog site is in the very same niche as yours and my users would genuinely benefit from some of the information you provide here. Please let me know if this okay with you. Regards! Letta Carce Stag

  8. Hurrah! After all I got a blog from where I know how to really take valuable data concerning my study and knowledge. Lucine Boone Floro

  9. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam vitae neqnsectetur adipiscing elit. Nam viae neqnsectetur adipiscing elit. Nam vitae neque vitae sapien malesuada aliquet. Aurie Dorie Lukas

  10. Party nude Feet porn sex pornography fuckin fingerfucker butt fucker clit porn motherfucks hooker nude butch mothafuckings sex Described Video nude Compilation Teen Latina porn sex German porn pissin porn nerd Verified Couples dyke nude asswipe porn mothafuckas Red Head phuked brown eye dink Vintage sex shitty ejaculates bull shit fistfuckers cocksucker nude stepsisters motherfucked butch motherfuckings kunilingus nude chink shitter fags Squirt nude cyberfucked jackass bung hole sex pissers fuckme ejaculation sex Fingering dong porn French jiz School porn cyberfuck dumbass Verified Couples nude nerd sex Celebrity Babysitter porn Bondage porn bitchin porn pisser sex prick sex retard fingerfuck ass buttfuck fistfuckings fart Small Tits sex muff nude cumshot phukking cyberfucking fucks butthead cocksucked Tattooed Women cooter prick POV sex porn Creampie nude Fingering porn motherfuckers cocksuck Babe sex wetback cobia mick Public porn bonehead lesbo Strap On Exclusive sex mothafucker browntown Pissing ejaculation sex merde dildos sex Massage Masturbation cunts farts fistfuck merde hard on porn mothafuckin sex Fisting dinks horniest dipshit pussys ejaculates Muscular Men nude farts cocksucked mick porn Cumshot porn Compilation Gay boy sex cuntlicking porn orgasims browneye Bondage sex sex College bitchers porn motherfucker ejaculated Muscular Men buttfuck porn mothafuckings nude bestial Teen porn sex Described Video sex punk School fucker bitching cunnilingus Big Dick cocksucks Smoking nude farted Bondage Amateur porn fagot hotsex shiting Role Play Toys sex porn pissers Pissing porn Smoking mothafuck Role Play sex College crap shitted fingerfucker Bisexual Male sex motherfucker Toys Orgy porn Hentai British porn porn phuq sex orgasims Tattooed Women farting sex queer bung hole Striptease porn Double Penetration cyberfuc fingerfucks blow job Romantic porn dick fatass motherfucked Red Head pissin mothafucked fucking cyberfucking Bisexual Male Big Tits fart mothafucking nude Music Rough Sex gangbang jiz wetback mothafuckaz nude butthead Amateur nude nudist beach bucket cunt nude cocksucked Indian nude browntown shitters Closed Captions nude cums big ejaculated butt hair nude Ebony boner cyberfucker Indian Gay sex bestiality fucks porn mothafuckin sex Fisting dinks horniest dipshit pussys ejaculates Muscular Men nude farts cocksucked mick porn Cumshot porn Compilation Gay boy sex cuntlicking porn orgasims browneye Bondage sex sex College bitchers porn motherfucker ejaculated Muscular Men buttfuck porn mothafuckings nude bestial Teen porn sex Described Video sex punk School fucker bitching cunnilingus Big Dick cocksucks Smoking nude farted Bondage Amateur porn fagot hotsex Jeri Ely Wind

  11. Excellent site. Lots of useful information here. I am sending it to a few friends ans also sharing in delicious. And naturally, thanks for your effort!| Merrielle Beltran Zebulon

  12. Hi there. I found your website by way of Google even as searching for a similar subject, your web site came up. It appears good. I have bookmarked it in my google bookmarks to visit later. Charmine Stearn Boggs

  13. I think you can give YouTube a trial and monetize your channel as well. People will subscribe and get notification every time you uplaod a new video. Your subscribers that business are inclined that watch on instagram will surely watch it on YouTube. Much more that the videos are 1-2 minute long, even if it is 5minute I think those that enjoys and learn from it will still subscribe and watch. Kym Duffy Jodee

  14. Hey! This is my first comment here so I just wanted to give a quick shout out and tell you I genuinely enjoy reading through your blog posts. Can you recommend any other blogs/websites/forums that deal with the same topics? Thanks for your time! Bertina Brantley Shena

  15. Pellentesque placerat mi quis iaculis dapibus. Aliquam rutrum vel risus at commodo. Quisque pulvinar feugiat massa pellentesque bibendum. Fusce lacinia consectetur risus, eu dapibus metus porta at. Minnaminnie Gibby Giff

  16. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Nam vitae neqnsectetur adipiscing elit. Nam viae neqnsectetur adipiscing elit. Nam vitae neque vitae sapien malesuada aliquet. Jacinthe Trever Narra

  17. One of my recent moments was when I made canadian bacon for the first time. I have never met anyone who makes it homemade and realized how incredibly easy it was. Not to mention the flavor. No more store bought C. B. for me!! Saundra Tadeo Sabba

  18. Greetings! Very helpful advice within this article! It is the little changes that produce the most important changes. Many thanks for sharing! Tobi Gerrie Mashe

  19. I always used to study paragraph in news papers but now as I am a user of web therefore from now I am using net for content, thanks to web. Gwendolin Rip Tawsha Janey Micheil Pepito

  20. After study a handful of the blog posts in your web site now, and I genuinely such as your technique of blogging. I bookmarked it to my bookmark web site list and will be checking back soon. Pls check out my web-site in addition and make me aware what you believe. Jemima Osgood Lethia

  21. Hi, I think your blog might be having browser compatibility issues. When I look at your blog in Firefox, it looks fine but when opening in Internet Explorer, it has some overlapping. I just wanted to give you a quick heads up! Other then that, terrific blog! Dannie Dewain Cralg

  22. Thank you for sharing these wonderful content. In addition, an excellent travel and medical insurance strategy can often relieve those worries that come with traveling abroad. The medical crisis can shortly become too expensive and that’s likely to quickly place a financial problem on the family’s finances. Setting up in place the ideal travel insurance package prior to leaving is definitely worth the time and effort. Cheers Glori Lucas Jelle

  23. Hi there, I discovered your website by means of Google even as looking for a comparable subject, your website got here up, it seems to be good. I have bookmarked it in my google bookmarks. Codee Heath Gelasias

  24. always i used to read smaller articles or reviews which as well clear their motive, and that is also happening with this paragraph which I am reading here.| Christian Pip Kries

  25. Hello I liked the article, thank you, I hope your site will be successful, you support me tooHello I liked the article, thank you, I hope your site will be successful, escort you support me too

  26. It’s actually a cool and helpful piece of information. I am happy that you simply shared this useful info with
    us. Please stay us up to date like this. Thank you for sharing.

  27. If some one needs to be updated with newest technologies afterward he must
    be pay a quick visit this website and be up to date every day.
    ps4 games 185413490784 ps4 games

  28. I know this web page provides quality depending posts and additional data, is there any other site which
    gives these kinds of data in quality? ps4 games allenferguson ps4 games

  29. Aw, this was an extremely good post. Taking a few minutes
    and actual effort to make a really good article…
    but what can I say… I hesitate a lot and don’t seem
    to get anything done.

  30. W88 Casino Review
    Established in 2013, W88 has been a leading online gaming site in Asia for quite
    a while. During this time, the operator has earned a great reputation among players thanks to its wide range of activities, which include casino games, sports betting, lottery,
    live casino games and poker.

    The W88 site is well-known for its commitment to customer service, offering players the
    chance to claim generous bonuses regularly. So, it while it seems like this casino is packed with great games and customer service, let’s find out if there’s more to it than this.

  31. Hey there I am so delighted I found your weblog, I really found you by mistake, while I was
    looking on Askjeeve for something else, Nonetheless I am
    here now and would just like to say thanks a lot for a marvelous post and a all round interesting blog (I also
    love the theme/design), I don’t have time to go through it
    all at the moment but I have book-marked it and also included
    your RSS feeds, so when I have time I will be back to read much more, Please do keep up the superb work.

  32. This is a good tip especially to those new to the blogosphere.
    Short but very precise information… Appreciate your
    sharing this one. A must read article! asmr 0mniartist

  33. You actually make it appear so easy together with your presentation however I find this matter to be actually one thing that I believe I might
    never understand. It sort of feels too complex and extremely wide for me.

    I’m taking a look forward to your subsequent post, I’ll attempt to get the hold of it!

  34. Good day! I know this is kind of off topic but I was wondering which blog platform are you using for this site?
    I’m getting fed up of WordPress because I’ve had problems with hackers and I’m looking at alternatives for another platform.
    I would be fantastic if you could point me in the direction of a good
    platform. asmr 0mniartist

  35. Hi there this is kinda of off topic but I was
    wanting to know if blogs use WYSIWYG editors or
    if you have to manually code with HTML. I’m starting a blog soon but have
    no coding expertise so I wanted to get advice from
    someone with experience. Any help would be greatly appreciated!
    0mniartist asmr

  36. each time i used to read smaller articles or reviews which also clear their motive, and that is also happening with
    this paragraph which I am reading here. asmr 0mniartist

  37. Hello I liked the article, thank you, I hope your site will be successful, you support me tooHello I liked the article, thank you, I hope your site will be successful, escort you support me too

  38. Metin2 pvp serverler ile en eğlenceli pvp oyunları bulabileceğin Forumexe.tc üzerinden istediğin oyun modunu bul ve metin2 heyecanını limitsiz tat! Metin2 pvp server tanıtım forumu üzerinden ücretsiz serverleri incele, sana uygun olanını seçerek anında maceraya katıl!

  39. İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu İzmir oruspu çocuğu

  40. Firmaların takipçi satın alma sebeplerinin arkasında prestijlerini artırmak yatıyor. Çok fazla takipçinin olması markanın daha fazla bilindiği imajını yaratsa da marka bilinirliğiniz gerçek anlamda sadece bir sayıdan oluşacaktır. Markaya değer katan gerçek kullanıcılardır.

  41. Terrific work! That is the type of info that are supposed
    to be shared around the internet. Disgrace on Google for no longer positioning this post higher!
    Come on over and seek advice from my web site
    . Thank you =)

  42. you are in point of fact a excellent webmaster. The site
    loading speed is incredible. It seems that you are doing any distinctive trick.
    Furthermore, The contents are masterpiece. you’ve performed a magnificent activity in this subject!

  43. No matter if some one searches for his vital
    thing, therefore he/she wishes to be available that in detail, so that thing is
    maintained over here.

  44. When I originally commented I clicked the “Notify me when new comments are added” checkbox
    and now each time a comment is added I get four emails with the same comment.
    Is there any way you can remove people from that service?
    Thank you!

  45. Hi there, I discovered your site by the use of Google even as searching for a similar topic, your web site came up, it
    appears to be like great. I’ve bookmarked it
    in my google bookmarks.
    Hi there, simply turned into aware of your weblog
    through Google, and located that it is truly informative.
    I am gonna watch out for brussels. I will be grateful for those who proceed this in future.
    Lots of other people shall be benefited from your writing.
    Cheers!

  46. First of all I want to say terrific blog! I had a quick question that I’d
    like to ask if you do not mind. I was curious to know
    how you center yourself and clear your mind prior to writing.
    I have had a tough time clearing my mind in getting my ideas out.

    I do take pleasure in writing however it just seems like
    the first 10 to 15 minutes are usually wasted simply just trying to figure out how to begin.
    Any suggestions or hints? Many thanks!

  47. I’m not sure why but this weblog is loading incredibly slow for me.

    Is anyone else having this problem or is it a issue on my end?
    I’ll check back later on and see if the problem still exists.

  48. I absolutely love your blog.. Very nice colors & theme.
    Did you develop this amazing site yourself?

    Please reply back as I’m looking to create my own personal site and want to know where you got this from or exactly what the theme is named.
    Appreciate it!

  49. Hey I know this is off topic but I was wondering if you knew of any widgets I could add to my blog that automatically tweet my newest twitter updates.
    I’ve been looking for a plug-in like this for quite some time and was hoping maybe you
    would have some experience with something like this.
    Please let me know if you run into anything.
    I truly enjoy reading your blog and I look forward to your new updates.

Leave a Comment

Your email address will not be published.