Build Software Robots with Python

Course is available here: http://pythonforengineers.com/web-scraping-course/

 

Sign up to know when the project will be available:

PS: Interested in leveling up your Python and getting a great job? Check out the Python Apprenticeship Program.

kickstart2

A software robot, or bot, is a program that can perform automated jobs over the internet. Their advantage is that a robot can do a lot of tedious and boring work, saving programmer time for more useful stuff.

Bots are infamous for use by spammers, but they have several legitimate uses as well.

For example, any price comparison website usually works using bots, as no human could check thousands of products and update prices regularly.

Many companies use bots to post Twitter updates, answer customer questions (usually by directing them to the FAQ), amongst other things.

I built a simple bot that can respond to posts on Reddit. The tutorial is here and the source code is here. To see it in action, go here. All the replies are by automated bots.

This course will describe how you can build bots to automate basic tasks. Stuff we’ll go over:

  • Web scraping: Parsing and downloading data from a website using Python.
  •  Legal problems with scraping, why you should use an API when provided
  •  Reddit API
  • Twitter API
  • Advanced Scraping:  Many website nowadays hide data behind Javascript. We need a different approach to extract this data (like Selenium).
  • Use Selenium to fill out forms, click buttons, and otherwise interact with a website.

We’ll build the following bots as part of the course:

  •  Parse a website, and search for words like sale or offer, and store that data.
  •  Parse a Javascript based (dynamic) webapp, and monitor the price. When it goes down, email the user
  •  Reddit Bot that can talk to the user. Like what we have now, but with more bells and whistles
  • Twitter Bot: A bot that can monitor Twitter trends (like how often #Python is used), and draw a graph of it
  •  Finally, we use all we have learnt to build an advanced bot.  A bot that will log in to a website, fill out a form and get some data back.

Something like a bot that books an appointment for you. So it logs in, fills out personal details, takes the first available appointment and emails you.

All the work will be done in a virtual machine (Vagrant), so you can follow along whichever OS you are using. You will get this VM and all the source code for free.

Sign up to know when the project will be available:

PS: Interested in leveling up your Python and getting a great job? Check out the Python Apprenticeship Program.

 

Leave a Reply