Selenium is a Web Browser Automation Tool. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that. It allows you to open a browser of your choice & perform tasks as a human being would, such as:
- Clicking buttons
- Entering information in forms
- Searching for specific information on the web pages
Setup and tools
I’m assuming you know how to use pip and virtual environments. If not, start here.
install selenium using pip
pip install selenium
install chrome driver
You will also need the Google Chrome Driver found here.
If you run into an error:
chromedriver executable needs to be in
PATH, you have a few options.
chromedriver to your system path variables.
chromedriver.exe to your Scripts folder of the virtual environment.
Third, explicitly give the location of the driver.
initiate the driver
I will use MacOS and the last option that explicitly point to
So, for using webdriver open Python console or create py-file and insert initialization commands:
from selenium import webdriver driver = webdriver.Chrome(executable_path='./chromedriver')
Our first example will be simple.
The script will open a browser, input some URL (
https://www.google.com) and get title of the loaded site.
Returned title will print into console.
from selenium import webdriver chrome = webdriver.Chrome(executable_path='./chromedriver') chrome.get('https://www.google.com') print(chrome.title) chrome.quit()
You can find proper documentation on selenium here.
Following methods will help to find elements in a webpage (these methods will return a list):
search in google
Once we make a request and it is successful we need to get a response. With response we can make some actions for example input a search string.
from selenium import webdriver from selenium.webdriver.common.keys import Keys chrome = webdriver.Chrome(executable_path='./chromedriver') chrome.get("https://www.google.com") element = chrome.find_element_by_xpath('//input[@name="q"]') element.click() element.send_keys('site:gainanov.pro Web Automation: Selenium WebDriver and Python') element.send_keys(Keys.RETURN) chrome.quit()
XPaths allow the script to determine the exact web element you want.
In the XPath, double forward slash (
//) means find an element anywhere on the page.
The star (
*) means find any element.
@ sign specifies the attribute you want.
In this case, I wanted input element with attribute name=”q” -
Once you grab a web element, you can act with elements.
For example, I send a search phrase and press return key with
Selenium Webdriver provides two types of waits - implicit & explicit. An explicit wait makes WebDriver wait for a certain condition to occur before proceeding further with execution. An implicit wait makes WebDriver poll the DOM for a certain amount of time when trying to locate an element.
Change the row with
find_element_by_xpath method to new the method with explicit waits.
from selenium import webdriver from selenium.webdriver.common.by import By from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC # ... element = WebDriverWait(chrome, 10).until( EC.visibility_of_element_located((By.XPATH, "//input[@name='q']"))) # ...
This waits up to 10 seconds before throwing a
TimeoutException unless it finds the element to return within 10 seconds.
An implicit wait tells WebDriver to poll the DOM for a certain amount of time when trying to find any element (or elements) not immediately available. The default setting is 0. Once set, the implicit wait is set for the life of the WebDriver object.
chrome.implicitly_wait(10) # seconds
You now have the foundational skills necessary to scrape websites and automate test a web interface. But this solution can be run only on local environment. For the best performance and remote testing read my next post. It will include materials about Selenoid can help you save time to run parallel scripts remote.
Thank you for reading.
- What is Selenium? Getting started with Selenium Automation Testing - good explanation from Edureka education platform
- How to Get Started with XPath in Selenium – XPath Tutorial - Getting started XPath
- Selenium with Python - Selenium library for Python