Selenium is an automated testing library for websites, but because it has a large number of automated libraries and can call browsers, it is often used in crawler technology. It is precisely because it invokes the browser, it has almost become an unsolvable crawler. In the field of neural networks, a large number of data sets are needed. Crawlers are a way to quickly obtain data. This is exactly my motivation for learning this library.
pip install -U seleniumto
Selenium needs the corresponding browser driver to call various browsers. I will use chrome to test. After the test is successful, it will be converted to the non-interface PhontomJS. To use Chrome, you need to download the corresponding driver and copy it to the Python installation folder, and use PhontomJS to copy the .exe file directly to the Python installation folder.
from selenium import webdriver
driver = webdriver.PhantomJS() #Open the PhantomJS browser driver = webdriver.Chrome() #Open Chrome browser
Visit the website with url, if you use Chrome, you will see the opened browser jump to the specified url
name_field = driver.find_element_by_id("username") submit_button = driver.find_element_by_tag_name("button") submit_button = driver.find_elements_by_link_text("Educational Administration System")
According to the element id, type and hyperlink name to get the element, in addition to the above methods, there are
There are also ways to get multiple elements:
name_field = driver.find_element_by_id('loginName') name_field.send_keys('...')
send_keys()method to input data after getting the element
submit_button = driver.find_element_by_id('loginAction') submit_button.click()
click()method to click the button after getting the element