We are attempting to create a small server side python script to perform some html scraping of another website. The problem is that the script returns nothing after the 'requests.get(....)' statement. To test this, we are using the test_rest.html file, and the output is always blank. We installed bunch, requests, bs4 (beautiful soup), and we even used the example 'math' script to see if everything was set up correctly, and that did return a correct answer.
from bs4 import BeautifulSoup
import requestsverb = event.request.method;
if verb != 'GET':
raise Exception('Only HTTP GET is allowed on this endpoint.');resource = event.resource;
params = event.request.parameters;
required = ['temp'];
if resource != "":
for element in required:
if params.get(element, "") == "":
raise Exception('Missing ' + element + ' in params.');isbn = str(params.temp);
if resource != 'isbn':
result = {'resource':['isbn']};
else:
result = []
r = requests.get('https://www.campusbooks.com/search/' + isbn)soup = BeautifulSoup(r.text) table = soup.find("table", { "class" : "table table-striped table-hover table-condensed" }) rows = table.find_all("table", {"class" : "table table-condensed"}) for row in rows: seller = row.find('span')['title'].split()[0] condition = row.find('td', {"class" : "condition"}).contents[0].split()[0] price = row.find('td', {"class" : "total"}).contents[0].split()[0] dic = {'seller':[],'condition':[],'price':[]} dic['seller'].append(seller) dic['condition'].append(condition) dic['price'].append(price) result.append(dic)
return result