4

I am currently experimenting with the Python version of Selenium Web Driver. Using the driver has been straight forward so far but I have been struggling recently with the issue of mocking HTTP responses. Is there a Python package that allows you to mock HTTP responses in web applications programmatically ? i.e. it allows Selenium to send real HTTP requests (by clicking on a link in the application for example) but then intercepts them and sends back fake HTTP responses that were created by me via code. Those fake responses will, of course, be displayed by the application.

It has to be one that can integrate perfectly with Python Web Driver and does not interfere with its communication with the web application. A bonus would be to be able to tally all the requests made to a specific URL (or URL regular expression)

Edit: Based on neo's suggestion below, I implemented the following steps:

1) I downloaded the original browsermob-proxy (JAVA version) and saved it in C:\Users\johnsmith\Computer_Code\browsermob-proxy-2.1.0-beta-1\

2) I installed the Python client of browsermob-proxy. The egg file was installed in C:\Python27\Lib\site-packages

3) I created a fake HTML page and saved it in C:\Users\johnsmith\Computer_Code\Python\Automation_Testing.

The HTML page code is shown below:

<!DOCTYPE html>
<html>
    <head>
        <title>Welcome to my fun page</title>
        <meta charset = "UTF-8">
    </head>
    <body>
        <h2>Search Python.org</h2>
        <h2>Mocking is not going well so far !!</h2>
        <img src = "http://i.imgur.com/EjcKmEj.gif">
    </body>
</html>

4) I navigated to the folder in step 3 above using the command line and then executed the following command: python -m SimpleHTTPServer This started a server on port 8000 for the purpose of serving the fake HTML file.

5) I wrote the following simple code that uses Selenium WebDriver to try and navigate to the Python home page. Using Browsermob's rewrite function, I tried to replace the real response to the request for http://www.python.org with the fake HTML page in step 3

# contents of Selenium_WebDriver_Mocking_test.py

import browsermobproxy
import selenium.webdriver as driver
import time

website_homepage_url = "http://www.python.org"

server = browsermobproxy.Server("C:\Users\johnsmith\Computer_Code\\browsermob-proxy-2.1.0-beta-1\\bin\\browsermob-proxy")
server.start()
proxy = server.create_proxy()

profile = driver.FirefoxProfile()
profile.set_proxy(proxy.selenium_proxy())
browser = driver.Firefox(firefox_profile=profile)

proxy.new_har("Python.org")
proxy.rewrite_url(website_homepage_url, 'http://localhost:8000/Fake_Search_Results_Page.html')
browser.get(website_homepage_url)

time.sleep(10)

proxy.har  # returns a HAR JSON blob

server.stop()
browser.quit()

The result of running the above code was a page with the following message:

Error response. Error code 404. Message: File not found. Error code explanation: 404 = Nothing matches the given URI.

Where did I go wrong ?

Improve this question
1
  • you need to escape backslashes in the path name to your fake file.
    – Corey Goldberg
    Jan 29, 2017 at 23:22

2 Answers 2

Reset to default
1

I had used browsermob-proxy and its Python client successfully in one of my project by using rewrite url to load mocking content. browsermob-proxy is a proxy server for the browser to connect with during testing.

Assume you are hosting a webserver with mock content on localhost:8899, below example will load the mock content whenever the browser try to load content from test.html

from browsermobproxy import Server
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities 
from selenium.webdriver.chrome.options import Options

# start a proxy server that intercepts browser requests
server = Server("browsermob-proxy/bin/browsermob-proxy")
server.start()
proxy = server.create_proxy()

# set the browser to use the proxy server
options = Options()
options.add_argument("--proxy-server={0}".format(proxy.proxy))
dCaps = DesiredCapabilities.CHROME
dCaps['loggingPrefs'] = { 'browser':'ALL' }
driver = webdriver.Chrome(chrome_options=options,desired_capabilities=dCaps)

# add the rewrite rule into the proxy server
proxy.rewrite_url('text.html', 'http://localhost:8899/mock.html')

and it supports regular expression,

PUT /proxy/[port]/rewrite - Redirecting URL's

matchRegex - a matching URL regular expression

replace - replacement URL

Improve this answer
3
  • That seems like an interesting solution. I have tried implementing it using Firefox as the browser. My attempt, however, failed. I edited my question above to explain the details of my attempt. Could you please have a look at it and point me in the right direction.
    – Wolf_Brigade
    Jul 13, 2015 at 8:29
  • The error message comes from Python server, that means the rewrite is working and you have successfully simulated a 404 response. Please try to open localhost:8000/Fake_Search_Results_Page.html from a browser to validate, or localhost:8000 to see if you could find the html file.
    – neo
    Jul 30, 2015 at 8:36
  • I can indeed open the file in the browser after starting the Python server by visiting the link localhost:8000/Fake_Search_Results_Page.html. I still don't undestand why that page does not appear inside FireFox when I run the Python browsermobproxy code above even though I clearly instructed it to do so in the proxy.rewrite_url line !
    – Wolf_Brigade
    Jul 31, 2015 at 13:08
-1

You can do:

browser.get('file:///path/to/my/html/file/testing.html')

Edit. You are trying to open a html fake page ( I assume for get the HAR content ), also trying to mock the request (you can do it, but it's more difficult than that), when you can get the same result loading the html file from selenium.

Improve this answer
2
  • Hi Joseph, how is this different than the line that is in OPs post already, browser.get(website_homepage_url)? And what does this do to mock responses by itself? Thanks!
    – corsiKa ♦
    Feb 4, 2016 at 16:09
  • Yeah, I need to explain more.
    – José Castro
    Feb 4, 2016 at 17:50

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.