How to modify HTTP request header in Selenium WebDriver with Java?

Question

I want to add "iv-user" request header to log into web app.
I'm using the newest ChromeDriver. Using extensions like ModHeader or ModifyHeaders is not working in this case. Fiddler is working, but I need a solution that will allow me to change accounts during the test case execution.

Can someone help me with this, please?

Now, I'm trying to use BrowserMob Proxy in Embedded mode, to achieve it, but something is not working. Proxy is running and ChromeDriver too, request header is set and sent to the app server, but after checking network log in browser, this custom header is not received by it.

Class Case:

@BeforeEach
public void setUp() throws Exception {
    LOGER.setLevel(Level.WARNING);
    loadProperties();

    if (driver != null) {
        driver.quit();
    }

    BrowserMobProxy proxy = new BrowserMobProxyServer();
    proxy.start(0);
    Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);

    proxy.addRequestFilter((request, contents, messageInfo)->{
        request.headers().add("iv-user", "login");
        System.out.println(request.headers().entries().toString());
        return null;
    });

    chromeOptions = new ChromeOptions();
    String proxyOption = "--proxy-server=" + seleniumProxy.getHttpProxy();
    chromeOptions.addArguments(proxyOption);
    chromeOptions.addArguments("-incognito");
    chromeOptions.addArguments("disable-infobars");
    chromeOptions.setCapability(CapabilityType.PLATFORM_NAME, Platform.WINDOWS);
    chromeOptions.setCapability(CapabilityType.UNEXPECTED_ALERT_BEHAVIOUR, UnexpectedAlertBehaviour.ACCEPT);
    chromeOptions.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
    System.setProperty("webdriver.chrome.driver", CHROMEDRIVER_PATH);
    driver = new ChromeDriver(chromeOptions);

    actions = new Actions(driver);
    driver.manage().timeouts().implicitlyWait(IMPLICITLY_WAIT_TIME_OUT, TimeUnit.SECONDS);

    driver.manage().window().maximize();
    new WebDriverWait(driver, 30);
    wait = new FluentWait<WebDriver>(driver)
            .withTimeout(ConstConfig.FLUENTWAIT_TIMEOUT_SEC, TimeUnit.SECONDS)
            .pollingEvery(1, TimeUnit.SECONDS)
            .ignoring(StaleElementReferenceException.class)
            .ignoring(NoSuchElementException.class)
            .ignoring(ElementNotVisibleException.class);
    jsExecutor = (JavascriptExecutor) driver;
}

Class Test:

public class Test_Proxy extends Case {

@Test
public void test_Proxy() throws Exception {

    try{
        driver.get(TESTAPP_URL);
        MainMenu mainMenu = new MainMenu();
        //..........

    } finally {//....................}}}

Answer 1

Here is the completed example that demonstrates how you can modify requests from your Selenium tests. To demonstrate one I have created a sample REST mock service via SoapUI that would return just a {"SUCCESS"} message for any response.

The test code (TestNg is used for test running, and borwsermob-proxy for proxying requests):

public class MiscTests {

    WebDriver driver;

    @BeforeTest
    public void setUp(){

        BrowserMobProxy proxy = new BrowserMobProxyServer();
        proxy.start(0);
        Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);

        // put our custom header to each request
        proxy.addRequestFilter((request, contents, messageInfo)->{
            request.headers().add("my-test-header", "my-test-value");
            System.out.println(request.headers().entries().toString());
            return null;
        });

        // Setting up Proxy for chrome
        ChromeOptions opts = new ChromeOptions();
        String proxyOption = "--proxy-server=" + seleniumProxy.getHttpProxy();
        opts.addArguments(proxyOption);
        System.setProperty("webdriver.chrome.driver", "E:/Dev/WebDrivers/chromedriver.exe");
        driver = new ChromeDriver(opts);
    }

    @Test
    public void testProxifying(){
        driver.get("http://localhost:8080/");
        Assert.assertEquals(driver.findElement(By.xpath("//body")).getText(), "{\"SUCCESS\"}");
    }

    @AfterTest
    public void tearDown(){
        if(driver != null){
            driver.quit();
            System.out.println("Driver was instantiated. Quitting..");
        }else{
            System.out.println("Driver was null so nothing to do");
        }
    }

}

In above code you set up a proxy and configure the request filter that process all the outgoing requests. In that filter you add a sample header (modify according to your purpose). In SoapUI log we can see that our added header has successfully came to the server.

P.S.: - Here you can find more examples of browsermob-proxy and selenium

Answer 2

Selenium doesn't have API to do that. You need to use something external.

As Alexey suggested one of the solutions would be setting up proxy like this.

Answer 3

You cannot achieve it by Selenium, but as you mentioned you are using BrowserMobProxy , I would recommend you to try once below .

1. Apache Module mod_headers ( Refer : http://httpd.apache.org/docs/current/mod/mod_headers.html#page-header )

2. You can also install Fiddler (http://www.fiddler2.com/fiddler2/) which is very easy to install (easier than Apache for example). After launching it, it will register itself as system proxy. Then open the "Rules" menu, and choose "Customize Rules..." to open a JavaScript file which allow you to customize requests. To add a custom header, just add a line in the OnBeforeRequest function:

   oSession.oRequest.headers.Add("MyHeader", "MyValue");

Hope this helps.

Answer 4

With the solutions already discussed above the most reliable one is using Browsermob-Proxy

But while working with the remote grid machine, Browsermob-proxy isn't really helpful.

This is how I fixed the problem in my case. Hopefully, might be helpful for anyone with a similar setup.

1. Add the ModHeader extension to the chrome browser

How to download the Modheader? Link

ChromeOptions options = new ChromeOptions();
options.addExtensions(new File(C://Downloads//modheader//modheader.crx));

// Set the Desired capabilities 
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(ChromeOptions.CAPABILITY, options);

// Instantiate the chrome driver with capabilities
WebDriver driver = new RemoteWebDriver(new URL(YOUR_HUB_URL), options);

2. Go to the browser extensions and capture the Local Storage context ID of the ModHeader

3. Navigate to the URL of the ModHeader to set the Local Storage Context

.

// set the context on the extension so the localStorage can be accessed
driver.get("chrome-extension://idgpnmonknjnojddfkpgkljpfnnfcklj/_generated_background_page.html");

Where `idgpnmonknjnojddfkpgkljpfnnfcklj` is the value captured from the Step# 2

4. Now add the headers to the request using Javascript

.

   ((Javascript)driver).executeScript(
         "localStorage.setItem('profiles', JSON.stringify([{  title: 'Selenium', hideComment: true, appendMode: '', 
             headers: [                        
               {enabled: true, name: 'token-1', value: 'value-1', comment: ''},
               {enabled: true, name: 'token-2', value: 'value-2', comment: ''}
             ],                          
             respHeaders: [],
             filters: []
          }]));");

Where token-1, value-1, token-2, value-2 are the request headers and values that are to be added.

5. Now navigate to the required web-application.

   driver.get("your-desired-website");

Answer 5

I was not able to see the added parameterin network tab. later developer told that I need to add to response and not to request. now I can see in chrome network tab

BrowserMobProxy proxy = new BrowserMobProxyServer();
//proxy.setTrustAllServers(true);
proxy.start();
Map<String, String> headers = new HashMap<String, String>();
headers.put("x-stack-b", "true");
proxy.addResponseFilter((response, contents, messageInfo)->{
response.headers().add("x-stack-b", "true");
System.out.println("filter: " +response.headers().entries().toString()); return ;
});

Answer 6

How can set up the same in VBA script? can anyone help on this? I need pass few headers so that akamai won't treat me as bot once i run the application