How to use proxy with authentification in Selenium with Chrome WebDriver?

Question

After hours of research I am still not able to use a proxy with authentification (username, password) in selenium using chrome WebDriver.

Here is what I tried:

• using a chrome extension -> not working in headless mode

• using AutoIT -> technically it's working, but for my usecase not good enough

• using chrome options arguments like below, doesn't work:

  chromeOptions.addArguments("--proxy-server=http://" + username + ":" + password + "@" + proxyAndPort);

• fill chrome authentification form with Selenium OOTB functionality: doesn't work because its an OS alert (thats why AutoIT can fill it)

Are there any other possibilities to connect with a proxy with authentification?

EDIT: After using browsermobproxy, I'm facing the issue that chrome shows me the error "ERR_EMPTY_RESPONSE". Here's my implementation:

    BrowserMobProxyServer proxy = new BrowserMobProxyServer();
    
    proxy.setChainedProxy(new InetSocketAddress(myIp, myPort));
    proxy.chainedProxyAuthorization(username, password, AuthType.BASIC);
    proxy.start(0);
        
    chromeOptions.setProxy(ClientUtil.createSeleniumProxy(proxy));
    WebDriver driver = new ChromeDriver(chromeOptions);

EDIT2: Use browsermob-proxy version 2.1.5 - everything works fine

Answer 1

You need to set Proxy-Authorization header to the request which are coming from your web browser. I would use browsermob-proxy for handling this.

Here you can find some example of how to use the proxy with your Selenium test. You need to amend the code from "Create test fish-bone" section so that you have the following setUpProxy() method:

public void setUpProxy(){
    proxy = new BrowserMobProxyServer();
    proxy.setChainedProxy(new InetSocketAddress("YOUR_PROXY_HOST", 1234));
    proxy.chainedProxyAuthorization("USER_NAME", "PASSWORD", AuthType.BASIC);
    proxy.start(0);
}

• Change new InetSocketAddress("YOUR_PROXY_HOST", 1234) to your proxy host and port
• Change "USER_NAME", "PASSWORD" to your credentials

The idea is that you configure your selenium to use the proxy which is started by your test code. That proxy in turn is to be configured to use the upstream proxy that you normally use in manual mode.

P.S. - Use browsermob-proxy version 2.1.5

Answer 2

I tested by hand with Firefox and my proxy is OK. This is my code.

    BrowserMobProxyServer proxyServer = new BrowserMobProxyServer();
    proxyServer.setChainedProxy(new InetSocketAddress(proxyHost, proxyPort));
    proxyServer.chainedProxyAuthorization(username, password, AuthType.BASIC);
    proxyServer.start(0);

    ChromeOptions ops = new ChromeOptions();
    ops.setProxy(ClientUtil.createSeleniumProxy(proxyServer));

    System.setProperty("webdriver.chrome.driver", chromePath);

    WebDriver driver = new ChromeDriver(ops);
    driver.navigate().to("https://youtube.com");

My code not working. I don't know what's the problem I met?