Can we get the text from the image using Selenium?

Question

This is the path of the image from where I have to get the text shown on an image and into the text for security purposes,

@FindBy(xpath = "//img[@src=\"/register/tb135/tb_getimage.php?uid=1677953817&start=JTNFJTIy\"]")
    private WebElement textShown;

This is the XPath where I fill the shown text on the image,

@FindBy(id = "metric_iframe")
    private WebElement fillTextShown;

I tried to get the text using textShown.getText() but it failed. So what I suppose to get the text shown on the image?

Answer 1

No, it is not possible to extract text directly from an image using Selenium. The getText() method retrieves the text content of a WebElement, but an image is not a text element.

However, you can use third-party OCR (Optical Character Recognition) libraries to extract text from an image. OCR technology is used to recognize text within images and convert it into machine-readable text. There are several OCR libraries available that can be integrated with your Selenium script to extract text from images.

One popular OCR library is Tesseract OCR, which is open-source and supports multiple languages. Here is an example of how you can use Tesseract OCR in your Selenium script to extract text from an image:

1 First, you need to download and install Tesseract OCR on your system.

2 Next, you can use the getScreenshotAs() method to take a screenshot of the image containing the text.

3 Then, you can pass this screenshot to Tesseract OCR for text extraction using the Tesseract OCR API.

4 Finally, you can retrieve the extracted text and use it in your test script.

5 Here is some sample code in Java that demonstrates how to use Tesseract OCR with Selenium:

// Import the necessary libraries
import java.io.File;
import net.sourceforge.tess4j.ITesseract;
import net.sourceforge.tess4j.Tesseract;
import org.openqa.selenium.WebElement;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.chrome.ChromeDriver;
import org.openqa.selenium.interactions.Actions;
import org.openqa.selenium.support.ui.WebDriverWait;
import org.openqa.selenium.OutputType;
import org.openqa.selenium.TakesScreenshot;
import org.apache.commons.io.FileUtils;

// Create a new instance of the Chrome driver
WebDriver driver = new ChromeDriver();

// Navigate to the page with the image
driver.get("https://example.com");

// Find the image element
WebElement imageElement = driver.findElement(By.xpath("//img[@src=\"/path/to/image.png\"]"));

// Take a screenshot of the image element
File screenshot = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);

// Use Tesseract OCR to extract text from the image
ITesseract tesseract = new Tesseract();
String text = tesseract.doOCR(screenshot);

// Output the extracted text
System.out.println(text);

// Close the browser
driver.quit();

Note that the above code is just an example and may need to be modified to suit your specific use case.