Using SIKULI API with Selenium Web driver for image based recognition


Sikuli API for Java provides image-based GUI automation functionalities to Java programmers. It is created and will be actively maintained bySikuli Lab. The effort of making this API is a response to the desire of many Sikuli users to use Sikuli Script’s functionalities directly in their Java programs, rather than writing Jython scripts. This new Java library has a re-designed API and includes several new functions that were not available in the original Sikuli Script, such as the abilities to match colors, handle events, and find geometric patterns such as rectangular buttons.

You can also download this eclipse project from :
https://drive.google.com/file/d/0B09BIsDTY_AuYVB4ZjJNM3h0ZlE/view?usp=sharing

Steps:
  1. Open Eclipse IDE
  2. Create a new Project
  3. Download Selenium Bindings
  4. Download Sukuli JAR
  5. Right click on you project
  6. Open New>Class


  1. Right click on you project

  1. Open Build Path>Configure Build Path


  1. Open Libraries Tab
  1. Click add External Jars button
  2. Add Following Imports
import java.io.File;
import java.util.concurrent.TimeUnit;


import org.junit.After;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.NoAlertPresentException;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.sikuli.api.*;
import org.sikuli.api.robot.Mouse;
import org.sikuli.api.robot.desktop.DesktopMouse;


  1. Add Selenium and Java Bindings
  2. Paste Following code in Your Class


   @BeforeClass
   public static void setUp() throws Exception {
       wd = new FirefoxDriver();
       wd.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
  }
   
   @Test
   public void TestCase1() throws InterruptedException {
   
   }
   
   @After
   public void tearDown() {
       wd.quit();
   }
   
   public static boolean isAlertPresent(FirefoxDriver wd) {
       try {
           wd.switchTo().alert();
           return true;
       } catch (NoAlertPresentException e) {
           return false;
       }
  }
  1. Open Flash Calculator Link in Browser “http://www.terrence.com/flash/calculator.html “
  1. Take small images of required number and operations like 1.png,2.png , equal.png and multiply.png etc. you can use snipping tool utility for this purpose its pre-installed in Win 7 or greater
Like These


  1. Now Create  function which takes path of image as string and click on that image
Code is:
public void click_Image(String img)
{
s = new DesktopScreenRegion();
target = new ImageTarget(new File(img));
r = s.find(target);


// Create a mouse object
mouse = new DesktopMouse();
// Use the mouse object to click on the center of the target region
mouse.click(r.getCenter());
}


  1. Now add Following code in your test case first navigate to URL by web driver and then click by images which you created example code is
 @Test
   public void register() throws InterruptedException {
    click_Image(“IMG\\1.png”);
    click_Image(“IMG\.png”);
    click_Image(“IMG\\plus.png”);
    click_Image(“IMG\\2.png”);
    click_Image(“IMG\\equal.png”);
   }


Your final code would be:


import java.io.File;
import java.util.concurrent.TimeUnit;


import org.junit.After;
import org.junit.BeforeClass;
import org.junit.Test;
import org.openqa.selenium.NoAlertPresentException;
import org.openqa.selenium.firefox.FirefoxDriver;
import org.sikuli.api.*;
import org.sikuli.api.robot.Mouse;
import org.sikuli.api.robot.desktop.DesktopMouse;


public class testAutomation {
public static FirefoxDriver wd;


ScreenRegion s;
Target target ;
ScreenRegion r;


// Create a mouse object
Mouse mouse ;


public void click_Image(String img)
{
s = new DesktopScreenRegion();
target = new ImageTarget(new File(img));
r = s.find(target);


// Create a mouse object
mouse = new DesktopMouse();
// Use the mouse object to click on the center of the target region
mouse.click(r.getCenter());
}
   @BeforeClass
   public static void setUp() throws Exception {
       wd = new FirefoxDriver();
       wd.manage().timeouts().implicitlyWait(60, TimeUnit.SECONDS);
   }
   
   @Test
   public void register() throws InterruptedException {
    click_Image(“IMG\\1.png”);
    click_Image(“IMG\.png”);
    click_Image(“IMG\\plus.png”);
    click_Image(“IMG\\2.png”);
    click_Image(“IMG\\equal.png”);
   }
   
   @After
   public void tearDown() {
       wd.quit();
   }
   
   public static boolean isAlertPresent(FirefoxDriver wd) {
       try {
           wd.switchTo().alert();
           return true;
       } catch (NoAlertPresentException e) {
           return false;
       }
   }


}

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s