Giter Site home page Giter Site logo

sikulisharp's Introduction

SikuliSharp

NOT MAINTAINED

I am not using this library anymore, and therefore it is not being actively maintained. I will however consider pull requests, but I will not implement new features.

Join the chat at https://gitter.im/christianrondeau/SikuliSharp

Yet another implementation of a Sikuli wrapper for using it in .NET.

Why Another Sikuli Wrapper?

There are already existing wrappers, sikuli-integrator and sikuli4net, but I had trouble running them, they use an additional level of wrapping, they do not seem very active, and especially they used a lot of static classes, which makes it difficult to extend. I then decided to try building an implementation myself.

How to Use

Check these steps first:

  1. Download Java
  2. Download Sikuli 1.1.0:
  3. Create an environment variable SIKULI_HOME that points to the Sikuli folder that contains the .jar files

Here is a simple example:

using(var session = Sikuli.CreateSession())
{
  var pattern = Patterns.FromFile(@"C:\Patterns\MyPattern.png"); 
  if(session.Exists(pattern))
  {
  	  Console.WriteLine("The pattern exists!");
  }
}

You can also simply run a project (1.0.1 and 1.1.0 only, 1.1.4 not supported yet):

Sikuli.RunProject(@"C:\Projects\MyProject.sikuli");

How Does it Work

A SikuliSession launches an instance of the Sikuli interactive script engine using a Jython interactive console. All commands are sent to the console's standard input, and the output is then parsed.

Documentation

Remember that this library simply wraps Sikuli; the same limitations apply. You can't use your computer while tests are running since Sikuli takes control of the mouse, and patterns may require fine tuning (using similarity).

Sikuli

The Sikuli object is the main entry point for all operations. It contains two methods:

  • CreateSession, which returns an ISikuliSession, with which you can execute Sikuli commands
  • RunProject which simply runs a .sikuli project and returns the console output

SikuliSession

All commands must be run within a ISikuliSession.

using (var session = Sikuli.CreateSession())
{
  // Execute commands here
}

All commands run against the ISikuliSession instance. They also can receive a timeoutSeconds parameter. If left empty, commands will wait "forever".

  • session.Exists checks if the pattern exists on the screen
  • session.Click Click to the Point offset distance from the pattern
  • session.DoubleClick Double-click on the pattern if it exists on the screen
  • session.RightClick Right-click to the Point offset distance from the pattern
  • session.Wait tries to click on the pattern if it exists on the screen
  • session.WaitVanish waits for the pattern to disappear from the screen
  • session.Type sends the characters to the application; don't forget to double-escape special characters (e.g. "\\n" should be "\\\\n" or @"\\n")
  • session.Hover Hover to the Point offset distance from the pattern
  • session.DragDrop Drags from a pattern to another
  • session.Find Find best match for pattern
  • session.FindAll Find all matches for pattern
  • session.Highlight Highlights an element on the screen

Patterns

Creating a pattern from a file path

var pattern = Patterns.FromFile(@"C:\Patterns\MyPattern.png"); 

You can also specify a similarity (between 0f an 1f)

var pattern = Patterns.FromFile(@"C:\Patterns\MyPattern.png", 0.6f); 

SikuliRuntime

If you need more functions, you can create your own. Here is an example:

using(var runtime = Sikuli.CreateRuntime())
{
  runtime.Start();

  var result = runtime.Run(
    @"print ""RESULT: OK"" if exists(""C:\\Patterns\\MyPattern.png"") else ""RESULT: FAIL""",
    "RESULT:",
    0d
    );

  Assert.That(result, Is.StringContaining("RESULT: OK"));
}

You must print a string that will show up regardless of whether the test succeeded or not. If you don't provide a timeout and the resultPrefix parameter is not printed in the console, the runtime will hang.

Also remember that this sends Jython to the console. Therefore, you must double-escape strings accordingly.

Contributions

This project is open for contributions through pull requests or feedback. This project is too small to have a contribution guide yet, but usual rules apply: make sure all tests work and try to keep the same coding style.

Here are some improvement ideas:

  • Get rid of the yellow banner (should be solved in 1.1.0)
  • Implement other sikuli functions
  • It may be interesting to provide other IPattern implementation, e.g. embedded resources
  • If possible, install Sikuli at runtime... not sure about this one though. Maybe a dedicated function such as Sikuli.InstallSikuli();

License

Copyright (c) 2019 Christian Rondeau, The MIT License

sikulisharp's People

Contributors

bradley8 avatar christianrondeau avatar exoow avatar flowerinthenight avatar gitter-badger avatar mnkjadhav avatar nightshadow92 avatar tokkyuuressha avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

sikulisharp's Issues

Issue when using Session.Click(pattern, timeout) function with timeout > 0f

Hi,
I have a quick look in the code and see that it maybe something wrong for this
[code]
protected bool RunCommand(string command, IPattern pattern, float timeoutInSeconds)
{
pattern.Validate();

		var script = string.Format(
			"print \"SIKULI#: YES\" if {0}({1}{2}) else \"SIKULI#: NO\"",
			command,
			pattern.ToSikuliScript(),
			AddTimeout(timeoutInSeconds)
			);
           ...
           }

[code]
This will ad a key modifier code into click function in stead of a timeout for click action.
Please help me check.

Support to set delay time

Hi Chris,
Could you help to create functions to support setting the speed value like mouse move, text typing etc....

Thanks and Best Regards,
Tai Nguyen

RunProject with 2 arguments is not properly working

Hi there,

Thank you for creating this wrapper.

I have a sikuli script that automates installation of firefox. It accepts command line argument - filename. In my c# code, I tried to call it with Sikuli.RunProject(args[0], args[1]) where
args[0] = path of firefox.sikuli
args[1] = "--args firefox-installer-here"

Upon running, i noticed that app focus remains the "cmd". I tried to edit my script to hardcode the filename and just call the Sikuli.RunProject(args[0]) and it runs successfully. Any recommendation?

Session.Type misses characters randomly

Hi Christian,

I've been using your Sikuli wrapper for couple of months now and it's working out pretty well for us.

However, there's this issue that creates a lot of problem here..

We've noticed that randomly Session.Type command will miss entering characters..

Example: Session.Type("abcdef") would type in "abcf" randomly and would work correctly other times.
I've tried to add a sleep also in between the click and type commands to see if it's related to timing, but it did not help.

Are you aware of this issue ? I am using SikuliSharp 1.0.3 release. Thanks much.

SikuliSharp always timeouts on Exists while same image works for other implementation

So... I was hoping to be able to use SikuliSharp but it fails to find the Google Logo in my test below.
Could be my mistake, but the exact same image and code works for the Sikuli4Net wrapper.

The SikuliSharp code timeouts after 7 seconds (as set in the method call) so everything appears to "work" properly, expect that the image isn't recognized while it should be (as proven by the other wrapper).

SikuliSharp code:

using (var session = Sikuli.CreateSession())
    {
        var gLogo = Patterns.FromFile(AppDomain.CurrentDomain.BaseDirectory + "\\Resources\\google-logo.PNG");
        var gSearch = Patterns.FromFile(AppDomain.CurrentDomain.BaseDirectory + "\\Resources\\google-search.JPG");
        var xResult = Patterns.FromFile(AppDomain.CurrentDomain.BaseDirectory + "\\Resources\\abc-detail.JPG");

        session.Exists(gLogo, 7);
        session.Wait(gSearch, 7);
        session.Type("abc");
        session.Exists(xResult, 5);
    }

Sikuli4Net code:

        Screen gHome = new Screen();
        Pattern gLogo = new Pattern(AppDomain.CurrentDomain.BaseDirectory + "\\Resources\\google-logo.PNG");
        Pattern gSearch = new Pattern(AppDomain.CurrentDomain.BaseDirectory + "\\Resources\\google-search.JPG");
        Pattern xResult = new Pattern(AppDomain.CurrentDomain.BaseDirectory + "\\Resources\\abc-detail.JPG");


        Assert.IsTrue(gHome.Exists(gLogo, 7));
        gHome.Type(gSearch, "abc");
        Assert.IsTrue(gHome.Exists(xResult, 7));

Support for sikuli 2.0.5 api as it offers better OCR and image recognition

Hi @christianrondeau,
I have been using the SikuliSharp package and it is quite powerful and fast. However, am facing issues while trying to extract text from the image.

I am using this command: var result = runtime.Run(@"print ""RE:""+ Region(553,548,91,24).text()", "RE:", 10).

Its extracting but the text is not accurate. I tried the same text extraction using the Sikuli IDE (version 2.0.5) and it is perfectly extracting it. As the latest Sikuli is coming with better OCR capabilities, Need your suggestion/help to invoke the latest sikuli api available here (https://raiman.github.io/SikuliX1/downloads.html) Appreciate your help. Thanks

SikuliRuntime command example

Hello,

I would like to get the bounds (coordinates) of a control after an exists command. As this is not available directly, I used the getLastMatch in the IDE and got a successful result. However when fire the command using the SikuliRuntime, I cannot get it to work. can you help me out. Here is the function I am trying to use:

` public void RunCommand()
{
using (var runtime = Sikuli.CreateRuntime(150))
{
runtime.Start();
var img = Patterns.FromFile(@"F:\Shashank\tools.png", .7f);
var result = runtime.Run(

         @"print ""RESULT:"" "+"getLastMatch() "+  "if exists(Patterns.FromFile("+"F:\\Shashank\\tools.png"+", .7f)" + ") else "+ "\"RESULT: FAIL\""   ,
          "RESULT:",
          0d
          );

    }
}`

The pattern img, will have a different image everytime.

SikuliSharp support for Sikuli 2.0.2

Hi I have enjoyed the SikuliSharp wrapper for a while for Siluli 1.0. We need to upgrade this to Sikuli 2.0.2 and was wandering if someone can help us with the updated packages

Catchup with SikuliX 1.1

In the new version the library and scripts names have changed, also the environment variable's name. Other stuff may have changed as well

Time out Exception

Hay, I just use your SikuliSharp, have some question.
Why I using
using (var session = Sikuli.CreateSession())
{
var pattern = Patterns.FromFile(@"C:\Patterns\MyPattern.png");
session.Exists(pattern);
}

And Sikuli.CreateSession() will throw time out exception?

Issue when using Session.<some action as Click, Wait...>(pattern) without specify timeout

Hi christianrondeau,
I have some doubt that when I run the code as below if the pattern cannot be found, my test application seem hangs in this state and cannot process to next action.
Session.Click(pattern);
I expected that it would return "SIKULI# : NO" as the script you specified for run in the RunCommand functions but it would not, after do some checks I found that some sikuli actions like "click","wait","exists"... would throw an exception FindFailed if the pattern cannot be found on the screen instead of return anything, that maybe the cause why the RunCommand function would not return.

Can you help to check this again.

Regards,

Sikuli.RunProject(@"C:\Project Path Contains Spaces\MyProject.sikuli") doesn't work if the project path contains spaces.

Is it possible to allow spaces in project path to work by changing this portion of the code ...

public ISikuliVersion WithProject(string projectPath, string args)
{
Arguments = string.Format("-jar \"{0}\" -r {1} {2}", _jar, projectPath, args);
return this;
}

... to this?

	public ISikuliVersion WithProject(string projectPath, string args)
	{
		Arguments = string.Format("-jar \"{0}\" -r \"{1}\" {2}", _jar, projectPath, args);
		return this;
	}

Not able to use SikuliSharp in Visual Studio 2017

  1. Downloaded the Java
  2. I do not find ◦sikulixapi.jar file anywhere hence I downloaded sikulixapi-1.1.0
  3. I changed the below if condition to get jar file folder added to Sikuli_Home
    if (jarFilenames.Contains("sikulixapi-1.1.0.jar") && jarFilenames.Contains("jython-standalone-2.7.1.jar"))
    but still getting below issue:
    '[error] RunTimeINIT: *** terminating: libs to export not found on above classpath: /sikulixlibs/windows/libs64.

Could any one please help me with this issue.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.