raiman/sikulix1

SikuliX version 2.0.0+ (2019+)

******** Paused (not available) till July 2024 ********

---

---

What is SikuliX
SikuliX automates anything you see on the screen of your desktop computer running Windows, Mac or some Linux/Unix. It uses image recognition powered by OpenCV to identify GUI components and can act on them with mouse and keyboard actions. This is handy in cases when there is no easy access to a GUI's internals or the source code of the application or web page you want to act on. More details

Great thanks for the new logo and all the help with the new webpage to @Waleed Sadek

---

2.0.6 (branch release_2.0.x) preparing for release - snapshots available

Latest Upload: April 17th, 2023

Direct IDE downloads      > for Windows <    > for macOS Intel <    > for macOS Silicon Mx <    > for Linux <

You get files like sikulixidemac-2.0.6-20210708.194940-1.jar, which you can place wherever you want and rename them to whatever you want.

JAVA: must be Java 11 or later (best places to get it: Eclipse Temurin or Azul)

OCR (macOS/Linux): now using Tess4J/Tesseract 5 - have a Tesseract 5.x ready (tesseract runs on commandline)

OpenCV Support: Windows/macOS have it bundled - for Linux you have to make it ready yourself

more information coming sooner or later ;-)

---

2.1.0 (branch master) currently not useable - development suspended

---

Latest stable version is 2.0.5 (still works with Java 8, does not run on Mac mX machines)

Important: Read about changes/issues/enhancements

List of fixes

Get SikuliX ready to use

For use in Java Maven projects the dependency coordinates are:

<dependency>
  <groupId>com.sikulix</groupId>
  <artifactId>sikulixapi</artifactId>
  <version>2.0.5</version>
</dependency>

---

My Development environment

• Java 17 (current JDK LTS release)
• Source and target level for Java is version 11
• Maven project
• Windows 11 latest (Pro 64-Bit)
• latest macOS 12 (Monterey) on Intel and M1 machines
• Ubuntu latest LTS version running in Oracle VM VirtualBox on Windows 10
• Using IntelliJ IDEA CE in all environments

---

Contributions are welcome and appreciated

• for bugreports and requests for features or enhancements use the issue tracker here
• for bugfixes related to the latest release version you should create a pull request against the release branch (currently release_2.0.x), so your fix will be in the next bug-fix release (see milestones).
• for smaller bugfixes and/or feature enhancements related to the running development (currently branch master as version 2.1.0-SNAPSHOT and dev_... branches) you should create a pull request against the target branch
• a pull request should target only one branch. It is the resposibility and job of the maintainer to apply the changes to other branches in case
• for more complex revisions and/or enhancements you should ask for a development branch together with a short description of your ideas

Please respect the following rules and guidelines when contributing

• Start with smaller fixes. E.g. choose an issue from the issue tracker and try to fix it. Or fix issues you encounter while using SikuliX.
• Only fix cosmetic stuff if it's related to an issue you want to fix.
• Before you change stuff like dependencies / overall code style and so on, talk with the maintainer beforehand.
  Sometimes there is a a reason that things are as they are (... and sometimes not :-)).
• Try to accept the individual coding styles of the acting contributors, even if some of the stuff might be sub-optimal in your eyes.
  But feel free to talk about your ideas and the reasons behind.

sikulix1's Issues

Where can I find sikulixapi 1.1.4 final in Maven?

The documentation tells:

The repository URL:http://oss.sonatype.org/content/groups/public
The coordinates are:
com.sikulix
sikulixapi
1.1.4-SNAPSHOT

But where can I find 1.1.4 final?
As you can see, in Maven central, there is only till version 1.1.2.

If I try to use 1.1.4-SNAPSHOT:

checksum format error: ~/.cache/coursier/v1/http/oss.sonatype.org/content/groups/public/com/sikulix/sikulixapi/1.1.4-SNAPSHOT/sikulixapi-1.1.4-SNAPSHOT.pom.sha1

Finder with Image in Image throws NPE on next()

Below is the example code. Problem is in Finder.java 484 where _region gets added alas is NULL
I suspect the next() method would need some overhaul regarding the optional private fields.
In my scenario there would be also no parentScreen so line 491 would be also problematic I assume...

BufferedImage img = .......
 BufferedImage screenshot = .......

    org.sikuli.script.Image i = new org.sikuli.script.Image(screenshot);
    Finder finder = new Finder(img);
    finder.find(i);
    while (finder.hasNext()) {
      try {
        Match m = finder.next(); // PENG here comes the NPE
        System.out.println("Score:"+ m.getScore());
        if (m.getScore() >= wantedScore) {
          return;
        }
      } catch (Exception e) {
        System.out.println("ERROR while retrieving matcher!");
        e.printStackTrace();
      }
    }

Happy for any suggestion son how to overcome ;) FWIW I am using it in combination with AssertJ's Swing module to do Java Swing GUI testing and am retrieving screenshots of the gui via AssertJ's ScreenshotTaker to be able to restrict it to specific components...
br
Thomas

Run from Step or Line Number

Would it be possible to add some sort of "Run from this step" or "Run from this line number" feature?

For instance, you're developing a script and it fails halfway through, I don't want to start from the beginning each time.

Or, perhaps I'm just not efficiently using it and I should develop or code differently?

bug on captureDeviceScreenMat

Hi i found it!

if (y + actW > currentH) { actH = currentH - y; }
please fix it! :)

Mat <-> BufferedImage converters do not work for subimages of BufferedImages

Using BufferedImage.getSubimage(...) to limit the area for Finder does not work,
Using the converters (forth and backwards) on such images show that the original image gets distorted.

Workaround is to create a new BufferedImage of Type TYPE_INT_RGB with limiting width/height and drawing the large image onto that....

ADBScreen class not available currently

This might be more of a question. Somewhere in the site I saw reference to ADBScreen to work with android device / emulator but I couldn't import it from the jar.

Am I missing something ?

[1.1.4] CvException when using find() in a project upgraded from 1.1.3 due to not supported png-image-type

Being eager to make use of the both the improved OCR and the transparent masking, this week I upgraded an existing, in-progress project from 1.1.3 to 1.1.4 to attempt to make use of the new features.
Noticing that everything required is now bundled into one .jar file, I removed my old Sikuli 1.1.3 install directory & jar file, and imported the nightly build via maven. However, I can no longer run any of my tests using sikuli, because any attempt to find an image results in this error:

CvException [org.opencv.core.CvException: cv::Exception: OpenCV(3.4.2)  

C:\build\3_4_winpack-bindings-win64-vc14-static\opencv\modules\imgproc\src\templmatch.cpp:1102: error:  
(-215:Assertion failed) (depth == 0 || depth == 5) && type == _templ.type() && _img.dims() <= 2 in  
function 'cv::matchTemplate'
]

I'm unclear what could be causing this issue, as I'm not very clued up on the underlying opencv libraries, but I was previously able to make use of this exact image before so I don't believe that it is the issue unless a significant requirement for saved images has changed.

Eager to get to the bottom of this as, like I said, I'm excited to make use of the new stuff.

click function: strange meta keys in log and does not always click

When using SikuliX (1.1.4) inside a docker image (https://gitlab.com/romangrothausmann/dind-sikuli) for automated fog testing in GitLab CI with Xvfb, I noticed that the click function does not always lead to a click event on the running GUI even though the mouse pointer moved to the correct click position.
I found some similar reports but only RaiMan/SikuliX-2014#260 seems related. By repeating the GL CI job a few times (3-6), some succeed all right. I observe this problem also on a slower laptop running sikuli on a default X-server (even if running the sikuli script in "slow motion").
It noticed that often (possibly always) the logs report some meta keys being held while clicking (even though that is not part of the sikuli script), e.g.:

[log] CLICK on L(955,281)@S(0)[0,0 1280x1024] (591 msec)
[log] Meta+Alt Graph+CLICK on L(300,953)@S(0)[0,0 1280x1024] (728 msec)
[log]  TYPE "Input.txt"
[log] CLICK on L(250,972)@S(0)[0,0 1280x1024] (570 msec)
[log] Meta+Alt Graph+CLICK on L(300,953)@S(0)[0,0 1280x1024] (601 msec)

However, I do get the very same log messages even for jobs that were executed correctly and succeeded.

Any ideas what could be causing this click effect variation?

runsikulix command not available after setup on MacOS 10.11.6

I dowloaded and installed v. 1.1.2 SikuliX (the repo for that version is archived so I can't post issues). I can use the IDE to create and run scripts, but I can't get it to work from the command line, as the runsikulix command doesn't work.

$ runsikulix
-bash: runsikulix: command not found

I also tried doing it this way:

open ./Applications/SikuliX.app -r ./agilent/test.sikuli

and I got this response:

open: invalid option -- r

Can you give me a pointer? Thanks for the help and the tool (I think I'll be using it all the time now)

compiler errors when running sikulisetup on Ubuntu 18.04

java -jar sikulixsetup-1.1.3-20180711.082147-40-forsetup.jar hangs in the RunBuild script compiling sources.
When I start /home/testy/.Sikulix/sikulixlibs/linux/runBuild I get the following error messages:

/home/testy/.Sikulix/sikulixlibs/linux/Build/Source/vision.cpp: In static member function ‘static void sikuli::Vision::initSParameters()’:
/home/testy/.Sikulix/sikulixlibs/linux/Build/Source/vision.cpp:489:4: error: ‘_sparams’ was not declared in this scope
    _sparams["OCRLang"] = "eng";
    ^~~~~~~~
...
----- pyramid-template-matcher.cpp
In file included from /home/testy/.Sikulix/sikulixlibs/linux/Build/Source/pyramid-template-matcher.h:14:0,
                 from /home/testy/.Sikulix/sikulixlibs/linux/Build/Source/pyramid-template-matcher.cpp:7:
/home/testy/.Sikulix/sikulixlibs/linux/Build/Source/vision.h:84:16: error: ‘map’ in namespace ‘std’ does not name a template type
    static std::map<std::string, float> _params;
                ^~~

This is with g++ (Ubuntu 7.3.0-16ubuntu3) 7.3.0
and also the same with g++ (Ubuntu 6.4.0-17ubuntu1) 6.4.0 20180424

The setup jar hangs, as the compiler is run behind pipes, so that its stderr is redirected to some other process, that apparently does not read (all of) the output. Guessing that much from poking around with strace.

[1.1.4] sikulixapi.jar should not use system.exit() - now throws SikuliXception instead --- revised build #172

If something goes wrong with the Sikulax jar API initilialisation, it makes a call to 'system.exit(...)'.

It is a major issue, because it is killing at the same time each application currently launched and running on the same JVM.

Rather than a 'system.exit' an exception should be raised, and let the host decides what to do with it.

[1.1.4] Application won't launch if parameters are supplied using setUsing(parameterString)

When Launching my target application without parameters, it will launch fine. However, when supplying the parameters needed for my test cases, I find that the application does not launch. It appears that this is because the parameters are being split wrongly, despite the fact that the App instance has them correct.

The App instance in question has the following:
C:\Program Files\Unity\Editor\Unity.exe as the executable
-createProject C:\TestUnityProject\ as the options

When launched with the above options I am greeted with the following dialogue box:

I believe this to be a recent change, as my testbed uses the latest version available in Maven, and this was not true last time I ran this project before taking holiday (2 weeks ago)

Missing part of source code

Hi Raiman,
I'm using your source code by download in github. Because I want to use alone sikulixapi.jar only not contain extend libraries (slf4j include sikulixapi.jar file is not working in my project and if I use other slf4j, it will conflict) so I download source to build not include slf4j, but com.sikulix.vnc.*; is missing, I can't find it and build .jar file, can use show me where I can find it or share me sikulixapi.jar file not include slf4j. Thanks

vidstige jadb dependency version no more available in jitpack rep

While trying to use sikulix api the following dependency cannot be found anymore

<dependency>
      <groupId>com.github.vidstige</groupId>
      <artifactId>jadb</artifactId>
      <version>-v1.0-g94ebf38-23</version>
      <!--License Apache 2-->
</dependency>

After some investigation it seems the corresponding release / tag is no more there in the github repo and the build is not available anymore in jitpack.io repository.

Thus preventing use of sikulix API 1.1.3+ in code.

How to not stop sikulix

Apoligice for my english but I hate use the translator ;-) . the question is:
How can I do to not stop and close the script when the script fails to click an image.
Im using Jython sikulix lastest version.

I have a code like this.
(Scan ratio 0.1)

While exists(image,0.3):
If exists(image,0.3):
Click(image)

I wrote the sentence "if" couse without it takes like 2 times to fail to click it couse sikulix cant find it.

Then I changed that and I got like 20min of running without problems but still failing sometimes to click the image.

I changed mouse speed to 0 when start the "while" and setup 0.5 when finish but still getting the problem that cant find it and the script stops.

I want that if don't exists the click then jump to the next line code and dont close the script. Maybe if is not possible jump to the next lane code just restart the script and start from cero. My code have so much click sentences and the images are not freezed on the screen and can appears or disappears in an instants and thats couse my script stops.

Thank you in advantage ^^

[1.1.4] exit status of script not passed to the shell executing java --- fixed 2019-01-30

Using SikuliX (1.1.4) inside a docker image (https://gitlab.com/romangrothausmann/dind-sikuli) for automated fog testing in GitLab CI (many thanks for Sikuli and its maintenance, it's just amazing what can be done with it!), I noticed that when executing a script with java -jar /opt/sikulix.jar -r test.script/ the exit code that the shell receives (e.g. echo $? is not correlating to the exit code of the script. For example if the script has an exit(10) the log says: [info] Exit code: 10 but a following echo $? reports 0. I found a few related issues on this from old:
https://answers.launchpad.net/sikuli/+question/193835
https://answers.launchpad.net/sikuli/+question/203783
sikuli/sikuli#38
but none seems to give an answer how to pass on the exit status of a script to the shell executing java. I had a look at the source (

SikuliX1/IDE/src/main/java/org/sikuli/scriptrunner/ScriptingSupport.java

Line 448 in 98ac987

System.exit(exitCode);

) but can't see where it gets lost.
The exit code to the shell is finally used by GL CI to determine if the job/test succeeded, so my current workaround is to grep the output from java for [info] Exit code: 0 but in case of multiple return values that's not ideal.

Should a script be called differently for such cases or is there another way to pass on the exit code of the script to the shell?

Textrecognizer: setPSM: should set psm

SikuliX1/API/src/main/java/org/sikuli/script/TextRecognizer.java

Line 144 in 911be91

tess.setOcrEngineMode(this.oem);

should be: set psm instead of seting oem there.

[1.1.4] Left side bar missing

on 1.1.2 there is a side bar with quick actions.

Settings
Find
Mouse Actions
Keyboard Actions

There is a menu checkbox under View for this.

On 1.1.4, the view menu options is missing, and the left sidebar is also missing.

[1.1.14] No installed runner supports (text/python)

I tried upgrading to 1.1.14 (sikulix.jar) and when I open it I get a prompt indicating:

... serious problem detected!
No installed runner supports (text/python)
Trying to run the script will crash IDE!

IDE opens and I can see the last script I opened in 1.1.12, when when I click RUN I get:

System.out is broken (console output)! you will not see any messages anymore! Save your work and restart the IDE! You may ignore this on your own risk!

After I click OK, the window disappears, but java.exe and javaw.exe are still running.

Questions:

1. Are there other pre-requisites needed to install beyond what 1.1.12 installed?
2. How do I resolve this?

What I have:

1. JAVA installed (1.8.0_181)
2. Python 2.7 installed

[1.1.4] IDE: Jython: Interactive sessions no longer supported --- use an installed Jython

Sikuli version: 1.1.4, build 2018-08-23_12:18
Platform: Windows 8.1, 64-bit

>java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

I seem to be unable to get a functioning interactive Sikuli session with the latest nightly build. The session starts, but it does not accept any keyboard input. The only way to exit is via CTRL-C. After forcing termination of the interactive session, I can see that all of my attempted input is actually fed as input to the Windows command prompt instead of Sikuli, though this is hidden until the Sikuli interactive session is terminated.

Resulting output:

C:\SikuliX\1.1.4~nightly20180823_1218>java -jar sikulix.jar -i -d
Hello, this is your interactive Sikuli (rules for interactive Python apply)
use the UP/DOWN arrow keys to walk through the input history
help()<enter> will output some basic Python information
... use ctrl-d to end the session

[The session hangs here regardless of keyboard input, except until CTRL-C is pressed.]

C:\SikuliX\1.1.4~nightly20180823_1218>
C:\SikuliX\1.1.4~nightly20180823_1218>
C:\SikuliX\1.1.4~nightly20180823_1218>
C:\SikuliX\1.1.4~nightly20180823_1218>
C:\SikuliX\1.1.4~nightly20180823_1218>
C:\SikuliX\1.1.4~nightly20180823_1218>
C:\SikuliX\1.1.4~nightly20180823_1218>help()
This command is not supported by the help utility.  Try "() /?".

C:\SikuliX\1.1.4~nightly20180823_1218>
C:\SikuliX\1.1.4~nightly20180823_1218>
C:\SikuliX\1.1.4~nightly20180823_1218>^D
'♦' is not recognized as an internal or external command,
operable program or batch file.

C:\SikuliX\1.1.4~nightly20180823_1218>

Debug log:

[2657 debug (8/23/18 12:49:36 PM)] Debug: setLogFile: C:/SikuliX/1.1.4~nightly20180823_1218/SikulixLog.txt
[2672 debug (8/23/18 12:49:36 PM)] Runner: commandline: 
[2672 debug (8/23/18 12:49:36 PM)] --- Sikuli parameters ---
[2672 debug (8/23/18 12:49:36 PM)] 1: -i
[2672 debug (8/23/18 12:49:36 PM)] 2: -d
[2672 debug (8/23/18 12:49:36 PM)] ScriptingSupport: initScriptingSupport: enter
[5750 debug (8/23/18 12:49:39 PM)] JythonSupport: init: starting
[5750 debug (8/23/18 12:49:39 PM)] JythonSupport: init: success
[5750 debug (8/23/18 12:49:39 PM)] JythonSupport: ***** Jython sys.path
 0: C:\Users\Brett Keller\AppData\Roaming\Sikulix\Lib
 1: C:\Users\Brett Keller\AppData\Roaming\Sikulix\Extensions\Lib
 2: C:\Users\Brett Keller\AppData\Roaming\Sikulix\Extensions\jython-standalone-2.7.1.jar\Lib
 3: __classpath__
 4: __pyclasspath__/
[5750 debug (8/23/18 12:49:39 PM)] JythonSupport: ***** Jython sys.path end
[5860 debug (8/23/18 12:49:40 PM)] Jython: sikuli: Sikuli: starting init
[7032 debug (8/23/18 12:49:41 PM)] Mouse: init start
[7532 debug (8/23/18 12:49:41 PM)] Mouse: init end
[7610 debug (8/23/18 12:49:41 PM)] RunTimeIDE: loadLib: WinUtil.dll
[7688 debug (8/23/18 12:49:41 PM)] Jython: requested to use as default region: S(0)[0,0 1440x900]
[7891 debug (8/23/18 12:49:42 PM)] Jython: sikuli: Sikuli: ending init
[7907 debug (8/23/18 12:49:42 PM)] JythonScriptRunner: running Jython 2.7.1 
[7907 debug (8/23/18 12:49:42 PM)] ScriptingSupport: initScriptingSupport: added: jython
[7907 debug (8/23/18 12:49:42 PM)] ScriptingSupport: initScriptingSupport: added: JavaScript
[7907 debug (8/23/18 12:49:42 PM)] ScriptingSupport: initScriptingSupport: exit with defaultrunner: jython (py)
[7922 debug (8/23/18 12:49:42 PM)] Jython: requested to use as default region: S(0)[0,0 1440x900]
[30782 debug (8/23/18 12:50:04 PM)] Sikulix: endNormal: 0
[30782 debug (8/23/18 12:50:04 PM)] Sikulix: cleanUp: 0
[30782 debug (8/23/18 12:50:04 PM)] RunTimeIDE: final cleanup
[30797 debug (8/23/18 12:50:04 PM)] FileManager: deleteFileOrFolder:
C:\Users\BRETTK~1\AppData\Local\Temp\Sikulix_1703971813

multi monitor issue

Hi team,

i want to extend my primary screen display by using dragdrop option in sikuli to secondary screen, does sikuli 1.1.3 supports this feature?

ex: if i am running a youtube in chrome and i want to drag and drop this chrome tab to secondary monitor, when i tried this sikuli didn't allow me to capture image to drop tab in secondary screen

How can we try to resolve this issue?

Can't run jython standalone script

I can't run jython standalone script, I'm using sikulixapi.jar for version 1.1.4.
My script has following lines at the beginning of script:

import org.sikuli.script.SikulixForJython
from sikuli import *

But when I'm trying to execute this script with following command jython -Dpython.path=path-to-jar/sikulixapi.jar script.py I got this error:
[error] RunTimeINIT: *** terminating: no valid Java context for SikuliX available (java.security.CodeSource.getLocation() is null)

[1.1.2] [1.1.4] Screen.capture not allowing region + filename args

Unless I am misunderstanding the usage of the 1.1.2 feature in documentation, this is not allowing me to send in the filename. It appears that the text or region is required, along with the filename, but the path is optional. That's 2 arguments. The error says I must have 0, 1, or 4 arguments.

filename = screen.capture(screen.getBounds(), "file.png")

[error] Error caused by: Traceback (most recent call last): File "C:\sikulix\EndUserLocation.sikuli\EndUserLocation.py", line 12, in filename = s.capture(s.getBounds(), "file.png") TypeError: capture(): expected 0-1 or 4 args; got 2

https://sikulix-2014.readthedocs.io/en/latest/screen.html?highlight=screenshot#Screen.capture

capture(region | text, [path, ]name)
only available in Python scripting

Parameters:
| text (region) – an existing region object or text to display in the interactive mode.
path – a path to a folder where the image is stored
text – name of the image file (.png can be omitted)
Returns:
the absolute path of the stored image as path/name.png or None if no success

[features request] Can keep "Message" pannel below "Code" in IDE 1.1.4 ?

Hi @RaiMan ! 🥇
SikuliX is very impressive and intuitive whatever the powerful complex guts ! Thanks a lot, this is a great tool, a dream-come-true ! A few days and it works as a charm, fast and reliable !
I started with SikuliX 1.1.2 IDE and Jython on win7, with enough Jython background to feel easy with it.
I write a Lua extension for the VLC-Media Player and running UI tests with SikuliX is a bliss. I was using AutoHotkey before and I spare a lot of aspirin :-) Thanks a lot !

Yesterday I began with the SikuliX 1.1.4 version, and my code needs some rewriting ...

Request 1 : Would you mind keeping an option to allow the user to have the "Message" panel below the "Code" panel, like in version 1.1.2 ? Some reasons :

• Creating many small(er) regions to get matches faster and accurate involves longer names to keep the code being self-documenting; also, inline coding with comprehensions is known to execute faster than for-loops : both result in many "horizontal" code chunks, and the IDE don't allow line-warping.
• Large horizontal screen-captures involve a large display width too.
• The current panel hides so much that it gets cumbersome to show|hide with a mouse-double-click, when it was easy to do this way with the previous version.
• The debug messages are quite "horizontal" too and IMHO they fit much better in a full width panel ( on my laptop display ).

Request 2 : Would it be possible to have an option to reload myfile.py in the IDE ? ( the whole myfile.sikuli project reloaded, in fact ... )

While SikuliX 1.1.4s' features sound very appealing, the IDE is still quite rough ! A Reload script menu entry, just bellow the Reset entry would be of great value : setting graphical objects and executing inside the SikuliX IDE and writing the "kitchen code" in "our" favorite code editor ! e.g. notepad++ updates nicely files that are edited by some other apps.
Sound nice for the intermediate user with a poor machine that < don't want to | cannot > deal with memory-hogs IDEs ( even PyCham is greedy on my laptop and wont do the job that SikuliX does ), without having to implement the ugly "copy-paste framework" :-)

I read at 010-command-line that there is an option to send log messages to some other buffer with the --logfile option, I will do that, but it it is still valuable to stay inside the same IDE ...

Thank you again for making this complex machine be so easy to use.
Please don't forget : laptop users work on smaller screens :-)
Best regards,
Stan.

HighDPI support

see bug:
https://bugs.launchpad.net/sikuli/+bug/1730645

some insights with Java 8 and 13

If you know the scaling factor, that has to be applied to all images, to work in the new environment, then you can use
Settings.AlwaysResize = <factor>
where <factor> is a decimal denoting the scaling to be applied.
Internally the scaling will be applied to every image before it is used in search.

Tested on Windows 10:

• with Java 8 having taken images with a scaling factor of 100% (Windows screen settings) a factor of 1.2 helps, when running the script with scaling of 125%.

• with Java 13 nothing has to be done in this scenario. The search works and even the mouse position is correct.

Still I do not have an environment to test with a hiDPI display.

---

original statement

Um, so it looks like the program doesn't support scaling right. Please add a way to scale the GUI so it is usable on highDPI displays. I've included a screenshot below:

https://i.imgur.com/TjJ2cAd.png

warning message window 10 and java 10.0.1

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.sikuli.ide.CloseableModernTabbedPaneUI (file:/E:/p-tools/sikulix/sikulix.jar) to constructor javax.swing.plaf.basic.BasicTabbedPaneUI$Actions(java.lang.String)
WARNING: Please consider reporting this to the maintainers of org.sikuli.ide.CloseableModernTabbedPaneUI
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

java 10.0.1 2018-04-17
Java(TM) SE Runtime Environment 18.3 (build 10.0.1+10)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.1+10, mixed mode)

Microsoft Windows [Version 10.0.17134.137]

1.1.3-SNAPSHOT

Its not available at sonatype.

Best Regards

[1.1.4] setAutoWaitTimeout apparently not working

Sikulix doesn´t wait the specified time anymore; immediately cancels when a picture is not on screen.

IDE module not compiling with JDK 1.8 due to lack of desktop package in awt

After checking out and building the SikuliX1 1.1.3 all modules I am getting a compilation error on this line

SikuliX1/IDE/src/main/java/org/sikuli/idesupport/IDEMacSupport.java

Line 10 in dc855c6

import java.awt.desktop.*;

As I understood the source and compiled code is compatible with JDK 1.8 but seems this package was introduced in JDK 9 . May be its simply a documentation issue that we need to build the project with JDK 9 or higher ? not sure.

VNC is broken since 1.1.3

VncScreen.java:40

 public static VNCScreen start(String theIP, int thePort, String password, int cTimeout, int timeout) throws IOException {
    VNCScreen scr = null;
    if (RunTime.get().runningMac && !RunTime.get().isJava9("VNCScreen not yet working on Mac")) {
      scr = new VNCScreen(VNCClient.connect(theIP, thePort, password, true));
      screens.put(scr, scr.client);
    }
    return scr;
  }

method start always return null on non Mac machine

Error when using sikulix from Maven repository

Version 1.1.1 was working fine.
But version 1.1.2 fails in this way:

java.lang.ExceptionInInitializerError
  org.sikuli.script.Sikulix.cleanUp(Sikulix.java:459)
  org.sikuli.script.Sikulix.endError(Sikulix.java:415)
  org.sikuli.script.RunTime.initSikulixOptions(RunTime.java:1688)
  org.sikuli.script.RunTime.get(RunTime.java:310)
  org.sikuli.script.RunTime.get(RunTime.java:122)
  org.sikuli.script.RunTime.get(RunTime.java:337)
  org.sikuli.util.JythonHelper.<clinit>(JythonHelper.java:23)
java.lang.NullPointerException
  org.sikuli.script.RunTime.makeFolders(RunTime.java:804)
  org.sikuli.script.RunTime.libsExport(RunTime.java:913)
  org.sikuli.script.RunTime.libsLoad(RunTime.java:844)
  org.sikuli.script.RunTime.loadLibrary(RunTime.java:1019)
  org.sikuli.android.ADBScreen.<clinit>(ADBScreen.java:21)
  org.sikuli.script.Sikulix.cleanUp(Sikulix.java:459)
  org.sikuli.script.Sikulix.endError(Sikulix.java:415)
  org.sikuli.script.RunTime.initSikulixOptions(RunTime.java:1688)
  org.sikuli.script.RunTime.get(RunTime.java:310)
  org.sikuli.script.RunTime.get(RunTime.java:122)
  org.sikuli.script.RunTime.get(RunTime.java:337)
  org.sikuli.util.JythonHelper.<clinit>(JythonHelper.java:23)

I'd be happy to use a newer version, but they aren't in any Maven repo.

BTW, it has been hard to use sikulix2tigervnc-2.0.0-SNAPSHOT, because it isn't published.

Java API: findLine failing to convert to Match type --- fixed 2018-11-24 build #131

Whilst trying to make use of the Region.findLine() method I'm noticing that some of my calls to findLine() are giving me the following error:

java.lang.ClassCastException: org.sikuli.script.Finder cannot be cast to org.sikuli.script.Match
	at org.sikuli.script.Region.findLine(Region.java:2746)

Looks like findLine is struggling to cast the result of doFindText() to Match in some cases, I have a hunch that it's only when the searched for text is not found but that is unconfirmed.

[1.1.4] error RunTimeAPI on VM native libraries not found

Hello again!

I got problems using sikulixapi.jar for textrecognition on a vm.

[error] RunTimeAPI: *** terminating: TextRecognizer: start: native libraries not found.

On my local pc it´s working fine.

System properties of the vm:

WIN7 Prof 64bit, SP1, Java Version: 1.8.0_181 Java SE Runtime Environment, jruby 9.2.0.0

It´s definitely no occasional problem; all my vms show the same behavior. But I can´t except for my windows version identify differences concerning the system properties.

Problem not solved yet ^^

Sorry for open a new issue but the problem still happening.

If you go to the closed issue you will see my last post but I give a summary:

Is possible when sikulix fails to click, find or wait, jump to the next line code? And not exit the script. Or maybe start from 0 the script.

Example:

Click(image 1)
Find(image 2)
Wait(image 3, 40)

If image 1 is not available jump to find, if find its over and image 2 not exists jump to wait, etc. Just not close the scripts and continue running. hahaha

[1.1.4] Can't export project as runnable jar --- feature retracted until new solution

I tried to export my sikuli project as runnable jar, but got this error:
[error] FileManager: makingScriptJar: sikulixapi.jar missing. run setup first!
I treid to add sikulixapi.jar to directory with my sikulix.jar, but it didn't help me.
I'm using sikuli 1.1.4

[NotAnIssue] SikuliX @ chocolatey

Hi!

If it's not too much trouble for you, could you please update the chocolatey package? If possible, maybe somehow make it autoupdate?

Thanks for this great tool!

App objects do not recognize when their application has gracefully terminated

Sikuli version: 1.1.4, build 2018-08-23_12:18
Platform: Windows 8.1, 64-bit

>java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

If the application an App object references terminates in a graceful way (e.g., an exit is requested via the application's GUI), it does not appear that the App object is ever updated to reflect the new non-running status. This makes calls to methods like isRunning() return unreliable information to Sikuli scripts.

Here's a small Sikuli script I wrote to demonstrate and reproduce the problem. Make sure all instances of Notepad are closed before running this script:

def print_divider():
	print('------------------------------------')

def status_check():
	print('Checking to see if Windows thinks Notepad is running:')
	print_divider()
	print(run('tasklist /NH /FI "IMAGENAME eq notepad.exe"').strip())
	print_divider()
	
	print("\nHere's what Sikuli thinks:")
	print_divider()
	print('Notepad PID is: ' + str(notepad_app.getPID()).strip())
	print('Is Notepad running?: ' + str(notepad_app.isRunning()))
	print_divider()


notepad_app = App('notepad.exe')

status_check()
print("\nOpening Notepad...")
notepad_app.open()
wait(1)
status_check()
print("\nClosing Notepad...")
notepad_app.focus()
type(Key.F4, Key.ALT)
wait(1)
status_check()

Generated output on my test system:

Checking to see if Windows thinks Notepad is running:
------------------------------------
[info] runcmd: tasklist /NH /FI "IMAGENAME eq notepad.exe" 
0 
INFO: No tasks are running which match the specified criteria.
------------------------------------

Here's what Sikuli thinks:
------------------------------------
Notepad PID is: -1
Is Notepad running?: False
------------------------------------

Opening Notepad...
[log] App.open [6972:notepad.exe]
Checking to see if Windows thinks Notepad is running:
------------------------------------
[info] runcmd: tasklist /NH /FI "IMAGENAME eq notepad.exe" 
0 
notepad.exe 6972 Console 1 8,208 K
------------------------------------

Here's what Sikuli thinks:
------------------------------------
Notepad PID is: 6972
Is Notepad running?: True
------------------------------------

Closing Notepad...
[log] App.focus: [6972:notepad.exe]
[log] ( Alt ) TYPE "#F4."
Checking to see if Windows thinks Notepad is running:
------------------------------------
[info] runcmd: tasklist /NH /FI "IMAGENAME eq notepad.exe" 
0 
INFO: No tasks are running which match the specified criteria.
------------------------------------

Here's what Sikuli thinks:
------------------------------------
Notepad PID is: 6972
Is Notepad running?: True
------------------------------------

SikuliX IDE to fill more than one input field

I try to use SikuliX IDE to fill more than one input fields on the same screen. All the fields look like the same. I captured input field, click on it and use type to set value, but text is not present after that process.

"Export as jar" output will not run

When I perform a File > Export as jar to generate a jar from the IDE and run java -jar test_sikuli.jar in the command prompt I get the following error: Error: Invalid or corrupt jarfile test_sikuli.jar.

This script will run correctly if I export via File > Export as runnable jar. I have been trying to perform a workaround by launching the jar via the class path java -cp test_sikuli.jar [some path] but have been unsuccessful finding the correct class to use.

Is there a solution or possible workaround to this problem? I am using Java Version 8 Update 181

IDE: [error] EditorPane: Problem while trying to clean bundle (not used images): null

I get following error message after I saved project.

[error] EditorPane: Problem while trying to clean bundle (not used images): null

SikulixIDE 1.1.4-SNAPSHOT(2018-12-05)

FIXED 1.1.3 (2018-03-15 Build 6) ERROR at line org.sikuli.script.Finder.java:81

log(-1, "imagefile not found:\n%s");
java.util.MissingFormatArgumentException: Format specifier '%s'

vnc is not working on linux ubuntu 16.04 with jython 2.7

I have installed sikulix and when I write in the IDE:

vnc = vncStart(ip="192.168.1.1", port=5901, password="vncpassword")
print vnc

It gives me None
Why is this happening?

Unable to open App with parameters

While opening app using App.open(), on 1.1.4 it fails with an error, "Windows cannot find C:\myapp.exe --config C:\myapp.conf --skip-splash". Make sure you typed the name correctly and try again.

It works fine on 1.1.2. If I don't pass config (No space in command) then it opens my app.

[1.1.4] problems with my own ADB based implementation based on Sikulix - worked in 1.1.3

I have implemented my own screen and region for Android ADB and the update to 1.1.4 seems to have introduced a bug where minimizing the search area will lead to a find fail. For example and image with the coordinates and size x: 50 y: 50 w: 50 h: 50, after creating a sub region with those exact coordinates or using a function like grow(-100) on screen then using the exists("image.png") on the smaller region leads to a find fail but works fine on the screen region itself.

This behavior was not present in 1.1.3

With full screen:

[1705106 debug] Image: cached: by.png (6 KB) (# 1 KB 6 -- 0 % of 64 MB)
[1705107 debug] Region: findAll: waiting 3.0 secs for (multiple) by.png to appear in NonLocal, 2160x1080
[1705864 debug] Finder2: makeMat: INT_RGB (2160x1080)
[1705885 debug] Finder2: makeMat: TYPE_4BYTE_ABGR (51x34)
[1705885 debug] Finder2: doFind: start
[1706110 debug] Finder2: doFind: in original: %100.0000 (?10) 225 msec 
[1706110 debug] Finder2: doFind: end 225 msec
[1706110 debug] Region: findAll: by.png has appeared

by.png: M[936,786 51x34]@S(NonLocal, 2160x1080) S:1.00 C:961,803 [-1 msec]
by.png: M[1269,786 51x34]@S(NonLocal, 2160x1080) S:0.99 C:1294,803 [-1 msec]
by.png: M[1603,786 51x34]@S(NonLocal, 2160x1080) S:0.97 C:1628,803 [-1 msec]

With sub region:

[1674822 debug] Image: cached: by.png (6 KB) (# 1 KB 6 -- 0 % of 64 MB)
[1674822 debug] Region: findAll: waiting 3.0 secs for (multiple) by.png to appear in NonLocal, 306x856
[1675095 debug] Finder2: makeMat: INT_RGB (306x856)
[1675103 debug] Finder2: makeMat: TYPE_4BYTE_ABGR (51x34)
[1675103 debug] Finder2: doFind: start
[1675127 debug] Finder2: doFind: in original: %16.5181 (?10) 24 msec 
[1675127 debug] Finder2: doFind: end 24 msec

by.png: M[817,720 51x34]@S(NonLocal, 2160x1080) S:0.17 C:842,737 [-1 msec]
by.png: M[871,818 51x34]@S(NonLocal, 2160x1080) S:0.16 C:896,835 [-1 msec]
by.png: M[920,824 51x34]@S(NonLocal, 2160x1080) S:0.16 C:945,841 [-1 msec]
by.png: M[736,882 51x34]@S(NonLocal, 2160x1080) S:0.16 C:761,899 [-1 msec]
by.png: M[791,225 51x34]@S(NonLocal, 2160x1080) S:0.15 C:816,242 [-1 msec]
by.png: M[710,812 51x34]@S(NonLocal, 2160x1080) S:0.15 C:735,829 [-1 msec]
by.png: M[898,875 51x34]@S(NonLocal, 2160x1080) S:0.15 C:923,892 [-1 msec]
by.png: M[845,246 51x34]@S(NonLocal, 2160x1080) S:0.15 C:870,263 [-1 msec]
by.png: M[771,338 51x34]@S(NonLocal, 2160x1080) S:0.14 C:796,355 [-1 msec]
by.png: M[870,741 51x34]@S(NonLocal, 2160x1080) S:0.14 C:895,758 [-1 msec]
by.png: M[848,549 51x34]@S(NonLocal, 2160x1080) S:0.14 C:873,566 [-1 msec]
by.png: M[733,687 51x34]@S(NonLocal, 2160x1080) S:0.14 C:758,704 [-1 msec]
by.png: M[813,811 51x34]@S(NonLocal, 2160x1080) S:0.14 C:838,828 [-1 msec]
by.png: M[949,769 51x34]@S(NonLocal, 2160x1080) S:0.14 C:974,786 [-1 msec]
by.png: M[717,981 51x34]@S(NonLocal, 2160x1080) S:0.14 C:742,998 [-1 msec]
by.png: M[708,520 51x34]@S(NonLocal, 2160x1080) S:0.14 C:733,537 [-1 msec]
by.png: M[719,317 51x34]@S(NonLocal, 2160x1080) S:0.13 C:744,334 [-1 msec]
by.png: M[867,578 51x34]@S(NonLocal, 2160x1080) S:0.13 C:892,595 [-1 msec]
by.png: M[708,428 51x34]@S(NonLocal, 2160x1080) S:0.13 C:733,445 [-1 msec]
by.png: M[791,903 51x34]@S(NonLocal, 2160x1080) S:0.13 C:816,920 [-1 msec]
by.png: M[957,980 51x34]@S(NonLocal, 2160x1080) S:0.13 C:982,997 [-1 msec]
by.png: M[918,903 51x34]@S(NonLocal, 2160x1080) S:0.13 C:943,920 [-1 msec]
by.png: M[935,719 51x34]@S(NonLocal, 2160x1080) S:0.13 C:960,736 [-1 msec]
by.png: M[899,267 51x34]@S(NonLocal, 2160x1080) S:0.13 C:924,284 [-1 msec]
by.png: M[750,826 51x34]@S(NonLocal, 2160x1080) S:0.13 C:775,843 [-1 msec]
by.png: M[823,360 51x34]@S(NonLocal, 2160x1080) S:0.12 C:848,377 [-1 msec]
by.png: M[706,939 51x34]@S(NonLocal, 2160x1080) S:0.12 C:731,956 [-1 msec]
by.png: M[804,847 51x34]@S(NonLocal, 2160x1080) S:0.12 C:829,864 [-1 msec]
by.png: M[953,853 51x34]@S(NonLocal, 2160x1080) S:0.12 C:978,870 [-1 msec]
by.png: M[831,934 51x34]@S(NonLocal, 2160x1080) S:0.12 C:856,951 [-1 msec]
by.png: M[740,232 51x34]@S(NonLocal, 2160x1080) S:0.12 C:765,249 [-1 msec]
by.png: M[809,782 51x34]@S(NonLocal, 2160x1080) S:0.12 C:834,799 [-1 msec]
by.png: M[956,952 51x34]@S(NonLocal, 2160x1080) S:0.12 C:981,969 [-1 msec]
by.png: M[903,931 51x34]@S(NonLocal, 2160x1080) S:0.12 C:928,948 [-1 msec]
by.png: M[907,776 51x34]@S(NonLocal, 2160x1080) S:0.12 C:932,793 [-1 msec]
by.png: M[953,288 51x34]@S(NonLocal, 2160x1080) S:0.12 C:978,305 [-1 msec]
by.png: M[882,698 51x34]@S(NonLocal, 2160x1080) S:0.12 C:907,715 [-1 msec]
by.png: M[920,548 51x34]@S(NonLocal, 2160x1080) S:0.12 C:945,565 [-1 msec]

Cannot compile Jython because an SyntaxError from sikulixapi.jar

Without the API all works, but when I add it to my project as library, a syntax error ocurred.

I'm using JDK8 and Jython 2.7

Output:

Traceback (innermost last):
  (no code object) at line 0
  File "C:\Users\jese5\IdeaProjects\Test\lib\sikulixapi.jar", line 1
	PK��
	  ^
SyntaxError: Lexical error at line 1, column 3.  Encountered: "\u0003" (3), after : ""

Process finished with exit code -1

My code:

class TOC(object):
    def __init__(self):
        print("Hello World")
TOC()

And the libraries:

Action Pane in IDE

Version 1.1.3 of the IDE had a pane on the left side from which it was possible to drag actions like click to the editor. This one has vanished from the 1.1.4 version. Any plans to bring it back?

App.focus() on Windows not working with unicode characters in window title --- fixed 2019-03-05 build#220

Bug status
currently only workaround --- fix will take time (bug tracking)

App("someApp.exe").focus() will not work, if the title of the 
frontmost app window contains unicode characters.

workaround:
App.focus("part of window title") will only work 
if the given string is ascii-only (no unicode characters),
but it will find and focus also windows, whose title contains Unicode characters.

So currently there is no way to focus apps/windows, whose window titles are completely unicode.

Hi
I've just started to use Sikulix for my task.
I have tested sikulix 1.1.4 2018-11-27_13:43 (Build 137) on Windows 10.
When I tried to focus on app named with Unicode character, it failed with error "App.focus failed: no window for App_name_including_unicode_characters_and_English_also_PID"
I've reviewed Sikulix code and realised that switchApp method in org/sikuli/natives/WinUtil.java line 332 not implemented.
Probably because of this, focus() is not working?
Sorry I'm not actively using Java but Python.
Could you please fix this when you have time?
Also, is there any better way to work around to set focus on the app easily? I assume it can be done with fidn images but except this :)

[1.1.4] JRuby: sikulixapi.jar: NameError: missing classname: ImageFinder

java version "10.0.2" 2018-07-17
Java(TM) SE Runtime Environment 18.3 (build 10.0.2+13)
Java HotSpot(TM) 64-Bit Server VM 18.3 (build 10.0.2+13, mixed mode)

jruby 9.2.0.0 (2.5.0) 2018-05-24 81156a8 Java HotSpot(TM) 64-Bit Server VM 25.121-b13 on 1.8.0_121-b13 +jit [mswin32-x86_64]

sikulix v1.1.0.3 (gem)

Win10 pro (64bit)

Using the latest version I get the following exception:

NameError: missing class name (`org.sikuli.script.ImageFinder')
method_missing at org/jruby/javasupport/JavaPackage.java:258
module:Sikulix at C:/jruby-9.2.0.0/lib/ruby/gems/shared/gems/sikulix-1.1.0.3/lib/sikulix/sikulix.rb:23

at C:/jruby-9.2.0.0/lib/ruby/gems/shared/gems/sikulix-1.1.0.3/lib/sikulix/sikulix.rb:7

The elder sikulixapi.jar worked/works reliably.

[1.1.4] export libs in Eclipse application bundle --- fixed build #165 (Windows: tesseract lib not yet fixed)

Hello,

Our context is an Eclipse RCP based application. This application is built as a set of plugins.
In one of this plugin, we have embedded the sikulixapi.jar (nighty build version).

If we are making a call to the Sikulix API, the execution is failing (the JVM is shutdown).

If before launching we manually unzip content of the sikulixapi.jar in C:\Users****\AppData\Roaming\Sikulix, everything is working fine.

After a bit of search in master code, issue seems to be at line 1569 of class RunTime.java:
fFolder = new File(uFolder.toURI());

The uFolder URL provided here by the class loader is a bundle URL (for example: bundleresource://311.fwk673068808:1/...) , and afaik there is no way to build a file directly from such URL.

[1.1.4] IDE: attempting to use Find throws an error --- currently switched off

Sikuli version: 1.1.4, build 2018-08-23_12:18
Platform: Windows 8.1, 64-bit

>java -version
java version "1.8.0_181"
Java(TM) SE Runtime Environment (build 1.8.0_181-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.181-b13, mixed mode)

When attempting to use the "Find" function in the IDE, either through Edit -> Find -> Find..., or via CTRL-F, the IDE throws an error:
[error] IDE: Problem when trying to invoke menu action doFind Error: null

No further activity occurs, and "Find" cannot be used.