serivesmejia/eocv-sim

FTC Library EasyOpenCV simulator for testing vision pipelines in a computer

EOCV-Sim has been migrated to a new repo in the DeltaCV organization

DO NOT USE THIS REPO AS IT'S NO LONGER UPDATED, ALL FUTURE UPDATES WILL BE MADE IN THE NEW AFOREMENTIONED REPO

DO NOT FOLLOW THE INSTRUCTIONS BELOW EITHER, THIS README IS KEPT FOR HISTORIC PURPOSES

Welcome!

EOCV-Sim (EasyOpenCV Simulator) is a straightforward way to test your pipelines in a simple user interface directly in your computer, simulating the EasyOpenCV library & a bit of FTC SDK structure, allowing you to simply copy paste directly your pipeline code once you want to transfer it onto your robot!

If you'd like to learn how to use the simulator, you can find a complete usage explaination here

Compatibility

Since OpenCV in Java uses a native library, which is platform specific, the simulator is currently limited to the following platforms:

• Windows x64 (tested)
• Windows x32 (untested)
• MacOS x64 (tested)
• Linux x64 (tested for Ubuntu 20.04)
  

Installation

1. Download & install the Java Development Kit if you haven't already:
   
   JDK 8 is the minimum required one, any JDK above that version will probably work fine.
   You can download it from the Oracle webpage, and here is a step by step video of the installation process
   

Recommended method

1. Make sure you have downloaded a JDK as mentioned above

2. Go to the releases page on this repo and find the latest version (or click here)

3. Download the jar file, named EOCV-Sim-X.X.X-all.jar, available at the bottom on the "assets" section

4. Choose and install an IDE/text editor
   
   The recommended text editor is VS Code, with the Java Extension Pack. EOCV-Sim provides direct support for it, for creating a "VS Code Workspace" from a template, although it can also be imported into IntelliJ IDEA since it's just a normal Gradle project.

   This installation method provides the benefit of "runtime compiling", which means that the user pipelines are compiled and loaded on the fly and therefore the changes made in code can be reflected immediately, as opposed to the old IntelliJ IDEA method in which the simulator had to be closed, compiled and then opened again to apply the smallest change made in a pipeline. Plus, VS Code is a lightweight editor which provides Java syntax highlighting and IntelliSense with the Java Extension Pack, making development of pipelines easy with tools like code completion.

   You can download and install VS Code from the Visual Studio page. The Java Extension Pack can be installed from the VS Code extension marketplace.

   Here's a tutorial video explaining how to download and install VS Code & the Java Extension Pack

5. Running EOCV-Sim
   
   For running the sim, simply double click the jar file downloaded from the releases page, or it can also be executed from the command line:

   java -jar "EOCV-Sim-X.X.X-all.jar"

   When running on Linux (distros such as Ubuntu, Linux Mint, etc) or Unix-like secure operating systems, it might prohibit you to run it by double clicking the file from a file explorer. This can be fixed by giving execute permissions to the jar file with the following command

   chmod +x EOCV-Sim-X.X.X-all.jar

Now the sim should be running without any issues! If you find any problem feel free to open an issue, and check the usage explanation for more details about how to use the simulator (and VS Code).

Altenative installation method (IntelliJ IDEA)

No complicated setup is required for this method either, it's straight up importing the EOCV-Sim project into IntelliJ IDEA:

*The downside of this method is that this repo has grown to a considerable amount of space, due to a bloated history, and takes some time to clone, and also builds can be slower depending on your device.

1. Make sure you have downloaded a JDK as mentioned here

2. Download & install IntelliJ IDEA Community IDE if you haven't already:
   
   You can download it from the JetBrains webpage
   Here is another great step by step video for IntelliJ installation.

3. Clone and import the project:
   

   1. Open IntelliJ IDEA and in the main screen click on "Get from Version Control"
      

      
      
      Alternatively, if you already had another project opened, go to File > New > Project from Version Control...
      
      

   2. Another window will show up for cloning and importing a repository into IntelliJ
      

      1. In the "URL" field, enter: https://github.com/serivesmejia/EOCV-Sim.git
         
      2. The directory can be changed, but it will be automatically filled so it's not necessary.
      3. Make sure the "Version control" is set to "Git".
         
         
         
      4. After that, click on the "Clone" button, located at the bottom right and the cloning process will begin...
         
         
         
      5. After the cloning finishes, the project should automatically import and you'll have something like this:
         
         
         

And you're ready to go! Refer to the usage explanation for further details on how to utilize the simulator.

From the command-line

1. Clone EOCV-Sim repo and cd to the cloned folder

 git clone https://github.com/serivesmejia/EOCV-Sim.git
 cd EOCV-Sim

*Or it can also be manually downloaded as a ZIP file from GitHub

2. Run EOCV-Sim through gradle:

 gradlew runSim

*On some command lines (like Windows PowerShell) you might need to execute "./gradlew" instead

And that's it! You might need to wait a bit for gradle to download all the dependencies but EOCV-Sim will open eventually.

From repl.it

1. Click here to go to repl.it, you might require to create an account if you haven't already. Once you do that, it will automatically create a new project and start cloning the EOCV-Sim repo.

2. After the cloning is finished, click on the green "Run" button at the top and EOCV-Sim should start.

*Please note that this method is not widely supported and you might run into some issues or lack of some functionality.

Adding EOCV-Sim as a dependency

Gradle

repositories {
    maven { url 'https://jitpack.com' } //add jitpack as a maven repo 
}

dependencies {
   implementation 'com.github.serivesmejia:EOCV-Sim:3.0.0' //add the EOCV-Sim dependency
}

Maven

Adding the jitpack maven repo

 <repositories>
 	<repository>
 	    <id>jitpack.io</id>
 	    <url>https://jitpack.io</url>
 	</repository>
 </repositories>

Adding the EOCV-Sim dependecy

 <dependency>
     <groupId>com.github.serivesmejia</groupId>
     <artifactId>EOCV-Sim</artifactId>
     <version>3.0.0</version>
 </dependency>

Contact

For any quick troubleshooting or help, you can find me on Discord as serivesmejia#8237 and on the FTC discord server. I'll be happy to assist you in any issue you might have :)

For bug reporting or feature requesting, use the issues tab in this repository.

Change logs

v3.0.0 - Compiling on the fly! Yay!

• This is the 9th release for EOCV-Sim

  • Changelog:

    • Runtime building! The sim now supports building pipelines on the fly, which allows for more quick and efficient testing. (Running the sim with a JDK is required for this feature to work, since normal JREs don't include a compiler to use)
    • Workspaces & VS Code is the new (and recommended) way of developing pipelines. A VS Code workspace template can be created from the sim, see the usage explanation for more details.
    • A file watcher was implemented, so when any modification happens in the current workspace under the "source" or "resource" folders specified in the eocvsim_workspace.json, a new build will be automatically triggered every 8 seconds.
    • VS Code can be executed by the sim if the current system has the code command. It is triggered by manually opening it in the top menu bar or when creating a VS Code workspace.
    • Files can now be drag and dropped into the sim to add them as Input Sources. The sim will automatically open a create dialog depending on the file extension.
    • Added a "Workspace" menu under the top menu bar, which contains the new features regarding the runtime compiling.
    • The UI now has smoother icons, by using a smoothing option on Java swing which makes them look a little nicer (but not much).
    • Current pipeline state is now stored and reestablished if a restart happens.
    • When a build is finished, the simulator tries reinitializes the currently selected pipeline if it exists, to ensure the changes were applied. Or it falls back to the DefaultPipeline if the old pipeline doesn't exist anymore, it also saves the state of the old pipeline and tries to apply the snapshot of the pipeline before it was reinitialized if the names of the old and new classes match.
    • The sim now uses a .eocvsim folder under the user directory to store its files, to avoid annoying the user with unwanted files now that the runtime compiling exists and it has to store the build output somewhere. If the user has previously run an older version of eocv sim which created eocvsim_sources.json and/or eocvsim_config.json under the user home directory, it automatically migrates them to the new folder.
    • Builds created by IntelliJ Idea (the common programming style) are now considered as "dev". This helps to distinguish between official & published builds created in a CI workflow and local builds, when an issue happens and it's reported.
    • The sim compiling target was changed back to Java 8, since this is one of the most widely used versions and we weren't really using many Java 9 features that couldn't have been replaced or handle different. This is also more convenient and provides better support for users directly downloading the jar and executing it.

  • Bugfixes:

    • Fixed issues with the source selector regarding to selection when a modification or error happens. When a new source is added, it's automatically selected. And when a source is deleted, the previous source in the list is selected
    • Fixed the color picker cursor size on non-windows systems
    • Fixed pause not working when a tunable field that uses a combo box in the UI is included.
    • Fixed an (apparently random, but it's just the garbage collector being weird) null pointer exception with Enum fields

  • Internals:

    • Improved event handlers to be more idiomatic and less weird. Bye bye KEventListener!
    • Improved some messy parts of the internal code and logic

v2.2.1 - JVM crashing hotfix

• This is the 8th release for EOCV-Sim

  • Changelog:

    • Removed "Java memory" message in the title since it's practically useless for the end user
    • Updated to Gradle 7.0 for Java 16+ support (#25)

  • Bugfixes:

    • Fixed JVM crashing error caused by releasing all mats in a MatRecycler finalization (#26)
    • Improved memory usage by deleting unused BufferedImageRecyclers, memory is now slightly freed when allocating or recycling buffered images of different sizes (which means that the memory usage is reduced a little bit when zooming in the viewport)

v2.2.0 - Variable Tuner Upgrade

• This is the 7th release for EOCV-Sim

  • Changelog:

    • Pipelines now have a timeout all of the three methods so that the main loop doesn't get compromised due to a "stuck" pipeline (using kotlin coroutines)
    • processFrame has a timeout of 4.2 seconds
    • init is executed in the same scope as processFrame, when it has to be called, the timeout is doubled (16.4)
    • When either processFrame or init methods timeout, the sim automatically falls back to the default pipeline and discards any frame that the old timeouted pipeline could return.
    • onViewportTapped is still called from the U.I Thread, but it now has a timeout of 4.2 seconds too
    • Added EnumField which handles the type Enum (accepts all classes of type enum, including the ones declared by the user)
    • Major improvements to the variable tuner, added new features for color picking, tuning with sliders, configuration... See usage explanation for further details.
    • GUI improvement: Dropped some external dialogs in favor of simple "popups" for more practicality
    • Internals:
      • Continued rewrite to kotlin
      • Splitted visualizer class components into different classes
      • Improved EventHandler doOnce listeners

v2.1.0 - Video Update

• This is the 6th release for EOCV-Sim

  • Changelog:

    • Added support for VideoSources! You can now input your pipeline with a moving video (*.avi format is the most supported and tested, other codecs might depend on the OS you're using)
    • Added support for video recording, accessible at the bottom of the pipeline selector. Save format is AVI
    • Added a new TunableField type: RectField, which handles the OpenCV type "Rect" (might be useful for rect pipelines 👀)
    • Improved uncaught exception handling and added a crash report generator
    • Added support for more themes from FlatLaf
    • Added new config option to change the output video recording size
    • Added support for EOCV's TimestampedOpenCvPipeline
    • Internals:
      • Major rewrite to kotlin! (Still mostly Java but that might change soon)
      • A bit of code cleaning and restructuring

v2.0.2 - TaskBar hotfix

• This is the 5th release for EOCV-Sim.

  • Bugfixes:

    • Fixes UnsupportedOperationException with the TaskBar API in some operating system

v2.0.1 - BooleanField hotfix

• This is the 4th release for EOCV-Sim.

  • Bugfixes:

    • Fixes ArrayIndexOutOfBoundsException when initial value of a boolean field was true which would make the sim enter into a frozen state.

v2.0.0 - Major Update

• This is the 3rd release for EOCV-Sim.

  • Changelog:

    • Gradle is now used as the main build system
    • Added variable tuner for public non-final supported fields in the pipeline, accessible on the bottom part of the image viewport.
    • Pipeline pause and resume option to save resources, pauses automatically with image sources for one-shot analysis
    • Top Menu bar containing new features/convenient shortcuts:
      • Save Mat to disk option in File submenu
      • Restart feature in File submenu
      • Shortcut for creating input sources under File -> New -> Input Source
      • Settings menu under Edit submenu
      • "About" information screen under Help submenu
      • Appereance themes via the FlatLaf library, selectable in the settings window
    • Telemetry now is passed to the pipeline via the constructor rather than an instance variable, check usage explaination for further details
    • Mat visualizing into the viewport is now handled in another thread to improve performance
    • Pipeline FPS are now capped at 30
    • Zooming viewport is now supported, using mouse wheel while holding Ctrl key

  • Bugfixes:

    • Removed call to the gc in the main loop due to performance issues
    • Fixed BufferedImage mem leak by recycling previously used buffered images and trying to flush them
    • Some internal code cleaning & reestructuration
    • Fixed issues with native lib loading (mostly on Mac) with the OpenCV package provided by OpenPnP

v1.1.0 - Telemetry Update

• This is the 2rd release for EOCV-Sim.

  • Changelog:

    • Added a Telemetry implementation displayed in the UI. Replicates the FTC SDK one, it can be used directly in pipelines.
    • Added an option to define the CameraSource resolution when creation.
    • Added MacOS support (thnx Noah)
    • Changed default resolution to 320x280 everywhere since it is the most commonly used in EOCV
    • Native libs are now downloaded by the simulator from another GitHub repo to avoid bloating the repository with heavy files
    • Java libraries, such as classgraph, opencv and gson are now delivered in compiled jars to improve compile times

  • Bug fixes:

    • Fixed a bug where the InputSources would return a BGR Mat instead of RGB, which is the type EOCV gives.
    • Regarding the last point, the visualizer now expects for the given mats to be RGB
    • Improved general IO error handling everywhere, from file accessing to input sources reading, so that the simulator doesn’t enter in a freeze state if any IO related operation fails
    • Improved multi threading handling for changing pipelines and inputsources.
    • Fixed issue in Linux where the buttons would be moved to an incorrect position when resizing out and then trying to resize back to the original size

v1.0.0 - Initial Release

• Initial EOCV-Sim release.

eocv-sim's Issues

Fails to load native library on Linux x64

00:00  INFO: [EOCVSim] Initializing EOCV Sim v1.0.0

00:00  INFO: [SysUtil] Copying native lib "linux64_opencv_java440.so"
00:00  INFO: [SysUtil] Copy of linux64_opencv_java440 cancelled since file already exists

00:00  INFO: [SysUtil] Loading native lib "linux64_opencv_java440.so"
00:00 ERROR: [SysUtil] Failure loading lib "linux64_opencv_java440", retrying with different architecture... (0 attempts)
00:00  INFO: [SysUtil] Copying native lib "linux32_opencv_java440.so"

00:00  INFO: [SysUtil] Loading native lib "linux32_opencv_java440.so"
Java HotSpot(TM) 64-Bit Server VM warning: You have loaded library /tmp/linux64_opencv_java440.so which might have disabled stack guard. The VM will try to fix the stack guard now.
It's highly recommended that you fix the library with 'execstack -c <libfile>', or link it with '-z noexecstack'.
java.lang.UnsatisfiedLinkError: /tmp/linux64_opencv_java440.so: /tmp/linux64_opencv_java440.so: invalid ELF header (Possible cause: endianness mismatch)
	at java.lang.ClassLoader$NativeLibrary.load(Native Method)
	at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1941)
	at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1824)
	at java.lang.Runtime.load0(Runtime.java:809)
	at java.lang.System.load(System.java:1086)
	at com.github.serivesmejia.eocvsim.util.SysUtil.loadLib(SysUtil.java:90)
	at com.github.serivesmejia.eocvsim.util.SysUtil.loadCvNativeLib(SysUtil.java:55)
	at com.github.serivesmejia.eocvsim.EOCVSim.init(EOCVSim.java:43)
	at com.github.serivesmejia.eocvsim.Main.main(Main.java:8)

Java 16 does not work

Describe the bug
Compiling in java 16 fails although java 13 does. (Deprecation of some api(s) i believe)

To Reproduce
Install Java 16

Expected behavior
To compile in the :compileJava: task.

Desktop (please complete the following information):

• OS: Windows 10
• Version 2.2.0

Additional context
Java 13 does work correctly!

Feature Request: Add Video Source Support

OpenCV has support for video sources. I know for a fact that in python you can do

import cv2
vid = cv2.VideoCapture('video.mp4')
success, img = vid.read()
while success:
  success, img = vid.read()

and it works just fine with img representing each image frame. I'm sure Java has something similar, if not C++. Would be great if I can bypass the camera and simply use an mp4 or other video format to test pipelines (super useful for remote stuff).

Samples copied from EasyOpenCV repository do not have license header

Currently this repo has the following samples copied from the EasyOpenCV repository, but omits the license header:

In order to not be in violation of the original MIT license under which these files were released, the following header must be placed at the top of each of them:

/*
 * Copyright (c) 2020 OpenFTC Team
 *
 * Permission is hereby granted, free of charge, to any person obtaining a copy
 * of this software and associated documentation files (the "Software"), to deal
 * in the Software without restriction, including without limitation the rights
 * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
 * copies of the Software, and to permit persons to whom the Software is
 * furnished to do so, subject to the following conditions:
 *
 * The above copyright notice and this permission notice shall be included in all
 * copies or substantial portions of the Software.
 * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
 * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
 * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
 * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 * SOFTWARE.
 */