cnescatlab / docker-cat Goto Github PK
View Code? Open in Web Editor NEWDocker CAT is a quality analysis platform including CNES plugins and configurations.
License: GNU General Public License v3.0
Docker CAT is a quality analysis platform including CNES plugins and configurations.
License: GNU General Public License v3.0
Since upgrade to SQ 9 we have some missing plugins, see #67
After analyze, we have side effects on reintegrate plugin as-is
{"errors":[{"msg":"For input string: \"AY3P71XQVPxrBVhsqr6R\""}]}
(AY3... is the CNES Quality Gate ID)Hi,
First of all, thank you for sharing us this repo.
I have been trying to use docker-cat on a CentOS 7 VM, unfortunately I'm stuck...
Here is the command I executed:
docker run -v /Docker:/media/sf_Shared:rw -p 9000:9000 -p 9001:9001 -e ALLOWED_GROUPS="1000" lequal/docker-cat
The error says "web[][o.s.s.p.Platform] Web server startup failed" and was caused by "java.util.zip.ZipException: error in opening zip file"
Here the logs obtained after running the command
[INFO] Docker-cat is now adding user permissions to Sonarqube. [INFO] Docker-cat is giving group permissions GID no. 1000 to Sonarqube. [INFO] Docker-cat permissions for sonarqube finished. [INFO] docker-cat, initiating connection with Sonarqube. 2019.08.07 15:24:44 INFO app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp 2019.08.07 15:24:44 INFO app[][o.s.a.es.EsSettings] Elasticsearch listening on /127.0.0.1:9001 2019.08.07 15:24:44 INFO app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='es', ipcIndex=1, logFilenamePrefix=es]] from [/opt/sonarqube/elasticsearch]: /opt/sonarqube/elasticsearch/bin/elasticsearch -Epath.conf=/opt/sonarqube/temp/conf/es 2019.08.07 15:24:44 INFO app[][o.s.a.SchedulerImpl] Waiting for Elasticsearch to be up and running 2019.08.07 15:24:54 INFO app[][o.e.p.PluginsService] no modules loaded 2019.08.07 15:24:54 INFO app[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin] [INFO] docker-cat, retrieving Sonarqube's service status.. [INFO] docker-cat, detected status for Sonarqube, expecting it to be UP.. [INFO] docker-cat, retrieving Sonarqube's service status.. [INFO] docker-cat, detected status for Sonarqube, expecting it to be UP.. 2019.08.07 15:25:19 INFO app[][o.s.a.SchedulerImpl] Process[es] is up 2019.08.07 15:25:19 INFO app[][o.s.a.p.ProcessLauncherImpl] Launch process[[key='web', ipcIndex=2, logFilenamePrefix=web]] from [/opt/sonarqube]: /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djava.io.tmpdir=/opt/sonarqube/temp -Xmx512m -Xms128m -XX:+HeapDumpOnOutOfMemoryError -Djava.security.egd=file:/dev/./urandom -cp ./lib/common/*:./lib/server/*:/opt/sonarqube/lib/jdbc/h2/h2-1.3.176.jar org.sonar.server.app.WebServer /opt/sonarqube/temp/sq-process1789222769243259198properties 2019.08.07 15:25:22 INFO web[][o.s.p.ProcessEntryPoint] Starting web 2019.08.07 15:25:24 INFO web[][o.a.t.u.n.NioSelectorPool] Using a shared selector for servlet write/read [INFO] docker-cat, retrieving Sonarqube's service status.. 2019.08.07 15:25:27 INFO web[][o.e.p.PluginsService] no modules loaded 2019.08.07 15:25:27 INFO web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.index.reindex.ReindexPlugin] 2019.08.07 15:25:27 INFO web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.join.ParentJoinPlugin] 2019.08.07 15:25:27 INFO web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.percolator.PercolatorPlugin] 2019.08.07 15:25:27 INFO web[][o.e.p.PluginsService] loaded plugin [org.elasticsearch.transport.Netty4Plugin] 2019.08.07 15:25:31 INFO web[][o.s.s.e.EsClientProvider] Connected to local Elasticsearch: [127.0.0.1:9001] 2019.08.07 15:25:31 INFO web[][o.s.s.p.LogServerVersion] SonarQube Server / 6.7.4.38452 / b62b59da1adf991c41ed0730684f813d4116f6cc 2019.08.07 15:25:32 INFO web[][o.s.s.p.d.EmbeddedDatabase] Starting embedded database on port 9092 with url jdbc:h2:tcp://127.0.0.1:9092/sonar 2019.08.07 15:25:32 INFO web[][o.s.s.p.d.EmbeddedDatabase] Embedded database started. Data stored in: /opt/sonarqube/data 2019.08.07 15:25:32 INFO web[][o.sonar.db.Database] Create JDBC data source for jdbc:h2:tcp://127.0.0.1:9092/sonar 2019.08.07 15:25:32 WARN web[][o.s.d.DatabaseChecker] H2 database should be used for evaluation purpose only 2019.08.07 15:25:34 INFO web[][o.s.s.p.ServerFileSystemImpl] SonarQube home: /opt/sonarqube 2019.08.07 15:25:34 INFO web[][o.s.s.u.SystemPasscodeImpl] System authentication by passcode is disabled 2019.08.07 15:25:34 INFO web[][o.s.s.p.d.m.h.MigrationHistoryTableImpl] Creating table schema_migrations 2019.08.07 15:25:34 ERROR web[][o.s.s.p.Platform] Web server startup failed java.lang.IllegalStateException: Unable to read plugin manifest from jar : /opt/sonarqube/extensions/plugins/sonar-cnes-scan-plugin-1.3.jar at org.sonar.updatecenter.common.PluginManifest.<init>(PluginManifest.java:125) at org.sonar.core.platform.PluginInfo.create(PluginInfo.java:395) at org.sonar.server.plugins.ServerPluginRepository.loadPreInstalledPlugins(ServerPluginRepository.java:142) at org.sonar.server.plugins.ServerPluginRepository.start(ServerPluginRepository.java:108) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.invokeMethod(ReflectionLifecycleStrategy.java:110) at org.picocontainer.lifecycle.ReflectionLifecycleStrategy.start(ReflectionLifecycleStrategy.java:89) at org.picocontainer.injectors.AbstractInjectionFactory$LifecycleAdapter.start(AbstractInjectionFactory.java:84) at org.picocontainer.behaviors.AbstractBehavior.start(AbstractBehavior.java:169) at org.picocontainer.behaviors.Stored$RealComponentLifecycle.start(Stored.java:132) at org.picocontainer.behaviors.Stored.start(Stored.java:110) at org.picocontainer.DefaultPicoContainer.potentiallyStartAdapter(DefaultPicoContainer.java:1016) at org.picocontainer.DefaultPicoContainer.startAdapters(DefaultPicoContainer.java:1009) at org.picocontainer.DefaultPicoContainer.start(DefaultPicoContainer.java:767) at org.sonar.core.platform.ComponentContainer.startComponents(ComponentContainer.java:134) at org.sonar.server.platform.platformlevel.PlatformLevel.start(PlatformLevel.java:90) at org.sonar.server.platform.platformlevel.PlatformLevel2.start(PlatformLevel2.java:92) at org.sonar.server.platform.Platform.start(Platform.java:211) at org.sonar.server.platform.Platform.startLevel2Container(Platform.java:177) at org.sonar.server.platform.Platform.init(Platform.java:87) at org.sonar.server.platform.web.PlatformServletContextListener.contextInitialized(PlatformServletContextListener.java:45) at org.apache.catalina.core.StandardContext.listenerStart(StandardContext.java:4745) at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5207) at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:150) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1419) at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1409) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) Caused by: java.util.zip.ZipException: error in opening zip file at java.util.zip.ZipFile.open(Native Method) at java.util.zip.ZipFile.<init>(ZipFile.java:225) at java.util.zip.ZipFile.<init>(ZipFile.java:155) at java.util.jar.JarFile.<init>(JarFile.java:166) at java.util.jar.JarFile.<init>(JarFile.java:130) at org.sonar.updatecenter.common.PluginManifest.<init>(PluginManifest.java:120) ... 32 common frames omitted 2019.08.07 15:25:36 INFO web[][o.s.s.p.d.EmbeddedDatabase] Embedded database stopped [INFO] docker-cat, detected status for Sonarqube, expecting it to be UP.. 2019.08.07 15:25:37 INFO app[][o.s.a.SchedulerImpl] Process [web] is stopped 2019.08.07 15:25:38 INFO app[][o.s.a.SchedulerImpl] Process [es] is stopped 2019.08.07 15:25:38 INFO app[][o.s.a.SchedulerImpl] SonarQube is stopped 2019.08.07 15:25:38 WARN app[][o.s.a.p.AbstractProcessMonitor] Process exited with exit value [es]: 143
I'm really new to docker, I spent half a day trying to solve this issue unsuccessfully. Maybe there is something wrong with the parameters I entered, even though I am sure of the group ID.
Any help would really be appreciated.
I remain at your disposal if you need any additional information.
Cheers,
When analyzing a C/C++ software, changing the criticality of a rule in one quality profile, this change is taken into account in SonarQube dashboard and in the Excel file generated by the CAT plugin.
It is also taken into account in the summary table of the Word file (see chapter 3), but not in the table of violations (chapter 4) of the Word report document. In this table, the criticality remains that of the parent profile.
Docker image pulled from DockerHub should run without failure on SonarQube start.
There should not be NoSuchFileException
on such file /opt/sonarqube/temp/README.txt
.
docker run --rm -v /tmp/:/media/sf_Shared:rw -p 9000:9000 -p 9001:9001 -e ALLOWED_GROUPS="xxxx" lequal/docker-cat
[INFO] Docker-cat is now adding user permissions to Sonarqube.
[INFO] Docker-cat is giving group permissions GID no. 1000 to Sonarqube.
[INFO] Docker-cat permissions for sonarqube finished.
[INFO] docker-cat, initiating connection with Sonarqube.
tail: cannot open './logs/es.log' for reading: No such file or directory
2019.11.27 20:59:56 INFO app[][o.s.a.AppFileSystem] Cleaning or creating temp directory /opt/sonarqube/temp
Exception in thread "main" java.nio.file.NoSuchFileException: /opt/sonarqube/temp/README.txt
at java.base/sun.nio.fs.UnixException.translateToIOException(Unknown Source)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
at java.base/sun.nio.fs.UnixException.rethrowAsIOException(Unknown Source)
at java.base/sun.nio.fs.UnixFileAttributeViews$Basic.readAttributes(Unknown Source)
at java.base/sun.nio.fs.UnixFileSystemProvider.readAttributes(Unknown Source)
at java.base/sun.nio.fs.LinuxFileSystemProvider.readAttributes(Unknown Source)
at java.base/java.nio.file.Files.readAttributes(Unknown Source)
at java.base/java.nio.file.FileTreeWalker.getAttributes(Unknown Source)
at java.base/java.nio.file.FileTreeWalker.visit(Unknown Source)
at java.base/java.nio.file.FileTreeWalker.next(Unknown Source)
at java.base/java.nio.file.Files.walkFileTree(Unknown Source)
at org.sonar.application.AppFileSystem.createOrCleanTempDirectory(AppFileSystem.java:96)
at org.sonar.application.AppFileSystem.reset(AppFileSystem.java:62)
at org.sonar.application.App.start(App.java:62)
at org.sonar.application.App.main(App.java:98)
docker run --rm -v /tmp/:/media/sf_Shared:rw -p 9000:9000 -p 9001:9001 -e ALLOWED_GROUPS="xxxx" lequal/docker-cat
2.0.1
A clear and concise description of what the bug is.
Some times it is impossible for CNES Scan to write thesonar-project.properties
or for Sonar Scanner to write the.scannerwork
.
Steps to reproduce the behavior.
Launch an analysis with random access write permissions on source code.
A clear and concise description of what you expected to happen.
Docker CAT should try not to write inside source code directory. Docker CAT would be more user-friendly.
Please complete the following information.
Running this version on C++ project I encounter an error:
ERROR: Error during SonarQube Scanner execution
ERROR: Metric 'files' should not be computed by a Sensor
Looking around it seems to be an incompatibility between SonarQube version an plugins as SonarQube depreciated a 'files' metric. Example: sonar-scala/sonar-scala#181
Looking at https://github.com/SonarOpenCommunity/sonar-cxx/wiki/SonarQube-compatibility-matrix we need 1.3.1 at least for 7.9 compatibility.
The project should provide more information for the community:
There is only few information in the readme.
See GitHub repo.
All until 2.0.1
Currently all SonarQube instances send telemetry data to SonarSource servers.
To disable it, we have to set sonar.telemetry.enabled
to false
in sonarqube/conf/sonar.properties
.
Currently the Quality Gate does not match any of recommended gates.
Conditions on coverage should be removed in context of Docker CAT but condition on failed and errored conditions could be added.
We should be able to play complete validation tests when merging on master branch: end-to-end tests on representative project samples.
We only test Docker-cat starting.
Play tests.
2.0.1
I am using Docker and docker-cat image on Windows 10.
Docker run command requiers an --env parameter in order to map groups. But, in my Windows configuration, there is no groups (except generic WORKGROUP).
Trying command such as : " docker run -v //C/Users/Anatole/sources:/media/sf_Shared -p 9000:9000 -p 9001:9001 -e ALLOWED_GROUPS="ANATOLE\Anatole" lequal/docker-cat " (computer-name\account-name) leads to a warning => "groupmod: invalid group ID 'CHENES\Chenes'".
Container creation continues (see attached log) but at the end SonarQube is not launched.
Please could you provide us with a aprameter guide on Windows ?
Thanx
During start up, Docker CAT has to run many configuration script before being ready to use.
No formal message is displayed in log to be sure Docker CAT is ready whereas it should.
Add message [INFO] Docker CAT is ready to go and find bugs!
when Docker CAT is ready.
The first step when running C/C++ analysis is to launch externals Tools (cppcheck, Rats, etc.).
Each of these tools generates an XML result file located in the workspace directory.
So that if the source directory is the same as the workspace, the XML files will be scanned in the next step.
iCode CNES tool evaluates the followings metrics SH.MET.LineOfCode, SH.MET.RatioComment, SH.MET.Nesting, SH.MET.ComplexitySimplified as rules at functions levels.
But CAT dont implemented these rules.
If these metrics were provided in the form of rules, this would have the advantage of allowing progress to be tracked with software releases.
The default CNES quality gate should be compliant with CNES requirements and SonarQube features.
Criteria system for SonarQube quality gate has changed in LTS 7.9, so the configuration of our Quality Gate should do.
Compare Quality Gates > CNES
with expected result in configuration file.
2.0.1
(I open this issue mainly to initiate a discussion)
What about creating two (other) Docker images: one for server, one for scanner?
With such solutions, it will be possible to:
The questions related to such deployment: what about CNES' plugins? Will they work on such deployment? Are they still pertinent? For example:
I can try such refactoring, but I lack some knowledge on these plugins. Any help to identify what can be kept and what should be fired, would be appreciated.
A clear and concise description of what the bug is.
No custom pylintrc is defined in Docker CAT: it should be in order to set CNES threshold.
Steps to reproduce the behavior.
A clear and concise description of what you expected to happen.
A pylintrc should be used for each criticality level (A, B,C D) containing corresponding threshold.
If applicable, add screenshots to help explain your problem. Then, append SonarQube log and do not forget to obfuscate them._
Please complete the following information.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
This image cannot handle Docker projects.
A clear and concise description of what you want to happen.
Last release of hadolint and sonar-hadolint-plugin should be installed and available: https://github.com/hadolint/hadolint & https://github.com/cnescatlab/sonar-hadolint-plugin
When starting container:
[INFO] Docker-cat permissions for sonarqube finished.
[INFO] docker-cat, initiating connection with Sonarqube.
Error: Unable to access jarfile lib/sonar-application-6.7.4.jar
When setting variable environment like SONARQUBE_JDBC_*
or like sonar.*
(as allowed in recent release of sonarqube docker) they are ignored.
Run the docker-compose.yml
script provided.
Variables like SONARQUBE_JDBC_*
or like sonar.*
should be passed to bin/run.sh
.
A clear and concise description of what the bug is.
In the Dockerfile (first stage), the dependency libocamlgraph-ocaml-dev
is listed twice (line 16 and 20).
Only one is enough to have it installed.
Steps to reproduce the behavior.
A clear and concise description of what you expected to happen.
If applicable, add screenshots to help explain your problem. Then, append SonarQube log and do not forget to obfuscate them._
Please complete the following information.
A clear and concise description of what the bug is.
In the Dockerfile (first stage), lines
git clone --single-branch https://github.com/Frama-C/Frama-C-snapshot.git .
git checkout -b tags/20.0
should be replaced by
git clone --single-branch -b 20.0 https://github.com/Frama-C/Frama-C-snapshot.git .
Steps to reproduce the behavior.
A clear and concise description of what you expected to happen.
It should clone the version 20.0 of Frama-C not cloning the master branch and creating a new branch called tags/20.0
which does not look like a branch name.
If applicable, add screenshots to help explain your problem. Then, append SonarQube log and do not forget to obfuscate them._
Please complete the following information.
A clear and concise description of what the bug is.
Some times it is impossible for CNES Scan to scan symbolic links because of Docker volume.
Steps to reproduce the behavior.
Launch an analysis with source code containing symbolic links.
A clear and concise description of what you expected to happen.
Docker CAT should try not to analyze symbolic links. Docker CAT would be more user-friendly.
Please complete the following information.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Analysis tools and SonarQube plugins should be updated.
A clear and concise description of what you want to happen.
Update at least:
A clear and concise description of what the bug is.
When analysing C or C++ with CNES Scan / CNES Cxx, no reports of Cppcheck, Vera++ and RATS are imported whareas they are generated.
Steps to reproduce the behavior.
More
> CNES Analysis
Run C/C++ tools
Analyze
A clear and concise description of what you expected to happen.
Make the reports for Cppcheck, Vera++ and RATS being automatically imported when they are generated by CNES Scan/Cxx.
Please complete the following information.
Debian
2.0.2
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Docker CAT should use up-to-date tools for Python.
A clear and concise description of what you want to happen.
Consider updating Pylint and its CNES extension:
A clear and concise description of what the bug is.
When analyzing clicking on CNES Report
and then any other CNES ***
in the menu, nothing append and the view is blocked on CNES Report
view.
All work fine and passing from CNES Report
to another official page. The CNES Report
view should be fixed.
It is possible to bypass this issue by refreshing the page with F5
.
Steps to reproduce the behavior.
More
> CNES Analysis
CNES Analysis
view is displayedMore
> CNES Report
CNES Report
view is displayedMore
> CNES Analysis
CNES Analysis
view is not displayedA clear and concise description of what you expected to happen.
We should be able to navigate without any problem between different plugins' pages.
Please complete the following information.
Debian
2.0.2
A clear and concise description of what the bug is.
The file custom-java-rules-template.json
contains errors that prevent some rules to be added correctly when configuring the container.
Steps to reproduce the behavior.
docker run --rm --name=cat -v $PWD:/media/sf_Shared:rw -p 9000:9000 -e ALLOWED_GROUPS="<YOUR_GID>" lequal/docker-cat:2.1.0
[WARNING] docker-cat, impossible to create the rule CNES Javadoc Variable due to : "Value 'VARIABLE_DEF' must be one of : ENUM_CONSTANT_DEF.".
[WARNING] docker-cat, impossible to update the rule CNES Javadoc Variable due to : "Invalid rule key: null".
A clear and concise description of what you expected to happen.
I do not expect warnings from this file.
If applicable, add screenshots to help explain your problem. Then, append SonarQube log and do not forget to obfuscate them._
See above.
Please complete the following information.
User should be able to run a c/c++ analysis.
Cppcheck execution failed during Docker-CAT analysis: infinite loop.
Run a C/C++ analysis.
Docker-CAT : 1.0.3
A clear and concise description of what the bug is.
There are many useless profiles like:
ALL_*
And configuration files for external tools like findbugs, checkstyle and pmd should be removed too.
Steps to reproduce the behavior.
Quality Profiles
A clear and concise description of what you expected to happen.
Previously named resources should be removed.
Please complete the following information.
Debian
2.0.2
Master branch does not build anymore.
Update links to broken dependencies.
We must sync quality gates with our sonarqube image
The test logs indicates an error but the docker-cat image is published
Steps to reproduce the behavior.
CI may crash on errors
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Docker CAT is long to start because of start-up configuration.
A clear and concise description of what you want to happen.
Consider setting configuration (custom rules, quality profiles and quality gates) of the embedded H2 database at build and let configuration be rerun as option (e.g.: RECONFIGURE
) if the user want to use an external database which is not yet configured.
The start up of the application should be speeded up.
containers use a H2 database that does not handle large volume of data.
At the bottoms of the dashboard pages, the following message appears :
Embedded database should be used for evaluation purpose only. The embedded database will not scale, it will not support upgrading to newer versions of SonarQube, and there is no support for migrating your data out of it into a different database engine."
What about using an other DB like Postgres, MysSQL, etc.?
Thanx
The image is based on Ubuntu but a choice was made to get the packages directly from the Debian bullseye repository
"deb http://ftp.fr.debian.org/debian/ bullseye main contrib non-free"
Find why this choice was made and understand if this choice is still usefull. Otherwise, we can directly download packages from Ubuntu. Some packages are maybe useless now.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Docker CAT must benefit of the new cnesreport version.
A clear and concise description of what you want to happen.
Version 3.3.0 of cnesreport must be installed in Docker CAT.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.