Giter Site home page Giter Site logo

usdot-fhwa-stol / carma-streets Goto Github PK

View Code? Open in Web Editor NEW
10.0 7.0 10.0 4.73 MB

CARMA Streets is a component of CARMA ecosystem, which enables such a coordination among different transportation users. This component provides an interface for CDA participants to interact with the road infrastructure. Doxygen Source Code Documentation: https://usdot-fhwa-stol.github.io/documentation/carma-streets/

CMake 3.04% C++ 93.54% Shell 0.94% Python 1.10% Dockerfile 1.38%

carma-streets's Introduction

CI Build Status Sonar Code Quality
CI Quality Gate Status

DockerHub Release Builds

Scheduling Service Message Services Intersection Model Signal Opt Service Tsc Service
Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status

DockerHub Release Candidate Builds

Scheduling Service Message Services Intersection Model Signal Opt Service Tsc Service Sensor Data Sharing Service
Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status

DockerHub Develop Builds

Scheduling Service Message Services Intersection Model Signal Opt Service Tsc Service Sensor Data Sharing Service
Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status Docker Cloud Build Status

CARMA Streets Base Image Builds

Build and Push Base Images

CARMA Streets is a component of CARMA ecosystem, which enables such a coordination among different transportation users. This component provides an interface for CDA participants to interact with the road infrastructure. CARMA Streets is also an edge-computing unit that improves the efficiency and performance of the Transportation Systems Management and Operations (TSMO) strategies.

Architecture

CARMA Streets architecture is based on a scalable services and layered architecture pattern that allows for easy deployment. Service components are packaged to contain one or more modules (classes) that represent a specific reusable function (e.g., decode a particular ASN.1 message) or an independently deployable business function (e.g., control interface to a signal controller). Services interact with each other via lightweight messaging service (e.g., Kafka) which allows for them be deployed either together or distributed for scalability and performance. A high-level abstract view of the architecture to communicate the design pattern is shown in Upcoming Figure. A more detailed Unified Modeling Language class and packaging diagrams to define the interfaces between services and layers and their interactions will be developed and documented here during implementation following an Agile Development Methodology.

Deployment

Docker is the primary deployment mechanism to containerize one or more services. The CARMA Streets application and other major frameworks such as Kafka will run in their own separate containers. This document will be updated with a detailed Docker deployment strategy during later design phases.

Development

This repository includes configurations for devcontainer VSCode extension. This extension allows us to standup a containerized development environment. More information about the CARMA Streets Dev Container Setup can be found here.

Base Images

To make creating new CARMA Streets services easier and to make our CI/CD more efficient, we have introduced new CARMA Streets base images. These images can be used as a starting point and common build/runtime environment for CARMA Streets services. There are currently two CARMA Streets base images , for which documentation and Dockerfiles can be found Streets Service Base and Streets Service Base Lanelet Aware.

CARMAStreets

The primary carma-streets repository can be found here and is part of the USDOT FHWA STOL github organization. Documentation on how the carma-streets functions, how it will evolve over time, and how you can contribute can be found at the above links, as well as on the Doxygen Source Code Documentation.

Data Collection

CARMA Streets services have several source from which data about usecase performance can be pulled. Kafka Message Topics can be pulled using the collect_kafka_logs.py script and includes message traffic on a configurable list of topics. Service log files can be found in a given CARMA Streets service logs/ directory. The collect_service_logs.sh adds all log files in this directory to a zip file and then deletes the originals. Some CARMA Streets services can configurable log performance in .csv file in the logs/ directory.

Collect Kafka Logs

This script uses docker exec to ssh into a running kafka container. Then using kafka container scripts to read all kafka data from a list of provided topics.

usage: collect_kafka_logs.py [-h] [--start_timestamp START_TIMESTAMP] [--end_timestamp END_TIMESTAMP] [--start_hours_ago START_HOURS_AGO] [--end_hours_ago END_HOURS_AGO]
                             [--topics TOPICS [TOPICS ...]] [--timeout TIMEOUT] [--zip ZIP]
                             outdir

Script to grab data from kafka

positional arguments:
  outdir                Folder name for the resulting folder logs are placed in

options:
  -h, --help            show this help message and exit
  --start_timestamp START_TIMESTAMP
                        Unix timestamp (seconds) for the first message to grab. Exclusive with start_hours_ago.
  --end_timestamp END_TIMESTAMP
                        Unix timestamp (seconds) for the last message to grab. Exclusive with end_hours_ago.
  --start_hours_ago START_HOURS_AGO
                        float hours before current time to grab first message. Exclusive with start_timestamp.
  --end_hours_ago END_HOURS_AGO
                        float hours before current time to grab last message. Exclusive with start_timestamp.
  --topics TOPICS [TOPICS ...]
                        list of topics to grab data from
  --timeout TIMEOUT     timeout for receiving messages on a topic, default is 5 seconds
  --zip ZIP             bool flag. When set to true, folder is compressed into a zip file.

Collection Service Logs

This script collects all CARMA Streets service log files, adds them to a zip file.

usage: collect_service_logs.sh [-h | --help] [--output <zip_file_name>.zip]
                                     [-c | --clear]

options:
-h, --help                Show usage
--output, -o              Name of the resulting zip file container CARMA Streets Service logs
--clear, -c               Adding this flag will delete log directories after creating zip.

Contribution

Welcome to the CARMA contributing guide. Please read this guide to learn about our development process, how to propose pull requests and improvements, and how to build and test your changes to this project. CARMA Contributing Guide

Code of Conduct

Please read our CARMA Code of Conduct which outlines our expectations for participants within the CARMA community, as well as steps to reporting unacceptable behavior. We are committed to providing a welcoming and inspiring community for all and expect our code of conduct to be honored. Anyone who violates this code of conduct may be banned from the community.

Attribution

The development team would like to acknowledge the people who have made direct contributions to the design and code in this repository. CARMA Attribution

License

By contributing to the Federal Highway Administration (FHWA) Connected Automated Research Mobility Applications (CARMA), you agree that your contributions will be licensed under its Apache License 2.0 license. CARMA License

Contact

Please click on the CARMA logo below to visit the Federal Highway Adminstration(FHWA) CARMA website.

CARMA Image

carma-streets's People

Contributors

abey-yoseph avatar adamlm avatar adev4a avatar afrye51 avatar codygarver avatar dan-du-car avatar kjrush avatar mishkamn avatar paulbourelly999 avatar saeid-solei avatar saikrishnabairamoni avatar snallamothu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar

carma-streets's Issues

Refactor intersection_model to use streets_service_base_lib

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • [ x ] Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Refactor intersection_model to use streets_service_base_lib to parse and store manifest.json configuration values.

Carma streets version where this issue was discovered

Expected behavior

All CARMA-Streets services use streets_service_base_lib to parse and store configuration values.

Actual behavior

Intersection_model had custom configuration parsing code.

Steps to reproduce the actual behavior

Related work

zapier test

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Carma streets version where this issue was discovered

Expected behavior

Actual behavior

Steps to reproduce the actual behavior

  1. Do this
  2. Then do this...

Related work

Remove OpenAPI REST server generated source code from intersection_model

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • [ x ] Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

This is a code organizational issue. The intersection_model src directory currently includes partially modified OpenAPI REST server code. This code needs to be moved to a location to clearly indicate that it is not CARMA-Streets developed code ( Possibly to ext or to utils). This code also needs to be regenerated to remove any edits made to include it. This may require developing a standard procedure and how and where to include generated code in CARMA-Streets and may require updating documentation.

Carma streets version where this issue was discovered

Expected behavior

NA

Actual behavior

NA

Steps to reproduce the actual behavior

NA

Related work

Sonar Scan Issues: Scheduling service

Types of Issue

Addressing the code quality issues and other issues got from the sonar cloud static metrics

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Carma streets version where this issue was discovered

Expected behavior

sonar scan addressed should be cleared.

Actual behavior

Steps to reproduce the actual behavior

  1. Do this
  2. Then do this...

Related work

https://sonarcloud.io/project/issues?id=usdot-fhwa-stol_carma-streets&resolved=false

docker-compose kafka broker ip

Types of Issue

The current broker ip is 127.0..0.1. This will not work when remote computer tries to connect to kafka broker

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Need to add environment variable for the IP address kafka broker connect to. Allow to dynamically update the IP address.
DOCKER_HOST_IP: ${DOCKER_HOST_IP}
ZK: ${DOCKER_HOST_IP}:2181
KAFKA_ADVERTISED_HOST_NAME: ${DOCKER_HOST_IP}

Carma streets version where this issue was discovered

N/A

Expected behavior

Running kafka broker in AWS EC2 instance.
Use kafka client in local VM to connect to kafka broker
Kafka client can view list of current available topics
create topic
pub data to topic
sub data from topic

Actual behavior

cannot connect to kafka broker.

Steps to reproduce the actual behavior

ssh into carma-streets ec2 instance
git clone
docker-compose up

open a terminal from local VM
download https://mirrors.advancedhosters.com/apache/kafka/2.8.0/kafka_2.13-2.8.0.tgz
tar -xzf kafka_2.13-2.8.0.tgz
cd kafka_2.13-2.8.0
bin/kafka-topics.sh --bootstrap-server 44.192.246.200:9092 --list

Related work

Sample test for Zapier Jira connection

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Carma streets version where this issue was discovered

Expected behavior

Actual behavior

Steps to reproduce the actual behavior

  1. Do this
  2. Then do this...

Related work

scheduling_service integration testing with one vehicle fixes

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Vehicle not able to stop at the intersection or the scheduling plan is not updated when the vehicle is stopped at the stop bar.

Carma streets version where this issue was discovered

Expected behavior

The scheduling plan shall be updated when the vehicle stop at the stop bar

Actual behavior

Steps to reproduce the actual behavior

  1. Do this
  2. Then do this...

Related work

Add logging statement to the scheduling service

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

This issue focuses on adding logs to the scheduling service and storing the necessary information for debugging and data processing in a CSV file.

Carma streets version where this issue was discovered

Expected behavior

The scheduling service creates a CSV file that contains the necessary information for debugging and data processing.

Actual behavior

Currently, only few information are just logged in CARMA Streets and nothing will be stored in a file.

Steps to reproduce the actual behavior

  1. Adding a message_logger service (header and cpp files) to the scheduling service
  2. ...

Related work

Create Standards for manifest.json configuration parameters

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • [ x ] Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Standardize naming convention for configuration parameters in manifest json files. Standardize documentation of these parameters. Also standardize valid formatting of manifest JSON files ( We could use parent/child node relations to included descriptions of parameters or to group parameters by functionality).

Carma streets version where this issue was discovered

Expected behavior

NA

Actual behavior

NA

Steps to reproduce the actual behavior

Related work

#115 #119

Sonar Scan Issue : Kafka clients

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Addressing the code quality issues and other issues got from the sonar cloud static metrics

Carma streets version where this issue was discovered

Expected behavior

sonar scan addressed should be cleared.

Actual behavior

Steps to reproduce the actual behavior

sonar scan reporting issue

Related work

https://sonarcloud.io/project/issues?id=usdot-fhwa-stol_carma-streets&resolved=false

Add CMake package install target for all internal CARMA-Streets libraries

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

CARMA-Streets contains several internal libraries that services depend on. Including these libraries and their dependencies manually through CMake commands can be a tedious, error-prone and messy process. Instead these libraries should all be installed as CMake packages, grouped into namespaces. As a package, a libraries internal dependencies are automatically retrieved by CMake reducing the complexity of CMake file to only describe their own target dependencies. This would allow for less messy and less error prone practices of including internal libraries.

Carma streets version where this issue was discovered

Expected behavior

internal libraries and their dependencies are retrievable using CMake find_package()

Actual behavior

internal libraries and their dependencies must be explicitly linked and included in order to be used.

Steps to reproduce the actual behavior

NA

Related work

Create common utility library to parse manifest.json configurations

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • [ x ] Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Each service in carma-streets currently parses its manifest json with its own implementation. This are sometimes substantially different. It may be worth will creating a utility library to establish a common way to stand up a carma-streets service and also parse these configuration files. This may help make each carma streets service implementation be more consistent with each other and make the services easier to read.

Carma streets version where this issue was discovered

Expected behavior

NA

Actual behavior

NA

Steps to reproduce the actual behavior

NA

Related work

#119 #118

Remove generated code from sonar scanner analysis

Types of Issue

  • [ x ] Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Currently the intersection model source code includes OpenAPI generated code for the REST server under the directory intersection_model/src/server/src/. This code should be excluded from the sonar scan report.

Carma streets version where this issue was discovered

Expected behavior

Sonar Scanner Analysis on CARMA-Streets code only

Actual behavior

Sonar Scanner Analysis on Open API REST Server Generated Code.

Steps to reproduce the actual behavior

  1. https://sonarcloud.io/code?id=usdot-fhwa-stol_carma-streets&selected=usdot-fhwa-stol_carma-streets%3Aintersection_model%2Fsrc%2Fserver%2Fsrc

Related work

Sonar Scanner Coverage Report Anomaly

Types of Issue

  • [ x ] Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Currently sonar scanner coverage statistics seem to be inaccurate with the gcovr report.

CircleCI gcovr coverage report for kafka clients


File                                       Lines    Exec  Cover   Missing
include/kafka_consumer_worker.h               48      14    29%   33,35-36,40,42-43,45-46,48-50,52-53,57,59-60,62-63,66,68-72,88,93-95,101-103,105-107
include/kafka_producer_worker.h               28      21    75%   29,31-33,59-61
src/kafka_client.cpp                          46      25    54%   46,61-62,74-75 [* 15,17-19,31,33-35,48,50-52,78,80-82]
src/kafka_consumer_worker.cpp                 92      56    60%   21-22,28-29,36-37,42-43,49-50,55,62-63,73-74,80-81,103-104,135-144,146-148,150,152-154
src/kafka_producer_worker.cpp                 63      44    69%   25-26,32-33,39-40,47-48,58,60,71,75-76,91-92,104-105 [* 148,151]
src/main.cpp                                  40       0     0%   3,5-12,14,18-19,21,24,26-27,29,33,35,38,40-46,48,52-53,55,57,59,61,64,67,69-72
TOTAL                                        317     160    50%

Sonar Scanner coverage report for kafka clients

Lines of Code | Bugs | Vulnerabilities | Code Smells | Security Hotspots | Coverage | Duplications
  |   | kafka_client.h | 26 | 0 | 0 | 0 | 0 | — | 0.0% |  
  |   | kafka_consumer_worker.h | 129 | 0 | 0 | 13 | 0 | 0.9% | 0.0% |  
  |   | kafka_producer_worker.h | 80 | 0 | 0 | 10 | 0 | 1.6% | 0.0%
  |   | kafka_client.cpp | 82 | 0 | 0 | 0 | 0 | 48.9% | 0.0% |  
  |   | kafka_consumer_worker.cpp | 131 | 0 | 0 | 6 | 0 | 0.0% | 0.0% |  
  |   | kafka_producer_worker.cpp | 106 | 0 | 0 | 6 | 0 | 0.0% | 0.0% |  
  |   | main.cpp | 67 | 0 | 0 | 1 | 0 | 0.0% | 31.5%

Carma streets version where this issue was discovered

release_fiesta

Expected behavior

Coverage results in gcovr and sonar scanner should match or at least be similar.

Actual behavior

Coverage results in gcovr are substantially different from sonar scanner

Steps to reproduce the actual behavior

  1. Trigger CircleCI workflow
  2. Observe code coverage in CircleCI "build" workflow
  3. Observe sonar scanner coverage

Related work

Simplify Scheduling logic and introduce configurable coefficients for performance tuning

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • [ x ] Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Simplify Scheduling logic and introduce configurable coefficients for performance tuning. This will allow for configuring carma-streets scheduling to give vehicles the smoothest possible trajectory for different scenarios.

Carma streets version where this issue was discovered

3.11.0

Expected behavior

NA

Actual behavior

NA

Steps to reproduce the actual behavior

  1. NA

Related work

#38

Intersection Model service design investigation or update if needed

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

The current Intersection Model package has a REST web service hosted on a server. It is deployed as a separate container.
Investigate how to incorporate this REST API in message_service.
Answer Questions:
Can we spin up another thread in message service to run the REST API?
Can we eliminate having both message_service and intersection model reading lanelet2 library.

Carma streets version where this issue was discovered

N/A

Expected behavior

The questions need to be answered

Actual behavior

The intersection model itself is a self-contained service.

Steps to reproduce the actual behavior

N/A

Related work

Message services integration testing with one vehicle issues

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

The message service sometimes get the staled BSM message and consider it during the mapping. That cause the vehicle status and intent message current vehicle location inconsistent with the actual vehicle location.

The vehicle stop at the stop bar for more than 3 seconds.

Carma streets version where this issue was discovered

Expected behavior

The vehicle status and intent should report the correct information about the vehicle.
The vehicle stop at the stop bar for less than 3 seconds.

Actual behavior

The message service sometimes get the staled BSM message and consider it during the mapping. That cause the vehicle status and intent message current vehicle location inconsistent with the actual vehicle location.

Steps to reproduce the actual behavior

NA

Related work

Docker-compose dependencies fix and image names update

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Update the dependencies between containers: message_service depends on Kafka. scheduling_service depends on intersection model server and message service. V2xhub depends on mysql db. php depends on the v2xhub and mysql db.

Update docker-compose to pull the correct images:
usdotfhwastoldev/message_services:latest
usdotfhwastoldev/scheduling_service:latest
usdotfhwastoldev/intersection_model:latest

It might be better to move the vector_map into one folder like MAP. Then message_service and intersection model will read the vector_map from that location.

Carma streets version where this issue was discovered

Expected behavior

Update the dependencies between containers: message_service depends on Kafka. scheduling_service depends on intersection model server and kafka. V2xhub depends on mysql db. php depends on the v2xhub and mysql db.
Image names are updated as described above.
The vector_map location should be updated.

Actual behavior

Steps to reproduce the actual behavior

NA

Related work

Sonar Scan Smell [BUG] in streets_singleton

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

https://sonarcloud.io/project/issues?resolved=false&sinceLeakPeriod=true&types=BUG&id=usdot-fhwa-stol_carma-streets&open=AX__ydd1OqmeBkYbLTxs

Carma streets version where this issue was discovered

Expected behavior

NA

Actual behavior

NA

Steps to reproduce the actual behavior

NA

Related work

scheduling_service integration testing with one vehicle issues

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Vehicle not able to stop at the intersection or the scheduling plan is not updated when the vehicle is stopped at the stop bar.
The schedule plan transit from null directly to access = 1, the vehicle does not stop. The vehicle only stops when it receives the updated schedule with access = 0, otherwise it will cruise as normal.

Carma streets version where this issue was discovered

Expected behavior

The scheduling plan shall be updated when the vehicle stop at the stop bar

Actual behavior

Steps to reproduce the actual behavior

NA

Related work

Message correlation service

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Design Message Correlation Library/Service including how we will create/use dependent libraries and how it will interface with the Scheduling Service. Some design questions that need to be answered:

What form will the Message Correlation part of CARMA-Streets take ( Separate Service/Docker Image, Library dependency for other Services to use)?

How will we persist/publish correlated updates ( In memory DB, Kafka)?

How will we correlate BSM, MobilityOperation and MobilityPath ( static vehicle ID, BSM ID, sequence ID)

What dependent libraries are necessary ( lanelet2 and extensions, Map Library )

Carma streets version where this issue was discovered

Expected behavior

Design and initial implementation for message service to satisfy those answer the above questions.

Actual behavior

N/A

Steps to reproduce the actual behavior

N/A

Related work

Investigate using more lightweight base images for carma-streets services.

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Might be useful to investigate using more lightweight containers like alpine (see intersection_model) instead of ubuntu images.

Carma streets version where this issue was discovered

Expected behavior

NA

Actual behavior

NA

Steps to reproduce the actual behavior

Related work

Refactor kafka_client directory to use streets_service_base configuration library.

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • [ x ] Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Refactor kafka_client directory to use streets_service_base configuration library

Carma streets version where this issue was discovered

Expected behavior

All carma-streets use common streets_service_base configuration library for parsing manifest.json files

Actual behavior

Custom code to parse manifest.json configuration file inside kafka_clients.

Steps to reproduce the actual behavior

Related work

street_configuration singleton does not update loglevel

Types of Issue

  • [ x ] Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Streets Configuration singleton update does not check or update loglevel parameter if changed while docker container is up.

Carma streets version where this issue was discovered

Expected behavior

Any changes to the manifest.json configuration file will cause updates to the street_configuration singleton.

Actual behavior

Changes to loglevel parameter will not update the loglevel for the streets_configuration singleton.

Steps to reproduce the actual behavior

  1. Deploy any carma-street container.
  2. Edit loglevel parameter
  3. Does loglevel update?

Related work

Update Ubuntu 16.04-based machine images to 20.04 images

Types of Issue

This issue to update the Ubuntu 16.04-based machine images on CircleCI to latest images as most of the old version images are going to be removed in March 2022.

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Images that are going to be deprecating is fount in this link (https://circleci.com/blog/ubuntu-14-16-image-deprecation/?mkt_tok=NDg1LVpNSC02MjYAAAGCnda7yQbo-EdeB3o7wskWySZY-1PzTIxmfx_iRlT1Czz_LQiWizACRuKwkXJgTQefhaW877aNs5TIJwP71Z7IWxFuExI74LnMs3F13z5bYPJ1)
Email from Circle CI ops as below:
We are deprecating Ubuntu 16.04-based machine images on CircleCI in preparation for an EOL on Tuesday, May 31, 2022 to ensure your builds remain secure. For a detailed overview of how this will affect your workflow, read the blog article here.

We will also be conducting temporary brownouts on Tuesday, March 29, 2022, and again on Tuesday, April 26, 2022 during which these images will be unavailable.

We are contacting you because one or more of your projects has a job that uses an Ubuntu 16.04-based image.

The project(s) using an image based on Ubuntu 16.04 are:
• [V2X-Hub] [carma-streets]

If you have specified an Ubuntu 16.04-based image please see our migration guide to upgrade to a newer version of Ubuntu image in order to avoid any service disruption during the brownout & subsequent EOL.

We will also be releasing a CircleCI Ubuntu 22.04 image on April 22nd offering the flexibility to upgrade to the latest LTS version of Ubuntu image before we remove older versions permanently. A beta version of the image will be available March 21st.

Our team is committed to getting you the most performant builds while maintaining security.

Carma streets version where this issue was discovered

Expected behavior

Actual behavior

Steps to reproduce the actual behavior

  1. Do this
  2. Then do this...

Related work

Docker deployment process improvement

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Carma streets is reusing the components from v2xhub. There are some volumes from V2xhub, and those volume configuration need to be manually copied over from v2xhub to carma-streets during the deployment. Looking for a solution to automate all the steps:
Script to copied the volumes from v2xhub to carma-streets.
Script to create secret files: mysql_password.txt and mysql_root_password.txt. Making sure the v2xhub is up and running successfully with these credentials.
Successfully push the images (message_service, intersection_model, scheduling_service) into dockerhub registry

Carma streets version where this issue was discovered

NA

Expected behavior

Script to copied the volumes from v2xhub to carma-streets.
Script to create secret files: mysql_password.txt and mysql_root_password.txt. Making sure the v2xhub is up and running successfully with these credentials.
Successfully push the images (message_service, intersection_model, scheduling_service) into dockerhub registry

Actual behavior

None of the above applies.

Steps to reproduce the actual behavior

NA

Related work

Memory Leak in message service

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

As large volume of live data feed in into the message service, the service dynamically allocates memory, and the allocated memory is not freed in time, this cause the host machine (During the validation testing) run out of memory. The host machine terminated the message service process.

Carma streets version where this issue was discovered

3.11

Expected behavior

The message service memory usage should not increase over time.

Actual behavior

The message service memory usage increases over time.

Steps to reproduce the actual behavior

Publish live MobilityOperation, MobilityPath, and BSM data as fast as it can. The message service constantly consumes the messages and generates the vehicle status and intent messages. Monitoring the memory usage from terminal.

Related work

Sonar Scan Issue : Scheduling service

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Addressing the code quality issues and other issues got from the sonar cloud static metrics

Carma streets version where this issue was discovered

Expected behavior

sonar scan addressed should be cleared.

Actual behavior

Steps to reproduce the actual behavior

  1. Do this
  2. Then do this...

Related work

https://sonarcloud.io/project/issues?id=usdot-fhwa-stol_carma-streets&resolved=false

Remove unnecessary kafka topics

Types of Issue

  • [ x ] Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

The docker-compose deployment for kafka includes two topics that are no longer used. I propose these be removed to avoid any confusion. The two topics v2xhub_in and v2xhub_out were initially created to demo the CARMA-Streets V2X-Hub plugins capability to transmit J2735 messages to a CARMA-Streets deployment via kafka. We have since created more specific topics for each message type that is transferred and so I believe we can remove these.

Carma streets version where this issue was discovered

NA

Expected behavior

Docker-compose kafka deployment only creates necessary topics

Actual behavior

Docker-compose kafka deployment created two unnecessary topics

Steps to reproduce the actual behavior

  1. Run docker-compose up and use kafka scripts or kowl to list topics. Two topics that are created will never have any traffic.

Related work

Core dump issue when consumer consume message at 100 HZ

Types of Issue

Core dump issue when consumer consume message at 100 HZ

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Core dump issue when consumer consume message at 100 HZ or 10 HZ

Carma streets version where this issue was discovered

Expected behavior

Consumer can consumer msg as normal does

Actual behavior

Core dump issue when consumer consume message at 100 HZ or 10 HZ

Steps to reproduce the actual behavior

Related work

Sample Issue to test the Zapier connection for Enhancement

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Carma streets version where this issue was discovered

Expected behavior

Actual behavior

Steps to reproduce the actual behavior

  1. Do this
  2. Then do this...

Related work

Check and update BSM ID to make sure BSM, MobilityOperation and MobilityPath can coorelate with each other

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Check and update BSM ID to make sure BSM, MobilityOperation and MobilityPath can coorelate with each other

Carma streets version where this issue was discovered

NA

Expected behavior

BSM, MobilityOperation and MobilityPath can coorelate with each other.
BSM ID from the above received three messages should all use decimal. If the received BSM ID is in hex format, it needs to be converted into decimal format for comparision.

Actual behavior

BSM, MobilityOperation and MobilityPath cannot coorelate with each other

Steps to reproduce the actual behavior

NA

Related work

Remove scheduling_service unused configuration parameters

Types of Issue

  • [ x ] Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

scheduling_service contains a configurable parameter called MAX_VALID_SPEED which is currently unused. We should either use it or remove it

Carma streets version where this issue was discovered

3.11.0

Expected behavior

NA

Actual behavior

NA

Steps to reproduce the actual behavior

NA

Related work

#115 #118

Allow forward slash in the GitHub branch name

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Allow forward slash in the GitHub branch name

Carma streets version where this issue was discovered

Expected behavior

Allow forward slash in the GitHub branch name

Actual behavior

The Circle CI build fails if there is a slash in the branch name

Steps to reproduce the actual behavior

NA

Related work

Create Map Library

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • [ x ] Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Create a library that is includes lanelet2 and the autware.io extensions to consume a lanelet2 osm file. This component will be responsible for translating long,lat location information into lane specific information and providing calculation about lane conflicts, distance to end of lane, and regulatory elements.

Carma streets version where this issue was discovered

Expected behavior

Actual behavior

Steps to reproduce the actual behavior

  1. Do this
  2. Then do this...

Related work

Zapier connection test

Types of Issue

Test Zapier connection with carma streets

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Carma streets version where this issue was discovered

Expected behavior

Actual behavior

Steps to reproduce the actual behavior

  1. Do this
  2. Then do this...

Related work

Docker Image Build Revision

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • [ x ] Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Currently we are building all docker images for CARMA-Streets using CircleCI. The docker build process can be moved to docker hub which would clean up CircleCI workflow. We are also not utilizing any custom base images. Using a custom base image for lanelet2 dependent services would reduce resources step on building these dependencies, possibly reduce frequency of building these dependencies to only when they change and reduce build time for lanelet2 dependent services.

Carma streets version where this issue was discovered

Expected behavior

Actual behavior

Steps to reproduce the actual behavior

  1. Do this
  2. Then do this...

Related work

Investigation: Using MAP information to get intersection geometry information

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • [ x ] Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

After reading documentation, MAP messages seem to have much of the information about intersection geometry needed for the scheduling service including entry lanes and lane ids. For UC3 to map signal state to intersection lane state we will likely need this information anyways. This issue is to investigate whether we can remove some or all lanelet2 dependency from carma-streets and instead use MAP information, which I believe also includes the absolute stopping line position.

Carma streets version where this issue was discovered

Expected behavior

CARMA-Streets can consume MAP messages and understand intersection geometry

Actual behavior

CARMA-Streets is able to consume a custom lanelet2 map and can understand intersection geometry from that.

Steps to reproduce the actual behavior

NA

Related work

Sonar Scan Issue : Kafka clients

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Addressing the code quality issues and other issues got from the sonar cloud static metrics

Carma streets version where this issue was discovered

Expected behavior

sonar scan addressed should be cleared.

Actual behavior

Steps to reproduce the actual behavior

sonar scan reporting issue

Related work

https://sonarcloud.io/project/issues?id=usdot-fhwa-stol_carma-streets&resolved=false

Scheduling Service Memory Core dump

Types of Issue

  • [ x ] Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

After running for multiple days we have twice ran into a memory core dump of the scheduling service that caused the container to restart. This happened Friday, January 28th and Tuesday February 1st. Both log files of the restart are available on the shared drive. Could not find core.dump files in the /tmp/ directory of the relevant container or the host machine. Searches online indicate that we may want to create volumes that map to our local /tmp directory to make sure core dumps are not lost but this may require further investigation. By checking for the restart in the logs we can determine if this memory core dump was through while the schedule was populated or not. In both occasions the scheduling_service was running for at least two days.

Carma streets version where this issue was discovered

release_fiesta

Expected behavior

No memory core dumps

Actual behavior

Memory core dumps

Steps to reproduce the actual behavior

  1. Turn scheduling delta really low to increase scheduling frequency
    NA (Requires investigation)

Related work

Remove dependencies from Scheduling Service Test Case

Types of Issue

  • [ x ] Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Scheduling Service test cases currently depend on the integration model service being up and running. Without this, code coverage reports are substantially lower. This dependency should be mocked out using gtext

Carma streets version where this issue was discovered

Expected behavior

CI Coverage Reports return correct coverage results

Actual behavior

CI Coverage Reports return incorrect coverage results

Steps to reproduce the actual behavior

  1. Trigger CI build workflow.

Related work

Enable Docker Build Layer Caching

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • [ x] Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Enable CircleCI to cache previous build layers and only rebuild changed layers. This will expedite the build process and the pulling process.

Carma streets version where this issue was discovered

release_fiesta

Expected behavior

Build layer caching is enabled.

Actual behavior

Each build has completely unique build layers regardless of actually changes

Steps to reproduce the actual behavior

  1. trigger CircleCI build process.
  2. inspect build layers of sequential CircleCI pushed docker images

Related work

Create Sample Lanelet2 Data for Unit Tests

Types of Issue

  • [ x ] Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Many of the current unit tests for the message service currently require the THFRC lanelet2 map to run and pass test cases. This map can not be committed to a public repo. We need to develop a way to load sample lanelet2 map information to allow our unit tests to run as part of the circle ci build process. Sample lanelet2 map we are allowed to use can be found here.

Carma streets version where this issue was discovered

Expected behavior

All unit tests cases run during circle ci build process

Actual behavior

Many unit test cases never run due to missing lanelet2 map

Steps to reproduce the actual behavior

  1. Run CircleCI workflow
  2. Message service should be scanned by the sonar scanner and coverage report should be generated.

Related work

Fix scheduling service ST estimation issues

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

The estimated stopping time (ST) in the schedule plan is not always correct. Sometimes, the estimated ST drops to 1-3 seconds later than the current timestamp while the vehicle is still far from the stop line.

In this issue, we aim to find the source of the issue and resolve it.

Carma streets version where this issue was discovered

Expected behavior

Actual behavior

Steps to reproduce the actual behavior

  1. Do this
  2. Then do this...

Related work

Fix build and coverage script for Unit Test Coverage

Types of Issue

  • [ x ] Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Currently many of the unit test case are reporting incorrect coverage statistics due to configuration relative file paths being incorrect. Also for gcovr we are including unnecessary files. Both the build.sh script and and coverage.sh script need to be updated.

Carma streets version where this issue was discovered

NA

Expected behavior

Gcovr return correct coverage statistics

Actual behavior

Unit test fail on loading configuration files and return incorrect unit test coverage.

Steps to reproduce the actual behavior

  1. Run unit test cover

Related work

Add debug flags to Dockerfile cmake calls

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • [ x ] Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Add -DCMAKE_BUILD_TYPE="Debug" flag to cmake calls in dockerfile for services to allow for line tracing in core dump files

Carma streets version where this issue was discovered

Expected behavior

core dump files do not contain line numbers

Actual behavior

core dump files contain line numbers

Steps to reproduce the actual behavior

  1. Trigger core dump
  2. Use gdb to process core dump file

Related work

Improve spdlog deployment to allow asynchronous, rotating logs with log level configuration.

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • [ x ] Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Currently we are using the most basic deployment of the an spd logger. To allow for more configurable and higher performance logs we want to implement our terminal logger to be asynchronous, rotating and allow for configurable log levels. Also we want to configure this logger to write to a file and the output

Carma streets version where this issue was discovered

release_fiesta

Expected behavior

Configurable, asynchronous, rotating file and terminal logs

Actual behavior

Hard coded log-level, output to terminal only

Steps to reproduce the actual behavior

  1. Run system

Related work

Implement CircleCI workflow trigger from github tags

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • [ x ] Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

CircleCI offers workflows trigger by github tags to allow automatic release builds. This will allow us to create releases and push appropriate issues simply by pushing a github tag. See V2X-Hub Implementation. We would need additional work flows for push release candidate images to correct organizations.

Carma streets version where this issue was discovered

Expected behavior

Automatically trigger release builds

Actual behavior

Manual release builds

Steps to reproduce the actual behavior

  1. Do this
  2. Then do this...

Related work

Lanelet2 related functions performance issues

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

There are about min 300 to max 900 milliseconds delay in these lanelet2 related functions for one vehicle testing. As more vehicles and more vehicles sending messages concurrently, the delay can be incrementally larger.
Attached the testing logs that show the delay:
scheduling_service_01_20_run1.txt
scheduling_service_01_20_run2.txt

Carma streets version where this issue was discovered

NA

Expected behavior

The service shall finish calculation/translation as soon as it can.

Actual behavior

It is slow

Steps to reproduce the actual behavior

NA

Related work

Replace Global Variables in Scheduling Service with Singletons

Types of Issue

  • Anomaly report (something appears to not work correctly)
  • [ x ] Enhancement request (describe the enhancement being requested)
  • Other (please ensure the description clarifies why the issue doesn’t fall into either of the above categories)

Descriptive summary

Currently, in the scheduling service we have multiple global variables. Using global variables in C++ is not preferred since unrestricted access to a variable in an entire C++ project can create hard to read/follow functionality and introduce bugs in the form of concurrent modification and other unintentional behavior. These should be replaced with thread safe Singletons.

Carma streets version where this issue was discovered

3.11.0

Expected behavior

NA

Actual behavior

NA

Steps to reproduce the actual behavior

  1. NA

Related work

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.