saif-ashraf99/lidar-obstacle-detection

Lidar-Obstacle-Detection

The directory structure of the starter code looks like:

• src
  • render
    • box.h: struct definition for box objects
    • render.h, render.cpp: classes and methods for rendering objects
  • sensors
    • lidar.h: functions using ray casting for creating PCD
  • environment.cpp: main file for using PCL viewer and processing/visualizing PCD
  • processPointClouds.h, processPointClouds.cpp: functions for filtering, segmenting, clustering, boxing, loading and saving PCD

Lidar and Point Clouds

1. Create a Lidar pointer object on the heap, taking two parameters: std::vector<Car> and setGroundSlope of 0. The Lidar::scan() method does a ray casting and returns a point cloud pointer object pcl::PointCloud<pcl::PointXYZ>::Ptr. Call renderRays() to plot rays on the viewer.

2. Increase the lidar resolution by tweaking the constructor of Lidar class: numLayers, horizontalAngleInc. Set minDistance to 5 meter to remove points from the vehicle's rooftop. Set sderr to 0.2 to add some noises to the PCD.

3. Remove rendering for the highway scene and rays, but enable rendering for the point cloud using renderPointCloud().

Point Cloud Segmentation

1. In the processPointCloud.cpp, to implement the ProcessPointClouds::SegmentPlane() method, use a pcl::SACSegmentation<PointT> object to segment the planar component from the input point cloud. Next to implement the SeparateClouds() helper method, use a pcl::ExtractIndices<PointT> object to extract the points not belong to the plane as the obstacles.

2. In the environment.cpp, create a ProcessPointClouds<pcl::PointXYZ> object, call SegmentPlane() to separate the plane and obstacle. Finally, render the plane and obstacle point clouds.

Identify Different Obstacles using Euclidean Clustering with PCL

1. In the processPointCloud.cpp, to implement the ProcessPointClouds::Clustering() method, create a Kd-tree representation pcl::search::KdTree<PointT>::Ptr for the input point cloud, configure the parameters for the Euclidean clustering object pcl::EuclideanClusterExtraction<PointT> and extract the clusters in the point cloud. In the environment.cpp, call the clustering function on the segmented obstacle point cloud, render clustered obstacle in different colors.

Bounding Boxes

1. Once point cloud clusters are found, we can add bounding boxes around the clusters. The boxed spaces should be considered as an area/object that our car is not allowed to enter, otherwise it would result a collision. Call ProcessPointClouds::BoundingBox() method, which finds the max and min point values as the boundary values for the Box data structure. Then render the Box structure for each cluster.

Load Real PCD

1. Create a new point processor for the real PCD from a cityBlock, the code is similar to the simpleHighway function. The point type is pcl::PointXYZI where the I indicates the intensity. The real PCD files are located at src/sensors/data/pcd/data_1/ directory.

Filter with PCL

1. To implement the ProcessPointClouds::FilterCloud() in the processPointClouds.cpp, pcl::VoxelGrid<PointT> class is applied for Voxel Grid filtering, and pcl::CropBox<PointT> class is applied for ROI-based filtering. The Eigen::Vector4f class has four parameters representing x, y, z coordinates and the last one should be 1.0. We are interested in a good amount of distance in front or at back of the car and surroundings of the car. Point cloud data outside of the ROI should be removed, including the rooftop points.

2. In the environment.cpp, call ProcessPointClouds::FilterCloud() function in the cityBlock(). Input a leaf size of 0.2m, so that the voxel size is large enough to help speed up the processing but not so large that object definition is preserved.

Obstacle Detection with Real PCD

1. Once having a filtered PCD, we can deploy the same segmentation and clustering techniques implemented previously in the cityBlock().

2. Tweak the Eigen::Vector4f minPoint/maxPoint for ProcessPointClouds::FilterCloud() and int minSize/maxSize for ProcessPointClouds::Clustering().

Stream PCD

1. Create a vector stream to store paths to the PCD files chronologically. Create a new cityBlock() function, which processes the input point cloud from the external. In the main() function of environment.cpp, inside the viewer update loop, read PCD file, process it and update the viewer.

III. Lidar Obstacle Detection Project

The previous exercises have created a processing pipeline for detecting the obstacles using PCL segmentation and clustering methods. In the final project, we have to implement the same obstacle detection pipeline but with the 3D RANSAC segmentation, KD-Tree, and Euclidean clustering algorithm we created in the quizzes.

1. Create a ProcessPointClouds::SegmentPlaneRansac() function in the processPointClouds.cpp, which uses the 3D RANSAC segmentation for plane implemented in the quiz. And call this segmentation function in the cityBlock() of environment.cpp.

2. Create ProcessPointClouds::ClusteringEuclidean and its helper function ProcessPointClouds::clusterHelper in the processPointClouds.cpp, which reuses the Euclidean clustering with KD-Tree with some modification from the quiz. And call this clustering function in the cityBlock() of environment.cpp.