awesome self-driving-car_a review of tracking, prediction and decision maki

Awesome Self-Driving Cars(https://github.com/sindresorhus/awesome)

Introduction

A curated list of all awesome things related to self-driving car.

Table of Contents

• Opensource code
  • Fullstack
  • Library
• Courses
• Papers & Blogs
  • Overview
  • HD Map
  • Simulation & Data generation
  • Localization
  • Calibration
  • Perception
  • Prediction
  • Planning
  • Control
  • End-to-End
  • Misc
• Datasets & Benchmarks
• Algorithms
  • Overview
  • HD Map
  • Simulation & Data generation
  • Localization
  • Calibration
  • Perception
  • Prediction
  • Planning
  • Control
  • End-to-End
  • Misc
• Systems
  • RTOS
• Cloud service
  • Simulation Service
  • HD Map Service
  • Data Service
  • Monitor Service
  • OTA
• Hardware
  • Computing Unit
  • sensors
    • GPS/IMU
    • Camera
    • LiDAR
    • RADAR
    • Ultrasonic Sensor
  • CAN card
  • Drive by wire
  • V2X
  • HMI Device
  • Black Box
• Big Players
• Legislation

Autonomous driving technology stack

First, let’s look at the technology stack for autonomous driving. In order to understand the full stack of autonomous driving. After that, you can learn the corresponding skill tree.
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-iQ9dYuKe-1628597229733)(technology_stack.png)]

Opensource

Fullstack

• apollo - Apollo is an open source autopilot platform that contains almost everything. Including hardware, systems, vehicle platforms, cloud services, etc. You can quickly build a self-driving system of your own by Apollo.
• autoware - The original Autoware project built on ROS 1. Launched as a research and development platform for autonomous driving technology.
• openpilot - Openpilot is an open source driver agent. Use the iphone to control the car, which provides adaptive cruise control (ACC) and lane keeping assist (LKAS).

Library

• ROS - The Robot Operating System (ROS) is a flexible framework for writing robot software. It is a collection of tools, libraries, and conventions that aim to simplify the task of creating complex and robust robot behavior across a wide variety of robotic platforms.
• OpenCV library - OpenCV (Open Source Computer Vision Library) is an open source computer vision and machine learning software library. OpenCV was built to provide a common infrastructure for computer vision applications and to accelerate the use of machine perception in the commercial products.
• Point Cloud Library - Point Cloud Library (PCL) is a standalone, large open project for 2D / 3D imagery and point cloud processing. Widely used to process laser point cloud data.
• TensorFlow - TensorFlow is an open source software library for numerical computation using data flow graphs. Used for automatic driving perception and prediction.
• ompl - The Open Motion Planning Library.

Courses

• Udacity Self-Driving Car Nanodegree - Udacity’s flagship program is sponsored by many self-driving car hiring partners. The nanodegree program includes 3 terms: 1) Term 1: Computer Vision and Deep Learning, 2) Term 2: Sensor Fusion, Localization, and Control, and 3) Term 3: Path Planning, Elective, and Systems. Each term costs $800.
• MIT 6.S094: Deep Learning for Self-Driving Cars - This class is an introduction to the practice of deep learning through the applied theme of building a self-driving car. It is open to beginners and is designed for those who are new to machine learning, but it can also benefit advanced researchers in the field looking for a practical overview of deep learning methods and their application. By the way, it’s free!
• BitTiger Build Your Own Autonomous Vehicle Mastery Program - Two weeks of live classes in Bay Area taught by engineers from Vector.ai on building a self-driving mini car from ground up. Topics include deep learning, ROS, sensors, computer vision, localization, mapping and control. The program costs $7,000.
• Apollo Autopilot Introduction - Note: it’s a Chinese tutorial. This is an open class between Baidu and Peking University. It is very comprehensive and detailed, which is a good introductory course.

Papers & Blogs

Overview

• A Survey of Autonomous Driving: Common Practices and Emerging Technologies - 2020
• A Survey of Deep Learning Techniques for Autonomous Driving - 2020
• Self-Driving Cars: A Survey - 2019
• Towards Fully Autonomous Driving: Systems and Algorithms - 2011

HD Map

• Highly Efficient HD Map Creation: Accelerating Mapping Process with GPUs - An introduction PPT (2017)

Simulation & Data generation

Localization

• A survey of the state-of-the-art localization techniques and their potentials for autonomous vehicle applications - JIOT 2017
• Robust and Precise Vehicle Localization based on Multi-sensor Fusion in Diverse City Scenes - ICRA 2018
• Map-Based Precision Vehicle Localization in Urban Environments -
• Robust Vehicle Localization in Urban Environments Using Probabilistic Maps -

SLAM

• Simultaneous localization and mapping: A survey of current trends in autonomous driving - 2017
• Past, Present, and Future of Simultaneous Localization And Mapping: Towards the Robust-Perception Age - 2016
• A critique of current developments in simultaneous localization and mapping - 2016

Perception

Survey

• Computer Vision for Autonomous Vehicles: Problems, Datasets and State-of-the-Art - CVPR 2017
• Deep Multi-modal Object Detection and Semantic Segmentation for Autonomous Driving: Datasets, Methods, and Challenges - boschresearch’s collection about datasets, detection and segmentation.

Object Detection

• Object Detection in 20 Years: A Survey - CVPR 2019
• Object Detection With Deep Learning: A Review - CVPR 2018
• Deep Learning for Generic Object Detection: A Survey - CVPR 2018
• 50 Years of object recognition: Directions forward - 2013

3D Object Detection

• 3D Object Detection for Autonomous Driving: A Survey - CVPR 2021

Object Tracking

• Multiple Object Tracking: A Literature Review - CVPR 2014

Lane Detection

• Recent progress in road and lane detection: a survey -

Data Fusion

• Multisensor data fusion: A review of the state-of-the-art
• A Review of Data Fusion Techniques
• A Comprehensive Review of The Multi-sensor Data Fusion Architectures
• A Survey of Multisensor Fusion Techniques, Architectures and Methodologies

Prediction

• A Review of Tracking, Prediction and Decision Making Methods for Autonomous Driving - LG 2019

Planning

Survey

• Planning and Decision-Making for Autonomous Vehicles - 2018
• A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles - Robotics 2016
• A Review of Motion Planning Techniques for Automated Vehicles - 2016
• Real-time motion planning methods for autonomous on-road driving: State-of-the-art and future research directions - 2015

Other

• ChauffeurNet: Learning to Drive by Imitating the Best and Synthesizing the Worst - Waymo’s paper about how to train a policy for autonomous driving via imitation learning that is robust enough to drive a real vehicle.
• Baidu Apollo EM Motion Planner - A real-time motion planning system based on the Baidu Apollo (open source) autonomous driving platform.

Control

• A Survey of Motion Planning and Control Techniques for Self-driving Urban Vehicles - Robotics 2016

End-to-End

• End to End Learning for Self-Driving Cars - 2016 NVIDIA

Misc

• An Introduction to LIDAR - Awesome introduction by Voyage about the key sensor of self-driving cars.
• Learning a Driving Simulator - comma.ai’s approach for self-driving cars is based on an agent that learns to clone driver behaviors and plans maneuvers by simulating future events in the road. This paper investigates variational autoencoders with classical and learned cost functions using generative adversarial networks for embedding road frames. A transition model is learned in the embedded space using action conditioned Recurrent Neural Networks (RNNs).
• 16 Questions About Self-Driving Cars - a16z’s Frank Chen goes over the 16 most commonly asked questions, technical and non-technical, about self-driving cars.
• Ways to think about cars - Awesome blog post by a16z’s Benedict Evans on electric cars, on-demand car services, and self-driving cars.
• The Third Transportation Revolution - Awesome blog post by John Zimmer on Lyft’s vision for self-driving cars. Spoiler alert, John predicts self-driving cars will account for the majority of Lyft rides within 5 years. And by 2025, private car ownership will all-but end in major U.S. cities.
• Cars and second order consequences - Benedict Evans on the impact of electric and autonomy on cars and beyond.

Datasets & Benchmarks

• KITTI Vision Benchmark Suite [Images] - Large vision benchmark dataset with objection detection evaluation training/testing images and leaderboard on cars and pedestrians.
• Cityscapes - Semantic, instance-wise, dense pixel annotations of 30 classes.
• comma.ai’s Driving Dataset [Videos] - Seven and a quarter hours (~ 80 GB) of largely highway driving. With this dataset, comma.ai’s founder George Hotz trained a self-driving car all by himself.
• Udacity’s Driving Dataset [Videos] - Eight hours (over 280 GB) of driving data collected for their open source self-driving car challenges. Udacity also provides convenient scripts to port the data.
• Washington DC’s Lidar Data - Lidar point cloud of the entire Washington DC area is made available by the District of Columbia’s Office of the Chief Technology Officer (OCTO).
• Apolloscape - Apolloscape provides images with 10x higher resolution and pixel-level annotation. And also Provides multiple levels of scene complexity.
• nuScenes - The nuScenes dataset (pronounced /nuːsiːnz/) is a public large-scale dataset for autonomous driving provided by nuTonomy-Aptiv.
• Waymo Open Dataset - The Waymo Open Dataset is comprised of high-resolution sensor data collected by Waymo self-driving cars in a wide variety of conditions.

Traffic Sign

• STSD - More than 20 000 images with 20% labeled, Contains 3488 traffic signs.
• LISA - 7855 annotations on 6610 frames.
• Tsinghua-Tencent 100K - 100000 images containing 30000 traffic-sign instances.
• German Traffic Sign [Images] - More than 50,000 images and 40 classes of traffic signs.

Algorithms

Systems

RTOS

Cloud service

Simulation Service

• Udacity’s Self-Driving Car Simulator - This simulator is built for Udacity’s Self-Driving Car Nanodegree to teach students how to train cars how to navigate road courses using deep learning. It is used for the project of Behavioral Cloning.
• Microsoft’s AirSim - An open-source and cross platform simulator built for drones and other vehicles. AirSim is designed as a platform for AI research to experiment with deep learning, computer vision and reinforcement learning algorithms for autonomous vehicles.
• MIT’s Moral Machine - Moral machine provides a “platform for 1) building a crowd-sourced picture of human opinion on how machines should make decisions when faced with moral dilemmas, and 2) crowd-sourcing assembly and discussion of potential scenarios of moral consequence”. If you are a fan of the trolley problem, you can’t miss this.
• MIT’s Google Self-Driving Car Simulator - Self-driving car simulated completely by visual programming language Scratch.
• Carla - CARLA has been developed from the ground up to support development, training, and validation of autonomous driving systems.
• Lgsvl - The LGSVL Simulator is a simulator that facilitates testing and development of autonomous driving software systems. The LGSVL simulator is developed by the Advanced Platform Lab at the LG Electronics America R&D Center, formerly the LG Silicon Valley Lab.

HD Map Service

Data Service

Monitor Service

OTA

Hardware

Computing Unit

Sensors

GPS/IMU

Camera

LiDAR

RADAR

Ultrasonic Sensor

CAN card

Drive by wire

V2X

HMI Device

Black Box

Big Players

If I have seen further it is by standing on ye shoulders of Giants.
                                                                - Isaac Newton
1
2

Legislation

• Arizona - Executive order 2015-09 was signed to direct all state agencies to undertake any necessary steps to support the testing and operation of self-driving vehicles on public roads within Arizona. The executive order also specifies that in Arizona the operator of a self-driving vehicle does not have to be physically inside the vehicle. The vehicle can be directed remotely in self-driving mode.
• Colorado - Colorado passes first law to regulate self-driving cars. Companies who plan to test self-driving cars in Colorado need to first check in with the state Department of Transportation and State Patrol.
• California - Application required for testing self-driving cars. Manufacturers are required to provide accident reports and disengagement reports. The regulations for post-testing deployment of self-driving cars have been drafted and will establish the requirements for manufacturers to meet prior to operation on California’s on public roads.
• New York - It is announced on May 10, 2017 that the state is accepting application for self-driving car testing. The pilot testing program is included in FY 2018 and can be further extended.
• Texas - Bill SB 2205 is proposed to implement minimum safety requirements and accelerate testing of self-driving cars on public roads. The bill passed the Texas Senate Transportation Committee and it is currently pending Senate debate and referral to the House.
• Virginia - Arguably the most friendly state to self-driving cars with no application or permit required and $25M per year fund set to facilitate self-driving cars. Virginia Automated Corridors is announced to offer self-driving car developers the opportunity to test their technologies on Virginia roads covering more than 70 miles of interstates and arterials in the Northern Virginia region.
• Singapore -
• China -
• South Korea -
• Japan -
• Europe -