devbox
Cpp五条
这个屌丝很懒，什么也没留下！
关注作者
热门标签
热门文章
当前位置:   article > 正文

TuSimple数据集格式标注自己的数据集_tusimple数据集构建

作者：Cpp五条 | 2024-03-31 16:27:46

赞

踩

tusimple数据集构建

目录

1.视频数据提取图像帧

2.labelme标注车道线

3.运行脚本,生成标注数据

由于需要使用自己的数据制作TuSimple格式的数据集,但是未找到合适的标注工具,只好自己写一个简单的脚本实现.

目前实现功能只能对直线进行自动标注,曲线部分还没有时间搞.

步骤:

  1. 将视频数据提取图像帧保存;
  2. 使用labelme工具对车道线进行标注,使用Line标注,存储json文件;
  3. 运行脚本程序,自动将标注数据存储到test_label.json文件中;

结果:

  1. {"lanes": [[-2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, 545, 551, 557, 564, 570, 576, 582, 588, 594, 600, 606, 612, 619, 625, 631, 637, 643, 649, 655, 661, 667, 674, 680, 686, 692, 698, 704, 710]], "h_sample": [160, 170, 180, 190, 200, 210, 220, 230, 240, 250, 260, 270, 280, 290, 300, 310, 320, 330, 340, 350, 360, 370, 380, 390, 400, 410, 420, 430, 440, 450, 460, 470, 480, 490, 500, 510, 520, 530, 540, 550, 560, 570, 580, 590, 600, 610, 620, 630, 640, 650, 660, 670, 680, 690, 700, 710], "raw_file": "clips/143/1641432916348132559.jpg"}
  2. {"lanes": [[-2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, 556, 560, 565, 569, 574, 578, 582, 587, 591, 596, 600, 605, 609, 614, 618, 622, 627, 631, 636, 640, 645, 649, 654, 658, 662, 667, 671, 676]], "h_sample": [160, 170, 180, 190, 200, 210, 220, 230, 240, 250, 260, 270, 280, 290, 300, 310, 320, 330, 340, 350, 360, 370, 380, 390, 400, 410, 420, 430, 440, 450, 460, 470, 480, 490, 500, 510, 520, 530, 540, 550, 560, 570, 580, 590, 600, 610, 620, 630, 640, 650, 660, 670, 680, 690, 700, 710], "raw_file": "clips/143/1641432907120901694.jpg"}
  3. {"lanes": [[-2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, 538, 543, 548, 553, 558, 563, 568, 573, 577, 582, 587, 592, 597, 602, 607, 612, 616, 621, 626, 631, 636, 641, 646, 651, 656, 660, 665]], "h_sample": [160, 170, 180, 190, 200, 210, 220, 230, 240, 250, 260, 270, 280, 290, 300, 310, 320, 330, 340, 350, 360, 370, 380, 390, 400, 410, 420, 430, 440, 450, 460, 470, 480, 490, 500, 510, 520, 530, 540, 550, 560, 570, 580, 590, 600, 610, 620, 630, 640, 650, 660, 670, 680, 690, 700, 710], "raw_file": "clips/143/1641432928394809381.jpg"}

1.视频数据提取图像帧

        此功能代码是用C++实现的,功能比较简单,可以自己写一个python脚本.

  1. /*
  2.  * @Author: hw
  3.  * @Date: 2022-01-05 16:35:58
  4.  * @LastEditTime: 2022-01-10 15:40:00
  5.  * @Description: 将视频数据的图像帧保存
  6.  * @FilePath: /lane_detect_opencv/src/lane_det/src/tools/video2frame.cpp
  7.  */
  8. #include "ros/ros.h"
  9. #include <opencv2/core/core.hpp>
  10. #include <opencv2/highgui/highgui.hpp>
  11. #include <opencv2/imgproc/imgproc.hpp>
  12. #include <iostream>
  13.  
  14. int main(int argc, char **argv)
  15. {
  16.     ros::init(argc, argv, "video2frame");
  17.     ros::NodeHandle nh_;
  18.     ROS_INFO("[video2frame] node run");
  19.  
  20.     //打开视频文件：其实就是建立一个VideoCapture结构,"path to video"
  21.     cv::VideoCapture capture("/home/hw/project/LaneDetection/datasets/video/lane_test.flv");
  22.  
  23.     //检测是否正常打开:成功打开时，isOpened返回ture
  24.     if (!capture.isOpened())
  25.         std::cout << "fail toopen!" << std::endl;
  26.  
  27.     //获取整个帧数
  28.     long totalFrameNumber = capture.get(CV_CAP_PROP_FRAME_COUNT);
  29.     std::cout << "整个视频共" << totalFrameNumber << "帧" << std::endl;
  30.  
  31.     //设置开始帧()
  32.     long frameToStart = 1;
  33.     capture.set(CV_CAP_PROP_POS_FRAMES, frameToStart);
  34.     std::cout << "从第" << frameToStart << "帧开始读" << std::endl;
  35.  
  36.     //设置结束帧
  37.     long frameToStop = totalFrameNumber;
  38.  
  39.     if (frameToStop < frameToStart)
  40.     {
  41.         std::cout << "结束帧小于开始帧，程序错误，即将退出！" << std::endl;
  42.         return -1;
  43.     }
  44.     else
  45.     {
  46.         std::cout << "结束帧为：第" << frameToStop << "帧" << std::endl;
  47.     }
  48.  
  49.     //获取帧率
  50.     double FPS = capture.get(CV_CAP_PROP_FPS);
  51. 	std::cout << "帧率为:" << FPS << std::endl;
  52.  
  53.     //定义一个用来控制读取视频循环结束的变量
  54.     bool stop = false;
  55.  
  56.     //承载每一帧的图像
  57.     cv::Mat frame;
  58.  
  59.     //显示每一帧的窗口
  60.     // namedWindow( "Extractedframe" );
  61.  
  62.     //两帧间的间隔时间:
  63.     // int delay = 1000/FPS;
  64.     double delay = 1000 / FPS;
  65.  
  66.     //利用while循环读取帧
  67.     // currentFrame是在循环体中控制读取到指定的帧后循环结束的变量
  68.     long currentFrame = frameToStart;
  69.  
  70.     while (!stop)
  71.     {
  72.         //读取下一帧
  73.         if (!capture.read(frame))
  74.         {
  75.             std::cout << "读取视频失败" << std::endl;
  76.             return -1;
  77.         }
  78.  
  79.         // cout << "正在读取第" << currentFrame << "帧" << endl;
  80.         // imshow( "Extractedframe", frame );
  81.  
  82.         //此处为跳帧操作
  83.         if (currentFrame % 20 == 0) //此处为帧数间隔，修改这里就可以了
  84.         {
  85.             std::cout << "正在写第" << currentFrame << "帧" << std::endl;
  86.             std::stringstream str;
  87.             std::string time = std::to_string( ros::Time::now().toNSec());
  88.             std::string image_folder = "/home/hw/project/LaneDetection/datasets/video2frame/lane_test/";
  89.             str << image_folder << time << ".jpg"; /*图片存储位置*/
  90.             std::cout << str.str() << std::endl;
  91.             cv::imwrite(str.str(), frame);
  92.         }
  93.  
  94.         // waitKey(intdelay=0)当delay≤ 0时会永远等待；当delay>0时会等待delay毫秒
  95.         //当时间结束前没有按键按下时，返回值为-1；否则返回按键
  96.         int c = cv::waitKey(delay);
  97.         //按下ESC或者到达指定的结束帧后退出读取视频
  98.         if ((char)c == 27 || currentFrame > frameToStop)
  99.         {
  100.             stop = true;
  101.         }
  102.         //按下按键后会停留在当前帧，等待下一次按键
  103.         if (c >= 0)
  104.         {
  105.             cv::waitKey(0);
  106.         }
  107.         currentFrame++;
  108.     }
  109.  
  110.     //关闭视频文件
  111.     capture.release();
  112.     cv::waitKey(0);
  113.     return 0;
  114. }

2.labelme标注车道线

我的文件结构如下,其中clips存放图片,clips_jsons存放标注后的json文件.

TuSimple_self
├── clip_jsons
│   ├── 143
│   ├── 144
│   └── test
└── clips
    ├── 143
    ├── 144
    └── test

3.运行脚本,生成标注数据

  • 数据集为720P图像，标注只有一条直线
  1. import json
  2. import os
  3.  
  4.  
  5. # 计算线段与行直线的交点
  6. def getLinesData(x1, y1, x2, y2):
  7.     intersection_x = []
  8.     for h in h_samples:
  9.         if h < y1 and h < y2:
  10.             intersection_x.append(-2)
  11.         else:
  12.             sign = 1
  13.             a = y2 - y1
  14.             if a < 0:
  15.                 sign = -1
  16.                 a = sign * a
  17.             b = sign * (x1 - x2)
  18.             c = sign * (y1 * x2 - x1 * y2)
  19.             x = -(b * h + c) / a
  20.             intersection_x.append(int(x))
  21.     return intersection_x
  22.  
  23.  
  24. # 得到文件夹下所有label.json的文件名列表
  25. def getLabelFilesNameList():
  26.     # 遍历文件夹内所有json文件,找出所有标注数据文件
  27.     file_list = os.listdir(clip_json_path)
  28.     json_name = 'json'
  29.     json_list = []
  30.     for file_name in file_list:
  31.         if json_name in file_name and file_name != 'test_label.json':
  32.             json_list.append(file_name)
  33.     return json_list
  34.  
  35.  
  36. # 计算车道线与h_lines的交点,并存储到json文件中
  37. def saveJson(json_list):
  38.     for file_name in json_list:
  39.         file_path = clip_json_path + file_name
  40.         image_name = file_name.strip('.json')
  41.         # 读取直线两个端点
  42.         file = open(file_path, 'r', encoding='utf-8')
  43.         data = json.load(file)
  44.         x1 = int(data['shapes'][0]['points'][0][0])
  45.         y1 = int(data['shapes'][0]['points'][0][1])
  46.         x2 = int(data['shapes'][0]['points'][1][0])
  47.         y2 = int(data['shapes'][0]['points'][1][1])
  48.         intersection_x = getLinesData(x1, y1, x2, y2)
  49.         file.close()
  50.         # 构建test_label.json数据结构,写入数据
  51.         lanes = [intersection_x]
  52.         row_file = 'clips/144/' + image_name + '.jpg'
  53.         info1 = {"lanes": lanes, "h_sample": h_samples, "raw_file": row_file}
  54.         fr = open(test_label_path, 'a')
  55.         model = json.dumps(info1)
  56.         fr.write(model)
  57.         fr.write('\r\n')
  58.         fr.close()
  59.         print(file_name + " " + "handle succeed")
  60.  
  61.  
  62. # 创建test_label.json文件
  63. def createJson():
  64.     if not os.access(test_label_path, os.F_OK):
  65.         file = open(test_label_path, 'a')
  66.         file.close()
  67.         print("create test_label.json success")
  68.     else:
  69.         print("test_label.json exist")
  70.  
  71.  
  72. # 存放结果文件夹路径
  73. folder_path = '/home/hw/project/LaneDetection/datasets/TuSimple_self/'
  74. # label文件存储路径
  75. clip_json_path = folder_path + 'clip_jsons/144/'
  76. # 结果文件名
  77. test_label = 'test_label.json'
  78. # 结果文件路径
  79. test_label_path = folder_path + test_label
  80.  
  81. # 图片行位置
  82. h_samples = [160, 170, 180, 190, 200, 210, 220, 230, 240, 250, 260, 270, 280, 290, 300, 310, 320, 330, 340, 350, 360,
  83.              370, 380, 390, 400, 410, 420, 430, 440, 450, 460, 470, 480, 490, 500, 510, 520, 530, 540, 550, 560, 570,
  84.              580, 590, 600, 610, 620, 630, 640, 650, 660, 670, 680, 690, 700, 710]
  85.  
  86. # Press the green button in the gutter to run the script.
  87. if __name__ == '__main__':
  88.     print('fuck day')
  89.     createJson()
  90.     label_json_list = getLabelFilesNameList()
  91.     saveJson(label_json_list)
  • 数据集为1080p，标注包含多条直线或曲线
    1. '''
    2. Descripttion: 曲线标注文件生成Tusimple数据集格式的json文件
    3. version: 1.0.0
    4. Author: hw
    5. Date: 2022-05-05 13:43:12
    6. LastEditors: hw
    7. LastEditTime: 2023-07-11 17:50:52
    8. '''
    9.  
    10. import json
    11. import os
    12. from turtle import color
    13. from cv2 import cv2
    14. import random
    15.  
    16.  
    17. '''
    18. name: getCrossPoint
    19. msg: 将标注线与h_samples的交点存储在lane_result
    20. param {*} x1 
    21. param {*} y1
    22. param {*} x2
    23. param {*} y2
    24. param {*} lane_result
    25. return {*}
    26. '''
    27.  
    28.  
    29. def getCrossPoint(x1, y1, x2, y2, lane_result):
    30.     for index, h in enumerate(h_samples):
    31.         if y1 == y2 and y1 == h:
    32.             lane_result[index] = y1
    33.             continue
    34.         # print("h:"+str(h)+","+"y1:"+str(y1)+","+"y2:"+str(y2))
    35.         if y1 <= h <= y2 or y2 <= h <= y1:
    36.             # print("h:"+str(h)+","+"y1:"+str(y1)+","+"y2:"+str(y2))
    37.             sign = 1
    38.             a = y2 - y1
    39.             if a < 0:
    40.                 sign = -1
    41.                 a = sign * a
    42.             b = sign * (x1 - x2)
    43.             c = sign * (y1 * x2 - x1 * y2)
    44.             x = -(b * h + c) / a
    45.             # print("x:"+str(x))
    46.             lane_result[index] = int(x)
    47.  
    48.  
    49. '''
    50. name: getLabelFilesNameList
    51. msg: 将clip_jsons文件夹下的json文件名存储在list中
    52. param {*}
    53. return {*}
    54. '''
    55. # 得到文件夹下所有label.json的文件名列表
    56.  
    57.  
    58. def getLabelFilesNameList():
    59.     # 遍历文件夹内所有json文件,找出所有标注数据文件
    60.     file_list = os.listdir(clip_json_path)
    61.     json_name = 'json'
    62.     json_list = []
    63.     for file_name in file_list:
    64.         if json_name in file_name and file_name != 'test_label.json':
    65.             json_list.append(file_name)
    66.     return json_list
    67.  
    68.  
    69. # 计算车道线与h_lines的交点,并存储到json文件中
    70. def saveJson(json_list):
    71.     for file_name in json_list:
    72.         print(file_name + " " + "handle start")
    73.         file_path = clip_json_path + file_name
    74.         image_name = file_name.strip('.json')
    75.         # 读取直线两个端点
    76.         file = open(file_path, 'r', encoding='utf-8')
    77.         data = json.load(file)
    78.         lanes = []
    79.         j = 0
    80.  
    81.         if len(data['shapes'])>4:
    82.             print("shapes length > 4")
    83.             continue
    84.         while(j < (len(data['shapes']))):
    85.             lane_result = [-2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2,
    86.                            -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2,
    87.                            -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2,
    88.                            -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2, -2,
    89.                            -2, -2, -2, -2, -2, -2, -2, -2, -2]
    90.             i = 0
    91.             while(i < len(data['shapes'][j]['points'])-1):
    92.                 x1 = int(data['shapes'][j]['points'][i][0])
    93.                 y1 = int(data['shapes'][j]['points'][i][1])
    94.                 x2 = int(data['shapes'][j]['points'][i+1][0])
    95.                 y2 = int(data['shapes'][j]['points'][i+1][1])
    96.                 i = i+1
    97.                 getCrossPoint(x1, y1, x2, y2, lane_result)
    98.             lanes.append(lane_result)
    99.             j = j+1
    100.         file.close()
    101.         img = cv2.imread(folder_path+image_path+image_name+".jpg")
    102.         point_size = 1
    103.         thickness = 4
    104.         color_num1 = [0, 100, 255, 65]
    105.         color_num2 = [255, 0, 0, 0]
    106.         color_num3 = [255, 255, 255, 255]
    107.         j = 0
    108.         while(j < len(lanes)):
    109.             i = 0
    110.             point_color = (color_num1[j], color_num2[j], color_num3[j])
    111.             # print("lanes["+str(j)+"]:"+str(lanes[j]))
    112.             for point_x in lanes[j]:
    113.                 center_coordinates = (point_x, h_samples[i])
    114.                 if point_x != -2:
    115.                     cv2.circle(img, center_coordinates,
    116.                                point_size, point_color, thickness)
    117.                 i = i+1
    118.             j = j+1
    119.         check_img_folder_path = folder_path+"check/"
    120.         save_path = check_img_folder_path + image_name+".jpg"
    121.         # 检查文件夹是否存在
    122.         if not os.path.exists(check_img_folder_path):
    123.             # 使用os模块中的mkdir()函数创建文件夹
    124.             os.mkdir(check_img_folder_path)
    125.         # cv2.imshow('test', img)
    126.         # cv2.waitKey()
    127.         cv2.imwrite(save_path, img)
    128.         # 构建test_label.json数据结构,写入数据
    129.         row_file = image_path + image_name + '.jpg'
    130.         info1 = {"lanes": lanes, "h_samples": h_samples, "raw_file": row_file}
    131.         fr = open(test_label_path, 'a')
    132.         model = json.dumps(info1)
    133.         fr.write(model)
    134.         fr.write('\r\n')
    135.         fr.close()
    136.         print(file_name + " " + "handle succeed")
    137.  
    138.  
    139. # 创建test_label.json文件
    140. def createJson():
    141.     if not os.access(test_label_path, os.F_OK):
    142.         file = open(test_label_path, 'a')
    143.         file.close()
    144.         print("create test_label.json success")
    145.     else:
    146.         print("test_label.json exist")
    147.  
    148.  
    149. # 存放结果文件夹路径
    150. folder_path = 'path to dataset'
    151. # label文件存储路径
    152. # folder_path + 'clip_jsons/train/'
    153. clip_json_path = folder_path + 'clip_jsons/'
    154. # 结果文件名
    155. test_label = 'label_data_train.json'  # 'label_data_test.json'
    156. # 图片路径
    157. image_path = 'clips/'  # 'clips/test/'
    158. # 结果文件路径
    159. test_label_path = folder_path + test_label
    160.  
    161. # 图片行位置
    162. h_samples = [160, 170, 180, 190, 200, 210, 220, 230, 240, 250, 260, 270, 280, 290, 300, 310, 320, 330, 340, 350, 360,
    163.              370, 380, 390, 400, 410, 420, 430, 440, 450, 460, 470, 480, 490, 500, 510, 520, 530, 540, 550, 560, 570,
    164.              580, 590, 600, 610, 620, 630, 640, 650, 660, 670, 680, 690, 700, 710, 720, 730, 740, 750, 760, 770, 780,
    165.              780, 790, 800, 810, 820, 830, 840, 850, 860, 870, 880, 890, 900, 910, 920, 930, 940, 950, 960, 970, 980,
    166.              990, 1000, 1010, 1020, 1030, 1040, 1050, 1060, 1070]
    167.              
    168. # Press the green button in the gutter to run the script.
    169. if __name__ == '__main__':
    170.     print('fuck day')
    171.     createJson()
    172.     label_json_list = getLabelFilesNameList()
    173.     saveJson(label_json_list)
    数据集整理完成之后需要使用ultra fast lane detection 项目下的scripts/convert_tusimple.py生成train_gt.txt文件，如果数据集为1080p图像需要改程序90行的像素参数 
    label = np.zeros((1080,1920),dtype=np.uint8)

    运行指令为：python convert_tusimple.py --root path_to_dataset

 

声明：本文内容由网友自发贡献，不代表【wpsshop博客】立场，版权归原作者所有，本站不承担相应法律责任。如您发现有侵权的内容，请联系我们。转载请注明出处：https://www.wpsshop.cn/w/Cpp五条/article/detail/345240
推荐阅读
相关标签

Copyright © 2003-2013 www.wpsshop.cn 版权所有，并保留所有权利。

  

闽ICP备14008679号