Autonomous driving with miniature cars
The Autonomous driving with toy car project aims to train a robot car to autonomously drive across a model environment. Once trained the car will be able to collect surrounding information (mainly video) and then use that information to navigate through the course while following some baseline traff
2025-06-28 16:30:31 - Adil Khan
Autonomous driving with miniature cars
Project Area of Specialization Artificial IntelligenceProject SummaryThe Autonomous driving with toy car project aims to train a robot car to autonomously drive across a model environment. Once trained the car will be able to collect surrounding information (mainly video) and then use that information to navigate through the course while following some baseline traffic rules avoiding any collisions with other cars and pedestrians. For this purpose we will be using concepts of machine learning and convolutional neural networks and implement them on a miniature car chassis on which a computational board is mounted.
Project ObjectivesThe scope of our project is to enable autonomous driving of toy cars in a controlled environment. Achieving this is an immense challenge as the toy cars have to avoid all the obstacles which it encounters by stopping and waiting until the road is clear. These obstacles could be pedestrians and other bots which need to be avoided on the basis of different computational methods.
All of the computations necessary for autonomous driving will be preferably performed onboard the individual raspberry pi module of each robot car. All the information will be gathered for processing using a fisheye lens camera module mounted on top of the car.
Data Collection, Training, and Deployment:
A python script was written for collecting the training data. A camera mounted on the car is used to record video in H264 format with a resolution of. The python script initiates to record the video on a key press. To control the car with a keyboard a system was needed to log the commands being sent to drive the car. For this purpose pressing the key and sending individual key
was too slow of a process. Therefore an inbuilt library for python was used which could detect key presses. These key presses which are set according to a predetermined command scheme are detected and then the appropriate action is performed based on the detected key. The user is able to initiate the system by pressing “o”. Initiating the system causes the video to start recording and the serial communication line being opened between the pi board and the arduino. By pressing “s” the car starts with both the wheels having same speed. Once the car has started the motion is controlled by changing the speeds of the wheels. Left and right motor speeds are controlled by pressing “l” and “j” keys. To stop the user must press “k”. To stop recording the video and end the recording session. During the drive the arduino continuously generates a log of the command being sent to it and the current PWM signals being sent to the motors. These are also being serially transmitted to the pi board where it is saved in a text file for labeling. As the video is recording at a rate of 25 frames per second, the arduino is set to generate these logs 25 times a second. Therefore we are able to determine what command is being sent respective to each frame. When the user is not pressing any key “n” is being noted down which represents no input.
The video that is collected during driving along with the respective log file is then moved to another workstation where the frames are extracted from the video and are then labeled according to the log file. As the frames are extracted the frame is labeled (named) using the command that was noted in the log file. Then the frames which belong to the same label i.e. command are grouped together in the same folders. In MATLAB we use an image data store object where we can store and access multiple images and store information about them. This image data store will then contain the files (frames) with appropriate labels that will be used for training the network. The MATLAB deep learning toolbox was used for the purpose of
training the system from which we have set up a convolutional neural network architecture with the following configurations.
After training the network is ported to the raspberry pi board. We can extract the network parameters from the trained MATLAB network and then inference will be generated in real time on the pi board. The camera will collect the frame which will be then processed on board and a command will be generated and sent to the arduino for motor control.
The scope of our project is to enable autonomous driving of toy cars in a controlled environment. Achieving this is an immense challenge as the toy cars have to avoid all the obstacles which it encounters by stopping and waiting until the road is clear. These obstacles could be pedestrians and other bots which need to be avoided on the basis of different computational methods. All of the computations necessary for autonomous driving will be preferably performed onboard the individual raspberry pi module of each robot car. All the information will be gathered for processing using a fisheye lens camera module mounted on top of the car. Our platform most closely resembles Duckietown. Duckietown is an open and flexible platform for autonomy education. The Duckietown project was conceived in 2016 as a graduate class at MIT. Our project has the potential of being a source of learning for student as it has the ability to model real time scenarios in an inexpensive, controlled and safe environment. In future we could even expand our product in various ways for example we can add navigation system and localization within a map.
Technical Details of Final DeliverableThe conclusion of this project can be summarized as that the approach of using the convolutional neural network as mentioned above can work but it would be much more efficient to introduce some other means of data collection and training that would lead to better results. The proof of concept is present as the network that was trained was able to generate
commands when fed with a frame on the track. But the desired accuracy required for an autonomous drive was not achievable. Using image processing techniques for determining the steering angle can be a better approach to solve the same problem. Interfacing all the various hardware components to produce accurate and consistent results does become a challenge to overcome. The requirement of being able to respond in real time according to the track was achieved but better accuracy would be needed for a fully autonomous drive. Further multiple other techniques can be incorporated as well to eliminate problems such as different illumination of the scenes while collecting training data.
| Item Name | Type | No. of Units | Per Unit Cost (in Rs) | Total (in Rs) |
|---|---|---|---|---|
| Total in (Rs) | 55400 | |||
| 4 fish Eye lens camera | Equipment | 4 | 3800 | 15200 |
| car chassis | Equipment | 4 | 1550 | 6200 |
| Raspberry pi 3 | Equipment | 4 | 6000 | 24000 |
| Model Environment + DC motors + Motor Driver shields | Miscellaneous | 1 | 10000 | 10000 |