Self Driving Car With Deep Learning

Part 1 : Self Driving Car With Deep Learning

Git: https://github.com/dhiraa/medium/tree/master/self-driving-car

Huh we are not Tesla, Google or any other big billion company out there to have our own AI car to try out our curiosity in Reinforcement Learning. Does that stops us from trying? Nope!

We are gonna use https://kivy.org/ to create our own environment and our own car with some minimalistic sensors.

We are gonna keep the well known OpenGym (https://gym.openai.com/) out for time being!

This post helps you to get used to the Kivy UI components step by step through simple examples. Most of the explanation needed for the understanding are developed from the sites ping-pong example. It is highly recommended for the readers to go through the links referred while following the examples(steps).

Kivy

We need an environment to emulate our car and sensors with our own rules. Kivy is a Python UI development framework, which can be used for this purpose, as it allows user interaction programming very easy, like mouse / touch interactions.

Environment Basics:

Following are few points that I wanted to highlight after following Ping-Pong tutorial:

• *.kv file is used to define the graphic objects (eg: car) look and feel
• Name of the graphics objects and its corresponding python reference should be in sync
• The python reference is used to update the graphic object (eg: movement of the car)
• https://stackoverflow.com/questions/23971752/understanding-glpushmatrix-and-glpopmatrix

Example @ https://github.com/dhiraa/medium/tree/master/self-driving-car/ping-pong

Original:

• Run below command, to start an emulated environment where you can see a toy car
• Use your mouse, left click and drag, to drop sands
• With mouse now you can create your own road/sand ways, where the car will learn to drive on its own.
• Go a head and create different paths and see how it adopts to the environment
• If you wanna develope this from scratch, then follow Steps 1 to 5, which introduces you the Kivy UI framework from Step 1 to Step 4
• In Step-5 we use Torch/Tensorflow to implement Deep Learning based Q Learning, a special RL algorirthm that learns the environment and helps the agent(car) to navigate avoiding the sand

python map.py

Car Environment:

Step-1:

• Lets start putting a small car in our virtual environment

• So basically we have two pieces, working together:
• .kv file, that defines how the shapes should be
• A python file that control those shapes
• In our first step, we have self_drive_env.py and selfdrive.kv
• Ok, our car is a simple rectangle with three ellipse as its sensors. I know it doesn’t look a like car, you have to bare with it as I didn’t get a nice example/tutorial, so that I can do Ctrl+C & Ctrl+V here ;)

Step-2:

• From step-1, we have added support to move up/down/right/left/rotate using arrow keys and Q,W keys
• Check self_drive_env.py for more details on the code
• As you can see the rotation of the sensor is not in sync with the Car/rectangle, lets fix this in next step.
• python self_drive_env.py #Keys: Arrows, Q,W

Step-3:

• From step-3, we have updated support to move up/down/right/left/rotate using arros keys and QWS keys
• Check self_drive_env.py for more details on the code
• As you can see the rotation of the sensor is not in with the Car/rectangle.
• python self_drive_env.py #Keys: Arrows, Q,W,S(animation)
• https://kivy.org/doc/stable/examples/gen__canvas__rotation__py.html
• By default all the sensors are stacked in once, unlike precious step, upon on first update the sensors takes its proper position

Step-4:

• From step-3, we have added support for Mouse interaction
• By clicking and dragging the mouse pointer we can drop sand on the environment.
• So what does that means? We have sensors which can detect sand and we have rules for the car to slow down when on sand.

python self_drive_env.py

Step-5:

• From step-4, we have added support for sand dropping
• By clicking and dragging the mouse pointer we can drop sand on the environment.
• So what does that means? We have sensors which can detect sand and we have rules for the car to slow down when on sand.
• With mouse now you can create your own road/sand ways, where the car will learn to drive on its own.
• Go a head and create different paths and see how it adopts to the environment

python self_drive_env.py

Stay tuned for more details on the maths behind the Q-Learning and its implementation!