Carla Simulator

Let's try to use a car simulator to not only do perception but also control.

In the rest of the tutorial, we will accept that you have a carla simulator running at localhost:2000 the default carla configuration.

Check out Carla Simulator installation page

The quickest way is to use docker with:

docker pull carlasim/carla:0.9.13
docker run --privileged --gpus all --net=host -e DISPLAY=$DISPLAY carlasim/carla:0.9.13 /bin/bash ./CarlaUE4.sh -carla-server -world-port=2000 -RenderOffScreen
# More info at https://carla.readthedocs.io/en/latest/build_docker/

Once the simulator is up and running, We can add the code within the getting started to detect obstacles on our way. To do that we only need to replace the webcam node that is sending the image. The oasis agent is connected to the carla server and captures images from the front camera of the ego vehicle. We can use this agent instaad of the webcam to send image data to the downstream operators in the dataflow graph.

nodes:
  - id: oasis_agent
    custom:
      inputs:
        tick: dora/timer/millis/400
      outputs:
        - position
        - speed
        - image
        - objective_waypoints
        - lidar_pc
        - opendrive
      source: shell
      # With Carla_source_node
      args: python3 ../../carla/carla_source_node.py
      #
      # Or with the OASIS AGENT
      #
      # args: >
        # python3 $SIMULATE --output 
        # --oasJson --criteriaConfig $CRITERIA
        # --openscenario $XOSC
        # --agent $TEAM_AGENT
        # --agentConfig $TEAM_AGENT_CONF
        # --destination $DESTINATION
  
  - id: yolov5
    operator: 
      outputs:
        - bbox
      inputs:
        image: oasis_agent/image
      python: ../../operators/yolov5_op.py

  - id: plot
    operator:
      python: ../../operators/plot.py
      inputs:
        image: oasis_agent/image
        obstacles_bbox: yolov5/bbox
        position: oasis_agent/position

To run:

dora up
dora start graphs/oasis/oasis_agent_yolov5.yaml --attach

You should see a window showing the view from a camera within the carla simulator. On this camera stream, you should see object detection happenning.

You can ctrl+c to kill your running dataflow.

This very first step is done to show you how to connect carla simulator with dora. It does not provide any control to move the car.