Railway simulator for obsacle detection project

In order to do so, a set of HW sensors, a sophisticated architecture for data processing, and artificial intelligence tools for final identification of obstacles and their interpretation will be used. As a key support for development of the detection system, the project includes the development of a software simulator for virtualization of railway conditions as well as implementation of test rides in a laboratory environment.

In such an environment, it will not only be possible to create crisis situations, which can analogously arise on a real railway track, but also to simulate various weather phenomena such as rain, snow, and fog, which can influence the generated output data. The simulator solution will include specification of particular critical scenarios that may occur, and selection of important virtual environment parameters that can be influenced, in particular, to define the complexity of the simulator while ensuring the required variability and credibility of data necessary to train the detection algorithm.

The project, in which IT4Innovations is responsible for creating a 3D virtual simulator environment and generating enough quality data to train a detection algorithm, is currently in its first phase. IXPERTA have designed the construction for the sensors, fitted the test railway vehicle with these sensors, and started data collection. A visual camera, a thermal camera, a laser sensor, and a train positioning sensor were used with the aim of enabling detection of objects at all times on the track. Analogous sensors will also be used when creating a software simulator at IT4Innovations.

Unlike in the case of automotive transport, there is currently no available, generic software on the market that can do virtual replication of inputs for a learning detection system on a real railway track, including climatic and other limiting conditions. “In order to create a reliable virtual railway environment, it was necessary to take into account large amounts of data such as terrain elevation, vegetation grids, real train speed profiles, and different types of static or dynamic objects. Due to the computationaldemands of the data generated, the IT4Innovations infrastructure is used as part of the project,” says Petr Strakoš, Principal Investigator of the project at IT4Innovations.

In the first phase of the project, a section of the railway track between Třebenice town and Dlažkovice stations was modeled. A 3D scene containing the underlying terrain, vegetation objects, and the track curve used to model the railway tracks (embankment, track, and sleeper objects) has been created. Based on the date and time specified, the particular phase of the day was calculated and the position of the sun and the appearance of the sky set. Static and dynamic objects, which are then placed in the scene, have been created for the track. A prototype train object with a camera and other features necessary for simulating sensors has been incorporated into the created environment. Based on the given events, the movement/animation of the train and other dynamic objects has been calculated. Various climatic conditions and weather phenomena can be simulated in the generated environment. In the first phase of the project, phenomena such as clouds and snow were in the experimental stage.

The current outputs that the simulator is able to generate are images from an RGB camera, a depth map, LIDAR, and a segmentation map with a ground truth classification. The depth map is obtained automatically using the ray tracing method, and can be used in the future as a resource to calculate LIDAR data so far having been obtained in a different way. Ground truth recording is an important output that is necessary to train the obstacle detection algorithm, which will take place in the next phase of the project. At the same time, train location information in GPS coordinates and train orientation information (azimuth, elevation) is also generated in a textual form.

The second phase of the project will continue with the simulator implementation. Emphasis will be placed on achieving a higher degree of lifelike synthetic data visualization by modifying augmentation filters based on neural networks. A railway track between Žďár nad Sázavou and Nedvědice stations will also be modeled with the aim of achieving maximum correspondence between the model and the real section of the railway track. Verification of selected concepts used in the simulator with real behaviour will also play an important role in this phase of the project. This may include, for example, the fidelity of the virtual train movement and a comparison of the similarity of the simulated selected sections with real sections of the railway track, etc.

The Research and Development of a Functional Sample of a Railway Vehicle Enabling Collection of Data and Software – a Simulator Enabling Generation of Data to Train Obstacle Detection under Simulated Conditions project, ID FW01010274, is co-funded with the state aid of the Technology Agency of the Czech Republic within the TREND Program.