The highest grade of automation, GoA4, has become familiar for metro and monorail and will soon be adopted in other segments of railway transport. Railway market players accumulate data to train vision systems, prepare relevant infrastructure, and draft regulations. If there are no obstacles to the wide use of GoA4 in shunting locomotives, the commercial application of the technology in urban rail transport is expected no earlier than the mid-2020s. Only after that will come the turn of regional trains, freight traffic, and possibly even high-speed railways.
From Rolling Stock Market Trends almanac issued for the PRO//Motion.Expo 2023 International Railway Fair
Projects that stand out
The heavy haul freight transportation system of Rio Tinto company in Western Australia is so far the only GoA4 project on non-isolated railway infrastructure in the world. Since 2019, the $940 mln AutoHaul project — its system integrator is Hitachi Rail — has been allowing unattended operation of more than 200 diesel locomotives delivering iron ore from 16 mines to the ports of Dampier and Cape Lambert. Up to 50 heavy trains of 240 freight cars run simultaneously on a network of more than 1,800 km with the Operations Centre 1,500 km away from the mines. Drivers operate trains only in terminals, but loading and unloading are automated.
Unattended train at Rio Tinto’s mine. Source: Hitachi Rail
The implementation increased the average speed of locomotives by 5–6% and reduced fuel consumption for traction, as Rio Tinto estimates. Elimination of the need to transport the personnel by car eliminated the need to cover costs for travelling 1.5 mln km a year. The company wants to improve the system and is selecting a computer vision supplier. 4AI Systems, Australia, and Rail Vision, Israel, have already been invited to the tests.
However, it is not possible to transfer the AutoHaul project scheme to public tracks because Rio Tinto’s network is not just in an isolated desert area, it runs identical trains of the same owner.
The automation of the Moscow Central Circle (MCC) in Russia has been under development since 2017 by NIIAS, a leading Russian Railways institute engaged in intelligent control and train safety systems. Over the circle line of 54 km, Lastochka EMUs carry half a million people a day, and passenger traffic will continue to grow. The operator needs to reduce intervals between trains to three minutes or less. With such a tight schedule, a deviation of only 15 seconds can cause transport chaos, so the choice of AI is obvious, as Alexander Dolgiy, CEO of NIIAS, explained in an interview with RZD TV.
The testing of the GoA3 operation of Lastochka began in 2020. A year later, the train received a certified vision system; Russian Railways claim it is the first certified system in Europe. Trains sensors used in these trains are more sensible than the human eye. They detect an adult pedestrian at a distance of 1 km, and a child at 600–700 m, and react to an obstacle in 0.3 sec. Now the system is trained to work with more complex types of signals and is prepared for the GoA4 certification. At this grade of automation, no driver’s participation is required to operate a train, diagnose the condition of its equipment, or open or close doors.
Grades of automation in rail transport (enlarge). Based on the infographics by ZDmira journal
In March 2023, an experiment was conducted on the MCC: one operator remotely controlled two Lastochka trains at once. Later, one operator is expected to control four trains simultaneously, and energy consumption to reduce due to optimisation of operation modes. The deputy CEO of NIIAS Pavel Popov said earlier the GoA4 technology would be ready by the end of 2023. It will be used at ES104 trains by Ural Locomotives to replace Lastochka trains previously localised in partnership with Siemens Mobility. A batch of 22 five-car EMUs is scheduled for delivery to Russian Railways since November immediately after certification.
Tests of the unmanned Lastochka EMUs operation on MCC. Source: Russian Railways
The developers have already adjusted the interaction of the modernised on-board systems with MCC’s infrastructure. It is worthy of note that MCC will have no platform doors traditionally used to isolate unmanned trains for safety on metro lines.
The automation of the Copenhagen S-bane, Denmark, makes a case for the most ambitious automation project on rial. According to the plan, the 170 km network of 86 stations will become completely unmanned in 2037. Today, 300,000 passengers a day use its lines to travel.
In achieving this goal, the national operator DSB initiated the world’s largest procurement of GoA4 rolling stock to replace Alstom’s GoA2 vehicles running the lines now. For €3.5 bln a contractor will deliver 226 trains with a minimum operating speed of 120 km/h and ensure 30-year maintenance. Selection of the contractor is scheduled for the spring of 2025, the beginning of the train testing for 2028, and the commissioning will take another ten years or so. In the autumn of 2022, Siemens installed CBTC, a communications-based train control system, on the network, and will later supply signalling equipment for the new fleet.
A train of the Copenhagen S-bane. Source: Thomas de Laine/myldretid.dk
As the strategy and rolling stock director at DSB Jürgen Müller told pro.ing.dk, initially it was proposed to use the distributed acoustic sensing (DAS) technology with sensors at the front of the train to ensure safety on the tracks. Now, the company is leaning towards doing away with platform doors, using track fencing between stations, and installing object detection systems at stations.
The automation of the Luzhskaya marshalling yard in the Leningrad Region, Russia, which is the largest port cargo hub in the country, began in 2015. Two years later, three TEM7A shunters were converted to GoA2. Russian Railways engaged its research centres — NIIAS, VNIKTI, and the Locomotive Engineering Design Bureau — to create software and onboard systems, and adopted the MSR32 system by Siemens for the marshalling management (now, when the German company left Russia, it is maintained by NIIAS).
TEM7A diesel shunter at the Luzhskaya station, Russia. Source: SsVMedia/youtube
Diesel locomotives shunt independently, and a driver intervenes only in case of incidents. An obstacle detection unit equipped with sensors identifies objects at a distance of up to 200 m and decides whether to move on or brake.
In South Korea, a similar project was announced in April 2023. The national operator Korail automated the coupling, uncoupling, and movement of freight trains at the Jecheon station. In Europe, shunting locomotives with technical vision have been actively tested only in the last two years.
Unmanned solutions at the Luzhskaya station are an important part of Russian Railways’ Digital Railway Station concept which is designed to delegate operations to AI as much as possible and replicate this project at 25 marshalling yards.
What is going on around the world
2022 saw GoA4 testing of at least two shunters in Europe. In Finland, the Otso 4 diesel locomotive manufactured in 1975 was equipped with a computer vision system and converted to hybrid traction. The state technology company VTT provided the system, private businesses participated in the modernisation, and the total cost of the project amounted to €3.6 mln. Regular operation of autonomous vehicles on public railway tracks was reported to start in 2023.
An automated shunter on test in the Voikkaa industrial area, Finland. Source: railwaygazette.com
In the Netherlands, a computer vision system by Elta Systems, a company of IAI defence concern, Israel, was tested on a shunting locomotive. The tests confirmed its compatibility with Alstom’s automatic train operation system. The Elta equipment uses high-resolution digital radar and multispectral electro-optical devices, so it is efficient even in bad weather and low visibility. At the same time, the test drive showed its sensors detect obstacles on the track only from 500 m, not from 1 km as claimed.
The Elta technical vision system at the shunter of Lineas tested in the Netherlands. Source: Alstom
Another obstacle detection system, Main Line, is proposed by Rail Vision, in which Knorr-Bremse holds a strategic stake. According to the company, its sensors mounted on the cab roof can see vehicles at a distance of up to 2 km, people up to 1.5 km, and the end of tracks up to 600 m in any weather and lighting conditions. In early 2023, Rail Vision signed its first $1.4 mln serial order contract for the supply and maintenance of 10 such systems with the national operator Israel Railways. The system positioned to help a driver or remote operator on main lines can be used at GoA4 vehicles. For three months in 2022, Main Line was tested on Rio Tinto’s unmanned ore trains in Australia, and then it was announced that the tests would continue.
Testing of Main Line obstacle detection system at the rolling stock of Israel Railways. Source: railwaygazette.com
In the USA, one more system improving the safety of shunting operations is under test run, Switch Yard (200 m range) by Rail Vision.
In June 2023, the project Sensors4Rail came to an end in Germany. Implemented by the national operator Deutsche Bahn since 2019, the project tested an obstacle detection system on four GoA2 trains in Hamburg. Automobile and industrial sensors by Bosch and Ibeo were adapted to the railway in the laboratory. The system comprised six Lidar sensors making more than 30,000 measurements at 15 Hz; an 8–14 µm thermographic camera capable of detecting stationary and moving objects even in absolute darkness; three visible-band cameras for close, medium (300 m) and far (550 m) range; four 76–77 GHz radars, and an onboard computing complex which processes signals. Train positioning and system integration were the responsibility of Siemens Mobility.
The train with the computer vision system on the S-Bahn in Hamburg, Germany. Sensors4Rail project. Source: here.com
The one-year testing resulted in recording 450 Tbytes of data which allowed us to assess how weather and time of day affect object recognition, and how accurately the train location is identified especially in areas where there is no satellite communication. A data centre was created to process all the collected information and prepare on its basis a catalogue of scenarios for serial systems. The AI training involves both real data and simulation modelling.
The next step towards automated train operation in Germany was taken in July 2023. The future Mireo Smart train by Siemens and one of the trains for Stuttgart were announced to have a GoA4 operation system. As it was with GoA2, the system will be installed on top of the European Train Control System ETCS. The €42.6 mln project AutomatedTrain is implemented at the account of the state budget. It will take three years to test how trains automatically enter and leave lines, and Deutsche Bahn expects to present the results at Innotrans–2026. In addition, the country plans to develop standards for testing and AI use in regional trains by the end of 2024.
Siemens continues to develop its collision warning system, Mainline Assistant, capable of scanning the space in front of the train for 600 m. It is claimed to support high grades of automation and adapt to different types of trains. The system is tested on a high-speed train, ICE TD, operated on the Deutsche Bahn’s network.
Advanced TrainLab at the Innotrans 2022 Fair in Berlin. Source: bahnblogstelle.com
Two years ago, France represented by a consortium led by the national operator SNCF announced the launch of prototypes of fully autonomous trains, a freight and a passenger one, by 2023. The testing at the GoA2 level involves Alstom and Thales and has been underway since 2021. Next year, Thales and its partners in Canada, including Lumibird Canada, a developer of Opal 3D Lidar, prototyped a $2 mln unmanned control system for urban and suburban trains.
In autumn 2022 in Germany, a rolling laboratory, Lucy, by Thales was tested at the Smart Rail Connectivity Campus to assess remote train control from a distance of 300 km. The company made a cooperation agreement with Knorr-Bremse producing braking systems to work on automated train operation for the Digital Freight Train concept.
The rolling laboratory Lucy by Thales in the Ore Mountains, Germany, operated from a distance of 300 km. Source: Deutschlandspricht über 5G
Japan is preparing for a more ambitious task, the automation of high-speed railways and E7/W7 series trains manufactured by a conglomerate of Japanese manufacturers. The operator JR East plans to adopt GoA4 on the section from the depot to the central station of Niigata in the late 2020s. Transition to GoA3 passenger service on the Joetsu line between Niigata and Tokyo is expected from the mid-2030s.
What Russia does in the computer vision sector
In Russia, the development of unmanned railway technologies is led by NIIAS and TMH Smart Systems. Until recently, they were competing with Cognitive Pilot, a joint venture of Sber and Cognitive Technologies, but the company has switched to other market segments.
NIIAS is ready to scale up a certified on-board vision system that assists drivers of locomotives, multiple units, and special vehicles. The system detects obstacles, identifies traffic lights, the condition of tracks, and the position of switches. It complies with the technical regulation TR EAEU 001/2011, so it can be implemented in Russian Railways’ network and supplied to the EAEU countries. The system’s sensor set includes two close-range video cameras (up to 50 m) and two far-range cameras (up to 200 m). The video control units are manufactured in Russia, and the software is developed by NIIAS.
Onboard computer vision system by NIIAS. Source: NIIAS
TMH Smart Systems offers a monitoring and control system, Ctrl@Vision, with a claimed range of up to 600 m which can be installed on any rail vehicle. In 2020, the TGM6A shunting locomotive of the Cherepovets Steel Mill was the first to receive the system. Its variations are tested at industrial sites in Russia and Kazakhstan. For example, if the company needs to move trains with cars ahead of the locomotive, the standard module is supplemented with a video monitoring unit on the tail car which transmits the image to the cabin.
The concept of Ctrl@Vision has been extended in Ctrl@Traffic 200, an automatic train operation system. This system is an option for the new shunting locomotive TEM23 by TMH. It has three operating modes: unmanned operation, obstacle detection, and remote control.
TEM23-0001 shunting locomotive during controlled demonstration operation at Cherepovets Steel Mill. Source: Vyacheslav Zlobin/Railgallery
Being at the forefront of unmanned technologies in mainline transport, NIIAS tests them on the Moscow Central Circle. According to its estimates, it is about a year ahead of global trends in terms of interaction between on-board systems and track infrastructure, while its computer vision system is as efficient as European models. However, NIIAS recognises there is a need to create a data fabric for training neural networks similar to the one created for Sensors4Rail.
