Virtual reality (VR) and augmented reality (AR) do not apply only to computer games or movie special effects. Such solutions are also often applied in the army.

Can you find yourself in the middle of a battlefield, while still sitting in your armchair? With a gun in your hand, fight your enemy lurking just around the corner? Plan and conduct an attack, and get help, if needed? Or perhaps fly your fighter, and in extremely harsh conditions train the blackest scenarios? All that while save in the knowledge that no harm is done to anyone. Yes, you can do all that – using modern training methods of simulation. Although simulation has been used for years, the ways of its use are multiplying with every technological advancement in the field of remote conduct of trainings and exercises.

Realism is Crucial

Trainings with the use of simulation programs, systems and devices are nothing new in the army. In Poland, all soldiers of all service branches use them. The array of skills and knowledge one can learn this way is vast. In the conditions reflecting the real ones, the trainees learn how to pilot an aircraft, drive a vehicle or shoot. This way they also master their soft skills, such as planning, decision-making or strategy development. Their skills and knowledge is next verified during practical trainings and exercises in the field.

Trainings with the use of various simulations are offered to officer cadets by, for example, the General Tadeusz Kościuszko Military University of Land Forces (Akademia Wojsk Lądowych, AWL) in Wrocław. “Before they go out in the field, we want our soldiers to learn how to react properly, make right decisions, and be prepared for real battlefield activity,” says Maj Aleksander Ziemiński, the AWL Tactical Simulation Institute.

Such capability is provided by, e.g., the Polish Academy Command and Staff Trainer (PACAST) for training commanders of units and staffs and larger subunits, and by the state-of-the-art Virtual Battle Space 3 (VBS3) simulator. “This is a kind of virtual battlefield where several tens of soldiers of different service branches operate at the same time. A significant database of equipment, objects and characters in any environment and conditions provides a wide spectrum of training possibilities,” adds Maj Ziemiński. The Laser Shooting Simulator (LSS) is used to train shooting, where a bullet is replaced by a laser impulse. Although during training all is done without combat ammunition, without any loss of people or equipment, the LSS ensures faithful copy of tactical activities.

According to LtCol Józef Ledzianowski, Dean of the AWL Management Faculty, this realism is one of the most crucial features of simulated trainings. “As much as the computer game designer invents the specific elements of the game, such as the range of the gun, tactical properties of some equipment or its fire vulnerability, in the systems we use all is based on real assumptions. Only in this way we can provide our soldiers with the touch of the training on a real training field,” says LtCol Ledzianowski.

Similar goals have the instructors at the AWL Leadership Laboratory. Here, the cadets in a simulated environment learn decision-making and tactical task-solving. “We set temperature between –8 and +50, we add some generator-produced wind, stroboscope light, and low- or high-frequency sound. That’s how we prepare soldiers to operate in a situation of time pressure, stress and no comfort,” says LtCol Piotr Pietrakowski, AWL.

World Tendencies

Although such solutions in the army are extremely effective, it’s time to take a step further. In many spheres of life, yet more new technologies are implemented, which offer some brand new possibilities, such as the ones using virtual reality (VR) and augmented reality (AR). Most common use of the former one, being the entirely created world, is entertainment, including computer games or movie special effects. Augmented reality links the physical world with a computer creation: in a popular smartphone game Pokemon Go, which uses phone camera and location services, a player must physically walk on the real-city streets or in the real park to catch hiding Japanese anime creatures. The game is actually played in two worlds – the real one and the virtual one.

The multiplicity of possibilities offered by VR and AR initiates the attempts to find yet another, much more serious, applications than those for entertainment. “The solutions are applied in such sectors as public administration, marketing, industry or culture. They have been widely used for some time in aviation, quite often now in medicine. They allow for faster and cheaper specialist trainings with virtually no limitations,” says Radosław Janicki of lune.xyz, the company creating apps and software for virtual reality trainings. The specialist admits that the effect of such trainings is quite measurable.

“Putting VR goggles on, we experience full immersion, which means we almost physically experience other reality, nothing distracts us, and we’re totally focused on our task. What’s more, we don’t engage in the training process many other people, because we’re able to learn on our own by simply following written instructions,” says Janicki.

The VR and AR solutions often set directions for development in modern armies. The use of these technologies allows soldiers to experience the actual harshness of battlefield, and better adjust to battlefield conditions. Małgorzata Gawlik-Kobylińska, PhD, the Faculty of Management and Command at War Studies University (Akademia Sztuki Wojennej, ASzWoj), admits that these innovative solutions are perfect for training. “It’s complementary addition to available training base. Apart from obvious advantages such as cost reduction or security of a trainee, AR and VR technologies allow for early error detection,” she says. She adds that AR can be inestimable already when performing activities: “In such a variant, augmented reality effectively supports soldiers. They receive visual and verbal instructions e.g. from a paramedic, who’s observing entire situation through his camera and adjusts optimal medical procedures for a diagnosed injury.”

Although virtual reality and augmented reality may be considered as breakthrough technologies, still the efforts to apply them in the army are not new at all. First examples of the AR use in the armed forces include the head-up displays (HUDs), initially used in, e.g., fighters, and now also in vehicles. They display crucial information on transparent surfaces, which allows a pilot/operator to observe their surroundings. The research on such devices were conducted as early as in 1950s, and the first HUD was installed in the British Blackburn Buccaneer aircraft in 1958. The equipment, constantly developed and improved, was multiplying its capabilities, and provided fighter pilots with more and more information, such as flight parameters or armament data.

Another step forward was the development of head-mounted displays (HMDs), also used mostly in aviation. They give pilots a complete set of data, regardless of their head’s position. Contemporary HUDs are quite enhanced and allow for, e.g., weapon guidance or cooperation with night-vision devices. In Poland, this kind of equipment is used on M-346 Bielik trainers. American pilots are now introducing to service the helmet-mounted display systems of 3rd generation.
The VR and AR capabilities have also been appreciated by civil enterprises. The example here being the perfected for many years Google Glass and Microsoft HoloLens products – intelligent goggles functioning as modern computers. Depending on configuration, they allow for creating virtual reality or using the augmented one. The US Army got interested in the Microsoft product, and in the fall of 2018, the company won a contract for delivery of the military versions of the HoloLens2 goggles with AR technology. The first integrated visual augmentation systems (IVASs) will go to soldiers in 2022. They will allow for a simultaneous observation of real-time image and computer data, such as a map with precisely marked location, or medical parameters of the goggles’ user.

The equipment with the most state-of-the-art AR and VR achievements is now being increasingly used in the US Army. Soldiers have at their disposal, e.g., battlefield management systems or train in shooting with laser weapons to holograms. Virtual and augmented reality may however find its application in all kind of military service branches, such as in the fully digital BattleView 360 system, on which the British BAE Systems has been working for several years. The system is to increase situational awareness of combat vehicle crews on a battlefield by means of using augmented reality according to models known from HMDs.

Breakthrough Technology

The Polish Ministry of National Defense is also interested in the VR and AR technology. The MoND had developed the MoND Priority Research Directions During 2017-2026. In cooperation with NATO member states, Allied Command Transformation, NATO Science and Technology Organization, and the European Defense Agency, top twenty breakthrough technologies were listed. The effect of their implementation is to find new technical solutions, the application of which shall change the future battlefield. According to experts, one of such technologies is virtual and augmented reality. In the MoND document, it was specified that VR and AR can be applied in the army, e.g. in simulation systems for training: troops, technical service personnel, medical personnel, and for building situational awareness on a battlefield.

The urge to follow world tendencies in army training is also seen by, e.g., military academic centers. The ASzWoj Center for Simulation and Computer War Games for years has been organizing all kinds of simulation-supported exercises for commands and military staffs, or for public administration. The training uses the updated in 2016 American JTLS-GO (Joint Theatre Level Simulation Global Operations). “Virtual training can be planned in any place in the world, and simulate equipping any army troops,” says Col Marek Sołoducha, PhD in Eng., Head of the ASzWoj Center for Simulation and Computer War Games. JTLS is not a new nor perfect system, but is NATO-certified for exercises in commands and staffs of all service branches of the armed forces. “NATO sees the need of changes and creating a universal system, which will link constructive simulation with a real and virtual ones,” emphasizes Col Sołoducha.

The VR potential can be used for training Polish chemical forces, the example of which is the concept of a virtual CBRN (chemical, biological, radiological and nuclear) island prepared as part of research and development project by Paweł Maciejewski, PhD in Eng. and Małgorzata Gawlik-Kobylińska, PhD. “A virtual island is for preparatory training, preceding the practical training in the Chemical Training Center in Drawsko-Pomorskie. The learning process is taking place in the synchronous and asynchronous mode, and is supervised by an instructor who assesses whether the trainees have reached the level required for training in the situation of real contamination,” says Gawlik-Kobylińska, and emphasizes: “This shows how the VR and AR tools give a possibility to create safe and controlled learning environment, and painless learning on your own mistakes. All training is based on scenarios adjusted to current needs, and is limited barely by imagination of a training system’s designer,” admits the ASzWoj academic.

Two Worlds

Experts notice that a sort of arms race in designing even more perfect VR and AR training tools and systems is taking place all around the world. The government agencies announce competitions, select winners, and then finance educational projects with most perspectives. In Poland the situation is similar. The example here is the MoND research grant for ASzWoj to develop modern environment for research on the use of virtual reality in army training.

As part of the project, the installation of mini-CAVE (CAVE Automatic Virtual Environment) was built, one of the most advanced devices of this type in Europe. It is for creating and implementing specific training scenarios, the exercising of which in real conditions can be dangerous to human life and health, such as deactivating explosives, operating in contaminated environment or making decision on a battlefield. ASzWoj works on this project in cooperation with Jacek Lebiedź, PhD in Eng., Gdańsk University of Technology (Politechnika Gdańska, PG).

Also academics at AWL thoroughly track the new training perspectives. “We want such solutions which would provide us with simulated activities, but in real-life conditions, and with the use of existing terrain, e.g., a tactical exercise belt in Wędrzyn or a training field in Nowa Dęba,” says the academic at the AWL Tactical Simulation Institute. The investments in modern training equipment for soldiers are also planned by for example the AWL Laboratory for Managing Drones in Tactical Operations. It is about drone simulators, which allows for virtual use and testing of UAVs. The university also works on creating the implementation of a treadmill for VBS3 application for full immersion in time of advanced tactical training. The plans also include the purchase of a VR and AR digital mock-up table. “While standing on such a table, a commander with special goggles on will be able to observe in 3D entire operational environment, and test various activity options as they happen. It can also be used for training in building equipment and vehicles,” says Maj Zieliński.

LtCol Józef Ledzianowski pays particular attention to one more possible use of virtual and augmented space – in developing tactical objectives for delivered army equipment: “Obviously, we can order 500 tanks, but what if it turns out that in practice they do not meet our expectations? Perhaps, before we buy and implement them, we shall virtually program them, assume specific parameters and test them in 3D reality on a virtual battlefield?.”

Virtual Tests

A similar method, applying modern imaging technologies for designing new devices, has been tested by the US Army. During the work on the long-range hypersonic weapon (LRHW), as many as three kinds of realities were applied: VR, AR, and MR (mixed reality, where we can interact with created things). In February 2020, a virtual prototype of such weapon was introduced to soldiers, their future users. The use of this technology was to allow for more accurate details of the project, including optimizing individual elements (e.g. accessibility to trouble spots), and presenting visualization to the future users was to eliminate at the early stage any possible errors. Eventually, it all was to translate into the shorter time of design and production phase, and decrease the cost of implementing the equipment into service.

The world tendencies are also followed by the Central Military Bureau of Design and Technology (Wojskowe Centralne Biuro Konstrukcyjno-Technologiczne, WCBKT). The company is the only Polish busines producing devices for aircraft ground service used both at the military and civil airports, including: electrical power supply units, hydraulic devices, gas distributors and others. “These are technologically complex devices. In order to lower high potential cost of fixing breakdowns, we decided to implement technologies supporting the work of both, personnel using the device, as well as service personnel. One of such solutions is VR holographic near-eye displays (holographic lens), which link the real image with a virtual one,” says Piotr Kisiel, WCBKT President of the Board. When a user puts them on and stand on a real device, a holograph will be displayed with all kinds of instructions and schemes of a given equipment. When there is a problem with fixing a bug or deleting an error, a technical support person can find assistance via goggles and a communicator by contacting the manufacturer’s expert. “This helps us to minimize the risk of errors,” emphasizes Piotr Kisiel. Holographic near-eye displays since 2020 have been used as part of pilotage project in the Polish Armed Forces, but some time in the future they will be introduced as an integral equipment to each serially-produced equipment.

The VR and AR training is today a necessity. There’s no way back. What’s important, though, is to find a balance between what is virtual and what is real. Because a soldier, even if perfectly trained in simulated conditions, must ultimately smell the real gunpowder and successfully manage on a real battlefield.