FY19 STUDY TORS 21st Century Training and Education Technologies (TET) Background The Air Force uses some virtual technologies to support increased proficiency and readiness in areas, such as large force employment training, pre-deployment certification, and mission rehearsal. Recent commercial-world advances in augmented reality (AR) and virtual reality (VR) technologies are creating potentially new forms of mixed reality that extend beyond the traditional live-virtual- constructive paradigm and could provide opportunities for improved Air Force education and training to a higher proficiency level and at lower cost. Virtual and real objects can now be made nearly indistinguishable, contextually relevant information can be presented digitally overlaid on the real environment, and other uses of AR/VR can be envisioned that could substantially enhance training. However, leveraging AR/VR technology for improved Air Force training would need to overcome key challenges not present in most commercial-world applications. These include multi-level security and compatibility with legacy training systems. The Air Force will benefit from a clearer understanding of specific types of Air Force education and training that could utilize modern AR/VR technologies, and an understanding of how associated technical, managerial, and governance issues can be resolved to enable effective augmented technologies for education and individual, team and large force training. Charter The study will: Survey present uses of virtual and constructive elements in Air Force training, their effectiveness, and identify needs that could potentially be met by augmented reality training. Determine current state-of-the-art commercial capabilities for augmented reality that are potentially relevant to Air Force education and training, as well as emerging near, mid, and far- term AR/VR technologies. Identify opportunities for Air Education and Training Command (AETC) and other MAJCOMs to leverage AR/VR and other technologies to increase the speed and depth of learning when it comes to complex educational and training requirements. Examine how the other Services are using AR/VR to improve their education and training needs. Assess the benefits for Air Force initial, recurring and specific task training enabled by augmented reality that cannot be obtained from current training approaches. Weigh time, cost and quality of training versus the investment needed for additional augmented reality. Determine shortfalls in current and projected AR/VR capabilities relevant to Air Force education and training, and identify those that are likely to be filled by commercial-world solutions and those that are unique to Air Force training needs and likely to require unique development or modification. Identify technologies and processes that could help keep AR/VR training systems up-to-date and concurrent with Air Force systems over their life cycle to maintain configuration control. Provide a roadmap for near, mid, and far-term Science and Technology efforts to enable the Air Force to leverage advanced training and education capabilities. Identify techniques (e.g., biometric data) to measure the impact of potential AR/VR training capabilities. Study Products Briefing to SAF/OS and AF/CC in July 2019. Publish report in December 2019. Fidelity of MS&A to Support Air Force Decision Making (MSA) Background The Air Force corporate process to make acquisition and investment decisions uses the AF Warfighting Integration Capability (AFWIC) to evaluate options. AFWIC in turn makes extensive use of modeling, simulation, and analysis (MS&A) to support its understanding of potential characteristics, effectiveness and value of candidate systems. Models are used in the Developmental Planning Process as well as Analyses of Alternatives and then instantiated in a Model-Based Systems Engineering (MBSE) approach to system life-cycle management. The complexity of the environments that weapon systems will operate in and the complexity of the weapon system capabilities that must be traded off cannot be adequately evaluated or understood without the support of these analytic methods. This allows an understanding of the effect of each major parameter on total system performance and informs trade-offs in reaching a realistically affordable weapon system with needed performance. However, the underlying analytic capabilities typically involve a system-of-systems framework, in which an interdependent assemblage of models and tools with varying levels of fidelity might be used to simulate mission performance. It can be difficult to quantify or understand the fidelity of results from this aggregated MS&A due to the obscured traceability to individual, underlying physics-based principles. The Air Force will benefit from an improved understanding of how the fidelity of MS&A can be characterized and improved and how MS&A credibility might be increased to support Air Force investment decision making. Charter The study will: Review and characterize the current state of the art for MS&A of complex systems and systems of systems, excluding cost models. Include capabilities within DoD, FFRDCs/UARCs, national laboratories and gaming industries. Identify best-of-breed models and integration frameworks relevant to Air Force missions. Characterize environments that are used to link physics-based simulations to engagement-level, mission-level, and campaign-level simulations, including the propagation of errors and uncertainties between levels, and identify key elements that limit overall resulting fidelity. Consider security implications of these linkages, including the need for MS&A that spans multiple security levels. Identify improvements to the VV&A processes used to certify MS&A fidelity and maintain configuration management of complex models and the supporting modeling environment. Assess if modern approaches to uncertainty quantification (UQ) in physics-level simulations can be extended to provide improved quantification of system-level MS&A fidelity and recommend an implementation plan. Recommend appropriate Design of Experiment and uncertainty quantification techniques that can increase decision-maker confidence in MS&A to support investment decisions. Recommend S&T efforts that should be pursued to enable improved fidelity and quantification of uncertainty in complex system-level MS&A environments. Study Products Briefing to SAF/OS and AF/CC in July 2019. Publish report in December 2019. Multi-Source Data Fusion for Target Location and Identification (DFT) Background Future Air Force operations in contested environments will require novel sensing, strike, and command and control architectures to enable mission success. In particular, the ability to detect, locate and identify (ID) targets from conventional AF ISR platforms is likely to be challenged by advanced kinetic and non-kinetic threats. Alternative sensing CONOPS leveraging DoD space, ground, and non- traditional air-based sensors coupled with the emerging, large space-based commercial earth-sensing constellations may provide the raw data needed for timely target surveillance, location and ID. Data from intelligence sources, cyber sources, or even publicly available information (PAI) may provide additional, possibly predictive, target location, ID and intent information. This disparate, multi-source data will need to be transported, verified, integrated, and fused into actionable products at speeds consistent with decision-making, tasking and engagement timelines. Recent commercial investments in data fusion, object recognition, and decision aid technologies leveraging advanced machine learning and data analytics techniques have enabled the fusion of large volumes of multi-source data. However, Air Force missions may impose unique data quality, latency, robustness, reliability and security constraints that will influence decisions on the viability of particular fusion architectures. The Air Force will benefit from a better understanding of the options and potential benefits of a robust data fusion architecture to maximize the utility of multi-source non-traditional sensing. Charter The study will: Determine the data products needed to support target location and identification, along with the associated resolution, fidelity, latency, accuracy, and timeliness requirements. Identify combinations of traditional and non-traditional data sources that support location and ID of targets in a contested environment, including government and commercial ground, air, space, cyber, and PAI. Determine current access, use, storage and network pathways for this data. Characterize the effectiveness of current and emerging government and commercial fusion capabilities, including advanced decision support, machine learning, data analytics and human machine interfaces. Identify gaps in current fusion capabilities and assess relevant future S&T efforts to address these gaps. Define a robust data fusion process and architecture that is agnostic to the characteristics of the input data and information source. Assess data management strategies to support this fusion. Assess the impact of current AF network bandwidth limitations on the utility and timeliness of the data products. Consider trades between centralized and “at the edge” data processing to manage these constraints. Identify any network improvements needed for optimal utilization of the data products. Identify potential countermeasures and vulnerabilities to the above approaches, including cyber vulnerabilities in the data fusion process, and degradation of the communications networks. Recommend approaches to assess and evaluate promising fusion capabilities, including data collections, modeling, simulation and analysis (MS&A), experiments and operational demonstrations. Study Products Briefing to SAF/OS & AF/CC in July 2019. Publish report in December 2019.