Transformers for Orbit Determination Anomaly Detection and Classification

Nathan Parrish Ré, Matthew Popplewell, Michael Caudill, Timothy Sullivan, Tyler Hanf, Benjamin Tatman, Kanak Parmar, Tyler Presser, Sai Chikine, Michael Grant, Richard Poulson; Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 2024, pp. 6819-6827

This paper presents the judicious application of machine learning algorithms to solve two fundamental challenges in spacecraft orbit determination (OD): identification of systematic anomalies (nominal vs anomalous behavior) and the classification of these anomalies to explain probable causes (such as unexpected spacecraft maneuvers or mis-modeled small forces). Traditional OD is based on well-tested iterative linearization methods (variations of the Kalman filter). These are commonly understood in the astrodynamics community to require manual tuning for a given set of mission assumptions and re-tuning when those assumptions are invalidated. OD data are typically long sparse multi-variate sequences consisting of multiple observation phenomenologies. These characteristics make the OD problem fundamentally well-suited to the same machine learning architectures (namely transformers) that have found success with language modeling and other sequence-based data modeling. The approach taken here is to simulate various failure modes in the traditional OD process using NASA's Monte software then process the simulated data in three different transformer-based architectures. The three transformer architectures all act as classifiers and can be described at a high level as: 1) treat each epoch with data as a feature vector with the input data comprising a single long sequence; 2) similar to the first but with nested self-attention to efficiently handle longer sequences; and 3) plot the data then classify the plot with a vision transformer (ViT) model. Model performance is studied as a function of the hyperparameter trade space.