Intelligent summarization of data

In contexts where there is an overwhelming abundance of data, we need to separate the signal from the noise, and curate meaningful representations of data that are sufficient for training AI models. Two main approaches involve filtering approaches that sample data for accurate representations of the underlying distribution, as well as compression approaches, which try to reduce the size of a data set (e.g. dimensionality reduction). This project will be focused on anomaly detection in aircraft, where many streams of high-frequency data are available, making it difficult for AI to scale to real-world demands. We are interested in using anomaly detectors in order to forecast periods of unexpected aircraft behavior so we may turn on data recording elements only when needed, and ignore the overwhelming amount of data that corresponds to nominal operation modes.