Franz Luef, Department of Mathematical Sciences, NTNU
Henry McNulty, Department of Mathematical Sciences, NTNU, and Cognite AS
Industry is awash with time series data, from process industry to renewable energy, from natural systems to equipment monitoring. Processing this data appropriately is one of the main goals of industrial data science. Since most time series data are assumed to be time-dependent, functional data analysis can be an attractive framework for industry. However for infinite dimensional function spaces, finding appropriate representations of data is non-trivial. In one possible approach, one can use wavelet or time-frequency methods for functions, although these are for the most part not tuned to specific data sets. In another approach, using a method such as (functional) PCA gives a data set-specific decomposition, while one loses certain advantages of time-frequency or wavelet analysis, in particular stability from redundancy. By leveraging tools from Quantum Harmonic Analysis (QHA), we aim to combine the advantages from both approaches, giving a data set-specific, stable representation of functional data. briefly described.
.
Spectrograms and Localization Operators
In order to understand not only frequency concentration of a function, but also where in the function frequencies are found, one can consider the Short-Time Fourier Transform (STFT) also known as the sliding window Fourier transform. While the Fourier transform considers the whole function simultaneously, the STFT takes a “window” function (usually concentrated around the origin such as the Gaussian) and multiplies the original function by a translation of the window before taking the Fourier transform, and is hence a function in both frequency and time (corresponding to the translation of the window). Like the Fourier transform, the STFT is a unitary transform, although between the original function space and the function space with double the variables. The STFT is intimately related to the Fourier transform and convolutions and can be expressed in a form corresponding to each of these. This joint time-frequency approach, known as Time-Frequency Analysis (TFA), is used in signal and audio processing, image processing, and biomedicine, where the modulus squared of the STFT, known as the spectrogram, often appears.
In particular, the STFT (or spectrogram) is often used as a preprocessing step to machine learning, especially CNNs, where the network can then detect spectro-temporal patterns. A useful tool in analyzing time-frequency composition of functions is the Localisation Operator, which amplifies or preserves certain parts of the time-frequency plane, and removes others based on a weight function or “mask”, and two window functions. One can consider localisation operators as an extension of the bandpass filter to the time domain as well, although there are some subtleties related to the support of the STFT which one must bear in mind.
Quantum Harmonic Analysis
Before we discuss quantum harmonic analysis we recall some facts for classical harmonic analysis, mainly those related to convolution. Recall that the convolution of functions blends one function with another and expresses the amount of overlap of one function as it is shifted over another function. There are many problems where the convolution of two functions is central. Let us mention a few: (i) As a weighted moving average in statistics. (ii) In digital signal processing for the design of linear time-invariant systems and in the study of such systems (iii) In image processing convolutions are used in edge detection, image blurring and denoising. (iv) Convolutional neural networks (CNNs) are a class of deep networks which have many layers such as convolution, ReLu and pooling. CNNs are used to analyze images where they classify images and extract features. They are also applied to analyze audio signals and time-series. (v) Smoothing of “rough” functions or in approximating a not so well-behaved function by a regularized one.
There is an alternative way to describe convolutions via the Fourier transform. Namely, the Fourier transform of the convolution of two functions is equal to the product of the Fourier transforms. Hence, in many situations one studies problems concerning convolutions of functions on the frequency domain, e.g. the design of filters. The interplay between convolution and the Fourier transform is the key to the understanding of convolutions.
The roots of quantum harmonic analysis may be traced back on the one hand to Norbert Wiener’s ground-breaking work on Tauberian theorems (Wiener, 1932), which is about the limiting behavior of summation methods, and on the other hand to describe the transition between classical mechanics and quantum mechanics in a rigorous way. A key step in Wiener’s argument was a method to approximate “rough” functions by superpositions of shifts of “well-behaved” functions, such as Gaussians or B-splines. This is nowadays known as Wiener’s approximation theorem.
In addition to his fundamental and innovative contributions to mathematics Wiener had a keen interest in engineering and computers which eventually led him to formulate the concept of cybernetics and to seminal contributions to information theory. He was also convinced that the 21st century is going to be the area of “Mathematical engineering”, or data science as we call it now, in the sense that computers, robots and algorithms are going to be central to our society.
We now turn to a seemingly unrelated topic, the quantization schemes of quantum physics. Quantization involves associating to a function or distribution an operator, corresponding to the quantum physics equivalent to a classical state. Conversely, a dequantization scheme associates to an appropriate operator a function. In the seminal work (Werner, 1984) an extension of Fourier analysis to the framework of operators was proposed with the motivation on transferring structures between classical and quantum theories such as the physical distinguishability of quantum states and classical notions, see (Werner 2004) or the characterization of informational completeness (Kiukas et al. 2012) and several further consequences for quantum information theory.
In this extension from classical harmonic analysis to operators, known as Quantum Harmonic Analysis (QHA) one can then define convolutions between two operators, and a convolution between an operator and function, as well as a Fourier transform for operators, the Fourier-Wigner transform, which follows a similar Fourier convolution theorem as the function case. There are analogs of the Fourier multiplication theorem turning these operator convolutions into products, Wiener’s approximation theorem and many other basic theorems in classical harmonic analysis in quantum harmonic analysis. The applications of convolution have also analogs in the setting of operators.
While initially the concept of an operator convolution based on the function assigned to it by a certain quantization scheme may seem slightly abstract, the convolutions and operator Fourier transform return familiar objects in the case of rank-one operators. Taking the operator-operator convolution of two rank-one operators gives a product of STFTs, while a function-operator convolution for a rank-one operator gives a localization operator with the function taking the role of mask, and the operator corresponding to the windows (Luef et al. 2018). Quantum harmonic analysis can also be used to describe the layers of CNNs with STFT preprocessing (Doerfler et al. 2021).
From Quantum States to Functional Data Sets
While in classical TFA one analyzes a particular function or signal, in the realm of functional data analysis, we are instead presented with a collection of (possibly time dependent) data points. In this way, QHA can be seen as the appropriate framework with which to approach TFA for functional data sets. There are several ways of associating an operator to a functional data set. We can consider the elementwise tensor products of datapoints, or take the tensor products of the data points with an arbitrary orthonormal basis or frame. The former corresponds to the covariance operator of the probability measure defined by the data set, while the latter operator is of particular interest in cases where the data points have a natural order, for example a time series, since the orthonormal basis gives a way of preserving this structure, as we will see below.
As a first example of QHA for functional data, consider the case of data augmentation. The goal of data augmentation is to expand a given data set with synthetic data based on the original data, to either give a more realistic distribution of data, to make models more robust to slight transformations of the data, such as rotations and translations in image classification, or to allow for the use of larger network architectures. In the case of audio or signal processing, where one can assume that small time-frequency shifts do not affect the fundamental properties of data points, data augmentation can be done by adding to a data set all data points shifted by time-frequency points from a given domain.
In the framework of QHA, we can then consider the covariance operator corresponding to this new augmented data set. This turns out to be precisely the mixed-state localization operator (Luef et al. 2020), given by the convolution of the original covariance operator with the indicator function of the augmentation domain.
By considering data augmentation as a QHA object, we can deduce several properties. Firstly, while functional principal components of a general Hilbert-Schmidt operator are not necessarily smooth or even continuous, by augmenting a data set we can ensure that the PCAs are not only continuous, but in fact have pleasing regularity. This is especially important for real world applications, where physical processes are largely assumed to be continuous and smooth, while data may be noisy. This perspective of data augmentation also informs the domain by which we might augment; one finds that the areas of strongest local correlations between data points in the time-frequency plane are precisely the points by which one should augment the original data set to maintain the effective dimensionality of the data.
We can also use the framework of QHA to analyze time-frequency correlations within data sets. Taking the operator convolution of the operator associated to the data set gives the Total Correlation Function introduced in (Dörfler et al., 2021). This function describes where in the time-frequency plane the cross-correlations between data points are strongest and can therefore be used to select an optimal set of functions to represent the data.
However, in the case of ordered data, such as a functional time series, we may wish to know how the index of the data points affects their time-frequency cross-correlation. To that end we might consider the Operator STFT (OSTFT) (Dörfler et al., 2024). While the total correlation function takes the sum of pairwise STFTs, the OSTFT is operator valued, and hence the ordering of the data points will inform the structure of the operator, while the Hilbert-Schmidt norm of the operator will be the total correlation function.
To illustrate this, we consider two data sets with identical total correlation functions shown above, but whereas one is the result of a stationary process, the other is the result of a time-dependent process. This is then reflected in the structure of the OSTFT at selected points in the time-frequency plane:
The implications for this approach in industry are vast, as many physical systems are expected to have both continuous/smooth data, and have properties reflected by frequency concentrations. Functional data appears naturally in the context of energy production and trading systems, in sensor data in continuous process industry and equipment monitoring. In all these instances QHA provides a complementary framework to traditional functional data techniques.
References
Dörfler, M., Luef, F., McNulty, H., & Skrettingland, E. (2024). Time-frequency analysis and coorbit spaces of operators. Journal of Mathematical Analysis and Applications, 534(2).
Dörfler, M., Luef, F., & Skrettingland, E. (2021). Local Structure and effective Dimensionality of Time Series Data Sets. arxiv:2111.02153.
Luef, F. & Skrettingland, E. (2018). Convolution for localization operators. Journal de Mathématiques de Pures et Appliquées, 118, 288-316.
Luef, F. & Skrettingland, E. (2019). Mixed-state localization operators: Cohen’s class and trace class operators. Journal of Fourier Analysis and Applications, 25, 2064-2108.
Werner, R. (1984). Quantum harmonic analysis on phase space. Journal of Mathematical Physics, 25(5), 1404-1411.Wiener, N. (1932). Tauberian Theorems. Annals of Mathematics, 33(1), 1-100.
