Class Oscillations2014

This is the latest of three implementations to detect oscillations in a spectrogram. This implementation is generic, that is, it attempts to find any and all oscillations in each of the frequency bins of a short duration spectorgram.

There are three versions of the generic algorithm implemented in three different methods:

1. uses auto-correlation, then FFT
2. uses auto-correlation, then singular value decomposition, then FFT
3. uses wavelets

I gave up on wavelets after some time. Might work with persistence! Singular value decomposition is used as a filter to select the dominant oscillations in the audio segment against noise.

The Oscillations2012 class uses the DCT to find oscillations. It works well when the sought oscillation rate is known and the DCT can be tuned to find it. It works well, for example, to find canetoad calls. However it did not easily extend to finding generic oscillations.

Oscillations2014 therefore complements the Oscillations2012 class but does not replace it.

Inheritance

Object

Oscillations2014

Inherited Members

Object.Equals(Object)

Object.Equals(Object, Object)

Object.GetHashCode()

Object.GetType()

Object.MemberwiseClone()

Object.ReferenceEquals(Object, Object)

Object.ToString()

Namespace: AudioAnalysisTools

Assembly: AudioAnalysisTools.dll

Syntax

public static class Oscillations2014

Fields

DefaultFrameLength

Declaration

public static int DefaultFrameLength

Field Value

Type	Description
Int32

DefaultResampleRate

Declaration

public static int DefaultResampleRate

Field Value

Type	Description
Int32

DefaultSampleLength

Declaration

public static int DefaultSampleLength

Field Value

Type	Description
Int32

DefaultSensitivityThreshold

Declaration

public static double DefaultSensitivityThreshold

Field Value

Type	Description
Double

Methods

AdjustSampleSize(Int32, Int32)

Adjusts sample length i.e. patch size for short recordings.

Declaration

public static int AdjustSampleSize(int framecount, int sampleLength)

Parameters

Type	Name	Description
Int32	framecount	number of frames in the frequency bin to be processed.
Int32	sampleLength	the number of frames to be included in each patch.

Returns

Type	Description
Int32	appropriately reduced patch size.

GenerateOscillationDataAndImages(FileInfo, Dictionary<String, String>, Boolean)

Generates the FREQUENCY x OSCILLATIONS Graphs and csv I have experimented with five methods to search for oscillations: 1: string algorithmName = "Autocorr-FFT"; use this if want more detailed output - but not necessrily accurate! 2: string algorithmName = "Autocorr-SVD-FFT"; use this if want only dominant oscillations 3: string algorithmName = "Autocorr-Cwt"; a Wavelets option but could not get it to work well 4: string algorithmName = "Autocorr-WPD"; another Wavelets option but could not get it to work well 5: Discrete Cosine Transform The DCT only works well when you know which periodicity you are looking for. e.g. Canetoad.

Declaration

public static Tuple<Image<Rgb24>, double[, ]> GenerateOscillationDataAndImages(FileInfo audioSegment, Dictionary<string, string> configDict, bool drawImage = false)

Parameters

Type	Name	Description
FileInfo	audioSegment
Dictionary<String, String>	configDict
Boolean	drawImage

Returns

Type	Description
Tuple<SixLabors.ImageSharp.Image<SixLabors.ImageSharp.PixelFormats.Rgb24>, Double[,]>

GetConfigDictionary(FileInfo)

Declaration

public static Dictionary<string, string> GetConfigDictionary(FileInfo configFile)

Parameters

Type	Name	Description
FileInfo	configFile

Returns

Type	Description
Dictionary<String, String>

GetDefaultConfigDictionary(FileInfo)

Declaration

public static Dictionary<string, string> GetDefaultConfigDictionary(FileInfo sourceRecording)

Parameters

Type	Name	Description
FileInfo	sourceRecording

Returns

Type	Description
Dictionary<String, String>

GetFrequencyByOscillationsMatrix(Double[,], Double, Int32, String)

Declaration

public static double[, ] GetFrequencyByOscillationsMatrix(double[, ] spectrogram, double sensitivity, int sampleLength, string algorithmName)

Parameters

Type	Name	Description
Double[,]	spectrogram
Double	sensitivity
Int32	sampleLength
String	algorithmName

Returns

Type	Description
Double[,]

GetFreqVsOscillationsDataAndImage(BaseSonogram, String)

Only call this method for short recordings. If accumulating data for long recordings then call the method for long recordings - i.e. double[] spectralIndex = GenerateOscillationDataAndImages(FileInfo audioSegment, Dictionary configDict, false, false).

Declaration

public static Oscillations2014.FreqVsOscillationsResult GetFreqVsOscillationsDataAndImage(BaseSonogram sonogram, string algorithmName)

Parameters

Type	Name	Description
BaseSonogram	sonogram
String	algorithmName

Returns

Type	Description
Oscillations2014.FreqVsOscillationsResult

GetFreqVsOscillationsImage(Double[,], Double, Double, Int32, String)

Creates an image from the frequency/oscillations matrix. The y-axis scale = frequency bins as per normal spectrogram. The x-axis scale is oscillations per second.

Declaration

public static Image<Rgb24> GetFreqVsOscillationsImage(double[, ] freqOscilMatrix, double framesPerSecond, double freqBinWidth, int sampleLength, string algorithmName)

Parameters

Type	Name	Description
Double[,]	freqOscilMatrix	the input frequency/oscillations matrix.
Double	framesPerSecond	to give the time scale.
Double	freqBinWidth	to give the frequency scale.
Int32	sampleLength	to allow calculation of the oscillations scale.
String	algorithmName	the algorithm used to compute the oscillations.

Returns

Type	Description
SixLabors.ImageSharp.Image<SixLabors.ImageSharp.PixelFormats.Rgb24>	bitmap image.

GetOscillationArrayUsingCwt(Double[,], Double, Int32)

Declaration

public static double[] GetOscillationArrayUsingCwt(double[, ] xCorrByTimeMatrix, double framesPerSecond, int binNumber)

Parameters

Type	Name	Description
Double[,]	xCorrByTimeMatrix
Double	framesPerSecond
Int32	binNumber

Returns

Type	Description
Double[]

GetOscillationArrayUsingFft(Double[,], Double)

returns an oscillation array for a single frequency bin.

Declaration

public static double[] GetOscillationArrayUsingFft(double[, ] xCorrByTimeMatrix, double sensitivity)

Parameters

Type	Name	Description
Double[,]	xCorrByTimeMatrix	derived from single frequency bin.
Double	sensitivity	a threshold used to ignore low ascillation intensities.

Returns

Type	Description
Double[]	vector of oscillation values.

GetOscillationArrayUsingSvdAndFft(Double[,], Double, Int32)

reduces the sequence of Xcorrelation vectors to a single summary vector. Does this by: (1) do SVD on the collection of XCORRELATION vectors (2) select the dominant ones based on the eigen values - 90% threshold Typically there are 1 to 10 eigen values depending on how busy the bin is. (3) Do an FFT on each of the returned SVD vectors to pick the dominant oscillation rate. (4) Accumulate the oscillations in a freq by oscillation rate matrix. The amplitude value for the oscillation is the eigenvalue.

NOTE: There should only be one dominant oscillation in any one freq band at one time. Birds with oscillating calls do call simultaneously, but this technique will only pick up the dominant call. #.

Declaration

public static double[] GetOscillationArrayUsingSvdAndFft(double[, ] xCorrByTimeMatrix, double sensitivity, int binNumber)

Parameters

Type	Name	Description
Double[,]	xCorrByTimeMatrix	double[,] xCorrelationsByTime = new double[sampleLength, sampleCount].
Double	sensitivity	can't remember what this does.
Int32	binNumber	only used when debugging.

Returns

Type	Description
Double[]

GetOscillationArrayUsingWpd(Double[,], Double, Int32)

Declaration

public static double[] GetOscillationArrayUsingWpd(double[, ] xCorrByTimeMatrix, double sensitivity, int binNumber)

Parameters

Type	Name	Description
Double[,]	xCorrByTimeMatrix
Double	sensitivity
Int32	binNumber

Returns

Type	Description
Double[]

GetOscillationUsingDct(Double[], Double, Double[,], out Double, out Double, out Double)

returns oscillations using the DCT.

Declaration

public static void GetOscillationUsingDct(double[] array, double framesPerSecond, double[, ] cosines, out double oscilFreq, out double period, out double intenisty)

Parameters

Type	Name	Description
Double[]	array
Double	framesPerSecond
Double[,]	cosines
Double	oscilFreq
Double	period
Double	intenisty

GetSpectralIndex_Osc(AudioRecording, Int32, Int32, Double)

Declaration

public static double[] GetSpectralIndex_Osc(AudioRecording recordingSegment, int frameLength, int sampleLength, double sensitivity)

Parameters

Type	Name	Description
AudioRecording	recordingSegment
Int32	frameLength
Int32	sampleLength
Double	sensitivity

Returns

Type	Description
Double[]

GetSpectrogramMatrix(AudioRecording, Int32)

Declaration

public static double[, ] GetSpectrogramMatrix(AudioRecording recordingSegment, int frameLength)

Parameters

Type	Name	Description
AudioRecording	recordingSegment
Int32	frameLength

Returns

Type	Description
Double[,]

GetXcorrByTimeMatrix(Double[], Int32)

Returns a matrix whose columns consist of autocorrelations of freq bin samples. The columns are non-overlapping.

Declaration

public static double[, ] GetXcorrByTimeMatrix(double[] signal, int sampleLength)

Parameters

Type	Name	Description
Double[]	signal	an array corresponding to one frequency bin.
Int32	sampleLength	the length of a sample or patch (non-overllapping) for which xcerrelation is obtained.

Returns

Type	Description
Double[,]

LogTransformOscillationVector(Double[], Int32)

Declaration

public static double[] LogTransformOscillationVector(double[] vector, int sampleCount)

Parameters

Type	Name	Description
Double[]	vector
Int32	sampleCount

Returns

Type	Description
Double[]

TESTMETHOD_DrawOscillationSpectrogram()

Declaration

public static void TESTMETHOD_DrawOscillationSpectrogram()

TESTMETHOD_GetSpectralIndex_Osc()

test method for getting a spectral index of oscillation values.

Declaration

public static void TESTMETHOD_GetSpectralIndex_Osc()