SDFM data outliers

Edit:  Response prompted me to check on our data rates.  They seem to have been too high for our own good and are now within the acceptable range which fixed the problem with over or under valued reads.

The modulator clock is produced by a PWM on the F28377S.  It is physically wired from one pin to another.  The input data rate is much lower (we were testing using 400 KHz or less.)

I have an ISR which is checking the AF flag before reading.  My code is:

interrupt void Sdfm1_ISR()
{
	static uint32_t sdfm1f3_prev = 0;

	// check the interrupt flags to see what brought us here
	if (Sdfm1Regs.SDIFLG.bit.AF3 == 1) // there is new data on filter 3
	{
		// TODO: Do something with the new data
		sdfm1f3_data = Sdfm1Regs.SDDATA3.bit.DATA32HI;
		sdfm1f3_data = sdfm1f3_data << 16; // shift for 32 bit
		sdfm1f3_data |= Sdfm1Regs.SDDATA3.bit.DATA16; // read rest of data

		sdfm1f3_data = TwosComplement(sdfm1f3_data); // undo two's complement so its a number we understand


		sdfm1f3_prev = sdfm1f3_data;  // update previous data

		Sdfm1Regs.SDIFLGCLR.bit.AF3 = 1; // clear interrupt trigger
	}
	if (Sdfm1Regs.SDIFLG.bit.MF3 == 1) // there was a modulator failure (the clock input likely crashed or was not present, etc.)
	{
		Sdfm1Regs.SDIFLGCLR.bit.MF3 = 1; // clear interrupt trigger
		// TODO: Error handling and/or recovery
	}
	if (Sdfm1Regs.SDIFLG.bit.IFL3 == 1) // we're below threshold
	{
		Sdfm1Regs.SDIFLGCLR.bit.IFL3 = 1; // clear interrupt trigger
		// TODO: Correct output to exceed low threshold
		// this data is coming from the comparator and the data is not available now, only that the threshold has been violated.
	}
	if (Sdfm1Regs.SDIFLG.bit.IFH3 == 1) // we're above threshold
	{
		Sdfm1Regs.SDIFLGCLR.bit.IFH3 = 1; // clear interrupt trigger
		// TODO: Correct output to below high threshold
		// this data is coming from the comparator and the data is not available now, only that the threshold has been violated.
	}

	// clear master interrupt flag
	Sdfm1Regs.SDIFLGCLR.bit.MIF = 1; // clear master interrupt
	// acknowledge interrupt in PIE group
	PieCtrlRegs.PIEACK.all = PIEACK_GROUP5; // ACK interrupt, so that other group 5 interrupts can fire
}