Data loss on MSP430FR5739 RS232 while transferring bulky data

chethu gowda said:

What could be the reason for this?

Trying to transfer 1MB without a proper protocol is a disaster waiting to happen.

It just happened!

eg, XMODEM is simple to implement & very widely (almost universally?) supported.

chethu gowda said:

the FR5739 just copying the RX register data into TX register in the ISR

Another disaster waiting to happen!

You really need to ensure that the Tx register is ready for a new byte before you load another!!

You need, at the very least, to implement your software flow control on both sides (receiving & transmitting) of the MSP430FR5739 

And remember that software flow control is never "instant" - your receiver must cope with "a few" more characters arriving after it has sent the XOFF signal...

Andy Neil said:

XMODEM is simple to implement & very widely (almost universally?) supported.

I shall try implementing some sort of protocol with FR5739 if possible. since XMODEM does not support error correction I shall implement some other protocol.

Andy Neil said:

You really need to ensure that the Tx register is ready for a new byte before you load another!!

I thot its understood that befor txing the data the tx buffer is checked so din post it in the thread:)  otherwise the actual code in the ISR code,

    while (!(UCA1IFG&UCTXIFG));
    UCA1TXBUF = UCA0RXBUF;

Andy Neil said:

And remember that software flow control is never "instant" - your receiver must cope with "a few" more characters arriving after it has sent the XOFF signal.

I guess it is not possible to have software flow control on MSP430FR5739, so only solution will be to making use of some protocol for the data integrity.

chethu gowda said:

since XMODEM does not support error correction I shall implement some other protocol

XMODEM supports error detection - which allows you to correct errors by simply retransmitting the "errored" packet. This should be entirely sufficient!

Implementing an error-correcting protocol is going to be an order of magnitude more complex - are you sure you are ready for that?

chethu gowda said:

I thot its understood

If you don't state it, it can't be taken for granted - especially as there seem to be some pretty basic issues here

chethu gowda said:

    while( !(UCA1IFG&UCTXIFG) );
    UCA1TXBUF = UCA0RXBUF;

That clearly will cause received characters to be lost!!

chethu gowda said:

I guess it is not possible to have software flow control on MSP430FR5739

Not at all - you simply have to design the software carefully with enough buffering for it to work...

These are really basic issues that need to be addressed before you even think of fancy stuff like error-correcting protocols!

Let me thanq u for ur honest reply,

Andy Neil said:

Implementing an error-correcting protocol is going to be an order of magnitude more complex - are you sure you are ready for that?

No, not really . A retransmission mechanism should be fine for me as I can compromise with the communication speed for some extent.

Andy Neil said:

Not at all - you simply have to design the software carefully with enough buffering for it to work...

Yup, I got it.. thnq u very much..

Andy Neil said:

before you even think of fancy stuff like error-correcting protocols!

I know the effort and pain involved in developing the error correction mechanism.. I just meant about acking the sender about the error and getting it corrected by retransmission request.. may be I was not clear with my explanation.

I do not understand why you need to put MSP430FR5739 between your two devices. Looks like a "null modem" arrangement will do the job.

The "24MHz" is actually 23.8 MHz +/- 3%. Are you sure you got the baud-rate correctly. When a incoming RXD has error (caused by marginal baud-rate mismatch), by default, it is ignored. Could that be the cause of losing of data?

Hi,

old_cow_yellow said:

why you need to put MSP430FR5739 between your two devices

Actually the MSP430FR5739 sits between two different devices as below with different communication protocol on another side,

                              RS232                                         SPI

<My Device1> <----------> MSP430FR5739   <-----------> <Their Device2>

currently the <Their Device2> is not available with us, so for time being using two <My Device1>s to check the bulk data transfer reliability (my requirement is 1Mb).

old_cow_yellow said:

Are you sure you got the baud-rate correctly.

Yup, the baudrate is calculated taking DCO=24MHz  and the communication is working fine except few Kbs of data (found ~15Kb) loss for transmission of 1Mb file.

old_cow_yellow said:

The "24MHz" is actually 23.8 MHz +/- 3%

with the above range of DCO frequency (23.086MHz to 24.514Hz), how could one select the proper DCO frequency :(  

chethu gowda said:

Are you sure you got the baud-rate correctly.

Yup, the baudrate is calculated taking DCO=24MHz  and the communication is working fine except few Kbs of data (found ~15Kb) loss for transmission of 1Mb file. ...

[/quote]

But the DCO is not exactly 24MHz. Thus the baud-rate is not exactly correct. That could well be the reason why 0.1% 1.5% of the data are lost.

Could you show how the UART is setup?

old_cow_yellow said:

Could you show how the UART is setup?

void UCA1_init()
{
  P2SEL1 |= BIT5 + BIT6;
  P2SEL0 &= ~(BIT5 + BIT6);
  UCA1CTL1 |= UCSWRST;
  UCA1CTL1 = UCSSEL_2;                     
  UCA1BRW = 0x0006;
  UCA1MCTLW |= 0xAA00;
  UCA1CTL1 &= ~UCSWRST; 
  UCA1IE |= UCRXIE;                        
}

void clock_init()
{
    CSCTL0 = 0xA500;
    CSCTL1 = 0x0086;
    CSCTL3 = 0x0050;
}

void main(void)
{
         ;
  clock_init();
  UCA1_init();

         ;

}

So, the baud-rate is 3.66Mb/s subject to +/-3% accumulative error due to DCO inaccuracy and additional error due to fractional baud-rate modulation support.

old_cow_yellow said:

So, the baud-rate is 3.66Mb/s

how did u arrive at 3.66Mb/s.. The baudrate derived is 115200bps.

chethu gowda said:

how did u arrive at 3.66Mb/s.. The baudrate derived is 115200bps.

I'd recommend feeding full 24MHz into the eUSCI, and using a larger baudrate divider there. It miniizes the jitter/bit error caused by modulation.

Regarding your code, you must read data when it comes and not when you see that you can forward it.
Assume your own baudrate is 1% slower than that of the peripherals. Then you get e.g. 1k data per second into RXBUF but are capable of sending 990 bytes per second only. You lose 10 bytes. Or, if you buffer them, then you pile up 10 byte sper second which have to be sent to the target when the source has finished sending and you find additional time.

So if yo don't implement a flow control and can stop the sender wehn your buffer runs full, you'll have to ensure that your sending frequency is a tad higher than the receiving frequency. But in your case it seems to be 1.5% lower. Hence the data loss of 1.5%

Sorry, like JMG said, I missed your CSCTL3 = 0x0050; which means SMCLK = 23.8/32 instead of 23.8 MHz. If you are lucky you may be able to squeeze by using the following changes:

void UCA1_init()
{
  P2SEL1 |= BIT5 + BIT6;
  P2SEL0 &= ~(BIT5 + BIT6);
  UCA1CTL1 |= UCSWRST;
  UCA1CTL1 = UCSSEL_2;                     
  UCA1BRW = 0x0006; 208;
  UCA1MCTLW |= 0xAA00; 0xAD00
  UCA1CTL1 &= ~UCSWRST; 
  UCA1IE |= UCRXIE;                        
}

void clock_init()
{
    CSCTL0 = 0xA500;
    CSCTL1 = 0x0086;
    CSCTL3 = 0x0050; 0x0000;
}

void main(void)
{
         ;
  clock_init();
  UCA1_init();

         ;

}

Jens-Michael Gross said:

I'd recommend feeding full 24MHz into the eUSCI, and using a larger baudrate divider there. It miniizes the jitter/bit error caused by modulation.

I shud try this.

Jens-Michael Gross said:

you must read data when it comes and not when you see that you can forward it.

Absolutely, thats d flaw in my code.. Now I am using the buffer to handle this data and a tailored XMODEM protocol.

Jens-Michael Gross said:

I'd recommend feeding full 24MHz into the eUSCI, and using a larger baudrate divider there. It miniizes the jitter/bit error caused by modulation.

Regarding this I have a big confusion, If at all there is very little baudrate difference between transmitter and receiver, is the jitter carried on till the completion of the frame or carried on continuously for all the data stream?

chethu gowda said:

I am using the buffer to handle this data and a tailored XMODEM protocol.

Note that XMODEM is self-pacing; ie, the protocol itself inherently performs flow control - you do not need to provide anything else. Just be sure that you only ACK a frame when you're ready to accept another one...

chethu gowda said:

a tailored XMODEM protocol

What do you mean by that?

One of the big advantages of XMODEM is its ubiquitous support - you kill that if you mess with it!

Andy Neil said:

What do you mean by that?

I only used the mechanism of XMODEM and its not exactly the XMODEM. for example my packet does not have the block number/inverse block number, instead it has the packet length in place.

Andy Neil said:

One of the big advantages of XMODEM is its ubiquitous support - you kill that if you mess with it!

I am not planning to provide a generic communication support as the interface will always remain fixed between my device and the MSP430 device. Both are programmed at my end only. So I guess it should not be the problem.

chethu gowda said:

my packet does not have the block number/inverse block number

So how will you detect if you lose a packet?

Also, the "value-and-inverse" gives a check that you haven't missed the start of a packet - otherwise, there is no way to tell if your length is valid or not!

chethu gowda said:

I am not planning to provide a generic communication support

But how will you test it...?

My sole intention behind this XMODEM like protocol implementation is to avoid data loss in the system. again my system is as follows,

                              RS232                                         SPI

<My Device1> <----------> MSP430FR5739   <-----------> <Their Device2>

Since there is no FIFO buffer available on MSP430FR5739 I thought this would be d best solution.

Andy Neil said:

So how will you detect if you lose a packet?

The distance between <My Device> and MSP430FR5739 is almost negligible, i.e FR5739 will be like a daughter card in this case which plugs into <My Device1>, so assuming that there will not be loss of data in between. And if you are talking about losing the data in other ways as part of protocol, <My Device1> will not send the next packet until it receives the ACK from FR5739. So if at all <My Device1> has to resend the packet it should be the previous packet itself which is sent on receiving the NACK from FR5739 or on timeout.

Andy Neil said:

But how will you test it...?

Simple, if the file received at <Their Device2> is same as the file sent from <My Device1>, then its working.

chethu gowda said:

assuming that there will not be loss of data

But you've shown & stated yourself that this is not a valid assumption - your system does lose data!!

Andy Neil said:

your system does lose data!!

Yes, my system is losing data and thats what this thread is all about.

But how the data being lost is the key factor here. The data is not getting lost due to hardware connection. But its happening only due to data overrun. i.e I am not reading the data until my tx line is free.

chethu gowda said:

my system is losing data and thats what this thread is all about.

Indeed - and XMODEM was my suggestion to combat this issue!

But how the data being lost is the key factor here. The data is not getting lost due to hardware connection. But its happening only due to data overrun

The mechanism is irrelevant.

The fact is that data gets lost, and you need some way to detect that, and recover. XMODEM offers this - but you are compromising it by removing essential elements of the protocol.

Andy Neil said:

The fact is that data gets lost, and you need some way to detect that, and recover. XMODEM offers this - but you are compromising it by removing essential elements of the protocol.

I am verifying the checksum and in case of mismatch sending back the NACK to <My Device1> which will resend the same packet. How is this different from XMODEM's?

You can't tell if you have missed a complete block!

Also, the true+complement in the block number field gives you added protection against errors in the "header" section.

chethu gowda said:

I'd recommend feeding full 24MHz into the eUSCI, and using a larger baudrate divider there. It miniizes the jitter/bit error caused by modulation.

Regarding this I have a big confusion, If at all there is very little baudrate difference between transmitter and receiver, is the jitter carried on till the completion of the frame or carried on continuously for all the data stream?[/quote]

Well, teh UART cna only generate a bit change on a clock edge. So if you have a clock/baudrate ratio of, say,  1.5, each bit can only be either 1 or 2 clock cycles long. Modulation controls how often it is n clock cycles and how often if is n+1 clock cycles. In this example, firs tbit would be 2 cycle slong, second would be 1 cycle long, third 2, fourth 1. So after one byte, all ten bits are sent in 15 clock cycles. However, the edges between two bits will be 0% or 50% off. Well, that's an extreme example.

In your case, the divider is a bit higher, so the bit rate error is lower. But the maximum allowed error is only a few % and this is for both sender and receiver side. Exceeding this limit may cause single bit errors on certain bit pattern up to completely unreliable communication.
The higher the baudrate divider is, the better can the modulaiton distribute between two integer dividers.

The error is reset on the next start bit edge that synchronizes sender and receiver for the next byte.

Note that this is of importance even if the average baudrate of sender and receiver is an exact match. Still individual bits appear to have a too-high and too-low baudrate, straying from the ideal baudrate way more than the average baudrate.