• State TI Thinks Resolved
  • Locked Locked
  • Replies 14 replies
  • Answers 3 answers
  • Subscribers 56 subscribers
  • Views 1902 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

RTOS/CC1310: Drift in Seconds_getTime() vs clock timing causing issues.

Craig Easdale
Expert 2200 points

Part Number: CC1310
Other Parts Discussed in Thread: CC1350,

Tool/software: TI-RTOS

I've noticed something a bit strange in one of my applications: A star network comprised of a CC1310 receiver/hub device and multiple CC1350 sensor nodes.

One task of the receiver is to monitor the regular transmissions of the nodes, and raise an alarm if there has been too long a delay since the last transmission from any one device. Nodes are constantly time-sync'd in their Tx using the RAT timer.

To do this I have a clock that times out every 10 seconds, whose callback posts an event to provide an update over serial terminal with a list of the last time it received transmissions from each node. I timestamp (in seconds and milliseconds) the serial update itself using the Seconds_getTime() function.

What I've noticed is that over a period of ~60s the time drifts by around 1ms. So after about 10 mins (600s) of running, the timestamp is "599.990s". 

This itself would not be troublesome, but after a while this drift surpasses the allowable delay for node transmissions, and constantly triggers alarms. 

So firstly, why am I seeing this drift? And secondly, is there a better way to keep track of time that won't drift like this and cause me problems?

{System start}

{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 0.462}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 0.711}
(Receiver: 13f1ee79  Timestamp: 9.999s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 10.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 10.518}
(Receiver: 13f1ee79  Timestamp: 19.999s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 20.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 20.518}
(Receiver: 13f1ee79  Timestamp: 29.999s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 30.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 30.518}
(Receiver: 13f1ee79  Timestamp: 39.999s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 40.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 40.518}
(Receiver: 13f1ee79  Timestamp: 49.999s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 50.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 50.518}
(Receiver: 13f1ee79  Timestamp: 59.999s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 60.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 60.518}
(Receiver: 13f1ee79  Timestamp: 69.998s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 70.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 70.518}
(Receiver: 13f1ee79  Timestamp: 79.998s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 80.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 80.518}
(Receiver: 13f1ee79  Timestamp: 89.998s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 90.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 90.518}
(Receiver: 13f1ee79  Timestamp: 99.998s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 100.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 100.518}
(Receiver: 13f1ee79  Timestamp: 109.998s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 110.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 110.518}
(Receiver: 13f1ee79  Timestamp: 119.998s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 120.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 120.518}
(Receiver: 13f1ee79  Timestamp: 129.998s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 130.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 130.518}
(Receiver: 13f1ee79  Timestamp: 139.997s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 140.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 140.518}
(Receiver: 13f1ee79  Timestamp: 149.997s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 150.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 150.518}
(Receiver: 13f1ee79  Timestamp: 159.997s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 160.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 160.519}
(Receiver: 13f1ee79  Timestamp: 169.997s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 170.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 170.519}
(Receiver: 13f1ee79  Timestamp: 179.997s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 180.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 180.519}
(Receiver: 13f1ee79  Timestamp: 189.997s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 190.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 190.519}
(Receiver: 13f1ee79  Timestamp: 199.996s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 200.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 200.519}
(Receiver: 13f1ee79  Timestamp: 209.996s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 210.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 210.519}
(Receiver: 13f1ee79  Timestamp: 219.996s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 220.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 220.519}
(Receiver: 13f1ee79  Timestamp: 229.996s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 230.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 230.519}
(Receiver: 13f1ee79  Timestamp: 239.996s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 240.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 240.519}
(Receiver: 13f1ee79  Timestamp: 249.996s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 250.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 250.519}
(Receiver: 13f1ee79  Timestamp: 259.996s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 260.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 260.519}
(Receiver: 13f1ee79  Timestamp: 269.995s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 270.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 270.519}
(Receiver: 13f1ee79  Timestamp: 279.995s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 280.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 280.519}
(Receiver: 13f1ee79  Timestamp: 289.995s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 290.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 290.519}
(Receiver: 13f1ee79  Timestamp: 299.995s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 300.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 300.519}
(Receiver: 13f1ee79  Timestamp: 309.995s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 310.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 310.519}
(Receiver: 13f1ee79  Timestamp: 319.995s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 320.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 320.519}
(Receiver: 13f1ee79  Timestamp: 329.994s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 330.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 330.519}
(Receiver: 13f1ee79  Timestamp: 339.994s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 340.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 340.519}
(Receiver: 13f1ee79  Timestamp: 349.994s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 350.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 350.519}
(Receiver: 13f1ee79  Timestamp: 359.994s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 360.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 360.519}
(Receiver: 13f1ee79  Timestamp: 369.994s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 370.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 370.519}
(Receiver: 13f1ee79  Timestamp: 379.994s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 380.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 380.519}
(Receiver: 13f1ee79  Timestamp: 389.993s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 390.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 390.519}
(Receiver: 13f1ee79  Timestamp: 399.993s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 400.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 400.519}
(Receiver: 13f1ee79  Timestamp: 409.993s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 410.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 410.519}
(Receiver: 13f1ee79  Timestamp: 419.993s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 420.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 420.519}
(Receiver: 13f1ee79  Timestamp: 429.993s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 430.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 430.519}
(Receiver: 13f1ee79  Timestamp: 439.993s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 440.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 440.519}
(Receiver: 13f1ee79  Timestamp: 449.993s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 450.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 450.519}
(Receiver: 13f1ee79  Timestamp: 459.992s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 460.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 460.519}
(Receiver: 13f1ee79  Timestamp: 469.992s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 470.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 470.519}
(Receiver: 13f1ee79  Timestamp: 479.992s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 480.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 480.519}
(Receiver: 13f1ee79  Timestamp: 489.992s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 490.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 490.519}
(Receiver: 13f1ee79  Timestamp: 499.992s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 500.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 500.519}
(Receiver: 13f1ee79  Timestamp: 509.992s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 510.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 510.519}
(Receiver: 13f1ee79  Timestamp: 519.991s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 520.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 520.519}
(Receiver: 13f1ee79  Timestamp: 529.991s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 530.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 530.519}
(Receiver: 13f1ee79  Timestamp: 539.991s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 540.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 540.519}
(Receiver: 13f1ee79  Timestamp: 549.991s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 550.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 550.519}
(Receiver: 13f1ee79  Timestamp: 559.991s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 560.268}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 560.519}
(Receiver: 13f1ee79  Timestamp: 569.991s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 570.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 570.519}
(Receiver: 13f1ee79  Timestamp: 579.990s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 580.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 580.519}
(Receiver: 13f1ee79  Timestamp: 589.990s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 590.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 590.519}
(Receiver: 13f1ee79  Timestamp: 599.990s)
{Heartbeat from 197707e2: ▒▒▒▒▒▒ @ 600.269}
{Heartbeat from 19770a87: ▒▒▒▒▒▒ @ 600.519}

  • 0
    TI__Guru 58425 points
    Hi Craig,

    Some clock drift is to expect as the crystal is not ideal. Assuming a 20ppm 32kHz crystal you would potentially have a ~1.7s / day drift. The drift you are seeing seems a bit higher but I would still expect this being due to the crystal source. The seconds module (as most modules in TI-RTOS for this device) depends on the RTC clock which means that if your timestamps are off it is due to the RTC being off which extends to the clock source used.

    Could you verify the clock frequency of the device?
  • 0 in reply to IUsedToBeARealPerson (M-W)
    Expert 2200 points

    I can't verify the frequency right now, but I'm using a CC1310 Launchpad if that helps.

    So if I understand correctly, the RAT timer and clock module, and the RTC are timed on separate crystals? If this is the case, is there any way to get the time from the same clock source as the RAT and clock module?

  • 0 in reply to Craig Easdale
    TI__Guru 58425 points

    Hi Craig,

    So the RAT and Clock modules does not run of the same clock when active, the RAT clock are however "re-synced" with the RTC timer following a power-up as it does not retain the value when the RF Core is shutdown.
    If the timestamps compared are derived from both RTC and RAT timer value I would guess you could experience more drift, is this the case?

  • 0 in reply to IUsedToBeARealPerson (M-W)
    Expert 2200 points
    The clock module times the callback which in turn gets the time from Seconds_getTime() and does the serial write. So the mismatch is between whatever oscillator the clock module runs on, and the oscillator the RTC is based on.

    Doesn't matter so much that there is drift, but it does matter that the Seconds_getTime() function drifts at a different rate from the clock timing.
  • 0 in reply to IUsedToBeARealPerson (M-W)
    Guru 26200 points
    Hello Craig, M-W,

    One of MSP430 LaunchPads uses a 4MHz crystal with tolerance 0.1%, sic !!!, and a temp. drift 20 ppm.
    I had a time drift over 30s per day.

    I have checked the CC1310 LP BOM list.
    32 kHz has 20 ppm tolerance, aging +/- 3 ppm/year plus some temp. drift.
    24 MHz has 10 ppm tolerance, aging +/- 1 ppm /year plus +/-1, of0 ppm temp. drift.
    Lucky you.

    I am not a hardware guy.
    I understand that a different clock sourcing for RAT and the clock module comes from a device architecture and its operations.
    I have found that 32 kHz oscillator has max sourcing at 1uW, however 24MHz has max sourcing at 200uW with recommended up to 100.


    Am I correct saying that a different clock sourcing comes from a different power optimizations for a different power states?

    I am asking this question because if I am right, it would help me to faster define the expectations and a software architecture for many other TI device families.
  • 0 in reply to Tomasz Kocon
    Expert 2200 points
    @M_W, do you have any further thoughts on this?
    I need to resolve the drift between clock module and seconds_getTime() and/or find away to track the time from the same crystal as the RAT and clock module since it is causing issues within my application.
  • 0 in reply to Craig Easdale
    TI__Guru 58425 points

    Hi Craig,

    There should be zero drift between the Seconds and Clock module (on the same device) as both modules runs of the RTC and thus are based on the same values.

    I could imagine multiple sources of drift if you are using timestamps from the RAT timer and compare that to the Clock/Seconds time. However, the drift you claim to be seeing is in the range of > 80 s / day which is far more then one would expect from these combinations.

    For example, assuming we only have the RTC on each side, if one side can be assumed to have a 1.7s / day drift, if two devices are drifting in opposite direction (one is faster and one is slower) we could expect around 3.4 s /day between the two devices.

    Edit: Miss-calculated the numbers, 1 ms / m drift equals 1.44 s / day which is within the expected clock drift of the RTC. 
    The RAT timer should not be a big issue here as the RF driver will turn the RF Core on and off as much as possible which will trigger a re-sync with the RTC.

    Could you share how you are calculating and sending the timestamp from the Seconds module? A schematic picture would also help to understand just what each timestamp is sourced from as it is multiple devices involved.

  • 0 in reply to IUsedToBeARealPerson (M-W)
    Guru 26200 points
    Hi M-W,

    according to Craig:
    after about 10 mins (600s) of running, the timestamp is "599.990s".
    The claimed drift is 1.44s / day so it is wee within the specs.

    Using RAT for the timestamps is an issue.
  • 0 in reply to Tomasz Kocon
    TI__Guru 58425 points
    You are absolutely correct, I calculated the numbers as being 1 ms / s instead of 1 ms / 60 s. Good catch!

    In that case the drift is as you say expected and the RAT timestamps should be OK as the RAT timer is re-synced each time the RF Core is powered up by the RF Driver.
  • 0 in reply to IUsedToBeARealPerson (M-W)
    Guru 26200 points
    I see two options:
    - timers synchronization: how to do it? how often? how precise?
    - change software architecture and before posting a measured data, replace RAT with RTC + something based on an identified RAT creep.
  • 0 in reply to IUsedToBeARealPerson (M-W)
    Expert 2200 points

    The drift of the RAT timer is to be expected. This isn't important since the nodes Rx time is checked on each transmission and they updated when to next transmit. This period (10s) is not long enough to provide any notable drift between RTC and RAT timer.

    But pleease do not focus on the time sync aspect of this...

    If the clock module and RTC are based off the same clock source then the problem is more unexplained than I had first realised..

    If you look at original sample of serial output youl'll see the drift I noted.

    (Receiver: 13f1ee79  Timestamp: 189.997s)
    ... 
    (Receiver: 13f1ee79  Timestamp: 599.990s)
    The big problem here is that this serial output is triggered by the clock module. If the seconds_getTime() value that is printed is based on the same crystal then this does not make any sense that it should have any variance at all!

  • 0 in reply to Craig Easdale
    TI__Guru 58425 points
    Hi Craig,

    If the clock is triggering the serial output, then most likely it is seeing the same drift.
    What are you comparing to when saying this is not the case, the "Heartbeat" printout?

    Could you maybe provide the code on how this is setup to make it clearer how things are calculated and from where they origin?
  • 0 in reply to IUsedToBeARealPerson (M-W)
    Expert 2200 points

    Hi M-W,

    I'm on the wrong computer to send code snippets right now, but here's the basic setup which I hope adds a little clarity.

    • Initialise clock in task_init, period and timeout of 10s (set to automatically start)
    • ...
    • 10 seconds in, clock callback occurs. Immediately use Seconds_getTime() to record the timestamp at that moment.
    • Do some other processing then Post an event, which in turn calls a function to output the timer value over serial (seen above)
    • Wait again for the clock to trigger the callback and repeat the process.

    As I mentioned, the timestamp I am outputting over the serial is drifting lower. I thought maybe the time to service the clock interrupt could be the cause, but if that was the case the recorded timestamp would drift high, not low.

    My other thought was that the clock may not automatically restart until the callback exits, but again, this would cause drift in the other direction.

  • 0 in reply to Craig Easdale
    TI__Guru 58425 points
    Some code here would be helpful here. As you say, there should be no difference between the clock that triggers the Clock interrupt and what the Seconds module reads out (other then the actual latency between the two calls). I'm wondering if this is a scaling issue somewhere maybe.

    The drift could very well be negative, if the RTC frequency is running slower then expected, then the drift would be negative.