MCT8316A: Some drivers are stuck on startup or during operation

Update: Not entirely unexpected, a power cycle brought the controller back to life. Who knows when it will fail for the next time. In case it matters, here is the register dump of the controller when the fan is actually running at approx. 10krpm:

isd_config               (0x0080): 0x4404c140
motor_startup1           (0x0082): 0x75489197
motor_startup2           (0x0084): 0x1a2d7980
closed_loop1             (0x0086): 0x0b3b2400
closed_loop2             (0x0088): 0x020024f8
closed_loop3             (0x008a): 0x4cc40110
closed_loop4             (0x008c): 0x000ce944
const_speed              (0x008e): 0x00a00500
const_pwr                (0x0090): 0x66664c80
150_deg_two_ph_profile   (0x0096): 0x36db6da6
150_deg_three_ph_profile (0x0098): 0x36db6d80
trap_config1             (0x009a): 0x054ba106
trap_config2             (0x009c): 0x3a880000
fault_config1            (0x0092): 0x7fff9071
fault_config2            (0x0094): 0x0f47a009
pin_config1              (0x00a4): 0x00000050
pin_config2              (0x00a6): 0x00101462
device_config            (0x00a8): 0x7fff0000
gd_config1               (0x00ac): 0x1c40030c
gd_config2               (0x00ae): 0x16a00000
gate_driver_fault_status (0x00e0): 0x00000000
controller_fault_status  (0x00e2): 0x00000000
sys_status1              (0x00e4): 0x00ef0bf0
sys_status2              (0x00ea): 0x2001197e
sys_status3              (0x00ec): 0x000b0047
algo_ctrl1               (0x00e6): 0x00000000
device_ctrl              (0x00e8): 0x00000000
input_duty               (0x040c): 0x02fc0000
current_duty             (0x04f6): 0x00000000
set_duty                 (0x0506): 0x00020000
motor_speed_pu           (0x05b2): 0x0000330e
dc_bus_power_pu          (0x06f4): 0x000000cb

Hi Philipp,

Thanks for providing details. I will look into this and update you by mid of next week.

Thanks and Best Regards

Venkatadri S

Hi Venkatadri,

are there any news yet? I took the driver unit showing the most failures out of our device and have it now running on the table (with exactly the same type of motor and external controller) and it just works without issues so far. Is there any kind of special control signal waveform or electrical noise of a specific frequency or something, which could trigger such a behavior?

Thanks and best regards,
Philipp

Hi Philipp

I will review and reply by Monday. 

Thanks and Best Regards

Venkatadri S

Hi Philipp,

I am aiming to reply to you by 6th .

Thanks and Best Regards

Venkatadri S

Hi Philipp,

I looked at EEPROM configuration ,

1. Faults (MTR_LOCK_MODE, LOCK_ILIMIT_MODE, CBC_ILIMIT_MODE and lock fault detection) all are disabled.

2. Motor startup is slow 1st cycle - many time slow first cycle frequency may not work , we can set this after verifying and selecting acceleration which suits well for mot0r inertia 

I have copied to GUI and made few changes (as listed above)

Also, what is the hardware used? Can you share schematic and bulk capacitor values sued?

Thanks and Best Regards

Venkatadri S

Hi Venkatadri,

thanks for your response. It is correct that the faults are disabled as far as possible. I had to do this to reliably drive the motor (except for the described failures), otherwise it would just randomly stop due to a fault condition. However, the faults all seem to be false-positives, as I wasn't able to measure any abnormal conditions when it turned off.

Please see my initial posting for schematic and layout.

Please also note that we had two failure modes:

- The motor fails to start at all

- The motor fails during runtime

While I agree that a start failure may be linked to the selection of startup algorithm, the stall during runtime must have a different cause.

I'll review your proposed changes to the configuration and try if these improve the situation. And if the controller is still able to drive the motor at all, of course.

Best regards,
Philipp

Hi Philipp,

I will wait for your test result.

Meanwhile I will review the schematic and provide my observation,

Thanks and Best Regards

Venkatadri S

Hi Venkatadri,

I'll only have a chance to test the proposed settings tomorrow, but I reviewed the changes. While some will probably work, I have doubt that other changes will still allow the motor to run, as it needed quite some trial and error to get it working in the first place. I decoded the bits by hand, so there might be a mistake somewhere, but these are the changes that I found:

MTR_STARTUP: 11 -> 00 (Slow 1st cycle -> Align)
ALIGN_RAMP_RATE: 0101 -> 0111 (5V/s -> 10V/s)
ALIGN_TIME: 1010 -> 1010 (1s)
ALIGN_CURR_THR: 0100 -> 0011 (0.4V -> 0.3V)
CL_DEC: 01110 -> 01110 (20V/s)
PWM_FREQ_OUT: 11001 -> 01111 (70kHz -> 20kHz)
CBC_ILIMIT_MODE: 1111 -> 0111 (CBC limit disabled)
LOCK_ILIMIT: 1111 -> 1111 (1.5V)
LOCK_ILIMIT_MODE: 1111 -> 0000 (Ilimit lock det disabled -> Ilimit lock det latched fault)
LOCK_ILIMIT_DEG: 0010 -> 0010 (5ms)
CBC_RETRY_PWM_CYC: 000 -> 000 (0 PWM cycles)
MTR_LCK_MODE: 1110 -> 0000 (Lock det disabled -> Lock det latched fault)
LCK_RETRY: 001 -> 001 (500ms)
LOCK1_EN: 0 -> 1
LOCK2_EN: 0 -> 1
LOCK3_EN: 0 -> 1
LOCK_ABN_SPEED: 1111 -> 1011 (4kHz -> 3kHz)
OVP_EN: 0 -> 1
DAC_XTAL_CONFIG: 1 -> 0
PIN_CONFIG2_RESERVED_23_20: 0000 -> 0001
SLEEP_TIME: 11 -> 00 (200ms -> 50us)
EXT_WD_FAULT: 0 -> 0
EXT_WD_FREQ: 00 -> 00
PIN_CONFIG2_RESERVED_12_0: 00000 00000000 -> 10010 00000000
PERIPH_CONFIG: 0x00000000 -> 0x41C01F00 (?)

Is this as you changed the values in the GUI? Questions/doubts arise especially with the following:

• Startup mode: An align time of 1s is most certainly far too long and will cause an overcurrent condition.
• PWM_FREQ_OUT: 70kHz -> 20kHz: The motor has very low inductance and I even needed to add inductance to the board, so I doubt that reducing the PWM frequency will have a positive effect; in fact, the 70kHz already are a compromise, because the commutation seems to often fail at higher frequencies.
• In the PIN_CONFIG2 register, some reserved bits have changed . Do they have any special meaning?
• The PERIPH_CONFIG register is not documented in the datasheet (it is only mentioned in the address table with a recommended value of 0x00000000). What does the change from 0x00000000 to 0x41C01F00 do?

Hi Philipp

I am out of office until Friday, I will explain reasons .

Thanks and Best Regards 

Venkatadri S 

Hi Venkatadri,

I was able to test the driver's behavior with the proposed register settings. To my surprise (honestly) it actually was able to run even at 20kHz, although the power dissipation is somewhat higher. Then I reduced the align time to 100ms and also lowered the open loop voltage settings somewhat to have it start up a bit smoother.

Unfortunately, there is no improvement concerning the observed controller crashes (or whatever it is). But since I had it hooked up to the EVM (only GND, SCL and SDA), I could test a bit more and discovered something like a pattern:

1. The motor is running at the duty cycle as configured

2. With the automatic motor status readings, the reported electrical speed is always changing by a few Hz

3. Suddenly, the reported electrical speed does not change anymore

4. I²C access is still fine, all registers can be read and the duty cycle can be changed via the I²C override

5. The motor is still spinning and it responds to changes in duty cycle

6. Even with large speed changes, the reported electrical speed does not change anymore

7. I can turn the motor off by setting the duty cycle to 0

8. Even with the motor stopped, the state machine is still reported to be in MOTOR_RUN and the electrical speed is also reported at the same value as before

9. When I now again set the duty cycle to a nonzero value, the motor does not start anymore

10. The only way to get the controller back working is by power-cycling it

The behavior was the same also with 30kHz and 80kHz PWM frequency and otherwise the same settings.

I then disconnected the controller PCB (configured for 20kHz) from the EVM and connected it to our own hardware with the analog speed signal to drive it with a simple test procedure (5%, wait 5s, 25%, wait 5s, 10%, wait 5s, 50%, wait 5s, 0%, wait 10s, repeat). This worked for the first cycle, but as soon as the motor was turned off, it refused to start again. Same behavior as when connected to the EVM, but this time without even having something connected to the I²C interface. I then did a power cycle and kept the test program running and since then it is working fine.

This is really driving me crazy. The failures happen far too often (at least with some drivers) for any serious application, but I can't reliably reproduce them by any means.

Any ideas?

Here are the actual register settings that I used now:

{
  "signature": "oneui-register-data",
  "data": [
    [
      {
        "idx": 0,
        "id": "isd_config",
        "value": "0x4404C140"
      },
      {
        "idx": 1,
        "id": "motor_startup1",
        "value": "0x06A69197"
      },
      {
        "idx": 2,
        "id": "motor_startup2",
        "value": "0x012D7980"
      },
      {
        "idx": 3,
        "id": "closed_loop1",
        "value": "0x0B39E400"
      },
      {
        "idx": 4,
        "id": "closed_loop2",
        "value": "0x020024F8"
      },
      {
        "idx": 5,
        "id": "closed_loop3",
        "value": "0x4CC40110"
      },
      {
        "idx": 6,
        "id": "closed_loop4",
        "value": "0x000CE944"
      },
      {
        "idx": 7,
        "id": "const_speed",
        "value": "0x00A00500"
      },
      {
        "idx": 8,
        "id": "const_pwr",
        "value": "0x66664C80"
      },
      {
        "idx": 9,
        "id": "150_deg_two_ph_profile",
        "value": "0x36DB6DA6"
      },
      {
        "idx": 10,
        "id": "150_deg_three_ph_profile",
        "value": "0x36DB6D80"
      },
      {
        "idx": 11,
        "id": "trap_config1",
        "value": "0x054BA106"
      },
      {
        "idx": 12,
        "id": "trap_config2",
        "value": "0x3A880000"
      }
    ],
    [
      {
        "idx": 0,
        "id": "fault_config1",
        "value": "0x7BFA1021"
      },
      {
        "idx": 1,
        "id": "fault_config2",
        "value": "0x7B47A008"
      }
    ],
    [
      {
        "idx": 0,
        "id": "gd_config1",
        "value": "0x1C44090C"
      },
      {
        "idx": 1,
        "id": "gd_config2",
        "value": "0x16A00000"
      }
    ],
    [
      {
        "idx": 0,
        "id": "pin_config1",
        "value": "0x00000050"
      },
      {
        "idx": 1,
        "id": "pin_config2",
        "value": "0x00101462"
      },
      {
        "idx": 2,
        "id": "device_config",
        "value": "0x7FFF0000"
      },
      {
        "idx": 3,
        "id": "peri_config",
        "value": "0x41C01F00"
      }
    ],
    [
      {
        "idx": 0,
        "id": "ana_trim3",
        "value": "0x48004800"
      },
      {
        "idx": 1,
        "id": "ana_trim4",
        "value": "0x00000000"
      },
      {
        "idx": 2,
        "id": "ana_trim5",
        "value": "0x00000013"
      },
      {
        "idx": 3,
        "id": "ana_trim6",
        "value": "0x00000000"
      },
      {
        "idx": 4,
        "id": "ana_trim7",
        "value": "0x00000000"
      },
      {
        "idx": 5,
        "id": "ana_trim8",
        "value": "0x00000861"
      },
      {
        "idx": 6,
        "id": "ana_trim9",
        "value": "0x0099F820"
      },
      {
        "idx": 7,
        "id": "ana_trim10",
        "value": "0x536F0139"
      }
    ],
    [
      {
        "idx": 0,
        "id": "algo_reserved1",
        "value": "0x00000000"
      },
      {
        "idx": 1,
        "id": "algo_reserved2",
        "value": "0x2433407D"
      },
      {
        "idx": 2,
        "id": "algo_reserved3",
        "value": "0x00000000"
      }
    ],
    [
      {
        "idx": 0,
        "id": "gate_driver_fault_status",
        "value": "0x00000000"
      },
      {
        "idx": 1,
        "id": "controller_fault_status",
        "value": "0x00000000"
      }
    ],
    [
      {
        "idx": 0,
        "id": "sys_status1",
        "value": "0x00F00050"
      },
      {
        "idx": 1,
        "id": "sys_status2",
        "value": "0x60010000"
      },
      {
        "idx": 2,
        "id": "sys_status3",
        "value": "0x00000000"
      }
    ],
    [
      {
        "idx": 0,
        "id": "device_ctrl",
        "value": "0x00000000"
      }
    ],
    [
      {
        "idx": 0,
        "id": "algo_ctrl1",
        "value": "0x00000000"
      }
    ]
  ]
}

Thanks.

Best regards,
Philipp

Hi Philipp,

I will verify this condition, I will get back to you on Tuesday if I need any more information.

Thanks and Best Regards

Venkatadri S

Hi Venkatadri,

are there any news on this?

Thanks and best regards,
Philipp

Hi Philipp

I  could not attend to this as planned. 

I am out of office, I request time until Monday / Tuesday.  I will make sure to reply .

Thanks and Best Regards 

Venkatadri S 

Hi Philipp,

Going back to original issue, is this behavior on few devices and reproducible or frequent?

Thanks and Best Regards

Venkatadri S

Hi Venkatadri,

as described in my initial post, we've observed the erratic behavior on about 10% of the units tested so far (~5 bad ones). Due the instability, we don't have a larger batch in operation currently, so I don't know whether more of them would show the same failures at a later time.

The issue is usually reproducible as long as nothing is changed, i.e., the same driver with the same blower in more or less the same position. Connecting a different blower or even power-cycling the unit a few times may "fix" it, but the issue may just come back a bit later. So it's definitely not easily reproducible. The failure interval is probably a few hours, at least for those failing during operation. For those failing during startup, I'd need to come up with a suitable test rig first, to automatically power-cycle and try to start a unit over and over again.

Are there any hints so far, about what could be wrong?

Thanks and best regards,
Philipp

Hi Philipp,

We are analyzing this, we will get back to you.

When motor is running, we generally don't recommend doing burst transaction over I2C which may affect the high priority motor run state.

Can you relax the sampling interval.

Thanks and Best Regards

Venkatadri S

Hi Venkatadri,

please see my initial posting:

- In normal operation, analog control mode is used and I²C is not connected (but pulled up of course)

And also in my report a few days after:

I then disconnected the controller PCB (configured for 20kHz) from the EVM and connected it to our own hardware with the analog speed signal to drive it with a simple test procedure (5%, wait 5s, 25%, wait 5s, 10%, wait 5s, 50%, wait 5s, 0%, wait 10s, repeat). This worked for the first cycle, but as soon as the motor was turned off, it refused to start again. Same behavior as when connected to the EVM, but this time without even having something connected to the I²C interface. I then did a power cycle and kept the test program running and since then it is working fine.

The issue happens when X2 doesn't even have a cable connected, so SDA and SCL are just pulled up using a pair of 100k resistors and there is definitely no I²C traffic. However, I also observed the same failure while having the unit connected to the TI EVM with the GUI regularly reading the motor status, so this doesn't seem to make any difference.

When the failure occurred using the EVM and also after I post-mortem connected the I²C interface on a failing unit, I observed that the interface itself was still working, but the status registers were not updated anymore. I.e., they still indicated a running motor and a constant electrical speed, despite the motor definitely was not running.

Best regards,
Philipp

Hi Philipp,

I sent request to discuss about this issue.

I am discussing about this issue with team, I will get back to you.

Thanks and Best Regards

Venkatadri S

Hi Philipp,

I have another question, can you confirm DRVOFF pin status in your board?

Thanks and Best Regards

Venkatadri S

Hi Venkatadri,

DRVOFF is fixed to GND, as can be seen in the schematic.

Best regards,
Philipp

Hi Philipp,

I see that, thank you.

Also, another question about I2C pull up, 100K is too weak can we reduce to 5K on the board?

I will reply to you about the other issues which is reported.

Thanks and Best Regards

Venkatadri S

Hi Venkatadri,

the 100k Pull-Ups are just to keep the I²C lines at a defined state when nothing is connected to the interface. When it is in use, there are lower-valued resistors (10k) on the master PCB. The I²C is only used during initial configuration/programming of the units (which works just fine) and not connected during normal operation. So unless the I²C lines are extremely susceptible to a tiny bit of electrical noise, I don't expect that lowering these resistance values would have any effect. But I can try lower values when I'm back in the office next week, though it will be hard to determine if there is a change, due to the very limited reproducibility of the issue.

Best regards,
Philipp

Hi Philipp,

Understood, Thanks for confirmation.

Thanks and Best Regards

Venkatadri S

Hi Philipp,

I will test with your JSON on the EVM, can you provide motor type used? What type of inertia and power level?

Thanks and Best Regards

Venkatadri S

Hi Venkatadri

From my initial posting:

• The driver runs a high-speed radial fan at up to 35krpm or about 2.3kHz electrical

• The fan type is TF037E-2000-F

• 10uH inductors are on the PCB, as the motor's own low inductance otherwise causes OCP trips

• Motor operation is 24/7, but the rotation speed is usually 8-20krpm with daily peaks of about 1min. to 35krpm

With a free blowing fan, the 35krpm are reached at around 50% duty cycle with a 24V supply. The power draw in this case is around 30W and increases quickly if even higher rotation speeds are used. The rotor inertia is specified in the datasheet as 1.9×10⁻⁶ kg・m².

Best regards,
Philipp

Hi Philipp,

Thank you, I am out of office tomorrow.

I will aim to respond by week end.

Thanks and Best Regards

Venkatadri S

Hi Venkatadri,

are there any news on this?

Thanks and best regards,
Philipp

Hi Philipp,

Sorry for delaying to respond, I will respond before 30th .

Thanks and Best Regards

Venkatadri S