TCA9517A - VccA turns on and off

The datasheet says:

V_CCB and V_CCA can be applied in any sequence at power up. The TCA9517A includes a power-up circuit that keeps the output drivers turned off until V_CCB is above 2.5 V and the V_CCA is above 0.8 V.

and:

All inputs and I/Os are over-voltage tolerant to 5.5 V, even when the device is unpowered (V_CCB and/or V_CCA = 0 V).

It does not explicitly say anything about the power-down sequence, but it does not look as if there are any special requirements.

In any case, consider disabling EN before powering down, just to be sure.