• State
  • Locked Locked
  • Replies 20 replies
  • Subscribers 32 subscribers
  • Views 3082 views
  • Users 0 members are here
Support feedback
Options
Options
Related

This thread has been locked.

If you have a related question, please click the "Ask a related question" button in the top right corner. The newly created question will be automatically linked to this question.

Sensor Hub Booster Pack: Random Failures of the BMP180 Pressure Sensor and the SHT21 Humidity Sensor

Jesus Torrado
Prodigy 160 points
Other Parts Discussed in Thread: TPS54426, TPS2411, LM2585

Greetings E2E Community


I am experiencing permanent failures of the humidity and pressure sensors of the booster pack at random occasions. The booster pack is used to monitor the state (temperature, humidity and orientation) of a sealed underwater vehicle. Typical conditions of the vehicle's sealed environment are temperatures ranging from 0C to 30C and a pressure of about 1 atmosphere.

Diagnostic test are performed on the vehicle before deployments and after recovery to assess its working state. In addition a log of the sensor readings is kept during deployments on a SD card.

At random occasions when performing a pre-deployment diagnostic test or normal bench test of the vehicle the pressure or humidity sensor would fail. This has happened on three separate occasions with 3 different vehicles and three separate booster packs.  The pressure sensor has failed once, exhibiting sporadic readings, usually in the upper range of the sensors capabilities. and the humidity sensors have failed twice, exhibiting constant readings, regardless of changing environmental temperatures.

The power supply of the 3.3V rail of the vehicle is powered by a TPS54426 buck switching converter, with an average output of 3.36V. The environmental and electrical characteristics of the operating environment are withing acceptable ranges for both sensors and no other sensors in the booster pack have exhibited these failures.

Since the failures are exhibited between power on-off cycles of the vehicle I suspect the failures have something to do with a transient voltage in the power supplies, but I am no entirely sure and currently investigating. I am writing to see if someone has experienced these failures before or if someone has any insight on the problem. Any advice is greatly appreciated and thanks, in advance for your time.

  • 0
    TI__Guru**** 244400 points
    Hello Jesus,

    Since the communication interface is I2C for the sensor pack, I would suggest checking the following

    1. The external I2C Line Pull Up are strong enough
    2. The noise filter in the I2C peripheral has been switched ON and set to maximum if the communication is at 100KHz with 80MHz system clock on the TM4C device.

    Regards
    Amit
  • 0 in reply to Amit Ashara
    Guru 47900 points

    Hi Amit,

    As poster has noted, "Permanent Failures" - and these (always and only) occur between Power On/Off cycles - this may suggest (other) than communication related. I vote w/poster - that damaging transients are - prime suspects.

    Certain of my small firm's devices must survive in environments far more hostile than poster's - we employ well chosen MOVs and carefully data-log any/all critical (power & signal) lines - to insure that harmful transients are bounded/absorbed - thus cannot wreak their (normal) havoc!
     
    Poster's ability to fully/properly monitor ALL signal & power lines - attached to those two (failing) sensors - seems a solid, "First attack."

  • 0
    Guru 27665 points
    Two questions

    What are you using for a power supply to your regulator?
    What other demands/loads are on that supply? I can envision a small battery pack and electric motors (BTW if you do have motors what kind?).

    Robert
  • 0 in reply to Robert Adsett
    Prodigy 160 points
    Hi Robert

    The power supply is comprised of two 7.4V 2-cell LiPO Batteries, each battery feeds two rails (redundant) of 3.3V and 12V. The 3.3V is converted using a TPS54426 Buck converter and the 12V rail is converted using a LM2585 boost converter. The redundant rails are OR-ed together with the TPS2411 controllers.

    The loads on the system are:
    - 1 Tiva C Series Micro-controller (30 mA @ 3.3V)
    - 2 MSP430 Micro-controllers (10~15 mA @ 3.3V)
    - 1 Bluetooth module (30 mA @ 3.3V)
    - RF transmiter (less than 1mA @ 3.3V)
    - Led strobe light (~50mA @ 3.3V)
    - 1 SensorHub unit (~1mA @ 3.3V)
    - Occasional 300 mA @ 12V constant current load turn off/on via PWM control. In a duration of 5 seconds, the current load start at 0 mA and ramps up to 300 mA and vice-versa. This was done to avoid strong current surges and transients.

    I hope this can give you a clearer picture of the power supply capabilities and its demands.
  • 0 in reply to Jesus Torrado
    Guru 27665 points
    That's a good picture. My worry was that you had motor or contactor loads. I'd judge the biggest worry would be the RF units and the strobe.

    Reflecting on the symptoms though (and if you verify that you do not have failures in operation) my largest suspicion would be static (ESD). You can make design changes to help if that's the case but a lot would need to be done to the board itself. If you have the room and weight allowance to do it something like a copper mesh cage (connected to the frame) to protect the boards and sensors could be a big help. Other than those it would be procedure changes during testing and retrieval. You may want to make those changes anyway.

    Robert
  • 0 in reply to Robert Adsett
    Guru 47900 points

    Hi Robert,

    Agreed that ESD is (always) lurking - but poster has linked these "2 sensor failures" to Power OFF/ON cycling.   That's what I seized upon - in my opening post.

    Twas me - I'd investigate the "known tendency" - either or both of those sensors - to be "failure prone."   Usually - but not always - sufficient number of failures will be known - and at some point - admitted.   We've no report of poster's effort - in that (basic) direction.

    Again - w/in opening post I suggested careful, strategic probing/data-logging of each/every sensor pin - as close to the device as possible - and especially aimed at  any/all pins "shared" between each sensor.   My belief - "One transient signal ... ... TWO dead sensors!"

    Another obvious test would be to perform long-term test - NOT allowing power to cycle - to (really) confirm that Power OFF/ON is the (real) killer.   (cause)

  • 0 in reply to cb1
    Guru 27665 points
    Yep, I keyed in on that it was occurring during deployment and recovery testing which means handling. And thus ESD rises higher.

    One test might be to do multiple power cycles in the lab w/o handling.

    Robert

    BTW Amit the frequency of "unknown error" upon posting seems to be increasing
  • 0 in reply to Robert Adsett
    Prodigy 160 points
    Hi Robert

    Thanks for the suggestions ESD is a sneaky problem, I will investigate the feasibility of a copper mesh on the circuits on a soon to be completed lab test unit. I think it will be a beneficial addition to the design, event if it might not be the main culprit, it can reduce future "sporadic failures".

    Jesus
  • 0 in reply to cb1
    Prodigy 160 points
    Hey cb1-

    I apologize for the delay in a power transient test, we are currently in the process of assembling 4 new vehicles with "fresh" components (we lost 2 vehicles at sea). One of them is meant for lab tests we expect to finish the soldering and assembly of them by Friday. Once complete I hope to be able to perform monitor tests on the power rails closest to the sensors, and see if they provide any insight. I will keep you posted once the test are performed, thanks for the advice.

    Jesus
  • 0
    Guru 47900 points

    Hi Jesus,

    Thank you - poster/friend Robert raises the point of "handling."   Yet - repeated read/review of your writing - essentially remains (silent) in that regard.

    Jesus Torrado said:
    Since the failures are exhibited between power on-off cycles of the vehicle

    I interpreted that as "only" power OFF/ON - w/minimal or NO handling.   It is never/ever wise to burden any test w/unnecessary handling!   Might you clarify if such handling occurred?   (and if it did - I'd stop that handling - and then continue with repeated power cycles...)

    Now to those sensors - I don't believe the intent of any booster-pack is to deploy "latest/greatest" and (especially) the most robust.   Game here is Sales - and that's realized (most always) via low-cost - reduction in user labor - and a desired array of features.   Long term - device robustness - I'd not expect.

    I should state that my firm - along w/several others - has produced control for (possibly) similar "under-water vehicles" - which were past tested off Catalina Isle.   (26 miles S.W. of Los Angeles.   I'm a sailor - my boat has made that passing many times - sometimes w/out incident.   In/around the less active areas of Catalina harbor serves as an excellent environment for such testing.   (we/others do NOT employ the sensors you note!   And thus far - none have failed...)

    Reinventing the wheel is seldom fun - nor efficient - nor worthwhile.   Perhaps your call to the sensor's maker - keying on your failures - will speed your diagnosis.

    My first response outlined (some) of the methods which (may) have allowed our, "underwater data harvesting/observation" to persist w/out sensor failure.

  • 0 in reply to cb1
    Guru 27665 points

    cb1- said:
    Now to those sensors - I don't believe the intent of any booster-pack is to deploy "latest/greatest" and (especially) the most robust.   Game here is Sales - and that's realized (most always) via low-cost - reduction in user labor - and a desired array of features.   Long term - device robustness - I'd not expect.

    Agreed. Also true of 'duino like systems from what I've observed.

    cb1- said:
    perhaps your call to the sensor's maker - keying on your failures - will speed your diagnosis.

    You may even be able to get them to check the sensors to see if they can observe the likely cause. Some failure modes will leave particular signatures on the silicon. Of course sometimes all they can say is it failed.

    Robert

  • 0 in reply to Robert Adsett
    Guru 47900 points

    Robert Adsett said:
    You may even be able to get them to check the sensors to see if they can observe the likely cause. Some failure modes will leave particular signatures on the silicon. Of course sometimes all they can say is it failed.

    I'd say that poster's chances - in this regard - are directly proportional to his "documented" purchasing volume.   Commodity device - as one assumes these are - are unlikely to receive "deep probing" - unless client has great, "pull."

    In some cases - different grades of sensors are manufactured - and this may enable a direct "swap" which can "re-use" your existing board & software.   And will - most always - prove more costly.   (and sometimes demand minimal quantity purchase...)

    One notes a "duller/lesser" knife - wielded (this time) by poster/Friend Robert...   (Cb1 firm hound (almost) approaches)

  • 0 in reply to cb1
    Guru 47900 points
    I've - just now - noted that the humidity sensor is "SHT-21" and that - to my mind/experience - is a, "first cabin" device. (especially if it is other than "bottom of barrel/fire-sale" version.) Voltage transient (severe) rises in the opinion of this reporter!
  • 0 in reply to cb1
    Prodigy 160 points
    Hi cb1-

    What Robert pointed out in handling is a clear issue I did not consider previously. Thought the vehicles circuits are mostly sealed they are sometimes handled by marine scientist, which might not practice ESD prevention headlining. That is why I will consider Roberts advice of the copper mesh and also instruct the users of the vehicle on electronic handling.

    In terms of the sensors, I did no expect high performance from these sensors nor much robust. But I also didn't expect them to be short lived especially in a pressure and humidity controlled environment (like the vehicle's enclosure). These sensors are mainly to monitor the vehicles interior for leaks (humidity) and provide some idea of its dynamics as it descends the water column (acc/gyro/compass) .

    I will attempt to contact the sensor's makers to see, if they can shed some light on the problem. Thank you for your input cb1- and Robert, they are immensely appreciated.

    Jesus
  • 0 in reply to cb1
    Prodigy 160 points
    by "first cabin" device do you mean: a sensor very sensitive to its electrical environment and I should focus more on the voltage transients? or that it is in fact a good sensor that can handle a not so calm electrical environment?

    I am more inclined towards the first but I just want to make sure.
  • 0 in reply to Jesus Torrado
    Guru 27665 points
    Hmm, some time ago I was asked my opinion on a MEMS based pressure sensor (I suspect that's what you have). I read the data sheet and noted it was incompatible with water (and that seems fairly common for such sensors since the water affects the silicon).

    Given your use and environment before and after deployment, is there a chance you are seeing condensation related failures?

    Robert
  • 0 in reply to cb1
    Guru 27665 points

    cb1- said:
    I'd say that poster's chances - in this regard - are directly proportional to his "documented" purchasing volume.   Commodity device - as one assumes these are - are unlikely to receive "deep probing" - unless client has great, "pull."

    Yep, although on occasion I've actually had a rep volunteer such service w/o what I would have thought was sufficient volume.

    Robert

  • 0 in reply to Robert Adsett
    Prodigy 160 points
    Hi Robert

    We had one occasion of accidental condensation inside the vehicle's hull, where many failures, like the humidity sensor were recorded. Thought these failures were no investigated too deeply because of the hull failure and the occurrence shorts in the IC's that were clearly present (salty water).

    In terms of humidity levels during deployments, the air inside the hull of the vehicle is first sucked out passed through a desiccant and the hull is refilled with dryer air through a very small port until the pressure inside the hull is about 1 atmos. The process is repeated until a relative humidity of 20% to 25% is achieved inside the hull. This is to avoid condensation in colder waters during deployments, the humidity sensor is used to monitor this process and the humidity levels during deployments. Typical humidity levels observed range from 25% to 45%. This is of-course ruling out that occasion when we had a leak in the vehicle's where we recorded up to 93% humidity until the circuits failed.

    Hope this answers your question Robert, let me know if I misinterpreted it.

    Jesus
  • 0 in reply to Jesus Torrado
    Guru 27665 points
    Partly, sounds like a good procedure but note that humidity is insufficient to determine if there is condensation. All you need for condensation is a surface below the dewpoint and you have an excellent heat sink outside to bring the temperature of any surface in contact with it down. You may be setup to avoid that.

    Are you conformally coating your boards? That wouldn't protect the sensors from direct contact (if that's an issue) but it would prevent or reduce the chance of on board shorts. And dielectric grease to protect the connectors.

    Robert
  • 0 in reply to Jesus Torrado
    Guru 117855 points

    By, "first cabin" I meant that SHT21 is a well-known, serious device - designed & implemented to well-recognized standards.   Environmental standard "AEC-Q100, Rev "G" describes the standard to which the device was tested - and passed!   

    Specific to your needs that testing includes:

    • TC - temperature cycling: -50° - 125°C, 1000 cycles
    • ELFR - Early Life Failure Rate:  - 125°, 48 hrs
    • ESD Immunity: HBM (human body model) +/-4 kV,  CDM 750/500V (corner/other) pins

    This device is very well qualified - yet most (all) makers will not "guarantee" operation to spec. - w/in your (hostile) environment (as described).

    Maker recommended that 3V0 (3.0V) rather than 3V3 be the supply voltage for these devices.  

    Abs. Max Ratings:

    • VDD to VSS: Min: -0.3,  Max: 5.0V
    • I/O to VSS:    Min: -0.3,  Max: VDD + 0.3

    Minus your (recommended) data-logging - I'd hazard the guess that your Power OFF/ON cycling (likely) exceeded either VDD-VSS(max) or descended below the -0.3V specification.   It may be that the I/O pins are most susceptible.   (that's my belief from past working @ major semi firm...)