SN74ALS541: When 74ALS541 is not powered, can applying 3.4V (less than absolute maximum ratings) to output pin, will this cause any damage to this device?

The absolute maximum ratings allow applying a voltage up to 5.5 V to the output pins only when they are in the high-impedance state. The datasheet does not say that the outputs of an unpowered device are in the high-impedance state.

However, figure 23 of Input and Output Characteristics of Digital Integrated Circuits at 5-V Supply Voltage implies that ALS 3-state outputs are overvoltage tolerant even when active.

If you want to be sure, you have to measure whether the output sinks any current in this situation.