Q: Can I drive high impedance loads with a headphone amplifier?

A: It is ok to drive higher impedance loads and, in general, should give better performance than lower loads such as 16-ohms. Actually, this is a good opportunity to place an RC LPF to remove high frequency noise assuming that the load is of sufficiently high impedance that the gain error is negligible. As an added bonus, this also 'isolates' the internally compensated amplifiers from direct capacitve loads. See http://www.ti.com/litv/pdf/slaa313 for resistor/cap values.