It seems the omap_hsmmc driver in TI kernel 2.6.37.6 is spending excessive time in the EDMA callback handler. Mounting an MMC/SD volume on the DM8148 EVM (no matter if ext3 or fat formatted) delays other EDMA handlers by up to 3 ms. This is a problem for EDMA based low latency audio or other EDMA based drivers.
Mounting an USB drive doesn't cause such delayed EDMA callbacks. So it doesn't seem to be a problem with globally disabled IRQ higher in the filesystem stack.