We formulate the induced potential in a finite temperature cold atomic medium between two heavy impurities, or polarons, which is shown to be complex-valued in general. The imaginary part of the complex-valued potential describes a decoherence effect, and thus, the resulting Schrödinger equation for the two polarons acquires a non-Hermitian term. We apply the developed formulation to two representative cases of polarons interacting with medium particles through the s-wave contact interaction: (i) the normal phase of single-component (i.e., spin-polarized) fermions using the fermionic field theory, and (ii) a superfluid phase using the superfluid effective field theory, which is valid either for a Bose-Einstein condensate (BEC) of a single-component Bose gas or for the BEC-BCS crossover in two-component fermions at a low-energy regime. Computing the leading-order term, the imaginary part of the potential in both cases is found to show a universal r^-2 behavior at long distance. We propose three experimental ways to observe the effects of the universal imaginary potential in cold atoms. [1] Y. Akamatsu et al., arXiv:2312.08241.