what would cost someone to have trouble remembering faces?

Depends on how bad the problem is. Is it poor memory or there is actually a condition for this

“Prosopagnosia (also known as face blindness or facial agnosia) is a neurological disorder characterized by the inability to recognize faces.”