What is the understudy doing during a play? Are they backstage singing along/speaking along as the play progresses, or are they at home reading a book or something?

It probably depends on the production, but I believe they usually in the cast in a different role. For example the understudy for the lead performer might be a supporting actor, that actor has an understudy who is a supernumerary. That way everyone can shift up and around and cover a number of possible circumstances. It’s also common for the understudy to cover multiple roles and their understudies to cover multiple roles. It’s entirely likely the majority of the cast is trained in performing almost all the roles.