A lot of things here aren't 'right', necessarily...

[He trails off, though he seems to mean it as a way to let her off the hook — an abortive gesture, and he sighs after her words. Seeing her even a little upset on his behalf... it bothers him, terribly.

His hand fidgets with the cross and amulet on the chain around his neck, for a moment.]

Can it not be swept under the rug if I am the one sweeping it?

[It's a half-hearted joke, but...]

... I imagine anything I could say won't make you feel much better about it.