They don't have to. But the photo ops always show them with a hand resting on a black book and the commentators always talk about the Bible.

In the private swearings-in, they also have the option - that's when Ellison chose the Koran and earned criticism.

I'd be curious to know how many people do it without thinking about what a strange contradiction that is.