But let's examine this. What are they really saying?
To read something literally means to take it word for word at face value. Like, "I am going to the store today" means, literally, "I am going to the store today."
But, "Go fly a kite" does not mean to literally go fly a kite, but is a figure of speech which means for you to go away.
Sometimes the Bible uses figures of speech. Sometimes it speaks plainly, or literally. And it's important to know which is which! When Jesus says you should hate your father and mother, he's obviously not speaking literally. But when it says he rose from the dead, it is!
Many Christians and sects get into trouble when they interpret passages from the Bible using an incorrect grammatical approach. Literal speech should be taken literally. Figurative figuratively. Apocalyptic, Poetic, Epistlary... all these forms of literature have their own rules for interpretation. Break the rules and you will get the meaning wrong.
This is, unforunately what has happened with many who teach all sorts of wacky things about the end times. They take literally what they shouldn't.
Likewise, many protestants take the literal words of Christ in a figurative or symbolic way, while "This is my body" really means it is, literally, somehow, his body!
Back to the original question. What do people mean when they say "I don't take the Bible literally"? They don't likely mean that they clearly distinguish between Scripture which is and isn't a figure of speech. I don't think this sort of statement often comes from a well-reasoned and researched view of the Bible. In my experience it's a tactic of argumentation meant to eliminate an entire line of debate.
I think what they really mean is, "I don't take the Bible seriously."