Did the people hear the words of the revelation directly at Sinei?
At the end of Shemot 19, Moshe has gone down from the mountain to speak to the people. Then Shemot 20 begins with the decalogue, with God speaking the commandments. We're then told (20:15) that all the people witnessed the thunder, lightning, blare, and smoke, but it doesn't mention words. The people fall back and then in Shemot 20:16 they say:
“You speak to us,” they said to Moses, “and we will obey; but let not God speak to us, lest we die.”
A plain reading of this passage suggests that God spoke to everybody, with Moshe at the foot of the mountain with the people, and then the people asked him to be an intermediary. (In other words, 20:16 is a reaction to having heard the preceding commandments.) However, since I was a child I've understood the revelation to have happened with Moshe atop the mountain, and I've been unclear on whether the people below heard the actual words or just the "sound effects".
I have heard that there is a teaching (somewhere) that they did hear the decalogue, and every word (or every commandment?) blew them back some distance and they had to keep returning to the mountain. I've also heard (somewhere) that they only heard the first commandment, or perhaps only the first word of the first commandment.
According to our sources (please cite them), how much of the revelation did the people hear as words? If these sources also address my confusion about Moshe's location please include that, but my primary question is about what the people heard.