Brachos 15 – ברכות טו

Rav Yosef says that regarding the rest of the Torah (other than shema) everyone agrees that one is required to have his words audible to his ears.  Rashi explains this to mean all the other brachos.  How can Rav Yosef derive from a passuk a halacah of brachos when we know that brachos are merely of rabbinic origin (other than birchas hamazon)?