Fertility rates in the US are already the lowest they've been in 50 years, and have been on a downward trend ever since then:
http://www.pregnantpause.org/numbers/fertility.htm
http://en.wikipedia.org/wiki/Total_fertility_rate
It's the issue that China is just starting to grapple with now, and realizing that having that many fewer working age individuals supporting x number non working age individuals becomes an increasing burden to the point of unsustainability. We're not there yet in the US, but that doesn't mean we won't get there.