Oh, yes, from that point of view I agree with you. In my household we have five properties (so we're pikers compared to you haha) and I've never looked at them for income, though I'm always cash flow positive. I have never incorporated their income into my personal spending.
I believe it's better to borrow more on the rentals (because the interest payments reduce taxable income from the rent) and pay more into my personal residence or retirement accounts (ie: no or deferred tax). Also paying enough money into the rental to make the net monthly income significant means that return on equity is reduced from the point of view of house price appreciation; and a large part of the attraction of real estate is the ability to leverage at low interest (compared to say a margin account at a brokerage or a small business loan).