For most of us, when we buy our first house, having a mortgage or not isn’t really a decision we have to make.  Few Americans, even high-income professionals, can afford to pay cash for a house.  In fact, for many high-income, low-net-worth professionals like doctors just out of residency, it’s tough just to come up with a 20% down payment, leading us to look very carefully at “doctor loans.”  Later in life, however, almost all of us will have to wrestle with the question of when/if to pay down the mortgage.  There truly is no right answer to this question and it isn’t all that dissimilar to the “student loans vs invest” question most deal with shortly out of residency.  However, I’ve been thinking about some new ways to look at this decision.

Pre-pay the Mortgage When Your Risk Is Higher Than The Bank’s Risk

When you first take out a mortgage, especially a 0% down mortgage, the bank is the one taking most of the risk.  All you’re risking is your down payment (if any) and your credit score.  The bank could be left holding the bag if you become unable to make your payments or choose not to make them in the event of the house declining in value.  As time goes on, the bank takes on less and less risk.  For example, when you only owe $100K on a $500K home, the bank only has $100K at risk, and if you default, stands to make $400K on the deal.  Meanwhile, a foreclosure costs you $400K and only saves you $100K.  Why put $400K of home equity at risk for a relatively small benefit of carrying a $100K mortgage?

Mortgage Interest Deduction Becomes Less Valuable

One of the reasons many high-income professionals buy a home is so their monthly payment can become partially deductible.  As long as you itemize, mortgage interest and property taxes may be deductible on your taxes (although the Pease law reinstated in 2013 limits the deduction for those with an AGI greater than $250K ($300K married)).  However, as you pay down the mortgage, more and more of each payment goes toward principle.  This decreases the interest you have to pay, but also decreases the interest you get to deduct.  At a certain point, it may not be worth it to you.  In fact, it may even be possible that without a big mortgage interest payment that your itemized deductions are close to or less than the standard deduction, effectively raising your after-tax mortgage interest rate.  If you end up cutting back and decreasing your income (or retire completely), putting yourself into a lower marginal tax bracket, that interest deduction also becomes less valuable.   $10K of mortgage interest at a 40% marginal rate is worth $4K.  At 25%, it’s only worth $2500.

Paying Off The Mortgage Is Like Disability Insurance

One of my partners considers paying off his mortgage the equivalent of buying a disability insurance policy that would produce the same amount of income.  For example, to cover a mortgage payment of $3K a month with disability insurance would cost $1000-1500 per year in disability insurance premiums.  This is similar to raising the interest rate on your mortgage.

It’s Like Life Insurance Too

If you owe $400K on your mortgage, you probably need to have an extra $400K in life insurance to protect your family in the event of your death.  A 40 year old healthy male buying 30 year level term insurance would pay at least $500 a year for a $400K policy.  Let’s say you have 15 years left on a 5% mortgage and owe $400K on it.  Your payments are $3059, of which $1497 is principal and $1562 is interest.  If you also add in the cost of disability insurance ($1500 per year) and life insurance ($500 per year) that’s the same as increasing your interest rate from 5% to 5.53%.

Skeptics would argue that if you invest that money instead of paying down the mortgage then you should have that $400K (or possibly more) on the side in an investing account.  That may or may not be true.  First, you need the discipline to actually invest that money instead of spending it.  Second, you actually need to earn a return equivalent to or better than your after-tax mortgage interest rate.  Third, you have to pay investment expenses and taxes in order to liquidate the investment.  Paying down the mortgage is a guaranteed return equivalent to your after-tax interest rate plus the costs of the disability insurance and life insurance equivalent to that amount.  That guarantee is worth something.

Paying Off Your Mortgage Allows For A Smaller Nest Egg

Paying off your mortgage early is even more attractive while in or near retirement.  If a retiree can count on spending ~4% of his nest egg each year, then a $3000/month mortgage payment requires a nest egg of $900K, just to pay the mortgage!  Far better to use $400K to pay off the mortgage, and then use the remaining $500K to produce an income of $20K per year.  It’s easy to see that carrying a mortgage into retirement isn’t particularly wise, no matter how you feel about it during your working years.

Asset Protection Considerations

Asset protection laws are state-specific.  Most states protect your 401K and IRAs from creditors.  No states protect a taxable investing account.  Some states protect a significant amount of your home equity from creditors.  In Texas, where up to $1M of home equity is protected, there is an additional benefit to paying down your mortgage and “hiding your money in your house.”  In Virginia, where only $5K of home equity is protected, that might not be the best plan.

Behavior Vs Math

As you can see, there is more to the pre-pay mortgage decision than just weighing a psychological benefit of being debt-free with the ability to invest using low-interest, non-callable margin debt like a mortgage.  It’s an individual decision, but recognize that it is likely that there is a certain point in the term of your mortgage and in your life when it probably makes sense for you to pre-pay your mortgage, especially when the alternative is investing in a taxable account.

My Recommendations

I recommend you have your mortgage paid off by the time you wish to cut back on your workload or retire.  People with debt of any kind aren’t really ready for retirement in my opinion.  One of my financial goals is to be able to cut back on my clinical workload by age 50, so I plan to have my mortgage paid off by then.  Rather than just amortizing it over the next 12-13 years, at some point I imagine I’ll just pay off the last $20K, $50K, or even $100K very rapidly to reduce my risk and improve my cash flow.  In general, I think using money that would otherwise be invested in a taxable account to pay down student loan or mortgage debt is a fine move, but I would hesitate to give up significant tax breaks to reduce low-interest debt.  In other words, max out your 401K, Stealth IRA and Backdoor Roth IRAs before paying down your mortgage.

What do you think?  How do you plan to pay your mortgage off?  Comment below!