Is It Better To Pay Off My Mortgage Or Invest At Retirement?

When preparing to retire, one question that is on most people’s minds is, “Is it better to pay off my mortgage or invest at retirement?” A common financial goal for many people is to retire debt-free, but most people approaching retirement are still paying off their mortgage.

Before you do anything, it’s important to evaluate your financial situation, available options, and opportunity costs.

Paying off a mortgage early can provide invaluable peace of mind and improve your cash flow. However, you might get a better return on your money by investing in the stock market, your business, or real estate while enjoying tax benefits from your mortgage.

So what’s best? Let’s examine the costs and benefits of paying off your mortgage versus investing at retirement.

Does Paying Off My Mortgage Early Make Sense?

One of the most important factors is to understand where you get the best return on your money. You will want to compare different rates of return and the risks associated with each investment option.

It is also critical to understand the impact of taxes.

Taxes are at the core of every financial decision and need to be accounted for in your analysis.

Pay off mortgage or invest

Getting the Best Return on Your Savings

In the early years of a mortgage, payments are comprised primarily of interest with a little principal. Towards the end of the loan, the opposite is true. Your payments are mostly principal with a little bit of interest.

The reason for this is that towards the end of the loan, the mortgage balance is lower, so a smaller amount of interest is being charged to the mortgage balance.

Thus, any extra principal reduction in the early life of the loan will greatly reduce the overall interest cost, saving money. The higher the interest rate, the more you will save on the life of the loan.

However, for the better part of the last decade, a 30-year mortgage has been in the 3% range. Much lower than the high-interest rates in the 5-6% range we saw just 15 years ago.

The opposite is true when investing in a diversified portfolio over time. The benefit of compounded interest and growth is significant towards the latter years of the investment. Since 1926 the S & P 500 index has averaged a double-digit return.

Your retirement plan should account for other long-term growth options that can outpace the interest savings of paying off your mortgage early. One example could be to invest in rental property instead of paying off your primary mortgage.

If you’re a business owner, consider what you could achieve by expanding or enhancing your operations. Reinvesting in your business could achieve a much greater return than saving interest on your mortgage.

As you can see, it’s important to think about where your money works best for you. And that’s different for everyone.

The Math: Pay off My Mortgage or Invest

When looking at the numbers, the difference between paying off your mortgage or investing those funds in your investment portfolio can be startling. If you decided to pay off your mortgage rather than invest, you would have paid a high opportunity cost for the feeling of being debt-free.

Historically over longer periods of time (10+ years), you’ll rarely find a time when market interest rates are better than average annual returns from a diversified investment portfolio. 

The table below compares the total interest paid on a 30-year loan to an investment portfolio growing for 30 years. We use a balance of $500,000 in our example. The mortgage rate is 3%, while the growth rate of the investment portfolio is 6%. 

Pay off Mortgage or Invest

The first thing that jumps out is the total comparison of total interest paid on the mortgage vs. total interest earned after 30 years. The total amount of interest paid over the life of the loan is $258,887 while the total return from the investment portfolio is $2,371,746! That is a massive difference in total return.

Of course, the portfolio return is twice that of the mortgage, but these are real numbers based on recent economic and market conditions over the last 10 years. The primary reason for the disparity in the totals is compounding.

Over time, your mortgage balance gets reduced, resulting in you paying less interest. You will notice that interest paid steadily declines every year in the “Interest” column.

However, the payment stays the same. So that means that more of your payment is going to the principal in the later years of the loan as you can see in the “Principal” column.

Also, note the first column titled “Pmt No.” This refers to the monthly payment being made out of 360 total payments (30 years x 12 mos). When you are halfway through the loan at payment number 180, you have paid nearly 71% of the total interest over the entire loan.

This is another reason why it doesn’t make a lot of sense to pay off a mortgage towards the end. You aren’t saving yourself much interest. It can be more beneficial to just continue making payments.

The Benefits of Paying off Your Mortgage at Retirement

It feels good not to owe money, psychologically and emotionally. Many baby boomers share the goal of paying off their mortgage by retirement. Realistically, this doesn’t happen since people move frequently and refinance in their working years.

A Paid Off Mortgage Provides Peace of Mind

Removing what for many people is their largest expense can feel liberating. Your psychological well-being can improve by removing a debt overhang and owning your own home. You will have lower cash needs on a monthly basis, allowing you to spend or invest those dollars elsewhere.

Home equity is also a good source of low-cost debt to tap in order to pay off high-interest debt, like credit cards. It can also be used to help out children, grandchildren or free up capital for all sorts of other reasons.

Another benefit is that lower monthly expenses require a smaller emergency fund. It’s a rule of thumb to have 3-9 months of your monthly expenses in an emergency fund.

Lastly, you remove the financial risk an unexpected life event could pose by impairing your ability to make payments.

No Mortgage Payment Means Lower Retirement Account Withdrawals

You can reduce your withdrawal rate from your retirement savings in the absence of your mortgage payment. This can reduce your tax liability if most of your withdrawals are coming from IRAs and 401k’s. Distributions from these accounts are income taxable.

Since you would use your non-retirement accounts to pay off a mortgage, lower withdrawals can preserve your retirement accounts more. Your average annual return can be higher in the long run by keeping those funds invested over time.

Conversely, you could keep the same withdrawal rate and afford a higher quality of living. If your retirement budget is higher than your pre-retirement budget, that means you will need the extra cash from retirement accounts. A paid-off mortgage will again keep withdrawals more manageable.

Drawbacks of Paying off Your Mortgage at Retirement

As we explained above, the opportunity cost of mortgage interest saved versus interest earned can be quite shocking. We recently discussed this in a recent post titled, “How to Retire Early at 55.”

Paying off Your Mortgage Ties Up Your Money

A mortgage from your bank is a financial partnership. They provide you with capital and you pay interest in return. As long as you keep making mortgage payments, you are the owner of the property and receive all of the appreciation of the home.

So let’s say you decide to fire your banking partner because you don’t want to pay the interest or have a payment. The trade-off is that now the equity of the property is tied up. For you to access your equity, you need to take out a home equity line of credit (HELOC) at an unknown interest rate in the future.

With lower interest rates over the last 10 years, it makes sense to use the interest rates to your advantage and obtain a mortgage. As long as there isn’t a pre-payment penalty, you can always pay it off or refinance it in the future when you have extra money.

Home equity is also a good source of low-cost debt to tap in order to pay off high-interest debt, like credit cards.

Paying off Your Mortgage Can Result in Lost Tax Benefits

When you file your taxes every year, you choose to take a standard deduction or itemize your deductions – whichever amount is greater. The standard deduction for taxpayers that are Married Filing Jointly is $24,000 ($12,000 for single taxpayers) in 2021.

Every year, you add up your itemized deductions with the hope that they are higher than your standard deduction. Expenses that qualify as itemized deductions are broken down into the following categories:

  • Medical and Dental Expenses

    To the extent that they exceed 7.5% of your Adjusted Gross Income (AGI)

  • Taxes You Paid

    Includes State and Local Income Taxes (SALT)

  • Interest You Paid

    Interest/Points you paid on your mortgage and/or vacation home

  • Gifts to Charity

    Charitable gifts made in the current year including carryover disallowed gifts made in previous years

  • Casualty and Theft Losses

    Generally, you may deduct losses caused by a federally declared disaster or by theft

  • Other Itemized Deductions

    You can only claim certain unreimbursed employee expenses if you fall into a qualified category of employment

Since mortgage interest is tax-deductible, losing the interest rate deduction by paying off your mortgage can increase your tax liability. If you purchased a home prior to 2017 and itemize your deductions, interest is deductible on the first $1,000,000 of your mortgage. The Tax Cut and Jobs Act (TCJA) of 2017 reduced this amount to $750,000.

The Act also doubled the standard deduction and reduced the State and Local Taxes (SALT) deduction by imposing a $10,000 limit. This has had a negative impact on residents of states with higher income and property taxes like California and New York.

Let’s take a closer look at Schedule A below to explain how losing your mortgage interest deduction can increase your tax liability.

Schedule A Itemized Deductions and SALT

In our example, although the state income taxes are $46,500 and the real estate taxes are $20,000, the total allowable deductible amount is limited to $10,000. The limitation causes this high-income household to miss out on $56,508 of deductions!

Additionally, you can see the total deductible mortgage interest in the amount of $26,532. The mortgage balance in our example is $1.5 million. That means that this taxpayer actually paid about $53,000 in mortgage interest for the year.

Prior to the TCJA legislation that limited the SALT deduction, there was no limitation. So using our example, prior to 2017, the total itemized deductions would have been $93,040, not $36,532. There is currently a proposal in the Build Back Better legislation to increase the SALT cap.

The bill has passed the House but faces a tougher battle in the Senate. How do you use this information to your benefit? You may have itemized deductions like medical expenses or charitable giving that put you close to the standard deduction amount. 

Mortgage interest could increase your itemized deductions to the point they are now greater than the standard deduction. So it’s an important consideration, especially if the SALT limitation is increased or goes away completely.

Final Thoughts

Like any big financial decision, it is important to weigh the costs and benefits. Financial decisions like this are complex and require a good understanding of all the variables involved.

Use a nuanced approach, giving weight to what matters most to you. Can’t decide whether you want a mortgage or not? A balanced approach is another option. You could pay off part of your mortgage and invest the remainder in whatever split you’re most comfortable with.

Maybe you decide to finance only an amount that would allow all the interest to be deductible for you.

Every decision involves risks that must be measured and managed for any particular path you choose. If you aren’t confident in making these decisions on your own, consult a fee-only financial planner to ensure you don’t leave anything out.

The wealth management industry will undergo monumental shifts in the next ten years. Consumers today increasingly use Skype or FaceTime to discuss personal matters and connect with friends and family globally. Factors such as evolving new customer segments and rapid technological advancements are why virtual financial advisors are the new normal.

Large global wealth management firms are taking notice. More focus is being placed on virtual advice centers that attempt to deliver a high level of personal service from high-caliber advisors. This is in stark contrast to more traditional call centers with less personalized advice.

What Does it Mean to be a Virtual Financial Advisor?

A virtual financial advisor is someone who has either very limited or no face-to-face meetings with prospects and clients. Rather, the advisor utilizes video conferencing platforms and robust financial technology in their meetings with prospects and clients.

It does not necessarily mean that the advisor does not have a physical office. Virtual Financial Advisors can have their own physical office space or utilize virtual workspaces such as WeWork.

Less commuting and face-to-face meetings also mean fewer carbon emissions and paper waste. Virtual financial advisors help save you time, money and even help save the environment.

Photo Credit: Karolina Grabowska

Does Working With a Virtual Financial Advisor Save You Time?

Picture this…It’s 5:30 pm and you just got off of work. You and your spouse have a 6 pm meeting scheduled at your financial advisor’s office 15 miles away. You got stuck in a meeting that ran 10 minutes over.

You’re stuck in traffic, frantically trying to call your spouse because you are running 15 minutes late. The sweat builds up around your collar as you try to navigate through traffic to get there on time. You finally get there to find your spouse waiting patiently in the reception area.

You join her in the lobby, but you don’t really want to be there. All you really want is to be home in comfortable clothes spending some meaningful time with your family.

With a more personalized “Netflix-type” approach, meeting with a virtual advisor is a much more seamless process. They can share your financial plan and other online portals to better explain financial strategies and concepts to you through virtual conferencing software. You can also share your screen if you need help logging into your 401k portal or have a question about a financial document. And of course, you will save time by cutting down on commuting and sitting in traffic.

Does Working Virtually Save You Money?

The fees that financial advisors charge should be commensurate with the services provided. Does your advisor only provide investment management? Or do you work with a comprehensive fee-only financial planner that also provides comprehensive financial planning advice? As an example, a prospective retiree will need advice above and beyond investments because retirement planning is complicated. They have other problems they need solutions to:

  • What age can I retire?
  • How much can we spend monthly without outliving our assets?
  • What’s the best Social Security Strategy?
  • How do unforseen health events impact our plan?
  • What’s the best investment strategy for us?
  • How do we reduce taxes in retirement?

So the level of service and value provided is different amongst financial advisors. It’s important to understand your biggest needs to ensure you get the right answers to your questions. And that’s where a virtual financial advisor can really add a ton of value.

Does A Virtual Financial Advisor Provide Better Advice?

As consumers increasingly turn to web search engines for their shopping needs, more options continue to become available. Search engines like Google are constantly trying to improve results for consumer web searches, and the financial planning profession is no different.

Number of Households That Prefer a Virtual Financial Advisor

Gone are the days when you Googled “financial advisor near me” and picked a firm from the first page of search results. There are too many options, and in most major metro cities, there can be hundreds or thousands of financial advisors that will be eager to work with you. This is not an optimal way to find an advisor because you are casting too wide of a net.

The 3 Most Important Questions to Ask Your Financial Advisor

If you are looking for unbiased advice to solve your specific problems, the 3 most important criteria in selecting an advisor are:

  1. Are they fee-only? This means their fees are directly paid by clients and not from third parties where conflicts of interest could arise.
  2. Are they experienced and credible? You may be looking for an advisor that is a CFP® because you need more financial planning help. A minimum level of experience could also be a priority for you.
  3. What is their area of expertise? You wouldn’t go to an opthamologist if you had back pain and the same is true in selecting the right advisor.

I can say from experience that most consumers do not ask the third question, which is just as important as the first two. You want the best advice for your needs, so you need to find a financial partner that is familiar with your financial situation.

The global economy and the ever-changing U.S. Tax Code make it too difficult for advisors to excel at every single area of financial planning, so most advisors are generalists.

The good news is that virtual financial advisors are making it easier than ever for people to find an optimal financial partner. If you are open to a virtual relationship, your Google search results will yield you a greater number of advisors that SPECIALIZE in solving your problems.

Casting a wider, more intentional net allows you to not only catch more fish – but the fish you really want to catch!

Is Working With a Virtual Financial Advisor Safe?

Nearly every American has an account with a large public financial institution. Whether it’s your bank, credit card company, or 401k provider, our sensitive personal information is out there. And when it comes to our finances, safety and security are of the utmost importance.

The good news is that most databases that contain this type of information have 256-bit encryption. This encryption supports the largest bit size and is practically unbreakable by brute force based on current computing power, making it the strongest encryption standard. Nearly all financial technology software contains 256-bit encryption.

Percentage of People Comfortable Working with a Virtual Financial Advisor by Age

The other aspect of security is a physical breach of your confidential information. Most virtual financial advisors save your financial documentation in digital format, rarely keeping hard copies of your financial documentation laying around. This significantly reduces the likelihood that your information falls into the wrong hands.

Regardless of whether your financial advisor is virtual, you need them to act with diligence and integrity when it comes to your financial information. One thing you can do is run a FINRA background check on the advisor for any red flags. FINRA is a not-for-profit organization working under the supervision of the SEC. They actively engage with and provide essential tools for investors, member firms, and policymakers.

How Do I Find a Virtual Financial Advisor?

Performing a Google search of “Virtual Financial Advisor” will probably yield you more results than you care to get. We recommend adding a specialty or expertise that you are looking for in your advisor. Here are some common specialty keywords that could help you narrow down your search:

  • Millenial
  • Generation XY
  • Retirement
  • Business Planning
  • College Planning
  • Tax Planning
  • Federal Employee (FERS)
  • State Employee (PERS)
  • Stock Option
  • Real Estate

Don’t forget to add the keywords, “CFP, Certified Financial Planner, Fee-only” or other descriptive words. The goal is to continue to refine your search to produce a better list of qualified candidates. If you are looking for retirement planning help, then your Google search should look something like this…

Certified Financial Planner + Retirement Planning + Virtual Financial Advisor

Another way to find an advisor is through financial advisor trade and non-profit organizations. Most of these organizations are advocates for more transparency and higher standards for financial advisors.

This helps to ensure that advisors are always acting in good faith, always putting their client’s interests first. The Certified Financial Planner Board of Standards regulates the prestigious CFP® designation. They have a great database to find a financial planner. The National Association of Personal Financial Advisors (NAPFA) also has a great search tool to find an advisor by area of expertise.

Final Thoughts on Virtual Financial Advisors

Through rapid technological advancement, time is increasingly becoming a more valuable commodity. The information age has provided consumers with an abundant amount of options that provide convenience and save time. More consumers are streaming entertainment from home, having restaurants and groceries delivered, and even meeting with their doctors virtually.

Investors now have better access to advisors that specialize in providing solutions to their specific needs. If you live in a smaller city, the value add by working with a virtual financial advisor is even greater.

The convenience and time saved are the icing on the cake. The more time you save from commuting, waiting in a lobby, and finding parking, the more time you have to enjoy the things you love to do.

Lastly, working with a virtual financial advisor lessens our carbon footprint through less driving and unnecessary paper documents.