What You Need to Know to Retire Early at 55

Most Americans consider age 65 the normal age of retirement because it was the full retirement age for Social Security Benefits until 1983. But the goal of retiring early has gained widespread popularity in recent years. In this article, we’ll cover what you need to know to retire early at 55.

From formulating a budget to creating tax-efficient retirement income, there are a lot of factors and variables to consider prior to making such an important decision. Making major mistakes will result in lifestyle restrictions or a forced return to the labor force.

Retirement is your reward for a lifetime of hard work and saving money. So let’s talk about how to retire early – and stay retired.

Planning to Retire Early at 55

Knowing exactly how you will spend your time in retirement will be critical to your success. When every day is Saturday there is a high likelihood that you will be spending more money.

Your retirement account withdrawals need to be reasonable to ensure you don’t outlive your savings and investments. Inflation is another obstacle you need to overcome. Your retirement plan needs to account for the higher cost of goods and services in future years.

Thus, having a step-by-step plan to overcome these types of obstacles is necessary. 

How Much Do I Need to Retire Comfortably?

One common goal that everyone has is maximizing their quality of life in retirement. This means spending money confidently, without fear, worry, or anxiety. 

Traveling the world will require more financial resources than playing bridge or having coffee with friends every day. Consider traveling costs to visit your children and gifts to grandchildren. What does your bucket list look like? Make sure to create a good framework detailing all of your retirement goals.

Mapping out your daily, weekly, and monthly lifestyle goals will assist in determining what a comfortable retirement will feel like. Knowing your expenses is the first step in determining your withdrawal rate. Your withdrawal rate is the amount withdrawn divided by the value of your retirement portfolio.

How Will My Expenses Change in Retirement?

Most retirees don’t take time to create a monthly or annual budget. We recommend creating a pre-retirement budget as well as a retirement budget. This will force you to see how expenses will change, line item by line item.

Retiring at 55 can increase your expenses. Health insurance is one of the most important considerations when factoring in higher expenses.

Losing employer-sponsored health coverage is costly until Medicare eligibility kicks in. Paying for health care costs out of your own pocket is unsustainable. For married couples, deciding whether one spouse works longer to retain affordable health coverage is a conversation worth having.

Some families have children entering college when they plan to retire. This can increase expenses dramatically at the same time employment income disappears. Planning for this change in expenses is paramount to a sound financial plan.

Some things are beyond your control. It is important to focus on what you can control now.

What is the Tax Rate on My Retirement Income?

When creating retirement income, how much you get to keep after taxes is what really matters. It’s important to understand how income and capital gains tax rates work. Having a sound financial plan will help you determine how to withdraw the money you need from retirement accounts – while paying the least amount of tax.

Taxable, tax-deferred, and tax-free accounts are the three primary types of accounts you can withdraw from. Withdrawals from tax-deferred accounts like Traditional IRAs increase your ordinary income tax rate. Tax-free withdrawals from Roth IRAs are just that, tax-free.

2021 Ordinary Income and Capital Gain tax rates for Single and Married Filing Jointly Taxpayers

Non-retirement accounts are subject to short-term and long-term capital gains tax rates. Selling an appreciated asset held for less than one year is called a short-term capital gain. Short-term capital gains are taxed at your marginal income tax rate. Assets held for longer than one year are taxed at more favorable long-term capital gain tax rates.

Selling an asset that has gone down in value from when you purchased it results in a capital loss. You can use a capital loss to offset a capital gain. This can be beneficial in reducing your tax liability every year. So just knowing which position to sell in an account can have major tax implications for you.

How Much Money Do I Need to Retire Early at 55?

The key question that every retiree wants to know is how much they can spend in retirement without outliving their nest egg. The challenge with retiring at 55 is that the earliest you can begin Social Security Benefits is age 62.

While many retirees that work for a state or federal government have pensions, the earliest age you can begin benefits can range anywhere from 55 to 65. But drawing your pension at 55 can substantially reduce your benefit.

It may make more sense to rely on retirement account withdrawals and delay your pension start date. These are some of the difficult, yet important decisions that need thorough analysis to ensure that you maximize your income in retirement.

Beyond the Four Percent Rule: How Much Can You Spend in Retirement?

The “four percent rule” states that no more than 4.2% should be withdrawn annually (adjusted for inflation) to ensure you don’t outlive your retirement savings. However, this popular rule of thumb is based on a retirement age of 65. Retiring early at age 55 requires the withdrawal rate to be lower.

It really depends on a number of factors, but you may only be able to draw 2-3% in your 50’s, 4% in your 60’s, and 5% or greater in your 70’s. Having a retirement plan that addresses the timing of your retirement income benefits, retirement account withdrawals, and limits tax liabilities will help optimize your financial decisions.

How Do I Generate Income in Retirement?

The biggest challenge when you retire early at 55 is bridging the income gap until Social Security or Pension eligibility. For married couples, coordinating retirement dates with their spouse is important. Extending group health insurance coverage helps reduce expenses until Medicare eligibility. Another option is part-time work or consulting, depending on your goals and skills.

Many retirees pursue entrepreneurial passions they never had a chance to realize during their working years. A lifetime hobby or passion can now have space to blossom into a business and provide supplemental retirement income.

Real estate investors should evaluate their real estate portfolios as they approach retirement. The goal is to invest/retain properties that produce higher income in retirement.

Social Security Benefit Reductions for Early Retirement

Social Security eligibility begins when you reach age 62. But drawing benefits early will reduce your payouts by 20-30%. Delaying retirement benefits to age 70 can increase benefits by as much as 30%!

If you work while receiving benefits prior to reaching your full retirement age, the Social Security Administration will reduce your benefits depending on how much employment income you earn. For example, at full retirement age, they deduct $1 in benefits for every $3 you earn above $50,520. This matters if you plan to work again in one form or another. If you do, it might pay to delay Social Security even if you retire early.

You also want to ensure that you have worked for 40 quarters (10 years) to become eligible for Social Security Benefits. If you are short, it makes a lot of sense to postpone your early retirement at 55 and consider a later age.

How Do I Avoid an Early Withdrawal Penalty from my 401K or IRA?

Retirement accounts have early withdrawal penalties that are prohibitive by design. The intention of having tax-free and tax-deferred growth is to help Americans save more effectively for retirement. By nature, this helps alleviate reliance on programs such as Social Security.

Rules on retirement accounts restrict withdrawals for non-retirement-related purposes. However, there are some exceptions and strategies that can be extremely valuable for early retirees.

Roth IRA Taxes and Penalties

Retirement Account Early Withdrawal Penalties – The 59 ½ Rule

Withdrawals from retirement accounts are subject to a 10% withdrawal penalty prior to reaching age 59 and 1/2. This penalty is taken out of the distribution amount prior to assessing the amount of the taxable distribution.

Eligible withdrawals prior to age 59 1/2 are limited to the following exceptions:

  • First-time home purchases
  • Educational expenses
  • Birth or adoption related expenses
  • Disability
  • Death
  • Medical expenses

Remember that distributions from all tax-deferred accounts like IRA’s and 401K’s are taxed at ordinary income tax rates.

Using the Rule of 55 to take Early 401K Withdrawals

A great strategy for accessing money from retirement accounts when you retire at 55 is the Rule of 55. If you are laid off or retire early at 55, the IRS waives the 10% penalty for early distributions from 401k or 403b plans. For public service employees, the rule applies in the calendar year they reach age 50.

The IRS Rule of 55

Not all employer-sponsored retirement plans support the rule of 55 and some plans require that proceeds be taken in a lump sum.

It is advised that you check with your plan provider prior to making this important decision.

It’s also worth mentioning that the rule of 55 does not apply to old 401ks from previous employers or IRA accounts. It only applies to 401k or 403b plans with your current employer.

Can I Avoid Penalties for Early IRA Withdrawals?

Another strategy to avoid penalties for early retirement distributions is Rule 72(t). This rule allows penalty-free withdrawals from retirement accounts, with some caveats. Individuals must take five “substantially equal periodic payments” (SEPP).

The amount varies upon life expectancy calculations. The IRS has three approved methods of calculation. You must also adhere to the SEPP schedule for a minimum of five years or until the age of 59 1/2. Rule 72(t) applies to IRAs as well as 401k and 403b plans.

Are Roth IRA Distributions Subject to the Early Withdrawal Penalty?

You can always withdraw contributions from a Roth IRA with no penalty at any age. At age 59½, you can withdraw both contributions and earnings with no penalty, provided your Roth IRA has been open for at least five tax years.

This five-year rule governing Roth IRAs applies to three scenarios:

  • You withdraw earnings from your Roth IRA.
  • Convert a Traditional IRA to a Roth IRA.
  • Inherit a Roth IRA

Roth IRAs offer more accessibility than other retirement accounts, but also provide tax-free growth. It’s better to use them as a last resort if possible.

How to Invest When Retiring Early at 55

The asset allocation of your portfolio should become more conservative as you near retirement. This rationale is based upon time horizons. You can withstand volatility and recover from loss when you have a longer time horizon.

When you reach retirement you need your portfolio to provide retirement income. This means that you need some safety in your portfolio to fund these income needs.

Investing Safely for Early Retirement

Fixed income (bonds) are considered to be a “safer” asset class than stocks. Fixed income plays a major role in diversification and wealth preservation.

Historically, bonds have an inverse relationship with stocks. During market downturns when most stocks decrease in value, investment-grade bonds increase in value. This makes them ideal positions to liquidate if stocks are down to provide the income needed in retirement.

In addition to being a hedge against stock declines, bonds pay higher interest than cash. One mistake that some retirees make is having large cash positions in retirement. Cash historically does not keep pace with inflation and is a poor way to invest in the long-term.

Should I Still Invest in Stocks if I Retire Early at 55?

The average retirement represents a long time horizon anywhere from 20-30 years. But if you want to retire at 55, you need to plan for a timeframe of 40 years. This longer time frame is sufficient to absorb risk while participating in upside growth. Think of the stocks in your portfolio as the money you will spend 10 to 20 years down the road.

A well-diversified portfolio should include stocks from a number of different asset classes. You want to have exposure to small, medium, and large companies. Your portfolio should also hold international stocks in addition to U.S. stocks. Holding different types of asset classes in your portfolio spreads the risk around.

Important Strategies for your Retirement Investment Portfolio

Asset location (different from asset allocation) involves determining which accounts to house assets. For example, Roth IRAs provide tax-free distributions making them ideal for high-risk/return asset classes. You want to position low-risk/low-return asset classes in accounts with the highest tax rates like Traditional IRAs.

Rebalancing your portfolio is an important investment management strategy that should be performed consistently. Stock returns outpace bond returns in the long run. If you don’t rebalance your portfolio, the percentage of stocks in your portfolio will continue to increase, making it riskier than you originally intended.

Tax-loss harvesting is another technique to reduce your taxable income. In taxable accounts, selling an asset at a loss and purchasing another will create a realized loss. These losses can then be used to offset gains in the current year or carried forward indefinitely.

What Else to Consider Before You Retire Early at 55

As we have discussed, there are a number of things to contemplate when deciding to retire early. Forming a retirement budget, minimizing taxes, and navigating the complexity of retirement account rules are primary considerations.

Since everyone’s situation is different, there will always be unique situations that require more difficult decisions to be made. Now let’s take a look at some of these situations and some unforeseen risks that could throw a wrench in your plans.

Paying Off Your Mortgage Early vs. Investing: Which Is Best?

Many baby boomers make it a goal to be debt-free by the time they retire at 55. While that may feel like a great decision psychologically, it isn’t necessarily the best financial decision. Leveraging “good debt” like mortgages can really help you out financially. Over the last 20 years, mortgage rates have hovered around 3-4%.

From 1926-2020, the average return of portfolio invested 60% in stocks and 40% in bonds averages over 9% per year. It’s easy to see why utilizing a mortgage and letting your money grow in your investment portfolio is a good long-term strategy. Having a mortgage may make you feel uncomfortable, but changing your perspective about carrying debt can make all the difference in retirement.

Pay off mortgage or invest in stocks?

Mortgages aren’t the only type of liability retirees have to deal with upon retirement. Some retirees also have children entering college at the same time they retire. In the absence of a 529 plan or college fund, this added expense can cause a significant strain on cash flows. These decisions are integral to your success if you want to retire at 55.

The Five Most Common Retirement Risks You Should Know

There are five main risks every prospective retiree should plan for prior to retirement. Neglecting these risks could inhibit you from your goal to retire early at 55.

Market Risk

A decline in asset values similar to the Great Recession of 2008 is a good example of market risk. At the same time, you can’t be too conservative in retirement. Having a well-diversified portfolio that takes into account your retirement income needs is key.

Longevity and Mortality Risk

The risk of outliving your assets is what every retiree fears. Another risk related to longevity is premature death. This can impact your spouse and any other dependants in your household.

These risks can dictate whether to purchase/retain any life insurance policies. They may also assist you in making decisions like when to take Social Security benefits and pensions.

Health Risk

More trips to the doctor should be accounted for in retirement. We already discussed the higher cost of purchasing your own healthcare before you become eligible for Medicare. But you also have to account for more frequent doctor visits, co-pays, and deductibles.

Health risks increase as you grow older so make sure this is planned for in your retirement expenses. These costs should be higher in your budget until Medicare eligibility at age 65.

Event Risk

There are some events that could have low odds of occurring, but you still want to be prepared in case they do. A long-term care event such as Alzheimer’s could require round-the-clock care for years. Self-insuring this risk is cost-prohibitive for most people. Regular health insurance will not provide coverage against this event.

Relying on an inheritance without 100% certainty could be another example. An earthquake in California could devastate most retirees as only 13% of California homeowners have earthquake insurance.

We can’t predict what curveballs life will throw next. At the very least, you will want to know how certain types of these events will impact you financially.

Tax and Policy Risk

One thing that will be consistent during your retirement is frequent legislative changes. Changes to retirement account rules and the tax code can change year to year. One example is the Tax Cuts and Jobs Reconciliation Act (TCJA) bill that was passed in 2018. In 2020, we saw the SECURE and CARES Acts passed, which had a dramatic impact on retirement distribution rules.

Making sure you have a short and long-term tax plan may be the most important factor in maximizing retirement income. Once a plan is in place, it’s important to keep on top of the policy and tax changes every year and adjust your retirement plan accordingly.

Should I Hire a Financial Advisor or Do it Myself?

We covered a lot up to this point and you should have a pretty good idea of what it takes to retire early at 55. But here’s the million-dollar question:

Do I have the expertise to understand, implement, and monitor every facet of my finances in retirement – and do I really want to?

When making this decision, be honest with yourself and think about the repercussions of making mistakes. Keep in mind that the long time horizon in retirement compounds any mistakes you make. A large amount of personal finance knowledge and expertise is needed, which can increase your margin for error.

Will you really spend the time to keep up on changes to the economic, legislative, and tax landscapes? Is this something you are really interested in doing? If the answer is no, then it probably makes sense to consult with a few financial planners. At the very least, you will be able to get a good sense of the planning work that’s needed after speaking with them.

A Certified Financial Planner™ (CFP®) can help you custom tailor a comprehensive plan that helps you accomplish all of your goals. Above all, a CFP® will provide invaluable peace of mind when things change. Remember that your retirement plan will need to be adjusted on an ongoing basis.

It pays to have an experienced guide on the most important journey of your life.

Final Thoughts on Retiring Early at 55

Early retirement at age 55 is an idea that continues to grow in popularity. Ultimately, it means accumulating enough financial resources to live off for another 5-10 years. A successful early retirement plan hinges upon organization and preparation and is the blueprint to accomplishing your goals.

Making adjustments to your plan on a year-to-year basis is equally as important as formulating a good initial plan. There will be constant change and unpredictability with a number of things that are out of your control. How you react to these changes will play a big part in successfully retiring early at 55.

Nonqualified Deferred Compensation Plans

A deferred compensation plan is a retirement plan that allows employees to defer some of their compensation to a later date. Common types you may already be familiar with are 401(k) and 403(b) plans. If you are a key employee, however, your employer may offer one or more nonqualified deferred compensation plans (NQDC’s).

NQDC plans are governed by section 409A of the U.S. tax code and differ from most other employer-sponsored retirement plans. Whether or not you should participate in an NQDC requires some careful consideration. In this article, we explain nonqualified deferred compensation plans and whether one is right for you.

What are Deferred Compensation Plans?

First, it’s important to understand the differences between qualified and nonqualified deferred compensation plans. The Employee Retirement Income Security Act (ERISA) governs qualified plans. This legislation sets standards for health and retirement plans offered by employers to protect employees.

An NQDC plan is exempt from ERISA law’s strict rules, making them more flexible and customizable for employees. The primary purpose is to attract and retain highly compensated key employees and executives.

Qualified Deferred Compensation Plans

ERISA law ensures that qualified plans have minimum standards for vesting, eligibility, benefit accrual, funding, and the overall administration of plans. It also guarantees certain benefits for participants through the Pension Benefit Guaranty Corporation (PBGC).

Participants in a qualified plan are also subject to strict parameters that govern the plan. Required minimum distributions, annual contribution limits, and penalties for early withdrawal apply in most scenarios. The complex rules of qualified plans position the employer as a fiduciary on behalf of the plan participants.

The table below lists other common characteristics along with the different types of qualified plans.

Common types and characteristics of qualified plans

The single most important trait of qualified plans is that employee and vested employer contributions are assets belonging to the participant. This is the significant distinguishing characteristic between qualified and nonqualified deferred compensation plans.

Nonqualified Deferred Compensation Plans

A nonqualified deferred compensation plan or 409A plan, allows participants to defer some of their compensation into the future. For key employees and executives, this can be a very attractive benefit. Top-heavy and non-discrimination requirements limit key employees’ participation in qualified plans.

Additionally, highly compensated employees are subject to annual contribution limits of only $19,500 in a qualified plan. The ability to defer larger amounts of compensation into a nonqualified plan is a very attractive benefit to high-income earners.

Like most endeavors involving money, these plans do not come without risk. The compensation you defer into future years is considered an asset of the employer. This means that your NQDC plan would be at substantial risk if the company you work for becomes financially unstable.

The Benefits of Nonqualified Deferred Compensation Plans

As we just mentioned, the primary reason to participate in an NQDC is to minimize tax liabilities. Reducing compensation in the current year helps keep you out of higher marginal tax brackets.

And taking distributions in retirement years when tax rates are lower is the objective with retirement planning.

A Great Tool to Retain and Attract Key Employees

Employers want to attract and retain highly skilled executive talent. A nonqualified deferred compensation plan can be a key component of an executive benefits package. Even when the company has a qualified plan in place like a 401(k), an executive’s contribution amount faces more hurdles.

Percent of Fortune 1000 companies that provide non-qualified deferred compensation plans to key employees

The annual contribution limit for employee deferrals to a qualified deferred compensation plan is $19,500 in 2021.

There are rules in place to prevent a plan from benefiting higher-income employees over rank and file employees.

These rules limit contributions of high-income earning employees to a lesser amount.

The qualified annual contribution limit of $19,500 is less than ideal for top earners. NQDC plans don’t have contribution limits, allowing employees the option to defer a large percentage of their salary.

For example, let’s say you accept a job offer that pays a salary of $450,000. Even without any other household income, this would place you in a 32% federal marginal tax bracket.

In reviewing your benefits package, you find out that you can defer $100,000 of your salary every year. This would save you $32,000 of tax liability!

Tax-advantaged Compound Growth

There are different types of nonqualified deferred compensation plans offering various investment options (more on that later). Most plans offer stocks, bonds, and mutual funds. This can provide substantial long-term growth in addition to tax savings.

Let’s look at an example of how powerful compound tax-deferred growth is in accumulating wealth. Following our previous example, let’s assume an employee in a 32% marginal tax bracket defers $100,000 into their NQDC. The tax savings in year one would be $32,000.

Assuming the plan’s investments grow at a rate of 7% over 10 years, the $32,000 grows into $62,949! So instead of paying $32,000 to the IRS, you created nearly $100,000 of additional wealth in retirement.

Spreading Income Tax Liability into Retirement Years

Once retirement begins, household income usually decreases, which means that tax liabilities decrease as well. So it’s extremely important to have a good idea of what your income tax situation looks like upon retirement.

Coordinating distributions with other sources of retirement income is crucial for achieving tax efficiency. Deciding when to take Social Security and pension benefits should be coordinated with scheduled distributions.

The same goes for coordinating retirement dates with your spouse and accounting for Required Minimum Distributions (RMD’s). Most of us are in the highest tax brackets in our working years. So deferring taxable income to your retirement years can make a ton of sense.

The Risks of Nonqualified Deferred Compensation Plans

Funds are Assets of the Employer

Unlike ERISA plans, salary deferrals into an NQDC plan are not assets of the employee, but assets of the company. In a nutshell, your deferrals become a promissory note between you (the borrower) and your company (the lender).

Since your deferrals are assets of the company, should they become financially insolvent, your funds would be at risk. This is by far the most important consideration when choosing how much to defer into your deferred compensation NQDC plan.

There May Be Limitations of Investment Options

Aside from stock option plans, most nonqualified deferred compensation plans will have an approved list of mutual funds to choose from. This mutual fund lineup is often the same as in the company’s qualified plan like a 401k.

In the case of a stock option plan, your deferrals purchase stock in the company. This creates additional risk if your company encounters financial trouble. Even if they remain solvent, a concentrated position in the company stock could experience large declines.

Distribution Options are Limited

The year prior to deferring money into your plan, you need to decide when distributions will begin. Plans can offer a lump sum or multi-year payout options, but 5-10 years is a common election. Distributions don’t start until a triggering event occurs. There are six triggering events:

  1. A fixed date
  2. Separation from service (usually retirement)
  3. A change in ownership of control of the company
  4. Disability
  5. Death
  6. An unforeseen emergency

Distributions are difficult to change once selected and require a five-year waiting period if allowed at all.

How Nonqualified Deferred Compensation Plans Work

In some cases, the triggers for deferred comp distribution are beyond your control. For example, death or disability will force you (or your heirs) to take distributions.

Choosing a fixed date can be useful when anticipating future income needs as well. A good example is having a child enter college around the same time you retire. Scheduling distributions when the child begins school can alleviate pressure to draw funds from other sources.

No Loans in Nonqualified Deferred Compensation Plans

If you really needed to access money from your 401(k) plan, many plans offer loans. This is not the case with an NQDC plan. Section 409A of the U.S. tax code governs these plans, subjecting withdrawals only to the six triggering events above.

This means that you should consider deferrals into the plan to be irreversible. So making sure you don’t overcontribute to the plan is important. It’s helpful to create a budget including annual income tax liability to avoid an income shortfall at end of the year.

Is a Nonqualified Deferred Compensation Plan Right for Me?

Do I Need Another Retirement Savings Account?

The first place to maximize retirement contributions is qualified plans like 401k’s and IRA’s. But few options reduce taxes and provide tax-deferred growth like an NQDC. Also, there aren’t any other types of deferred compensation plans with unlimited contribution limits.

Thus the biggest advantage of deferred compensation plans is a tax-advantaged way to save annual income surplus.

How Financially Secure is the Company I Work For?

Again, since NQDC plans are assets of the employer, the financial strength of your employer is of paramount importance. Your company needs to be around in the long run for you to receive distributions from the plan. The longer you are away from your triggering event, the more risk you carry.

Concentration risk

Another consideration you want to make is how much of your wealth you tie to your employer. Do you already have other types of nonqualified deferred compensation plans? Do you already own a high concentration in company stock?

Too much concentration can be dangerous, as we saw with big companies like Chrysler in 2008 and Hertz in 2020. Having a significant amount of your wealth AND income tied to your employer can be extremely risky. If you feel comfortable with a high concentration of company stock, there are ways to obtain third-party insurance.

Key Considerations in Participating to a NQDC Plan

How Will My Tax Rate Change in the Future?

Tax savings are amongst the primary benefits of a nonqualified plan. It’s important to weigh your future income needs and tax liability when electing to participate in your plan. Accounting for all your sources of income such as Social Security and RMD’s is vital upon retirement.

Remember that your contributions are invested into a portfolio with an expected rate of return. If you are ten years away from retirement, you should estimate your contributions’ value in year ten.

This is because all distributions are income taxable. You must be careful not to defer too much and create a future tax problem. It’s recommended to consult with a financial planner or tax advisor before you make your elections since they can’t be changed.

What are My Distribution Options?

We have discussed how important it is to coordinate your NQDC distributions with your other retirement income sources. The big challenge is choosing the distribution schedule at the same time as the deferral election.

Let’s go through an example of how this works. It’s September and you receive your enrollment package to make your deferral for the following year. Your salary is $300,000 and you anticipate your bonus will be $100,000 next year.

You can defer up to 50% of your salary and 100% of your bonus. You decide to defer 33% of salary and 100% of bonus to your retirement date 10 years down the road. Your distribution options are to take a lump sum or payment over 10 years.

You must decide on which distribution to take, even though it won’t start for 10 years. To make a good decision, you will need to coordinate distributions with your future income sources. This takes some serious financial planning.

The final step is to choose how your deferral will be invested. The performance of the investments you choose will determine what the final amount will be upon retirement. As you can see, there are a lot of variables to consider when making this decision.

Key Takewaways

Nonqualified deferred compensation NQDC plans are a great tool for both employers and employees. Employers have added ways to enhance benefits packages for employees. And employees have another way to save in addition to their qualified retirement plan.

Employees need to weigh the benefits, risks, and whether an NQDC fits into their financial plan. They are a great way to reduce taxes and boost tax-deferred growth for retirement. But remember that the funds in these plans belong to the employer until the employee actually receives them.

These plans are very complex and should be discussed with your financial advisor prior to participating.

Retirement Tax Planning - 6 Things You Must Know

Tax planning strategies are an essential component to maximizing retirement income. And the more income you have, the more you get to spend or pass down to your heirs.

There are a lot of retirement tax planning articles out there. But our goal is to provide more of a “how-to” so you aren’t just relying on someone else’s word. So we’re going to explain the importance of retirement tax planning – 6 things you must know.

Anytime I am at a social gathering and people find out I’m a financial planner, the response is always an inquisitive, “Oh?” I can see them scrambling through their brain trying to think of a financial question for me. As you can guess, I’ve had some really interesting conversations with strangers. Some of the most common questions I get are:

What will my tax rate be when I retire?

Will my Social Security Benefits be taxed?

How can I plan ahead for retirement?

You are going to get answers to these and many other questions by the end of this article.

How Much Income is Needed in Retirement?

Whether you work with a financial planner or not, a successful plan starts with you. You and only you have to decide on what your life in retirement will be.

If you’re anything like me, you are going to want to know how often you can dine out. Or how many weeks you can live abroad every year. Or….maybe you just want to order that $4 side of guacamole and without feeling guilty.

The point is that the first step is to figure out how much we need to spend. And that is unique to each of us.

Creating Your Personal Financial Statements

So now that you know what you need to spend in retirement, it’s time to figure out how to make it happen. You are going to want to know how much you need to withdraw every year…and where to draw it from.

Your balance sheet is the first financial statement you will need. It contains a list of your assets and liabilities in separate columns. The difference between these amounts is your net worth.

Then you will list your annual income and expenses on your personal income statement. The difference between these amounts is your income shortfall.

Think of your net worth like your water well and your income shortfall as the water. You have to make sure there is always enough water in the well or you’ll go thirsty. Or in our case, you risk outliving your assets.

Income Needs Will Increase Over Time

One of the biggest obstacles for retirees is inflation. Every year the cost of goods and services increases. Luckily, Social Security benefits have a cost of living adjustment (COLA) to help keep pace.

But if you have a private pension, there may not be any COLA included. That means that your income will lose purchasing power over time.

As you can see from the chart, inflation has averaged over 2% over the last 20 years. It’s a lower rate than the historical average of 3.1% since 1913. But it still takes a big bite out of your purchasing power.

The Impact of Inflation Over Time

Something that cost $100 in 1913 would cost $2,555 now! That’s why it’s important to know how much you need to spend now and in the future.

How Much of Your Income Will be Taxable?

The good news is that it’s all up to you. The bad news…um yeah, it’s all up to you. You have nearly full control of your income decisions in retirement. The goal is clear: to maximize your income while paying the least amount of tax.

The first thing you need to know is how our tax system works. You’re going to have to know Social Security and retirement account rules. There will also be plenty to know about tax-efficient investing.

But that’s why you’re here in the first place, isn’t it? So let’s start with some retirement tax planning basics: how taxes in retirement work.

How Taxes are Calculated

Ordinary Income Tax Rates

Our federal income tax system is progressive. That means that the percentage of tax increases along with income levels. Ordinary income can consist of wages, salaries, and interest. But more concerning for retirees are taxable Social Security benefits and IRA withdrawals.

We all pay federal taxes starting at a rate of 10%. The federal marginal tax brackets go all the way up to 37%. If you look at your most recent tax return, line 10 on schedule 1040 gives you your taxable income. Looking at the table, you can see which marginal tax bracket that number puts you in.

Any extra dollar that’s earned will be taxed at your marginal tax rate. As an example, let’s say a single-filing taxpayer has taxable income for the year of $40,000. They are close to going from the 12% marginal tax bracket to the 22% bracket. But their bank account is running low and they need another $30,000 for living expenses.

If they take the $30,000 needed from an IRA account, the entire amount counts as ordinary income. That means they pay federal income tax at a marginal tax rate of 22%. Keep in mind that this does not include state income taxes (depending on your state of residence).

If you withdraw the $30,000 from a Roth IRA or taxable account you could avoid the extra tax bill. So you need to think twice about where your income will come from each year.

2021 Ordinary Income and Capital Gain Tax Rates for Single and Married Filing Jointly Taxpayers

Capital Gain Tax Rates

Favorable capital gain tax rates are a key retirement tax planning tool. When you sell a capital asset for more than what you paid for, the result is a capital gain. Capital assets include stocks, bonds, real estate, precious metals, cryptocurrency, etc.

If you sell a stock for a profit and hold it for one year it is taxed as a long-term capital gain (LTCG) as shown in the table. You can see that these rates are always lower than your marginal tax rates.

If you sell a capital asset for a gain that was held for less than a year, the gain is taxed as ordinary income. This is called a short-term capital gain (STCG) and will be subject to your federal marginal tax rate.

What if you sold a capital asset at a loss? In that case, here is how you would treat the loss:

List all your realized short or long-term losses

Offset short-term gains with short-term losses

Offset long-term gains with long-term losses

Finally, compare your net long-term gain/loss vs. your net short-term gain/loss

Here is how the results are treated:

A Net LTCG – taxed at long-term capital gain rates

A Net STCG – taxed at your marginal income tax rate

And if there is a net loss? Whether short or long-term, you can use $3,000 of the loss as a deduction in the current tax year. The remaining losses carry forward indefinitely. They can be used to offset gains in future years. In years with no gains, you can always use up to $3,000 of your carryforward losses as a deduction against your ordinary income. 

Withdrawals from Retirement Accounts

Retirement account withdrawals are where you can save BIG on taxes in retirement. This is where all that prudent saving you did in your working years pays off. Without an income distribution plan, you can kiss a lot of those savings goodbye to taxes. But not you. You are going to use your newly acquired retirement tax planning skills to turn the tables in your favor.

Taxable Accounts

Taxable accounts are any non-retirement accounts. Think of your bank or brokerage accounts or that stock certificate your granny gave you as a kid. For married couples, these accounts are usually jointly owned or in the name of their trust.

The capital gain rates that we discussed earlier apply to assets owned in these accounts. In your taxable account is where you need to pay attention to the cost basis and holding periods. 

Qualified dividends from stocks held in a taxable account are also taxed at lower capital gain rates. But interest income held in a taxable account will be treated as ordinary income, along with STCG’s.

As you can see, a sale of an asset here can be subject to two different tax rates. As can the income an asset produces, dividends and interest. And with capital loss rules at your disposal, you can save yourself a lot of money with a good investment strategy. More on that later… 

Tax-deferred Accounts

Tax-deferred accounts are the most common types of retirement accounts. Think of 401k’s, 403b’s, IRA’s, and pensions. Every time you contributed to these accounts, that money avoided taxation.

There should be a lot of growth from compounding if you have invested it well. But you see that number at the top of your statement that says “Account Value?” That’s not all yours. No, you have a silent partner named the IRS that is waiting for your distributions to begin.

And when they begin, they are taxed at the highest rates. So you want to be extra careful when making withdrawals or rollovers from these accounts.

Tax Diversification: Tax-Free, Taxable and Tax-deferred

Tax-free Accounts

Roth IRA’s and Roth 401k’s are the only retirement accounts that provide tax-free distributions. Roth accounts are funded with after-tax contributions, so no immediate tax benefit is received. in exchange for that immediate benefit, all the growth in the account is tax-free.

A Roth is a great account to have in retirement because they allow you to spend more without climbing into a higher marginal bracket. So, when you find out you need a new roof, you won’t have an extra tax bill to make things worse.

Every year you should calculate your income needs and tax bracket and then choose the most efficient withdrawal strategy. When your nest egg consists of all three types of accounts, you have what we call tax diversification. It is the foundation of your retirement tax planning strategy.

Will Your Social Security Benefits be Taxed?

Social Security is a big part of most American’s retirement income. It is also very complex and takes some careful analysis to ensure you are making the best decision. Next, we’re going to focus on three tax-related factors that impact Social Security Benefits.

When You Should Take Social Security Benefits

Take a look at your most recent Social Security Benefit statement or view it by logging in to your account. Find where it states your benefit at full retirement age (FRA). This is the age that you can begin receiving your full benefit. On your statement, you can also see a reduced benefit if you take it at age 62. You can also delay to age 70 for a higher benefit.

Take it at your earliest age and you face a reduction in benefits of over 25%. But delaying it allows it to grow at about 8% per year. There aren’t many things you can invest in and get a guaranteed 8% return! However, when you should take it depends on how much you have saved as well as health factors. And of course, how much tax you will pay on your benefit. 

Taking Social Security While Working

This is quite fine if you start your benefits at your FRA. But if you are under full retirement age for the entire year, your benefit is reduced by $1 for every $2 you earn above the annual limit.

For 2021, that limit is $18,960. In the year you reach full retirement age, your benefits are reduced by $1 for every $3 you earn above a different limit.

In 2021, the limit on your earnings is $50,520. Your earnings are counted up to the month before you reach your full retirement age, not your earnings for the entire year. 

How Your Social Security is Taxed

Your Social Security income is either 0%, 50%, or 85% subject to taxation. In other words, if your benefit was $30,000 and 50% was taxable, then $15,000 would be added to your ordinary income. Subject to – you guessed it, your marginal tax rate. The key factor in determining how much is taxable is your other sources of income as you can see from the formula below.

Social Security Taxation Formula

Since half of your Social Security benefits are included in the calculation, most of us will pay tax on about 85 percent of benefits. Here are the Internal Revenue Service (IRS) rules. If you:

file a federal tax return as an “individual” and your combined income is between $25,000 and $34,000, you may have to pay income tax on up to 50 percent of your benefits. More than $34,000, up to 85 percent of your benefits may be taxable.

file a joint return, and you and your spouse have a combined income that is between $32,000 and $44,000, you may have to pay income tax on up to 50 percent of your benefits. More than $44,000, up to 85 percent of your benefits may be taxable.

are married and file a separate tax return, you will probably pay taxes on your benefits. In summary, the timing of your Social Security benefits is a huge part of controlling taxes in retirement. To oversimplify, you want to keep as much of your benefit in your pocket.

There is a lot of thought that goes into selecting the right Social Security strategy. It’s a balancing act between maximizing your benefit and protecting it from taxes. Social Security and pension benefits must always factor into your retirement tax planning decisions.

Retirement Tax Planning Strategies

You should now have a good idea of how our tax system works and how to create retirement income. You now have some great tools to reduce your tax bill! But there is still so much more you can do…

Plan for RMD’s

Employer-sponsored retirement plans are the primary retirement savings vehicles for retirees. One reason is that the contribution limit is much higher than that of IRA’s. In 2021, the contribution limit for a defined contribution plan is $19,000 per year. In contrast, the annual IRA contribution limit is only $6,000.

Uniform Life Expectancy or RMD Table for 2021 and 2022

These accounts are where most Americans have saved their nest egg. And as we mentioned earlier, the IRS wants to collect its share of the pie. So, at age 72, you must begin taking Required Minimum Distributions (RMD’s) from these tax-deferred accounts. And the penalty for not taking your RMD is 50% of the amount!

A big mistake that many people make is not planning ahead for RMD’s. When you add that income to your other retirement income sources, it can easily put you in a higher marginal bracket.

It typically is not a good strategy to avoid distributions from your tax-deferred accounts. It may make sense to take some out during years when your tax bracket is extremely low. This ensures the income is taxed at lower rates.

Convert to a Roth IRA

Another great retirement planning strategy is Roth Conversions. It consists of transferring money from your Traditional IRA to your Roth IRA. The entire amount of the conversion gets treated as ordinary income.

But the benefit here is converting at tax rates that you know are less than or equal to future tax rates. If you retire at 60, you probably do not have any fixed income sources. This is when your marginal tax rates are usually the lowest. It may make sense to pay tax now and allow for tax-free growth going forward.

Roth Conversions also reduce your tax-deferred balance, which lowers future RMD’s. Lastly, they help your wealth transfer strategy since beneficiary withdrawals are also tax-free.

Contributions to Retirement Plans

It’s very common for one spouse to continue working after the other retires. This can present the opportunity to contribute to their 401k or other employer plans.

They could also be eligible to make Traditional or Roth IRA contributions for themselves and their retired spouse. Yet another tool to lower your income or contribute more to tax-free accounts.

Charitable Gifting in Retirement

There are several ways to give to charities and reduce taxes. You can give away your RMD through a qualified charitable distribution (QCD). You have to be age 72, but you won’t have to report RMD income using this strategy.

Gifting highly appreciated stock is another common tax reduction strategy.
It allows the taxpayer to avoid paying capital gain tax.
A donor-advised fund is another way taxpayers can give charitably and get a tax break.

It is like a charitable investment account that supports charities of your choosing. There are several more charitable gifting strategies for investors that donate to charities. 

Invest Tax-efficiently

Earlier, we stated that income from taxable accounts can be taxed at ordinary income or capital gain tax rates. There are a lot of portfolio management strategies you can use to control taxes. If you work with a financial advisor, don’t be shy in asking how they are managing your portfolio. 

Tax-efficient Investment Vehicles

Mutual funds are professionally managed investment vehicles that provide market exposure. There is a fund today for any asset class you can think of – like the Proshares Pet Care ETF (PAWZ).

But when it comes to selecting funds, you first need to know if you are investing in an active or passive fund. Not only are they higher in cost, but most active funds are not tax efficient. The manager is buying and selling positions throughout the year.

This turnover can generate capital gains that you have to pay tax on. As a matter of fact, your fund could have an unrealized loss for the year and still distribute capital gains!

Index funds and ETF’s focus on tracking a benchmark. An example of a benchmark is the S&P 500. Since these benchmarks have minimal turnover, the respective funds also have low turnover. Investors also benefit by paying much lower expense ratios in passively managed funds.

Depending on your tax bracket, municipal bonds can also make a lot of sense in taxable accounts. These are state-issued bonds that provide state and federally tax-exempt interest. To qualify, you must reside in that state of issuance. They pay lower interest than taxable bonds, so you have to do some math to see if they fit in your portfolio. 

Tax-loss Harvesting

When you buy an asset in a taxable account and you sell it for a gain, you pay capital gains tax. And you know now that 1 year is the holding period that determines whether it is short or long term.

Since you need to liquidate securities frequently in retirement, it’s important to use capital losses wisely. Tax-loss harvesting does just that by using market volatility to your advantage. It is best to use an example to explain it.

Let’s assume you invest $50,000 in a fund and the market subsequently declines, decreasing your fund’s value to $40,000. You don’t want to panic and sell it and go to cash. And you can always just ignore it and wait for the markets to bounce back.

Or you can sell the fund and immediately purchase a substitute fund in its place. This way, you can still recover your $10,000 paper loss. But since you sold your initial investment, you have now created a capital loss.

Going back to our capital gain rules, you can use the loss to offset gains in the current year. If there are no gains, you can carry forward the losses. And you can deduct $3,000 of losses every year against your ordinary income. For large taxable account balances, tax-loss harvesting is a popular retirement tax planning tool. 

Asset Location

Asset Allocation is the process of investing in different asset classes to balance risk/reward. This means selecting the right weighting of stocks, bonds, and other asset classes in your portfolio.

But you may have never heard of asset location. This strategy involves determining which accounts should hold each asset class. The goal is to maximize after-tax returns and it works like this…

If you have all three types of accounts and you could pick where you want the most growth, which would it be? Well, we know that Roth accounts will provide tax-free distributions. So, it would make sense to hold your high-risk/return asset classes in Roth accounts.

Then it also makes sense to hold low risk/return asset classes in accounts with the highest tax rate. Of course, this is to keep your ordinary income lower in retirement. Bonds for example are great to hold in tax-deferred accounts.

And we already discussed many of the benefits and strategies to help control taxes in taxable accounts. Even if you only have two types of accounts, asset location is still key to your after-tax investment returns. 

Final Thoughts on Retirement Tax Planning

Give yourself a nice pat on the back if you have made it this far! We threw a lot of information at you readers, and you may feel a bit overwhelmed. But now you can start planning your retirement today!

The first step is to really think hard about what your retirement will look like. This will give you the answer to a key variable – how much income you need. Next, you need to get familiar with how the U.S. tax code works.

You also need to decide on Social Security and retirement account withdrawal strategies. Not to mention making sure your financial advisor is managing your portfolio in the most tax-efficient manner. Lastly, don’t miss out on any other tax-planning strategies like Roth Conversions or charitable gifting.

Taxes are at the center of all your retirement planning decisions. If this part isn’t clear to you, we highly recommend you get the help of a fee-only financial planner or other tax professional.

You only retire once. The less tax you pay, the more resources you have for a rich, fulfilling retirement.