One big issue with early retirement is being able to access your retirement accounts without paying the stiff 10% early withdrawal penalty.
It’s great to invest in the traditional 401k and IRA because you don’t have to pay tax on that portion of your earnings. This is an excellent way to defer tax especially if your income is high. Uncle Sam will get eventually get his cut from this pile of money, though. You will have to pay tax at the earned income rate when you take withdrawal from these retirement accounts. Deferring tax until after retirement is usually advantageous because you make less money in retirement and should be in a lower income tax bracket. However, there is a 10% early withdrawal penalty if you want to access the money before you’re 59 ½. This could be a problem if a significant portion of your net worth is in the tax-deferred accounts.
Our Retirement Accounts
We love our 401k and IRA because it is the easiest way for us to save and invest. I also put off paying tax on a significant portion of my blogging income every year. I love paying less tax in April, who doesn’t? We have been saving in our pre-tax accounts for 20 years and it has grown to be the biggest part of our net worth.
Our retirement accounts make up 44% of our net worth and they should continue to grow over time. Currently, most of our new investment is going into those retirement accounts because we want to defer tax.
What if we want to access our pre-tax accounts at some point? In 13 years, RB40Jr will graduate from high school and hopefully heading off to college. At that point, we’d probably want to retire full-time and mostly relax. We will have some income from our dividend stock portfolio, and I’d probably be out of the rental property business. If the income from our dividend portfolio isn’t enough to support our lifestyle, then I would consider taking early withdrawal from our IRA. As long as we spend less than 4% of the total value of our portfolio, we should be able to sustain a lengthy retirement. In 13 years, I won’t be 59 ½ yet, so I need to work around the 10% early withdrawal penalty.
Actually, Mrs. RB40 plans to retire in 2020 so our income will drop quite a bit. We will be in the lower tax brackets and it will be a great time to withdraw some money from our IRA. The withdrawal is taxed at the earned income rate so it’s better to spread out the withdrawal over a longer period. You will pay less tax that way because you will be in the lower tax bracket.
Don’t forget to consider social security benefit. Once you start taking your social security benefit, it will increase your income and possibly push you to a higher tax bracket. The goal for early retirees is to stay in the 15% tax bracket as much as possible. That’s the sweet spot for minimizing tax. You don’t even have to pay tax on dividend income! There are several ways to avoid paying the 10% early withdrawal penalty when you withdraw before 59 1/2.
1. Roth IRA conversion
Personally, I think the best way to access your retirement fund early is to build a Roth IRA ladder. Here is how to do it.
- Convert 1 year of living expense to Roth IRA. (You will have to pay tax when you do this.)
- Wait 5 years
- Withdraw the 1 year of expense from the Roth IRA
Just repeat this every year until you’re 59 ½. The drawback here is you have to wait 5 years before you can access the first payment to avoid the 10% penalty*. This is doable for us because we could draw down our dividend portfolio and other taxable accounts during the first 5 years. Also, don’t forget to add inflation and tax to the amount you convert every year.
* With the Roth IRA, you can withdraw your regular contribution at anytime without having to pay the penalty. Conversions will have to wait 5 years to avoid the penalty.
Actually, it’s a good idea to move as much of your assets to the Roth IRA as possible. You won’t have to pay tax on that account ever again. When Mrs. RB40 retires, our income will be relatively low and we’ll probably convert a small portion to Roth IRA every year.
2. IRS Rule 72(t)
Another way to access your traditional IRA is to use the IRS rule 72(t). This will help you avoid that early withdrawal penalty, but you’ll have to follow some rules. First, you will have to take “substantially equal periodic payments” (SEPPs) every year. Once this starts, you must continue to do so for at least five full years, or if later, until age 59 ½.
The calculation for the rule 72(t) is pretty complicated and that alone would deter a lot of people. You probably need to hire a financial advisor or tax accountant to help with the process. I’ll use Bankrate’s 72(t) calculator to give us an idea of how much we could withdraw.
- Account balance: $2,000,000
- Reasonable interest rate: 2.1%
- Your age: 55
- Beneficiary age: 54
- Choose life expectancy table: Single life expectancy
I could pick one of the following methods depending on how much money we want to withdraw.
- RMD method: $67,568
- Amortization method: $91,414
- Annuitized method: $90,956
I’d probably go with the RMD method to keep tax low. It’s tough to look ahead 13 years because we don’t know how much inflation we’d see. The tax bracket will adjust for inflation, but this looks like a lot of income to me.
The main problem with using rule 72(t) is you’d have to keep withdrawing for at least 5 years. If you mess up somewhere, you’d have to pay 10% penalty on the total amount withdrawn. I like the Roth IRA ladder method better because you have more control over how much to convert.
3. Early 401(k) withdrawal
Here is an exception to the 59 ½ rule that’s now widely known. If you retire after 55, you can withdraw from your 401(k) with no penalty. This only applies to a company established 401(k). The age limit is further reduced to 50 for retiring police officers, firefighters, and medics. That’s a pretty nifty benefit for our public servants. Note that this provision does not apply to your IRA.
We won’t be able to use this exception because we’d both be long retired before we’re 55. I guess I could go back to work and do a reverse rollover. This would move my IRA into an employer sponsored 401(k) and enable me to take advantage of this exception. Of course, that would mean going to work for a company which would be my last choice.
4. Tap your IRA to pay various expenses
Lastly, there are a few exceptions to the IRA early withdrawal penalty. This could be a good way to take some money out of your retirement accounts if you have these expenses anyway.
- Pay for higher education. We plan to help RB40Jr with his college education so this could work for us. If our 529 isn’t enough to pay for college, then we can make some withdrawals from our IRA.
- Health insurance premium while unemployed. This one seems complicated. I don’t think we’d qualify as unemployed when we’re retired.
- Unreimbursed medical expenses over 10% of AGI.
- IRS levy back taxes. This one is very interesting. If you owe taxes to the IRS, they can levy your IRA and you won’t have to pay the 10% penalty. This seems like an incentive to not pay tax on time. Of course, you’d have to pay a penalty for late payment so I’m not sure if this makes sense financially. The late payment penalty is the federal short-term interest rate plus 3%.
- First time home buyer. If you haven’t own a home for 2 years, then you could withdraw up to $10,000 to help with the down payment.
5. Reader’s comments
- If you have a 457 plan, you can make a withdrawal without the 10% penalty. This plan is available to some state and local public employee. Some non profit organizations offer this plan.
Accessing your retirement account
Out of these options, I like the Roth IRA ladder option best. We have enough in our taxable account to pay the first 5 years of living expense. After that, we’ll be able to withdraw from our Roth IRA without paying the penalty. I like having more control over the accounts. Also, it’s always a good idea to move money into your Roth IRA because that account is tax free forever.
Taking money out of your retirement accounts early usually isn’t a good idea because most of them are woefully underfunded. The average retirement saving of Americans between the ages of 55 and 64 is just $104,000. If you take early withdrawal from that account, you’d run out of money pretty quickly. On the other hand, what if you’re lucky enough to have $2 million in your traditional IRA at the age of 55? In that scenario, it would be better focus on minimizing tax and make some early withdrawal. When you turn 70½, you would have to withdraw the required minimum distribution (RMD) on your IRA. You don’t want to have a huge amount in your traditional IRA at that point because the RMD would kick you into a high tax bracket. It’s still a better problem to have than not having enough money.
Roth IRAs are not subject to the RMD rule in your lifetime. That’s another reason to move money into your Roth IRA.
Are you planning to retire early? Do you have a plan to access your retirement accounts?
Image credit dbking
Passive income is the key to early retirement. This year, Joe is increasing his investment in real estate with CrowdStreet. He can invest in projects across the U.S. and diversify his real estate portfolio. There are many interesting projects available so sign up and check them out.
Joe also highly recommends Personal Capital for DIY investors. He logs on to Personal Capital almost daily to check his cash flow and net worth. They have many useful tools that will help DIY investors analyze their portfolio and plan for retirement.