401k Loans: Is It Ever a Good Idea to Withdraw From a Retirement Account?

Hey Bank of Dad. I’m in the process of buying a home and have been told that, in this situation, it’s okay to withdraw from my 401k, which, right now, has about 100K in there. I’d need to take a $40K loan out to make the down payment. There are numerous articles about the dangers of borrowing from the 401k but also those that discuss times when it is fine to do so. What do you think? Am I dumb to take out this loan? I know it comes down to looking at the interest I would gain on the loan were I to have kept it untouched in the account as well as the value accrued in my home. But are there any penalties for taking money out? Also: How do I take the money out and are there ever any times when borrowing from that account is the right move? I intend to put the money back in the account. — George, via email.

At first glance, borrowing from your own retirement account sounds like a pretty great deal. No credit check? Low origination fees? Interest that you pay to yourself instead of a bank? What’s not to like?

But like shiny jewels sold from the trunk of an ’92 Lincoln, 401(k) loans appear much less enticing the closer you look. When it comes down to it, they make the most sense as a last-resort source of funds – not something you want to lean on when making a big purchase. Why? Because pulling money out of your nest egg is one of the surest ways to derail your long-term savings and potentially find yourself with a huge tax bill.

It’s true that if your employer is among the more than 80 percent of companies who offer loans, you should be able to access at least some of that money. IRS rules permit you to pull out 50 percent of your vested account balance, up to $50,000, for loans. The key here is the “vested” part. In your case, the sum of your contributions and rollover amounts, plus any vested matching funds, would have to be at least $80,000 to take out a $40,000 loan.

You typically have to pay back the principal and interest over a five-year period. A unique feature of 401(k) loans is that the interest you pay – often times the prime rate plus one percentage point – gets added to your account balance. You’re not losing any of that money to a bank or other lender.

But, my oh my, are they loaded with land mines. “I prefer to think of retirement savings as sacrosanct,” says Rebecca Kennedy, a financial planner with Denver-based IMPACTfolio. “Frankly, the idea of taking out a $40,000 loan from a $100,000 account balance concerns me.” Here’s how a 401(k) loan that size can backfire:

1. You’ll Experience a Big Cash Crunch

With a mortgage, you have the option to spread out payments over a 30-year period. But, often, you have to pay back a 401(k) loan in just five years. You’ll be making much bigger payments, and that means less cash to pay your mortgage, put into an emergency fund and, you know, eat. “The plan might allow for longer repayment since it is being used for a home purchase,” says Kennedy. “But it could still translate to a hefty monthly or quarterly payment that needs to be factored into cash flow.”

1. It’s a Huge Drag on your Retirement Savings.

When you’re paying back the loan, you’ll have less money to invest when you’re in that accelerated repayment schedule. That’s a huge opportunity wasted. One of the absolute keys to smart retirement planning is starting early. Every dollar you put in while you’re young has the chance to earn compounded growth when it stays in your account. So the $100 you invest in your 20s ends up being a lot more valuable than the $100 you throw in right before retirement. You want to keep that money in the account, where it can grow.Plus, you’re repaying yourself with post-tax money. Compare that to the tax-deductible 401(k) contributions you could be making if you didn’t have the loan. You’re forgoing a huge benefit in the tax code.

1. You Could Get Stuck with a Huge Tax Bill.

Any loan amount that you don’t repay on time gets treated as an early distribution if you’re under 59½. That means you’ll have to not have to pay income taxes on that amount, but incur a 10-percent penalty from Uncle Sam. Yikes. Perhaps you’ve done the math and don’t think falling behind on your loan is a big worry. Keep in mind, though, that if you leave your job for any reason, you’ll likely have to pay back the entire amount by April 15 of the following year to avoid a tax penalty. According to a 2015 working paper for the National Bureau of Economic Research, as many as 86 percent of people who leave their job during repayment default on their loan. Eighty-six percent! If you’ve already drank the 401(k) borrowing Kool-Aid, that statistic alone should jolt you into sobriety.

I can certainly see why people get jittery about the stock market, given its inevitable ups and downs. However, it’s generated much higher returns over the long haul than real estate.

Don’t think the upward climb of property values is always a sure thing, either.

“Buying a house isn’t always a profitable venture, as people in certain housing markets across the country learned during the 2008/2009 financial crisis,” says Kennedy. “Hindsight will tell if we’re near the peak or not, but all home purchases now should be made with the intention of staying put for a while.”

If you don’t have the means to buy a home without tapping into your 401(k), that might be a signal that you’re getting in over your head. And if you’re mainly looking at the home as an investment, you’re probably better off using pre-tax money to bulk up your retirement account. As long as you invest using an age-appropriate asset mix, you’re potential for growth will be much greater.