Skip to main content
Mar2, 2023

Should I Downsize in Retirement?

Where to listen

In this episode of the Great Retirement Debate, Ed and Jeffrey debate whether or not it’s a good strategy to downsize your home in retirement.

Episode Transcript

[00:00:00] INTRO: Hi, I’m Ed Slott, and I’m Jeff Levine. And we are two guys who just love to talk about retirement and taxes. Look, our mission is simple to educate you, the savers, so that you can make better decisions because better decisions on the whole lead to better outcomes.

[00:00:14] INTRO: And here’s how we’re going to do that.

[00:00:15] INTRO: Each week, Jeff and I will debate the pros and the cons of a particular retirement strategy or topic with the. Helping you keep more of your hard-earned money. Yeah, but we won’t know which side of the debate we’re taking until we flip a coin winner of the coin flip gets to pick which side of the debate they want to argue, and both of us will have to argue in favor of our respect positions, whether we agree with them or not.

[00:00:40] INTRO: At the end of each debate, there’s going to be one clear winner you, a more informed saver who can hopefully apply the merits of each side of the debate to your own personal situation to decide what’s best for you and your family.

[00:00:52] INTRO: So here we go. Welcome to the Great Retirement Debate.


[00:01:02] Jeffrey Levine: Welcome back to the Great Retirement Debate, Ed, I am fired up for our next episode and ready to argue. So tell me, Ed, what are we arguing about?

[00:01:12] Ed Slott: Well, here’s an interesting point. Should I downsize my home in retirement? I get that question a lot from clients, especially older ones like me, and I have to remind our listeners, Jeff and I have a generational divide, so our point of view could be, uh, skewed towards that age. Jeff’s about 30 years younger than me, so I may have a different point on that. Uh, as usual, we’ll flip a coin to see which side of the debate we’ll come out on. You ready?

[00:01:39] Jeffrey Levine: I will say, Ed, we’re, we’re finally getting to the point though, where we have about the same amount of hair between the two of us.

[00:01:46] Ed Slott: Yeah, that’s why we do audio podcasts.

[00:01:49] Jeffrey Levine: That’s all right. You, were headed, you know, it was kind of a weird race here, you know? First I had more hair than you had more hair, and now we’re tied, so, yeah.

[00:01:55] Ed Slott: Right.

[00:01:57] Jeffrey Levine: Alright, well that’s an interesting one, Ed, your [00:02:00] topic, i’m gonna call it, I’m gonna go, uh, I’m gonna say tails.

[00:02:03] Ed Slott: All right. Tails.

[00:02:07] Jeffrey Levine: Tails! I finally win one. You know what? My natural inclination would be to argue in favor of downsizing, but I’m gonna go the other way. Um, and I’ll, and I’ll, and I’ll say, I’m gonna go the other way because of a particular interaction, uh, with a client of mine. And that is, I’m going to say, I’m gonna argue in favor of not downsizing.

[00:02:26] Jeffrey Levine: My beginning point that I’d love to make, uh, just to kick it off, is when I envision myself, you know, one of the things that uh, I can share now is recently I’ve built a house, uh, that is, is meant to be a forever home and it’s a larger house. Uh, you know, there’s plenty of room for our family now, but also plenty of room for our family as our family grows.

[00:02:46] Jeffrey Levine: I’m lucky enough to have three healthy boys, God willing, one day they all grow up. I mean, the oldest right now is only eight, so it’s many years from now. But God willing, they all grow up. They find, uh, you know, people that they love to spend their lives with. And [00:03:00] maybe at least one or two of them, maybe all of them, uh, end up, you know, bringing home grandchildren one day.

[00:03:05] Jeffrey Levine: I’d love for this place to be the place where we can all gather. Each and every year for those important holidays, continue family traditions, etc. And I had a client who, who did this in retirement. He actually upsized because of that.

[00:03:19] Ed Slott: Oh boy.

[00:03:19] Jeffrey Levine: He wanted to have a place for his family to come spend in retirement.

[00:03:23] Jeffrey Levine: He said it was the best decision that he’s ever made. And I look at all the memories that he’s created in that house with his family, and I just want the same for myself and my family.

[00:03:32] Ed Slott: I can see that. But again, as a grandparent and say 30 years you’re a senior. Uh, you know, I have to..

[00:03:38] Jeffrey Levine: Say that one more time. I just like to hear that.

[00:03:40] Ed Slott: Yeah. I want people to know where I’m coming from. It used to be the American dream was to own your own home. Now it’s getting the kids out of it. Uh, I think if you have a large home now you talked about your three boys bringing everybody home. What if they’re still living there?

[00:03:57] Ed Slott: Uh, , you know, it’s very common [00:04:00] now. In today’s economy, people coming back in. I like the idea of downsizing, just to simp again, simplify my life, get rid of otherwise my, my kids will end up cleaning up the mess. All these little things. You know, I had this, uh, when my mom died, we’re going through all this stuff now.

[00:04:17] Ed Slott: She did downsize after my father passed away, but my father was more in your school. He loved having the house and everybody came over the house, but after he passed. So it may be what situation you’re in, uh, as a single person, you might be better off downsizing because you don’t have to worry about if you have a home cutting the lawn and doing all these chores around the house and just having, making life simple, less bills to pay and less things to take care of.

[00:04:45] Ed Slott: I always say same thing with your investments. People always ask me about having all these IRAs have 50 IRAs, you only need one IRA, you know, . So sometime it just makes life simpler. If you downsize, plus you can get your money out of the [00:05:00] house at the top of a market and, uh, even rent, you might not even have to buy something.

[00:05:05] Jeffrey Levine: Well, you know, the, those are, uh, interesting, interesting points, Ed. And I think one of the things that I would, uh, you know, that that kind of hits me as you talk through that, is the idea of simplifying life as you get older. And I understand that, and in fact, that’s probably the most common argument I hear from folks is, I don’t want to take care of this anymore.

[00:05:25] Jeffrey Levine: It’s too much house for me. I’ve gotta worry about this breaking and that breaking. But I think one of the other things that’s important to realize too, is that as you get older, you know, there is some benefit to routine, particularly for those who begin to lose, you know, cognitive abilities, etc.

[00:05:41] Jeffrey Levine: There’s a lot of evidence that would show that, you know, maintaining a consistent routine can help improve quality of life. You know, think about going from one place to another and then all of a sudden not remembering, you know, where you know where the bathroom is or where you store, you know, the, the drawer that you have your brush in for [00:06:00] the last 40 years.

[00:06:01] Jeffrey Levine: It’s kind of, you know, It’s just mem it, it, it’s automatic for a lot of people at some point. And then to have to pick up and move into a new environment can be challenging. And all the gains that are made, let’s say, from not having to pay the, you know, the lawn bill, well, so much of that today can be automated with auto payments and so forth.

[00:06:19] Jeffrey Levine: Or with a good power of attorney being able to take care of that for you. But those little things and the comfort of being in a place that you have known for so long is also one that I would, you know, would point out.

[00:06:32] Ed Slott: And you could be comfortable. Those are all good points and absolutely health is a big, is the, the big factor in all of this.

[00:06:41] Ed Slott: Do you have the wherewithal? It depends. Maybe you’re taking care of a spouse or maybe as I said, a spouse has died and you’re alone. You might like to be in more of a community in a smaller apartment or condo complex where there are people around that things are taken care of. So it’s, uh, I think health is a [00:07:00] big thing, and lifestyle.

[00:07:01] Ed Slott: Maybe you wanna travel more and you don’t have to worry about locking up the house every time you go away. You just, you know, if you’re in an apartment or a condo or co-op or something, you lock the door and you’re on your way. You don’t have to worry about any of these things. So again, I like the idea, uh, not me personally, but the idea of simplicity and not having to worry about other things. As you get older, people don’t wanna worry about a lot of other things. You know, there’s something to be said for a, a nice big, comfortable house, but you can have those same comforts, uh, in a smaller home or an apartment or a condo or something like that.

[00:07:38] Ed Slott: And, uh, sometimes in some of these things, as you’ve seen, uh, I mean, they’re all over the country. I was gonna say Florida, but they’re all over. Uh, you have amenities, you have your own small unit, but you have access to all kinds of amenities and activities and other people, and that’s really important if you’re alone.

[00:07:56] Jeffrey Levine: Now I got another one for you, Ed. I’m gonna have to go to the financial side [00:08:00] of this because I have a feeling arguing on, on, on the practical side, it’s gonna be a losing battle for me. So I’m gonna go to the financial side of this for a moment and say, If you have, uh, you know, purchased a house that has gone up significantly in value, the, there is under current tax law, an exemption for a certain amount of gain.

[00:08:19] Jeffrey Levine: If you are a single individual, you have up to $250,000 of gain that can be tax free. If you are married and file a joint return, you can have up to $500,000. But as home prices continue to increase, and we’ve seen substantial increases in recent years, and as we start off with larger amounts, it takes a smaller percentage gain to exceed those 250 and $500,000 thresholds.

[00:08:42] Jeffrey Levine: And if you are going to go and sell your. And your gain exceeds those excludable amounts, that gain is actually taxable to you even though you’re not able to like depreciate unless you’re taking a home off this deduction, but you’re not generally able to like depreciate your own personal home. And if you lose money on [00:09:00] your house, you don’t get to take a a, a loss.

[00:09:03] Jeffrey Levine: You do have to pay taxes, uh, income tax, that’s capital gains tax, plus the 3.8% sur tax if your income exceeds certain amounts on the gain from the sale of your house. But if you hold onto that house and you enjoy it throughout your lifetime, when you pass away, your heirs will be able to get that house with what they refer to as a step up and basis, meaning..

[00:09:26] Ed Slott: Right.

[00:09:26] Jeffrey Levine: You could have bought your house for 500,000, you know, 40 years ago now it’s worth two and a half million. You have $2 million of gain and that entire 2 million can escape taxation forever, as long as you hold onto it until you die.

[00:09:42] Ed Slott: Well, you know, that’s true. And I thought about that for myself. Do I wanna sell the house or maybe just hold on to death and get the step up in basis.

[00:09:51] Ed Slott: My heirs won’t have to pay any tax on it, but that’s more about them than me. Uh, it’s about my lifestyle and [00:10:00] maybe I don’t want to keep a whole house like that. And first of all, uh, there’s no guarantee the market, well, I shouldn’t say there’s no. There is no guarantee the market will always go up, even though traditionally for like our lives, 50 years.

[00:10:15] Ed Slott: I mean, I think of it, my father bought the house, my mother sold for 16,000. She sold it, you know, only a few decades later for half a million dollars. It’s crazy. Same house. Uh, so there, there is that, uh, but, the reason I was thinking of maybe downsizing is to aleve other people of all the record keeping, I was thinking of all the improvements.

[00:10:37] Ed Slott: You talk about the gain, but the longer you’re in a house, the, the more likely it is you put tens, if not hundreds of thousands into it. I mean, if you’re in a house 30, 40, 50 years, there’s certain things,

[00:10:49] Jeffrey Levine: New roof, new heater.

[00:10:51] Ed Slott: Yeah, you, it wouldn’t be standing if you didn’t put tens of thousands of dollars into it.

[00:10:56] Ed Slott: So maybe this is the time to, you know, it won’t de tax, your [00:11:00] issue won’t be as much as you think. First of all, a married couple. If you’re thinking while you’re married, you do get the 500,000. It’s double. So, uh, that, that may cover a lot of the gain. Remember, the 500,000 is not off the selling price, the gain, and the gain, the selling price less your original cost, plus all of those improvements. Now, true what Jeff said, you don’t need to know any of that if you hold it till death. Everybody, uh, the beneficiaries get the step up and basis, but then, uh, that’s like I said about the beneficiaries, making it easier for them and a little easier for you maybe cause you don’t wanna change anything.

[00:11:38] Ed Slott: So you keep piling on your junk in the house for somebody else to take away. And at some point, I think I’d like to just simplify and, uh, you know, get a, a smaller place and maybe there might be some tax to pay, but it all depends on how much you put into the house, but then it’s, it’s a lot simpler. You don’t have to worry about all these things.

[00:11:59] Ed Slott: And a lot [00:12:00] of the things I said before about, uh, especially if a spouse has died or you are, you don’t have a spouse and, you just wanna be active with people there. You know, you, you know, these places here, the villages in Florida, these people love that stuff.

[00:12:15] Jeffrey Levine: Yeah. They love a lot of things, Ed. Uh, I mean, I’m sure you’ve seen the reports that it is the, uh, you know,

[00:12:20] Ed Slott: Yeah, yeah, don’t even go there..

[00:12:21] Jeffrey Levine: Yeah. Right. Yeah, go ahead and Google the Villages folks. If you don’t know what I’m talking about.

[00:12:25] Ed Slott: All what Jeff is talking about is that they love life.

[00:12:29] Jeffrey Levine: Yeah, they love love, yes. Yeah.

[00:12:33] Jeffrey Levine: Yeah, that’s, uh, it’s an interesting place. Uh, you know, I would also say, Ed, you know, one other thing on, on the, the, the financial side of this is that, you know, most people when you think about retirement, they’ve got a tremendous amount of assets, or, or at least a lot of people who are probably listening to this type of podcast have a, a significant portion of their assets tied up in financial assets.

[00:12:54] Jeffrey Levine: Things like stocks, bonds, mutual funds, ETFs, etc. And if you [00:13:00] sell your house and you walk away with a boatload of cash, what are you gonna do? You’re probably gonna go and invest it in stocks, bonds, mutual funds, ETFs, your house while you own it, is effectively a, a non correlated asset with the market, with stocks and bonds.

[00:13:15] Jeffrey Levine: In other words, it’s a, it’s a diversifier to your portfolio. And while someone listening may say, yeah, but I can’t spend my kitchen. You can in a way, right? There are home equity loans, there are reverse mortgages, which, uh, are much, much more cost affiction and much more regulated than they were in previous years.

[00:13:35] Jeffrey Levine: Not advocating for it today, maybe on a future podcast, I’ll have to, uh, but you know, there are all these things that come up from time to time and, and ultimately, you know, that is, your house is an asset. There is a way to access the value in your house. While you are still there, and even if the market, the stock market that is, is down the real estate market, your house may still be valuable enough that you can take a loan from that [00:14:00] of substantial amounts.

[00:14:01] Jeffrey Levine: And in fact, sometimes you may even want to access the money in your house as a loan to avoid selling when the stock market is down to give it time to recover before you sell those financial assets.

[00:14:14] Ed Slott: Right. Uh, that’s, that is a good financial argument that you can tap the funds, but you still have to deal with maintaining the home.

[00:14:22] Ed Slott: And I know some people tell me they keep the home because they wanna leave it to their kids. At 90, uh, I heard this is not a statistic, it’s just something somebody said, but it makes,

[00:14:31] Jeffrey Levine: It’s just something you made up, but it sounds good for the podcast.

[00:14:34] Ed Slott: Somebody else made up. He was talking about reverse mortgages, that’s what reminded me when you said that. He said, and there’s probably some truth to it. 99% of homeowners, parents, uh, think their kids want the house when they die. 99% of kids don’t want the house. They want the money.

[00:14:52] Jeffrey Levine: Yeah, I would agree with that. And I mean, forget about that. What if you differ in opinion?

[00:14:56] Jeffrey Levine: Right? I mean that’s, uh, you know, that is one that, uh, you know, [00:15:00] you’ll notice that I definitely didn’t use that as an argument. That it’s what the kids want.

[00:15:03] Ed Slott: What do you mean different opinion with the two spouses?

[00:15:05] Jeffrey Levine: No, the two kids. Right. Like what if

[00:15:07] Ed Slott: Oh, the kids. But I, I, when you said that, I was thinking of my parents, they did differ in opinions.

[00:15:12] Ed Slott: My father wanted to stay in the house. He was just like, your first argument, this is my palace. This is where everybody comes. I’m not moving. But he died early, , and it didn’t take a year or two. My mother said, I’m outta here. I’m doing what I always wanted to do, get my little apartment. She sold the house and she didn’t even buy anything else.

[00:15:29] Ed Slott: She just rented and it worked out great. The money lasted up for the rest of her life.

[00:15:34] Jeffrey Levine: Sounds like a win.

[00:15:36] Jeffrey Levine: Ed, I don’t know. It’s a good discussion. I, I don’t have, I have I convinced you, are you rethinking selling your house now or do you still think you’ll downsize? I think that may be the ultimate judge of how we debate.

[00:15:46] Ed Slott: Yeah. I still would like to downsize cause I’d like to do it on my terms, see what goes and, you know, take care of everything. So I don’t leave other people a problem, but there’s another side to that. Leave them the problem.

[00:15:59] Jeffrey Levine: Yeah. [00:16:00] That’s what I’m gonna do.

[00:16:00] Ed Slott: A step up and basis. They can hire accountants and attorneys. They’ll have plenty more money.


[00:16:05] Jeffrey Levine: I’ve got probably 30, 40 years before I have to make that decision. But that’s my, that’s where I’m leaning at least for right now.

[00:16:10] Ed Slott: All right.

[00:16:11] Jeffrey Levine: Yeah. There we go. Well, you know, another, another fun, uh, discussion Ed, and, you know, certainly two sides to this coin, as there are with, uh, most everything in life, but..

[00:16:20] Ed Slott: This is very practical. This is a lot of people, uh, are grappling with this now.

[00:16:24] Jeffrey Levine: Yeah, yeah. But, but our listeners, you know, their life and their retirement decisions are too important to leave up to that coin flip. So, you know, that’s why the one thing Ed you and I always agree on is making sure that, uh, people are talking with a knowledgeable financial advisor or tax professional when it comes to any sort of big decision. This way they can weigh the pros and the cons of different options against their specific set of goals and objectives.

[00:16:49] Jeffrey Levine: As a reminder, if you’d like to continue the discussion with Ed and I, we love to hear from you. Uh, we always look forward, uh, to seeing what’s coming through in our inboxes or on Twitter. You can reach out to Ed on [00:17:00] Twitter using the handle @TheSlottReport. That’s with two ts, @TheSlottReport. Or you can reach out to me using the handle @CPAPlanner. Once again, that’s @CPAPlanner. Uh, and we would love to hear from you.

[00:17:14] Jeffrey Levine: Let us know what you thought. You think you’re gonna downsize, you’re gonna stay in your current home. Why? And if you have a topic for us for a future episode, we’d love to know.

[00:17:23] Ed Slott: Well, I think it’s important we hit good topics, uh, good points on this, but one of the big points, I just wanna reiterate the point you made, you have to really have some good records.

[00:17:33] Ed Slott: I mean, you have to know what you paid. You talked about the gain. You have to know what you paid, and a lifetime’s worth could be decades worth of improvements. You know, pe sometimes people don’t keep records. This is one area where you’ve had to keep these records forever, as long as you own that.

[00:17:49] Ed Slott: Because everything you can prove will increase your basis and may get you to the point that you can sell with that $500,000 exclusion and have no tax. Anyway, [00:18:00] something to think about and we’ll see you next time on the Great Retirement Debate.

[00:18:06] Outro: Jeffrey Levine is Chief Planning Officer for Buckingham Wealth Partners.

[00:18:09] Outro: This podcast is for informational and educational purposes only, and should not be construed as specific investment accounting, legal or tax advice. Certain information mentioned may be based on third party information, which may become outdated or otherwise superseded without notice. Third party information is deemed to be reliable, but it’s accuracy and completeness cannot be guaranteed.

[00:18:26] Outro: The topic discussed in corresponding arguments are those of the speakers and may not accurately reflect those of Buckingham Wealth partners.