Sadly Giving Up On Retirement And Going Back To Work

16 min read Original article ↗

After doing a deep dive on the different ways to pay for college and how to get good financial aid, I realized the best thing for my family is for me to go back to work. Coming up with $1,500,000 for two kids' college experience in 12-15 years is daunting.

Going back to work feels like giving up since I helped kickstart the modern-day FIRE movement in 2009. But the reality is, times are always changing. And we must adapt with the times.

Sitting idly on my hands, hoping my investments will miraculously cover such an expensive nut in the future feels uncomfortable. Given I have young children, I face expenses that are inflating at the fastest rates (tuition and unsubsidized healthcare at $2500/month). As a result, I want to take action while I'm still healthy and full of energy!

When I retired in 2012, my wife and I had planned to live a simple life on Oahu with less than $100,000 a year. I was 34 and she was 31. We would help take care of my grandfather's fruit farm, sell some mangos and papayas to the local supermarkets, and feed ourselves with what was left over.

After trying for so long, we thought kids would not be in our destiny. But our son was finally born in 2017 and then our daughter in 2019. Suddenly, our whole world had changed and so had our expenses.

Going Back To Work Isn't A Total Failure

In 2017, I made a promise that I would be a stay-at-home-dad for five years. After five years, my son would be able to go to kindergarten full-time. I fulfilled my promise plus an additional year because of the birth of our daughter. I wanted to give her the same amount of time as I had given my son.

However, a ~20% decline in the S&P 500 hit our net worth by ~7% in 2022. Both my kids' 529 plans also got hit almost equally as bad because bonds performed terribly in 2022 as well. The funds are in index target date funds.

My investments in private real estate across the heartland have also performed well during the pandemic, especially in 2021. However, after 11 rate hikes since 2022, the real estate market is now on shaky ground.

Going back to work will replenish the bank account and provide a buffer for a potential decline in passive income. To be truly financially independent, one needs to have enough passive income to cover one's desired living expenses.

If a recession returns in the next couple of years, then everything from dividend payout ratios to rental occupancy rates will be at risk. Here's a recession preparation checklist to follow. I think the likelihood of a recession is 70% by 2025.

Going Back To Work Is A Compromise

I do feel guilty about not spending as much time with my daughter as I did with my son. However, she still received a lot of love for the four years before she started preschool part-time.

With my wife always at home, I am clearly my daughter's second love. I've experienced many hurtful rejections that have made me question the point of being a stay-at-home father for so long. But as she gets older, things are getting better.

In a sad way, going back to work protects my heart, while also optimizing my purpose. I've spoken to many dads on paternity leave who can't help but feel restless after the third month. If they were the breadwinners before, they felt an immense draw to go back to work to earn.

Having two stay-at-home parents is great for the first of a child's life. There's so much that needs to be done to take proper care of a baby. The second year of life is pretty wonderful given most toddlers will begin walking and playing by then.

However, after the second year, I think it's more efficient for one parent to go back to work. It's better for financial security, peace of mind, and work/life balance. Usually, that one parent is the father because the child needs and connects with the mother more.

House Is Getting Small Again

No matter how much you love someone, it can sometimes be hard to be around them 24/7. You might scare the bejeezus out of them when you silently get something in the kitchen. Or maybe you might get annoyed by the mess they leave behind in the living room.

Whatever the case may be, having some separation is healthy for any relationship. There's a reason why best friends sometimes no longer are best friends after a long roadtrip.

I've done my best to get out of the house every day by playing sports. However, my body can no longer play for longer than two hours before my back, heels, hips and right shoulder start hurting! As a result, my outings don't last as long as they have in the past. Further, I usually rest the entire next day, which means staying at home.

My wife enjoys being at home more than I do. She definitely doesn't want to go back to work, no matter how much I've encouraged her. As a result, it is up to me to adapt. I don't mind because I enjoy meeting new people. Luckily, life is back to normal.

Can't Buy A Mansion For Relief

Buying a larger house in 2020 gave us three years of extra comfort so far. However, as our kids grow bigger and accumulate more stuff, our house seemed to grow smaller.

I strongly considered buying an even larger house. However, my wife doesn't want to move. Further, I'm hesitant to concentrate even more of our wealth on San Francisco real estate. Our total annual property tax bill is already too high for comfort.

Instead, I want to continue diversifying more into heartland real estate. Working from home and the spreading out of America to lower-cost areas are here to stay. Check out Fundrise if you're interested in investing in this long-term trend as well. Dollar-cost averaging in this weak market is attractive as I think real estate prices will eventually rebound and catch up to stock prices.

Going back to work now that both our children are in school gives us more space, which is healthy for our relationship.

Fundrise

Work Seems Easier Today

Early retirement is becoming obsolete thanks to the pandemic. The amount of flexibility people have at work today is incredible compared to the past.

If I had the freedom some workers have today, I would have worked for at least five more years. Back then, I had to sneak away to take a 15-minute nap in my car! Today, you can play pickleball for 1.5 hours during the weekday and then take a half-an-hour nap after if you want.

As an efficient person, I'm sure I can get eight hours of work done in four hours or less. So long as I'm not encumbered with meetings, I have no doubt work will be more manageable.

I'm also pleased that companies seem to be less focused on face time and more focused on production. So long as you get your work done, that's mostly what matters. With so much more work flexibility, going back to work is more digestible.

Hard To Pay The Bills As A Writer

They say “follow your passion and the money will come.” Good luck with that occupation strategy! Instead, it's better to do something that pays and then pursue your passion on the side.

Despite writing Buy This, Not That, a WSJ bestseller, I still have not earned enough from the book to support my family. To do so, I need to be raising my national profile regularly. But I'm disinterested in the spotlight.

Despite having a 60,000-subscriber newsletter, I feel bad moving to a subscription model where I could probably make at least $150,000 a year. The goal of my newsletter is to help anyone and everyone interested in building more wealth while also staying on top of the most important financial topics.

Given the difficulty of being a professional writer, the logical solution is for me to get a day job or consult again. I'll still write, just not as often. Or maybe I’ll leverage AI to write my content for me.

For all you creatives out there, don’t lose hope! I had a really good run for 12 years, but all good things must come to an end. It’s still better to try than never try at all.

More Active Income And Good Benefits

The ideal income composition doesn't consist of 100% passive income. Instead, it's better to have at least 10% of your total income coming from active day job income. This way, you feel like you're doing something productive in society.

One of the biggest negatives of early retirement is feeling lost. Without a day job bringing you purpose, it's easy to fall into despair as you question what is the point of life. The longer you work, the more you will have your identity wrapped up in work.

Giving Up On Retirement And Going Back To Work
Working hard in Istanbul

Having a day job, no matter how mundane, helps provide structure and purpose in our lives. With school teachers now taking care of educating our children, a large part of my purpose as a stay-at-home dad is gone.

I'll certainly continue to provide supplemental education after school and during the weekends. Being the family driver will always be my duty. But I'm now left with a void that needs to be filled.

It'll be nice to earn a steady paycheck and receive subsidized healthcare benefits again. I'm also looking forward to getting a 401(k) match or profit sharing once more.

Even though I’m going back to work, at least I won't have any regrets trying to make it as a professional writer. I don't have regrets leaving banking before making Managing Director because I at least tried for one year.

Trapped By The School Schedule

So long as your children are attending grade school in person, parents are trapped by the school schedule. Therefore, parents might as well spend some of the school time to make some money as we wait for spring, summer, and winter breaks.

Homeschooling is in the cards given that's what we did from March 2020 to August 2021. Kids learn so much quicker than regular schooling. However, given our children are still young, they won't appreciate their travels just yet.

Going back to work helps utilize the downtime when kids are in school. If my family wants to slow travel or live abroad, it's also best to go back to work and make some money while I can before we do.

Having a goal of saving up for years of world travel makes going back to work more meaningful.

My New Job Thanks To A Return To Normalcy

So what will I be doing?

Well, I love basketball so I've decided to apply for a job as the video coordinator for the Golden State Warriors. Although the job only pays $40,000 a year, I get to be around the coaching staff and the players.

I feel like I've got good analytical insights to share with the team. Matchups intrigue me and so does the ability to build winning teams. Besides, the NBA needs more Asian diversity!

When I coached high school tennis, we won back-to-back Northern California Sectional championships partially due to lineup strategies. Before then, the school had never won an NCS championship.

There you have it. Starting in 2023, I hope to be busy working through April or May 2024. The season will be hectic and I may have to do some traveling.

But it'll be fun! Barista FIRE for the win. And at least I'll still get five or six months off a year, depending if we get to the playoffs.

In A Different Life, Perhaps

OK, I was joking about landing a job with the Warriors. But everything I've written in this post is true.

I need to find a way to get out of the house more often, make active income, have more purpose. and fill my days once both kids are in school full-time in fall 2024. For seven years, so much of my life has evolved around raising young kids. With college tuition and a rapid rise in costs for everything, it’s prudent to do some consulting or go back to work again after 12 years of retirement.

Last year, I was invited to a players and coaches event with the Warriors and it was exhilarating. It felt wonderful to be a part of a team, especially one that has won for so long. I miss the camaraderie.

If you love basketball, the NBA, or the Golden State Warriors, I talk to my friend, Shaun Livingston, who won four NBA championships with the Warriors. Have a listen. You can also go the podcast on Apple, Spotify or wherever you listen to podcasts.

In the meantime, I've got some other ideas on how to fill the void:

  • Host a book signing event for up to 100 attendees in San Francisco at the Sports Basement at Stonestown Mall. Anybody want to come?
  • Become a pickleball product ambassador or affiliate partner. I met a guy who is one for a pickleball energy drink. Maybe I can use my platform to meet new people, attend events, and play in tournaments.
  • Join an author group and write my next personal finance book with Portfolio Penguin.
  • Become a school teacher in Hawaii where I’d like my kids to go. One of the benefits will be a discounted tuition and easier entrance. I’ll also get to see them more often during the day.
  • Find a job in Artificial Intelligence since it is a booming industry and so many AI companies are based in San Francisco, where I am. I've applied to several companies so far with no luck. The difficulty of find a good job after retiring for so many years is hard!
  • Forsake tech work and get a doctorate degree in real estate.

So Old And Tired To Work As Hard As Before

Despite some serious upcoming expenses in the future, it's difficult for me to get a regular day job again. I'm too old to grind as I once did. There needs to be a job that provides better balance and more fun. As a result, I'm leaning more toward consulting part time like I did for Empower (Personal Capital) back in 2013-2015.

Doing part-time consulting for 15-25 hours a week will help fill the void once both kids start school full-time in September 2024. We are purposefully only sending our daughter to school three days a week because she will be our last child. As a result, we want to spend as much time with her as possible.

In addition, I realized if I send my kids to community college, I won't have to come up with $1.5 million to pay for two private college tuitions in the future. Hence, there pressure to make a lot of money at work won't be as high.

If you are under 45, please don't waste your energy! Not having kids is your FIRE super power! Your energy will eventually fade. When it does, you had better have a pile of money saved or invested. Otherwise, you may feel trapped in an increasing state of misery.

I hope this post has provided some insights into some of the complexities of life. The more you have (time, children, money, etc), the more decisions you will need to make.

Don't think that just because you reached your target net worth, got that promotion, or finally gave birth to a girl after two boys, you'll be forever happy. You'll be elated for a bit, and then it's back to your steady state of being.

Life takes work! Maybe that's the job we should really be focusing on all along.

Went Back To Work And Couldn't Last

Well folks, I gave going back to work a try from November 25, 2023 through March 2024, and couldn't last for more than four months. I recap my experiencing about returning to work after FIRE to help you realize how difficult it might be. It's a tricky situation if you've tasted years of freedom and the fit isn't perfect.

But I am appreciative for giving work a try again. It made me realize my weaknesses. I got a better sense of what type of work I want to do. Finally, I'm more appreciative of the time I have.

Feeling Better Financially Due To A Rebound In The Stock Market

Thankfully, 2023 and 2024 turned out to be fantastic years for the stock market, pushing my stock investments to all-time highs. As a result, I do feel better about my financial situation.

However, I made a self-inflicted financial wound in the second half of 2023 by selling stocks (and bonds) to buy another forever home. The property was discounted 14% from its 2022 asking price and featured rare Golden Gate Bridge views on a massive, usable lot—an opportunity too good to pass up.

The purchase, however, came at a cost. It reduced my passive income by approximately $150,000 a year, bringing it down to $230,000. Over the past year, I’ve clawed back my passive income to around $300,000 by renting out my old house and saving and investing aggressively. Despite this progress, I’m still about $80,000 short of having my investment income fully cover our living expenses.

As a result, it’s back to work for me, with a clear goal: to achieve $380,000 in passive income by December 31, 2027, or build an additional $2 million in net worth. The $2 million comes from using a 4% withdrawal rate.

Reader Questions And Suggestions

Readers, what are some fun jobs you'd love to take if you had the opportunity? How has your life changed after retirement?

If you're looking to invest in artificial intelligence, take a look at the Fundrise venture product. The Fundrise invests in AI, DevOps, FinTech, PropTech, and Data Infrastructure private growth companies. Unlike traditional venture capital funds where you must commit capital first and then hope the general partners make good investments, you can see what Fundrise is invested in first before investing.

I've personally invested over $500,000 in Fundrise Venture so far to gain more exposure to private growth companies. I want to own as much OpenAI, Anthropic, Databricks, and Anduril as possible to hedge against my children not being able to find jobs in the future. Below is my children's Fundrise investment dashboard.

Fundrise Innovation Kids Account 200K

For more nuanced personal finance content, join 60,000+ others and sign up for the free Financial Samurai newsletter and posts via e-mail. Financial Samurai is one of the largest independently-owned personal finance sites that started in 2009. 

Fundrise is a sponsor of Financial Samurai and Financial Samurai is a six-figure investor in Fundrise funds.