Self-defense isn't purely about having a tool or practicing a martial art.
Both of those things are good, but they will only add to what you bring to the table when that self-defense happens. Same thing with adrenaline. It will only add to what you bring to the table.
If you're slow and sloppy, you're really not contributing anything more to the odds of your survival or the defense of your loved ones. Defense is also about confidence. A strong confident presence will contribute to your ability to deter a threat.
Look, when it's "go time" and every second counts in your defense, you want to move quick and hit like a ton of bricks.
When we're young bucks, it's pretty easy to be cocky and over confident. Doesn't mean that we can actually back it up, but we're pretty confident until we're proven otherwise. Some learn that lesson with maturity, some only learn it when they get run over by a threat. But as we get older, our fitness is no longer simply a function for looking good at the beach, it really does start to become about your survival.
If you haven't been to the gym lately, get going. If it's been a long time, go find a good trainer to help you get back in shape. Your family will appreciate it, both in how you'll feel, and if you ever have to defend them.
I can't tell you how good Andrew from Defensive Fitness is. He's a true professional who loves his craft and gets serious results for his clients. Check him out at defensive-fitness.com