Guys can father children at 90... But not so women.
Women who have never been pregnant prior to age 35 face a significant challenge. Women who have given birth in their younger years fare much better at getting pregnant beyond age 35, but for a first time pregnancy, it becomes increasingly less likely beyond 35, and often entails expensive medical mediation.
This is the facts... Saying its never too late is like saying anyone can become rich... It CAN happen, but is False hope for the vast majority.
Thus the OP is faced with either finding a woman who already has kids, and is willing to risk her aging eggs and his aging sprem on producing another healthy child...
In which case, his attitude towards her prior kids had better be pretty great...
Or, he has to find a significantly younger woman willing to mate with a much older guy... Not so bad if he goes for a 35 year old... Kinda rare if he tries for 20 year olds.
Most women who are attracted to much older men either have serious daddy issues, or they are seeking an older man because his attainments by that age offer security they value.
( yes... Sometimes a young woman goes for an older guy because he is just perfect for her and age isn't an issue... And SOMEONE wins the lottery every year, but that don't make lottery tickets a good retirement plan)
These are the facts... Frankly I agree with the sentiment that the woman who reneged on having more babies with him did so because it became clear he wasn't good father stock...
A conclusion supported by his sudden, very late, interest in family.
I have little sympathy for people who try so hard to be 18 when they ought to be 32. Because they ARE 32.
Adulthood starts at 18... Sometime soon after that milestone is when you are supposed to find a mate you ache for and are willing to step up and be responsible for.
If family is something you want... Don't count on the universe providing for you when you finally get around to it... You should be aware that women are designed to be best at and most interested in making babies in their 20s and 30s ( if not earlier).
That is reality... And starting your adult life at 40 is possible... But much much harder, and with fewer opportunities.
I know guys in their mid and late forties who decided to grow up at 40.
Every one of them is still single, every one of them still hunting for that younger woman who will overlook their primary character flaw... Being too self centered or immature to have committed to a woman, to a family, when they were younger.
The OP had better stop thinking about what he wants, and had better start focusing on what the woman willing to mate with him wants... And start working to be able to provide that for her.