There is a bit to tackle in your response.
First, let us agree that the particular journey that this or that theist or atheist took to get to their position is completely irrelevant when assessing the question of truth and validity. How Einstein came up with Relativity is fascinating as a study of human ingenuity and psychology, but completely irrelevant to the question of the truth or predictive accuracy of Relativity. Likewise, how an atheist or a theist got there is irrelevant to the rational, cognitive question “should I be an atheist?”
Now, to what I think is relevant in your response. First, do people believe things that they know are not true… I think psychologists are not in consensus about this. There is a fascinating book called Vital Lies, Simple Truths that argues that we lie to ourselves all the time about ourselves. Kind of along the lines of the Prairie Home Companions' Lake Wobegon. But, I do not wish to argue about that. I hold that it is contentious, though, to claim we only believe things we think are true. I am a statistician; in applied statistical decision theory we sometimes make decisions using biased forecasts because of asymmetric loss functions. For example, I estimate that the plane will leave the airport before it’s scheduled departure because the cost of being early is less than the cost of being late. This is kind of like Pascal’s Wager, which we don’t need to talk about because we both will agree it has problems. But, this admits of a different interpretation: I forecast unbiased departure time, but have a utility function, preference ordering, that selects the action of arriving early. This gets into the problem of the circular inferences of the action-belief-desire triad. Only actions are observable, but the belief-desire pair is not mapped one-to-one to actions. We might like to assume that our interlocutor’s desire is to come to the truth, but, as your reference to mumpsimus suggests, even you are not quite certain the truth is my desire.
But, to your point that “and you should too” is a natural addendum to one’s beliefs, this means that the reasons need to be intersubjective, which I understand as the means by which we try to achieve objectivity. If the reasons are purely subjective, there is nothing stopping a person from devising entailment relations that have no intersubjective validity or plausibility (think of conspiracy theorists, I recall one in particular after one of the school shootings in the US that inferred that no kids were outside of the school while the shooting was taking place as a good reason to suppose it was all a fake, as though in a real shooting the kids would be sitting there in the school parking lot waiting to be shot). You may be interested in the literature surrounding Wittgenstein’s private language argument.
So, the reasons have to be intersubjectively valid or plausible. But, at this point, we have to concede that the reasons have to be for something more substantive than “my belief that…”. The reasons have to be for some claim about the intersubjective world (I honestly don’t care about what you believe, no offense, as I am sure you don’t care about what I believe). At this point, the atheists, as you define them, can be from one of two camps: the agnostic, who is going to make epistemic claims about the limits of human knowledge, and the position that is as yet unnamed that makes the ontological claim that there is no supernatural at all. The reasons each of these two will give have to be categorically different. Yet, they will both fall under atheism as you define it. In my book, conflating different positions under the same name is bad defining work. We have a different name for democratic socialists and social democrats because they are different political philosophies. We have a different name for classical logic and intuitionistic logic because they are different views of logical entailment. Defining atheism as “lacking a belief that…” conflates two positions that make mutually inconsistent claims. The agnostic claims the questions “is there a supernatural?” cannot be answered by humans. The other position says it can and it’s answer is “no”. Hence, I think the definition of atheism as lacking a belief is inadequate for rational purposes.