There are different ways of being an atheist or an agnostic. A person can be agnostic in the sense that they don’t believe in the supernatural, but think that it might really exist. Or, a person can be an agnostic in the sense that they believe in some kind of supernatural realm, just not any described by the world’s religions.
Most atheists say they just don't believe in God. These atheists can also be called agnostics.
"But if I were to go on to say that, since my assertion cannot be disproved, it is intolerable presumption on the part of human reason to doubt it, I should rightly be thought to be talking nonsense. If, however, the existence of such a teapot were affirmed in ancient books, taught as the sacred truth every Sunday, and instilled into the minds of children at school, hesitation to believe in its existence would become a mark of eccentricity and entitle the doubter to the attentions of the psychiatrist in an enlightened age or of the Inquisitor in an earlier time. It is customary to suppose that, if a belief is widespread, there must be something reasonable about it. I do not think this view can be held by anyone who has studied history."
So, the argument goes, there may theoretically be a God, just as there may be a celestial teapot, but there is no reason to believe it. This is what is called "weak atheism," since it does not flatly refuse the possibility of there being a God.
The problem with the teapot argument is that it rests on the premise that there is no difference between the ideas of God and the celestial teapot. Yet, there is a difference. Unlike the notion of God, we can define teapots in empirical and logical terms. We can be agnostic about the teapot, because we know what it would mean for there to be a teapot in orbit around the Sun. (Of course, this presumes that the teapot is in some way measurable, even if our best telescopes cannot pick it up. Perhaps it emits a unique, herbal radiation.)
We have no way of understanding the notion of God, however, and so we cannot know what it would mean for there to be a God. There is no meaningful way to even discuss the possibility of God’s existence.
Thus, I cannot be a "teapot agnostic" about the existence of God. You could instead say I’m a theological noncognitivist, but I wouldn’t recommend introducing that phrase into many conversations. A simpler though less descriptive way to put it is that I'm a "strong atheist."