There are numerous reasons that someone wouldn't be on a census record or you can't find them. 1950 was the very first census where people could fill out their own information instead of having someone else fill out the information. Should be interesting to see the 1950 census once it's released next year.
Anyway, prior to 1950 there were people hired to go around and fill out the requested information on the census. Since this person is highly likely an outsider that doesn't know the family, errors are pretty common. I've seen some people change genders between census records. One time I saw 8 year old Johnny listed as a male. But 18 year old Johnny after hitting puberty became female because 10 years later it was pretty obvious that Johnny was a female. 8 year old Johnny it wasn't as obvious. Fun fact, before about 1950 it was common to put young boys in a dress to make them look cute. This is how I'm sure errors are made among gender because the person filling out the information gets a brief visual on the person and assumes gender based upon name.
Also, if no one in the family was home when the person who was working the census, then they would simply move onto the next house. It wasn't worth the time to wait around for someone to come home. They may try again of course but I doubt people working the census would go more than a handful of times to the same house when no one was home.
Moving is another indicator. If someone packs up and moves during a census year they could either be missed completely or even counted in the census twice in both locations.
And yes, I feel your pain on census records. Take a look at my ancestor Butcher-4229
Born in Missouri he married, had a few children, then moved to Louisiana around 1902 or 1903. The family should be in the 1900 census in Missouri. Yet they are nowhere to be found. His wife supposedly lived until after the 1930's. She should be somewhere in the census records. Yet she cannot be found in 1930.
Then with my last name, Kerbow, we run into multiple other problems. There are numerous ways to spell Kerbow and they all come from, according to family legend, 3 brothers who came to America from France. Kerbo, Curbo, Curbow, Kerbi, Kerby, Corbeau, Corbo, Corbeaux, Kerby (though I'm not sure if Kerby is part of the ancestral line) and Kierbow are common to bounce around between census records from one decade to the next.