I see 16 Cameron households enumerated in the 1841 Census of PEI, but no children are listed. Like you, I don't see anyone by the name of Samuel Cameron in the baptismal records in that period, but they are by no means complete. There are lots of reasons for that: it was a tiny little colony with a severe shortage of clergy, and high rates of illiteracy. One of my ancestors, as a deacon, performed marriages, baptisms and funerals in his neighborhood literally for decades, waiting for someone qualified to arrive. If the person or people who stepped in for the clergy in your family's community wasn't literate, or wasn't organized, there may not be a birth or baptismal record.
Many of the records that do exist were compiled retroactively by residents in later years (I'd say 1860 and later) when there were more clergy, mandatory education, and a law requiring record-keeping wasn't put into place until after PEI joined Canada in 1873. So if Samuel Cameron born 1823 left PEI before marriage, buying land etc, then he may not have left many footprints in the usual records there.
But I wouldn't give up hope. I understand there may be genealogical records in the paper archives that haven't been put online. For example, the local newspapers published pretty much anything they were given, such as a birth, and if Samuel ever corresponded with his family back home, I've also seen the newspapers of that time publish updates about residents who had gone off on adventures, like to Australia. I suggest contacting the PEI Public Archives to see what arrangements can be made for remote lookup of newspapers and any other records that they can suggest: https://www.princeedwardisland.ca/en/service/search-public-archives-material-online
