I'm pretty convinced that the Olivier that married Virginie Lebeau had Pierre/Peter and Josephte/Josephine as parents.
If you follow Pierre and Josephte, they're still in Saint-Ours in the 1861 Canada census (along with 15 year old Olivier; 4 year old Philomene, and other children). This has got to be the same family that shows in the 1870 US census in Sutton, Worcester, MA (indexed as "Labasea" on Ancestry). Pierre & family must have immigrated in the 1860s. Olivier and Virginie/Jane (indexed "Labarca") are married on the 1870 census *also in Sutton MA*.
Following Virginie backwards, you find her family on the 1860 census in Uxbridge, Worcester, MA as "Labo"; and her baptism in 1847 in Iberville, Bas-Canada. Her family was still in Canada as of the 1851 census, so they must have moved in the 1850s.
In short, the fact that Pierre and Josephte move to the same place we find Olivier and Virginie, together with Olivier (husband of Virginie's) listed birthplace and parents names on his death certificate, I'm convinced.
--
As for the Olivier Labossiere that married Amanda Daigle and Delima Lucier, as you described, this may be someone who was born with a different name. Do you have a copy of his 1926 Connecticut death certificate? If not, maybe look into ordering it - maybe it will illuminate something.