The slavery Charles Sumner was referring to in the mid 1800's was by the Barbary Pirates in north Africa to Morocco. This is different from what we typically think about slavery, primarily happening about the same time, but in USA, Caribbean, down to Brazil, where whites were the slave holders.