Deborah,
I also believe these are the same unit, just a different location description. I think there's only one 4th Regiment in the Massachusetts Militia. Uxbridge is in the central part of the state in Worcester County, and Plymouth County is on the East coast. Both were created prior to the Revolutionary War.