Joseph T. Ball, Jr. was born Feb. 21, 1804 in Cambridge, Massachusetts. His mother Mary Montgomery Drew of Cambridge, Massachusetts, was white while his father was Jamaican-born Joseph T. Ball, Sr who came to Massachusetts in 1790. The elder Ball founded a society to help African American widows