Jeremiah Haralson was born near Columbus, Georgia on April 1, 1846. The slave of Georgia planter John Haralson, he was taken to Alabama where he remained in bondage until 1865. It is unclear as to what he did in the earlier years of his freedom, but there are records that suggest he may have been a farmer and clergyman. Haralson taught himself to read and write and later became a skilled orator and debater.

In 1868, Haralson made his first unsuccessful attempt for a seat in the Forty-first Congress, representing Alabama’s First District of Alabama.  Two years later he won a seat in Alabama’s House of Representatives and in 1872 was elected to the State Senate.  In 1874 Haralson again ran for the U.S. House of Representatives.  Haralson narrowly won the Republican primary over Liberal Republican Frederick G. Bromberg.  Soon after the primary Bromberg accused Haralson of voter fraud and sought to deny him his seat.  The Democrats who controlled the U.S. House of Representatives supported Haralson and on March 4, 1875 he took his seat in Congress.   

Congressman Jeremiah Haralson supported the policies of President Ulysses Grant and urged black voters to remain loyal to the Republican Party.  Appointed to the House Committee on Public expenditures, he introduced legislation to use proceeds from public land sales for educational purposes and for the relief of the Medical College of Alabama. Haralson broke with other Republican-era black Congressmen by criticizing the use of federal soldiers to control violence and ensure orderly voting in the South.  He also favored general amnesty for former Confederates.

In 1876 Haralson stood for election in the newly created, predominately black Fourth Congressional District.  This district was already represented by black Congressman James T. Rapier.  Both he and Haralson ran in the Republican Primary.  Haralson won the primary but lost narrowly to Democrat Charles M. Shelley.  Haralson contested the results before the House of Representatives but on this occasion, the