During the founding of our country the word freedom had a different meaning. Freedom in early America simply meant land ownership or at least being your own boss. If you had to work for someone else in the days of the open ballot, then politically you were not free. Your employer could insist you vote for their candidate and there were no laws stopping them from firing you if you did not. Hence, voting was restricted only to those who were free meaning only to those who owned their own land or business.
Historically Speaking 