An African-American pastor of Freedom House Church in Charlotte, N.C. church came under over the weekend for requesting only white people