it depends what sort of t-test you want to do (one sided or two sided dependent or independent) but it should be as simple as:
from scipy.stats import ttest_ind
cat1 = my_data[my_data['Category']=='cat1']
cat2 = my_data[my_data['Category']=='cat2']
ttest_ind(cat1['values'], cat2['values'])
>>> (1.4927289925706944, 0.16970867501294376)
it returns a tuple with the t-statistic & the p-value
see here for other t-tests http://docs.scipy.org/doc/scipy/reference/stats.html