Going paleo means stocking your kitchen with some new ingredients.  Your first trip the grocery store will not be your cheapest purchase, but keep in mind that you will not be buying these things every time you shop.  Having the following things on hand makes cooking gourmaleo a little easier:

coconut milk

coconut flour

coconut oil

almond meal

almond butter

almond milk

dates

golden raisins

chicken and beef stock

olive oil

Investing in some extra equipment might make your life easier as well. 

Slow Cooker

Food Processor (the best $200 investment I have ever made!)

Blender

Roasting Pan

A good non-stick pan