I occasionally eat ice cream. I have tonic or ginger ale with my vodka. Cheese cake is popular in circles I run in and I often indulge at family functions. Peanut butter aint exactly paleo but I do eat a spoonful every now and then. As far as strictness of calories consumed, I am probably 90% compliant.
Yes, I consume this stuff. I don't go hog wild and jump back on the Standard American Diet. I eat it, enjoy it, and don't stress about it. I am not in a weight loss mode so I may be able to tolerate this stuff better than some. I can tell you I have had 2 vacations recently where I had indulged more than usual and put on 5 pounds each time. It virtually falls off me when I get back to routinely eating my normal food and I suspect the weight gain is due to fluid retention more than anything.
I don't believe in cheat days. That is flying off the reservation for too long. Indulging once in a while is OK for me.