My tomatoes develop black spots after harvesting

Pretty soon after I harvest my homegrown tomatoes and bring them inside, they tend to develop black or dark spots, which start mostly on the sides, and some of which look like something like a mold is growing on the spots. The spots can grow deep into the fruit. They spread over the tomato and I believe they spread to tomatoes sitting nearby. The plants look reasonably ok and the spots don't appear on the tomatoes until a little while after I bring them inside.

I neither wash nor refrigerate my tomatoes. Any ideas on how to avoid these spots?
Thanks!!!