I'm going to guess that you're using city water from the tap.
I get that every single time if I use heated city water from the tap. I believe it is the chlorine added to most city water.
By heated, I mean from warm but I can put my hand in it without it being too uncomfortable, to boiling hot. Flash rust every time when I run my first drying patch after cleaning.
Room temp or cold works fine for cleaning, but hot is a bit nicer for evaporating so the barrel dries faster.
But you can use room temp to cold water from the tap and most likely won't see that anymore.
If you want to use hot water but not see the flash rust, do one of two things. Buy distilled water by the gallon to have on hand to use, or bring your city water to a rolling boil for a solid 10 minutes to get rid of the chlorine. Both have worked 100% of the time for me so far.
Give it a try, I think you'll see that color go away.