Even without external bad things it’s still not great. You still have the possibility of conscription into the guard. You still have a somewhat feudal government system. Depending on the dominant faction in the sector, you have to deal with their beliefs, on top of not ever appearing heretical. Punishment for heresy can include entire family trees, or entire blocks.

On top of all that, you still have the constant threat of warp storms. They could consume the planet or just block travel to your system. Either is likely a death sentence.