I rode with a broken big toe a couple of summers ago. I tried on a bunch of shoes afterwards until I found one that seemed to protect it pretty well. It’s discussed in this thread.
It may not hurt to ride, but it’ll definitely hurt - and hurt the healing - to whack them again. I took the opportunity to work a lot on low-risk things like stillstands during the first couple of weeks of healing.