The Web has been voluntarily opt-out since the early to mid- 1990s.

Before that, there was no opt-ing out.

The act of putting something onto the (open) Web was (and still is) understood as consent for others to use your data.

This is the norm of the Web — and has been for decades.

I think it is unlikely this norm is going to change any time soon.

If you run a website and you think that adding anything to robots.txt "blocks" #ChatGPT or any other #WebCrawler from accessing your site, you may want to think that process through a bit more.

#RobotsDotTxt is *asking nicely* for the robot not to look places. If you don't trust the ethics of the scanning party, I'm not sure why you would think that asking them nicely not to mine your free content for their profit would even slow them down.