[ci] make shell scripts stricter

[jameslamb]

Proposes standardizing on 2 practices for shell scripts in this project:

• using #!/bin/bash (not sh or other shells)
• running shell scripts under stricter settings, to get earlier and louder errors

Benefits of these changes

Prevents situations where tests are silently failing... like the rchk tests have been for over a year (#6332).

Reduces debugging time by ensuring that CI logs tend to end with the first thing that was broken. For example, no more brew install silently failing and then only finding out about it via cryptic errors from R CMD check.

Notes for Reviewers

Per the set man page (link):

      -e  Exit immediately if a command exits with a non-zero status.
      -o option-name
          Set the variable corresponding to option-name:
              allexport    same as -a
              ...
              pipefail     the return value of a pipeline is the status of
                           the last command to exit with a non-zero status,
                           or zero if no command exited with a non-zero status
              ...
      -u  Treat unset variables as an error when substituting.

I've kept the following scripts using sh, so they can be executed in the widest possible range of environments (including Windows), without needing to worry about setting up bash (which tends to be less common on non-Linux platforms than sh):

• build-python.sh
• build-cran-package.sh
• .ci/check_python_dists.sh

[jameslamb]

This rchk job has been silently broken for over a year.

I'm working on fixing it in #6332 . See that PR for details.

In this PR, I'm proposing removing it entirely to save some CI time and to not have to introduce complexity to workaround its issues as a part of this PR.

[borchero]

Nice, thanks!