The top option “Bypass Duo authentication when offline (FailOpen)” is the setting that controls whether users can bypass active MFA and log into Windows if Duo’s service can’t be reached. Learn more about that setting in step 4 of setup here or in the FAQ item here.
The “Only prompt for Duo authentication when logging in via RDP” doesn’t distinguish between online and offline states. When it is enabled, then local console logins never prompt for Duo MFA, whether the system can contact Duo’s cloud service or not. When disabled, both local console logins and RDP logins prompt for Duo MFA.