rtweet_user() 403 errors.

[Arf9999]

Problem

I have two twitter apps, both of which work with rtweet when authenticating using rtweet_app() and rtweet_bot(). However if I try to set up an interactive auth, I get 400 errors.

Expected behavior

When specifying the api_key and api_secret in an rtweet_user() call, I expect an auth token to be returned.

Reproduce the problem

authenticating using rtweet_bot. (keys and secret redacted)

> rtweet_bot(api_key = "xxxxxxxxxx", api_secret = "xxxxxxxxxxxxx", access_token = "xxxxxxxxxx", access_secret =  "xxxxxxx")
<Token>
<oauth_endpoint>
 request:   https://api.twitter.com/oauth/request_token
 authorize: https://api.twitter.com/oauth/authenticate
 access:    https://api.twitter.com/oauth/access_token
<oauth_app> rtweet
  key:    xxxxxxxxxxxxx
  secret: <hidden>
<credentials> oauth_token, oauth_token_secret
---

However, when using rtweet_user() with identical api_key and api_secret, which should start the process of interactive authorisation via a browser (unless I'm mistaken):

> rtweet_user(api_key = "xxxxxxxx", api_secret = "xxxxxxxx")
Error in httr::init_oauth1.0(endpoint, app, permission = permission, is_interactive = is_interactive,  : 
  Forbidden (HTTP 403).

Please note that exactly the same api_key and api-secret were used. I have replicated this with two separate twitter apps. Am I doing something stupid? Is it a setting that I've missed (I was able to get this type of auth to work on rtweet 0.7, so I don't think it is an issue with the twitter app settings)

rtweet version

## copy/paste output
> packageVersion("rtweet")
[1] ‘1.0.2’

Session info

## copy/paste output
> sessionInfo()
R version 4.1.3 (2022-03-10)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Monterey 12.6

Matrix products: default
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
 [1] magrittr_2.0.3         trundler_0.1.27        lubridate_1.8.0        forcats_0.5.1         
 [5] stringr_1.4.0          dplyr_1.0.8            purrr_0.3.4            readr_2.1.2           
 [9] tidyr_1.2.0            tibble_3.1.7           ggplot2_3.3.5          tidyverse_1.3.1       
[13] rtweet_1.0.2           rtweetXtras_0.4.1.0000

loaded via a namespace (and not attached):
 [1] httr_1.4.3        tidygraph_1.2.0   bit64_4.0.5       jsonlite_1.8.0    modelr_0.1.8     
 [6] assertthat_0.2.1  askpass_1.1       triebeard_0.3.0   urltools_1.7.3    cellranger_1.1.0 
[11] yaml_2.3.5        remotes_2.4.2     progress_1.2.2    pillar_1.7.0      backports_1.4.1  
[16] glue_1.6.2        digest_0.6.29     promises_1.2.0.1  gridtext_0.1.4    rvest_1.0.2      
[21] colorspace_2.0-3  httpuv_1.6.5      pkgconfig_2.0.3   broom_0.7.12      haven_2.4.3      
[26] shrtcts_0.1.1     scales_1.1.1      processx_3.5.2    later_1.3.0       tzdb_0.2.0       
[31] openssl_2.0.2     generics_0.1.2    farver_2.1.0      ellipsis_0.3.2    withr_2.5.0      
[36] cli_3.3.0         crayon_1.5.1      readxl_1.3.1      ps_1.6.0          ggtext_0.1.1     
[41] fs_1.5.2          fansi_1.0.3       xml2_1.3.3        pkgbuild_1.3.1    progressr_0.10.0 
[46] tools_4.1.3       prettyunits_1.1.1 hms_1.1.1         lifecycle_1.0.1   munsell_0.5.0    
[51] reprex_2.0.1      callr_3.7.0       compiler_4.1.3    rlang_1.0.2       grid_4.1.3       
[56] rstudioapi_0.13   rappdirs_0.3.3    igraph_1.2.11     labeling_0.4.2    gtable_0.3.0     
[61] DBI_1.1.2         roxygen2_7.1.2    curl_4.3.2        R6_2.5.1          knitr_1.37       
[66] bit_4.0.4         utf8_1.2.2        rprojroot_2.0.2   stringi_1.7.6     Rcpp_1.0.8.2     
[71] vctrs_0.4.1       dbplyr_2.1.1      tidyselect_1.1.2  xfun_0.30 

[llrs]

If I recall correctly, user authentication cannot use API credentials but I might be wrong.

Have you tried using just rtweet_user()? This will use the credentials of the authenticated user in your default browser.

Note for myself: The help page does not have a value section so it is not clear in the documentation what is the returned value of the functions documented there. I probably need to document better the rtweet_user system.

[Arf9999]

If I use retweet_user(), it'll authorise to the default rtweet twitter app, not so? That is problematic as I won't have control over rate limits.

[llrs]

Mmh, indeed. All the authentication process is a bit confusing to me but I'll check how to fix this and if there have been some changes in the authentication mechanism.

[llrs]

The validation error happens on the step 1 of the OAuth dance at the request_token step. The internal error response from Twitter is code 415: "Callback URL not approved for this client application. Approved callback URLs can be adjusted in your application settings." Reference

This means that I didn't set up correctly this configuration:
from my app at the project-apps portal.

I added a required field (type of app) and then I got a Client ID and a Client Secret. If instead of the API key and API key secret I use those I get a 401 error with internal twitter code 32: "Could not authenticate you". If your app worked for authenticating users before it might still work if you use the right client id and client secret.

Please let me know if you succeed and which callback urls do you have set up, as I might have hit a rate limit there. As per issue #251 I tried with the callback site I introduced for my app and with the default 127.0.0.1 but I ran into the same problem.

I won't be able to try it again in a week or so, but I think that instead of api_key and api_secret they might be better named as client_id and client_secret.

---

Check the slightly improved documentation on the devel branch (version 1.0.2.9010+1)

[llrs]

Hi @Arf9999, I hope to release a new version of rtweet soon, could you check if it works for your app in the latest version ? Remember using the client ID and client secret instead of the api key and api token? And if you could report back what IP have you set in the configuration page it might help me figure out what might be wrong. Thanks!

[llrs]

I am not sure if something changed but httr encodes the callback url making it impossible to validate the user (see r-lib/httr#732). It will need to wait until it is fixed upstream (I don't want to ship a modified version of httr just for this).

The same happens with oauth 2.0 in httr2 (r-lib/httr2#193) which is what prevents accessing the bookmarked endpoint in #344

[llrs]

Ok, so it seems that somehow I messed up with the authentication mechanism at this has gone through the release. It should work again in the devel branch (I also added an option to set your app name for rtweet_bot to distinguish from other rtweet apps). But I'll be testing this more extensively from now on, so not for immediate use yet (but I'm closing the issue).