-
Notifications
You must be signed in to change notification settings - Fork 0
/
day02d-currency.qmd
240 lines (174 loc) · 6.95 KB
/
day02d-currency.qmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
---
title: "Exercise: Currency Live!"
format:
html:
toc: true
engine: knitr
webr:
packages: [jsonlite]
filters:
- webr
- pyodide
---
## Overview
In this exercise, we'll explore creating a dynamic document that once published will continually update based on upstream data without needing to be re-run.
We'll want to use a [public web API](https://github.com/public-api-lists/public-api-lists?tab=readme-ov-file) that does not require any API key and has both [HyperText Transfer Protocol Secure (HTTPS)](https://developer.mozilla.org/en-US/docs/Glossary/HTTPS) and [Cross-origin resource sharing (CORS)](https://developer.mozilla.org/en-US/docs/Web/HTTP/CORS) enabled. One such API is the [Frankfurter](https://www.frankfurter.app/docs/#historical) API that tracks foreign exchange references.
Just as before, we'll focus on using the `quarto-webr` extension. However, you can substitute `quarto-webr` with `quarto-pyodide` if you prefer working with Python.
## Step-by-step
### Web API Background
When working with web APIs, we have access to data by formatting a URL in a specific manner.
In this case, we can directly retrieve information related to a date range and a specific currency (e.g. `USD`) by using:
```sh
https://www.frankfurter.app/2024-05-01..?to=USD
```
- `2024-05-01`: denotes the start date in the format of `Year-Month-Day`.
- `..`: denotes a date range up to present day.
- `?to=USD`: denotes a filter to only focus on conversion from the Euro to USD.
- Optional:
- You can specify a closed date range using `2024-05-01..2024-05-03`
- You can omit `?to=` and receive all currencies.
### Downloading Data
When using WebAssembly distributions of R and Python, we have to be careful when working with remote data. In the case of R, this means either using `download.file()` or `url()` inside of `readLines()` to obtain data. Inside of `pyodide`, there are similar limitations; but, there is built-in support for handling this data call (c.f. [`pyodide-http`](https://github.com/koenvo/pyodide-http)).
::: {.panel-tabset group="programmingLanguage"}
#### R
For R's case, we have:
```{webr-r}
#| autorun: true
#| read-only: true
data_url = "https://www.frankfurter.app/2024-05-01..?to=USD"
download.file(data_url, "currency-data.json")
```
This saves the file contents into `currency-data.json`, which sits on webR's virtual file system. The response data we have is governed by:
```{webr-r}
#| context: output
cat(paste0(readLines("currency-data.json", warn=FALSE), collapse="\n"))
```
#### Python
Inside of Python, we need to make a request to download the data using the `requests` library.
:::{.callout-note}
The `quarto-pyodide` uses a special version of the `requests` library.
:::
```{pyodide-python}
import requests
# URL where JSON data is located
data_url = "https://www.frankfurter.app/2024-05-01..?to=USD"
# Fetch JSON data from the URL
response = requests.get(data_url)
data_dict = response.json()
# Show data
data_dict
```
::::
## Data Wrangling
From here, we need to data wrangle the JSON data. So, we'll likely want to include a code cell that loads the necessary JSON handling package and converts the JSON file into a data frame.
::: {.panel-tabset group="programmingLanguage"}
#### R
For R, the `jsonlite` package handles the conversion from JSON into an R list through `fromJSON()` function. The result should be saved into `currency_json`.
```{webr-r}
#| read-only: true
#| autorun: true
library(jsonlite)
currency_json <- fromJSON("currency-data.json")
```
We can further wrangle the data by using:
```{webr-r}
#| read-only: true
#| autorun: true
currency_df <- data.frame(
amount = currency_json$amount,
base = currency_json$base,
start_date = currency_json$start_date,
end_date = currency_json$end_date,
rates_date = as.Date(names(currency_json$rates)),
rate = unlist(lapply(currency_json$rates, function(x) x$USD))
)
head(currency_df)
```
#### Python
In Python, we only need to extract the `rates` key from the dictionary and use `pandas` construct the data frame containing rates and alongside a date.
:::{.callout-note}
We do drop a few variables with this approach.
:::
```{pyodide-python}
import pandas as pd
# Extract rates data
rates_data = data_dict['rates']
# Convert rates data into a DataFrame
df = pd.DataFrame(rates_data).T.reset_index()
df.columns = ['date', 'USD']
# Convert 'date' column to datetime type
df['date'] = pd.to_datetime(df['date'])
# Show final data frame
df.head()
```
:::
Once done, the problem defaults to figure out the best way to visualize and model this kind of time series data.
### Tasks
Create a Quarto document powered by either webR or Pyodide. The document should:
1. Download data starting from January 1st, 2024 and run to present day for the USD to EURO.
2. Create graphics to visualize the time series data.
- [R Graph Gallery](https://r-graph-gallery.com/)
- [Python Graph Gallery](https://python-graph-gallery.com/)
- [Graph Selector](https://www.data-to-viz.com/)
3. (Optional) Try to model the time series data.
Finally, publish the document on Quarto Pub.
### Extra
In the preceeding exercises, we only focused on working with the **EURO** to **USD** conversion rate. How can we improve the analysis (graphics / ) by allowing for all currencies rates to be present? e.g.
```sh
https://www.frankfurter.app/2024-05-01..
```
To help, here's some data wrangling code:
::: {.panel-tabset group="programmingLanguage"}
#### R
```{webr-r}
data_url = "https://www.frankfurter.app/2024-05-01.."
download.file(data_url, "currency-data-all.json")
library(jsonlite)
data_list <- fromJSON("currency-data-all.json")
# Initialize an empty dataframe
df <- data.frame()
# Iterate over each date and its rates
for (date in names(data_list$rates)) {
# Create a temporary dataframe for each date's rates
temp_df <- data.frame(
date = date,
currency = names(data_list$rates[[date]]),
rate = unlist(data_list$rates[[date]])
)
# Bind the temporary dataframe to the main dataframe
df <- rbind(df, temp_df)
}
# Add other columns
df$amount <- data_list$amount
df$base <- data_list$base
df$start_date <- data_list$start_date
df$end_date <- data_list$end_date
# Print the dataframe
head(df)
```
#### Python
```{pyodide-python}
import pandas as pd
import json
import requests
# URL where JSON data is located
# (Note, this URL may or may not cause issues!)
data_url = "https://www.frankfurter.app/2024-05-01.."
# Fetch JSON data from the URL
response = requests.get(data_url)
data_dict = response.json()
# Extract rates data
rates_data = data_dict['rates']
# Initialize an empty list to store rows
rows = []
# Iterate through the rates data and extract each date and its corresponding rates
for date, rates in rates_data.items():
for currency, rate in rates.items():
rows.append({'date': date, 'currency': currency, 'rate': rate})
# Convert the list of dictionaries into a DataFrame
df = pd.DataFrame(rows)
# Convert 'date' column to datetime type
df['date'] = pd.to_datetime(df['date'])
df.head()
```
:::