kenmirrors/MagicMirror
1
0
Fork 0
mirror of https://github.com/MichMich/MagicMirror.git synced 2026-06-28 11:18:26 -07:00
Code Issues Projects Releases Wiki Activity

[weather] refactor: migrate to server-side providers with centralized HTTPFetcher (#4032)

Browse Source 
This migrates the Weather module from client-side fetching to use the
server-side centralized HTTPFetcher (introduced in #4016), following the
same pattern as the Calendar and Newsfeed modules.

## Motivation

This brings consistent error handling and better maintainability and
completes the refactoring effort to centralize HTTP error handling
across all default modules.

Migrating to server-side providers with HTTPFetcher brings:
- **Centralized error handling**: Inherits smart retry strategies
(401/403, 429, 5xx backoff) and timeout handling (30s)
- **Consistency**: Same architecture as Calendar and Newsfeed modules
- **Security**: Possibility to hide API keys/secrets from client-side
- **Performance**: Reduced API calls in multi-client setups - one server
fetch instead of one per client
- **Enabling possible future features**: e.g. server-side caching, rate
limit monitoring, and data sharing with third-party modules

## Changes

- All 10 weather providers now use HTTPFetcher for server-side fetching
- Consistent error handling like Calendar and Newsfeed modules

## Breaking Changes

None. Existing configurations continue to work.

## Testing

To ensure proper functionality, I obtained API keys and credentials for
all providers that require them. I configured all 10 providers in a
carousel setup and tested each one individually. Screenshots for each
provider are attached below demonstrating their working state.

I even requested developer access from the Tempest/WeatherFlow team to
properly test this provider.

**Comprehensive test coverage**: A major advantage of the server-side
architecture is the ability to thoroughly test providers with unit tests
using real API response snapshots. Don't be alarmed by the many lines
added in this PR - they are primarily test files and real-data mocks
that ensure provider reliability.

## Review Notes

I know this is an enormous change - I've been working on this for quite
some time. Unfortunately, breaking it into smaller incremental PRs
wasn't feasible due to the interdependencies between providers and the
shared architecture.

Given the scope, it's nearly impossible to manually review every change.
To ensure quality, I've used both CodeRabbit and GitHub Copilot to
review the code multiple times in my fork, and both provided extensive
and valuable feedback. Most importantly, my test setup with all 10
providers working successfully is very encouraging.

## Related

Part of the HTTPFetcher migration #4016.

## Screenshots

<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-06-54"
src="https://github.com/user-attachments/assets/2139f4d2-2a9b-4e49-8d0a-e4436983ed6e"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-02"
src="https://github.com/user-attachments/assets/880f7ce2-4e44-42d5-bfe4-5ce475cca7c2"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-07"
src="https://github.com/user-attachments/assets/abd89933-fe03-40ab-8a7c-41ae1ff99255"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-12"
src="https://github.com/user-attachments/assets/22225852-f0a9-4d33-87ab-0733ba30fad3"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-17"
src="https://github.com/user-attachments/assets/7a7192a5-f237-4060-85d7-6f50b9bef5af"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-22"
src="https://github.com/user-attachments/assets/df84d9f1-e531-4995-8da8-d6f2601b6a08"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-27"
src="https://github.com/user-attachments/assets/4cf391ac-db43-4b52-95f4-f5eadc5ea34d"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-32"
src="https://github.com/user-attachments/assets/8dd8e688-d47f-4815-87f6-7f2630f15d58"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-37"
src="https://github.com/user-attachments/assets/ee84a8bc-6b35-405a-b311-88658d9268dd"
/>
<img width="1920" height="1080" alt="Ekrankopio de 2026-02-08 13-07-42"
src="https://github.com/user-attachments/assets/f941f341-453f-4d4d-a8d9-6b9158eb2681"
/>

Provider "Weather API" added later:

<img width="1910" height="1080" alt="Ekrankopio de 2026-02-15 19-39-06"
src="https://github.com/user-attachments/assets/3f0c8ba3-105c-4f90-8b2e-3a1be543d3d2"
/>
This commit is contained in:
Kristjan ESPERANTO
2026-02-23 10:27:29 +01:00
committed by GitHub
parent 80c48791b2
commit 8ce0cda7bf
79 changed files with 28056 additions and 3954 deletions
Download Patch File Download Diff File Expand all files Collapse all files
defaultmodules/utils.js
-152
View File
@@ -1,154 +1,3 @@
/**
* A function to make HTTP requests via the server to avoid CORS-errors.
* @param {string} url the url to fetch from
* @param {string} type what content-type to expect in the response, can be "json" or "xml"
* @param {boolean} useCorsProxy A flag to indicate
* @param {Array.<{name: string, value:string}>} requestHeaders the HTTP headers to send
* @param {Array.<string>} expectedResponseHeaders the expected HTTP headers to receive
* @param {string} basePath The base path, default is "/"
* @returns {Promise} resolved when the fetch is done. The response headers is placed in a headers-property (provided the response does not already contain a headers-property).
*/
async function performWebRequest (url, type = "json", useCorsProxy = false, requestHeaders = undefined, expectedResponseHeaders = undefined, basePath = "/") {
const request = {};
let requestUrl;
if (useCorsProxy) {
requestUrl = getCorsUrl(url, requestHeaders, expectedResponseHeaders, basePath);
} else {
requestUrl = url;
request.headers = getHeadersToSend(requestHeaders);
}
try {
const response = await fetch(requestUrl, request);
if (response.ok) {
const data = await response.text();
if (type === "xml") {
return new DOMParser().parseFromString(data, "text/html");
} else {
if (!data || !data.length > 0) return undefined;
const dataResponse = JSON.parse(data);
if (!dataResponse.headers) {
dataResponse.headers = getHeadersFromResponse(expectedResponseHeaders, response);
}
return dataResponse;
}
} else {
throw new Error(`Response status: ${response.status}`);
}
} catch (error) {
Log.error(`Error fetching data from ${url}: ${error}`);
return undefined;
}
}
/**
* Gets a URL that will be used when calling the CORS-method on the server.
* @param {string} url the url to fetch from
* @param {Array.<{name: string, value:string}>} requestHeaders the HTTP headers to send
* @param {Array.<string>} expectedResponseHeaders the expected HTTP headers to receive
* @param {string} basePath The base path, default is "/"
* @returns {string} to be used as URL when calling CORS-method on server.
*/
const getCorsUrl = function (url, requestHeaders, expectedResponseHeaders, basePath = "/") {
if (!url || url.length < 1) {
throw new Error(`Invalid URL: ${url}`);
} else {
let corsUrl = `${location.protocol}//${location.host}${basePath}cors?`;
const requestHeaderString = getRequestHeaderString(requestHeaders);
if (requestHeaderString) corsUrl = `${corsUrl}sendheaders=${requestHeaderString}`;
const expectedResponseHeadersString = getExpectedResponseHeadersString(expectedResponseHeaders);
if (requestHeaderString && expectedResponseHeadersString) {
corsUrl = `${corsUrl}&expectedheaders=${expectedResponseHeadersString}`;
} else if (expectedResponseHeadersString) {
corsUrl = `${corsUrl}expectedheaders=${expectedResponseHeadersString}`;
}
if (requestHeaderString || expectedResponseHeadersString) {
return `${corsUrl}&url=${url}`;
}
return `${corsUrl}url=${url}`;
}
};
/**
* Gets the part of the CORS URL that represents the HTTP headers to send.
* @param {Array.<{name: string, value:string}>} requestHeaders the HTTP headers to send
* @returns {string} to be used as request-headers component in CORS URL.
*/
const getRequestHeaderString = function (requestHeaders) {
let requestHeaderString = "";
if (requestHeaders) {
for (const header of requestHeaders) {
if (requestHeaderString.length === 0) {
requestHeaderString = `${header.name}:${encodeURIComponent(header.value)}`;
} else {
requestHeaderString = `${requestHeaderString},${header.name}:${encodeURIComponent(header.value)}`;
}
}
return requestHeaderString;
}
return undefined;
};
/**
* Gets headers and values to attach to the web request.
* @param {Array.<{name: string, value:string}>} requestHeaders the HTTP headers to send
* @returns {object} An object specifying name and value of the headers.
*/
const getHeadersToSend = (requestHeaders) => {
const headersToSend = {};
if (requestHeaders) {
for (const header of requestHeaders) {
headersToSend[header.name] = header.value;
}
}
return headersToSend;
};
/**
* Gets the part of the CORS URL that represents the expected HTTP headers to receive.
* @param {Array.<string>} expectedResponseHeaders the expected HTTP headers to receive
* @returns {string} to be used as the expected HTTP-headers component in CORS URL.
*/
const getExpectedResponseHeadersString = function (expectedResponseHeaders) {
let expectedResponseHeadersString = "";
if (expectedResponseHeaders) {
for (const header of expectedResponseHeaders) {
if (expectedResponseHeadersString.length === 0) {
expectedResponseHeadersString = `${header}`;
} else {
expectedResponseHeadersString = `${expectedResponseHeadersString},${header}`;
}
}
return expectedResponseHeaders;
}
return undefined;
};
/**
* Gets the values for the expected headers from the response.
* @param {Array.<string>} expectedResponseHeaders the expected HTTP headers to receive
* @param {Response} response the HTTP response
* @returns {string} to be used as the expected HTTP-headers component in CORS URL.
*/
const getHeadersFromResponse = (expectedResponseHeaders, response) => {
const responseHeaders = [];
if (expectedResponseHeaders) {
for (const header of expectedResponseHeaders) {
const headerValue = response.headers.get(header);
responseHeaders.push({ name: header, value: headerValue });
}
}
return responseHeaders;
};
/**
* Format the time according to the config
* @param {object} config The config of the module
@@ -178,6 +27,5 @@ const formatTime = (config, time) => {
};
if (typeof module !== "undefined") module.exports = {
performWebRequest,
formatTime
};