ageng/ClearUrls
1
0
Fork 0
mirror of https://gitlab.com/KevinRoebert/ClearUrls synced 2025-12-19 07:35:36 +07:00
Code Issues Projects Releases Wiki Activity

Merge branch 'master' into next-gen

Browse Source
This commit is contained in:
Kevin Röbert
2020-03-20 20:18:37 +01:00
parent fa1c3b35ac 6ee6591019
commit b602b0431a
23 changed files with 700 additions and 166 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
CHANGELOG.md
View File

@@ -4,6 +4,30 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [1.16.0] - 2020-03-20
### Compatibility note
- Require Firefox >= 55
- Require Chrome >= 22
### Added
- Added ETag header filtering [#362](https://gitlab.com/KevinRoebert/ClearUrls/issues/362), [#440](https://gitlab.com/KevinRoebert/ClearUrls/issues/440). Hint: Cache must be cleared before first use, to delete the already existing ETags.
### Fixed
- Fixed spontaneous disappearance of the badged
- Fixed wrong counting of blocked elements (too little was ;D)
### Changed
- Updated all translation
- Changed badged font color to #FFFFFF
- Changed watchdog behavior as follows [#428](https://gitlab.com/KevinRoebert/ClearUrls/issues/428), [#431](https://gitlab.com/KevinRoebert/ClearUrls/issues/431), [#429](https://gitlab.com/KevinRoebert/ClearUrls/issues/429):
- Increased watchdog interval to 60 seconds
- Executed watchdog only if ClearURLs is also active
- Watchdog restarts ClearURLs at most 3 times and only if ClearURLs has rules
- Changed behavior of downloading rules [#428](https://gitlab.com/KevinRoebert/ClearUrls/issues/428), [#431](https://gitlab.com/KevinRoebert/ClearUrls/issues/431), [#429](https://gitlab.com/KevinRoebert/ClearUrls/issues/429):
- If download of hash file fails and no local rules are available, then ClearURLs displays `hash_status_code_5` and deactivates itself
- If download of rules file fails and no local rules are available, then ClearURLs displays `hash_status_code_5` and deactivates itself
## [1.15.0] - 2020-02-16
### Compatibility note

3
README.md
View File

@@ -1,6 +1,6 @@
<a href="https://www.buymeacoffee.com/KevinRoebert" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/default-orange.png" alt="Buy Me A Coffee" height="51" width="217"></a>
[<img src="https://addons.cdn.mozilla.net/static/img/addons-buttons/AMO-button_1.png" alt="for Firefox">](https://addons.mozilla.org/en-US/firefox/addon/clearurls/) [<img src="https://developer.chrome.com/webstore/images/ChromeWebStore_BadgeWBorder_v2_206x58.png" alt="for Chrome">](https://chrome.google.com/webstore/detail/clearurls/lckanjgmijmafbedllaakclkaicjfmnk)
[<img src="https://addons.cdn.mozilla.net/static/img/addons-buttons/AMO-button_1.png" alt="for Firefox">](https://addons.mozilla.org/en-US/firefox/addon/clearurls/) [<img src="https://gitlab.com/KevinRoebert/ClearUrls/-/raw/master/promotion/MEA-button.png" alt="for Edge">](https://microsoftedge.microsoft.com/addons/detail/mdkdmaickkfdekbjdoojfalpbkgaddei) [<img src="https://developer.chrome.com/webstore/images/ChromeWebStore_BadgeWBorder_v2_206x58.png" alt="for Chrome">](https://chrome.google.com/webstore/detail/clearurls/lckanjgmijmafbedllaakclkaicjfmnk)
# <sub><img src="https://gitlab.com/KevinRoebert/ClearUrls/raw/master/img/clearurls.svg" width="64px" height="64px"></sub> ClearURLs
@@ -29,6 +29,7 @@ Indeed most of the above URL is tracking code. Once ClearURLs has cleaned the ad
* Supports redirection to the destination, without tracking services as middleman
* Adds an entry to the context menu so that links can be copied quickly and cleanly
* Blocks hyperlink auditing, also known as *ping tracking* (see also [this article](https://html.spec.whatwg.org/multipage/links.html#hyperlink-auditing))
* Prevents ETag tracking
* Prevents tracking injection over history API (see also: [The replaceState() method](https://developer.mozilla.org/en-US/docs/Web/API/History_API#The_replaceState()_method))
* Prevents Google from rewriting the search results (to include tracking elements)
* Prevents Yandex from rewriting the search results (to include tracking elements)

22
_locales/de/messages.json
View File

@@ -306,8 +306,14 @@
"description": "Diese Zeichenkette wird als Titel für das Referral-Marketing verwendet."
},
"watchdog": {
"message": "[ClearURLs]: Der Watchdog hat ein Problem festgestellt. ClearURLs wird neugestartet.",
"description": "Diese Zeichenkette wird als Text für den Watchdog verwendet."
"message": "[ClearURLs]: Der Watchdog hat ein Problem festgestellt und ist bereits das $TIMES$-te mal gescheitert.",
"description": "Diese Zeichenkette wird als Text für den Watchdog verwendet.",
"placeholders": {
"times": {
"content": "$1",
"example": "3"
}
}
},
"domain_blocking_enabled": {
"message": "Erlaube Domain-Blocking (Kann zu Problemen auf Seiten führen, die AdBlocker nicht erlauben)",
@@ -328,5 +334,17 @@
"ping_blocking_enabled_title": {
"message": "Blockiere Hyperlink Auditing Anfragen",
"description": "Diese Zeichenkette wird als Titel für das Hyperlink Auditing-Blocking verwendet."
},
"eTag_filtering_log": {
"message": "ETag-Header wurde aus dieser Anfrage entfernt",
"description": "Diese Zeichenkette wird für das Filtern von ETag-Headern im ClearURL-Protokoll verwendet."
},
"eTag_filtering_enabled": {
"message": "Filtert die ETag-Header aus den Anfragen (Siehe auch <a href='https://en.wikipedia.org/wiki/HTTP_ETag#Tracking_using_ETags' target='_blank'>diesen Artikel</a>)",
"description": "Diese Zeichenkette wird als Beschreibung für das Filtern von ETag-Headern verwendet."
},
"eTag_filtering_enabled_title": {
"message": "Filtert ETag-Header. Info: Cache muss vor der ersten Benutzung geleert werden.",
"description": "Diese Zeichenkette wird als Titel für das Filtern von ETag-Headern verwendet."
}
}

22
_locales/en/messages.json
View File

@@ -306,8 +306,14 @@
"description": "This string is used as title for the referral marketing switch"
},
"watchdog": {
"message": "[ClearURLs]: The watchdog has detected a problem. ClearURLs is restarting.",
"description": "This string is used as text for the watchdog"
"message": "[ClearURLs]: The watchdog has detected a problem and has already failed $TIMES$ times.",
"description": "This string is used as text for the watchdog",
"placeholders": {
"times": {
"content": "$1",
"example": "3"
}
}
},
"domain_blocking_enabled": {
"message": "Allow domain blocking (Can lead to problems on pages that do not allow AdBlockers)",
@@ -328,5 +334,17 @@
"ping_blocking_enabled_title": {
"message": "Block hyperlink auditing",
"description": "This string is used as title for the hyperlink auditing blocking switch"
},
"eTag_filtering_log": {
"message": "ETag header was removed from this request",
"description": "This string is used on ETag header filtering in the ClearURLs log."
},
"eTag_filtering_enabled": {
"message": "Filters ETag headers from requests (See also <a href='https://en.wikipedia.org/wiki/HTTP_ETag#Tracking_using_ETags' target='_blank'>this article</a>)",
"description": "This string is used as label for the ETag header filtering switch"
},
"eTag_filtering_enabled_title": {
"message": "Filters ETag headers. Hint: Cache must be cleared before first use.",
"description": "This string is used as title for the ETag header filtering switch"
}
}

4
_locales/es/messages.json
View File

@@ -301,10 +301,6 @@
"message": "Permitir márketing de referidos",
"description": "This string is used as title for the referral marketing switch"
},
"watchdog": {
"message": "[ClearURLs]: El watchdog detectado un problema. ClearURLs será reiniciado.",
"description": "This string is used as text for the watchdog"
},
"domain_blocking_enabled": {
"message": "Permitir bloqueo de dominios (Puede provocar problemas con sitios que no permitan bloqueadores de anuncios)",
"description": "This string is used as label for the domain blocking switch"

4
_locales/fr/messages.json
View File

@@ -301,10 +301,6 @@
"message": "Permettre certaines actions commerciales",
"description": "Sert à afficher la signification de la fonction sur la page qui affiche les options."
},
"watchdog": {
"message": "[ClearURLs]: Temps ou délais d’exécution imparti dépassé. ClearURLs doit redémarrer.",
"description": "Sert à afficher la signification de la fonction sur la page qui affiche les options."
},
"domain_blocking_enabled": {
"message": "Bloquer les atteintes à la vie privée. Cependant, des sites décèlent la technique de protection et dysfonctionnent délibérément ou affichent des avertissements",
"description": "Sert à afficher la signification de la fonction sur la page qui affiche les options."

4
_locales/it/messages.json
View File

@@ -301,10 +301,6 @@
"message": "Permette l'utilizzo del referral marketing (esempio: Amazon)",
"description": "This string is used as title for the referral marketing switch"
},
"watchdog": {
"message": "[ClearURLs]: Il sistema di controllo ha identificato un problema. ClearURLs verrà riavviato.",
"description": "This string is used as text for the watchdog"
},
"domain_blocking_enabled": {
"message": "Permetti il blocco dei domini (può portare a problemi su pagine che non consentono AdBlocker)",
"description": "This string is used as label for the domain blocking switch"

4
_locales/ru/messages.json
View File

@@ -301,10 +301,6 @@
"message": "Разрешить рефералы маркетинга",
"description": "This string is used as title for the referral marketing switch"
},
"watchdog": {
"message": "[ClearURLs]: сторожевой таймер обнаружил проблему. ClearURLs будет перезапущен.",
"description": "This string is used as text for the watchdog"
},
"domain_blocking_enabled": {
"message": "Разрешить блокировку домена (может вызвать проблемы на сайтах, которые запрещают AdBlocker)",
"description": "This string is used as label for the domain blocking switch"

4
_locales/sv_SE/messages.json
View File

@@ -301,10 +301,6 @@
"message": "Tillåt referral marketing",
"description": "This string is used as title for the referral marketing switch"
},
"watchdog": {
"message": "[ClearURLs]: Ett problem har upptäckts. ClearURLs startar om.",
"description": "This string is used as text for the watchdog"
},
"domain_blocking_enabled": {
"message": "Tillåt domänblockering (Kan leda till problem på sidor som inte tillåter AdBlockers)",
"description": "This string is used as label for the domain blocking switch"

4
_locales/tr/messages.json
View File

@@ -301,10 +301,6 @@
"message": "Tavsiye pazarlamasına izin verir",
"description": "Bu dize tavsiye pazarlama anahtarı için başlık olarak kullanılır"
},
"watchdog": {
"message": "[ClearURL'ler]: Güvenlik Zamanlayıcısı bir sorun tespit etti. ClearURL'ler yeniden başlatıldı.",
"description": "Bu dize güvenlik zamanlayıcısı için metin olarak kullanılır."
},
"domain_blocking_enabled": {
"message": "Alan adı engellemeye izin ver (Reklam Engelleyicilere izin vermeyen sayfalarda sorunlara neden olabilir)",
"description": "Bu dize, alan adı engelleme anahtarı için etiket olarak kullanılır"

128
clearurls.js
View File

@@ -22,13 +22,10 @@
*/
var providers = [];
var prvKeys = [];
var badges = [];
var tabid = 0;
var siteBlockedAlert = 'javascript:void(0)';
var dataHash;
var localDataHash;
var os;
var currentURL;
/**
* Helper function which remove the tracking fields
@@ -37,9 +34,10 @@ var currentURL;
* @param {Provider} provider Provider-Object
* @param {string} pureUrl URL as String
* @param {boolean} quiet If the action should be displayed in log and statistics
* @param {requestDetails} request The request details
* @return {object} Object with changes and url fields
*/
function removeFieldsFormURL(provider, pureUrl, quiet = false) {
function removeFieldsFormURL(provider, pureUrl, quiet = false, request = null) {
let url = pureUrl;
let domain = "";
let fragments = "";
@@ -70,7 +68,7 @@ function removeFieldsFormURL(provider, pureUrl, quiet = false) {
pushToLog(beforeReplace, url, rawRule);
}
increaseBadged(quiet);
increaseBadged(quiet, request);
changes = true;
}
});
@@ -91,7 +89,11 @@ function removeFieldsFormURL(provider, pureUrl, quiet = false) {
url = decodeURL(re);
//Log the action
if (!quiet) pushToLog(pureUrl, url, translate('log_redirect'));
if (!quiet) {
pushToLog(pureUrl, url, translate('log_redirect'));
increaseGlobalURLCounter(1);
increaseBadged(false, request);
}
return {
"redirect": true,
@@ -131,7 +133,7 @@ function removeFieldsFormURL(provider, pureUrl, quiet = false) {
if (!quiet) pushToLog(tempBeforeURL, tempURL, rule);
}
increaseBadged(quiet);
increaseBadged(quiet, request);
changes = true;
}
});
@@ -146,7 +148,8 @@ function removeFieldsFormURL(provider, pureUrl, quiet = false) {
if (provider.isCaneling() && storage.domainBlocking) {
if (!quiet) pushToLog(pureUrl, pureUrl, translate('log_domain_blocked'));
increaseBadged(quiet);
increaseGlobalURLCounter(1);
increaseBadged(quiet, request);
cancel = true;
}
@@ -228,13 +231,26 @@ function start() {
}
/**
* Get the hash for the rule file on github.
* Deactivates ClearURLs, if no rules can be downloaded and also no old rules in storage
*/
function deactivateOnFailure() {
if(storage.ClearURLsData.length === 0) {
storage.globalStatus = false;
storage.dataHash = "";
changeIcon();
storeHashStatus(5);
saveOnExit();
}
}
/**
* Get the hash for the rule file on GitLab.
* Check the hash with the hash form the local file.
* If the hash has changed, then download the new rule file.
* Else do nothing.
*/
function getHash() {
//Get the target hash from github
//Get the target hash from GitLab
fetch(storage.hashURL)
.then(function (response) {
const responseTextHash = response.clone().text().then(function (responseTextHash) {
@@ -250,6 +266,7 @@ function start() {
}
} else {
dataHash = false;
deactivateOnFailure();
}
});
});
@@ -279,6 +296,8 @@ function start() {
storage.ClearURLsData = JSON.parse(storage.ClearURLsData);
toObject(storage.ClearURLsData);
saveOnDisk(['ClearURLsData', 'dataHash', 'hashStatus']);
} else {
deactivateOnFailure();
}
});
}
@@ -559,7 +578,8 @@ function start() {
if (storage.pingBlocking && storage.pingRequestTypes.includes(request.type)) {
pushToLog(request.url, request.url, translate('log_ping_blocked'));
increaseBadged();
increaseBadged(false, request);
increaseGlobalURLCounter(1);
return {cancel: true};
}
@@ -567,9 +587,8 @@ function start() {
* Call for every provider the removeFieldsFormURL method.
*/
for (let i = 0; i < providers.length; i++) {
if (providers[i].matchURL(request.url)) {
result = removeFieldsFormURL(providers[i], request.url);
result = removeFieldsFormURL(providers[i], request.url, false, request);
}
/*
@@ -629,39 +648,6 @@ function start() {
getHash();
setBadgedStatus();
/**
* Call by each tab is updated.
* And if url has changed.
*/
function handleUpdated(tabId, changeInfo, tabInfo) {
if (changeInfo.url) {
delete badges[tabId];
}
currentURL = tabInfo.url;
}
/**
* Call by each tab is updated.
*/
browser.tabs.onUpdated.addListener(handleUpdated);
/**
* Call by each tab change to set the actual tab id
*/
function handleActivated(activeInfo) {
tabid = activeInfo.tabId;
browser.tabs.get(tabid).then(function (tab) {
if (!browser.runtime.lastError) { // https://gitlab.com/KevinRoebert/ClearUrls/issues/346
currentURL = tab.url;
}
}).catch(handleError);
}
/**
* Call by each tab change.
*/
browser.tabs.onActivated.addListener(handleActivated);
/**
* Check the request.
*/
@@ -697,53 +683,3 @@ function start() {
["blocking"]
);
}
/**
* Function to log all activities from ClearUrls.
* Only logging when activated.
* The log is only temporary saved in the cache and will
* permanently saved with the saveLogOnClose function.
*
* @param beforeProcessing the url before the clear process
* @param afterProcessing the url after the clear process
* @param rule the rule that triggered the process
*/
function pushToLog(beforeProcessing, afterProcessing, rule) {
const limit = storage.logLimit;
if (storage.loggingStatus && limit !== 0) {
if (limit > 0 && !isNaN(limit)) {
while (storage.log.log.length >= limit) {
storage.log.log.shift();
}
}
storage.log.log.push(
{
"before": beforeProcessing,
"after": afterProcessing,
"rule": rule,
"timestamp": Date.now()
}
);
deferSaveOnDisk('log');
}
}
/**
* Increases the badged by one.
*/
function increaseBadged(quiet = false) {
if (badges[tabid] == null) badges[tabid] = 0;
if (!quiet) increaseURLCounter();
checkOSAndroid().then((res) => {
if (!res) {
if (storage.badgedStatus && !quiet) {
browser.browserAction.setBadgeText({text: (++badges[tabid]).toString(), tabId: tabid}).catch(handleError);
} else {
browser.browserAction.setBadgeText({text: "", tabId: tabid}).catch(handleError);
}
}
});
}

77
core_js/badgedHandler.js Normal file
View File

@@ -0,0 +1,77 @@
/*
* ClearURLs
* Copyright (c) 2017-2020 Kevin Röbert
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
/*jshint esversion: 6 */
/*
* This script is responsible for setting the badged.
*/
let badges = {};
/**
* Increases the badged by one.
*/
function increaseBadged(quiet = false, request) {
if (!quiet) increaseURLCounter();
if(request === null) return;
const tabId = request.tabId;
const url = request.url;
if(tabId === -1) return;
if (badges[tabId] == null) {
badges[tabId] = {
counter: 1,
lastURL: url
};
} else {
badges[tabId].counter += 1;
}
checkOSAndroid().then((res) => {
if (!res) {
if (storage.badgedStatus && !quiet) {
browser.browserAction.setBadgeText({text: (badges[tabId]).counter.toString(), tabId: tabId}).catch(handleError);
} else {
browser.browserAction.setBadgeText({text: "", tabId: tabId}).catch(handleError);
}
}
});
}
/**
* Call by each tab is updated.
* And if url has changed.
*/
function handleUpdated(tabId, changeInfo, tabInfo) {
if(!badges[tabId] || !changeInfo.url) return;
if (badges[tabId].lastURL !== changeInfo.url) {
badges[tabId] = {
counter: 0,
lastURL: tabInfo.url
};
}
}
/**
* Call by each tab is updated.
*/
browser.tabs.onUpdated.addListener(handleUpdated);

45
core_js/eTagFilter.js Normal file
View File

@@ -0,0 +1,45 @@
/*
* ClearURLs
* Copyright (c) 2017-2020 Kevin Röbert
*
* This program is free software: you can redistribute it and/or modify
* it under the terms of the GNU Lesser General Public License as published by
* the Free Software Foundation, either version 3 of the License, or
* (at your option) any later version.
*
* This program is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public License
* along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
/*jshint esversion: 6 */
/**
* Filters eTag headers from web requests.
*/
function eTagFilter(requestDetails) {
if(!requestDetails.responseHeaders || !storage.eTagFiltering) return {};
const responseHeaders = requestDetails.responseHeaders;
const filteredHeaders = responseHeaders.filter(header => {
return header.name.toLowerCase() !== "etag";
});
if(filteredHeaders.length < responseHeaders.length) {
pushToLog(requestDetails.url, requestDetails.url, translate("eTag_filtering_log"));
increaseBadged(false, requestDetails);
increaseGlobalURLCounter(1);
return {responseHeaders: filteredHeaders};
}
}
browser.webRequest.onHeadersReceived.addListener(
eTagFilter,
{urls: ["<all_urls>"]},
["blocking", "responseHeaders"]
);

4
core_js/settings.js
View File

@@ -115,6 +115,7 @@ function getData()
.then(() => loadData("referralMarketing"))
.then(() => loadData("domainBlocking"))
.then(() => loadData("pingBlocking"))
.then(() => loadData("eTagFiltering"))
.then(() => {
changeSwitchButton("localHostsSkipping", "localHostsSkipping");
changeSwitchButton("historyListenerEnabled", "historyListenerEnabled");
@@ -122,6 +123,7 @@ function getData()
changeSwitchButton("referralMarketing", "referralMarketing");
changeSwitchButton("domainBlocking", "domainBlocking");
changeSwitchButton("pingBlocking", "pingBlocking");
changeSwitchButton("eTagFiltering", "eTagFiltering");
}).catch(handleError);
}
@@ -189,6 +191,8 @@ function setText()
injectText("domain_blocking_enabled", "domain_blocking_enabled");
$('#ping_blocking_enabled').html(translate('ping_blocking_enabled'))
.prop('title', translate('ping_blocking_enabled_title'));
$('#eTag_filtering_enabled').html(translate('eTag_filtering_enabled'))
.prop('title', translate('eTag_filtering_enabled_title'))
}
/**

6
core_js/storage.js
View File

@@ -202,6 +202,8 @@ function initSettings() {
storage.logLimit = -1;
storage.domainBlocking = true;
storage.pingBlocking = true;
storage.eTagFiltering = true;
storage.watchDogErrorCount = 0;
if (getBrowser() === "Firefox") {
storage.types = ["font", "image", "imageset", "main_frame", "media", "object", "object_subrequest", "other", "script", "stylesheet", "sub_frame", "websocket", "xbl", "xml_dtd", "xmlhttprequest", "xslt"];
@@ -268,6 +270,10 @@ function storeHashStatus(status_code) {
case 3:
status_code = "hash_status_code_3";
break;
case 5:
status_code = "hash_status_code_5";
break;
case 4:
default:
status_code = "hash_status_code_4";
}

73
core_js/tools.js
View File

@@ -48,9 +48,11 @@ function isEmpty(obj) {
* Translate a string with the i18n API.
*
* @param {string} string Name of the attribute used for localization
* @param {string[]} placeholders Array of placeholders
*/
function translate(string) {
return browser.i18n.getMessage(string);
function translate(string, ...placeholders)
{
return browser.i18n.getMessage(string, placeholders);
}
/**
@@ -181,32 +183,6 @@ function loadOldDataFromStore() {
localDataHash = storage.dataHash;
}
/**
* Save the hash status to the local storage (RAM).
* The status can have the following values:
* 1 "up to date"
* 2 "updated"
* 3 "update available"
* @param status_code the number for the status
*/
function storeHashStatus(status_code) {
switch (status_code) {
case 1:
status_code = "hash_status_code_1";
break;
case 2:
status_code = "hash_status_code_2";
break;
case 3:
status_code = "hash_status_code_3";
break;
default:
status_code = "hash_status_code_4";
}
storage.hashStatus = status_code;
}
/**
* Increase by {number} the GlobalURLCounter
* @param {int} number
@@ -257,6 +233,9 @@ function setBadgedStatus() {
browser.browserAction.setBadgeBackgroundColor({
'color': color
}).catch(handleError);
browser.browserAction.setBadgeTextColor({
color: "#FFFFFF"
}).catch(handleError);
}
});
}
@@ -306,3 +285,41 @@ Object.prototype.getOrDefault = function (key, defaultValue) {
function handleError(error) {
console.log(translate('core_error') + ":" + error);
}
/**
* Function to log all activities from ClearUrls.
* Only logging when activated.
* The log is only temporary saved in the cache and will
* permanently saved with the saveLogOnClose function.
*
* @param beforeProcessing the url before the clear process
* @param afterProcessing the url after the clear process
* @param rule the rule that triggered the process
*/
function pushToLog(beforeProcessing, afterProcessing, rule) {
const limit = storage.logLimit;
if (storage.loggingStatus && limit !== 0) {
if (limit > 0 && !isNaN(limit)) {
while (storage.log.log.length >= limit) {
storage.log.log.shift();
}
}
storage.log.log.push(
{
"before": beforeProcessing,
"after": afterProcessing,
"rule": rule,
"timestamp": Date.now()
}
);
deferSaveOnDisk('log');
}
}
/**
* Checks if the storage is available.
*/
function isStorageAvailable() {
return storage.ClearURLsData.length !== 0;
}

12
core_js/watchdog.js
View File

@@ -24,15 +24,21 @@
*
* This watchdog restarts the whole Add-on, when the check fails.
*/
const CHECK_INTERVAL = 15000;
const CHECK_INTERVAL = 60000;
setInterval(function() {
if(isStorageAvailable() && storage.globalStatus) {
const dirtyURL = "https://clearurls.roebert.eu?utm_source=addon";
const cleanURL = "https://clearurls.roebert.eu";
if(pureCleaning(dirtyURL, true) !== cleanURL) {
console.log(translate('watchdog'));
storage.watchDogErrorCount += 1;
console.log(translate('watchdog', storage.watchDogErrorCount));
saveOnExit();
reload();
if(storage.watchDogErrorCount < 3) reload();
} else if(storage.watchDogErrorCount > 0){
storage.watchDogErrorCount = 0;
saveOnExit();
}
}
}, CHECK_INTERVAL);

111
data/data.min.json
View File

@@ -33,7 +33,10 @@
"aaxitk",
"hsa_cr_id",
"sb-ci-[a-zA-Z]+",
"rnid"
"rnid",
"dchild",
"camp",
"creative"
],
"referralMarketing": [
"tag"
@@ -126,6 +129,18 @@
],
"forceRedirection": true
},
"googleSearch": {
"urlPattern": "(https:\\/\\/|http:\\/\\/)([a-zA-Z0-9-.]*\\.)?(google)(\\.[a-zA-Z]{2,})\\/search\\?.*",
"completeProvider": false,
"rules": [
"client"
],
"referralMarketing": [],
"rawRules": [],
"exceptions": [],
"redirections": [],
"forceRedirection": false
},
"googlesyndication": {
"urlPattern": "(https:\\/\\/|http:\\/\\/)([a-zA-Z0-9-.]*\\.)?(googlesyndication)(\\.[a-zA-Z]{2,}).*",
"completeProvider": true,
@@ -229,7 +244,12 @@
"(https:\\/\\/|http:\\/\\/)(login\\.)(meijer\\.com)\\/.*\\?ref=.*",
".*(facebook\\.)\\w{2,}.*(\\/login_alerts\\/).*",
".*(facebook\\.)\\w{2,}.*(\\/should_add_browser\\/).*",
".*(facebook\\.)\\w{2,}.*(\\/ajax\\/).*"
".*(facebook\\.)\\w{2,}.*(\\/ajax\\/).*",
".*(api\\.taiga\\.io).*",
".*(\\.gog\\.com\\/click\\.html).*",
".*(login\\.progressive\\.com).*",
".*(sephora\\.com\\/api\\/).*",
".*(contestgirl\\.com).*"
],
"redirections": [],
"forceRedirection": false
@@ -1369,6 +1389,93 @@
"exceptions": [],
"redirections": [],
"forceRedirection": false
},"meetup.com": {
"urlPattern": "(https:\\/\\/|http:\\/\\/)([a-zA-Z0-9-.]*\\.)?(meetup)(\\.com).*",
"completeProvider": false,
"rules": [
"rv",
"_xtd"
],
"referralMarketing": [],
"rawRules": [],
"exceptions": [],
"redirections": [],
"forceRedirection": false
},"apple.com": {
"urlPattern": "(https:\\/\\/|http:\\/\\/)([a-zA-Z0-9-.]*\\.)?(apple)(\\.com).*",
"completeProvider": false,
"rules": [
"app",
"ign-itsc[a-zA-Z]+"
],
"referralMarketing": [],
"rawRules": [],
"exceptions": [],
"redirections": [],
"forceRedirection": false
},"alabout.com": {
"urlPattern": "(https:\\/\\/|http:\\/\\/)([a-zA-Z0-9-.]*\\.)?(alabout)(\\.com).*",
"completeProvider": false,
"rules": [],
"referralMarketing": [],
"rawRules": [],
"exceptions": [],
"redirections": [
".*url=([^&]*)"
],
"forceRedirection": false
},"newyorker.com": {
"urlPattern": "(https:\\/\\/|http:\\/\\/)([a-zA-Z0-9-.]*\\.)?(newyorker)(\\.com).*",
"completeProvider": false,
"rules": [
"source",
"bxid",
"cndid",
"esrc",
"mbid"
],
"referralMarketing": [],
"rawRules": [],
"exceptions": [],
"redirections": [],
"forceRedirection": false
}, "gog.com": {
"urlPattern": "(https:\\/\\/|http:\\/\\/)([a-zA-Z0-9-.]*\\.)?(gog)(\\.com).*",
"completeProvider": false,
"rules": [
"track_click",
"link_id"
],
"referralMarketing": [],
"rawRules": [],
"exceptions": [],
"redirections": [],
"forceRedirection": false
},
"tradedoubler.com": {
"urlPattern": "(https:\\/\\/|http:\\/\\/)([a-zA-Z0-9-.]*\\.)?(tradedoubler)(\\.com).*",
"completeProvider": false,
"rules": [],
"referralMarketing": [],
"rawRules": [],
"exceptions": [],
"redirections": [
".*url=([^&]*)",
".*_td_deeplink=([^&]*)"
],
"forceRedirection": false
},
"theguardian.com": {
"urlPattern": "(https:\\/\\/|http:\\/\\/)([a-zA-Z0-9-.]*\\.)?(theguardian)(\\.com).*",
"completeProvider": false,
"rules": [
"CMP"
],
"referralMarketing": [],
"rawRules": [],
"exceptions": [],
"redirections": [],
"forceRedirection": false
}
}
}

2
html/log.html
View File

@@ -114,7 +114,7 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
<script src="../browser-polyfill.js"></script>
<script src="../external_js/jquery-3.4.1.min.js"></script>
<script src="../external_js/bootstrap.min.js"></script>
<script src="../external_js/dataTables.min.js"></script>
<script src="../external_js/datatables.min.js"></script>
<script src="../external_js/fontawesome/all.min.js"></script>
<script src="../core_js/log.js"></script>
<script src="../core_js/write_version.js"></script>

7
html/settings.html
View File

@@ -161,6 +161,13 @@ along with this program. If not, see <http://www.gnu.org/licenses/>.
<span class="slider round"></span>
</label>
</p>
<p>
<label id="eTag_filtering_enabled" style="font-weight: bold;"></label><br />
<label class="switch">
<input type="checkbox" id="eTagFiltering">
<span class="slider round"></span>
</label>
</p>
<br />
<p class="text-center">
<button type="button" id="save_settings_btn"

6
manifest.json
View File

@@ -1,7 +1,7 @@
{
"manifest_version": 2,
"name": "ClearURLs",
"version": "1.15.0",
"version": "1.16.0",
"author": "Kevin Röbert",
"description": "__MSG_extension_description__",
"homepage_url": "https://gitlab.com/KevinRoebert/ClearUrls",
@@ -63,12 +63,14 @@
"core_js/message_handler.js",
"external_js/ip-range-check.js",
"core_js/tools.js",
"core_js/badgedHandler.js",
"core_js/pureCleaning.js",
"core_js/context_menu.js",
"core_js/historyListener.js",
"clearurls.js",
"core_js/storage.js",
"core_js/watchdog.js"
"core_js/watchdog.js",
"core_js/eTagFilter.js"
]
},
"content_scripts": [

BIN
promotion/MEA-button.png Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 6.0 KiB

290
promotion/MEA-button.svg Normal file
View File

@@ -0,0 +1,290 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->
<svg
xmlns:dc="http://purl.org/dc/elements/1.1/"
xmlns:cc="http://creativecommons.org/ns#"
xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#"
xmlns:svg="http://www.w3.org/2000/svg"
xmlns="http://www.w3.org/2000/svg"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
width="173mm"
height="62mm"
viewBox="0 0 173 62"
version="1.1"
id="svg8"
inkscape:version="0.92.1 r15371"
sodipodi:docname="MEA-button.svg">
<defs
id="defs2">
<style
id="style4487">.cls-1{fill:url(#linear-gradient);}.cls-2{opacity:0.35;fill:url(#radial-gradient);}.cls-2,.cls-4{isolation:isolate;}.cls-3{fill:url(#linear-gradient-2);}.cls-4{opacity:0.41;fill:url(#radial-gradient-2);}.cls-5{fill:url(#radial-gradient-3);}.cls-6{fill:url(#radial-gradient-4);}</style>
<linearGradient
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(1,0,0,-1,0,266)"
y2="84.03"
x2="241.67"
y1="84.03"
x1="63.33"
id="linear-gradient">
<stop
id="stop4489"
stop-color="#0c59a4"
offset="0" />
<stop
id="stop4491"
stop-color="#114a8b"
offset="1" />
</linearGradient>
<radialGradient
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(1,0,0,-0.95,0,248.84)"
r="95.38"
cy="68.91"
cx="161.83"
id="radial-gradient">
<stop
id="stop4494"
stop-opacity="0"
offset="0.72" />
<stop
id="stop4496"
stop-opacity="0.53"
offset="0.95" />
<stop
id="stop4498"
offset="1" />
</radialGradient>
<linearGradient
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(1,0,0,-1,0,266)"
y2="40.06"
x2="45.96"
y1="161.39"
x1="157.35"
id="linear-gradient-2">
<stop
id="stop4501"
stop-color="#1b9de2"
offset="0" />
<stop
id="stop4503"
stop-color="#1595df"
offset="0.16" />
<stop
id="stop4505"
stop-color="#0680d7"
offset="0.67" />
<stop
id="stop4507"
stop-color="#0078d4"
offset="1" />
</linearGradient>
<radialGradient
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(0.15,-0.99,-0.8,-0.12,176.64,-125.4)"
r="143.24"
cy="62.99"
cx="-340.29"
id="radial-gradient-2">
<stop
id="stop4510"
stop-opacity="0"
offset="0.76" />
<stop
id="stop4512"
stop-opacity="0.5"
offset="0.95" />
<stop
id="stop4514"
offset="1" />
</radialGradient>
<radialGradient
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(-0.04,1,2.13,0.08,-1179.54,-106.69)"
r="202.43"
cy="570.21"
cx="113.37"
id="radial-gradient-3">
<stop
id="stop4517"
stop-color="#35c1f1"
offset="0" />
<stop
id="stop4519"
stop-color="#34c1ed"
offset="0.11" />
<stop
id="stop4521"
stop-color="#2fc2df"
offset="0.23" />
<stop
id="stop4523"
stop-color="#2bc3d2"
offset="0.31" />
<stop
id="stop4525"
stop-color="#36c752"
offset="0.67" />
</radialGradient>
<radialGradient
gradientUnits="userSpaceOnUse"
gradientTransform="matrix(0.28,0.96,0.78,-0.23,-303.76,-148.5)"
r="97.34"
cy="567.97"
cx="376.52"
id="radial-gradient-4">
<stop
id="stop4528"
stop-color="#66eb6e"
offset="0" />
<stop
id="stop4530"
stop-opacity="0"
stop-color="#66eb6e"
offset="1" />
</radialGradient>
</defs>
<sodipodi:namedview
id="base"
pagecolor="#ffffff"
bordercolor="#666666"
borderopacity="1.0"
inkscape:pageopacity="0.0"
inkscape:pageshadow="2"
inkscape:zoom="0.7"
inkscape:cx="680.12936"
inkscape:cy="17.153298"
inkscape:document-units="mm"
inkscape:current-layer="layer1"
showgrid="false"
inkscape:window-width="2560"
inkscape:window-height="1377"
inkscape:window-x="-8"
inkscape:window-y="32"
inkscape:window-maximized="1" />
<metadata
id="metadata5">
<rdf:RDF>
<cc:Work
rdf:about="">
<dc:format>image/svg+xml</dc:format>
<dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" />
<dc:title></dc:title>
</cc:Work>
</rdf:RDF>
</metadata>
<g
inkscape:label="Ebene 1"
inkscape:groupmode="layer"
id="layer1"
transform="translate(0,-235)">
<rect
style="opacity:1;fill:#20989a;fill-opacity:1;stroke:none;stroke-width:0;stroke-miterlimit:4;stroke-dasharray:none;stroke-opacity:1;paint-order:stroke markers fill"
id="rect4485"
width="172"
height="60"
x="0.75595236"
y="236.24405"
ry="9.0862904"
inkscape:export-filename="F:\Programmieren\Git\Firefox Addons\ClearUrls\promotion\MEA-button.png"
inkscape:export-xdpi="25.4"
inkscape:export-ydpi="25.4" />
<g
id="g4586"
transform="matrix(0.18042983,0,0,0.18191038,10.112503,242.71284)"
inkscape:export-filename="F:\Programmieren\Git\Firefox Addons\ClearUrls\promotion\MEA-button.png"
inkscape:export-xdpi="25.4"
inkscape:export-ydpi="25.4">
<path
id="path4537"
transform="translate(-4.63,-4.92)"
d="m 235.68,195.46 a 93.73,93.73 0 0 1 -10.54,4.71 101.87,101.87 0 0 1 -35.9,6.46 c -47.32,0 -88.54,-32.55 -88.54,-74.32 A 31.48,31.48 0 0 1 117.13,105 c -42.8,1.8 -53.8,46.4 -53.8,72.53 0,73.88 68.09,81.37 82.76,81.37 7.91,0 19.84,-2.3 27,-4.56 l 1.31,-0.44 a 128.34,128.34 0 0 0 66.6,-52.8 4,4 0 0 0 -5.32,-5.64 z"
class="cls-1"
inkscape:connector-curvature="0"
style="fill:url(#linear-gradient)" />
<path
id="path4539"
transform="translate(-4.63,-4.92)"
d="m 235.68,195.46 a 93.73,93.73 0 0 1 -10.54,4.71 101.87,101.87 0 0 1 -35.9,6.46 c -47.32,0 -88.54,-32.55 -88.54,-74.32 A 31.48,31.48 0 0 1 117.13,105 c -42.8,1.8 -53.8,46.4 -53.8,72.53 0,73.88 68.09,81.37 82.76,81.37 7.91,0 19.84,-2.3 27,-4.56 l 1.31,-0.44 a 128.34,128.34 0 0 0 66.6,-52.8 4,4 0 0 0 -5.32,-5.64 z"
class="cls-2"
inkscape:connector-curvature="0"
style="opacity:0.35;isolation:isolate;fill:url(#radial-gradient)" />
<path
id="path4541"
transform="translate(-4.63,-4.92)"
d="M 110.34,246.34 A 79.2,79.2 0 0 1 87.6,225 80.72,80.72 0 0 1 117.13,105 c 3.12,-1.47 8.45,-4.13 15.54,-4 a 32.35,32.35 0 0 1 25.69,13 31.88,31.88 0 0 1 6.36,18.66 c 0,-0.21 24.46,-79.6 -80,-79.6 -43.9,0 -80,41.66 -80,78.21 a 130.15,130.15 0 0 0 12.11,56 128,128 0 0 0 156.38,67.11 75.55,75.55 0 0 1 -62.78,-8 z"
class="cls-3"
inkscape:connector-curvature="0"
style="fill:url(#linear-gradient-2)" />
<path
id="path4543"
transform="translate(-4.63,-4.92)"
d="M 110.34,246.34 A 79.2,79.2 0 0 1 87.6,225 80.72,80.72 0 0 1 117.13,105 c 3.12,-1.47 8.45,-4.13 15.54,-4 a 32.35,32.35 0 0 1 25.69,13 31.88,31.88 0 0 1 6.36,18.66 c 0,-0.21 24.46,-79.6 -80,-79.6 -43.9,0 -80,41.66 -80,78.21 a 130.15,130.15 0 0 0 12.11,56 128,128 0 0 0 156.38,67.11 75.55,75.55 0 0 1 -62.78,-8 z"
class="cls-4"
inkscape:connector-curvature="0"
style="opacity:0.41000001;isolation:isolate;fill:url(#radial-gradient-2)" />
<path
id="path4545"
transform="translate(-4.63,-4.92)"
d="m 156.94,153.78 c -0.81,1.05 -3.3,2.5 -3.3,5.66 0,2.61 1.7,5.12 4.72,7.23 14.38,10 41.49,8.68 41.56,8.68 a 59.56,59.56 0 0 0 30.27,-8.35 61.38,61.38 0 0 0 30.43,-52.88 c 0.26,-22.41 -8,-37.31 -11.34,-43.91 C 228.09,28.76 182.35,4.92 132.61,4.92 a 128,128 0 0 0 -128,126.2 c 0.48,-36.54 36.8,-66.05 80,-66.05 3.5,0 23.46,0.34 42,10.07 16.34,8.58 24.9,18.94 30.85,29.21 6.18,10.67 7.28,24.15 7.28,29.52 0,5.37 -2.74,13.33 -7.8,19.91 z"
class="cls-5"
inkscape:connector-curvature="0"
style="fill:url(#radial-gradient-3)" />
<path
id="path4547"
transform="translate(-4.63,-4.92)"
d="m 156.94,153.78 c -0.81,1.05 -3.3,2.5 -3.3,5.66 0,2.61 1.7,5.12 4.72,7.23 14.38,10 41.49,8.68 41.56,8.68 a 59.56,59.56 0 0 0 30.27,-8.35 61.38,61.38 0 0 0 30.43,-52.88 c 0.26,-22.41 -8,-37.31 -11.34,-43.91 C 228.09,28.76 182.35,4.92 132.61,4.92 a 128,128 0 0 0 -128,126.2 c 0.48,-36.54 36.8,-66.05 80,-66.05 3.5,0 23.46,0.34 42,10.07 16.34,8.58 24.9,18.94 30.85,29.21 6.18,10.67 7.28,24.15 7.28,29.52 0,5.37 -2.74,13.33 -7.8,19.91 z"
class="cls-6"
inkscape:connector-curvature="0"
style="fill:url(#radial-gradient-4)" />
</g>
<text
xml:space="preserve"
style="font-style:normal;font-weight:normal;font-size:14.11111069px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:0.26458332"
x="63.122021"
y="270.91965"
id="text4594"
inkscape:export-filename="F:\Programmieren\Git\Firefox Addons\ClearUrls\promotion\MEA-button.png"
inkscape:export-xdpi="25.4"
inkscape:export-ydpi="25.4"><tspan
sodipodi:role="line"
id="tspan4592"
x="63.122021"
y="283.79364"
style="font-size:14.11111069px;fill:#ffffff;stroke-width:0.26458332" /></text>
<g
id="g4607"
transform="translate(67.73337,2.7074)"
inkscape:export-filename="F:\Programmieren\Git\Firefox Addons\ClearUrls\promotion\MEA-button.png"
inkscape:export-xdpi="25.4"
inkscape:export-ydpi="25.4">
<text
id="text4590"
y="260.3363"
x="-0.79237193"
style="font-style:normal;font-weight:normal;font-size:14.11111069px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#ffffff;fill-opacity:1;stroke:none;stroke-width:0.26458332"
xml:space="preserve"><tspan
style="font-size:14.11111069px;fill:#ffffff;stroke-width:0.26458332"
y="260.3363"
x="-0.79237193"
id="tspan4588"
sodipodi:role="line">GET THE</tspan></text>
<text
id="text4598"
y="276.43811"
x="-0.23151043"
style="font-style:normal;font-weight:normal;font-size:10.58333302px;line-height:1.25;font-family:sans-serif;letter-spacing:0px;word-spacing:0px;fill:#000000;fill-opacity:1;stroke:none;stroke-width:0.26458332"
xml:space="preserve"><tspan
style="font-style:normal;font-variant:normal;font-weight:bold;font-stretch:normal;font-size:16.93333244px;font-family:sans-serif;-inkscape-font-specification:'sans-serif Bold';fill:#ffffff;stroke-width:0.26458332"
y="276.43811"
x="-0.23151043"
id="tspan4596"
sodipodi:role="line"><tspan
id="tspan4600"
style="font-size:19.75555611px">ADD</tspan>-ON</tspan></text>
</g>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 11 KiB