Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Proposal: StorageArea.getKeys() #601

Open
polywock opened this issue Apr 25, 2024 · 7 comments
Open

Proposal: StorageArea.getKeys() #601

polywock opened this issue Apr 25, 2024 · 7 comments
Labels
implemented: chrome Implemented in Chrome implemented: safari Implemented in Safari supportive: firefox Supportive from Firefox

Comments

@polywock
Copy link

polywock commented Apr 25, 2024 •
edited
Loading

Taken from #505, as an alternative proposal suggested by @tophf and @Rob--W. It's significantly different from my original proposal, so I'm creating a separate proposal for it.

Background

I regularly need to use storage keys that involve dynamic elements like flag:[tabId]. The only way to get all keys starting with the prefix flag: is using StorageArea.get() to get all the storage items and then to filter.

This isn't efficient. The storage areas support 10MB of data and even more with the unlimitedStorage permission. That's potentially a huge amount of storage data that needs to be serialized and sent over. There's also a special concern when using the Sync storage area.

Proposal

A method to get all keys without the corresponding values.

browser.storage.local.getAllKeys(): Promise<string[]>

Resources

https://developer.mozilla.org/en-US/docs/Web/API/IDBIndex/getAllKeys
@github-actions github-actions bot added needs-triage: chrome Chrome needs to assess this issue for the first time needs-triage: firefox Firefox needs to assess this issue for the first time needs-triage: safari Safari needs to assess this issue for the first time labels Apr 25, 2024
@polywock polywock mentioned this issue Apr 25, 2024
Closed
@xeenon xeenon added supportive: safari Supportive from Safari and removed needs-triage: safari Safari needs to assess this issue for the first time labels Apr 25, 2024
@Rob--W Rob--W added supportive: firefox Supportive from Firefox and removed needs-triage: firefox Firefox needs to assess this issue for the first time labels Apr 25, 2024
@dotproto dotproto mentioned this issue May 9, 2024
Closed
@oliverdunk oliverdunk added supportive: chrome Supportive from Chrome and removed needs-triage: chrome Chrome needs to assess this issue for the first time labels May 9, 2024
@Rob--W Rob--W mentioned this issue May 9, 2024
Merged
@oliverdunk
Copy link
Member

@polywock I plan to implement this in Chromium as part of some work I am doing on the storage API. I’ve started a proposal PR using the template - would you like to work together on finishing that, or are you happy for me to open it? Thanks!

@polywock
Copy link
Author

@oliverdunk I appreciate the offer to be involved. But, I'll probably slow down the process.

Please go ahead without me. Thank you!

@oliverdunk oliverdunk mentioned this issue Jun 28, 2024
Merged
@oliverdunk
Copy link
Member

I have opened a proposal for this in #649.

@polywock
Copy link
Author

polywock commented Jul 4, 2024 •
edited
Loading

Pull request has been approved and merged. 🚀

@polywock polywock closed this as completed Jul 4, 2024
@Rob--W
Copy link
Member

Rob--W commented Jul 30, 2024

Re-opening this ticket because we usually keep tickets open until there is nothing left to be done. In this case, we can use this ticket to signal the cross-browser implementation status of the feature (as issue labels here).

I have opened https://bugzilla.mozilla.org/show_bug.cgi?id=1910669 to track the implementation in Firefox.

@Rob--W Rob--W reopened this Jul 30, 2024
@oliverdunk
Copy link
Member

I've just landed the CL for this in Chromium (link). It will be available in Canary later today and more generally Chrome 130 as that rolls out across channels (schedule).

@oliverdunk oliverdunk added implemented: chrome Implemented in Chrome and removed supportive: chrome Supportive from Chrome labels Aug 21, 2024
@xeenon xeenon mentioned this issue Sep 26, 2024
Merged
@xeenon xeenon changed the title Proposal: StorageArea.getAllKeys() Proposal: StorageArea.getKeys() Sep 26, 2024
@Rob--W Rob--W mentioned this issue Sep 29, 2024
Merged
MoeBazziGIT added a commit to MoeBazziGIT/WebKit that referenced this issue Oct 1, 2024
@MoeBazziGIT
Implement browser.storageArea.getKeys() for Web Extension Storage API.
2da28ba 
https://bugs.webkit.org/show_bug.cgi?id=280275

Reviewed by NOBODY (OOPS!).

This patch implements a web extension API to retrieve all keys for a given storage area in the browser.storage API.
WECG Proposal: https://github.com/w3c/webextensions/blob/main/proposals/storage-get-keys.md
WECG Original Issue: w3c/webextensions#601

* Source/WebKit/UIProcess/Extensions/Cocoa/API/WebExtensionContextAPIStorageCocoa.mm:
(WebKit::WebExtensionContext::storageGetKeys):
* Source/WebKit/UIProcess/Extensions/Cocoa/_WKWebExtensionStorageSQLiteStore.h:
* Source/WebKit/UIProcess/Extensions/Cocoa/_WKWebExtensionStorageSQLiteStore.mm:
(-[_WKWebExtensionStorageSQLiteStore getAllKeys:]):
(-[_WKWebExtensionStorageSQLiteStore getValuesForKeys:completionHandler:]):
* Source/WebKit/UIProcess/Extensions/WebExtensionContext.h:
* Source/WebKit/UIProcess/Extensions/WebExtensionContext.messages.in:
* Source/WebKit/WebProcess/Extensions/API/Cocoa/WebExtensionAPIStorageAreaCocoa.mm:
(WebKit::WebExtensionAPIStorageArea::getKeys):
* Source/WebKit/WebProcess/Extensions/API/WebExtensionAPIStorageArea.h:
* Source/WebKit/WebProcess/Extensions/Interfaces/WebExtensionAPIStorageArea.idl:
* Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebExtensionAPIStorage.mm:
(TestWebKitAPI::TEST(WKWebExtensionAPIStorage, Errors)):
(TestWebKitAPI::TEST(WKWebExtensionAPIStorage, GetKeys)):
MoeBazziGIT added a commit to MoeBazziGIT/WebKit that referenced this issue Oct 1, 2024
@MoeBazziGIT
Implement browser.storageArea.getKeys() for Web Extension Storage API.
f935178 
https://bugs.webkit.org/show_bug.cgi?id=280275

Reviewed by NOBODY (OOPS!).

This patch implements a web extension API to retrieve all keys for a given storage area in the
browser.storage API.

WECG Proposal Issue: https://github.com/w3c/webextensions/blob/main/proposals/storage-get-keys.md
WECG Original Issue: w3c/webextensions#601

* Source/WebKit/UIProcess/Extensions/Cocoa/API/WebExtensionContextAPIStorageCocoa.mm:
(WebKit::WebExtensionContext::storageGetKeys):
* Source/WebKit/UIProcess/Extensions/Cocoa/_WKWebExtensionStorageSQLiteStore.h:
* Source/WebKit/UIProcess/Extensions/Cocoa/_WKWebExtensionStorageSQLiteStore.mm:
(-[_WKWebExtensionStorageSQLiteStore getAllKeys:]):
(-[_WKWebExtensionStorageSQLiteStore getValuesForKeys:completionHandler:]):
* Source/WebKit/UIProcess/Extensions/WebExtensionContext.h:
* Source/WebKit/UIProcess/Extensions/WebExtensionContext.messages.in:
* Source/WebKit/WebProcess/Extensions/API/Cocoa/WebExtensionAPIStorageAreaCocoa.mm:
(WebKit::WebExtensionAPIStorageArea::getKeys):
* Source/WebKit/WebProcess/Extensions/API/WebExtensionAPIStorageArea.h:
* Source/WebKit/WebProcess/Extensions/Interfaces/WebExtensionAPIStorageArea.idl:
* Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebExtensionAPIStorage.mm:
(TestWebKitAPI::TEST(WKWebExtensionAPIStorage, Errors)):
(TestWebKitAPI::TEST(WKWebExtensionAPIStorage, GetKeys)):
MoeBazziGIT added a commit to MoeBazziGIT/WebKit that referenced this issue Oct 1, 2024
@MoeBazziGIT
Implement browser.storageArea.getKeys() for Web Extension Storage API.
13a447a 
https://bugs.webkit.org/show_bug.cgi?id=280275

Reviewed by NOBODY (OOPS!).

This patch implements a web extension API to retrieve all keys for a given storage area in the
browser.storage API.

WECG Proposal: https://github.com/w3c/webextensions/blob/main/proposals/storage-get-keys.md
WECG Original Issue: w3c/webextensions#601

* Source/WebKit/UIProcess/Extensions/Cocoa/API/WebExtensionContextAPIStorageCocoa.mm:
(WebKit::WebExtensionContext::storageGetKeys):
* Source/WebKit/UIProcess/Extensions/Cocoa/_WKWebExtensionStorageSQLiteStore.h:
* Source/WebKit/UIProcess/Extensions/Cocoa/_WKWebExtensionStorageSQLiteStore.mm:
(-[_WKWebExtensionStorageSQLiteStore getAllKeys:]):
(-[_WKWebExtensionStorageSQLiteStore getValuesForKeys:completionHandler:]):
* Source/WebKit/UIProcess/Extensions/WebExtensionContext.h:
* Source/WebKit/UIProcess/Extensions/WebExtensionContext.messages.in:
* Source/WebKit/WebProcess/Extensions/API/Cocoa/WebExtensionAPIStorageAreaCocoa.mm:
(WebKit::WebExtensionAPIStorageArea::getKeys):
* Source/WebKit/WebProcess/Extensions/API/WebExtensionAPIStorageArea.h:
* Source/WebKit/WebProcess/Extensions/Interfaces/WebExtensionAPIStorageArea.idl:
* Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebExtensionAPIStorage.mm:
(TestWebKitAPI::TEST(WKWebExtensionAPIStorage, Errors)):
(TestWebKitAPI::TEST(WKWebExtensionAPIStorage, GetKeys)):
MoeBazziGIT added a commit to MoeBazziGIT/WebKit that referenced this issue Oct 1, 2024
@MoeBazziGIT
Implement browser.storageArea.getKeys() for Web Extension Storage API.
e02b00a 
https://bugs.webkit.org/show_bug.cgi?id=280275

Reviewed by NOBODY (OOPS!).

This patch implements a web extension API to retrieve all keys for a given storage area in the
browser.storage API.

WECG Proposal: https://github.com/w3c/webextensions/blob/main/proposals/storage-get-keys.md
WECG Original Issue: w3c/webextensions#601

This patch also includes the following clean-ups:
- Change 'auto' to 'auto *' when assigning the result of WebExtensionContext::storageForType()
 to explicitly denote pointer type.
- Update strongSelf retrieval to in _WKWebExtensionStorageSQLiteStore.mm methods to leverage
 ARC in extension code, allowing the use of Objective-C pointers instead of RetainPtr, which
is only necessary in other parts of WebKit that do not utilize ARC.
- Remove unnecessary explicit template types from WeakObjCPtr in _WKWebExtensionStorageSQLiteStore.

* Source/WebKit/UIProcess/Extensions/Cocoa/API/WebExtensionContextAPIStorageCocoa.mm:
(WebKit::WebExtensionContext::storageGet):
(WebKit::WebExtensionContext::storageGetKeys):
(WebKit::WebExtensionContext::storageGetBytesInUse):
* Source/WebKit/UIProcess/Extensions/Cocoa/_WKWebExtensionStorageSQLiteStore.h:
* Source/WebKit/UIProcess/Extensions/Cocoa/_WKWebExtensionStorageSQLiteStore.mm:
(-[_WKWebExtensionStorageSQLiteStore getAllKeys:]):
(-[_WKWebExtensionStorageSQLiteStore getValuesForKeys:completionHandler:]):
(-[_WKWebExtensionStorageSQLiteStore getStorageSizeForKeys:completionHandler:]):
(-[_WKWebExtensionStorageSQLiteStore getStorageSizeForAllKeysIncludingKeyedData:withCompletionHandler:]):
(-[_WKWebExtensionStorageSQLiteStore setKeyedData:completionHandler:]):
(-[_WKWebExtensionStorageSQLiteStore deleteValuesForKeys:completionHandler:]):
* Source/WebKit/UIProcess/Extensions/WebExtensionContext.h:
* Source/WebKit/UIProcess/Extensions/WebExtensionContext.messages.in:
* Source/WebKit/WebProcess/Extensions/API/Cocoa/WebExtensionAPIStorageAreaCocoa.mm:
(WebKit::WebExtensionAPIStorageArea::getKeys):
* Source/WebKit/WebProcess/Extensions/API/WebExtensionAPIStorageArea.h:
* Source/WebKit/WebProcess/Extensions/Interfaces/WebExtensionAPIStorageArea.idl:
* Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebExtensionAPIStorage.mm:
(TestWebKitAPI::TEST(WKWebExtensionAPIStorage, Errors)):
(TestWebKitAPI::TEST(WKWebExtensionAPIStorage, GetKeys)):
@xeenon xeenon added implemented: safari Implemented in Safari and removed supportive: safari Supportive from Safari labels Oct 2, 2024
MoeBazziGIT added a commit to MoeBazziGIT/WebKit that referenced this issue Oct 2, 2024
@MoeBazziGIT
Implement browser.storageArea.getKeys() for Web Extension Storage API.
b4568af 
https://bugs.webkit.org/show_bug.cgi?id=280275

Reviewed by NOBODY (OOPS!).

This patch implements a web extension API to retrieve all keys for a given storage area in the
browser.storage API.

WECG Proposal: https://github.com/w3c/webextensions/blob/main/proposals/storage-get-keys.md
WECG Original Issue: w3c/webextensions#601

This patch also includes the following clean-ups:
- Change 'auto' to 'auto *' when assigning the result of WebExtensionContext::storageForType()
 to explicitly denote pointer type.
- Update strongSelf retrieval to in _WKWebExtensionStorageSQLiteStore.mm methods to leverage
 ARC in extension code, allowing the use of Objective-C pointers instead of RetainPtr, which
is only necessary in other parts of WebKit that do not utilize ARC.
- Remove unnecessary explicit template types from WeakObjCPtr in _WKWebExtensionStorageSQLiteStore.

* Source/WebKit/UIProcess/Extensions/Cocoa/API/WebExtensionContextAPIStorageCocoa.mm:
(WebKit::WebExtensionContext::storageGet):
(WebKit::WebExtensionContext::storageGetKeys):
(WebKit::WebExtensionContext::storageGetBytesInUse):
* Source/WebKit/UIProcess/Extensions/Cocoa/_WKWebExtensionStorageSQLiteStore.h:
* Source/WebKit/UIProcess/Extensions/Cocoa/_WKWebExtensionStorageSQLiteStore.mm:
(-[_WKWebExtensionStorageSQLiteStore getAllKeys:]):
(-[_WKWebExtensionStorageSQLiteStore getValuesForKeys:completionHandler:]):
(-[_WKWebExtensionStorageSQLiteStore getStorageSizeForKeys:completionHandler:]):
(-[_WKWebExtensionStorageSQLiteStore getStorageSizeForAllKeysIncludingKeyedData:withCompletionHandler:]):
(-[_WKWebExtensionStorageSQLiteStore setKeyedData:completionHandler:]):
(-[_WKWebExtensionStorageSQLiteStore deleteValuesForKeys:completionHandler:]):
* Source/WebKit/UIProcess/Extensions/WebExtensionContext.h:
* Source/WebKit/UIProcess/Extensions/WebExtensionContext.messages.in:
* Source/WebKit/WebProcess/Extensions/API/Cocoa/WebExtensionAPIStorageAreaCocoa.mm:
(WebKit::WebExtensionAPIStorageArea::getKeys):
* Source/WebKit/WebProcess/Extensions/API/WebExtensionAPIStorageArea.h:
* Source/WebKit/WebProcess/Extensions/Interfaces/WebExtensionAPIStorageArea.idl:
* Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebExtensionAPIStorage.mm:
(TestWebKitAPI::TEST(WKWebExtensionAPIStorage, Errors)):
(TestWebKitAPI::TEST(WKWebExtensionAPIStorage, GetKeys)):
webkit-commit-queue pushed a commit to MoeBazziGIT/WebKit that referenced this issue Oct 3, 2024
@MoeBazziGIT @xeenon
Implement browser.storageArea.getKeys() for Web Extension Storage API.
972a6ce 
https://bugs.webkit.org/show_bug.cgi?id=280275

Reviewed by Timothy Hatcher.

This patch implements a web extension API to retrieve all keys for a given storage area in the
browser.storage API.

WECG Proposal: https://github.com/w3c/webextensions/blob/main/proposals/storage-get-keys.md
WECG Original Issue: w3c/webextensions#601

This patch also includes the following clean-ups:
- Change 'auto' to 'auto *' when assigning the result of WebExtensionContext::storageForType()
 to explicitly denote pointer type.
- Update strongSelf retrieval to in _WKWebExtensionStorageSQLiteStore.mm methods to leverage
 ARC in extension code, allowing the use of Objective-C pointers instead of RetainPtr, which
is only necessary in other parts of WebKit that do not utilize ARC.
- Remove unnecessary explicit template types from WeakObjCPtr in _WKWebExtensionStorageSQLiteStore.

* Source/WebKit/UIProcess/Extensions/Cocoa/API/WebExtensionContextAPIStorageCocoa.mm:
(WebKit::WebExtensionContext::storageGet):
(WebKit::WebExtensionContext::storageGetKeys):
(WebKit::WebExtensionContext::storageGetBytesInUse):
* Source/WebKit/UIProcess/Extensions/Cocoa/_WKWebExtensionStorageSQLiteStore.h:
* Source/WebKit/UIProcess/Extensions/Cocoa/_WKWebExtensionStorageSQLiteStore.mm:
(-[_WKWebExtensionStorageSQLiteStore getAllKeys:]):
(-[_WKWebExtensionStorageSQLiteStore getValuesForKeys:completionHandler:]):
(-[_WKWebExtensionStorageSQLiteStore getStorageSizeForKeys:completionHandler:]):
(-[_WKWebExtensionStorageSQLiteStore getStorageSizeForAllKeysIncludingKeyedData:withCompletionHandler:]):
(-[_WKWebExtensionStorageSQLiteStore setKeyedData:completionHandler:]):
(-[_WKWebExtensionStorageSQLiteStore deleteValuesForKeys:completionHandler:]):
* Source/WebKit/UIProcess/Extensions/WebExtensionContext.h:
* Source/WebKit/UIProcess/Extensions/WebExtensionContext.messages.in:
* Source/WebKit/WebProcess/Extensions/API/Cocoa/WebExtensionAPIStorageAreaCocoa.mm:
(WebKit::WebExtensionAPIStorageArea::getKeys):
* Source/WebKit/WebProcess/Extensions/API/WebExtensionAPIStorageArea.h:
* Source/WebKit/WebProcess/Extensions/Interfaces/WebExtensionAPIStorageArea.idl:
* Tools/TestWebKitAPI/Tests/WebKitCocoa/WKWebExtensionAPIStorage.mm:
(TestWebKitAPI::TEST(WKWebExtensionAPIStorage, Errors)):
(TestWebKitAPI::TEST(WKWebExtensionAPIStorage, GetKeys)):

Canonical link: https://commits.webkit.org/284602@main
@xeenon xeenon mentioned this issue Oct 10, 2024
Closed
@dotproto dotproto mentioned this issue Oct 16, 2024
Merged
@fregante
Copy link

Safari landed this in the their technology preview release: 🎉

Missing:

@fregante fregante mentioned this issue Oct 24, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
implemented: chrome Implemented in Chrome implemented: safari Implemented in Safari supportive: firefox Supportive from Firefox
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@xeenon @Rob--W @fregante @oliverdunk @polywock