Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add bundle conformance tests to verify documents are loaded correctly. #4337

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

wu-hui
Copy link
Contributor

@wu-hui wu-hui commented Jan 24, 2021

No description provided.

@changeset-bot
Copy link

changeset-bot bot commented Jan 24, 2021

⚠️ No Changeset found

Latest commit: d78ecf9

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Jan 24, 2021

Size Analysis Report

Affected Products

  • @firebase/analytics-exp

    • getAnalytics

      Size Table

      TypeBase (352eb47)Head (f6379d9)Diff
      size
      8.98 kB
      9.01 kB
      +36 B (+0.4%)
      size-with-ext-deps
      14.2 kB
      26.3 kB
      +12.1 kB (+84.8%)
    • isSupported

      Size Table

      TypeBase (352eb47)Head (f6379d9)Diff
      size
      9.03 kB
      9.07 kB
      +36 B (+0.4%)
      size-with-ext-deps
      14.3 kB
      26.4 kB
      +12.1 kB (+85.0%)
    • logEvent

      Size Table

      TypeBase (352eb47)Head (f6379d9)Diff
      size
      8.89 kB
      8.93 kB
      +36 B (+0.4%)
      size-with-ext-deps
      14.1 kB
      26.3 kB
      +12.1 kB (+85.9%)
    • setAnalyticsCollectionEnabled

      Size Table

      TypeBase (352eb47)Head (f6379d9)Diff
      size
      9.05 kB
      9.08 kB
      +36 B (+0.4%)
      size-with-ext-deps
      14.3 kB
      26.4 kB
      +12.1 kB (+84.9%)
    • setCurrentScreen

      Size Table

      TypeBase (352eb47)Head (f6379d9)Diff
      size
      9.13 kB
      9.17 kB
      +36 B (+0.4%)
      size-with-ext-deps
      14.4 kB
      26.5 kB
      +12.1 kB (+84.4%)
    • setUserId

      Size Table

      TypeBase (352eb47)Head (f6379d9)Diff
      size
      9.12 kB
      9.15 kB
      +36 B (+0.4%)
      size-with-ext-deps
      14.4 kB
      26.5 kB
      +12.1 kB (+84.5%)
    • setUserProperties

      Size Table

      TypeBase (352eb47)Head (f6379d9)Diff
      size
      9.19 kB
      9.23 kB
      +36 B (+0.4%)
      size-with-ext-deps
      14.4 kB
      26.6 kB
      +12.1 kB (+84.0%)
    • settings

      Size Table

      TypeBase (352eb47)Head (f6379d9)Diff
      size
      9.02 kB
      9.05 kB
      +36 B (+0.4%)
      size-with-ext-deps
      14.3 kB
      26.4 kB
      +12.1 kB (+85.1%)

@@ -207,7 +207,7 @@ export function bundleInitialProgress(
documentsLoaded: 0,
bytesLoaded: 0,
totalDocuments: metadata.totalDocuments!,
totalBytes: metadata.totalBytes!
totalBytes: parseInt(metadata.totalBytes!.toString(), 10)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use normailizeNumber for this: https://osscs.corp.google.com/firebase-sdk/firebase-js-sdk/+/master:packages/firestore/src/model/normalize.ts;l=73?q=normalizeNumber&ss=firebase-sdk%2Ffirebase-js-sdk

This is the function that we use in other places to deserialize large numbers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool!

];

// This is built by Java(proto3 json formatter) with above document and one additional field of `pathValue: db.doc('col1/ref1')`.
const BUNDLES_PROTOB3_CONFORMANCE_TEMPLATE = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does the B stand for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stands for I-cannot-type i guess.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix though :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done..

bytesValue: Blob.fromUint8Array(new Uint8Array([0x01, 0x02]))
};

// This is built by Node(protobuf.js) with above document and one additional field of `pathValue: db.doc('col1/ref1')`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that this is now a strange mix of ProtobufJS and Proto3 JSON.

@@ -252,3 +264,178 @@ apiDescribe('Bundles', (persistence: boolean) => {
});
});
});

apiDescribe('Bundles conformance', (persistence: boolean) => {
const CONFORMANCE_DOC = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The approach of "hardcoding" the JSON strings from Node and Java has some merits, but since it is Monday morning and we have a full week ahead of us, I would like to point out that you could also use the Web serializer to built both Proto3 JSON and ProtobufJS documents in the tests themselves. This approach has some benefits (less code to copy/paste, less chance that the code will go out of sync) and some downsides (less portable, may not catch bugs in the Node/Java clients) so I will leave the decision to you.

If we do decide to only support Proto3 JSON, we can also get rid of one of the tests here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added a todo instead, my 3 minute research did not tell me clearly how to do it..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to spend some more time on this since adding this and then removing it right away just adds code churn. You should be able to call wrap() with CONFORMANCE_DOC and get a JSON encoded Value proto. You may have to add an overload so you can control the useProto3JSON flag (and as discussed, this only works in api_internal)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline, this is harder than I expected, and does not bring much value because we still need to hard code elements that are not documents. Let's not do this for now?

@wu-hui wu-hui assigned schmidt-sebastian and unassigned wu-hui Jan 25, 2021
packages/firestore/test/integration/api/bundle.test.ts Outdated Show resolved Hide resolved
@@ -252,3 +264,178 @@ apiDescribe('Bundles', (persistence: boolean) => {
});
});
});

apiDescribe('Bundles conformance', (persistence: boolean) => {
const CONFORMANCE_DOC = {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to spend some more time on this since adding this and then removing it right away just adds code churn. You should be able to call wrap() with CONFORMANCE_DOC and get a JSON encoded Value proto. You may have to add an overload so you can control the useProto3JSON flag (and as discussed, this only works in api_internal)

];

// This is built by Java(proto3 json formatter) with above document and one additional field of `pathValue: db.doc('col1/ref1')`.
const BUNDLES_PROTOB3_CONFORMANCE_TEMPLATE = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please fix though :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants