Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Restrict use of backend-specific note identifiers. #888

Merged
merged 2 commits into from
Aug 7, 2023

Conversation

nuttycom
Copy link
Contributor

@nuttycom nuttycom commented Aug 2, 2023

This builds atop #886

@nuttycom nuttycom force-pushed the feature/tx_output_note_refs branch from 5ed49a4 to bede07d Compare August 2, 2023 20:31
@nuttycom nuttycom requested review from str4d and daira August 2, 2023 20:31
@codecov
Copy link

codecov bot commented Aug 2, 2023

Codecov Report

Patch coverage: 83.63% and project coverage change: +0.11% 🎉

Comparison is base (cdb904a) 70.89% compared to head (ef0b27c) 71.01%.
Report is 3 commits behind head on main.

❗ Current head ef0b27c differs from pull request most recent head f602ec1. Consider uploading reports for the commit f602ec1 to get more accurate results

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #888      +/-   ##
==========================================
+ Coverage   70.89%   71.01%   +0.11%     
==========================================
  Files         131      131              
  Lines       12656    12658       +2     
==========================================
+ Hits         8973     8989      +16     
+ Misses       3683     3669      -14     
Files Changed Coverage Δ
zcash_client_sqlite/src/wallet/init.rs 53.57% <ø> (ø)
zcash_client_backend/src/data_api.rs 44.18% <53.84%> (+3.20%) ⬆️
zcash_client_sqlite/src/lib.rs 68.68% <84.61%> (+0.94%) ⬆️
zcash_client_sqlite/src/wallet/sapling.rs 80.15% <85.71%> (+2.12%) ⬆️
zcash_client_backend/src/data_api/wallet.rs 85.83% <100.00%> (+0.05%) ⬆️
zcash_client_sqlite/src/wallet.rs 79.28% <100.00%> (+1.37%) ⬆️
...wallet/init/migrations/sapling_memo_consistency.rs 57.77% <100.00%> (+0.95%) ⬆️

... and 1 file with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@pacu pacu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UTACK

Left non blocking comment on possible breaking API

@@ -441,7 +440,7 @@ pub fn create_proposed_transaction<DbT, ParamsT, InputsErrT, FeeRuleT>(
min_confirmations: NonZeroU32,
change_memo: Option<MemoBytes>,
) -> Result<
DbT::TxRef,
TxId,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Non - Blocking] Although small, this is a great improvement!

I'm mostly sure it will break the ffi layer so it will be good to track this hiccup in the ffi repository on iOS and the android sdk r

@nuttycom nuttycom force-pushed the feature/tx_output_note_refs branch 4 times, most recently from 0181c95 to ef0b27c Compare August 7, 2023 14:50
Copy link
Contributor

@str4d str4d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed ef0b27c.

zcash_client_backend/src/data_api.rs Outdated Show resolved Hide resolved
zcash_client_backend/src/data_api.rs Show resolved Hide resolved
zcash_client_backend/src/data_api.rs Show resolved Hide resolved
zcash_client_backend/src/data_api.rs Outdated Show resolved Hide resolved
zcash_client_backend/src/data_api.rs Outdated Show resolved Hide resolved
zcash_client_backend/CHANGELOG.md Outdated Show resolved Hide resolved
ReceivedNoteId(i64),
}

impl fmt::Display for NoteId {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I confirmed that this Display impl was added in ffd5031 so that NoteId could be used as a generic parameter of crate::data_api::error::Error, but 9a7dc0d removed that generic parameter from the latter, so we don't need a Display impl on ReceivedNoteId.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, I missed that the NoteId generic parameter on Error was actually just renamed to NoteRef, so we did still need a Display impl. Fixed in #890.

Comment on lines +220 to 224
let sent_memo = wallet::get_sent_memo(self.conn.borrow(), note_id)?;
if sent_memo.is_some() {
Ok(sent_memo)
} else {
wallet::get_received_memo(self.conn.borrow(), note_id)
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This method is now concretely less efficient than before. I don't know by how much, but it is now O(num_received + num_sent + 2*num_internal), because we have two separate table joins against transactions (instead of a direct query of an indexed column), and for received notes we always query the sent notes table first. We'll know more once we get this into the mobile SDKs, so this is non-blocking, but we should track its effect on the history-rendering app views.

More non-blocking thoughts: Could we gain back or amortize some of the performance loss by combining the new queries into a single query against all three tables at once? AFAICT this is the only place that wallet::get_sent_memo and wallet::get_received_memo are used (they were removed from the public API in 0.7.0), so we can just have a single backing function. This does become slightly more annoying given the sapling_received_notes vs sent_notes split, but we can put off dealing with that to the Orchard work.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I contemplated doing this as a single query using MINUS and/or UNION but I figure that if fetching the memos is done in a hot loop over all the notes, there's something wrong elsewhere.

zcash_client_sqlite/src/wallet.rs Show resolved Hide resolved
@str4d
Copy link
Contributor

str4d commented Aug 7, 2023

Needs a rebase to fix merge conflicts after #889 was merged.

nuttycom and others added 2 commits August 7, 2023 11:27
In general, it is preferable to use globally relevant identifiers where
possible. This PR removes the `WalletRead::TxRef` associated type in
favor of using `TxId` directly for the transaction identifier, and
restricts the use of the `NoteRef` type to those scenarios where the
result of one query is intended to be used directly as the input to
another query.

Closes zcash#834
@nuttycom nuttycom force-pushed the feature/tx_output_note_refs branch from 47ac5cb to f602ec1 Compare August 7, 2023 17:28
@nuttycom nuttycom requested a review from str4d August 7, 2023 17:28
Copy link
Contributor

@str4d str4d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

utACK f602ec1

@str4d str4d merged commit f6eef20 into zcash:main Aug 7, 2023
10 checks passed
@nuttycom nuttycom deleted the feature/tx_output_note_refs branch August 8, 2023 17:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants