From 4418d1fb6e9f1637ea69416d22dcc7db4b382262 Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Mon, 21 Jun 2021 19:18:44 +0000 Subject: [PATCH 01/15] Update 'doc/developer-notes.md' --- doc/developer-notes.md | 53 ------------------------------------------ 1 file changed, 53 deletions(-) diff --git a/doc/developer-notes.md b/doc/developer-notes.md index c22ae3fe8..c4aa7e9c2 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -36,59 +36,6 @@ class Class } ``` -Doxygen comments ------------------ - -To facilitate the generation of documentation, use doxygen-compatible comment blocks for functions, methods and fields. - -For example, to describe a function use: -```c++ -/** - * ... text ... - * @param[in] arg1 A description - * @param[in] arg2 Another argument description - * @pre Precondition for function... - */ -bool function(int arg1, const char *arg2) -``` -A complete list of `@xxx` commands can be found at http://www.stack.nl/~dimitri/doxygen/manual/commands.html. -As Doxygen recognizes the comments by the delimiters (`/**` and `*/` in this case), you don't -*need* to provide any commands for a comment to be valid; just a description text is fine. - -To describe a class use the same construct above the class definition: -```c++ -/** - * Alerts are for notifying old versions if they become too obsolete and - * need to upgrade. The message is displayed in the status bar. - * @see GetWarnings() - */ -class CAlert -{ -``` - -To describe a member or variable use: -```c++ -int var; //!< Detailed description after the member -``` - -Also OK: -```c++ -/// -/// ... text ... -/// -bool function2(int arg1, const char *arg2) -``` - -Not OK (used plenty in the current source, but not picked up): -```c++ -// -// ... text ... -// -``` - -A full list of comment syntaxes picked up by doxygen can be found at http://www.stack.nl/~dimitri/doxygen/manual/docblocks.html, -but if possible use one of the above styles. - Development tips and tricks --------------------------- From 723a537e64eca17ad58d44b0ef5c2694cf1edf80 Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Mon, 21 Jun 2021 19:27:29 +0000 Subject: [PATCH 02/15] Update 'doc/developer-notes.md' --- doc/developer-notes.md | 31 +++++++++++++++++++++++++++++-- 1 file changed, 29 insertions(+), 2 deletions(-) diff --git a/doc/developer-notes.md b/doc/developer-notes.md index c4aa7e9c2..727f42fd8 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -1,5 +1,32 @@ -Coding -==================== +# Testing a Branch + +To test a branch called `zindexdb` with a fresh clone + +``` +git clone https://git.hush.is/hush/hush3 hush3-testing +cd hush3-testing +git checkout zindexdb +# you need 2GB RAM free per -jN +./build.sh -j2; make; make; make # this deals with build-system race condition bugs + +# we want to test a fresh sync, so backup current data +mv ~/.komodo/{HUSH3,HUSH3-backup} +mkdir ~/.komodo/HUSH3 + +# This is optional but will likely speed up sync time greatly +cp ~/.komodo/{HUSH3-backup,HUSH3}/peers.dat + +echo "zindex=1" >> ~/.komodo/HUSH3/HUSH3.conf + +# This log file is helpful for debugging more and will contain a history of the +# size of the anonset at every block height +./src/hushd &> hushd.log & +# to look at the log +tail -f hushd.log +``` + + +# Coding Various coding styles have been used during the history of the codebase, and the result is not very consistent. However, we're now trying to converge to From f780af1c6ac38ee068a15935bb76c6d58ada8653 Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Mon, 21 Jun 2021 19:34:48 +0000 Subject: [PATCH 03/15] Update 'doc/developer-notes.md' --- doc/developer-notes.md | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/doc/developer-notes.md b/doc/developer-notes.md index 727f42fd8..68e32a9a9 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -25,7 +25,27 @@ echo "zindex=1" >> ~/.komodo/HUSH3/HUSH3.conf tail -f hushd.log ``` +To get a CSV file of the value of the anonset size for every block height: +``` +grep anonset hushd.log | cut -d= -f2 +``` +This only needs to be calculated once, if we can verify it's correct. These are historical values that do not change. These values should match on all nodes: + +``` +46913,2547,2253,294 +46914,2549,2254,295 +46915,2549,2254,295 +46916,2553,2257,296 +46917,2553,2257,296 +``` + +We should also check a recent block height to verify it's working correctly. The big "test" for this `zindexdb` branch is: + + * If you stop a node, and restart, are the stats from `getchaintxtstats` correct, i.e. the anonset stats? For instance, `shielded_pool_size` should be close to 500000, if it's close to or exactly 0, something is wrong. + * Is there a new file called `zindex.dat` in `~/.komodo/HUSH3/` ? + * Is `zindex.dat` 149 bytes ? + # Coding Various coding styles have been used during the history of the codebase, From 8485e334eed72eafa035693ef67a832a3b5ca137 Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Mon, 21 Jun 2021 20:33:18 +0000 Subject: [PATCH 04/15] Update 'doc/developer-notes.md' --- doc/developer-notes.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/doc/developer-notes.md b/doc/developer-notes.md index 68e32a9a9..3633c9e95 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -13,11 +13,14 @@ git checkout zindexdb mv ~/.komodo/{HUSH3,HUSH3-backup} mkdir ~/.komodo/HUSH3 +# Use your previous config as a base +cp ~/.komodo/{HUSH3,HUSH3-backup}/HUSH3.conf +# Add zindex to your node +echo "zindex=1" >> ~/.komodo/HUSH3/HUSH3.conf + # This is optional but will likely speed up sync time greatly cp ~/.komodo/{HUSH3-backup,HUSH3}/peers.dat -echo "zindex=1" >> ~/.komodo/HUSH3/HUSH3.conf - # This log file is helpful for debugging more and will contain a history of the # size of the anonset at every block height ./src/hushd &> hushd.log & From e4e1077d994e799119994ef555146d9e80d87147 Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Mon, 21 Jun 2021 20:51:05 +0000 Subject: [PATCH 05/15] Add 'doc/privacy-basics.md' --- doc/privacy-basics.md | 8 ++++++++ 1 file changed, 8 insertions(+) create mode 100644 doc/privacy-basics.md diff --git a/doc/privacy-basics.md b/doc/privacy-basics.md new file mode 100644 index 000000000..06c792fa0 --- /dev/null +++ b/doc/privacy-basics.md @@ -0,0 +1,8 @@ +# Privacy Basics + + +## Browser Tips + +Firefox is likely the best browser for privacy. Use this to improve + customize: + +http://ebin.city/~werwolf/posts/firefox-hardening-guide/ \ No newline at end of file From d358971963674601c41e34d2fb975369e3948d3e Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Mon, 21 Jun 2021 20:58:04 +0000 Subject: [PATCH 06/15] Update 'doc/privacy-basics.md' --- doc/privacy-basics.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/doc/privacy-basics.md b/doc/privacy-basics.md index 06c792fa0..701d5de20 100644 --- a/doc/privacy-basics.md +++ b/doc/privacy-basics.md @@ -1,8 +1,10 @@ # Privacy Basics - ## Browser Tips Firefox is likely the best browser for privacy. Use this to improve + customize: -http://ebin.city/~werwolf/posts/firefox-hardening-guide/ \ No newline at end of file +http://ebin.city/~werwolf/posts/firefox-hardening-guide/ + +Those are helpful when NOT using a Tor browser. If you are using Tor Browser (which is a fork of Firefox), then you do not need the above guide and should probably use the default that Tor Browser devs provide. + From 559eaa71032ae1cf03101bec37bbb46561b680f0 Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Mon, 21 Jun 2021 20:58:50 +0000 Subject: [PATCH 07/15] Update 'doc/privacy-basics.md' --- doc/privacy-basics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/privacy-basics.md b/doc/privacy-basics.md index 701d5de20..2198a1cb9 100644 --- a/doc/privacy-basics.md +++ b/doc/privacy-basics.md @@ -4,7 +4,7 @@ Firefox is likely the best browser for privacy. Use this to improve + customize: -http://ebin.city/~werwolf/posts/firefox-hardening-guide/ +https://ebin.city/~werwolf/posts/firefox-hardening-guide/ Those are helpful when NOT using a Tor browser. If you are using Tor Browser (which is a fork of Firefox), then you do not need the above guide and should probably use the default that Tor Browser devs provide. From 947173a2ca59aa7b7657b838157df1479ee336ef Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Mon, 21 Jun 2021 21:00:24 +0000 Subject: [PATCH 08/15] Update 'doc/privacy-basics.md' --- doc/privacy-basics.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/privacy-basics.md b/doc/privacy-basics.md index 2198a1cb9..b231520c4 100644 --- a/doc/privacy-basics.md +++ b/doc/privacy-basics.md @@ -6,5 +6,7 @@ Firefox is likely the best browser for privacy. Use this to improve + customize: https://ebin.city/~werwolf/posts/firefox-hardening-guide/ +which is a guide around using and customizing the [Arkenfox](https://github.com/arkenfox/user.js/wiki/1.3-Implementation) user.js project + Those are helpful when NOT using a Tor browser. If you are using Tor Browser (which is a fork of Firefox), then you do not need the above guide and should probably use the default that Tor Browser devs provide. From 4bba0e2a5867385c92f3e071acadab003d7f4a11 Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Mon, 21 Jun 2021 21:28:55 +0000 Subject: [PATCH 09/15] Update 'doc/developer-notes.md' --- doc/developer-notes.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/doc/developer-notes.md b/doc/developer-notes.md index 3633c9e95..618d45ffe 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -10,14 +10,16 @@ git checkout zindexdb ./build.sh -j2; make; make; make # this deals with build-system race condition bugs # we want to test a fresh sync, so backup current data -mv ~/.komodo/{HUSH3,HUSH3-backup} +TIME=`perl -e "print time"` +mv ~/.komodo/{HUSH3,HUSH3-backup-$TIME} mkdir ~/.komodo/HUSH3 # Use your previous config as a base -cp ~/.komodo/{HUSH3,HUSH3-backup}/HUSH3.conf +cp ~/.komodo/{HUSH3-backup-$TIME,HUSH3}/HUSH3.conf # Add zindex to your node echo "zindex=1" >> ~/.komodo/HUSH3/HUSH3.conf + # This is optional but will likely speed up sync time greatly cp ~/.komodo/{HUSH3-backup,HUSH3}/peers.dat From bdf30a90a9a5e8987dd6d82d4481df5eb0d60eab Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Mon, 21 Jun 2021 21:53:01 +0000 Subject: [PATCH 10/15] Update 'doc/developer-notes.md' --- doc/developer-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/developer-notes.md b/doc/developer-notes.md index 618d45ffe..b54e2b2f9 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -32,7 +32,7 @@ tail -f hushd.log To get a CSV file of the value of the anonset size for every block height: ``` -grep anonset hushd.log | cut -d= -f2 +grep 'setting zstats' hushd.log | cut -d= -f2 ``` This only needs to be calculated once, if we can verify it's correct. These are historical values that do not change. These values should match on all nodes: From ea41d0428686169a9856efe5889e3ef86d528011 Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Mon, 21 Jun 2021 22:02:30 +0000 Subject: [PATCH 11/15] Update 'doc/developer-notes.md' --- doc/developer-notes.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/doc/developer-notes.md b/doc/developer-notes.md index b54e2b2f9..2533e1ac5 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -32,10 +32,13 @@ tail -f hushd.log To get a CSV file of the value of the anonset size for every block height: ``` -grep 'setting zstats' hushd.log | cut -d= -f2 +grep 'setting zstats' hushd.log | cut -d= -f2 > anonset.csv ``` -This only needs to be calculated once, if we can verify it's correct. These are historical values that do not change. These values should match on all nodes: +This only needs to be calculated once, if we can verify it's correct. These are historical values that do not change. The goal is a web page with a historical view of the HUSH anonset size. + + +These values should match on all nodes: ``` 46913,2547,2253,294 From add73be7b53b9ce4e8025db1d9673965927c1b43 Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Mon, 21 Jun 2021 22:14:01 +0000 Subject: [PATCH 12/15] Update 'doc/developer-notes.md' --- doc/developer-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/developer-notes.md b/doc/developer-notes.md index 2533e1ac5..3ae91bb41 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -32,7 +32,7 @@ tail -f hushd.log To get a CSV file of the value of the anonset size for every block height: ``` -grep 'setting zstats' hushd.log | cut -d= -f2 > anonset.csv +grep anonset hushd.log | cut -d= -f2 > anonset.csv ``` This only needs to be calculated once, if we can verify it's correct. These are historical values that do not change. The goal is a web page with a historical view of the HUSH anonset size. From 0a4b8ed5bfaba2695aba27153d77cc7d11aa0989 Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Tue, 22 Jun 2021 00:38:02 +0000 Subject: [PATCH 13/15] Update 'doc/developer-notes.md' --- doc/developer-notes.md | 28 ++++++++++++++++++++++++++-- 1 file changed, 26 insertions(+), 2 deletions(-) diff --git a/doc/developer-notes.md b/doc/developer-notes.md index 3ae91bb41..12a1dc9f9 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -1,8 +1,32 @@ -# Testing a Branch +# Fresh sync -To test a branch called `zindexdb` with a fresh clone +Many times, you will want to do a "fresh sync" test, to verify code works when syncing from the genesis block, which is a different code path than a "partial sync" which means you already have part of blockchain history and are "catching up" to get in sync. + +A "fresh sync" goes thru the Initial Blockchain Download (IBD) optimized codepath and is often faster than than rescanning all of history. Both ways of testing any important change should be done. + +One way is: ``` +cd ~/.komodo/HUSH3 +rm blocks chainstate database notarizations hushstate +``` + +If you are using `zindex=1` then you need to also delete zindex.dat + +``` +cd ~/.komodo/HUSH3 +rm zindex.dat blocks chainstate database notarizations hushstate +``` + +It's possible to confused hush if you ran old code, stop, restart, and then write out zindex.dat that is incorrect, with later hushds will load from disk and believe. + + +# Testing a Branch + +To test a branch called `zindexdb` with a fresh clone: + +``` +# TODO: this should probably become a script in ./contrib git clone https://git.hush.is/hush/hush3 hush3-testing cd hush3-testing git checkout zindexdb From c614e5adb3214e290154902eacf6b27e595ac676 Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Tue, 22 Jun 2021 00:41:34 +0000 Subject: [PATCH 14/15] Update 'doc/developer-notes.md' --- doc/developer-notes.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/doc/developer-notes.md b/doc/developer-notes.md index 12a1dc9f9..e73e8bce9 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -2,9 +2,11 @@ Many times, you will want to do a "fresh sync" test, to verify code works when syncing from the genesis block, which is a different code path than a "partial sync" which means you already have part of blockchain history and are "catching up" to get in sync. -A "fresh sync" goes thru the Initial Blockchain Download (IBD) optimized codepath and is often faster than than rescanning all of history. Both ways of testing any important change should be done. +A "fresh sync" goes thru the Initial Blockchain Download (IBD) optimized codepath and is often faster than than rescanning all of history. Fresh sync and partial sync testing any important change should be done for all important changes. -One way is: +A fresh sync preserves peers.dat, so it will always be faster than a "fresh clone", which has to learn enough p2p peers to being syncing, which can often add many minutes to completing a sync. When code related to peers.dat changes (such in the `p2p` branch) then doing a fresh clone is needed to fully test it. + +One way fresh sync is: ``` cd ~/.komodo/HUSH3 From bad7adb778806dbf2196aa113cf1403b4d847118 Mon Sep 17 00:00:00 2001 From: Duke Leto Date: Tue, 22 Jun 2021 00:42:20 +0000 Subject: [PATCH 15/15] Update 'doc/developer-notes.md' --- doc/developer-notes.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/doc/developer-notes.md b/doc/developer-notes.md index e73e8bce9..26be1e262 100644 --- a/doc/developer-notes.md +++ b/doc/developer-notes.md @@ -6,7 +6,7 @@ A "fresh sync" goes thru the Initial Blockchain Download (IBD) optimized codepat A fresh sync preserves peers.dat, so it will always be faster than a "fresh clone", which has to learn enough p2p peers to being syncing, which can often add many minutes to completing a sync. When code related to peers.dat changes (such in the `p2p` branch) then doing a fresh clone is needed to fully test it. -One way fresh sync is: +One way to do a fresh sync is: ``` cd ~/.komodo/HUSH3