Split hush unity build: move ~14k lines of implementation from headers into hush_impl.cpp and hush_nSPV_impl.cpp

- Add HUSH_PRIVATE_IMPLEMENTATION guard to hush.h separating implementation from declarations
- Create hush_impl.cpp to compile all hush_*.h implementation as a separate translation unit
- Create hush_nSPV_impl.cpp to compile all hush_nSPV_*.h headers separately
- main.cpp now includes declarations only, no longer a unity build for hush code
- Add both new compilation units to Makefile.am
This commit is contained in:
dan_s
2026-02-26 20:55:12 -06:00
parent 896fa9c107
commit 3d4e25e429
6 changed files with 364 additions and 7 deletions

123
doc/codebase-cleanup.md Normal file
View File

@@ -0,0 +1,123 @@
# Codebase Cleanup & Organization Options
## 1. Remove Build Artifacts from the Working Tree
The `depends/` directory alone is **1.8 GB**, with 479M in `depends/sources/`, 302M in `depends/built/`. The `release/` dir is 395M, and `repos/` is 164M. While `release/` and `repos/` are already in `.gitignore`, the `depends/built/` and `depends/work/` outputs are not explicitly ignored.
**Actions:**
- [x] Add `depends/built/`, `depends/work/`, `depends/x86_64-w64-mingw32/` to `.gitignore`
- [ ] Run `git clean -fdx` on build output directories periodically
- [ ] The `.o` and `.a` files in `src/` (e.g., 22M `libbitcoin_server.a`) are build outputs cluttering the source tree
## 2. Move Large Binary Params Out of Git History
`sapling-spend.params` (46M) and `sapling-output.params` (3.5M) are committed directly. These are cryptographic proving keys from the Zcash Sapling trusted setup ceremony and **cannot be regenerated**. The node shuts down without them.
**Options:**
- [ ] Use **Git LFS** to track them instead
- [ ] Download them at build/install time via a script (like Zcash's `fetch-params.sh`)
- [x] Keep as-is (current deliberate design choice — guarantees out-of-box operation)
## 3. Clean Up Tracked Editor/Autoconf Backup Files
`src/univalue/configure~` is a tracked backup file. The root `configure~` also exists. These should be removed from tracking and the `.gitignore` pattern `*~` should catch them going forward.
- [x] Remove `src/univalue/configure~` from git tracking
## 4. Split Monolithic Source Files
Several files are extremely large and would benefit from decomposition:
| File | Lines | Suggestion |
|------|-------|------------|
| `src/main.cpp` | 8,217 | Split validation, block processing, mempool logic into separate files |
| `src/wallet/rpcwallet.cpp` | 6,392 | Group RPC methods by category (send, receive, list, z-operations) |
| `src/chainparams.cpp` | 5,420 | Extract checkpoint data and chain-specific params to separate files |
| `src/wallet/wallet.cpp` | 5,059 | Split wallet transaction logic from key management |
- [ ] Split `src/main.cpp`
- [ ] Split `src/wallet/rpcwallet.cpp`
- [ ] Split `src/chainparams.cpp`
- [ ] Split `src/wallet/wallet.cpp`
## 5. Move Implementation Out of Header Files
There are **13,652 lines** of implementation code in `hush_*.h` headers:
| Header | Lines |
|--------|-------|
| `src/hush_utils.h` | 2,549 |
| `src/hush_gateway.h` | 2,531 |
| `src/hush_bitcoind.h` | 1,867 |
| `src/hush_curve25519.h` | 1,017 |
| `src/hush_nSPV_superlite.h` | 977 |
| `src/hush_nSPV_fullnode.h` | 914 |
| `src/hush_defs.h` | 656 |
| `src/hush_nSPV.h` | 603 |
| `src/hush_nSPV_wallet.h` | 505 |
| `src/hush_notary.h` | 469 |
| `src/hush_globals.h` | 360 |
| `src/hush_ccdata.h` | 272 |
| `src/hush_kv.h` | 204 |
| `src/hush_nSPV_defs.h` | 193 |
- [x] Move implementations from `hush_*.h` headers to `.cpp` files
- Created `src/hush_impl.cpp` — compiles all hush_*.h implementation via `HUSH_PRIVATE_IMPLEMENTATION` guard
- Created `src/hush_nSPV_impl.cpp` — compiles all hush_nSPV_*.h implementation
- Modified `src/hush.h` with `#ifdef HUSH_PRIVATE_IMPLEMENTATION` to separate implementation from declarations
- `main.cpp` now includes declarations only (no longer a unity build for hush code)
## 6. Deduplicate Vendored Code
`cJSON` exists in **4 copies**:
- `src/cJSON.c` + `src/cJSON.h`
- `src/hush_cJSON.c` + `src/hush_cJSON.h`
- `src/cc/dapps/cJSON.c`
- `src/cc/includes/cJSON.h`
- [ ] Consolidate to a single copy and have other modules link against it
## 7. Relocate Shell Scripts Out of `src/`
There are **15+ shell scripts and .bat files** tracked inside `src/`:
- `dragonx-cli`, `dragonxd`, `tush-cli`, `tushd`, `zush`
- `smartchains`, `purge`, `listassetchains`, `listassetchainparams`
- `testdragonx-cli`, `testdragonxd`, `testequihash-cli`, `testequihashd`
- `assetchains_stop`, `hush-arrakis-chain`
- `.bat` variants: `dragonx-cli.bat`, `dragonxd.bat`, `hush-arrakis-chain.bat`
- [x] Move shell scripts to `contrib/scripts/` directory
## 8. ~~Organize the `repos/` Directory~~ (N/A — `repos/` is gitignored)
## 9. Improve `.gitignore` Coverage
The current `.gitignore` has redundant and scattered entries (e.g., `src/cc/dapps/a.out` listed twice, many game-related paths for a rogue game that appears obsolete).
- [x] Group entries by category (build outputs, editor files, platform-specific)
- [x] Remove stale entries for files/features that no longer exist
- [x] Add `depends/built/`, `depends/work/`, `depends/x86_64-*/`
## 10. Consolidate Documentation
Docs are split across `doc/`, `contrib/README.md`, and the root.
- [ ] Create a `doc/architecture.md` describing the module structure
---
## Recommended Priority
**Highest-impact, lowest-risk changes** (don't touch compiled code):
1. Items **1, 3, 9**`.gitignore` cleanup and removing tracked backup files
2. Item **7** — relocate scripts out of `src/`
**Higher-impact, higher-risk changes** (affect build system and code):
4. Item **5** — move implementation out of headers
5. Item **4** — split monolithic source files
6. Item **6** — deduplicate cJSON
7. Item **2** — move params to LFS or download script

View File

@@ -277,6 +277,8 @@ libbitcoin_server_a_SOURCES = \
i2p.cpp \
init.cpp \
dbwrapper.cpp \
hush_impl.cpp \
hush_nSPV_impl.cpp \
main.cpp \
merkleblock.cpp \
metrics.h \

View File

@@ -27,6 +27,12 @@
#define HUSH_SMART_CHAINS_WAITNOTARIZE
#define HUSH_PAXMAX (10000 * COIN)
extern int32_t NOTARIZED_HEIGHT;
#ifdef HUSH_PRIVATE_IMPLEMENTATION
// ===================================================================
// Full implementation mode — only compiled via hush_impl.cpp
// ===================================================================
uint256 NOTARIZED_HASH,NOTARIZED_DESTTXID;
#include <stdint.h>
@@ -56,6 +62,9 @@ int32_t hush_parsestatefile(struct hush_state *sp,FILE *fp,char *symbol,char *de
#include "hush_events.h"
#include "hush_ccdata.h"
// Need NSPV struct definitions for hush_currentheight() below
#include "hush_nSPV_defs.h"
void hush_currentheight_set(int32_t height)
{
char symbol[HUSH_SMART_CHAIN_MAXLEN],dest[HUSH_SMART_CHAIN_MAXLEN]; struct hush_state *sp;
@@ -980,4 +989,162 @@ int32_t hush_connectblock(bool fJustCheck, CBlockIndex *pindex,CBlock& block)
}
}
#else
// ===================================================================
// Declarations-only mode — for files that call hush functions
// This provides type info and function declarations without pulling
// in 14,000+ lines of implementation code.
// ===================================================================
extern uint256 NOTARIZED_HASH, NOTARIZED_DESTTXID;
#include <stdint.h>
#include <stdio.h>
#include <pthread.h>
// Struct/type declarations from hush_structs.h
#include "hush_structs.h"
// nSPV type definitions and forward declarations
#include "hush_nSPV_defs.h"
// Extern globals defined in hush_globals.h but missing from hush_defs.h
extern uint8_t ASSETCHAINS_BURN;
extern uint32_t ASSETCHAINS_MINOPRETURNFEE;
extern uint32_t HUSH_STOPAT;
extern int32_t HUSH_REWIND;
// Macros from implementation headers needed by callers
#define BURN_ADDRESS "RD6GgnrMpPaTSMn8vai6yiGA7mN4QGPVMY"
#ifndef SATOSHIDEN
#define SATOSHIDEN ((uint64_t)100000000L)
#endif
#ifndef dstr
#define dstr(x) ((double)(x) / SATOSHIDEN)
#endif
#define HUSH_NOTARIES_HEIGHT1 814000
// Forward declarations — hush.h implementation functions
void hush_currentheight_set(int32_t height);
int32_t hush_currentheight();
int32_t hush_parsestatefile(struct hush_state *sp,FILE *fp,char *symbol,char *dest);
int32_t hush_parsestatefiledata(struct hush_state *sp,uint8_t *filedata,long *fposp,long datalen,char *symbol,char *dest);
int32_t memread(void *dest,int32_t size,uint8_t *filedata,long *fposp,long datalen);
void hush_stateupdate(int32_t height,uint8_t notarypubs[][33],uint8_t numnotaries,uint8_t notaryid,uint256 txhash,uint64_t voutmask,uint8_t numvouts,uint32_t *pvals,uint8_t numpvals,int32_t HUSHheight,uint32_t HUSHtimestamp,uint64_t opretvalue,uint8_t *opretbuf,uint16_t opretlen,uint16_t vout,uint256 MoM,int32_t MoMdepth);
int32_t hush_validate_chain(uint256 srchash,int32_t notarized_height);
int32_t hush_voutupdate(bool fJustCheck,int32_t *isratificationp,int32_t notaryid,uint8_t *scriptbuf,int32_t scriptlen,int32_t height,uint256 txhash,int32_t i,int32_t j,uint64_t *voutmaskp,int32_t *specialtxp,int32_t *notarizedheightp,uint64_t value,int32_t notarized,uint64_t signedmask,uint32_t timestamp);
int32_t hush_notarycmp(uint8_t *scriptPubKey,int32_t scriptlen,uint8_t pubkeys[64][33],int32_t numnotaries,uint8_t rmd160[20]);
int32_t hush_connectblock(bool fJustCheck, CBlockIndex *pindex,CBlock& block);
// Forward declarations — hush_globals.h functions
int32_t hush_baseid(char *origbase);
uint64_t hush_current_supply(uint32_t nHeight);
std::string devtax_scriptpub_for_height(uint32_t nHeight);
std::string devtax_address_for_height(uint32_t nHeight);
// Forward declarations — hush_utils.h functions
void vcalc_sha256(char deprecated[(256 >> 3) * 2 + 1],uint8_t hash[256 >> 3],uint8_t *src,int32_t len);
bits256 bits256_doublesha256(char *deprecated,uint8_t *data,int32_t datalen);
void calc_rmd160(char deprecated[41],uint8_t buf[20],uint8_t *msg,int32_t len);
uint32_t calc_crc32(uint32_t crc,const void *buf,size_t size);
void calc_rmd160_sha256(uint8_t rmd160[20],uint8_t *data,int32_t datalen);
int32_t bitcoin_addr2rmd160(uint8_t *addrtypep,uint8_t rmd160[20],char *coinaddr);
char *bitcoin_address(char *coinaddr,uint8_t addrtype,uint8_t *pubkey_or_rmd160,int32_t len);
int32_t hush_is_issuer();
int32_t bitweight(uint64_t x);
int32_t _unhex(char c);
int32_t is_hexstr(char *str,int32_t n);
int32_t unhex(char c);
int32_t decode_hex(uint8_t *bytes,int32_t n,char *hex);
char hexbyte(int32_t c);
int32_t init_hexbytes_noT(char *hexbytes,unsigned char *message,long len);
char *bits256_str(char hexstr[65],bits256 x);
int32_t dragon_rwnum(int32_t rwflag,uint8_t *serialized,int32_t len,void *endianedp);
int32_t dragon_rwbignum(int32_t rwflag,uint8_t *serialized,int32_t len,uint8_t *endianedp);
int32_t hush_scriptitemlen(int32_t *opretlenp,uint8_t *script);
uint64_t hush_block_prg(uint32_t nHeight);
int64_t hush_block_unlocktime(uint32_t nHeight);
long _stripwhite(char *buf,int accept);
char *clonestr(char *str);
int32_t safecopy(char *dest,char *src,long len);
char *parse_conf_line(char *line,char *field);
double OS_milliseconds();
void OS_randombytes(unsigned char *x,long xlen);
uint16_t _hush_userpass(char *username,char *password,FILE *fp);
void hush_statefname(char *fname,char *symbol,char *str);
void hush_configfile(char *symbol,uint16_t rpcport);
uint16_t hush_userpass(char *userpass,char *symbol);
uint32_t hush_smartmagic(char *symbol,uint64_t supply,uint8_t *extraptr,int32_t extralen);
uint16_t hush_smartport(uint32_t magic,int32_t extralen);
uint16_t hush_port(char *symbol,uint64_t supply,uint32_t *magicp,uint8_t *extraptr,int32_t extralen);
int32_t hush_whoami(char *pubkeystr,int32_t height,uint32_t timestamp);
uint64_t hush_max_money();
uint64_t hush_block_subsidy(int height);
uint64_t hush_sc_block_subsidy(int nHeight);
int8_t equihash_params_possible(uint64_t n, uint64_t k);
void hush_args(char *argv0);
void hush_nameset(char *symbol,char *dest,char *source);
void hush_prefetch(FILE *fp);
// Forward declarations — hush_bitcoind.h functions
struct hush_state *hush_stateptr(char *symbol,char *dest);
void hush_init(int32_t height);
int32_t hush_faststateinit(struct hush_state *sp,char *fname,char *symbol,char *dest);
int32_t hush_isrealtime(int32_t *hushheightp);
int32_t hush_longestchain();
uint64_t hush_paxtotal();
int32_t hush_block2height(CBlock *block);
int32_t hush_block2pubkey33(uint8_t *pubkey33,CBlock *block);
int32_t hush_blockload(CBlock& block, CBlockIndex *pindex);
CBlockIndex *hush_chainactive(int32_t height);
CBlockIndex *hush_getblockindex(uint256 hash);
uint32_t hush_chainactive_timestamp();
char *hush_issuemethod(char *userpass,char *method,char *params,uint16_t port);
int32_t hush_chosennotary(int32_t *notaryidp,int32_t height,uint8_t *pubkey33,uint32_t timestamp);
int32_t hush_notaries(uint8_t pubkeys[64][33],int32_t height,uint32_t timestamp);
bool hush_hardfork_active(uint32_t time);
// Forward declarations — hush_pax.h functions
void hush_paxpricefeed(int32_t height,uint8_t *pricefeed,int32_t len);
// Forward declarations — hush_notary.h functions
int32_t hush_notarized_height(int32_t *prevMoMheightp,uint256 *hashp,uint256 *txidp);
int32_t hush_MoMdata(int32_t *notarized_htp,uint256 *MoMp,uint256 *hushtxidp,int32_t nHeight,uint256 *MoMoMp,int32_t *MoMoMoffsetp,int32_t *MoMoMdepthp,int32_t *hushstartip,int32_t *hushendip);
int32_t hush_notarizeddata(int32_t nHeight,uint256 *notarized_hashp,uint256 *notarized_desttxidp);
// Forward declarations — hush_gateway.h functions
int32_t hush_isPoS(CBlock *pblock,int32_t height,bool fJustCheck);
int32_t hush_checkPOW(int64_t stakeTxValue, int32_t slowflag, CBlock *pblock, int32_t height);
void hush_passport_iteration();
int32_t hush_opretvalidate(const CBlock *block,CBlockIndex *previndex,int32_t height,CScript scriptPubKey);
int32_t hush_scpublic(uint32_t tiptime);
int32_t hush_checknotarypay(CBlock *pblock,int32_t height);
int32_t hush_checkpoint(int32_t *notarized_heightp, int32_t nHeight, uint256 hash);
// Forward declarations — hush_bitcoind.h additional functions
uint64_t the_commission(const CBlock *pblock, int32_t height);
// Forward declarations — hush_events.h functions
void hush_eventadd_pubkeys(struct hush_state *sp,char *symbol,int32_t height,uint8_t num,uint8_t pubkeys[64][33]);
void hush_eventadd_pricefeed(struct hush_state *sp,char *symbol,int32_t height,uint32_t *prices,uint8_t num);
void hush_eventadd_opreturn(struct hush_state *sp,char *symbol,int32_t height,uint256 txid,uint64_t value,uint16_t vout,uint8_t *buf,uint16_t len);
void hush_eventadd_notarized(struct hush_state *sp,char *symbol,int32_t height,char *dest,uint256 notarized_hash,uint256 notarized_desttxid,int32_t notarizedheight,uint256 MoM,int32_t MoMdepth);
void hush_eventadd_hushheight(struct hush_state *sp,char *symbol,int32_t height,int32_t hushheight,uint32_t timestamp);
void hush_event_rewind(struct hush_state *sp,char *symbol,int32_t height);
// Forward declarations — hush_ccdata.h functions
void hush_rwccdata(char *symbol,int32_t rwflag,struct hush_ccdata *ccdata,struct hush_ccdataMoMoM *MoMoMdata);
void hush_purge_ccdata(int32_t height);
// Forward declarations — hush_kv.h / other
int32_t gettxout_scriptPubKey(uint8_t *scriptPubkey,int32_t maxsize,uint256 txid,int32_t n);
bool check_pprevnotarizedht();
// Forward declarations — nSPV functions (defined in hush_nSPV_*.h, compiled in hush_nSPV_impl.cpp)
void hush_nSPVreq(CNode *pfrom,std::vector<uint8_t> request);
void hush_nSPVresp(CNode *pfrom,std::vector<uint8_t> response);
void hush_nSPV(CNode *pto);
#endif // HUSH_PRIVATE_IMPLEMENTATION
#endif

29
src/hush_impl.cpp Normal file
View File

@@ -0,0 +1,29 @@
// Copyright (c) 2016-2024 The Hush developers
// Distributed under the GPLv3 software license, see the accompanying
// file COPYING or https://www.gnu.org/licenses/gpl-3.0.en.html
//
// hush_impl.cpp — Compilation unit for all hush_*.h implementation code.
//
// Previously, the hush implementation headers were #included directly
// into main.cpp (a "unity build" pattern), making main.o enormous and
// coupling everything into one translation unit. This file replaces
// that pattern: it includes hush.h with HUSH_PRIVATE_IMPLEMENTATION
// defined, which pulls in all the implementation headers and compiles
// them as a separate object file (hush_impl.o).
//
// main.cpp now includes hush.h WITHOUT HUSH_PRIVATE_IMPLEMENTATION,
// receiving only forward declarations.
// Standard project headers needed by hush implementation code
#include "main.h"
#include "notarizationdb.h"
#include "init.h"
#include "txdb.h"
#include "hush_nSPV_defs.h"
#include "wallet/asyncrpcoperation_sendmany.h"
#include "wallet/asyncrpcoperation_shieldcoinbase.h"
// Enable full implementation mode and HUSH_ZCASH compatibility flag
#define HUSH_PRIVATE_IMPLEMENTATION
#define HUSH_ZCASH
#include "hush.h"

38
src/hush_nSPV_impl.cpp Normal file
View File

@@ -0,0 +1,38 @@
// Copyright (c) 2016-2024 The Hush developers
// Distributed under the GPLv3 software license, see the accompanying
// file COPYING or https://www.gnu.org/licenses/gpl-3.0.en.html
//
// hush_nSPV_impl.cpp — Compilation unit for nSPV (simplified payment
// verification) implementation headers.
//
// Previously, the nSPV implementation headers were #included directly
// into main.cpp after hush.h. This file replaces that pattern by
// compiling them as a separate object (hush_nSPV_impl.o).
// Standard project headers needed by nSPV code
#include "main.h"
#include "notarizationdb.h"
#include "rpc/server.h"
#include "init.h"
#include "txdb.h"
#include "core_io.h"
#include "key_io.h"
#include "merkleblock.h"
#include "script/sign.h"
#include "cc/CCinclude.h"
#include "wallet/asyncrpcoperation_sendmany.h"
#include "wallet/asyncrpcoperation_shieldcoinbase.h"
// Forward declare functions defined in other compilation units
bool Getscriptaddress(char *destaddr,const CScript &scriptPubKey);
// hush.h in declarations-only mode gives us hush function declarations
// that nSPV code calls (dragon_rwnum, dragon_rwbignum, hush_notarized_height, etc.)
#include "hush.h"
// nSPV implementation headers — defines, structs, serialization, then fullnode/superlite/wallet
#include "hush_nSPV_defs.h"
#include "hush_nSPV.h"
#include "hush_nSPV_fullnode.h"
#include "hush_nSPV_superlite.h"
#include "hush_nSPV_wallet.h"

View File

@@ -721,8 +721,9 @@ CBlockIndex* FindForkInGlobalIndex(const CChain& chain, const CBlockLocator& loc
CCoinsViewCache *pcoinsTip = NULL;
CBlockTreeDB *pblocktree = NULL;
// Hush globals
#define HUSH_ZCASH
// Hush declarations (implementation compiled separately in hush_impl.cpp)
#include "cc/CCinclude.h" // myGetTransaction, zeroid, NSPV_U
#include "key_io.h" // DecodeDestination
#include "hush.h"
UniValue hush_snapshot(int top)
@@ -6855,11 +6856,8 @@ void static ProcessGetData(CNode* pfrom)
}
}
#include "hush_nSPV_defs.h"
#include "hush_nSPV.h" // shared defines, structs, serdes, purge functions
#include "hush_nSPV_fullnode.h" // nSPV fullnode handling of the getnSPV request messages
#include "hush_nSPV_superlite.h" // nSPV superlite client, issuing requests and handling nSPV responses
#include "hush_nSPV_wallet.h" // nSPV_send and support functions, really all the rest is to support this
// nSPV implementation compiled separately in hush_nSPV_impl.cpp
// Forward declarations provided by hush.h (declarations-only mode)
bool static ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv, int64_t nTimeReceived)
{