fix: cap mempool memory usage (-maxmempool) to bound an OOM DoS

This fork never ported Bitcoin's fee-ordered mempool eviction: CTxMemPool has no
TrimToSize/Expire, and LimitMempoolSize's body + call site are both commented out and
reference an undefined DEFAULT_MAX_MEMPOOL_SIZE. So the mempool had no total-size
ceiling. Together with the just-restored removeExpired() and near-free tx admission, a
peer could flood transactions to exhaust every node's memory (incl. pool/payout nodes).

Add a simple admission cap in AcceptToMemoryPool: once the pool exceeds -maxmempool it
refuses new admissions with DoS(0) (no ban -- a full pool isn't the peer's fault). This
is not fee-ordered eviction (that needs the absent TrimToSize machinery) but it bounds
the footprint; removeExpired() already evicts unmineable expired txs on each block
connect. New DEFAULT_MAX_MEMPOOL_SIZE=300 (MB, Bitcoin's default) is far above DragonX's
normal mempool, so normal operation is unaffected. Reviewed: bytes-vs-bytes comparison,
read under LOCK(pool.cs) on a recursive mutex (no deadlock); only the reorg re-add path
and new sends route through it, and only at 300MB.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

src/main.cpp

@@ -2072,6 +2072,12 @@ bool AcceptToMemoryPool(CTxMemPool& pool, CValidationState &state, const CTransa
 
         {
             LOCK(pool.cs);
+            // Bound mempool memory: this fork never ported fee-ordered TrimToSize eviction, so
+            // instead of evicting we refuse new admissions once the pool exceeds -maxmempool.
+            // removeExpired() already clears unmineable expired txs on each block connect; this
+            // caps the total footprint against a flood of otherwise-minable/low-fee txs (OOM DoS).
+            if ( pool.DynamicMemoryUsage() > (size_t)GetArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000 )
+                return state.DoS(0, error("AcceptToMemoryPool: mempool full, rejecting tx %s", hash.ToString()), REJECT_INSUFFICIENTFEE, "mempool-full");
             // Store transaction in memory
             pool.addUnchecked(hash, entry, !IsInitialBlockDownload());
 

src/main.h

@@ -66,6 +66,8 @@ class PrecomputedTransactionData;
 
 struct CNodeStateStats;
 #define DEFAULT_MEMPOOL_EXPIRY 1
+/** Default for -maxmempool, maximum megabytes of mempool memory usage */
+#define DEFAULT_MAX_MEMPOOL_SIZE 300
 #define _COINBASE_MATURITY 100
 
 /** Default for -blockmaxsize and -blockminsize, which control the range of sizes the mining code will create **/