See here for background: https://bugreports.qt.io/browse/QTBUG-34748
libxcb temporarily had an abi breakage which caused crashes when qt was
compiled against a non-compatible version. Building qt with -qt-xcb should have
shielded us from this issue, except that incompatible headers were used when
building qt's wrapper.
Make sure those headers aren't picked up by qt's build.
Details:
qt's build adds a wrapper around the xcb libs when -qt-xcb is used. This is
done to avoid having to link to a handful of different libs, which may not be
api/abi stable. This build depends on include-order, so that its files are
found before the real libxcb headers.
Our build (for other reasons related to qt's complicated build-system) injects
our prefix into CXXFLAGS. Because libxcb is found in this path, that reverses
the include-order, negating the purpose of the wrapper.
To fix, libxcb's includes are simply moved to a subdir. pkg-config ensures that
they're still found properly when needed.
To make things even more interesting, this behavior in qt's .pro files is broken:
INCLUDEPATH += $$QMAKE_CFLAGS_XCB
The INCLUDEPATH variable is processed by qmake which automatically prefixes each
entry with "-I". The QMAKE_CFLAGS_XCB variable comes from pkg-config and
already contains -I, making the path look like "-I-I/path/to/xcb/headers".
To work around that, CFLAGS/CXXFLAGS are used here rather than INCLUDEPATH.
When re-indexing, there are a few cases where garbage data may be skipped in
the block files. In these cases, the indices are correctly written to the index
db, however the pointer to the next position for writing in the current block
file is calculated by adding the sizes of the valid blocks found.
As a result, when the re-index is finished, the index db is correct for all
existing blocks, but the next block will be written to an incorrect offset,
likely overwriting existing blocks.
Rather than using the sum of all valid blocks to determine the next write
position, use the end of the last block written to the file. Don't assume that
the current block is the last one in the file, since they may be read
out-of-order.
This was added a while ago for testing purposes, but was never intended to be
used. Remove it until upstream libsecp256k1 decides that verification is
stable/ready.
For Gitian releases:
- Windows builds remain unchanged. libstdc++ was already linked statically.
- OSX builds remain unchanged. libstdc++ is tied to the SDK and not worth
messing with.
- Linux builds now statically link libstdc++.
For Travis:
- Match the previous behavior by adding --enable-reduce-exports as
necessary.
- Use static libstdc++ for the full Linux build.
Backwards-compatibility for libstdc++ is not limited to straightforward abi
changes. Symbol visibility also needs to be taken into consideration, and
that really can't be addressed simply.
Instead, just static-link libstdc++ for backwards-compat.
This is really a packager's option. While it's helpful to encourage devs to
test this option for daily builds, it's not reliable in several real-world
use-cases. Some older libstdc++ runtimes (freebsd 9, debian wheezy, for
example) fail to properly catch exceptions due to mismatched type_info.
See https://gcc.gnu.org/bugzilla/show_bug.cgi?id=19664 for more info.
This makes it easier for us to replace it if desired, since it's now only in
one spot. Also, it avoids the openssl include from allocators.h, which
essentially forced openssl to be included from every compilation unit.
Since permissions and timestamps are changed for the sake of determinism,
. must not be added to the archive. Otherwise, tar may try to modify pwd when
extracting.
c++11 (libc++'s stdlib implementation anyway) doesn't allow for map types to be
forward-declared. for example:
class foo;
std::map<int, foo> bar; // error, foo has not been defined.
class foo{};
Since CWallet and CWalletTx are inter-dependent, but only std::map<*,CWalletTx>
is used, forward-declare CWallet instead and define CWalletTx first.
Despite the mangled git diff, this change only amounts to moving ~320 lines in
a single chunk.
The default font changed again.
The real fix is to compile qt against a >= 10.8 sdk, but this is simple enough
to backport to 0.10 to avoid having to do that there.
Note: NSAppKitVersionNumber is a double and there's no official value for
NSAppKitVersionNumber10_10. Since == isn't reliable for doubles, use Apple's
guidelines for testing versions here:
https://developer.apple.com/library/mac/releasenotes/AppKit/RN-AppKit/
Chinese and Japanese fonts have been hard-coded as well, otherwise they fail to
show up at all.
New versions of OpenSSL will reject non-canonical DER signatures. However,
it'll happily decode them. Decode then re-encode before verification in order
to ensure that it is properly consumed.
on rare occasions, rand() was returning duped values, causing duplicate
transactions.
BuildMerkleTree happily used these, but CPartialMerkleTree caught them and
returned a null merkle root.
Rather than taking changes with rand(), use the loop counter to guarantee
unique values.
At sipa's request, also remove the remaining uses of rand().
tl;dr: Update to the newer stable toolchain and SDK for OSX without giving up
any backwards compatibility. We can move to clang 3.5 as a next step which
allows use to use libc++ and the 10.10 sdk, but we'll need to find a build that
works in gitian/travis first.
Switch to a new, better maintained fork of cctools:
https://github.com/tpoechtrager/cctools-port
I've forked this and will be working on it some as well:
https://github.com/theuni/cctools-port
This brings in:
cctools v862
ld64: v241.9
It also fixes 64bit builds, so there's no longer any need to use a 32bit clang.
Since clang is no longer tied to an old/crusty 32bit build, clang has been
upgraded to 3.3. Unfortunately, there's a bug in 3.4 that breaks builds. 3.5
works fine, but there are no binary builds compatible with precise, which is
currently used for gitian and travis. We could always build our own if
necessary.
After updating to stable clang/linker/cctools, it's possible to use a more
recent SDK. The current SDK (10.7) through the most recent 10.10 have all been
built/tested successfully, both with and without 10.6 compatibility. However,
10.10 requires clang 3.5.
SDKs >= 10.9 use libc++ rather than libstdc++. This is verified working as well.
Broken hash logic caused all depends on some platforms (osx at least) to end up
with the same build-id. Without this fix, nothing will be rebuilt when recipes
or dependencies change.
Since the last commit will force rebuilds of all depends, take the opportunity
to clean up a few other things that would trigger rebuilds as well.
- Move source stamps to the sources dir so that SOURCES_PATH is respected for
"make download".
- Only print "fetching..." when actually downloading a file.
- Avoid using non-deterministic paths for the recipe hash (patch location).
This should ensure that all builders get the same resulting build-ids.
- Use a per-package source paths. This will allow for removing old source files
in the future.
- Use a host-agnostic path for downloads which gets cleaned up properly.
- Split linux32/linux64 releases
- Split win32/win64 zips
- Post-processing should no longer be required. The deterministic outputs are
ready for consumption.
Fixes default hidden symbol visibility for our linux->osx cross build. Without
this change, the check for working -fvisibility=hidden fails, and all symbols
are visible by default.
Ugly as this is, it's just a simple find/replace to fix a bug in Qt's configure.
They assume in an "XPLATFORM_MAC" block that the builder is capable of running
osx programs. This should be "BUILD_ON_MAC" instead.
Some users may have libtool libs (.la) installed in their linker search paths.
In this case, using -static-libtool-libs would try to link in .a's instead of
shared libs. That would be harmless unless the .a was built in a way that
would break linking, like non-fpic.
What we really want is "-static" here. Despite its name, it's actually less
aggressive than -static-libtool-libs. It causes only internal libs to be linked
statically (libbitcoinconsensus is the one were'a after).
Descriptors now make use of the dependencies builder, so results are cached.
A very new version (>= e9741525c) of Gitian should be used in order to take
advantage of caching.
We're not ready to switch to a static qt5 for Linux yet due to missing plugin
support. This adds a recipe for building a shared qt4 that we build and link
against, but don't distribute.
make USE_LINUX_STATIC_QT5=1 can be used to build static qt5 as before.
Attempt to codify the possible error statuses associated with script
validation. script/types.h has been created with the expectation that it will
be part of the public lib interface. The other flag enums will be moved here in
a future commit.
Logging has also been removed in order to drop the dependency on core.h. It can
be re-added to bitcoind as-needed. This makes script verification finally free
of application state and boost!
This is a long chain of errors, and there are likely other changes that could
be made to cope in other places along that chain.
If depends don't build successfully, don't bother trying again for the sake of
better logging. That's likely to hurt more than help. In this case, qt build
failed, and on the second attempt, it appeared to be successful. However, due
to a bad object from an internal gcc error on the first build, the resulting
lib was unusable. This caused bitcoin-qt to not be built, and tests and
packaging which expected bitcoin-qt to be there failed.
The root cause:
Mingw is especially crashy when using -jX, likely compounded by low-memory
environments. I've seen multiple problems with this combo in Gitian as well.
In this case:
i686-w64-mingw32-g++: internal compiler error: Killed (program cc1plus)
...
make[3]: *** [.obj/release/qdrawhelper.o] Error 4
The workaround:
Bump Travis down to using -j2 by default. Additionaly, enable --with-gui for
the windows builds. This will cause configure to fail if qt is not working
while also testing the config flag.
Other failures which may be worth revisiting separately:
- If a depends package fails, maybe remove the workdir so that it doesn't taint
subsequent runs
- See if there's anything repeatable about the ICE when building qt
tl;dr: This solves boost visibility problems for default/release build configs
on non-Linux platforms.
When Bitcoin builds against boost's header-only classes, it ends up with
objects containing symbols that the upstream boost libs also have. Since
Bitcoin builds by default with hidden symbol visibility, it can end up trying
to link against a copy of the same symbols with default visibility.
This is not a problem on Linux because 3rd party static libs are un-exported
by default (--exclude-libs,ALL), but that is not available for MinGW and OSX.
Those platforms (and maybe others?) end up confused about which version to use.
The OSX linker spews hundreds of: "ld: warning: direct access in <foo> to
global weak symbol guard variable for <bar> means the weak symbol cannot be
overridden at runtime. This was likely caused by different translation units
being compiled with different visibility settings."
MinGW's linker complains similarly.
Since the default symbol visibility for Bitcoin is hidden and releases are
built that way as well, build Boost with hidden visibility. Linux builds Boost
this way also, but only for the sake of continuity.
This means that the linker confusion logic is reversed, so the problem will
will now be encountered if Bitcoin is built with --disable-reduce-exports, but
that's better than the current situation.
Cory Fields