IRC meeting summary for 2017-02-23

• Link to this week logs
• Meeting minutes by meetbot

Main topics

• Git and SHA1 collisions
• Bitcoin Core 0.14.0 release
• Travis CI issue
• Code reorganization

Git and SHA1 Collisions

Background

A few hours before the beginning of the meeting, several researchers announced that they had produced the first case of two different files both having the same hash (a situation called a hash collision) when using the SHA1 hash function.

The Git revision control system used by Bitcoin Core and many other projects uses SHA1 hashes to allow people to ensure that they all have the exact same code. The ability to generate SHA1 collisions undermines that assurance.

Many members of the security community have been requesting Git change from SHA1 to SHA256 or another more-secure hash function for years, but the Git developers have strongly resisted this change (probably because it is backwards incompatible, meaning that it’s likely that all Git users will need to upgrade at roughly the same time).

Comments

Discussion initially focused on clarifying the severity of the situation and then moved towards describing potential ways the Bitcoin Core project can deal with the problem while everyone waits for the Git developers to release a more secure program. Solutions proposed included:

• “I wonder how hard it is to create an overlay that goes back in history, computes a sha256 for every tree and commit, and then include gpg signatures on those?” (Pieter Wuille)

• “We can update our dev scripts to do a sha256sum of all committed files and sign that as well […] eg the merge scripts could include a hash of sha256sum * in the commit message” (Matt Corallo). Gregory Maxwell proposed something very similar at roughly the same time.

• “I have a patch for git to use [sha1collissiondetect] as the hash function and abort() if it detects a potentially-bad hash.” (Matt Corallo)

Conclusion

No explicit conclusion was reached, but presumably developers will investigate some or all of the solutions proposed above while Git developers also work to upgrade their program.

Bitcoin Core 0.14.0 release

Note, this heading covers two related topics from the meeting, (1) “help cfields with adding performance-related release notes” and (2) “RC2 status”.

Background

Bitcoin Core 0.14.0’s first Release Candidate (RC1) was distributed after the previous week’s meeting. No major problems were found but some minor fixes and features remain to be merged.

Many of the major upgrades in 0.14.0 are performance improvements.

Meeting comments

Wladimir van der Laan asked those present to suggest ways to quantify the performance improvements so Cory Fields could perform any suggested measurements and add the results to the release notes in PR#9787.

Gregory Maxwell suggested, “sync with last release takes N hours, sync with new release takes Y.”

Fields agreed, “ok, I’ll add a vague 0.13 vs 0.14 sync time. The 0.13 will take time though, I haven’t had the patience to finish one yet.”

Jeremy Rubin suggested using Amazon EC2 as a testing environment but Wladimir van der Laan cautioned, “EC is not a good comparison environment: sloooow I/O”. Fields replied that, “I used EC for my sync benching because I figured it represented a very typical use-case.”

Discussion moved to the second Release Candidate (RC2), which seemed to be ready to tag in Git except for a minor translations update and PR#9829 which was ready to merge but had failing Travis Continuous Integration (Travis CI) tests.

Conclusion

Cory Fields will perform a test of the initial sync on EC2 (this has since been completed, Bitcoin Core 0.14.0 performed 5.7 times faster at intial sync than Bitcoin Core 0.13.2). The tests for PR#9829 were restarted after its author, Russell Yanofsky, indicated that he thought they failed because of an intermittent Travis CI problem.

Subsequent to the meeting, RC2 was tagged.

Travis CI issues

Background

Bitcoin Core development uses the Travis CI testing platform to run the project’s tests against each Pull Request (PR). The tests are only meant to fail when there’s a problem with the code in a PR, but sometimes the tests fail for reasons related to Travis CI. In the past these problems have included bugs on Travis as well as Bitcoin Core reaching Travis’s resource limits.

Comments

About a week before the meeting, the testing executable test_bitcoin began to fail intermittently during Travis CI tests. Issue 9825 was opened to diagnose the problem and work on solutions.

The meeting comments focused on how the testing code could create stack traces for the build log as well as how the tests could be changed to get executables, core dumps, and other build and testing artifacts out of Travis for analysis by developers.

Conclusion

Keep investigating the failures, isolate the problem, and work on fixing it.

Code reorganizing

Background

Many economic full nodes use Bitcoin Core to enforce the consensus rules. Other programs would benefit from reusing the same code Bitcoin Core uses to ensure they comply with the consensus rules, so several developers have been working over time to make the Bitcoin Core codebase more modular so parts of it can be used in other programs.

Although the goal of making the codebase more modular is well supported by project contributors, actually moving code around tends to break other developer’s pending changes and consumes part of the limited time experts have available to review changes to Bitcoin Core—while providing no new features or performance improvements. This makes code moves a good topic for team discussion so that changes can be agreed upon in advance in a way that minimizes disruption.

Comments

Jeremy Rubin began, “I have a [proof of concept] branch which moves most of the pure data structures to a datastructures directory.” He added, “non-bitcoin specific [data structures]. E.g., prevector.”

Gregory Maxwell replied, “I don’t think any of us care to maintain things like prevector for other usage. Making a good library takes a tremendous amount of work.”

Wladimir van der Laan agreed, “I don’t think a bitcoin-datastructures library makes sense. If we offer a library it should be something useful to bitcoin users.”

Conclusion

No explicit conclusion during the meeting (the meeting ran out of time on this topic). Discussion continued after the meeting with Maxwell and Wladimir van der Laan expanding upon their points.

Comic relief

<jonasschnelli> or syncs are roughly XYZ% faster...
<jonasschnelli> use the ~ and nobody will blame you afterwards. :)

<jeremyrubin> use two ~~ to be extra approximate

<wumpus> it's marketing not science :p hehe

<gmaxwell> but ~~ will just give you the same number you put in!

<jeremyrubin> The is-approximately operator is non-involutive ;)

<gmaxwell> Well people just have no general idea of the impact. Marketing would be saying that it's 2x faster rather than 3x faster because 2x is more plausible. :P

<gmaxwell> sipa: e.g. someday libstdc++ could get something that generalizes prevector, if it did, we'd drop prevector and use that.

<sipa> c++23

<gmaxwell> sipa: C++23 will just integrate libconsensus of course. template cryprocurrency.

Participants

Disclaimer

This summary was compiled without input from any of the participants in the discussion, so any errors are the fault of the summary author and not the discussion participants.