Peter did a code review on Spark that correlates to the white paper and test coverage assessments. So far it is pretty good.
Levon finished fixing the isMine() bug. However, it results in some RPC tests failing, which are being fixed.
Transaction indexing is fixed, which allows Explorer to lookup Spark-type addresses.
Levon edited the mint creation limit to set it to no limit.
Levon will be fixing the Spark spend issue where exceeding the maximum limit will result in failed transaction creation due to unoptimized code.
Peter had some outstanding PRs regarding refactors, and they will be finished soon.
Firo Client
Sproxet is almost finished with the test for Gui-client before moving on to transaction creation.
Miscellaneous
Bitcoin IP leak issue: Peter is investigating this. He doesn’t think we have it. However, this is probably a bad thing since we don’t have it. This is because transactions that never get broadcast to the mempool will not be rebroadcast in the future or ever (Daemon).GUI wallets have a function to rebroadcast, but not in daemon.
Ledger-live: Gevorg worked on the Firo integration. He will be meeting with Levon later this week to check his code.
Ledger-live: Fixed the build issue. To start testing.
Peter
worked on the PR to allow resending wallet transactions only once without leaking the sender’s IP address (Bitcoin inherited bug in some edge cases). Had to port some code from Bitcoin 0.24. There are a few tests before pushing the PR.
Worked on the PR that supports multiple wallets and increases performance, a prerequisite for the Spark refactor PR.
Newer versions of Bitcoin code assumes the possibility of multiple wallets being loaded onto the daemon which Firo does not. This results in some undesired behaviour and crashes during certain RPC calls.
Identified and working to improve reindex/wallet scan performance. Identified unnecessary reads and writes to the wallet database file, which impacted the performance for large wallets. Estimated 15-20% speed increase once fixed.
Spark: There are minor issues, and will issue a PR regarding the fix by the end of the week.
Levon
Spark: Worked on Spark mobile lib. To finalize the documentation for it
Spark: To implement unit tests for testing the transaction creation flow for mobile to simulate the process of calling several functions. To cleanup and fix the new unit tests next. 3–4 tests were made, and to continue thinking of new ideas for testing.
Assisting the Arcadia team on QT implementation with tests.
Ledger-live: Started Gevorg’s ledger live code review.
Minor Spark improvement
Sarang proposed a minor change to allow better hardware wallet support in a more adversarial scenario where you are also assuming that the helper device (laptop/mobile) could be compromised and can redirect funds (even though they don’t have access to the spend key)
Levon
Last Week:
-Assisting Arcadia
-implementing full API unitest for mobile lib
-adding subtractFeeFromAmount for spark mint creation
-Reviewed #PR1237
This week:
-Cleanup and provide mobile design documents
-Remove spend input number limit
-Change spark address prefix from p to s
Peter
Last week:
-wallet resend PR
-multiple wallets PR (will be completed next week)
-PR regarding minor spark fixes (tmr?)
This week:
-spark, multiple wallets
Gevorg
Last Week:
-(Ledger) Testing
-(Ledger) Fix errors
This Week:
-(Ledger) Continue testing
Levon
Last Week:
-Added subtract fee from amount option for mint creation
-Removed spend number limit and other minor fixes
This Week:
-Continue testing
-Preparing for devnet restart
Peter
Last week:
-minor fixes for the spark
-completion of wallet tx rebroadbast (need testing)
-multiple wallet fix
This week:
-spark testing and fixes
Sproxet
-working on tests for public transaction creation. Found a few bugs. Coverage is coming along. Will then need some adaptation for Spark.
Gevorg
-Still testing and fixing errors
-Finished with bridge now testing account syncing
Peter worked on a patch for multiple wallet/improvements for Spark PR. Ideally, the final form of this PR will have only one instance of CwalletDB.
Peter reviewed the code regarding Firo-ElectrumX/Electrum-Firo, Sproxet’s, and Levon’s.
More PRs regarding wallet refactoring are being done.
Levon worked on fixing minor spark issues and failing tests.
Peter will be preparing a devnet relaunch the day after this meeting.
Firo-Client
Sproxet continues work on transaction creation and undefined behavior issues. There are a couple of tests that need to be done for transaction creation.
Sproxet will look into the Masternode Firo Client issue, where it was reported that some masternodes are not displaying on the masternode tab list.
Miscellaneous
Ledger-live: Gevorg will be fixing some issues and is nearly done with testing. It will be finalized this week.
Devnet was restarted by Peter; the HF (devnet) for Spark is 1500 (the current block value is 877). Any deal-breaking issues must be resolved before the hard fork block because it will otherwise require several components to be reset (mining, masternode, etc.) and is very troublesome.
Levon investigates and fixes the spend conflict issues.
Spark UI: Levon was investigating and resolving current UI testing issues. Arcadia’s team was notified and reported that the fixes were working. Testing of the UI can begin shortly after.
Peter continued work on multi wallet support by the client and also, optimizing Lelantus Spark code by removing unnecessary reads and writes from the disk.
Spark QT: Levon started reviewing the changes made to the QT code by Arcadia to support Lelantus Spark, and it will be tested on Devnet this week. If everything goes well, Testnet will be ready to be launched within a week or two.
Reviews of GUI code and testing are still ongoing, and Spark Devnet Explorer needs to be updated.
Peter worked on a build problem on Fedora (Issue 1240) reported by erciccione, Unfortunately, another problem appeared, and will need to use a virtual machine to test fixes.
Firo
Sproxet is finishing up on public transaction creation logic code tests and moving on to the private transaction creation part.
Miscellaneous
Ledger-live: Gevorg is continuing work on the integration and finalizing it.
Levon reviewed code from Arcadia and assisted in UI implementation.
Peter worked on fixing compilation issues. A more recent version of autotools was used. Fixing these issues will avoid problems in future versions of Ubuntu/Debian. The problem lies with the bls-dash library. The build process changed sometime and was incompatible with GUIX. Peter will be pushing a quick fix for Fedora soon and also will need to update the bls library later.
Current issues:
Wallet crashes with UI; currently investigating by Levon
Devnet explorer; worked by Levon
Spark QT issues: address book, anonymize all, and crashes
Peter will be launching proper quorums on devnet
Peter will be working on wallet refactoring and associated syncing issues; PR will be pushed in two separate parts.
Firo
Sproxet worked on unifying transaction creation between different types of transactions, so CoinControl will only have one logic to select everything. Other parts of transaction creation besides input selection might have multiple logics, but input selection will be one logic for all types. Unifying transaction creation with abstract-based classes for different types of transaction outputs.
Miscellaneous
Ledger-live: Gevorg is finished with the integration and will be meeting Levon to test the code with him.
Levon tried setting up devnet explorer, but was not successful. Having issues building the explorer with Node.js, hopefully sproxet can find the issue faster
Peter is currently doing the wallet refactoring PR testing and fixes. Estimated to be done within this week. The two parts that are out of sync are now in sync.
Peter found that the Fedora compilation issue was not a distribution specific issue. It had something to do with the newest GCC tools.
Peter is reviewing the refactoring of existing Spark code, which is included in the wallet refactor PR.
Levon was reviewing UI code and bug fixes. There are some missing checks and crashes that were happening on Firo-qt and Firod.
Levon did some small wallet refactoring based on Aram’s comments.
Mobile lib will be updated based on all the changes.
After finishing with explorer and testing on devnet, hopefully we will be able to move on to testnet by the end of the week.
Firo
Sproxet gentrified the input selection as a template function and moved to getting Lelantus transaction creation to use that input selection function to make transaction creation less messy.
This algorithm selects transactions from the coinControl feature in a specific order. If the option to require all inputs (coinControl->fRequireAllInputs) is not set, it stops once it has enough inputs to cover the desired outputs. Otherwise, it consumes all available inputs. It starts by choosing the smallest unspent transaction output (UTXO) in the wallet. If this UTXO alone is enough to cover the output value, the selection process ends. If not, it looks for a combination of the smallest UTXO and any other UTXO that can fulfil the output value requirement. If no such combination exists, it proceeds to choose the largest UTXO, followed by the next smallest, and so on, until the output requirements are met or the maximum size (nMaxSize) or input limits (coinControl->nMaxInputs) are reached. At that point, it starts replacing the largest small inputs chosen with the largest inputs that haven’t been selected yet. If this replacement process fails, it indicates that the transactions in the wallet cannot fulfil the output requirements within the specified constraints, resulting in a failure.
Miscellaneous
Ledger-live: Gevorg did some refactoring of the code and will meet with Levon to test the hardware wallet
Levon found a fix for the Windows crashing issue and pushed it. The fix was tested successfully.
The wallet code refactor is almost done, and Levon will finalize it and do final testing on devnet (and regtest on Levon’s side) before it is ready for testnet
The UI code provided by Arcadia will be actively reviewed by Levon.
Testnet: Wallet refactoring needs to be finished before testnet can be launched as there are some changes to the walletdb.
Peter continues his work on walletdb refactoring (PR #1239), which optimizes performance by using a single instance of CWalletDB. It is about 90% done and is estimated to be done by the end of the week.
Compilation issue: The latest MacOS compilation fixes cause problems when compiling for Window. The problem lies with the Boost library, and one solution is to find an intermediate Boost version that works with both OSes.
Firo
Sproxet is continuing investigation into the undefined behavior issues and has integrated the transaction creation code.
Wallet refactor: Levon finalized wallet refactoring and pushed the changes. Conducted testing with regtest and private devnet, and everything is functioning correctly. Peter is still porting the code from Bitcoin 0.24. Changes the way walletDB is bound to the wallet itself. The work on it is about 95% done. Basic testing on devnet is required before it is ready for testnet.
Spark performance test: Levon implemented a performance test and provided the numbers to Aram for presentation.
Spark tx abandon code was reimplemented because it was missing as spark walletdb was not updated. It worked, and pushed.
Planning to move devnet past HardFork after updating to the latest Spark version.
Levon will be focusing mainly on testing and reviewing UI code.
Other Firo updates
Sproxet is working on crashes, there is undefined behavior when deserializing certain malformed MNauth messages. It doesn’t crash on x86 but crashes on ARM.
Sproxet worked on the CHDChain crash and to delete sigma tests (that causes the crash) due to it being broken
Sproxet worked on PR #1260: performance boosts in initial startup by 70% (saved 200 seconds) by moving PoW verification (for accepted blocks and loaded from disk) to a flag. However, it concerns anonymity sets being corrupted if block checks are skipped.
Compilation issues: Peter fixed Sproxet’s compilation issue PR and passed all the tests by removing the compilation of libbitcoinconsensus. It can be re-enabled with the “./configure” command.
Miscellaneous
Navek takes over Ledger integration task
Sproxet to work on coinswap for Firo Rich GUI Client
The Lelantus Spark testnet has been launched (internally), with Peter and Levon at the helm of this operation. They are currently testing the migration from Lelantus to Spark.
Additionally, Levon has been dealing with an unresolved mempool conflict issue. However, potential fixes have been identified and will be implemented if they prove effective.
Mac Issues
Peter and Sproxet have identified a bug in the dash-bls affecting Mac, which seems to originate from clang (Apple’s compiler). They’ve found that upgrading dash-bls doesn’t resolve the vulnerabilities. Their proposed solution is to change a specific flag. The next step is to test if any blockchain transaction might conflict with this dash verification procedure using bls.
Bugs
Sproxet has been working on addressing undefined behavior in a CHDChain PR.
Miscellaneous
Narek reported making progress with the Ledger Live integration
The Lelantus Spark testnet has been launched (internally) and tested. Although a few issues have emerged, all high-level functions are operational. Peter has been focusing on some specific issues:
The QT branch by Arcadia is currently crashing on startup.
Treatment of Lelantus post-transition to Spark needs refinement:
Joinsplit spend from Lelantus should be supported post-transition as consensus allows, which currently isn’t the case.
Lelantustospark conversion is leaving several Lelantus coins in the wallet and refusing to convert them to Spark.
Peter is also continuing to work on wallet refactoring to improve its overall performance and usability.
Levon has been deep-diving into the Spark testnet, investigating QT crashes, and testing various aspects of the platform. His recent activities include:
Investigating the QT crash issue and attempting to self-identify the bug. He will be reporting to Arcadia with his findings.
Refactoring the spendspark RPC and help strings for clarity and efficiency.
Checking and allowing Lelantus spends after the hardfork until the graceful block.
Identifying and fixing a bug on the miner side.
Trying to finalize the mempool conflict issue.
Working to fix the crash and Lelantus migration issues.
Working on fixing the testnet explorer.
Resolving the issue with ‘lelantustospark’ not returning txid as output.
Bugs
Peter has been investigating a bug discovered by Sproxet affecting macOS users. The issue lies with libgmp, which is utilized by relic which are also used by bls. A workaround exists, but it takes a substantial performance hit, impacting LLMQ’s formation speed and new block/transaction signing. The team continues to look for an optimal solution to this problem.
Miscellaneous
Sproxet has been mainly focused on the bls bug, investigating potential solutions, and checking whether fixing x18 CPU register (libgmp) will resolve the issue.
Narek has been working on addressing outstanding issues related to Ledger Live integration. More updates on this front will be provided in due course.
Peter has conducted tests on both the testnet and the Qt branch to further ascertain the stability and functionality of the system.
Levon has been intensively working on Lelantus Spark, his focus areas include:
Fixing failing tests.
Reviewing the QT crash fix (which currently appears satisfactory).
Refactoring the lelantusToSpark RPC to spend sigma coins.
Investigating a crash on non-mnemonic wallets.
Reviewing UI code.
Synchronizing the mobile library with the current code and finalizing documentation.
Finishing refactoring work.
Adding a list of transactions to the Lelantustospark output.
Additionally, Sproxet identified an issue with Bitcoin’s handling of nested transactions in the mempool which has anonymity implications. This is currently affecting the Lelantustospark conversion process.
Bugs
Peter has made progress with the Apple M1 issue, fixing a problem causing crashes. He noted the sync speed on M1 isn’t as efficient as it used to be and is looking into this. He is also continuing his work on refactoring the PR from Bitcoin’s core 24 (Firo is using core 14) which is currently 97% complete.
Firo
Sproxet has been focusing on the Firo Client, particularly working on CoinSwap. The refactoring work done should make adding swap providers easier in the future. However, theyencountered an issue with zeromq and node_gyp, making it challenging to cross-compile for Windows on Linux.
Miscellaneous
Narek has completed all code changes for Ledger Live integration. The next step is to get Firo added into explorers.api.
QT Code Review: Closely examined QT code, which Arcadia provided, and overall, the code meets the necessary standards. Some comments and suggestions were given to further optimize it.
Aram’s Code Feedback: Took a look at Aram’s code and commented about cleanup, but beyond that the code appears to be fine.
Mobile Integration: Updated the mobile library to incorporate the latest Spark features.
Crash Issue Resolved: A crash issue on non-mnemonic wallets, highlighted by Anwar, has been addressed. The fix will soon be available through a PR.
“Spark Coin Used” Issue: During the stress tests performed by Narek, an issue related to used Spark coins was uncovered. Levon caught the issue and has already devised a solution.
Peter
Mempool Conflict Solution: A significant challenge emerged with two mempool transactions sharing the same coin. A potential solution is in the works: maintaining a list of serials for coins to ensure no duplication.
WalletDB Refactor: Continuing the refactor of the wallet database. This involved merging 4 Bitcoin PRs and additional adjustments. With these changes, support for multiple wallets will need RPC/Qt adjustments but the core support is already integrated.
Narek
Stress Testing: Initiated Spark transactions on the testnet specifically for stress testing. An issue was found and will be fixing it asap.
Bug Resolution and Investigation
A Mobile (Lelantus) Issue: An issue with the mobile version of Lelantus, flagged by Diego, is being looked into by Levon. A fix will be finalized shortly.
Spark Mempool Conflict: Investigating a mempool conflict issue. Some initial changes have been pushed to address it and will continue to monitor if it occurs again.
Spark Qt UI: Reviewing the Qt UI code of Spark from Arcadia.
Spark Lockwallet issue: Working on fixing Spark’s lockwallet issue. A commit will be pushed soon. It is not a consensus change.
firod Mobile API Update: Updating firod’s mobile_api branch to align it with the latest developments.
Peter
Mempool/Blockchain Conflict: Currently researching mempool and blockchain conflicts in Spark
WalletDB Code Refactoring: Code refactoring continues to progress and will have additional pushes today
Bug
Mobile wallet Issue: Investigating an issue regarding the missing of funds on the mobile platform, which appears to be associated with counter reuse. A wallet was created via Campfire wallet app and later recovered on Stack wallet app
Miscellaneous Progress
Firo-Client: Sproxet continues work on Firo-Client.
Ledger Live Investigation: Narek continues investigation into Ledger Live integration. He will be communicating with the Ledger Live team to resolve any potential integration challenges.
lockwallet Issue: Addressing an issue related to the locked wallet case in Spark with his own fix. It is still ongoing
Sparkspend Fee Subtraction Issue: Resolved the issues involving Sparkspend fee subtraction. Has pushed a commit for it.
Mempool issue: Investigating issues where Spark transactions were not being mined and not appearing in the mempool.
Peter
Stress-Testing: Created a script for stress-testing Spark minting and spending processes, allowing for comprehensive assessment and optimization.
Mempool Bug: Investigating possible scenarios that could lead to a specific bug (some lTag is both in mempool and sparkstate at the same time), with successful reproduction of the issue multiple times.
Block Assembly: Working on problems related to block assembly (replacebyfee not compatible with InstantSend)
Wallet Refactor: Continuing works on the wallet refactor PR.
Code Review and Bug: code reviews are still being done and continuing identifying and addressing bugs.
Sproxet
Spark lockwallet Issue: Investigating lockwallet problem within Spark, particularly focusing on the possible issue of a locked wallet not verifying blocks. Determining the best approach to address this challenge is underway.
Client-API: Working to prepare Spark for the upcoming client-api integration.
Narek
Lelantus Manual-Anonymize Page: has been removed for Spark since Lelantus minting is no longer possible.
French QT Typo Fixed
Fixed RPC help text in mintspark and spendspark
UI Column Resizing: Working to resolve an issue with Qt resizing columns that continue resizing. Levon took a look and found that there are some commits in bitcoin by just removing the entire class which helps solve the issue and improve wallet performance. Alternatively Narek has a specific fix for the window resizing issue.
Code Review and Fixes: Reviewed and commented on Acadia fixes for issues #1288 and #1282. Also did some fixes for issues #1297 and #1268.
Code Cleanup: Enhanced the readability of the codebase by cleaning up some conditions - this has made private transactions look better in the code.
Upcoming Tasks: Plans to review Peter’s fixes, sync mobile-lib and API, finalize the documentation, and kickstart discussions on mobile implementation. Will also review QT code.
Peter
Block Connection Issue Fix: Resolved a mempool/blockchain issue that arose during the connection of blocks. Has identified three potential ways this bug could occur and is currently investigating to determine the exact cause.
Code Review and Refactor: Reviewing code and is in the process of doing a refactor, which should be completed in the next couple of days.
Sproxet
Client Integration: Working to integrate Arcadia changes into the firo-client/client-api.
Spark spend issue: There were some challenges in getting spark inputs properly recognized, but in the final stages of resolving this issue.
Test Infrastructure: Refining test infrastructure because manual testing wasn’t yielding the best results. Expecting to have a working binary soon.
Narek
Bug Fixes: Fixed issues #1281 and #1276 and merged them into the Spark branch. Also worked on issue #1303, which is currently under testing.
Week 3 August 2023
Spark Development Updates
Levon
Research and Testing: Conducted research on Peter’s fix, added a new check, and fixed unittest.
Spark IS: Issue has been successfully fixed.
Issue Investigation: Looked into issues #1287, #1285. After thorough investigation, it was found that #1285 and #1287 were not issues, but the amount was incorrect in each instance of the list.
Sync Mobile Library: Synchronized the mobile library and will be pushing updates soon.
Peter
Refactoring and Maintenance: Carried out refactoring PR and maintenance work on testnet/devnet nodes.
Bug Fixes: Continuing fixing bugs that has been brought up, and other remaining Lelantus fixes from Levon
Refactoring: Continue on refactoring task
Sproxet
Firo Client: Successfully merged Arcadia’s changes. Worked on issues that were originally planned for after the testnet release but needed immediate attention. Removed a few UI elements to simplify the user interface.
Narek
Server Investigation for Ledger live: Investigating the possibility of using electrumx servers in ledger live for Firo as a potential replacement for their servers.
Week 4 August 2023
Spark Development Updates
Peter
Bug Fixes: Resolved a QT-related startup bug that had been present in the code for some time. Also addressed issues with the Spark branch not functioning properly on the mainnet and a problem with SparkState::RemoveBlock that could lead to state corruption.
Spark Refactoring: Continue work on code refactoring
Sproxet
Platform Compatibility: Successfully got Spark working on Mac and is now working on setting up a Windows environment/build.
CoinSwap Implementation: Working on getting CoinSwap functioning properly.
Depends NO_WALLET=1 Setting: Successfully addressed the “depends NO_WALLET=1” setting issue.
Ledger Live Integration: Currently reviewing Ledger Live code that integrates the ElectrumX server API and finding a method for Ledger Live to connect to ElectrumX.