Weekly Dev Update #20

Hey Y’all,

Lots of hiring this week! We have brought on two new members who will be joining us in the coming weeks – I will introduce them once they complete their first pull requests.

Daemon


Service Nodes


LLARP / Lokinet
We are looking for experienced sysadmins to run nodes that connect to our toynet. Please join Discord room called #Lokinet and we can add you to the node operator role. You can host a hidden service or browse existing hidden services.

  • Progress continues on libllarp
    • More memleak fixes
    • Fix DHT bug
    • Bundle libsodium
    • Network event changes
    • Drop ninja support, use make -jX now where X is the number of cores you have available
    • MacOS: Tun flush
    • Limit introset propagation
    • Always use random established path
    • Only publish introset to nearest router
  • libAbyss
    • More implementation
    • Start TCP event work
  • DNS library
    • Refactoring of data structures
  • GitHub Pulse Stats for the last week: Excluding merges, 3 authors have pushed 65 commits to master and 74 commits to all branches. On master, 254 files have changed and there have been 24,917 additions and1,841 deletions.
  • NOTE: Loki Builder compile helper is now deprecated and integrated with main repo.

Loki Messenger


Thanks,  
Kee

Weekly Dev Update #19

Hey Y’all,

Things are moving along at a fast pace as we hire more developers.
Service Nodes

TLDR: Be very careful if you are using autostake in an open or closed pool.


LLARP / LokinetIf you’re lucky and join our Discord, you might catch Jeff or Ryan, the developers of LLARP, live streaming as they code.
Again, special thanks to our community contributors, Despair (AKA Pickle Rick) and Cathugger.

  • Released LokiNET 0.3.0: “i can’t believe it’s not i2p”
    • https://github.com/loki-project/loki-network/releases/tag/v0.3.0
    • More robust packet support, better path building, improved DHT handling of RCs, MacOS run support, new config locations, more sane timeouts, .loki hidden service addresses, Reverse DNS support, and better memory management
    • We urge all Lokinet toy node operators to upgrade ASAP, as they’ll be unable to communicate as the network upgrades.
  • Progress continues on libllarp
    • Make tunnel interface fragmentation-aware
    • Cleanup UDP handling
    • One’s complement math improvements
    • Tunnel packet filtering at destination
    • Drop too large packets, instead of truncating
    • Additional streaming improvements: support DCCP (improving congestion control) and UDP-Lite
    • TCP checksum improvements
    • Better IPv4 bogon checking on RCs to prevent bogons to end up on public routers descriptions
    • Handle expire intro before building next message
    • Log endpoint when requesting introsets
    • Make sure event loop doesn’t overflow
    • On HS lookup failure, try random path
    • IP range tools
    • Handle case of no backup path better by
    • More unit tests
    • Win32: separate read/write event ports to avoid overwriting queue events
    • Win32: clean up and various define clean up
    • Fix memory leak in routing messages
    • Change UTP queue size from 128 to 1024
  • libAbyss 
    • Set up basic README
  • DNS library
    • Improve handling of .loki lookups
    • Reverse DNS lookups for dynamically handed out IP addresses
    • Handle various error conditions better
    • Make reverse DNS matching smarter
  • GitHub Pulse Stats for the last week: Excluding merges, 4 authors have pushed 36 commits to master and 36 commits to all branches. On master, 46 files have changed and there have been 758 additions and 325 deletions.
  • Compile helper (include assets, initial seed routers and dependencies)
    https://github.com/loki-project/lokinet-builder

Loki Core


Loki Messenger

Thanks,  
Kee

Service Node Autostaking Announcement

We are very quickly approaching the date where the first Service Node will be deregistered because of a natural expiry. On October the 20th, 30 days since the first Service Node being registered, the first group of Service Nodes will undergo deregistration.

There is a couple of things we wanted to go through for Service Node operators so they understand what the process is of re-registering, and issue a few warnings about Autostaking. If you are presently intending on using autostaking, you should read this announcement very carefully.

Updating software

The downtime between two registration periods is one of the best times to update your Service Node to the latest version of the Service Node software, as it can be done without the risk of accidentally deregistering your node.
The latest version of the Service Node software can be found here https://github.com/loki-project/loki/releases/latest and we have also complied a written guide and a video guide on how to update your Service Node software.

Solo Operators

If you are a solo operator and used autostake, then you must ensure that whatever computer you issued the stake command from is online during your re-registration period. You must also ensure that there is still a process running called loki-wallet-cli as this wallet will scan the blockchain and recognise when your node has been de-registered, and as soon as your funds become unlocked it will restake your Loki.
If Autostaking fails for any reason then solo operators can run the prepare_registration command on their Service Node again and run the outputted command in their wallet. This will stake your Loki again.

Closed Pools

A closed pool is a pool in which each contributor is specified by their staking address and the amount they are contributing. If you are staking Loki without running a node, ask your operator whether you are in an closed or open pool, they should be able to give appropriate guidance.
If you are Autostaking in a closed pool as a participant we highly recommend that you contact your pool operator and switch away from autostaking back to manual staking. Disabling autostake is as simple as ending the wallet process (called loki-wallet-cli) on your machine, this can be done by using:

On Linux: pkill loki-wallet-cli or
ps aux | grep loki-wallet-cli and using the process id quoted and running kill <process ID>

On Windows: you can use task manager to find the loki-wallet-cli process and terminate it

On Mac: you can use the activity monitor to find and terminate loki-wallet-cli

The main reason we recommend disabling autostake is because in a closed pool if any participant decides they will not stake during the next registration period, you may lock up your Loki without being able to fill the vacancy left by the deserting party.

If you are staking without using autostake, contact your pool operator and ask them whether there has been any changes to their node, they will give you their Service Node pubkey and they can also tell you how much Loki is required during the next registration period (Remember the amount of Loki required to stake is decreasing overtime).

You can use this information and re-run the stake command, as stated above we recommend you do not use autostake unless you have high confidence that all your collaborators will also restake their contributions.

stake <Service Node Pubkey> <address> <contribution amount>

For operators of closed pools we recommend that you contact all participants in your pool and recommend they disable autostake before re-registration begins, unless you can ensure a scenario where you have high confidence in all staking participants. Operators who are also using autostake to re-register their Service Node should consider their pool participants and whether they will be staking in the next period, if not they may need to adjust their pool using the prepare_registration command.

Open Pools

Open pools are pools for which there is no specified contribution addresses. Anyone can contribute to an open pool without notifying the pool operator.
For open pools in version 1.0.4 we have disabled the autostake feature. This is due to the high risk for operators or contributors to inadvertently lock funds due to a lack of communication. Additionally using autostake in open pools in the current implementation fails to reduce the amount of the stake as per the decreasing staking requirement.

Therefore we recommend that anyone who operates or is a participant in an an open pool terminate any wallet which is autostaking. Open pools are still available in version 1.0.4, however they only allow the client to use the stake command without the auto flag.  

As with all other Loki, Loki that was staked in open pools will be unlocked and the user is free to use this Loki in the same open pool (if it exists) by using the stake command as shown below:

stake <Service Node Pubkey> <address> <contribution amount>

Operators of open pools can also continue running open pools without autostake by running the prepare_registration command and submitting output through their wallet, users can be notified of the specifics of the new open pool when it is available.

User experience
We are looking at making changes to the user experience of staking in the future which will make the process less clunky and also reduce the risk of the user locking their funds without acting intentionally dishonestly.

Weekly Dev Update #18

Hey Y’all,

Lots of code going in this week, with some big design decisions especially around Swarms and Loki Messenger.

There was also a new release of the the Loki CLI wallet and daemon, which all Service Node operators should update to. The release does not include consensus changes but does change how autostaking works, making users much less likely to lock funds unintentionally by autostaking.

The release is here: https://github.com/loki-project/loki/releases/latest and there is also a video guide on how to update here: https://youtu.be/u7kyAoBsQIY.


Service Nodes


LLARP / Lokinet

If you’re lucky and join our Discord, you might also catch Jeff or Ryan, the developers of LLARP, live streaming as they code: https://www.twitch.tv/uguu25519https://www.twitch.tv/neuroscr

Special thanks to our new contributor, CatHugger, who’s pitched in and helped organise some of our network structures.

  • Progress continues on libllarp
    • Windows work on tunnel code
    • Only publish HS if we have a minimum of 3 paths
    • Don’t publish all path builds
    • Put router profiling back in place, adjust algorithm
    • Play with threading crypto functions again
    • Update logging time format to be more human readable
    • Make logger memory churn lower
    • Better handover logic
    • Initial admin API for managing a live router
    • Several memory corruption fixes
    • Clear re-used restructures in tun interface to stop info leaking
    • A ton of tun interface protocol clean up
    • Fix Introset expiration bug
    • Introset code refactoring
    • Implement “No Logging” level
    • Lots of minor fixes
    • Improve documentation
  • DNS library
    • Additional cleanup
    • Started DNS Unit tests
    • Initialisation everything in header
    • Disable SOA packet crashes for now
    • Question parsing correctionness
    • Improve documentation
  • GitHub Pulse Stats for the last week: Excluding merges, 4 authors have pushed 80 commits to master and 80 commits to all branches. On master, 79 files have changed and there have been 2,116 additions and 594 deletions.Most changes can be found at: https://github.com/loki-project/loki-network/
  • Current version: v0.2.2 (next one coming soon!) https://github.com/loki-project/loki-network/releases/tag/v0.2.2
  • Compile helper (include assets, initial seed routers and dependencies) https://github.com/loki-project/lokinet-builder

Loki Messenger


Loki Core

Thanks,

Kee

Weekly Dev Update #17

Hey Y’all,

Another weekly Dev Update for everyone.

Service Nodes


LLARP / Lokinet

If you’re lucky and join our Discord, you might catch Jeff or Ryan, the developers of LLARP, live streaming as they code: https://www.twitch.tv/uguu25519 https://www.twitch.tv/neuroscr

  • Progress continues on libllarp
    • Optimise the router’s inbound links data structure
    • More keepalive tuning
    • Support tunnel “packet info” for better multiplatform support
    • MacOS tunnel support is now fixed
    • No longer stores Introsets when forwarding
    • Improve autoconfiguration
      • A default “hidden service” is made for client access, and configuration option to enable, disable or auto-config
      • -g for router auto-config, -c for client auto-config
      • Detect if you’re already using any private network ranges
      • Make it play more nicely with existing hidden service definitions
      • Don’t overwrite configs unless -f is present
    • Give lookup more time: adjust timeout from 5s to 10s
    • Only consider introductions that don’t expire in the next 2 minutes
    • Don’t establish outbound context for previously made inbound sessions
    • “cache convo tag and drop traffic”
    • Deallocate previous Introset before replacing
    • Brought Windows build up to date
    • Make sure Windows build doesn’t crash if only one network interface
    • Made AVX2 codepath 32-bit clean
    • Fix bencoding for ILP32/LLP64
    • MacOS/BSD (kqueue) updates
    • Try to improve path handovers
    • Improve transit message padding to the nearest MESSAGE_PAD_SIZE
    • Adjusted DST awareness on timestamps
    • Fix DHT double free
    • Remove dead code
    • Path Info hashing
    • Swap out intro with next intro as it is safe
    • Update protocol docs
    • Give paths more time to build: timeouts changed from 15s to 30s
    • Minimum time between each path build changed from 5s to 10s
    • Minimum path latency adjusted: from 10s to 5s
    • Path timeout changed from 20s to 10s
    • Stricter profiling of paths/routers
    • 2 path build fails before kicked (changed form 8)
    • Try to build path 5 times before failing
    • Don’t prematurely close sessions
  • DNS library
    • Rely on llarp::Addr more to allow easier IPv6 support
    • Make DNS server port configurable at compile time (DNS_PORT)
    • Made sure DNS relay works with default “hidden service”
    • Update DNS daemon to just run a router (similar to Lokinet), scheduled to be downgraded to just a integration test
    • Mapping an address now does it on all hidden services/tunnel as there is only one DNS server (on most non-“netns-linux” platforms)
  • GitHub Pulse Stats for the last week: Excluding merges, 3 authors have pushed 93 commits to master and 93 commits to all branches. On master, 86 files have changed and there have been 3,898 additions and 4,874 deletions.
  • Most changes can be found at: https://github.com/loki-project/loki-network/
  • Current version: v0.2.2 (next one coming soon!) https://github.com/loki-project/loki-network/releases/tag/v0.2.2
  • Compile helper (include assets, initial seed routers and dependencies) https://github.com/loki-project/lokinet-builder

Loki Core

  • In process of merging a huge chunk of upstream changes, this includes Bulletproofs.
  • IPv6 first implementation is now finished; this will add client support for IPv6 connections.  Just working on the final details now. https://github.com/loki-project/loki/pull/280

Thanks,

Kee

Weekly Dev Update #16

Hey Y’all,

The dev team had an interesting week, which was primarily spent responding to bugs in the Loki core implementation. Lokinet developers have been working super hard and are getting close to something that is usable by the end user.

***

Releases

This week we published a minor release which can be found here: https://github.com/loki-project/loki/releases/tag/1.0.2 This incorporates fixes for newly operating Service Nodes and prevents a burning bug, mentioned below.

This update does not contain any consensus rule changes and therefore is a non-mandatory upgrade. However, if you are a Loki merchant, you should update immediately. Service Nodes who consistently take their nodes offline should also update. We will publish a guide soon on how to safely update a Service Node, however you should not be concerned unless you are planning to take your Service Node offline for a period of time.

***

Service Nodes

***

LLARP / Lokinet

We are looking for experienced sysadmins for running nodes that connect to our toynet. Please join the Discord room called Lokinet, and we can add you to the node operator role. You can host a hidden service or browse existing hidden services.

  • Progress continues on libllarp
    • Prep for next release
    • FreeBSD Compile fixes
    • Various low level network updates
      • Give protocol a fresh buffer instead of resetting
      • Only tick ev if running
      • Properly close sockets so we don’t run out over time
      • Double max protocol message size to 4k to make sure intros fit inside one packet
      • Increase the UTP socket buckets
      • Improve llarp::Addr utility
      • Start IPv6 support for tun
      • Promote tun to enabled by default, disable tap
    • DHT changes
      • Make sure DHT old RCs updates don’t stomp new ones
      • Try fixing DHT crash
    • Configuration
      • Update testnet config generation
      • Update auto configuration
      • Bootstrap node
    • Sessions
      • Server shutdown now cleans up sessions
      • Prevent dead sessions from updating and building more paths
      • Up session inactivity cost and change timeout to 60s from 10s
    • Introsets
      • Allow node introset to have 10s clock difference
      • Don’t propagate or store introsets on crypto error
      • Made sure timestamp is added to introset (Protocol backwards compatible breaking change)
    • Intros
      • If intro is bad then potentially trigger a manual path rebuild
      • Prevent spamming of intro frames
      • Reset intros before reuse
      • Fix intro matching by removing expiration and latency comparisons
    • Hidden service
      • Make sure hidden service lookups are valid and handle response
      • Optimise hidden service publishing from a random establish path to using a “closer” algorithm
      • Require 6 paths for a hidden service instead of 4
      • Expire outgoing hidden service paths
    • Path building improvements
      • Track path data usage
      • Don’t kill paths when under heavy use
      • Make sure path is recently active (20s) before checking for expiration
      • Only timeout path if no messages received are ever received on it
      • Don’t accept introset if it expires in the next 60s
      • Separate send (40s) and connect (60s) hidden service timeouts
      • Enable path callbacks even if marked bad
      • Reduce outbound path failure by attempting to use parent endpoint’s pathset
      • Path building backoff to prevent DoS in such a way that make sure intros don’t die
      • Preemptively trigger rebuild before expiration to ensure continuity
  • DNS library
    • Support other tun gateways other than 10.x.x.x
    • Refactor .loki lookup into its own file
    • Fix SIGINT handling on DNS binary
    • Moved logic worker loop into DNSc (client) for timers
    • Support multiple DNS IP trackers for netns
    • Allow upstream DNS configuration per tun interface (nameresolver key)
    • Fix PTR query detection to allow reverse DNS to work
    • Fix NXDOMAIN (not found) packet to include question name
    • SOA packet fixes
    • Various header cleanup
    • Make sure all vectors use unique_ptr for tighter memory management
    • Prevent duplicate mappings
  • GitHub Pulse Stats for the last week: Excluding merges, 2 authors have pushed 90 commits to master and 90 commits to all branches. On master, 76 files have changed and there have been 3,606 additions and 1,553 deletions.
  • Most changes can be found at: https://github.com/loki-project/loki-network/
  • Current version: v0.2.2 (next one coming soon!) https://github.com/loki-project/loki-network/releases/tag/v0.2.2
  • Compile helper (include assets, initial seed routers and dependencies) https://github.com/loki-project/lokinet-builder

***

Loki core

  • Fixed a bug affecting Cryptonote coins where no warning was giving for non spendable outputs, leading to effective burning if coins were credited.

https://github.com/loki-project/loki/pull/268

Thanks,

Kee