Background

• @ramfox has been heading up a project to implement a streaming interface atop a libp2p connection for inter-process communication (IPC)

• On Friday May 16th, we got it to work!

• The result of it working: we also got a sense of the distance between what we need & where we are, and there is a lot of ground that needs to be covered to use libp2p streaming as a robust IPC protocol

• dig pulled together an experiment using gRPC over the weekend to understand how fast switching to gRPC could close the gap

• from discord, @dig :

  I just finished an experiment (it works) that uses gRPC (tonic impl) instead of libp2p for the rpc layer, and I think it is a considerably better choice for what we need to do in the short to medium term. I have been thinking about all the things that we have left to do on implementing a solid rpc layer, and that is quite considerable. so my proposal is lets switch to use grpc instead and replace it down the line if we ever find ourselves in the actual need for sth else code to play around and look at https://github.com/n0-computer/iroh/tree/grpc

• The decision in front of us:

  1. keep pouring time into streaming libp2p connections
  2. switch to gRPC

• given that we’re very pressed for time, we’ve decided to go to option 2: switch to gRPC

Consequences

• we’ve invested weeks of engineering time into the libp2p streaming interface, and that code isn’t going to make it to production
  • however, we do have the net result of that project realized in the form of gRPC
• @ramfox needs to overhaul her work schedule based on this change
• the net amount of code we need to maintain goes down with a switch to gRPC
• we gain possible integrations with gRPC measurement & debugging tools

Lessons Learned

• We should be more explicit about code we intend to run as experiments, declaring a hypothesis & outcomes if our hypothesis is confirmed or not viable
  • our experiments should have planned “cut for time” outcome like the one we encountered here
  • cut for time results should include an articulation of the “gap” between where we’ve gotten, and what would need to happen to make the experiment viable
  • that articulation should be connected to github history for later source code archeology
• We should ask why we should not do experiments before we take them on, and write those reasons why not down to help shape our understanding of a project