Integrate new p2panda high-level API#218
Merged
Merged
Conversation
adzialocha
force-pushed
the
adz/p2panda-high-level-api
branch
12 times, most recently
from
84af564 to
e3aa553
Compare
adzialocha
force-pushed
the
adz/fix-tests
branch
7 times, most recently
from
87b3021 to
8d83882
Compare
adzialocha
force-pushed
the
adz/p2panda-high-level-api
branch
from
e3aa553 to
a79f048
Compare
adzialocha
force-pushed
the
adz/fix-tests
branch
3 times, most recently
from
49c10ba to
9adc524
Compare
adzialocha
force-pushed
the
adz/p2panda-high-level-api
branch
from
a79f048 to
6e34c9e
Compare
adzialocha
force-pushed
the
adz/fix-tests
branch
from
9adc524 to
074a7f1
Compare
adzialocha
force-pushed
the
adz/p2panda-high-level-api
branch
from
6e34c9e to
a45d239
Compare
adzialocha
changed the title
Collaborator
|
Only had a quick look. Can you split code changes to sensible commits, e.g. creating the new database.rs file. Following the commit style https://handbook.gnome.org/development/commit-messages.html |
adzialocha
force-pushed
the
adz/p2panda-high-level-api
branch
from
a45d239 to
93cf54f
Compare
Member
Author
Yeah, the commits are a bit unfortunate. Not sure I'm able to do this as this refactor kinda requires me to touch upon everything at the same time. I'll try to split them up a bit but no guarantee they'll compile in each patch. |
This will assure that users have a chance to recover documents from past Reflection versions and move them into this new version with a migration script, etc.
This ensures that we're not able to sync with Reflection users of an older version.
The new p2panda Node API brings timestamp out-of-the-box from ephemeral streams, we don't need to provide them ourselves anymore.
There's more stream processing related errors we want to expose to -docs from p2panda aka -node. To avoid confusion around two SubscriptionError types I've renamed one of them which was anyhow more "publishing" related. I'm considering renaming more types in a subsequent patch to make the PubSub relationship clearer.
Topic is used already all over the place in p2panda and indicates the
identifier of the topic itself (32 byte string). A stream on top of a
topic is a "topic stream", we can subscribe to it ("topic subscription")
and publish into it, following a PubSub pattern.
Reflection "enhances" these types with "tracking", that is, we know
which topics have been used and which authors are connected to it and
when they are online.
This hopefully clarifies the types a bit better and stays consistent
with naming in p2panda to avoid confusion:
Topic -> TrackedTopic
Author -> TrackedAuthor
Subscription -> TopicStream
SubscribableTopic -> TopicSubscription
adzialocha
force-pushed
the
adz/p2panda-high-level-api
branch
4 times, most recently
from
e9edf01 to
8c2e190
Compare
If there are no active connections it is possible that an in-memory SQLite database will be dropped, causing any further queries to fail. This issue can be avoided if idle_timeout and max_lifetime values are both set to None. See issue: <p2panda/p2panda#1255>
adzialocha
force-pushed
the
adz/p2panda-high-level-api
branch
3 times, most recently
from
67a0872 to
ae56558
Compare
adzialocha
force-pushed
the
adz/p2panda-high-level-api
branch
from
ae56558 to
db1ae64
Compare
mycognosist
approved these changes
Jul 7, 2026
sandreae
approved these changes
Jul 7, 2026
sandreae
left a comment
sandreae
left a comment
Member
There was a problem hiding this comment.
Been testing a build of this branch for a few weeks and it works great. Reviewed the code now and all looks good, as well as the node integration there is some nice new logging and encoding optimisations.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This patch integrates the new
p2pandacrate into the "backend" of Reflection.p2pandais the new all-in-one high-level Node API which should make usage of p2panda features easier, removing a bunch of code inreflection-nodewhich is now handled in p2panda directly, such as:Reflection Node
Some features implemented in
reflection-nodestay as they're adding functionality on top ofp2panda. Some of them might be moved there as well in the future, such as:Unfortunately we need to remove one feature for now which is to change "connection modes", causing the network to be disabled / enabled.
reflection-nodehandled that well in the past by only disabling the networking layer while keeping the event processing layer intact. We will have to do something similar inp2pandato offer this feature as well since the new API surface will not allow such low-level access anymore. See related issue: p2panda/p2panda#1093Breaking Changes
These changes introduce breaking changes to the core operation format (
previousfield was removed) and the header extensions changed which means that all previously created, persisted operations will not be compatible anymore.THIS RESULTS IN LOOSING ALL PREVIOUSLY CREATED DOCUMENTS.
Since
p2pandais not stable yet this might happen again in the near future, though this was the most important and largest update and we're much closer to a stable API and protocol now.Future SQLite files will be written to a new path, the network id has also changed now. This prevents "old" nodes talking to "new" ones, similarily we allow users to retreive old documents from the database.
Final remarks
Some minor changes in
reflection-docwhere also necessary, I specifically decided to makep2panda-corea concrete dependency there since too much functionality is used to justify re-exports fromreflection-node.