Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

schema: atomic multi-property permanode updates #110

Open
bradfitz opened this issue Feb 12, 2013 · 7 comments
Open

schema: atomic multi-property permanode updates #110

bradfitz opened this issue Feb 12, 2013 · 7 comments
Assignees
Labels

Comments

@bradfitz
Copy link
Contributor

Right now we can only update one property of a permanode per blob (set-attribute,
add-attribute, del-attribute, or del-all-attributes).

To do read/write FUSE support we'll need to be able to set multiple attributes
atomically (e.g. a rename operation, removing a camliPath from one permanode, adding it
the same or different permanode).

So we'll need a way for the schema package to let you build multi-claim blobs, and we'll
need to document the format (once we decide it), and we'll need to parse it, index it,
and test it.
@bradfitz
Copy link
Contributor Author

bradfitz commented Sep 8, 2013

Comment 2:

This is only halfway complete.  We have some support in pkg/schema for creating them,
but they're not currently indexed.
pkg/search/handler.go's func (dr *DescribeRequest) populatePermanodeFields needs to be
updated, and probably some stuff that it uses.
Then it needs tests in handler_test.go.
And then pkg/fs/mut.go should use it.

@mpl
Copy link
Contributor

mpl commented Sep 11, 2013

Comment 3:

I reckoned before doing the search part we'd need to figure out how we want to index the
multi claims, so I started with that. Also started on a camput claim command so I could
easily generate multi claims. that same command can be the one that we'll use for claims
to delete permanodes/claims.
https://camlistore-review.googlesource.com/796

Status changed to Started.

@bradfitz
Copy link
Contributor Author

Comment 4:

New plan.
The "claimType": "multi" got too invasive.
Instead, we can make a set of N claims to be processed atomically still be N different
blobs, but link them together in new fields, so that the indexer itself will only
process them all together once they're all available.
The first claim can be:
     "claimBatchSize": 4,
And then claims 2, 3, and 4 can be:
     "claimBatchLeader": "sha1-xxxxxxxxxx"  (blobref of the first claim)
Then the indexer waits until it sees them all until it applies them, atomically.

@bradfitz
Copy link
Contributor Author

Comment 5:

Owner changed to @bradfitz.

@edrex
Copy link
Contributor

edrex commented May 10, 2016

I guess with the new plan, you'll be able to create a delete claim targeting a single claim in a previous batch, addressing the second point in @mpl's #562 (comment)?

@edrex
Copy link
Contributor

edrex commented May 10, 2016

I found the documentation of multi claims in https://camlistore.org/doc/schema/permanode, and assumed they were already implemented. Should we delete that section from the docs?

@edrex edrex added the Schema label May 10, 2016
@mpl
Copy link
Contributor

mpl commented May 10, 2016

I think it would look even worse to completely remove it as it would give the impression that we don't even have a plan for that.
If that itches you too much, I guess you could atomically (hah!) replace that section with Brad's above new plan in one commit?

Or even better, implement the new plan (if it still applies?), and then fix the doc.

camlistorebot pushed a commit that referenced this issue Jan 24, 2018
Related: issue #110

Change-Id: I2726f4e2aef714b62397f3a30633eb30e65ef079
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants