couchdb - Hyperledger Fabric private data collection t... https://stackoverflow.com/questions/55647360/hyperl...

Hyperledger Fabric private data collection to distribute

We are currently researching on Hyperledger Fabric and from the document we know that
a private data collection can be set up among some subset of organizations. There would
1 be a private state DB (aka. side DB) on each of these organizations and per my
understanding, the side DB is just like a normal state DB which normally adopts CouchDB.

One of our main requirements is that we have to distribute files (e.g. PDFs) among some
subset of the peers. Each file has to be disseminated and stored at the related peers, so a
centralized storage like AWS S3 or other cloud storage / server storage is not acceptable.
As the file maybe large, the physical copies must be stored and disseminate off-chain. The
transaction block may only store the hash of these documents.

My idea is that we may make use of the private data collection and the side DB. The
physical files can be stored in the side DB (maybe in the form of base64string?) and can be
distributed via Gossip Protocol (a P2P protocol) which is a feature in Hyperledger Fabric.
The hash of the document along with other transaction details can be stored in a block as
usual. As they are all native features by Hyperledger Fabric, I expect the transfer of the
files via Gossip Protocol and the creation of the corresponding block will be in-sync.

My question is:

1. Is this way feasible to achieve the requirement? (Distribution of the files to different
peers while creating a new block) I kinda feel like it is hacky.

2. Is this a good way / practice to achieve what we want? I have been doing research but
I cannot find any implementation similar to this.

Most of the tutorial I found online pre-assumes that the files can be stored in a single
centralized storage like cloud or some sort of servers, while our requirement demands a
distribution of the files as well. Is my idea described above acceptable and feasible? We
are very new to Blockchain and any advice is appreciated!

couchdb hyperledger-fabric hyperledger blockchain

Share Improve this question edited Apr 12, 2019 at 8:48 asked Apr 12, 2019 at 8:22
Follow shole
4,048 2 28 67
Join Stack Overflow to find the best answer to your technical question, help others answer
theirs.

2Sign up with email

1 of 3 4/28/23, 15:47
couchdb - Hyperledger Fabric private data collection t... https://stackoverflow.com/questions/55647360/hyperl...

Is this way feasible to achieve the requirement? (Distribution of the files to different
peers while creating a new block) I kinda feel like it is hacky.
3
So the workflow of private data distribution is that the orderer bundles the private data
transaction containing only a hash to verify the data to a new block. So you dont have to do
a workaround for this since private data provides this per default. The data itself gets
distributed between authorized peers via gossip data dissemination protocol.

Is this a good way / practice to achieve what we want? I have been doing research
but I cannot find any implementation similar to this.

Yes and no. Sry to say so. But this depends on your file sizes and amount. Fabric is
capable of providing rly high throughput. I would test things out and see if it meets my
requirements.

The other approach would be to do a work around and use IPFS (a p2p file system). You
can read more about that approach here here

And here is an article discussing storing 'larger files' on chain. Maybe this gives some
constructive insights aswell. But keep in mind this is an older article.

Share Improve this answer Follow answered Apr 12, 2019 at 8:38
kajuken
307 2 10

Thanks for your great input...I have read a lot on IPFS as well, yet our management would like to
adopt Hyperledger Fabric fully if possible, so if using native features (e.g. Private Data) can get the
job done, it is the best way. Just I am not sure if my idea is okay... – shole Apr 12, 2019 at 9:05

1 Haha, ok. Well then either tell your management that blockchain are not intended to be used as fully
capable file storage systems (that's why ipfs etc exist for p2p approaches) and they let you do so.
OR just implement a PoC where you use this private data approach and run a stress test :)
– kajuken Apr 12, 2019 at 9:09

The PoC way is a great suggestion and we actually are going to do that, thanks! At least the way
using private data would work, just the size of the file is limited and we have to test it out, I think it
makes sense. – shole Apr 12, 2019 at 9:10

Join Stack Overflow to find the best answer to your technical question, help others answer
theirs.

Sign up with email Sign up with Google Sign up with GitHub Sign up with Facebook

2 of 3 4/28/23, 15:47
couchdb - Hyperledger Fabric private data collection t... https://stackoverflow.com/questions/55647360/hyperl...

Check out IBM Blockchain Document Store, it is the implementation of storing any
document (pdf or otherwise) both on and off chain. It has been done.
1
And while the implementation isn't publicly available, there is vast documentation on it's
usage, can probably disseminate some information from it

Share Improve this answer Follow answered Apr 13, 2019 at 4:08
lindluni
505 4 6

Join Stack Overflow to find the best answer to your technical question, help others answer
theirs.

Sign up with email Sign up with Google Sign up with GitHub Sign up with Facebook

3 of 3 4/28/23, 15:47