How successful will the Storj Project (Decentralized Cloud Storage) be?

So recently, as in yesterday, the long awaited Storj V3 network has finally launched into production. With the prices below, how good of an alternative to using AWS or google cloud do you think it will be. Is it worth migrating all your stuff over since their prices are more economic? As insightful individuals your comments are appreciated.

Storage
$0.01 PER GB PER MONTH

Bandwidth
$0.045 PER GB PER MONTH

Comments

  • beaglebeagle OG
    edited March 2020

    @Neoon may have a thing or two to say about them.

  • Shit Shit Shit.

    Thanked by (1)mfs
  • InceptionHostingInceptionHosting Hosting ProviderOG

    That seems to cheap to survive imo.

    https://inceptionhosting.com
    Please do not use the PM system here for Inception Hosting support issues.

  • lentrolentro Hosting Provider

    Note: It's 45 dollars per TB of egress bandwidth, and ingress is free.

    @AnthonySmith said:
    That seems to cheap to survive imo.

    They pay node operators $1.50/TB/mo, far less than the $10/TB/mo they are charging. Add in some redundancy, and they are paying $4.13 per TB per month (They store data in 80 shards, of which 29 are needed to recreate it). Add in payroll, and they still should be making a bit of money.

    Same thing can be said with egress.

    It's pretty innovative given that they can be 20% faster than AWS, but at the same time, they shouldn't be competing on cost too much, as the companies going with AWS are usually large corporations (imagine banks), and I'd imagine developers might go with the cheaper companies like BackBlaze or Wasabi. Storj will certainly survive (if they manage money correctly), but I doubt they will thrive and become #1.

    Just my ideas. Opinions or counter thoughts?

  • joepie91joepie91 OGServices Provider
    edited March 2020

    @xyphos10 said: With the prices below, how good of an alternative to using AWS or google cloud do you think it will be.

    The main concern with ~decentralized storage~ thingems isn't the price, though that usually doesn't look great either. Rather, the problem is availability and durability. How sure can you be that the data you store now, will still be there in 6 years?

    To my knowledge, noone has figured out yet how to make such guarantees for a decentralized system at the same level that they are being made for centralized systems. Of course, such "decentralized" storage providers often conveniently forget to mention this aspect.

    Until I see a plausible and clear explanation from a vendor on how they've solved the "proof of storage" problem (proving that N separate copies exist that are stored by different parties), I frankly wouldn't trust any system like this for storing production data. Every single "solution" for this that I've seen, and I've seen many, fails to protect from basic attacks.

    Thanked by (2)uptime AlSwearengen
  • I still don't get how Storj is better than Wasabi, both pricing wise and redundancy wise.

  • lentrolentro Hosting Provider
    edited March 2020

    @sanvit said:
    I still don't get how Storj is better than Wasabi, both pricing wise and redundancy wise.

    Wasabi doesn’t have multiple regions.

    When you store with Storj, your data is stored in 80 shards around the world. If North Korea bombs Ashburn, you’ll lose all data on Wasabi. On Storj, if a user is in Ashburn, the remaining 79 shards can still rebuild your data (29 are needed).

    Thus, Storj is more reliable.

    Pricing wise, they are a bit more expensive, but they are faster, are basically a built-in CDN, are more reliable, and have all of the normal SLAs to match other cloud providers.

  • ElmoElmo OG
    edited March 2020

    I would also like to make an important note, that I don't see mentioned: The data is transferred and stored in an encrypted form. It gets encrypted even before leaving the "source". I'm not sure how other providers are handling this, but I usually read in various ToS that the provider -can- have access to your data. I understand that anyone can encrypt their data before sending it out but I doubt that most people go down that road.

    As for availability and durability, since the data is split among 80 different endpoints and only 29 are needed to recreate this data, I tend to believe it's statistically impossible to have 51+ problematic endpoints out of 80 scattered around continents. Besides, as far as I have read, there is an algorithm in place that check for the presence of stored data and if a node goes offline or whatever, this data is replicated to another node.

    Of course this is a new technology and should be treated accordingly. By all means try it, but as already stated avoid using it for production data, if this data is not saved anywhere else.

    About the pricing model... I don't believe that the price is or should be considered the biggest advantage if this technology. When this matures and provided that it doesn't face any serious issues in the meantime, I expect that future pricing models of Storj would be much more expensive than centralized alternatives. But for now and in order to be adopted, yes the prices should be low. Otherwise people might not be interested/willing to test/use it.

  • lentrolentro Hosting Provider

    @Elmo said:
    Of course this is a new technology and should be treated accordingly. By all means try it, but as already stated avoid using it for production data, if this data is not saved anywhere else.

    I would personally be fine using it for production, as they have not lost a single file so far and have SLAs that reimburse you. I see it as something better than centralized cloud services given your reasoning.

    As long as you abide by the 3-2-1 rule for backups, you will be fine.

  • joepie91joepie91 OGServices Provider
    edited March 2020

    @lentro said: When you store with Storj, your data is stored in 80 shards around the world. If North Korea bombs Ashburn, you’ll lose all data on Wasabi. On Storj, if a user is in Ashburn, the remaining 79 shards can still rebuild your data (29 are needed).

    Thus, Storj is more reliable.

    ... assuming that Storj can actually accurately determine how many copies of a piece of data exist, which they almost certainly can't because it's a trustless network (like I described above). That's the whole problem with it.

    The concept behind this sort of distributed storage is nice, but it just doesn't hold up in practice yet in a fully trustless scenario.

    @lentro said: I would personally be fine using it for production, as they have not lost a single file so far and have SLAs that reimburse you.

    No amount of "it's not lost files so far" makes up for a fundamentally unsound storage model...

    Thanked by (2)WSS saibal
  • lentrolentro Hosting Provider

    @joepie91 said:

    ... assuming that Storj can actually accurately determine how many copies of a piece of data exist, which they almost certainly can't because it's a trustless network (like I described above). That's the whole problem with it.

    Well, I do frequently get audit checks. You need to be on the network for quite some time to actually get data sent. If you fail the audit checks (I think they are checksums), then you lose your withheld amount. (Which is 75% of revenue for you for your first 3 months, 50% of revenue of the next 3 months, etc. )

    No amount of "it's not lost files so far" makes up for a fundamentally unsound storage model...

    True, but I think we might see them succeed. They came up with their original idea back in 2016, went through 2 iterations of their network (now it’s version 3). It seems that they’ve improved a lot. With 30 million dollars to play with, you can do a lot. Let’s see.

  • comicomi OG
    edited March 2020

    @joepie91 said: it's a trustless network

    But it isn't.
    Surely they use term "decentralised" a lot, but what they actually have decentralised is just harddrives, which is the simplest part of any storage subsystem. Everything else in their network seems pretty centralized, thus they can control how many shards are online.

    @joepie91 said: assuming that Storj can actually accurately determine how many copies of a piece of data exist

    if one host goes down they are supposed to reconstruct it somewhere else, right? This necessitates that they must know at all times how many copies of a piece of data exist. if they don't the whole thing is glaringly pointless as individual data hosts are supposed to be able to disappear at any moment. Is it that ridiculous?

    Let's see.
    From https://storj.io/storjv3.pdf :

    4.1.1 Actors.
    ...
    Storage node. This peer class participates in the node discovery system, stores data for others, and gets paid for storage and bandwidth.
    ...
    Satellite. This peer class participates in the node discovery system, caches node address information, stores per-object metadata, maintains storage node reputation, aggregates billing data, pays storage nodes, performs audits and repair, and manages authorization and user accounts. Users have accounts on and trust specific Satellites. Any user can run their own Satellite, but we expect many users to elect to avoid the operational complexity and create an account on another Satellite hosted by atrusted third party such as Storj Labs, a friend, group, or workplace.

    4.14 Data repair.
    ...
    If a node goes offline, the Satellite will mark that nodes’ file pieces as missing. The node discovery system’s caches have reasonably accurate and up-to-date information about which storage nodes have been online recently. When a storage node changes state from recently online to offline, this can trigger a lookup in a reverse index within a user’s metadata database, identifying all segment pointers that were stored on that node. For every segment that drops below the appropriate minimum safety threshold m the segment will be downloaded and reconstructed, and the missing pieces will be regenerated and uploaded to new nodes.
    ...
    A direct implication of this design is that, for now, the Satellite must constantly stay running. If the user’s Satellite stops running, repairs will stop, and data will eventually disappear from the network due to node churn.

    So if trusted centralized part of the system goes down you can lose your data. Not completely the same, but pretty similar to other providers. The good thing is you can run your own piece of control plane, the bad thing is how does economy add up then.

    If people are supposed to run the control plane themselves it must stay cheaper then fully centralized clouds. If it must stay cheaper then they might decide to lower already pretty low payouts for individual hosts at some point. If that happens it might become worth the trouble to run storage node only for people with large amounts of free space. And there is exponentially more people with a little unused space than people with a lot of free space. That means a payout drop can cause huge swing in supply with hardly predictable outcome. Yes, I'm bored.

    Thanked by (1)lentro
  • Hmm, thank you very much everyone for your insightful comments. I guess the most prudent option right now is to wait for the technology to mature and see where things go from there.

  • @xyphos10 said: wait for the technology to mature

    It is not so much technology is maturing as it is general public's understanding of the technology.
    Until the Internet stack is rewritten, which I very much hope will happen at some point, and to a degree is happening now, the best kind of real world decentralization that is practicly possible is more like DNS or email.

    Storj isn't necessarily bad, but they try to market on "decentralization". People start assuming things they shouldn't, like as if it was possible to fully decentralise stuff on the current Internet stack, then they feel betrayed and lied to when it's suddenly kinda half centralized.

    Anyway, it is always better to have more choises then not.

Sign In or Register to comment.