Local first is often just plain the better technical solution.
E.g. you put a pin on a map to mark the parking spot, and want to share it with somebody standing right next to you. But it does not work because the connection to some data center 1000 km away is flaky. This is just plain bad design.
Or you want to print something or get two machines to interact. Any remote third party you bring into this interaction is going to increase latency, reduce bandwidth, increase the probability of failure, etc.
That is why we use the local first approach for factory shop floor software at actyx.com . You don't want your factory to stop because the link to the cloud is down.
In developing nations, we often don't even have good internet. Uploading a 20 MiB file to Dropbox may fail a few times. But these days developers often forget these kind of situations.
In section 2 it says - "In cloud apps, the data on the server is treated as the primary, authoritative copy of the data; if a client has a copy of the data, it is merely a cache that is subordinate to the server."
Exactly! Local copy is just treated as cache in those apps which offer offline support. They are called "Offline-also" apps, not Offline-first apps.
I miss those days when I could just buy the software and own it. Collaboration can happen in offline-first apps too, it's tricky but it can be done.
Everything is online-first web apps these days because of the reasons listed in section 3.2.1. There is one more reason for popularity of online-only web apps:
When startups are testing their idea, it's far easier to put together an online-first app than offline-first. It's easier to get traction comparatively because it just requires a browser.
When they get some traction, then a desktop app is a low priority to them.
I want to bring back offline-first desktop apps back. I've already started with Brisqi[1] an offline-first personal Kanban app. Hope more people take an offline-first approach.
For similar reasons, I could write a whole essay on why we developers need to create a better open source cross-platform framework to develop offline-first desktop apps with permissive licensing.
My latest version of my ezInvoice app has a local-first option. It's also offline first.
The user has to install CouchDB on their desktop PC to use the local first option. Basically they just use a different html file to access the local CouchDB instead of the Cloud based CouchDB.
The app is also offline first and doesn't use the internet at all unless you turn on the "Live Sync" function in the app's preferences.
At that point it's still local first but it syncs with my Cloud based CouchDB and users can access that by using a different url. Since Live Sync works both ways whenever the Cloud CouchDB is updated those changes are pushed to the desktop PC if it's connected to the internet, or as soon as it is.
Users can also make backups and snapshots of their DBs on their desktop CouchDB.
I released ezInvoice back in 2002 and this is something I've wanted to do since then. Feels pretty good to have what's needed to get it working.
These kinds of ideas always get traction here on HN or similar places (slashdot), but I think that ship has long sailed. People, by and large, seem to be totally okay with storing data in the cloud -- from Dropbox, to Google Docs, to InVision. All things considered, data breaches are rare, and if they do happen, they're usually to be blamed on configuration blunders, not the services at large.
Other than a few highly-specialized niches (InfoSec, government, military), the de-facto reality is that online-first collaboration pros seem to outweigh the cons.
More and more, I am seeing many companies that realize they can do both: take the user's money and feed them ads. After all, that's more profitable than just taking their money.
As examples, T-mo, Verizon, my local gas station, various airlines all both take my money and give me ads. Some are even known to sell my data.
I think this is a good analogy, because the national debt is not a good measure of anything. People bring it up at all the wrong times, and takes grandstanding postures around it. Just like this, eh?
At least there might be a reason to care about privacy, but there actually isn't a reason to care about the national debt. There's no evidence the model of it being debt means anything - it certainly isn't household debt when it's in a currency you can just print more of.
And even under the model where printing money causes inflation, we're in a negative interest rate regime meaning printing money would cause /deflation/.
There’s a whole thing with growing or making your own food as much as you can. We keep outsourcing food production. In affluent areas that’s a boon, in others it’s an active problem.
I know plenty of enterprise guys who won't store critical documents, designs, code, or IP in cloud services. Also, local untethers from SaaS changes (prices/features), snooping, data loss, hacking, etc. at the expense of supporting, backup/restore, disaster recovery, and securing it yourself. It's fine if you have a competent IT shop and already have the scale to do it, but it's not the first choice for small, non-technical ventures.
Not just that, but there also a lot of cases that the local-first software will perform much better given the resources might be cached.
If you mix the two crowds: the people that actually care with the people that will just want faster apps, you get a powerful bootstrap traction.
Lets also not forget that things might just vanish and when we are able to collect them and make personal relations that make sense to us, we will want to be secure that things that matter to us wont go away because we are keeping them. Its pure psychology.
The parent poster is forgetting that before we get use of how things are right now in the digital realm, we had to deal with the material world and thats how we shaped our minds. And in that world we learn how important is to keep things nearby and have mnemonics to help our minds that needs to keep so much information that our devices become our second brains.
I'm working on a fancy chromium inspired prototype to tackle this exact issue and just with what i got here working in "prototype mode" i kind of don't want anything else.. it's just.. another level.
Its technologically superior and even ordinary, unaware people will just want this. Cloud have its place of course, but "local-first" will eat a lot of their lunch in the coming years.
> If you mix the two crowds: the people that actually care with the people that will just want faster apps, you get a powerful bootstrap traction.
Do we really though? Most people just buy a new phone, idly agree to all cloud sync stuff that the Android vendor has slapped on it and only notice "hey, I have 500GB free storage for the next 6 months!" and that's the end of it.
There is a market for local-first. Absolutely. But I question how big it is.
Maybe in corporate? If not there, I don't think you can make it big in that area. We at HN are a fairly negligible rounding error compared to most of the internet-connected humanity.
I agree with you. But that's because the state-of-the-art in clients are cloud hungry applications, giving their parent companies have much to profit from this behavior.
Local-first is a technological sophistication from the current tech status-quo, so it requires more labor to get at a point where it can stand against the current cloud-first applications.
We will only be able to see how far it can go, once the next-gen applications reach the same level.
It will be a harsh competition, but if we can just ship the platform along with it, and at least Android make it possible, users wont even notice, because it was the dev's that made that decision for them. And the dev's will do it because it might make their apps a little better, more responsive and fail proof.
Of course this all must happen before 5G or part of the leverage is gone unfortunately..
But we need at least try anyway, because a world where all the information is centralized on the FAANG titans is a world where freedom, civil and human rights are constantly endangered(especially in the age of AI automation).
Oh, I completely agree, especially with the point that most end users wouldn't notice the under-the-hood changes (except maybe for apps working better when connectivity is scarce or missing).
And yes, the world is heading into a dystopia. All of us that care better prepare some big ZFS clusters for redundant data and well-connected ISPs so we can still exchange data in an encrypted and decentralized manner.
But I feel that it's a lost battle. We missed the train. There's still time but it is shrinking every year.
everywhere outside the US this may be true, but in the US there is the millennial age group, which is a mini-population-boom. The millennial age group remembers stuff like handbrake and bittorrent, is socially conscious about the effects of centralization, and is about to become the largest demography with consumptive power (they are in their earnings/savings prime right now and having children).
Increasingly, millennials also are having children, and I think there is a market opportunity for these sorts of products (e.g. I want to sort pictures by my kids, but without giving facial recognition to fb/google).
People use the cloud because they are being forced through the software user interfaces. Working with local files takes increasingly more steps compared to cloud based solutions. So I wouldn't exactly say that people are ok with it, it feels more like they choice is being made for them.
If web pages could be more easily given access to a directory on your filesystem, Google Docs could more easily show you you files in your Documents folder.
'Access to shared data' for proprietary, untrusted "apps" is an anti-pattern that even iOS and Android are now getting away from. If a JavaScript app needs to access some sort of complex directory structure, that can happen within current API's by locally "uploading" and "downloading" a standard archive format, e.g. a .ZIP file.
However I would've said the same thing about Telegram/Signal/Defi and other decentralized/privacy first technology not too long ago.
I think there's a reasonable case for the idea that as the conversation around censorship and data ownership evolves, we'll see a restructuring of the market toward some level of personal data ownership. We're probably far from that today though.
People have no clue as to how computers and networks work. In fact, you may have even interviewed people for software jobs who don't have a solid grasp on these matters.
This is good news, as far as system architecture is concerned, as the opinion that matters is principally concerned with utility, convenience, usability, etc.
Secondly, the ship may have sailed, but interestingly these ships apparently are generational. There used to be a ship called AOL. /g
(There is a reason every ideological effort for authoritarian control heads straight for the school house and youth camps. The reason is the said 'ship' that sails, on schedule, every ~20 years.)
To be constructive here, if you care about these matters and have technical know-how, invest in the future and mentor the younger generation on how to build decentralized systems. Give them knowledge, blue prints, and tools. They'll crack the generational social code on their end.
> All things considered, data breaches are rare, and if they do happen, they're usually to be blamed on configuration blunders, not the services at large.
> Can't tell if you're intentionally or accidentally misleading.
All I'm doing is looking at the market cap of companies like Facebook and Dropbox. I mean, hell, Facebook literally sold personal data and sure, you occasionally see "boycott FB" movements, but they're all bark and no bite.
This is sort of like saying people are okay with cancer because they buy certain plastics in mass. The reality is people are ignorant of what is good for them in a ton of applications, including this one.
Only a handful really grasp the context of what is happening with their data and most importantly how their data could affect them.
People routinely do things against their own interest. It's up to the other people that care about this issue to make competing products that don't harm. There's simply no way to get everyone to really care about an issue with nuance and/or niche knowledge.
I don't think you're giving people enough credit. It's not exactly a mystery at this point that Facebook/IG make their money by selling access to your information to advertisers. People really just don't care which doesn't make them ignorant and doesn't mean they're wrong to not care. We're the privacy obsessed weirdos. I've been in conversations where people talk about products they "discovered" through IG ads.
The mindset where we're the only ones that really understand what's going on and everyone else are just sheeple -- sheeple I tell you is gross. This issue really isn't all that nuanced. That's just the lie we tell ourselves because we don't want to admit that we're the nutters being like "they're watching you scroll through Facebook and they're gonna use that to lower your credit score. THE ALGORITHMS!!"
I'm proud to be part of this weird club but let's not use it as an oppertunity to shit on our outgroup.
We may be interested in the deep details/debates the general public has no knowledge about. However i can assure you many people care about their privacy. They're not all ignorant, but they're usually powerless.
Wanna keep in touch with friends? Gotta use facebook. Wanna watch cool videos? Gotta use Youtube. etc.
But there's a very strong movement of non-tech persons for privacy, decentralization and free software/culture. In France, it's best exemplified by the Framasoft association which started from a network of teachers : https://framasoft.org/en/
Malice, apathy, ignorance, or simply don't care. They all look the same from a distance. With the same outcome.
You eat your ham sandwich. But did you hear the pig cry out at the slaughter? The same people reading your comment right now are the same people sticking the knife in you.
"It is difficult to get a man to understand something, when his salary depends on his not understanding it." --Upton Sinclair
In 1998 people turned their back on the FSF and joined the "open source" initiative. Everything that has happened since has been inevitable.
I'm in France, and have met some of the excellent folks who help framasoft with their project to degooglize the Internet.
I know a number of people who dispair, thinking they can't protect their privacy. But gradually, those who know a geek may be moving towards better software, as privacy respecting options with decent UX get developed, and as their FOSS-geek friend or family member patiently points out tools they can actually use.
From a business perspective, it doesn't really matter. There's still a large market of people that do care about it. Look at the explosive growth of Obsidian in large part because it's a local-first solution.
In addition, there's a difference between using those services and being "totally okay" with it. If you like Evernote, then sure, you'll be using the cloud. If you had a way to use Evernote without storing your data in someone else's cloud, a lot of people would be happy.
Even non-technical people talk all the time about not trusting services because of data issues. Many will not use a new service because they're worried about their data if the service stops. It's a really big issue for businesses.
Data breaches are not the only consideration. There's not many things more important to a business than their data and putting that entirely in a 3rd party's hands at a remote location only is very risky and could be catastrophic.
It's one thing to send a copy of a blueprint of a product design to China to have it manufactured and quite another to send them your only copy of it.
You can own your data if you use client-side encryption with cloud. When encrypted, you don't have to trust the cloud provider. This opens up new possibilities such as P2P networks with fair pricing, no provider lock-in, equal access and censorship resistance. I'm excited about Sia / Skynet which enables these kind of apps.
That works fine as long as you don't care that advances in computers and cryptography would one day (maybe after you are dead, and likely only with some notable effort) allow someone to read your data and (much more practically) you are extremely confident you can't lose control of your key somehow.
This is if you are storing all your data together in a single location. If you decouple it, using multiple accounts on the host, or even several hosts, you might have some of your data decrypted, but it would be worthless. If you decouple correctly, you should have data that is meaningless without some other data that is hosted elsewhere, on an account completely unrelated and unconnected to your other host accounts.
If it wasn't considered a solved problem, we wouldn't have public CA's for our browsers to trust, as their very premise is based on having a secure way to store private keys. I think a FIPS-140-2 level 4 HSM and hashicorp vault enterprise will cover 99.9999% of all other use cases for private CA's and PKI.
Keychains, Vault, GPG, HSMs, password managers. It's solved, just requires effort and understanding to setup. It's neither "huge" nor "unsolved" as pks have to live somewhere and cannot exist in the aether.
Yeah, I meant that the security tradeoff with usability is unsolved in personal use. It requires too much effort and understanding. If you use hardware keys, you still have to backup your keys somewhere safely.
It's funny that you mention the breaking of keys using outdated algorithms; I booted up my Standard Notes app and there's an "Encryption Upgrade Available" notice at the bottom which has you enter your password to locally regenerate your keys. I could imagine something similar working here, and perhaps let your progeny have access to your credentials postmortem.
Client-side encryption could help Confidentiality and Integrity, but still leaves you powerless on Availability. If your cloud provider decides to have a glitch, changes their offer, raises fees, … your whole setup is impacted by the cloud provider.
I wish OneDrive, Dropbox, etc. had a notion of a Personal Database File System, PDFS. With PDFS if they all shared the same API then you could build a database ontop of the PDFS that syncs to popular file systems.
If both of those things existed you would be able to build apps like LinkedIn such that LinkedIn could just access your PDFS so your profile is actually hosted directly on your PDFS - so if you delete your profile info on your PDFS it's gone from LinkedIn. Of course, this also depends on trusting LinkedIn to not just copy over the contents of your PDFS but that could be handled as well.
>Solid (Social Linked Data) is a web decentralization project led by Tim Berners-Lee, the inventor of the World Wide Web, developed collaboratively at the Massachusetts Institute of Technology (MIT). The project "aims to radically change the way Web applications work today, resulting in true data ownership as well as improved privacy" by developing a platform for linked-data applications that are completely decentralized and fully under users' control rather than controlled by other entities. The ultimate goal of Solid is to allow users to have full control of their own data, including access control and storage location. To that end, Tim Berners-Lee formed a company called Inrupt to help build a commercial ecosystem to fuel Solid.
You have to remember that all of these companies are either part of a large company or a startup. Locking you in is the reason they can justify their stock, their IPO, their exit and their high developer salaries.
So if it's just about protocol to exchange files, SSH and webdav are the de-facto standards for that (FTP has declining popularity).
If we're talking about the semantics of building desktop applications, then i think you're looking for freedesktop.org, although it quite POSIX-centered.
All four of these standards could receive extension proposals for more semantics for specific use-cases. However, Microsoft (who owns linked in) and other evil tech-multinationals will never adopt a standard because that would allow competitors to walk on their turf. Remember when gmail.com chat was federated with Jabber/XMPP until Google pulled the plug?
Microsoft was planning this 20 years ago with WinFS, to be shipped in what became Windows Vista. For many political and technical reasons it never came to pass. Some of the folks working on it went on to work on what became OneDrive though.
> What happens if you set up OneDrive and Dropbox (and Google and Apple) to sync the same folder.
Chaos, potentially. Think about deletes... delete a file from the web UI of one service, its client deletes the file locally, but that same instant the _other_ service sees the missing file and happily restores it, the first client deletes, the second client restores, etc. etc. until you've blown gigabytes and gigabytes of network bandwidth.
In other words, ultimate control needs to be on the client side. You delete from Dropbox, and the local machine marks the file as no longer being backed up by Dropbox but does not delete it. Some UI needs to present to the user: This file is backed up in these services, but not the other one. At that point, the user has the choice of fully removing the file, restoring it to Dropbox, or leaving it alone. They may even decide they don't want it backed up in any service, but want it to remain on the filesystem.
The headline is about the aspiration but not the implementation.
Within the contents of the paper, it talks about CRDTs to technically implement it.
I'm not familiar with the technical tradeoffs but I bookmarked some past HN threads mentioning CRDT did not fare as well as OT (Operational Transform):
Modern CRDTs like Automerge and Yjs should be able to perform basically just as well as an OT based systems (like my own ShareDB) now. I ran some benchmarks using Kleppmann’s approach a few months ago[1] and I think a well implemented crdt should easily be able to do millions of edits/second. The thing that’s holding us back at the moment is the lack of good high performance native implementations of these systems. But that’s being worked on.
This has only really started being true in the last year or so.
If javascript/non-native CRDT implementations are responsive enough compared to existing algorithms for collaborative editing like OT, what incentives or cases exist to demand high performance native implementations?
To be clear, I think this is exciting stuff and your article is great. I'm just curious if there are any open research problems in this area that would make them more appealing in production.
This sort of data structure is extremely hard to well optimize in javascript because a lot of the performance comes from finicky datastructure optimizations to keep allocations down in the fast path. JS implementations will be "fine", but I think there'd be a 20+x performance uplift between yjs and a zero allocation style rust/C implementation using btrees / skip lists.
So on the server I'd rather have a native implementation - which would also remove the need to bundle a JS VM in databases. And if we have that, it should be pretty easy to pack the same code into wasm for the browser.
And this is all assuming a JS-native web world. I also want CRDTs for native apps like Bear. And it'd be much easier to use a rust library than link to v8.
Neither of them says that. The xi CRDT was not about shared editing. The second paper just says that CRDTs are not superior per se. But CRDTs are being used in production for text editing and they are performant if implemented correctly (even in js).
> In the Trellis project we experimented with a “time travel” interface, allowing a user to move back in time to see earlier states of a merged document, and automatically highlighting recently changed elements as changes are received from other users. The ability to traverse a potentially complex merged document history in a linear fashion helps to provide context and could become a universal tool for understanding collaboration.
I’ve always wondered why word processors don’t take this approach out of the box.... ‘resume_v2_2021_final_done.docx’ -> resume.docz using some wrappers around git seems pretty straight forward to me... and I’d be using it in a heartbeat
Word absolutely has this functionality, and it's about as complicated as git, which is why only people who use Word professionally use the functionality. It's just too complicated compared to using the file metaphor you're already familiar with.
Re-watch the initial presentation of Google Wave. This was a complex demo that tried to explain what Google Wave is step by step so that a pretty tech-savy audience could follow. And it took several hours to talk them through.
Now imagine teaching all of that to a less technology minded audience. It probably won't go well.
Local-first software feels underrated especially in the UX realm. Peer-to-peer applications such as Manyverse load instantly due to not being blocked by network requests.
My hope is that more apps are developed with a local-first mindset. CRDTs likely introduce additional complexity around data storage, so this would certainly be a drawback when considering local-first.
Glad to hear someone say this! I'm itching to make a selfhosted Workflowy clone and have pretty much zeroed in on PouchDB/CouchDB to make the syncing easy. Would love to know if you found any pitfalls while working with the pair.
I did struggle with representing a graph and put that on ice. Though I only tried it with "raw" pouchdb, and researching this right now I found a plugin that supports many-to-many relationships[1].
I am completely behind the idea and would work full time on it (if I had my time for myself) but all such documents or sites are all talk and no code and it's getting tiring.
I get that they want to inspire, to show data backing up the idea that people are open to that software model, and in general to give hope and inform but... I am getting way too cynical and I can't help but wonder if actually writing a tool that builds the foundations of such a movement wouldn't have been a time better spent?
I mean, at what point will somebody do something about it? And does it have to be your average overworked programmer who is sacrificing their scarce free time? When will one of these organizations that periodically toot their own horn about much they care, will hire several hardcore programmers to do something?
> I am getting way too cynical and I can't help but wonder if actually writing a tool that builds the foundations of such a movement wouldn't have been a time better spent by the authors?
One of the authors wrote a book on this topic that gets high praise and frequent recommendations on this site. Several (all?) of the coauthors worked on prototypes on this topic that they discussed freely in order to enable others to follow their same path and explore it further. So yes, I'd say you're getting way too cynical.
If a doctor takes a picture of your wife's anus for a medical report, should that data be allowed to be saved and resent to anyone? Posted online? Distributed by one of those websites that lets you find people?
There are use-cases for data ownership, stewardship, management and security.
E.g. you put a pin on a map to mark the parking spot, and want to share it with somebody standing right next to you. But it does not work because the connection to some data center 1000 km away is flaky. This is just plain bad design.
Or you want to print something or get two machines to interact. Any remote third party you bring into this interaction is going to increase latency, reduce bandwidth, increase the probability of failure, etc.
That is why we use the local first approach for factory shop floor software at actyx.com . You don't want your factory to stop because the link to the cloud is down.