As I am sure many have already seen, this 404 Media article caused quite a stir a few days ago. Even putting aside the ethics of focusing on AI use over security and pentesting (which is already a horrendous idea for a supposedly security-focused company), the verbal abuse allegations alone should be a turn-off for anyone who cares about even a neutral workplace (let alone a good workplace). Having a CEO who berates employees asking for more security testing should be a red line for anyone who cares about the privacy and security of their data. And, with Proton, that is likely most of their customer base (we moved away from Google largely for privacy reasons, after all).

Then we have reactions like this one, which brings up the point that leaving may not accomplish anything (other companies will also disappoint you, you have to look for alternatives, and generally life gets more complicated). I think what this misses is that most people (and companies) who moved to Proton already went through this process once already, and we're willing to move again. Most of us are here because Proton promised “people over profit” (their words) and “privacy by default” (again, their words). Having a CEO who verbally abuses employees asking for more time to do proper security testing means those words are literally the antithesis of how Proton actually operates. Verbally abusing employees is not putting people over profits. Prioritizing speed over security testing (which is functionally what is happening) does not ensure privacy by default. This is not to say that Proton is compromised —– very likely, the privacy guarantee still holds true for now. Rather, it is that the trajectory and attitude means that at some point, that guarantee will cease to be true.

Will other companies disappoint? Probably. But that's like saying “What's the point of moving from Google? Other companies might disappoint you, so better to stay with the convenience of Google”. After all, Google's cloud suite is infinitely more convenient for most people than Proton is. Proton's suite is slower, has fewer features, and often feels incomplete. So by Widdershins' own argument, they should never have moved to Proton in the first place —– it was always less convenient than Google is. But they did move, implying that those arguments just are an argument to overlook this specific issue with Proton.

And that's fine! You're free to ignore Proton's work culture and AI boosterism. But those of us who have a problem with it (like we had a problem with Google and Microsoft and the rest monetizing our lives) will move on and do the work of finding other alternatives. And to pretend as though that is virtue signalling is dishonest when it is very much the same attitude that brought us to Proton in the first place. We moved to Proton for the same reason many of us are considering alternatives now —– we had a problem with the way our previous provider was operating (whether that is privacy in the case of Google or workplace abuse and AI boosterism in the case of Proton).

Unfortunately, my 2-year subscription renewed a few months ago, so there's no sense in my moving away from Proton for the foreseeable future. I am looking to see how things shake out and whether the Foundation does the right thing (kicking out Andy Yen and Bart Butler) to see whether I renew 2 years from now. I am already self-hosting Ente for photos and will likely consider something like Filen for cloud storage. This leaves mail and VPN, but that's something to think about in the future should this not shake out the way I hope.

I certainly understand folks cancelling their subscriptions how, however. The core, fundamental promises of Proton are now in jeopardy thanks to the attitudes of the CEO and CTO. While privacy guarantees are likely still intact for now, Proton is clearly not putting people above profits, and prioritizing AI over security will jeopardize the privacy guarantees in the future as well.

I've been a paying customer since 2019 or so, I think, and I don't feel good recommending Proton anymore if someone asks for a privacy-respecting email service. I doubt I am alone in this. Unless they get their stuff in order, Proton is about to lose many of the very people who were enthusiastically promoting them to friends and family.

#AI #ScientificML #CloudLLMs #ChatGPT #Gemini #Claude #Ente #Firefox #Mozilla

Types

First, we should be clear what we're talking about here, because “AI” can mean a ton of different things. AI could mean any and all of:

1. Scientific machine learning, used within limited contexts to elucidate complex relationships within data and tease out more complicated models for which we don't currently have analytical forms.
2. Cloud-based LLMs, of the chatbot style (e.g. ChatGPT, Gemini, etc).
3. Local ML models run on-device (e.g. Firefox's on-device translation feature, Ente's machine learning, etc).

Putting them all into the same bucket is sort of silly, right? So let's address each in turn.

Scientific ML

Scientific machine learning models tend to be limited in scope and the good ones build in validation protocols. I might have my own concerns here around whether having a model for which we don't have an analytical form is useful. There are certainly variants here which attempt to extract equations from data (which brings its own challenges), but the concerns here are around the scientific usefulness. The data all has good provenance (usually generated by the researcher themselves), the scope is limited (e.g. we want to better predict the behavior of this specific material), and energy usage etc is wrapped up into the broader HPC system (this could change of course, if we need to scale up the HPC systems to handle more ML work, but that's a concern in the future and assumes these types of methods are broadly applicable in every context and that we need to train these models on the HPC...quite often, if a model is well-designed, it can be trained locally anyway, bypassing this issue entirely).

Cloud-based LLMs

Cloud-based LLMs like ChatGPT, Claude, Gemini, and so on raise the most alarm bells for me, and these are the ones I am completely against. I find issues with data provenance (did folks consent to have their data used for training these models?) and the fact that these companies seem to want to gatekeep knowledge. The ideal case for these people is that we can no longer function without LLMs and we pay them money indefinitely to continue to be able to function. There are the privacy concerns (these companies are storing your chats and can produce them on request) and the environmental concerns (there are enough articles out there about this, from the renewed building of gas power plants to folks' water becoming impotable to the fact that these data centers are being built in drought-prone areas). There are the ethical concerns around the fact that the pitch to companies is that they can replace all of their workers with these LLMs. There is the fact that they have no notion of 'truth' (so they don't 'hallucinate', they just spit out bullshit —– hallucination implies that they have a working model of the world and are imagining things which are not there, which simply is not true of these LLMs). There are bandaids around some of these concerns. Some companies provide interfaces to these LLMs without storing chats (DuckDuckGo) or storing them client-side encrypted (Proton). But nothing addresses all of these concerns.

Then there are the practical concerns. I simply don't see a use for them. I enjoy coding. I enjoy thinking about how to design interfaces and classes and modules so that extending them is easy. I enjoy debugging (within reason, of course). I enjoy deriving equations to figure out if something is correct. And I enjoy thinking. Why would I give that up to an LLM? Why would I prefer chatting with a sycophantic plausible-sounding bullshit generator rather than an actual human being with their own ideas and their own viewpoints? So much of this just feels like hype for hype's sake, where something sounds 'cool' so people jump on the bandwagon.

The basic fact is that if I have to double-check everything an LLM states anyway, I'm better off doing the research myself. I'm better off using Google Scholar (as long as that's still around /sigh) to search for research papers than asking an LLM to find stuff for me. Is there a good chance it will find stuff? Sure. Is there also a decent chance it will hallucinate papers that don't exist or imply a paper says something it doesn't? Also yes. The number of times I've looked at a DuckDuckGo AI summary and found it claiming something which simply isn't true is...non-negligible. It's not super high, but it's also disturbingly common. And that's the issue, right? It's not so good that I can just rely on it, but it's also not so bad that I can just assume everything is bullshit. It's at this level where most of what it's saying is probably accurate, increasing the cognitive overhead beyond what it would take for me to just search for stuff on my own.

Local models

Local models, I think, are the future of “AI” insofar as we want natural language interactions with machines. They are cheaper to run. They can be designed with good data provenance (sidestepping potential lawsuits and ethical issues). They don't require gigantic data centers. But they're harder to build for all of these reasons.

Something like Ente's on-device ML for face recognition and semantic search is great because the facial recognition data is encrypted before being uploaded to the server and all ML processing happens on-device. Firefox, similarly, has an on-device translation option which doesn't touch the cloud. I know Ente uses models with good provenance, but I can't figure out where the Firefox Translations dataset is. At the very least, though, any local ML model must have open datasets and open weights and biases for it to be ethical.

Conclusion

My strong feelings on 'AI' mostly derive from my strong feelings about cloud-based LLMs. There is a bait-and-switch tactic where folks will imply someone is against other forms of ML (including scientific ML or local models) just because they are against what OpenAI, Anthropic, and the rest are doing. I'm not against AI or ML in general, but this space has been co-opted by these giant companies doing highly unethical things. We can simultaneously be pro-ML (used judiciously) while also being anti-'AI' (as defined by these giant companies).