I’m sure OPs RTX 3060 will have a tolerable effect on the school children and the rainforest.

It depends on how complex you want to make it.

The simplest way would be to use a GUI tool like LM Studio, it has the benefit of selecting the model quants that are likely to work on your hardware and having sane defaults for the various parameters that you can work from.

Typically you’ll want to increase the context window, the default is good for short one-off questions but if you’re going to do RAG or coding tasks you will need a much larger context window. You will typically find this through trial and error, increasing until you hit memory limits or notice a large slowdown on tokens/second.

You can also have the model respond to an OpenAI-like API interface on a designated port/interface if you want to run non-chat tools through the same model.

Then, depending on the task that you’re trying to solve, you may want to find an MCP server built for that task which can provide the ability for the AI to interact with tools in a way that’s more controlled (giving an AI raw terminal access on a user’s machine is typically a bad idea).

If you need something more flexible that you can incorporate into scripts and automation, llama.cpp has been popular for a while. Like Linux distros, everyone is going to have an opinion on the best and whatever you decide on will be the wrong choice. There’s also Ollama and vLLM if you want to run a datacenter.

Model-wise it completely depends on the task. Do you need a multi-modal model that supports vision, is it going to be coding, is it trained to use tools/MCP servers, does it use reasoning, etc. The Qwen models are open weight and available in many different quants. The version 3.5 models run well on consumer hardware and can use vision, reasoning and tools.

I’m running it through Llama.cpp but it will likely be available via LM Studio as well.

The beatings cyberattacks will continue until morale security improves

Think of Louisiana before Katrina. Those systems of pumps and dykes are expensive and had been neglected… which was fine right up until they were severely tested by Katrina.

Our orange city counselman stopped funding repairs and inspections on the dykes and pumps many years ago (when he destroyed CISA). They’ve been deteriorating from neglect ever since.

A large hurricane is going to hit at 14:00 UTC

(In 10 minutes)

It’ll almost certainly be handled quietly and nobody will know but system admins, but this could be a bigger thing than usual.

0118 999 881 999 119 725 3

We’re YOUR emergency services

AI is good at RAG tasks. When you’re asking it questions about text that is in the context window, the hallucination rate is significantly lower.

It can fill that gap between documentation and implementation that isn’t always obvious to someone who is new and answer question that they may not be able to form into a search string. The kind of simple questions that, in some Linux communities, would invite assholes (like ‘how do you set permissions’ or ‘I want to add my user to a group’) to tell them to RTFM. Now TFM can answer their questions directly.

There was a GTA6.exe that appeared on the torrent sites around the time of the first gameplay video leaks, I’m sure it got a few people.

Please provide a blood sample to verify your Age. Thank you for using Open FaceGoogazonthropic, we love you.

The features they’re talking about are secure from a cryptographic standpoint.

However, there are features built into the passkey system so that a company could decide that it will only accept passkeys from a specific piece of hardware or software.

This isn’t necessarily bad, you may only want your server to allow you access if the passkey comes from your phone’s hardware. A person could have stolen your passkey but they’re unable to fake the attestation step.

On the other end of things, a company like Google could provide a passkey service which requires you to use hardware provided by vendors that they approve of and they could abuse their position to force it’s implementation, such as how you can use Google to sign into websites via OIDC.

Google could just as easily provide a service for ‘secure’ passkeys which are only allowed to be issued by phones provided by Google, Verizon or AT&T. No more third-party OSs (at least, if you want to sign into anything on the Internet) and no more jailbroken phones. You would have to use the exact, spyware laden, operating system that comes installed directly from your cellular provider who would have remote root access to your device.

I think the idea that we can just boycott passkeys and prevent this is nonsense magical thinking. It requires some hypothetical world where we can perfectly coordinate a boycott of otherwise useful technology. In reality this issue will remain obscure and generally only known to people interested in the topic while Google can pour tens of millions in advertising to reach everybody on the planet explaining how good and secure attested passkeys are to mainstream Internet users.

Use passkeys, they’re useful and more secure than passwords. Not using them won’t prevent enshittification, just sit back and watch the slow motion attested passkey trainwreck as it unfolds over the next decade.

That seems like a lot of work for something that can be done with a search filter.

Make sure your dotfile directories are part of a git repo, it makes unbreaking things MUCH easier.

You’ll probably need to watch a short video on git, but for this purpose it’s pretty simple to use.

One day you’ll use Arch, btw

zswap

As an aside, I just read that zswap is getting a pretty big performance increase: https://www.phoronix.com/news/Linux-Better-Zswap-Performance

I was looking into upgrading storage and ended up opting to buy 20TB in HDDs for my NAS.

I have to shuffle Steam games around but at 10Gb, it goes pretty fast (and you can run most games directly off the NAS) and the array can survive if a drive fails (ZFS).

You need the extra hardware to run the NAS but HDDs are a significantly cheaper option and you don’t need SSD speeds for most things.

They sell enclosures also if you want to be able to see the print… but it’s hard to beat free.

Brim and also ensure that the room doesn’t have any active cooling.

A fan blowing on the print will cause corners (with a higher surface area) to cool faster and the contraction will result in curling. Another alternative would be to put a cardboard box over the printer to ensure the print stays warm while it’s printing.

This is already not the case for computers. And you’re trying to argue that we might one day be able to run the bigger models on consumer hardware, but I said it’s too expensive to train them. And you know that the cost isn’t just the RAM right?

If you replace everything I said about RAM with compute and replace the Bill Gates quote with Thomas Watson saying that the “world market [has room for] for maybe five computers” then it’s the same argument. They’re all ultimately built on technical advances in lithography and so growth in RAM and compute largely stem from similar sources.

Sure, my saying they will “always” be tied to concentrated capital is slightly hyperbolic. You never know. But it takes a whole lot of optimism to hope that LLMs will be decoupled from the billionaire class, hence why most analyses of the problems LLMs cause discuss them in this context.

I understand why it is done as shorthand.

I know this likely doesn’t apply to you, as you seem to be capable of understanding the nuance, it’s that a lot of people on social media (by people I’m including the bots) don’t make that distinction.

The same vitriol that is, rightfully, directed at OpenAI/Anthropic/Google/NVIDIA is also directed at things completely outside of the corporate LLM hellscape.

I’m mostly frustrated at seeing the sciences, researchers and open source AI projects catching strays and being painted with the same brush.

It’s useful shorthand among people who understand the nuance, but using that shorthand on social media invites misunderstanding and that is causing collateral damage.

This is mostly true, but you’re glossing over the fact that AI will always be tied to concentrated capital due to the cost of training state of the art models.

I think this is a false assumption.

We don’t know what the ultimate state of AI will be, we only know that we’re at the beginning of the journey.

Look at other technologies in similar positions. At one point computers were like this. Nobody but the military and the largest companies could afford them and now you can buy children’s toys with more storage and processing power than was available to the entire US military at the dawn of computers.

We had no idea what computers would become, even the experts making guesses that sounded wild at the time were off by several orders of magnitude. Bill Gates once said “640K [of RAM] ought to be enough for anybody”, modern PCs have around 100,000 times that.

The reason that AI seems like it’s only affordable to concentrated capital is because models using a large amount of parameter require a large amount of RAM. The 5TB of RAM estimated for some of the largest frontier models is a lot, about 100x a workstation PC. That’s a few orders of magnitude less than 100,000x. Given equal scaling speed, we’re closer to running frontier models on our desktop than we are to the dawn of computers.

In addition, the research is leaning towards there being a ceiling on how useful it is to add more parameters (use more RAM). Doubling parameters doesn’t result in doubling of capability so there is effectively a soft cap on the amount of parameters.

Hardware is growing near exponentially while the RAM demands of models have a ceiling.

Keep in mind that this only relates to the state of the largest LLMs. You can train generative models on consumer graphics cards. Object recognition, classification, etc are all tiny compared to LLMs. These are also AI and, I’d argue, that they actually produce more value than LLMs (which are failing to deliver on their promise of replacing white collar labor).

It’s unlikely that the market will open up any time soon so that you can get your LLM from someone who isn’t a fascist billionaire tech bro.

You can find open weight models (of all kinds, not just LLMs) freely available here: https://huggingface.co/

Many can be run on locally on gaming hardware. Some of the classification and object recognition models can be run on microcontrollers (a popular use it to use them to make aimbots in FPS games).

There are larger models that can be run on more expensive dedicated hardware (like NVIDIA DGX Spark, ~$5,000) if you want to keep your projects completely internal. Or you can run those open weight models on hardware hosted in datacenters ran by companies you research to be ethical.

You can use AI without using one of the AI companies.

Most of these arguments are conflating AI with the people using them.

Labor exploitation, scraping without consent, military contracts, surveillance policing, grid strain and hardware prices driven up by speculation are real harms. But none of them are properties of a statistical language model.

They come from concentrated capital and state power, and whatever technology arrives next will get used the same way.

Just like we’ve seen everything in your list, done by the same people, using different technology.

We have had hardware speculation during the crypto craze.

These same companies (Google) have been taking your data without consent to power their surveillance advertising.

Datacenters have always been power and water hungry, the ones powering LLMs are exactly the same ones that exactly the same companies have been building since the Internet began.

Essentially every major advance in technology has been used for state surveillance and in the military. AI is simply the latest capability.

Models reproducing licensed code, poor code quality and floods of junk contributions. These are real problems with the technology itself. And the solution to most of these things is to iterate, build better models trained on consentually obtained data.

Yes, you should still be mad at what the AI companies are doing. I am too. Just keep in mind that Google was evil before Gemini and will certainly be on the forefront of fuckery with whatever technology comes next as long as people keep getting tricked into misdirecting their anger onto the technology.